CakePHP 3: Model Unit Test fails - "duplicate key value" - postgresql

Question

I'm using Postgres (which I think is related to the problem), and CakePHP 3.
I have the following unit test to just check to make sure that a valid dataset can get saved by the model. When I run the following test, with a standard "bake'd" Model unit test, I get the error below.
I think this is the problem:
We are using fixtures to add some base data. This is the only place that I think might be causing a problem. To add credence to this, while the unit tests were running I ran the following command to get the next auto-incrementing id value and it returned 1, even though it returned the proper number in non-test DB. Select nextval(pg_get_serial_sequence('agencies', 'id')) as new_id;
Unit Test:
public function testValidationDefault()
{
$agencyData = [
'full_name' => 'Agency Full Name',
'mode' => 'transit',
'request_api_class' => 'Rest\Get\Json',
'response_api_class' => 'NextBus\Generic',
'realtime_url_pattern' => 'http://api.example.com',
'routes' => '{"123": {"full_route": "123 Full Route", "route_color": "#123456"}}'
];
$agency = $this->Agencies->newEntity($agencyData);
$saved = $this->Agencies->save($agency);
$this->assertInstanceOf('App\Model\Entity\Agency', $saved);
}
Error:
PDOException: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "agencies_pkey"
DETAIL: Key (id)=(1) already exists.
Things I've tried
Copied that same code into a controller, and it successfully added the entity in the table.
Adding an id of 200. Same error appears.
Update 1
The fixture for this does have the ID field set each record. Deleting them from the fixture does work, but it breaks other unit tests that rely on some relational data.

Answer 1

I don't like this solution, but adding the following before saving the entity does work.
$this->Agencies->deleteAll('1=1');

Answer 2

[UPDATE: My other answer is the real solution to this problem.! You don't have to do this anymore...]
Here is a less dirty workaround that doesn't require deleting all the records:
use Cake\Datasource\ConnectionManager;
...
$connection = ConnectionManager::get('test');
$results = $connection->execute('ALTER SEQUENCE <tablename>_id_seq RESTART WITH 999999');
//TEST WHICH INSERTS RECORD(s)...
It appears that the auto-incrementing doesn't get properly set/reset during the setUp() or tearDown()... so manually setting it to something really high (greater than the number of existing records) prevents the "duplicate key..." error.
The benefit of this hack (over deleteAll('1=1')) is that you can still subsequently run tests that reference existing DB data.

Answer 3

It might be a problem in your fixture definition. The Cake PHP documentation uses a _constraints field specifying that the id field is a primary key:
'_constraints' => [
'primary' => ['type' => 'primary', 'columns' => ['id']],
]

Answer 4

I believe I've finally figured out the REAL solution to this problem!
I believe this issue stems from a default fixture setting that results from using the bake command to generate fixtures.
When you bake a model it creates the boilerplate for it's fixtures. Notice the autoIncrement for the ID property in the code below? Contrary to what you might think, this should not but true. When I set it to null and remove the ids from the items in the $records array I no longer get uniqueness errors.
public $fields = [
'id' => ['type' => 'integer', 'length' => 10, 'autoIncrement' => true, 'default' => null, 'null' => false, 'comment' => null, 'precision' => null, 'unsigned' => null],
'nickname' => ['type' => 'text', 'length' => null, 'default' => null, 'null' => false, 'comment' => null, 'precision' => null],
...
public $records = [
[
// 'id' => 1,
'nickname' => 'Foo bar',
'width' => 800,
...
The ninja wizards on the CakePHP project are the heroes: source
CakePHP ticket

Answer 5

If id fields are removed from fixture records then they will utilize auto-incrementing when inserted, leaving the table's ID sequence in the right place for inserts that happen during tests. I believe that is why it works for #emersonthis as described above.
That solution has another problem, though: you can't create dependable relationships between fixture records because you don't know what IDs they will get. What do you put in the foreign ID field of a related table? This has led me back to his original solution of just altering the table sequence after records with hard-coded IDs have been inserted. I do it like this in affected TestCases now:
public $fixtures = [
'app.articles',
'app.authors',
];
...
public function setUp()
{
$connection = \Cake\Datasource\ConnectionManager::get('test');
foreach ($this->fixtures as $fixture) {
$tableName = explode('.', $fixture)[1];
$connection->execute("
SELECT setval(
pg_get_serial_sequence('$tableName', 'id'),
(SELECT MAX(id) FROM $tableName)
)");
}
}
This moves the auto-increment sequence to the highest previously-used ID. The next time an ID is generated from the sequence it will be one higher, resolving the problem in all cases.
Including one of these solutions in an upcoming CakePHP release is being discussed here.