I am trying to populate some tables using data that I extracted from Google BigQuery. For that purpose I essentially normalized a flattened table into multiple tables that include the primary key of each row in the multiple tables. The important point is that I need to load those primary keys in order to satisfy foreign key references.
Having inserted this data into tables, I then try to add new rows to these tables. I don't specify the primary key, presuming that Postgres will auto-generate those key values.
However, I always get a 'duplicate key value violates unique constraint "xxx_pkey" ' type error, e.g.
"..duplicate key value violates unique constraint "collection_pkey" DETAIL: Key (id)=(1) already exists.
It seems this is triggered by including the primary key in the data when initializing table. That is, explicitly setting primary keys, somehow seems to disable or reset the expected autogeneration of the primary key. I.E. I was expecting that new rows would be assigned primary keys starting from the highest value already in a table.
Interestingly I get the same error whether I try to add a row via SQLAlchemy or from the psql console.
So, is this as expected? And if so, is there some way to get the system to again auto-generate keys? There must be some hidden psql state that controls this...the schema is unchanged by directly inserting keys, but psql behavior is changed by that action.
I am happy to provide additional information.
Thanks
Related
I am using a tool called apgdiff 'https://www.apgdiff.com/' for finding the DDL diff between 2 postgres database. It parses 2 postgres dumps and generate the diff between the 2 dumps in terms of alter queries .
The tool actually doesn't mind the order of creating or dropping foreign key constraints while generating the diff. i.e. foreign key constraints should be created after primary key , or to be dropped before dropping the primary key . But still, what makes me curious is a line of code in their sourcecode, which says that all the primary keys should be dropped first and then all other non-primary keys should be dropped . Do we have any such constraint in Postgres that the primary keys should be dropped first and then the remaining constraints ..
If anything, other constraints should be dropped first, because foreign key constraints depend on primary key (or unique) constraints. It doesn't matter, though, if you use the CASCADE keyword when dropping the constraints.
I can't see a reason why dropping primary key constraints first should make a difference.
I am new to Redshift when pushing the data in Redshift, where created the primary key as Vin(Vehicle Identification Number). Even when pushing the same key twice not getting any constraint exception instead same data being saved as record.
And when doing with Foreign key constraint again getting the same issue. Am I missing any configurations for enabling the contrints in db ?
From the AWS documentation:
Define primary key and foreign key constraints between tables wherever appropriate. Even though they are informational only, the query optimizer uses those constraints to generate more efficient query plans.
Do not define primary key and foreign key constraints unless your application enforces the constraints. Amazon Redshift does not enforce unique, primary-key, and foreign-key constraints.
If I read this information correctly, the workaround you should follow is to check in your application layer that each VIN number to be inserted is unique.
I'm using postgres and I require a secondary key or a similar concept. I make full use of the primary key the id field. In addition to the primary key, I'm generating a useful 'conversational id' such as 'OH-15-001'.
I would like to ensure, that the above secondary key, is used only once even if is destroyed by the database. I've created an algorithm to generate these id's but I do not wish to save them in another table. I'm wondering if there is a feature in Postgres to ensure the field is unique, even if it is removed? (similar to the id field)
A PRIMARY KEY doesn't ensure uniqueness against removed previous values. You will always get a new, unique ID if you generate them from a sequence, but nothing is preventing you from manually inserting a record with a previously deleted ID.
Can you use a sequence as part of your ID generation algorithm? (that, plus a UNIQUE constraint will behaving exactly the same as a primary key)
On https://stackoverflow.com/questions/10356484/how-to-add-on-delete-cascade-constraints#= a user, kgrittn, commented saying that
But I notice that you have not created indexes on referencing columns... Deletes on the referenced table will take a long time without those, if you get many rows in those tables. Some databases automatically create an index on the referencing column(s); PostgreSQL leaves that up to you, since there are some cases where it isn't worthwhile.
I'm having difficulty understanding this completely. Is he saying that primary keys are not created automatically with an index or is he saying that foreign keys should be indexed (in particular cases that is). I've looked at the PostgreSQL documentation and it appears from there that an index is created for primary keys automatically. Is there a command I can use to list all indexes?
Thanks
A primary key is behind the scenes a special kind of a unique index. The quote referencing, that it might be a good idea to create an index also on columns, where the primary key is used as an foreign key.
I am trying to use Entity Framework DB first to do quick prototyping of a reporting website for a huge db. The problem is one of the tables doesn't have a key. I got an 'Error 159: EntityType has no key defined'. If I add a key on the model designer, I got 'Error 3024: Must specify mapping for all key properties'. My question is whether there is a way to workaround this WITHOUT adding a key to the table. The table is not in our control.
Huge table which does not have a key? It would not be possible for you or for table owner to search for anything in this table without using full table scan. Also, it is basically impossible to use UPDATE by single row without having primary key.
You really have to either create synthetic key, or ask owner to do that. As a workaround, you might be able to find some existing column (or 2-3 columns) which is unique enough that it can be used as unique key. If it is unique but does not have actual index created, that would be still not good for performance - you should create such index.