I am trying to populate some tables using data that I extracted from Google BigQuery. For that purpose I essentially normalized a flattened table into multiple tables that include the primary key of each row in the multiple tables. The important point is that I need to load those primary keys in order to satisfy foreign key references.
Having inserted this data into tables, I then try to add new rows to these tables. I don't specify the primary key, presuming that Postgres will auto-generate those key values.
However, I always get a 'duplicate key value violates unique constraint "xxx_pkey" ' type error, e.g.
"..duplicate key value violates unique constraint "collection_pkey" DETAIL: Key (id)=(1) already exists.
It seems this is triggered by including the primary key in the data when initializing table. That is, explicitly setting primary keys, somehow seems to disable or reset the expected autogeneration of the primary key. I.E. I was expecting that new rows would be assigned primary keys starting from the highest value already in a table.
Interestingly I get the same error whether I try to add a row via SQLAlchemy or from the psql console.
So, is this as expected? And if so, is there some way to get the system to again auto-generate keys? There must be some hidden psql state that controls this...the schema is unchanged by directly inserting keys, but psql behavior is changed by that action.
I am happy to provide additional information.
Thanks
I have a question concerning the error messages in PostgreSQL.
I noticed that in case of some failure PostgreSQL make a report in the form of text message but it does not contain error code id.
For instance:
ERROR: Relation "mytable" already exists or
ERROR: duplicate key value violates unique constraint "id"
Could you please suggest a way to make PostgreSQL including native error code id to messages for instance as follows:
42P07 ERROR: Relation "mytable" already exists or
23505 ERROR: duplicate key value violates unique constraint "id"
.
Is it possible ?
Thanks in advance.
You can change the parameter log_error_verbosity in postgresql.conf file to change the amount of information being logged during errors. By default, its value is default. You can change it to verbose to include more information about the errors.
In postgres I have created a table by name twitter_tweets. In this table I have assigned constraint for tweet_text column by using the command
ALTER TABLE ONLY twitter_tweets
ADD CONSTRAINT twitter_tweets_pkey PRIMARY KEY (tweet_text);
The constraint has applied by getting message i.e., alter table
but while parsing the data it showing runtime exception i.e.,
java.lang.RuntimeException: Failed to execute insert query insert into twitter_tweets (tweet_created_at, tweet_id, tweet_id_str, tweet_text, tweet_source, tweet_truncated, tweet_in_reply_to_status_id, tweet_in_reply_to_status_id_str, tweet_in_reply_to_user_id, tweet_in_reply_to_user_id_str, tweet_in_reply_to_screen_name, tweet_geo,tweet_coordinates, tweet_at_reply, tweet_is_quote_status, tweet_retweet_count, tweet_favorite_count, tweet_favorited, tweet_retweeted, tweet_lang, tweet_possibly_sensitive, tweet_filter_level, tweet_scopes_S)values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) at Demo.JdbcClient.executeInsertQuery(JdbcClient.java:62) at Demo.PsqlBolt.execute(PsqlBolt.java:91) at backtype.storm.daemon.executor$fn__5694$tuple_action_fn__5696.invoke(executor.clj:690) at backtype.storm.daemon.executor$mk_task_receiver$fn__5615.invoke(executor.clj:436) at backtype.storm.disruptor$clojure_handler$reify__5189.onEvent(disruptor.clj:58) at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:132) at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:106) at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) at backtype.storm.daemon.executor$fn__5694$fn__5707$fn__5758.invoke(executor.clj:819) at backtype.storm.util$async_loop$fn__545.invoke(util.clj:479) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "twitter_tweets_pkey" Detail: Key (tweet_text)=() already exists. at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:405) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2892) at com.zaxxer.hikari.proxy.StatementProxy.executeBatch(StatementProxy.java:116) at com.zaxxer.hikari.proxy.PreparedStatementJavassistProxy.executeBatch(PreparedStatementJavassistProxy.java) at Demo.JdbcClient.executeInsertQuery(JdbcClient.java:50) ... 11 more
The below image1 is the table to which i have used constraint
This is my output after keeping constraints
Your problem is described here:
ERROR: duplicate key value violates unique constraint "twitter_tweets_pkey" Detail: Key (tweet_text)=() already exists. at
You set tweet_text to be your PRIMARY KEY (PK), and as PK it cant get duplicated data.
At some point you already insert the data that you are trying to insert now into this column (tweet_text).
Now, why not create an Integer column, AUTO INCREMENTED, something like ID? The way as it now, you are telling me that no one should post a same text that was posted by other user.
Ex. If User A post a tweet with content (tweet_text) : "Hello World", no other user can post the same content.
Unique Constraint Violation
You asked for a primary key. A primary key in Postgres automatically creates an index and a UNIQUE constraint.
Then you inserted rows of data. At least two of those rows had the same value in that primary key field. The duplicate data violated the UNIQUE constraint. Postgres then did its duty in refusing to store the offending data. That refusal is reported back to you, the Java programmer, as an Exception.
At least that is my guess based on this excerpt from the middle of your error text:
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "twitter_tweets_pkey" Detail: Key (tweet_text)=() already exists.
I keep getting this error within my application
duplicate key value violates unique constraint "product_supplierinfo_pkey"
DETAIL: Key (id)=(409) already exists.
This is on table product_supplierinfo.
The actual next sequence number the key constraint needs to be is 5461 not 409.
Can someone please tell me the correct query to update this key unique constraint?
#chris Collins, please post the output of \d product_supplierinfo. I imagine you created this table with id serial.
You should see the name of the sequence from which the next default value for the id field will come. It will probably be product_supplierinfo_id_seq.
Then do, assuming the above names are correct, SELECT * from product_supplierinfo_id_seq;. You will probably see that the next value is 410.
If this is all correct, do SELECT setval('product_supplierinfo_id_seq', 5461);.
I have a composite UNIQUE set of columns in my table. Therefore if I insert into the table where the unique key is violated, Postgresql returns and error and my PHP script can read this error.
When inserting, instead of doing this:
SELECT id FROM table WHERE col1='x' and col2='y'
(if no rows)
INSERT INTO table...
(else if rows are found)
UPDATE table SET...
I prefer to use:
INSERT INTO table...
(if error occurred then attempt to UPDATE)
UPDATE table SET...
The kind of error returned from the above would be "ERROR: duplicate key value violates unique constraint "xxxxxxxx_key"
However, there is no point doing an UPDATE if the INSERT failed for some other reason, such as invalid data. Is there a way of "knowing" (from PHP/Postgres) if the error actually failed from this duplicate key issue, rather than invalid data? I'm just curious. Performing an UPDATE also would return an error anyway if the data were invalid, but what would you say is best practice?
Many thanks!
Just check the error message to see what kind of error you have. pg_result_error_field() shows it all. Check the PGSQL_DIAG_SQLSTATE and the PostgreSQL manual for the details.
You might want to look into this example in the official documentation.
You're free to add more WHEN EXCEPTION ... THEN handlers, list of available errors can also be found in the documentation.
Although in the example above the function will cause an exception in case any other error appears, except the unique_violation one, which is treated specially.