Upserting into one Postgres table from another table? - postgresql

I have two tables with identical structures. All columns are integers and are named "A" , "B" and "key".
I can insert from one table into another with some SQL like this:
INSERT INTO test_table_kewmfsznj
SELECT * FROM tmp_test_table_kewmfsznj_cnxtbkbq ta
But this doesn't work:
INSERT INTO test_table_kewmfsznj
SELECT * FROM tmp_test_table_kewmfsznj_cnxtbkbq ta
ON CONFLICT ("key")
DO NOTHING
My expectation was that this code would skip any row from "ta" where the key already exists in the table I'm inserting into. Instead I get this error:
ERROR: missing FROM-clause entry for table "ta"
Position: 152
Here's what I really want to do: When the key already exists in the table I'm inserting into, I want to update certain columns:
INSERT INTO test_table_kewmfsznj
SELECT * FROM tmp_test_table_kewmfsznj_cnxtbkbq ta
ON CONFLICT ("key")
DO UPDATE SET "A" = ta."A", "B" = ta."B"
Unfortunately this gives me (almost) the same error:
ERROR: missing FROM-clause entry for table "ta"
Position: 134
Can somebody explain what I am doing wrong here?
EDIT0: I tried it without upper case table names. The columns are now called "a", "b" and "key". The data remains unchanged - it's all integers.
INSERT INTO test_table_mrcvnaoia
SELECT * from tmp_test_table_mrcvnaoia_uuxkaidv ta
ON CONFLICT (key)
DO UPDATE SET a = ta.a, b = ta.b
... and now I get this error:
SQL Error [42P01]: ERROR: missing FROM-clause entry for table "ta"
Position: 129
To me, this suggests that there's something wrong with my ON CONFLICT statement, and probably not the first half of the query, but beyond that I'm out of clues. Can anybody help?

You almost had it, but you can't reference the table name, you reference EXCLUDED:
INSERT INTO test_table_mrcvnaoia
SELECT * from tmp_test_table_mrcvnaoia_uuxkaidv
ON CONFLICT (key)
DO UPDATE SET a = EXCLUDED.a, b = EXCLUDED.b;
Furthermore, to avoid errors in the future, make sure you explicitly specify the column names in the insert and select portions of your statement. They are the same for now, but that might not always be the case.

Related

Temp table column type error when values are null

I am using a temp table to hold some data while I tidy up a database table. Narrowed down, it goes something like this:
CREATE TEMP TABLE fw_temp AS SELECT time, version, device_id FROM fw_status;
DELETE FROM fw_status;
INSERT INTO fw_status SELECT * FROM fw_temp
On my INSERT statement I get some.. interesting behaviour when the values in the device_id column are null:
SQL Error [42804]: ERROR: column "device_id" is of type bigint but expression is of type text
Hint: You will need to rewrite or cast the expression.
As the error message may lead on, device_id is indeed bigint type in the table, but appears to have lost the type when we select from the temp table.
Making a general select,
CREATE TEMP TABLE fw_temp AS SELECT * FROM fw_status;
DELETE FROM fw_status;
INSERT INTO fw_status SELECT * FROM fw_temp
interestingly does not cause this error.
Now, the real temp table is a bit more complex, so sadly going with SELECT * is not an option. How can I go about this error?
Changing to
CREATE TEMP TABLE fw_temp AS SELECT time, version, device_id FROM fw_status;
DELETE FROM fw_status;
INSERT INTO fw_status (time, version, device_id) SELECT time, version, device_id FROM fw_temp
resolved the issue. Apparently, you can mix general select and general insert or specific select and specific insert, but not general select and specific insert. I am still puzzled as to what is going on. Any enlightenment would be appreciated

How can I insert into another table selecting from another ignoring the data type

I am running the query below and I am getting the error shown
core=# INSERT INTO transactionp SELECT * FROM transaction WHERE posting_date>'2021-08-17';
ERROR: column "balance" is of type u_money but expression is of type u_datetime
LINE 1: INSERT INTO transactionp SELECT * FROM transaction WHERE pos...
^
HINT: You will need to rewrite or cast the expression.
It's most likely because of the dreaded select * and not specifying target columns for the insert. Both are considered "code smell" and should be avoided.
The error message suggests that the order of columns is different in the source and target tables (they are matched by position not by name)
To fix this, specify column names in both parts: the INSERT and the SELECT statement:
INSERT INTO transactionp (balance, some_column, another_column, ...)
SELECT some_balance, column_one, column_two, ...
FROM transaction
WHERE posting_date>'2021-08-17';

UPSERT from table with different table sizes

I'm getting the error:
ERROR: column "some_col_name" does not exist Hint: There is a column named "some_col_name" in table "usert_test", but it cannot be referenced from this part of the query.
On UPSERT The cause of this error is that the source table (read in from API) doesn't always have the same number of fields as the table I'm looking to UPSERT. Within the UPSERT process is there a way to handle this? So far I've tried the below:
INSERT INTO scratch."usert_test" (many_cols)
SELECT *
FROM scratch.daily_scraper
ON CONFLICT (same_unique_id)
DO UPDATE
SET
many_fields = excluded.many_fields;
Name each column specifically in every instance.
insert into scratch."usert_test" (column_name1, column_name2, column_name3,column_name3)
select cola, colb, colc, colf
from scratch.daily_scraper
on conflict (column_name1, column_name4)
do update
set
column_name3 = excluded.column_name3
, column_name2 = excluded.column_name2;
How ever many columns you have properly name every one. (IMHO) As you should always do.

postgresql on conflict-cannot affect row a second time

I have a table, i have auto numbering/sequence on data_id
tabledata
---------
data_id [PK]
data_code [Unique]
data_desc
example code:
insert into tabledata(data_code,data_desc) values(Z01,'red')
on conflict (data_code) do update set data_desc=excluded.data_desc
works fine, and then i insert again
insert into tabledata(data_code,data_desc) values(Z01,'blue')
on conflict (data_code) do update set data_desc=excluded.data_desc
i got this error
[Err] ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
this is my real code
insert into psa_aso_branch(branch_code,branch_desc,regional_code,status,created_date,lastmodified_date)
(select branch_code, branch, kode_regional,
case when status_data='Y' then true
else false end, current_date, current_date
from branch_history) on conflict (branch_code) do
update set branch_desc = excluded.branch_desc, regional_code = excluded.regional_code,status = (case when excluded.status='Y' then true else false end), created_date=current_date, lastmodified_date=current_date;
working fine on first, but not the next one (like the example i give you before)
You can use update on the existing record/row, and not on row you are inserting.
Here update in on conflict clause applies to row in excluded table, which holds row temporarily.
In the first case record is inserted since there is no clash on data_code and update is not executed at all.
In the second insert you are inserting Z01 which is already inserted as data_code and data_code is unique.
The excluded table still holds the duplicate value of data_code after the update, so the record is not inserted. In update set data_code have to be changed in order to insert record properly.
I have been stuck on this issue for about 24 hours.
It is weird when I test the query on cli and it's works fine. It is working fine when I make an insertion using one data row. This errors only appear when I'm using insert-select.
It is not mostly because of insert-select problem. It is because the select rows is not unique. This will trigger the CONFLICT for more than once.
Thanks to #zivaricha comment. I experiment from his notes. Just that its hard to understand at first.
Solution:
Using distinct to make sure the select returns unique result.
This error comes when the duplicacy occurs multiple times in the single insertion
for example you have column a , b , c and combination of a and b is unique and on duplicate you are updating c.
Now suppose you already have a = 1 , b = 2 , c = 3 and you are inserting a = 1 b = 2 c = 4 and a = 1 b = 2 c = 4
so means conflict occurs twice so it cant update a row twice
I think what is happening here
when you do an update on conflict, it does an update that re conflicts again and then throws that error
We can find the error message from the source code, which we can simply understand why we got ON CONFLICT DO UPDATE command cannot affect row a second time.
In the source code of PostgreSQL at src/backend/executor/nodeModifyTable.c and the function of ExecOnConflictUpdate(), we can find this comment:
This can occur when a just inserted tuple is updated again in the same command. E.g. because multiple rows with the same conflicting key values are inserted.
This is somewhat similar to the ExecUpdate() TM_SelfModified case. We do not want to proceed because it would lead to the same row being updated a second time in some unspecified order, and in contrast to plain UPDATEs there's no historical behavior to break.
As the comment said, we can not update the row which we are inserting in INSERT ... ON CONFLICT, just like:
postgres=# CREATE TABLE t (id int primary key, name varchar);
postgres=# INSERT INTO t VALUES (1, 'smart'), (1, 'keyerror')
postgres=# ON CONFLICT (id) DO UPDATE SET name = 'Buuuuuz';
ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
Remember, the executor of postgresql is a volcano model, so it will process the data we insert one by one. When we process to (1, 'smart'), since the table is empty, we can insert normally. When we get to (1, 'keyerror'), there is a conflict with the (1, 'smart') we just inserted, so the update logic is executed, which results in updating our own inserted data, which PostgreSQL doesn't allow us to do.
Similarly, we cannot update the same row of data twice:
postgres=# DROP TABLE IF EXISTS t;
postgres=# CREATE TABLE t (id int primary key, name varchar);
postgres=# INSERT INTO t VALUES (1, 'keyerror'), (1, 'buuuuz')
postgres=# ON CONFLICT (id) DO UPDATE SET name = 'Buuuuuuuuuz';
ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.

How to Update/Insert (Upsert) for Multiple Values DB2

I am trying to do an UPSERT in DB2 9.7 without creating a temporary table to merge. I am specifying values as parameters, however I'm always getting a syntax error for the comma separating the values when I try to include more than one row of values.
MERGE INTO table_name AS tab
USING (VALUES
(?,?),
(?,?)
) AS merge (COL1, COL2)
ON tab.COL1 = merge.COL1
WHEN MATCHED THEN
UPDATE SET tab.COL1 = merge.COL1,
tab.COL2 = merge.COL2
WHEN NOT MATCHED THEN
INSERT (COL1, COL2)
VALUES (merge.COL1, merge.COL2)
I have also tried teknopaul's answer from Does DB2 have an “insert or update” statement, but have received another syntax error complaining about the use of SELECT.
Does anybody know how to correctly include a table with values in my merge, without actually creating/dropping one on the database?
I believe you need something like USING (SELECT * FROM VALUES ( ...) ) AS ...