I've created a temporary table DETAILS and follow the same syntax of creating and inserting in it. But I have not received any result set However, the CREATE and INSERT statements ran successfully and the Row was also affected in the INSERT statement . But the result set was empty when I ran the last SELECT statement to view the record .
DROP TABLE DETAILS ;
CREATE GLOBAL TEMPORARY TABLE DETAILS AS (
SELECT ins_id , firstname , pages FROM
INSTRUCTOR)DEFINITION ONLY;
INSERT INTO DETAILS
SELECT ins_id , firstname , pages
FROM INSTRUCTOR WHERE ins_id = '1';
SELECT * FROM DETAILS ;
If you want to preserve rows in CGTT after commit, you have to specify ON COMMIT PRESERVE ROWS option of the CREATE GLOBAL TEMPORARY TABLE statement.
ON COMMIT DELETE ROWS option is in effect otherwise, and such a table is cleared on commit.
Related
I am updating a table with data from another table from another database. Most columns can be moved with no change but some of them require mapping/change after import since they refer to the new database ids. My first attempt was to make insert with returning values and use those values in update on the same table, with additional joins. It does not work. Postgres inserts records and ignores updates, with no error. Simplified example of the case below.
Create table test(
idtst serial ,
name varchar(10),
nameid varchar(20));
with tst as ( -- makes insert, no update
Insert into test(name,nameid) values ('tsta','tsta')
returning idtst,nameid)
update test t
set nameid=t.nameid||t.idtst::varchar(5)
from tst i
where t.idtst=i.idtst;
with tst as ( -- makes insert, no update
Insert into test(name,nameid) values ('tstb','tstb')
returning idtst,nameid)
update test t
set nameid=i.nameid||i.idtst::varchar(5)
from tst i
where t.idtst=i.idtst;
Create temp table tmp_tst(
idtst integer,nameid varchar(20));
with tst as (
Insert into test(name,nameid) values ('tstc','tstc')
returning idtst,nameid)
Insert into tmp_tst Select * from tst; -- insert into tmp
update test t -- makes update
set nameid=i.nameid||i.idtst::varchar(5)
from tmp_tst i
where t.idtst=i.idtst;
Does it depend on postgres version/configuration ?
In test in result there are records
1,tsta,tsta
2,tstb,tstb
3,tstc,tstc3
No errors, no warnings, just psql ignores first two updates. Updates/Inserts on any other table are OK.
What is table i? Where does it come from?
My guess would be, that there is no lines which fullfill t.idtst=i.idtst
Maybe you should look at this:
SELECT *
FROM test as t, i
WHERE t.idtst=i.idtst
I am slowly working through a feature where I am importing large csv files. The contents of the csv file has a chance that when it is uploaded the contents will trigger a uniqueness conflict. I've combed stack overflow for some similar resources but I still can't seem to get my trigger to update another table when a duplicate entry is found. The following code is what I have currently implemented with my line of logic for this process. Also, this is implemented in a rails app but the underlying sql is the following.
When a user uploads a file, the following happens when its processed.
CREATE TEMP TABLE codes_temp ON COMMIT DROP AS SELECT * FROM codes WITH NO DATA;
create or replace function log_duplicate_code()
returns trigger
language plpgsql
as
$$
begin
insert into duplicate_codes(id, campaign_id, code_batch_id, code, code_id, created_at, updated_at)
values (gen_random_uuid(), excluded.campaign_id, excluded.code_batch_id, excluded.code, excluded.code_id, now(), now());
return null;
end;
$$
create trigger log_duplicate_code
after insert on codes
for each row execute procedure log_duplicate_code();
INSERT INTO codes SELECT * FROM codes_temp ct
ON CONFLICT (campaign_id, code)
DO update set updated_at = excluded.updated_at;
DROP TRIGGER log_duplicate_code ON codes;
When I try to run this process nothing happens at all. If I were to have a csv file with this value CODE01 and then upload again with CODE01 the duplicate_codes table doesn't get populated at all and I don't understand why. There is no error that gets triggered or anything so it seems like DO UPATE..... is doing something. What am I missing here?
I also have some questions that come to my mind even if this were to work as intended. For example, I am uploading millions of these codes, etc.
1). Should my trigger be a statement trigger instead of a row for scalability?
2). What if someone else tries to upload another file that has millions of codes? I have my code wrapped in a transaction. Would a new separate trigger be created? Will this conflict with a previously processing process?
####### EDIT #1 #######
Thanks to Adriens' comment I do see that After Insert does not have the OLD key phrase. I updated my code to use EXCLUDED and I receive the following error for the trigger.
ERROR: missing FROM-clause entry for table "excluded" (PG::UndefinedTable)
Finally, here are the S.O. posts I've used to try to tailor my code but I just can't seem to make it work.
####### EDIT #2 #######
I have a little more context on to how this is implemented.
When the CSV is loaded, a staging table called codes_temp is created and dropped at the end of the transaction. This table contains no unique constraints. From what I read only the actual table that I want to insert codes should have the unique constraint error.
In my INSERT statement, the DO update set updated_at = excluded.updated_at; doesn't trigger a unique constraint error. As of right now, I don't know if it should or not. I borrowed this logic taken from this s.o. question postgresql log into another table with on conflict it seemed to me like I had to update something if I specify the DO UPDATE SET clause.
Last, the correct criteria for codes in the database is the following:
For example, this is an example entry in my codes table
id, campaign_id, code
1, 1, CODE01
2, 1, CODE02
3, 1, CODE03
If any of these codes appear again somewhere, This should not be inserted into the codes table but it needs to be inserted into the duplicate_codes table because they were already uploaded before.
id, campaign_id, code
1, 1, CODE01.
2, 1, CODE02
3, 1, CODE03
As for the codes_temp table I don't have any unique constraints, so there is no criteria to select the right one.
postgresql log into another table with on conflict
Postgres insert on conflict update using other table
Postgres on conflict - insert to another table
How to do INSERT INTO SELECT and ON DUPLICATE UPDATE in PostgreSQL 9.5?
Seems to me something like:
INSERT INTO
codes
SELECT
distinct on(campaign_id, code) *
FROM
codes_temp ct
ORDER BY
campaign_id, code, id DESC;
Assuming id was assigned sequentially, the above would select the most recent row into codes.
Then:
INSERT INTO
duplicate_codes
SELECT
*
FROM
codes_temp AS ct
LEFT JOIN
codes
ON
ct.id = codes.id
WHERE
codes.id IS NULL;
The above would select the rows in codes_temp that where not selected into codes into the duplicates table.
Obviously not tested on your data set. I would create a small test data set that has uniqueness conflicts and test with.
I have a table, i have auto numbering/sequence on data_id
tabledata
---------
data_id [PK]
data_code [Unique]
data_desc
example code:
insert into tabledata(data_code,data_desc) values(Z01,'red')
on conflict (data_code) do update set data_desc=excluded.data_desc
works fine, and then i insert again
insert into tabledata(data_code,data_desc) values(Z01,'blue')
on conflict (data_code) do update set data_desc=excluded.data_desc
i got this error
[Err] ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
this is my real code
insert into psa_aso_branch(branch_code,branch_desc,regional_code,status,created_date,lastmodified_date)
(select branch_code, branch, kode_regional,
case when status_data='Y' then true
else false end, current_date, current_date
from branch_history) on conflict (branch_code) do
update set branch_desc = excluded.branch_desc, regional_code = excluded.regional_code,status = (case when excluded.status='Y' then true else false end), created_date=current_date, lastmodified_date=current_date;
working fine on first, but not the next one (like the example i give you before)
You can use update on the existing record/row, and not on row you are inserting.
Here update in on conflict clause applies to row in excluded table, which holds row temporarily.
In the first case record is inserted since there is no clash on data_code and update is not executed at all.
In the second insert you are inserting Z01 which is already inserted as data_code and data_code is unique.
The excluded table still holds the duplicate value of data_code after the update, so the record is not inserted. In update set data_code have to be changed in order to insert record properly.
I have been stuck on this issue for about 24 hours.
It is weird when I test the query on cli and it's works fine. It is working fine when I make an insertion using one data row. This errors only appear when I'm using insert-select.
It is not mostly because of insert-select problem. It is because the select rows is not unique. This will trigger the CONFLICT for more than once.
Thanks to #zivaricha comment. I experiment from his notes. Just that its hard to understand at first.
Solution:
Using distinct to make sure the select returns unique result.
This error comes when the duplicacy occurs multiple times in the single insertion
for example you have column a , b , c and combination of a and b is unique and on duplicate you are updating c.
Now suppose you already have a = 1 , b = 2 , c = 3 and you are inserting a = 1 b = 2 c = 4 and a = 1 b = 2 c = 4
so means conflict occurs twice so it cant update a row twice
I think what is happening here
when you do an update on conflict, it does an update that re conflicts again and then throws that error
We can find the error message from the source code, which we can simply understand why we got ON CONFLICT DO UPDATE command cannot affect row a second time.
In the source code of PostgreSQL at src/backend/executor/nodeModifyTable.c and the function of ExecOnConflictUpdate(), we can find this comment:
This can occur when a just inserted tuple is updated again in the same command. E.g. because multiple rows with the same conflicting key values are inserted.
This is somewhat similar to the ExecUpdate() TM_SelfModified case. We do not want to proceed because it would lead to the same row being updated a second time in some unspecified order, and in contrast to plain UPDATEs there's no historical behavior to break.
As the comment said, we can not update the row which we are inserting in INSERT ... ON CONFLICT, just like:
postgres=# CREATE TABLE t (id int primary key, name varchar);
postgres=# INSERT INTO t VALUES (1, 'smart'), (1, 'keyerror')
postgres=# ON CONFLICT (id) DO UPDATE SET name = 'Buuuuuz';
ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
Remember, the executor of postgresql is a volcano model, so it will process the data we insert one by one. When we process to (1, 'smart'), since the table is empty, we can insert normally. When we get to (1, 'keyerror'), there is a conflict with the (1, 'smart') we just inserted, so the update logic is executed, which results in updating our own inserted data, which PostgreSQL doesn't allow us to do.
Similarly, we cannot update the same row of data twice:
postgres=# DROP TABLE IF EXISTS t;
postgres=# CREATE TABLE t (id int primary key, name varchar);
postgres=# INSERT INTO t VALUES (1, 'keyerror'), (1, 'buuuuz')
postgres=# ON CONFLICT (id) DO UPDATE SET name = 'Buuuuuuuuuz';
ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
I have been using a TSQL trigger under the assumption that the inserted table always contains records for an insert or update and the deleted table always contains records for updates (I am ignoring deletes).
Here is the related MSDN article:
http://msdn.microsoft.com/en-us/library/ms191300.aspx
However, I have encountered situations where both inserted and deleted are empty. Here is a test trigger I have been using.
CREATE TRIGGER [dbo].[InsertUpdateTest] ON [dbo].[Test]
AFTER INSERT, UPDATE
AS
DECLARE #countInserted INT
DECLARE #countDelete INT
SET #countInsert = (SELECT COUNT(*) FROM INSERTED)
SET #countDeleted = (SELECT COUNT(*) FROM DELETED)
IF (#countInserted = 0 AND #countDelete = 0)
BEGIN
print 'Inserted and deleted are both empty'
END
Under what conditions does this occur?
Answering my own question. This occurs when an update statement on the triggered table doesn't update any rows. The trigger still fires but with empty inserted and deleted tables. For example:
UPDATE Test Set somefield=0 WHERE id='Values does not exist'
I want to push all data from SourceTable to TargetTable by inserting or updating accordingly. I don't need any entries removing from TargetTable, even if they are not present in SourceTable.
This is what I have so far. Is the MERGE command the most suitable approach for this? I've had a look at some example but not quite sure on how to go about this.
My current solution truncates the whole table and then inserts of the data. I have since realised I need to keep the PK values for values that existed previously in the TargetTable. How do I change this to INsert or update depending on whether AssociatedRefId exists in the PublicRefId column of the TargetTable?
-- Truncate lancrm01sql's Baseline Official table
TRUNCATE TABLE [TargetTable]
-- Populate table
INSERT INTO [TargetTable]
(PublicRefId
,FamilyName
,GivenName
)
SELECT
AssociatedRefId
, lastname
, firstname
FROM
#TempSourceTable
MERGE TargetTable AS T
USING #TempSourceTable AS S
ON (t.PublicRefId = s.AssociatedRefId)
WHEN NOT MATCHED BY TARGET
THEN INSERT(PublicRefId, FamilyName, GivenName) VALUES (S.AssociatedRefId, S.lastname, S.firstname)
WHEN MATCHED
THEN UPDATE SET T.FamilyName = S.lastname, T.GivenName = S.firstName
This article gives a pretty good explanation on using MERGE.