I'm investigating using a cassandra trigger to make inserts into multiple tables when insert to a table occurs.
So there is one table where I'd like to use the
insert into tbl (...) values(...) if not exists;
I'd like to know how to apply this if not exists logic to a RowMutation in an ITrigger. Is it possible?
Related
I would like to create trigger to execute function to truncate local database table and insert new data.
Trigger execution must start after new row have insert in foreign database table.
I have read a lot about creating triggers on foreign table, but for me its not working. Trigger seems to not execute function when new row will be inserted in foreign table. It seems like trigger cant see this new row insert event.
What I did:
Created foreign table in my local database, lets call it 'foreign_table'. I tested, I can read data.
Created function to truncate local table and insert new data:
CREATE or replace FUNCTION public.reset_insert_table()
RETURNS TRIGGER
LANGUAGE 'plpgsql'
SET search_path=public
AS $BODY$
BEGIN
create temporary table temporary_table_tmp
as select * from public.table1;
TRUNCATE TABLE public.table2;
insert into table2
select * from temporary_table_tmp;
DROP table temporary_table_tmp;
END;
$BODY$;
Created trigger to launch function 'reset_insert_table()'
CREATE TRIGGER local_table_update
AFTER INSERT
ON 'foreign_table'
FOR EACH ROW EXECUTE PROCEDURE reset_insert_table();
Made test: inserted new row in foreign database table 'foreign_table', but I cant see that table is truncated and new data is not inserted. Insertion to foreign_tale was done in foreign database.
Problem was also testing does this trigger function work, executing manually will produce error:
EXECUTE PROCEDURE reset_insert_table();
ERROR: syntax error at or near "execute"
Tried also CALL and SELECT.
I created same function for testing but instead defining 'RETURNS TRIGGER'used 'RETURNS VOID' and function is working.
Can anyone tell why my solution is not working and does trigger on foreign tables must see events happening in foreign tables?
According to your comments, you seem to be using logical replication.
While data modifications are replayed on the standby with logical replication, the parameter session_replication_role is set to replica to keep triggers and foreign key constraints from working.
If you want a trigger to be triggered by the replay of data via logical replication, you have to declare it as a replica trigger:
ALTER TABLE a2 ENABLE REPLICA TRIGGER trigger_name;
I'm using a PostgreSQL RDS instance in AWS. Basically, there is a query that inserts data into a first table, let's call it table. The data there can have duplicates in some fields (except for the primary key obviously).
Then there is the trigger that updates another table, infotable, allowing no duplicates.
The trigger:
CREATE TRIGGER insert_infotable AFTER INSERT ON table
FOR EACH ROW EXECUTE PROCEDURE insert_infotable();
The relevant part of the trigger function looks like this:
CREATE OR REPLACE FUNCTION insert_infotable() RETURNS trigger AS $insert_infotable$
BEGIN
--some irrelevant code
IF NOT EXISTS (SELECT * FROM infotable WHERE col1 = NEW.col1 AND col2 = NEW.col2) THEN
INSERT INTO infotable(col1, col2, col3, col4, col5, col6) values (--some values--);
END IF;
RETURN NEW;
END;
$insert_infotable$ LANGUAGE plpgsql;
The table infotable has a UNIQUE constraint on the columns col1 and col2.
In general all is working fine, but rarely, about once in 1k inserts, the trigger returns an error 'duplicate key value violates unique constraint "unique_col1_and_col2"' for table infotable. Which shouldn't happen since there is the IF NOT EXISTS part in the trigger function.
The first question is what might be the cause of this? The only thing I can think of is races where two users are getting the same info simultaneously, both trigger the trigger but then one updates the second table via trigger and the second user gets the duplicate error. And because of that his whole insert query fails, including the insert to the main table.
If that's the case, what can I do about it? Is using a lock on insert a good idea for a table that is supposed to have 100+ users inserting data simultaneously?
And if yes, what type of lock should I use and what table should I lock -- the main table, or the second one, which gets modified by the trigger? (or I guess should I have the lock with my main insert statement or inside the trigger function?)
Yes, this is a race condition. Two such triggers running concurrently won't see each other's modifications, because the transactions are not yet committed.
Since you have a unique constraint on infotable, you can simply use
INSERT INTO infotable ...
ON CONFLICT (col1, col2) DO NOTHING;
So I am new to using procedures and triggers and it is really confusing me
I have used temporal tables and want to basically create a history table of records inserted,updated or deleted.
Infact I have created my history table and works fine when I use this trigger sql
DROP TRIGGER if exists versioning_trigger on mytable;
CREATE TRIGGER versioning_trigger BEFORE INSERT OR UPDATE OR DELETE ON mytable FOR EACH ROW EXECUTE PROCEDURE versioning('sys_period', 'table_history', true);
This creates records of the rows updated or deleted,precisely copies the old row record from mytable into table_history table and updates the record in mytable.But I want to insert the updated record from mytable to table_history also so that it has records of all types('current active record'and 'record before updation').Also insert some other fields in table_history when the trigger is executed.
I want to ask
How is it possible to have different trigger events(BEFORE or AFTER) together in one CREATE TRIGGER query in temporal_tables?
Is it possible to insert new field values in table_history on trigger execution? How can I accomplish this?
https://www.postgresql.org/docs/current/static/plpgsql-trigger.html
A trigger procedure is created with the CREATE FUNCTION command,
declaring it as a function with no arguments and a return type of
trigger
and also
same trigger can't fire both before and after event - just create two triggers if you really need it
https://www.postgresql.org/docs/current/static/sql-createtrigger.html
Determines whether the function is called before, after, or instead of
the event.
use NEW instead of OLD for new values
https://www.postgresql.org/docs/current/static/plpgsql-trigger.html
NEW
Data type RECORD; variable holding the new database row for
INSERT/UPDATE operations in row-level triggers. This variable is
unassigned in statement-level triggers and for DELETE operations.
This question already has an answer here:
How to bulk insert only new rows in PostreSQL
(1 answer)
Closed 8 years ago.
I'm trying to achieve database abstraction in my project, but now I got stuck with doing a bulk INSERT in PostgreSQL. My project is in C# and I'm using PostgreSQL 9.3 with npgsql.dll 2.0.14.
For Microsoft SQL Server I'm doing the bulk INSERT simply by concatenating all statements and then performing an ExecuteNonQuery:
IF NOT EXISTS (SELECT id FROM table WHERE id = 1) INSERT INTO table (id) VALUES (1);
IF NOT EXISTS (SELECT id FROM table WHERE id = 2) INSERT INTO table (id) VALUES (2);
IF NOT EXISTS (SELECT id FROM table WHERE id = 3) INSERT INTO table (id) VALUES (3);
Though the IF-NOT-EXISTS clause can be substituted in PostgreSQL by a SELECT-WHERE, this approach unfortunately still doesn't work - because every single statement in PostgreSQL is committed separately.
So I googled for another solution and found the approach of using the COPY command along with NpgsqlCopySerializer/NpgsqlCopyIn to performantly "stream" the bulk data. But now I'm getting primary key violation errors all the time - 'cause the EXISTS/WHERE clause can seemingly not be used together with the COPY statement.
I would really like to avoid to do the INSERTs all one-by-one, as this will slow down my application extremely, so I hope that anyone solved this issue already!
Generally for this type of situation I'd have a separate staging table that does not have the PK constraint, which I'd populate using COPY (assuming the data were in a format for which it makes sense to do a COPY). Then I'd do something like:
insert into table
select a.*
from staging a
where not exists (select 1
from table
where a.id = b.id)
That approach isn't too far off from your original design.
I don't totally understand this part of your question, however, which doesn't even seem totally relevant to your question:
this approach unfortunately still doesn't work - because every single
statement in postgreSQL is committed separately.
That's not true at all, not for any RDBMS. Sure, auto-commit might be enabled on your client, but that doesn't mean that postgres commits every statement separately and that you can't disable the auto-commit. This approach would work:
begin;
insert into table (id) select 1 where not exists (select 1 from table where id = 1);
insert into table (id) select 2 where not exists (select 1 from table where id = 2);
insert into table (id) select 3 where not exists (select 1 from table where id = 3);
commit;
As you pointed out, however, if you've got more than a handful of such statements you'll quickly be hitting some performance concerns.
I have two tables representing two different types of imagery. I am using PostGIS to represent the boundaries of those images. Here is a simplified example of those tables:
CREATE TABLE img_format_a (
id SERIAL PRIMARY KEY,
file_path VARCHAR(1000),
boundary GEOGRAPHY(POLYGON, 4326)
);
CREATE TABLE img_format_p (
id SERIAL PRIMARY KEY,
file_path VARCHAR(1000),
boundary GEOGRAPHY(POLYGON, 4326)
);
I also have a cross reference table, which I want to contain all the IDs of the images that overlap each other. Whenever an image of type "A" gets inserted into the database, I want to check to see whether it overlaps any of the existing imagery of type "P" (and vice versa) and insert corresponding entries into the img_a_img_p cross reference table. This table should represent a many-to-many relationship.
My first instinct is to write a trigger to manage thisimg_a_img_p table. I've never created a trigger before, so let me know if this is a silly thing to do, but it seems to make sense to me. So I create the following trigger:
CREATE TRIGGER update_a_p_cross_reference
AFTER INSERT OR DELETE OR UPDATE OF boundary
ON img_format_p FOR EACH ROW
EXECUTE PROCEDURE check_p_cross_reference();
The part where I am getting stuck is with writing the trigger function. My code is in Java and I see that there are tools like PL/pgSQL, but I'm not sure if that's what I should use or if I even need one of those special add-ons.
Essentially all I need the trigger to do is update the cross reference table each time a new image gets inserted into either img_format_a or img_format_p. When a new image is inserted, I would like to use a PostGIS function like ST_Intersects to determine whether the new image overlaps with any of the images in the other table. For each image pair where ST_INTERSECTS returns true, I would like to insert a new entry into img_a_img_p with the ID's of both images. Can someone help me figure out how to write this trigger function? Here is some pseudocode:
SELECT * FROM img_format_p P
WHERE ST_Intersects(A.boundary, P.boundary);
for each match in selection {
INSERT INTO img_a_img_p VALUES (A.id, P.id);
}
You could wrap the usual INSERT ... SELECT idiom in a PL/pgSQL function sort of like this:
create function check_p_cross_reference() returns trigger as
$$
begin
insert into img_a_img_p (img_a_id, img_p_id)
select a.id, p.id
from img_format_a, img_format_p
where p.id = NEW.id
and ST_Intersects(a.boundary, p.boundary);
return null;
end;
$$ language plpgsql;
Triggers have two extra variables, NEW and OLD:
NEW
Data type RECORD; variable holding the new database row for INSERT/UPDATE operations in row-level triggers. This variable is NULL in statement-level triggers and for DELETE operations.
OLD
Data type RECORD; variable holding the old database row for UPDATE/DELETE operations in row-level triggers. This variable is NULL in statement-level triggers and for INSERT operations.
So you can use NEW.id to access the new img_format_p value that's going in. You (currently) can't use the plain SQL language for triggers:
It is not currently possible to write a trigger function in the plain SQL function language.
but PL/pgSQL is pretty close. This would make sense as an AFTER INSERT trigger:
CREATE TRIGGER update_a_p_cross_reference
AFTER INSERT
ON img_format_p FOR EACH ROW
EXECUTE PROCEDURE check_p_cross_reference();
Deletes could be handled with a foreign key on img_a_img_p and a cascading delete. You could use your trigger for UPDATEs as well:
CREATE TRIGGER update_a_p_cross_reference
AFTER INSERT OR UPDATE OF boundary
ON img_format_p FOR EACH ROW
EXECUTE PROCEDURE check_p_cross_reference();
but you'd probably want to clear out the old entries before inserting the new ones with something like:
delete from img_a_img_p where img_p_id = NEW.id;
before the INSERT...SELECT statement.