I'm using PostgreSQL 9.3 and I have some stored procedures created which contains several statements. I'm calling this stored procedures in a Java application with the help of a prepared statement.
Now I've read that each statement inside the stored procedure is executed as a transaction, i.e. one commit after each statement. But what I want is to have the whole stored procedure executed as one transaction, i.e. only one commit.
How can I do this? Perhaps deactivating autocommit on the JDBC level?
Well, basically stored procedures are atomic in nature and executed as one transaction.
CREATE TABLE xxx (id int PRIMARY KEY);
CREATE OR REPLACE FUNCTION f() RETURNS void AS $$
DECLARE
len int;
BEGIN
RAISE NOTICE 'Transaction ID: %', TXID_CURRENT();
INSERT INTO xxx VALUES (1);
RAISE NOTICE 'Transaction ID: %', TXID_CURRENT();
INSERT INTO xxx VALUES (2);
RAISE NOTICE 'Transaction ID: %', TXID_CURRENT();
SELECT COUNT(*) FROM xxx INTO len;
RAISE NOTICE 'Number of records: %', len;
RAISE NOTICE 'Transaction ID: %', TXID_CURRENT();
-- results in unique constraint violation
UPDATE xxx SET id = 3;
END;
$$ LANGUAGE plpgsql;
Then try invoking f() from psql.
stackoverflow=# show autocommit;
autocommit
------------
on
(1 row)
stackoverflow=# SELECT f();
NOTICE: Transaction ID: 15086
NOTICE: Transaction ID: 15086
NOTICE: Transaction ID: 15086
NOTICE: Number of records: 2
NOTICE: Transaction ID: 15086
ERROR: duplicate key value violates unique constraint "xxx_pkey"
DETAIL: Key (id)=(3) already exists.
CONTEXT: SQL statement "UPDATE xxx SET id = 3"
PL/pgSQL function f() line 20 at SQL statement
stackoverflow=# SELECT * FROM xxx;
id
----
(0 rows)
Related
I am experiencing an issue while debugging this postgresql trigger function. While attempting to insert data and observe its result, I am getting errors (expected during the debug process). There have been no successful inserts, and all tables have 0 rows despite my (failed) tests while debugging -- this is also expected. But, for some unknown reason, my reference to NEW.id is incrementing with each failed test. NEW.id is supposed to be the primary key value of the record being inserted. The primary key is supposed to auto-increment after each successful insert (like 0, 1, 2, 3, ...). So how is it possible that NEW.id is incrementing with each failed record INSERT despite row count remaining at 0?
This behavior is preventing me from moving forward in testing. This behavior breaks the logic I am relying on to keep the database clean and structured.
Below I will provide a copy of the trigger function. Below that, I will provide a copy of the error as I see it,
-- FUNCTION: public.distribute_process_data()
-- DROP FUNCTION IF EXISTS public.distribute_process_data();
CREATE OR REPLACE FUNCTION public.distribute_process_data()
RETURNS trigger
LANGUAGE 'plpgsql'
COST 100
VOLATILE NOT LEAKPROOF
AS $BODY$
DECLARE
time_stamp timestamp;
source_id int;
medium_id int;
country_id int;
request_id int;
error_id int;
success bool;
BEGIN
select CURRENT_TIMESTAMP into time_stamp;
INSERT INTO
sources(source, category)
VALUES(NEW.source, NEW.category) ON CONFLICT (source) DO NOTHING
RETURNING id INTO source_id;
INSERT INTO
mediums(sources_id, medium)
VALUES(source_id, NEW.medium) ON CONFLICT (sources_id, medium) DO NOTHING
RETURNING id INTO medium_id;
INSERT INTO
countries(country)
VALUES(NEW.country) ON CONFLICT (country) DO NOTHING
RETURNING id INTO country_id;
INSERT INTO
requests(sources_id, request_jsonstring, request_md5hash)
VALUES(source_id, NEW.request, md5(NEW.request::text)) ON CONFLICT (request_md5hash) DO NOTHING
RETURNING id INTO request_id;
CASE
WHEN NEW.error IS NOT NULL THEN
INSERT INTO
errors(timestamp, processes_id, error)
VALUES(time_stamp, NEW.id, NEW.error)
RETURNING id INTO error_id;
select FALSE INTO success;
ELSE
select NULL INTO error_id;
select TRUE INto success;
END CASE;
INSERT INTO
processes(
id, timestamp, requests_id, errors_id, sources_id, mediums_id, countries_id, successful,
date, web_visits, conversions, ad_impressions, ad_clicks, ad_cost, companygo_ad_visits,
companynetwork_ad_visits, companygo_ad_visits_bounce_rate, downloads_desktop, downloads_mobile,
initiated_registrations, completed_registrations, paid_users, revenue
)
VALUES(
NEW.id, time_stamp, request_id, error_id, medium_id, country_id, success,
NEW.date, NEW.web_visits, NEW.conversions, NEW.ad_impressions, NEW.ad_clicks, NEW.ad_cost,
NEW.companygo_ad_visits, NEW.companynetwork_ad_visits, NEW.companygo_ad_visits_bounce_rate,
NEW.downloads_desktop, NEW.downloads_mobile, NEW.initiated_registrations, NEW.completed_registrations,
NEW.paid_users, NEW.revenue
);
RETURN trigger;
END;
$BODY$;
ALTER FUNCTION public.distribute_process_data()
OWNER TO postgres;
Here is the error:
---------------------------------------------------------------------------
ForeignKeyViolation Traceback (most recent call last)
Cell In [4], line 1
----> 1 cursor.execute(sql)
ForeignKeyViolation: insert or update on table "errors" violates foreign key constraint "processes_fkey"
DETAIL: Key (processes_id)=(13) is not present in table "processes".
CONTEXT: SQL statement "INSERT INTO
errors(timestamp, processes_id, error)
VALUES(time_stamp, NEW.id, NEW.error)
RETURNING id"
PL/pgSQL function distribute_process_data() line 34 at SQL statement
Notice where it says (processes_id)=(13). That value of 13 began at 1. After each failed INSERT during debug, it incremented by +2. So it progressed like so: 1 -> 3 -> 5 -> 7 -> 9 -> .... Since it is referencing NEW.id though, the expected behavior is that it remains at value 1. Because my tests are failing, therefore row count remains 0, therefore PRIMARY KEY should not be incrementing.
Any help or leads here is very much appreciated.
Thank you.
I'm trying to keey track of a clients database with which we sync. I need to record records_added (INSERTs) and records_updated (UPDATEs) to our table.
I'm using an UPSERT to handle the sync, and a trigger to update a table keeping track of insert/updates.
The issue is counting records that have are updated. I have 40+ columns to check, do I have to put all these in my check logic? Is there a more elegant way?
Section of code in question:
select
case
when old.uuid = new.uuid
and (
old.another_field != new.another_field,
old.and_another_field != new.and_another_field,
-- many more columns here << This is particularly painful
) then 1
else 0
end into update_count;
Reproducible example:
-- create tables
CREATE TABLE IF NOT EXISTS example (uuid serial primary key, another_field int, and_another_field int);
CREATE TABLE IF NOT EXISTS tracker_table (
records_added integer DEFAULT 0,
records_updated integer DEFAULT 0,
created_at date unique
);
-- create function
CREATE OR REPLACE FUNCTION update_records_inserted () RETURNS TRIGGER AS $body$
DECLARE update_count INT;
DECLARE insert_count INT;
BEGIN
-- ---------------- START OF BLOCK IN QUESTION -----------------
select
case
when old.uuid = new.uuid
and (
old.another_field != new.another_field
-- many more columns here
) then 1
else 0
end into update_count;
-- ------------------ END OF BLOCK IN QUESTION ------------------
-- count INSERTs
select
case
when old.uuid is null
and new.uuid is not null then 1
else 0
end into insert_count;
-- --log the counts
-- raise notice 'update %', update_count;
-- raise notice 'insert %', insert_count;
-- insert or update count to tracker table
insert into
tracker_table(
created_at,
records_added,
records_updated
)
VALUES
(CURRENT_DATE, insert_count, update_count) ON CONFLICT (created_at) DO
UPDATE
SET
records_added = tracker_table.records_added + insert_count,
records_updated = tracker_table.records_updated + update_count;
RETURN NEW;
END;
$body$ LANGUAGE plpgsql;
-- Trigger
DROP TRIGGER IF EXISTS example_trigger ON example;
CREATE TRIGGER example_trigger
AFTER
INSERT
OR
UPDATE
ON example FOR EACH ROW EXECUTE PROCEDURE update_records_inserted ();
-- A query to insert, then update when number of uses > 1
insert into example(whatever) values (2, 3) ON CONFLICT(uuid) DO UPDATE SET another_field=excluded.another_field+1;
I'm using postgres 9.4; I have a table with a unique index. I would like to mutate the name by adding a suffix to ensure the name is unique.
I have created a "before" trigger which computes a suffix. It works well in autocommit mode. However, if two items with the same name are inserted in the same transaction, they both get the same unique suffix.
What is the best way to accomplish my task? Is there a way to handle it with a trigger, or should I ... hmm... wrap the insert or update in a savepoint and then handle the error?
UPDATE (re comment from #Haleemur Ali ):
I don't think my question depends on the details. The salient point is that I query the subset of the collection over which I want to enforce uniqueness, and
choose a new name... however, it would seem that when the queries are run on two objects identically named in the same transaction, one doesn't see the others' modification to the new value.
But ... just in case... my trigger contains ("type" is fixed parameter to the trigger function):
select find_unique(coalesce(new.name, capitalize(type)),
'vis_operation', 'name', format(
'sheet_id = %s', new.sheet_id )) into new.name;
Where "find_unique" contains:
create or replace function find_unique(
stem text, table_name text, column_name text, where_expr text = null)
returns text language plpgsql as $$
declare
table_nt text = quote_ident(table_name);
column_nt text = quote_ident(column_name);
bstem text = replace(btrim(stem),'''', '''''');
find_re text = quote_literal(format('^%s(( \d+$)|$)', bstem));
xtct_re text = quote_literal(format('^(%s ?)', bstem));
where_ext text = case when where_expr is null then '' else 'and ' || where_expr end;
query_exists text = format(
$Q$ select 1 from %1$s where btrim(%2$s) = %3$s %4$s $Q$,
table_nt, column_nt, quote_literal(bstem), where_ext );
query_max text = format($q$
select max(coalesce(nullif(regexp_replace(%1$s, %4$s, '', 'i'), ''), '0')::int)
from %2$s where %1$s ~* %3$s %5$s
$q$,
column_nt, table_nt, find_re, xtct_re, where_ext );
last int;
i int;
begin
-- if no exact match, use exact
execute query_exists;
get diagnostics i = row_count;
if i = 0 then
return coalesce(bstem, capitalize(right(table_nt,4)));
end if;
-- find stem w/ number, use max plus one.
execute query_max into last;
if last is null then
return coalesce(bstem, capitalize(right(table_nt,4)));
end if;
return format('%s %s', bstem, last + 1);
end;
$$;
A BEFORE trigger sees rows modified by the statement that is currently running. So this should work. See demo below.
However, your design will not work in the presence of concurrency. You have to LOCK TABLE ... IN EXCLUSIVE MODE the table you're updating, otherwise concurrent transactions could get the same suffix. Or, with a UNIQUE constraint present, all but one will error out.
Personally I suggest:
Create a side table with the base names and a counter
When you create an entry, lock the side table in EXCLUSIVE mode. This will serialize all sessions that create entries, which is necessary so that you can:
UPDATE side_table SET counter = counter + 1 WHERE name = $1 RETURNING counter to get the next free ID. If you get zero rows, then instead:
Create a new entry in the side table if the base name being created and the counter set to zero.
Demo showing that BEFORE triggers can see rows inserted in the same statement, though not the row that fired the trigger:
craig=> CREATE TABLE demo(id integer);
CREATE TABLE
craig=> \e
CREATE FUNCTION
craig=> CREATE OR REPLACE FUNCTION demo_tg() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE
row record;
BEGIN
FOR row IN SELECT * FROM demo
LOOP
RAISE NOTICE 'Row is %',row;
END LOOP;
IF tg_op = 'DELETE' THEN
RETURN OLD;
ELSE
RETURN NEW;
END IF;
END;
$$;
CREATE FUNCTION
craig=> CREATE TRIGGER demo_tg BEFORE INSERT OR UPDATE OR DELETE ON demo FOR EACH ROW EXECUTE PROCEDURE demo_tg();
CREATE TRIGGER
craig=> INSERT INTO demo(id) VALUES (1),(2);
NOTICE: Row is (1)
INSERT 0 2
craig=> INSERT INTO demo(id) VALUES (3),(4);
NOTICE: Row is (1)
NOTICE: Row is (2)
NOTICE: Row is (1)
NOTICE: Row is (2)
NOTICE: Row is (3)
INSERT 0 2
craig=> UPDATE demo SET id = id + 100;
NOTICE: Row is (1)
NOTICE: Row is (2)
NOTICE: Row is (3)
NOTICE: Row is (4)
NOTICE: Row is (2)
NOTICE: Row is (3)
NOTICE: Row is (4)
NOTICE: Row is (101)
NOTICE: Row is (3)
NOTICE: Row is (4)
NOTICE: Row is (101)
NOTICE: Row is (102)
NOTICE: Row is (4)
NOTICE: Row is (101)
NOTICE: Row is (102)
NOTICE: Row is (103)
UPDATE 4
craig=>
I am not able to trigger the exceptions NO_DATA_FOUND from functions in PostgreSql 8.2 even if the returned rows or result sets is zero.
Here is my code;
CREATE OR REPLACE FUNCTION func_ex() RETURNS trigger AS
$func_ex$
DECLARE
var_name name;
BEGIN
Select empname INTO var_name from emp_table1 WHERE empid = 161232;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RAISE EXCEPTION 'No data found';
RETURN NEW;
END;
return new
$func_ex$ LANGUAGE plpgsql;
-- End of Function
-- Creation of Trigger
CREATE TRIGGER insert_trigger1 AFTER update of empname
ON emp_table1 EXECUTE PROCEDURE func_ex();
-- insertion enteries.
INSERT INTO emp_table1 (empid, empname, salary) values (124, ' Sapmle_CustormerName', '3000');
To trigger NO_DATA_FOUND exception use:
Select empname INTO STRICT var_name from emp_table1 WHERE empid = 161232;
Details here: http://www.postgresql.org/docs/current/static/plpgsql-statements.html
If STRICT is not specified in the INTO clause, then target will be set
to the first row returned by the query, or to nulls if the query
returned no rows.
If the STRICT option is specified, the query must return exactly one
row or a run-time error will be reported, either NO_DATA_FOUND (no
rows) or TOO_MANY_ROWS (more than one row).
I am new to PostgreSQL. Could anybody please correct this query.
BEGIN TRANSACTION;
BEGIN;
CREATE TABLE "Logs"."Events"
(
EventId BIGSERIAL NOT NULL PRIMARY KEY,
PrimaryKeyId bigint NOT NULL,
EventDateTime date NOT NULL DEFAULT(now()),
Action varchar(12) NOT NULL,
UserId integer NOT NULL REFERENCES "Office"."Users"(UserId),
PrincipalUserId varchar(50) NOT NULL DEFAULT(user)
);
CREATE TABLE "Logs"."EventDetails"
(
EventDetailId BIGSERIAL NOT NULL PRIMARY KEY,
EventId bigint NOT NULL REFERENCES "Logs"."Events"(EventId),
Resource varchar(64) NOT NULL,
OldVal varchar(4000) NOT NULL,
NewVal varchar(4000) NOT NULL
);
COMMIT TRANSACTION;
RAISE NOTICE 'Task completed sucessfully.'
EXCEPTION;
ROLLBACK TRANSACTION;
RAISE ERROR #ErrorMessage, #LineNumber, #ErrorState --how to catch errors and display them????
END;
Questions:
How to print a message like 'PRINT' in T-SQL?
How to raise errors with exception information?
To catch the error message and its code:
do $$
begin
create table yyy(a int);
create table yyy(a int); -- this will cause an error
exception when others then
raise notice 'The transaction is in an uncommittable state. '
'Transaction was rolled back';
raise notice '% %', SQLERRM, SQLSTATE;
end; $$
language 'plpgsql';
Haven't found the line number yet
UPDATE April, 16, 2019
As suggested by Diego Scaravaggi, for Postgres 9.2 and up, use GET STACKED DIAGNOSTICS:
do language plpgsql $$
declare
v_state TEXT;
v_msg TEXT;
v_detail TEXT;
v_hint TEXT;
v_context TEXT;
begin
create table yyy(a int);
create table yyy(a int); -- this will cause an error
exception when others then
get stacked diagnostics
v_state = returned_sqlstate,
v_msg = message_text,
v_detail = pg_exception_detail,
v_hint = pg_exception_hint,
v_context = pg_exception_context;
raise notice E'Got exception:
state : %
message: %
detail : %
hint : %
context: %', v_state, v_msg, v_detail, v_hint, v_context;
raise notice E'Got exception:
SQLSTATE: %
SQLERRM: %', SQLSTATE, SQLERRM;
raise notice '%', message_text; -- invalid. message_text is contextual to GET STACKED DIAGNOSTICS only
end; $$;
Result:
NOTICE: Got exception:
state : 42P07
message: relation "yyy" already exists
detail :
hint :
context: SQL statement "create table yyy(a int)"
PL/pgSQL function inline_code_block line 11 at SQL statement
NOTICE: Got exception:
SQLSTATE: 42P07
SQLERRM: relation "yyy" already exists
ERROR: column "message_text" does not exist
LINE 1: SELECT message_text
^
QUERY: SELECT message_text
CONTEXT: PL/pgSQL function inline_code_block line 33 at RAISE
SQL state: 42703
Aside from GET STACKED DIAGNOSTICS is SQL standard-compliant, its diagnostics variables (e.g., message_text) are contextual to GSD only. So if you have a field named message_text in your table, there's no chance that GSD can interfere with your field's value.
Still no line number though.
Use the DO statement, a new option in version 9.0:
DO LANGUAGE plpgsql
$$
BEGIN
CREATE TABLE "Logs"."Events"
(
EventId BIGSERIAL NOT NULL PRIMARY KEY,
PrimaryKeyId bigint NOT NULL,
EventDateTime date NOT NULL DEFAULT(now()),
Action varchar(12) NOT NULL,
UserId integer NOT NULL REFERENCES "Office"."Users"(UserId),
PrincipalUserId varchar(50) NOT NULL DEFAULT(user)
);
CREATE TABLE "Logs"."EventDetails"
(
EventDetailId BIGSERIAL NOT NULL PRIMARY KEY,
EventId bigint NOT NULL REFERENCES "Logs"."Events"(EventId),
Resource varchar(64) NOT NULL,
OldVal varchar(4000) NOT NULL,
NewVal varchar(4000) NOT NULL
);
RAISE NOTICE 'Task completed sucessfully.';
END;
$$;
You could write this as a psql script, e.g.,
START TRANSACTION;
CREATE TABLE ...
CREATE TABLE ...
COMMIT;
\echo 'Task completed sucessfully.'
and run with
psql -f somefile.sql
Raising errors with parameters isn't possible in PostgreSQL directly. When porting such code, some people encode the necessary information in the error string and parse it out if necessary.
It all works a bit differently, so be prepared to relearn/rethink/rewrite a lot of things.
Just want to add my two cents on this old post:
In my opinion, almost all of relational database engines include a commit transaction execution automatically after execute a DDL command even when you have autocommit=false, So you don't need to start a transaction to avoid a potential truncated object creation because It is completely unnecessary.