How to save query errors in plpgsql to a table? - postgresql

I need to save in a table the error code (SQLSTATE) and the error message (SQLERRM) returned by an INSERT or an UPDATE. My procedure must execute an INSERT, and if an error occurs, it must be saved into an apposite table.
But the problem is that if I use an EXCEPTION block, when an error occurs the transaction is aborted and any command after cannot execute.
How can I save the error returned by a query in a table using PLPGSQL?

There are two possible solutions:
use a CSV format of PostgreSQL log. Later you can import pg log to table by \copy statement. This way is preferred if it is possible, because it has minimal negative impact on performance.
You can emulate autonomous transactions via more techniques
PostgreSQL dblink
PostgreSQL FDW driver
Example of dblink based emulation (by Jon Roberts and me):
CREATE OR REPLACE FUNCTION fn_log_error(_function varchar,
_location int, _error varchar)
RETURNS void AS $$
DECLARE
_sql varchar;
_exec_error varchar;
BEGIN
PERFORM dblink_connect('autonom_connection', 'dbname=...');
_sql := format('INSERT INTO error_log (function_name, location,
error_message, error_time) VALUES (%L, %s, %L, %L)',
_function, _location, _error, clock_timestamp());
PERFORM dblink_exec('autonom_connection', _sql, false);
_exec_error := dblink_error_message('autonom_connection');
IF position('ERROR' in _exec_error) > 0
OR position('WARNING' in _exec_error) > 0 THEN
RAISE EXCEPTION '%', _exec_error;
END IF;
PERFORM dblink_disconnect('autonom_connection');
EXCEPTION
WHEN others THEN
PERFORM dblink_disconnect('autonom_connection');
RAISE EXCEPTION '(%)', SQLERRM;
END;
$$ LANGUAGE plpgsql;
Some other examples:
http://raghavt.blogspot.cz/2012/05/autonomous-transaction-in-postgresql-91.html
http://tapoueh.org/blog/2013/10/14-autonomous-transactions

Related

PL/pgSQL procedures and transaction control

I'm new to Postgres, but with experience from Oracle. Trying to create a stored procedure which is going to:
Insert a row
Handle exceptions and in case of an exception insert a row into a log table by calling dedicated procedure
Emit an audit log record into a log table in case the whole procedure ran successfully
By pseudo code:
CREATE OR REPLACE PROCEDURE test.p_insert(IN p_test_param character varying)
LANGUAGE 'plpgsql'
SECURITY DEFINER
AS $BODY$
DECLARE
-- some declarations
BEGIN
BEGIN
INSERT INTO test.a(a) VALUES (p_test_param);
EXCEPTION
WHEN OTHERS THEN
-- GET STACKED DIAGNOSTICS
CALL test.p_insert_log(...); -- Inserts a row into a log table, another COMMIT may be required?
RAISE;
END;
COMMIT; -- CAN'T DO
BEGIN
IF (SELECT test.f_debug()) THEN
CALL test.p_insert_log(...); -- Audit the execution
END IF;
END;
COMMIT; -- CAN'T DO EITHER
END;
$$BODY$$;
However when I try to test the procedure out from an anonymous block in PgAdmin such as:
BEGIN;
DO
LANGUAGE plpgsql
$$
BEGIN
CALL test.p_insert(
p_test_param => 'test'
);
END;
$$
I'm getting an error ERROR: invalid transaction termination. How can I get rid of it? My objective is to let the procedure carry out the transaction control, I don't want the caller to COMMIT or ROLLBACK anything. If I remove both COMMIT commands from the code of the procedure, it executes well, however the invoker must explicitly COMMIT or REVOKE the transaction afterwards, which is not desired. In Oracle the pseudo code with COMMIT statements would work, in Postgres it doesn't seem to work as I would like to. Could you please help me out? Thanks
Your code will work as intended. Perhaps you made some mistake in calling the code:
you cannot call the procedure from a function
you cannot call the procedure in an explicitly started transaction:
BEGIN;
CALL p_insert('something); -- will fail
COMMIT;

postgresql: trigger that tests for dblink connection and establishes if not present

I'm trying to set up a trigger in postgresql 9.6 that will use dblink to insert a row into another database when a row is inserted into its own table. Since there can be a large volume of these inserts, I don't want to connect and disconnect to the database for every insertion, so I would prefer to have a persistent connection that gets used by every insert. But I think I would then have to test if the connection is available since a number of things could cause the connection to drop.
My pseudo-code is as follows:
-- 1. test if named dblink connection already exists
-- 2. if it does not, create a named dblink connection
-- 3. insert data via dblink
What I have so far is like this:
CREATE OR REPLACE FUNCTION db_link_trigger()
RETURNS trigger AS
$BODY$
BEGIN
-- 1. test if named dblink connection already exists
IF (SELECT COALESCE('dblinktest' = ANY (dblink_get_connections()), false)) = false THEN
-- 2. if it does not, create a named dblink connection
RAISE NOTICE 'dblink connection not established. Connecting now';
PERFORM dblink_connect('dblinktest', 'hostaddr=192.168.1.30 port=5433 dbname=otherdb user=myuser password=mypassword');
ELSE
RAISE NOTICE 'dblink connection already established';
END IF;
-- 3. insert data via dblink
PERFORM dblink_exec('dblinktest', 'insert into mytable(data) values(''' || NEW.data || ''');');
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
My main concerns are how to handle errors and how to handle near-simultaneous invocation of the trigger. Consider if two INSERTS come in at the same time, and there is no previous dblink named 'dblinktest'. When the first sees that the connection does not exist, it proceeds to set it up. Then the second could see that the link is also not established and try to connect itself, but it will fail since the first will connect before it, and it will raise an error as:
ERROR: duplicate connection name
How can I handle an error like that? Does postgresql have something like this python-inspired pseudo-code?
try:
if dblink connection is not established:
establish dblink connection
except 'ERROR: duplicate connection name':
pass # do nothing
finally:
insert row into other db via dblink connection
You can get the error code from this link https://www.postgresql.org/docs/10/errcodes-appendix.html i this case is:
42710 -> duplicate_object
and manage this error with PL/pgSQL BEGIN block with an EXCEPTION clause
for example:
CREATE OR REPLACE FUNCTION db_link_function()
RETURNS void AS
$BODY$
declare
rec record;
BEGIN
BEGIN
PERFORM dblink_connect('dblinktest', 'hostaddr=127.0.0.1 port=5435 dbname=dell user=postgres password=password');
--the EXCEPTION
EXCEPTION
WHEN duplicate_object THEN --code error 42710
RAISE NOTICE 'this connections exists';
END;
--select data via dblink
SELECT * FROM dblink('dblinktest','SELECT * FROM categories where category=1') AS t(cid int, cname text) into rec;
raise notice 'value of rec: %,%', rec.cid, rec.cname;
END;
$BODY$
LANGUAGE plpgsql;
I suggest using FDW instead of dblink https://www.postgresql.org/docs/10/postgres-fdw.html

Postgresql how to multiple stored procedure in transaction

I have many stored procedure in my postgresql db,
and for some reason i need to run many procedure in transaction so if there is a error it will rollback.
is there any way to do this?
edit 1
i run this through java and for some reason i cant make transaction from java and i cant run query string, just store procedure only.
I actually thinking making procedure like this
CREATE OR REPLACE FUNCTION ldt_pricing_rule_v1_api.start()
RETURNS VOID
LANGUAGE PLPGSQL
SECURITY DEFINER
AS $$
BEGIN
EXECUTE 'begin transaction'
RETURN;
END
$$;
select ldt_pricing_rule_v1_api.start();
but it's will display this
ERROR: cannot begin/end transactions in PL/pgSQL
HINT: Use a BEGIN block with an EXCEPTION clause instead.
BEGIN ... COMMIT should to work.
BEGIN
SELECT func1();
SELECT func2();
COMMIT;
PostgreSQL 11 (it is not released yet) has procedures where you can control transactions explicitly. Procedures are started by CALL statement like any other databases. Now, PostgreSQL functions doesn't allow control transactions (explicitly).
Any PostgreSQL function is executed under transaction - explicitly started by user (like my example), or implicitly started by system (by autocommit mode).
So outer BEGIN starts explicit transaction:
BEGIN
SELECT func1();
SELECT func2();
COMMIT;
and if there is any unhandled fail, then only ROLLBACK command is available.
or implicit transaction:
CREATE OR REPLACE FUNCTION outerfx()
RETURNS void AS $$
BEGIN
PERFORM func1();
PERFORM func2();
END;
$$ LANGUAGE plpgsql;
SELECT outerfx(); -- starts outer transaction implicitly.
Now, functions func1, func2 are executed under transaction too.

PostgreSQL 9.3 trigger function to insert into table with parameterized name

I'm trying to dynamically partition log entries in Postgres. I have 53 child tables (1 for each week's worth of log entries), and would like to route INSERTs to a child table using a trigger.
I run the function with INSERT INTO log5 VALUES (NEW.*), and it works.
I run the function with the EXECUTE statement instead, and it fails. Within the EXECUTE statement, it's recognizing NEW as a table name and not a variable passed to the trigger function. Any ideas on how to fix? Thanks!
The error:
QUERY: INSERT INTO log5 VALUES (NEW.*)
CONTEXT: PL/pgSQL function log_roll_test() line 6 at EXECUTE statement
ERROR: missing FROM-clause entry for table "new" SQL state: 42P01
My function:
CREATE FUNCTION log_roll_test() RETURNS trigger AS $body$
DECLARE t text;
BEGIN
t := 'log' || extract(week FROM NEW.updt_ts); --child table name
--INSERT INTO log5 VALUES (NEW.*);
EXECUTE format('INSERT INTO %I VALUES (NEW.*);', t);
RETURN NULL;
END;
$body$ LANGUAGE plpgsql;
My trigger:
CREATE TRIGGER log_roll_test
BEFORE INSERT ON log FOR EACH ROW
EXECUTE PROCEDURE log_roll_test();
CREATE FUNCTION log_roll_test()
RETURNS trigger
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format('INSERT INTO %I SELECT ($1).*' -- !
, to_char(NEW.updt_ts, '"log"WW')) -- child table name
USING NEW; -- !
RETURN NULL;
END
$func$;
You cannot reference NEW inside the query string. NEW is visible in the function body, but not inside EXECUTE environment. The best solution is to pass values in the USING clause.
I also substituted the equivalent to_char(NEW.updt_ts, '"log"WW') for the table name. to_char() is faster and simpler here.

ERROR: unterminated quoted string at or near

While executing below shown trigger code using ANT I am getting the error
org.postgresql.util.PSQLException: ERROR: unterminated quoted string at or near "' DECLARE timeout integer"
Position: 57
I am able to sucessfully execute the below code through PGADmin (Provided by postgres) and command line utility "psql" and the trigger function is added but while executing through ANT it fails everytime
BEGIN TRANSACTION;
CREATE OR REPLACE FUNCTION sweeper() RETURNS trigger as '
DECLARE
timeout integer;
BEGIN
timeout = 30 * 24 * 60 * 60 ;
DELETE FROM diagnosticdata WHERE current_timestamp - teststarttime > (timeout * ''1 sec''::interval);
return NEW;
END;
' LANGUAGE 'plpgsql';
-- Trigger: sweep on diagnosticdata
CREATE TRIGGER sweep
AFTER INSERT
ON diagnosticdata
FOR EACH ROW
EXECUTE PROCEDURE sweeper();
END;
I encountered this error in liquibase and this page was one of the first search results so I guess I share my solution at this page:
You can put your whole sql in a separate file and include this in the changeset.
Its important to set the splitStatements option to false.
The whole changeset would then look like
<changeSet author="fgrosse" id="530b61fec3ac9">
<sqlFile path="your_sql_file_here.sql" splitStatements="false"/>
</changeSet>
I always like to have those big SQL parts (like function updates and such) in separate files.
This way you get proper syntax highlighting when opening the sql file and dont have to intermix XML and SQL in one file.
Edit: as mentioned in the comments its worth noting that the sql change supports the splitStatements option as well (thx to AndreyT for pointing that out).
I had the same problem with the JDBC driver used by Liquibase.
It seems that the driver explodes each line ended by a semicolon and runs it as a separate SQL command. That is why the code below will be executed by the JDBC driver in the following sequence:
CREATE OR REPLACE FUNCTION test(text) RETURNS VOID AS ' DECLARE tmp text
BEGIN tmp := "test"
END;
' LANGUAGE plpgsql
Of course, this is invalid SQL and causes the following error:
unterminated dollar-quoted string at or near ' DECLARE tmp text
To correct this, you need to use backslashes after each line ended with semicolon:
CREATE OR REPLACE FUNCTION test(text)
RETURNS void AS ' DECLARE tmp text; \
BEGIN
tmp := "test"; \
END;' LANGUAGE plpgsql;
Alternatively, you can place the whole definition in one line.
I am using HeidiSQL client and this was solved by placing DELIMITER // before CREATE OR REPLACE statement. There is a also a 'Send batch in one go' option in HeidiSQL that essentially achieves the same thing.
This error arises as an interaction between the particular client used to connect to the server and the form of the function. To illustrate:
The following code will run without casualty in Netbeans 7, Squirrel, DbSchema, PgAdmin3
CREATE OR REPLACE FUNCTION author.revision_number()
RETURNS trigger AS
$BODY$
begin
new.rev := new.rev + 1;
new.revised := current_timestamp;
return new;
end;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
Please note that the 'begin' statement comes immediately after the '$' quoted string.
The next code will halt all the above clients except PgAdmin3.
CREATE OR REPLACE FUNCTION author.word_count()
RETURNS trigger AS
$BODY$
declare
wordcount integer := 0; -- counter for words
indexer integer := 1; -- position in the whole string
charac char(1); -- the first character of the word
prevcharac char(1);
begin
while indexer <= length(new.blab) loop
charac := substring(new.blab,indexer,1); -- first character of string
if indexer = 1 then
prevcharac := ' '; -- absolute start of counting
else
prevcharac := substring(new.blab, indexer - 1, 1); -- indexer has increased
end if;
if prevcharac = ' ' and charac != ' ' then
wordcount := wordcount + 1;
end if;
indexer := indexer + 1;
end loop;
new.words := wordcount;
return new;
end;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
The crucial difference in the second example is the 'declare' section. The ploy of using back-slashes raises an error with PgAdmin3.
In summary I suggest trying different tools. Some tools even though they are supposed to be writing text files put invisible stuff into the text. Notoriously this occurs with the Unicode BOM which will halt any php file that tries to implement sessions or namespaces.
Whilst this is no solution I hope it helps.
I had the same problem with zeos and c++ builder.
The solution in my case:
Change the property delimiter (usually ";") to another in the component (class) I used.
dm->ZSQLProcessor1->DelimiterType=sdGo;
Perhaps Ant have something similar.
I know this question was asked a long time ago but I had kind of the same issue with a Postgresql script (run from Jenkins) using Ant's SQL Task.
I tried to run this SQL (saved in a file named audit.sql):
DROP SCHEMA IF EXISTS audit CASCADE
;
CREATE SCHEMA IF NOT EXISTS audit AUTHORIZATION faktum
;
CREATE FUNCTION audit.extract_interval_trigger ()
RETURNS trigger AS $extractintervaltrigger$
BEGIN
NEW."last_change_ts" := current_timestamp;
NEW."last_change_by" := current_user;
RETURN NEW;
END;
$extractintervaltrigger$ LANGUAGE plpgsql
;
but got the error "unterminated dollar-quoted string". No problem running it from pgAdmin.
I found out that it is not the driver that split the script at every ";" but rather Ant.
At http://grokbase.com/t/postgresql/pgsql-jdbc/06cjx3s3y0/ant-sql-tag-for-dollar-quoting I found the answer:
Ant eats double-$$ as part of its variable processing. You have to use
$BODY$ (or similar) in the stored procs, and put the delimiter on its
own line (with delimitertype="row"). Ant will cooperate then.
My Ant SQL script looks like this and it works:
<sql
driver="org.postgresql.Driver" url="jdbc:postgresql://localhost:5432/jenkins"
userid="user" password="*****"
keepformat="true"
autocommit="true"
delimitertype="row"
encoding="utf-8"
src="audit.sql"
/>
This example worked for me with PostgreSQL 14.1 and HeidiSQL 9.4.0.5125
DROP TABLE IF EXISTS emp;
CREATE TABLE emp (
empname text NOT NULL,
salary integer
);
DROP TABLE IF EXISTS EMP_AUDIT;
CREATE TABLE emp_audit(
operation char(1) NOT NULL,
stamp timestamp NOT NULL,
userid text NOT NULL,
empname text NOT NULL,
salary integer
);
DELIMITER //
CREATE OR REPLACE FUNCTION process_emp_audit() RETURNS TRIGGER AS $$
BEGIN
--
-- Create a row in emp_audit to reflect the operation performed on emp,
-- make use of the special variable TG_OP to work out the operation.
--
IF (TG_OP = 'DELETE') THEN
INSERT INTO emp_audit SELECT 'D', now(), user, OLD.*;
RETURN OLD;
ELSIF (TG_OP = 'UPDATE') THEN
INSERT INTO emp_audit SELECT 'U', now(), user, NEW.*;
RETURN NEW;
ELSIF (TG_OP = 'INSERT') THEN
INSERT INTO emp_audit SELECT 'I', now(), user, NEW.*;
RETURN NEW;
END IF;
RETURN NULL; -- result is ignored since this is an AFTER trigger
END;
$$ LANGUAGE plpgsql;
DROP TRIGGER IF EXISTS emp_audit ON emp;
CREATE TRIGGER emp_audit
AFTER INSERT OR UPDATE OR DELETE ON emp
FOR EACH ROW EXECUTE PROCEDURE process_emp_audit();
I was receiving the same error because I had my semicolon in a new line like this:
WHERE colA is NULL
;
Make sure they are in a single line as
WHERE colA is NULL;