postgresql: trigger that tests for dblink connection and establishes if not present - postgresql

I'm trying to set up a trigger in postgresql 9.6 that will use dblink to insert a row into another database when a row is inserted into its own table. Since there can be a large volume of these inserts, I don't want to connect and disconnect to the database for every insertion, so I would prefer to have a persistent connection that gets used by every insert. But I think I would then have to test if the connection is available since a number of things could cause the connection to drop.
My pseudo-code is as follows:
-- 1. test if named dblink connection already exists
-- 2. if it does not, create a named dblink connection
-- 3. insert data via dblink
What I have so far is like this:
CREATE OR REPLACE FUNCTION db_link_trigger()
RETURNS trigger AS
$BODY$
BEGIN
-- 1. test if named dblink connection already exists
IF (SELECT COALESCE('dblinktest' = ANY (dblink_get_connections()), false)) = false THEN
-- 2. if it does not, create a named dblink connection
RAISE NOTICE 'dblink connection not established. Connecting now';
PERFORM dblink_connect('dblinktest', 'hostaddr=192.168.1.30 port=5433 dbname=otherdb user=myuser password=mypassword');
ELSE
RAISE NOTICE 'dblink connection already established';
END IF;
-- 3. insert data via dblink
PERFORM dblink_exec('dblinktest', 'insert into mytable(data) values(''' || NEW.data || ''');');
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
My main concerns are how to handle errors and how to handle near-simultaneous invocation of the trigger. Consider if two INSERTS come in at the same time, and there is no previous dblink named 'dblinktest'. When the first sees that the connection does not exist, it proceeds to set it up. Then the second could see that the link is also not established and try to connect itself, but it will fail since the first will connect before it, and it will raise an error as:
ERROR: duplicate connection name
How can I handle an error like that? Does postgresql have something like this python-inspired pseudo-code?
try:
if dblink connection is not established:
establish dblink connection
except 'ERROR: duplicate connection name':
pass # do nothing
finally:
insert row into other db via dblink connection

You can get the error code from this link https://www.postgresql.org/docs/10/errcodes-appendix.html i this case is:
42710 -> duplicate_object
and manage this error with PL/pgSQL BEGIN block with an EXCEPTION clause
for example:
CREATE OR REPLACE FUNCTION db_link_function()
RETURNS void AS
$BODY$
declare
rec record;
BEGIN
BEGIN
PERFORM dblink_connect('dblinktest', 'hostaddr=127.0.0.1 port=5435 dbname=dell user=postgres password=password');
--the EXCEPTION
EXCEPTION
WHEN duplicate_object THEN --code error 42710
RAISE NOTICE 'this connections exists';
END;
--select data via dblink
SELECT * FROM dblink('dblinktest','SELECT * FROM categories where category=1') AS t(cid int, cname text) into rec;
raise notice 'value of rec: %,%', rec.cid, rec.cname;
END;
$BODY$
LANGUAGE plpgsql;
I suggest using FDW instead of dblink https://www.postgresql.org/docs/10/postgres-fdw.html

Related

PL/pgSQL procedures and transaction control

I'm new to Postgres, but with experience from Oracle. Trying to create a stored procedure which is going to:
Insert a row
Handle exceptions and in case of an exception insert a row into a log table by calling dedicated procedure
Emit an audit log record into a log table in case the whole procedure ran successfully
By pseudo code:
CREATE OR REPLACE PROCEDURE test.p_insert(IN p_test_param character varying)
LANGUAGE 'plpgsql'
SECURITY DEFINER
AS $BODY$
DECLARE
-- some declarations
BEGIN
BEGIN
INSERT INTO test.a(a) VALUES (p_test_param);
EXCEPTION
WHEN OTHERS THEN
-- GET STACKED DIAGNOSTICS
CALL test.p_insert_log(...); -- Inserts a row into a log table, another COMMIT may be required?
RAISE;
END;
COMMIT; -- CAN'T DO
BEGIN
IF (SELECT test.f_debug()) THEN
CALL test.p_insert_log(...); -- Audit the execution
END IF;
END;
COMMIT; -- CAN'T DO EITHER
END;
$$BODY$$;
However when I try to test the procedure out from an anonymous block in PgAdmin such as:
BEGIN;
DO
LANGUAGE plpgsql
$$
BEGIN
CALL test.p_insert(
p_test_param => 'test'
);
END;
$$
I'm getting an error ERROR: invalid transaction termination. How can I get rid of it? My objective is to let the procedure carry out the transaction control, I don't want the caller to COMMIT or ROLLBACK anything. If I remove both COMMIT commands from the code of the procedure, it executes well, however the invoker must explicitly COMMIT or REVOKE the transaction afterwards, which is not desired. In Oracle the pseudo code with COMMIT statements would work, in Postgres it doesn't seem to work as I would like to. Could you please help me out? Thanks
Your code will work as intended. Perhaps you made some mistake in calling the code:
you cannot call the procedure from a function
you cannot call the procedure in an explicitly started transaction:
BEGIN;
CALL p_insert('something); -- will fail
COMMIT;

Not able to dynamically truncate table in Redshift Stored Procedure

I have a table in Redshift (let's call it a status table) where I set the status of tables which I want to truncate. I created a Redshift Stored Procedure in order to achieve that. Here is my code for the SP:
CREATE OR REPLACE PROCEDURE <schema>.truncate_table()
AS $$
DECLARE
v_tpsl RECORD;
exec_statement VARCHAR(256);
BEGIN
FOR v_tpsl in SELECT * from <schama>.tablename_process_status_log WHERE status = 'TRUE' LOOP
exec_statement = 'TRUNCATE TABLE <schema>.' + quote_ident(v_tpsl.staging_table_name) + '_test;';
RAISE INFO 'statement = %', exec_statement;
EXECUTE exec_statement;
END LOOP;
END;
$$
LANGUAGE plpgsql;
Now when I am CALLING the Stored Procedure, I am getting this error:
SQL Error [500310] [34000]: [Amazon](500310) Invalid operation: cursor does not exist;
I looked at the documentation of the SP to check if Truncate is possible or not. By looking at the examples, it looks like it's possible.
I am not sure what is going wrong in this. I am using RedshiftJDBC42-no-awssdk-1.2.34.1058.jar and connecting via DBeaver.
It looks like I have found the answer. According to this, Any cursor that is open (explicitly or implicitly) is closed automatically when a COMMIT, ROLLBACK, or TRUNCATE statement is processed. In my next iteration of the loop, it's trying to accessing the cursor which is already closed.

Postgresql how to multiple stored procedure in transaction

I have many stored procedure in my postgresql db,
and for some reason i need to run many procedure in transaction so if there is a error it will rollback.
is there any way to do this?
edit 1
i run this through java and for some reason i cant make transaction from java and i cant run query string, just store procedure only.
I actually thinking making procedure like this
CREATE OR REPLACE FUNCTION ldt_pricing_rule_v1_api.start()
RETURNS VOID
LANGUAGE PLPGSQL
SECURITY DEFINER
AS $$
BEGIN
EXECUTE 'begin transaction'
RETURN;
END
$$;
select ldt_pricing_rule_v1_api.start();
but it's will display this
ERROR: cannot begin/end transactions in PL/pgSQL
HINT: Use a BEGIN block with an EXCEPTION clause instead.
BEGIN ... COMMIT should to work.
BEGIN
SELECT func1();
SELECT func2();
COMMIT;
PostgreSQL 11 (it is not released yet) has procedures where you can control transactions explicitly. Procedures are started by CALL statement like any other databases. Now, PostgreSQL functions doesn't allow control transactions (explicitly).
Any PostgreSQL function is executed under transaction - explicitly started by user (like my example), or implicitly started by system (by autocommit mode).
So outer BEGIN starts explicit transaction:
BEGIN
SELECT func1();
SELECT func2();
COMMIT;
and if there is any unhandled fail, then only ROLLBACK command is available.
or implicit transaction:
CREATE OR REPLACE FUNCTION outerfx()
RETURNS void AS $$
BEGIN
PERFORM func1();
PERFORM func2();
END;
$$ LANGUAGE plpgsql;
SELECT outerfx(); -- starts outer transaction implicitly.
Now, functions func1, func2 are executed under transaction too.

How can I write a cursor in postgresql with update statement and using db link

I am going to update the table of huge record from one DB to another DB.
Here I'm using a cursor function:
CREATE OR REPLACE FUNCTION Drug("Drug" text)
RETURNS void AS
$BODY$
DECLARE
curs refcursor;
rec record;
BEGIN
OPEN curs FOR EXECUTE 'SELECT * FROM ' || quote_ident("Drug") FOR UPDATE;
LOOP
FETCH NEXT FROM curs INTO rec;
EXIT WHEN rec IS NULL;
RAISE NOTICE '%', rec."Id";
EXECUTE format('update statement with dblink', tbl)
USING rec.ctid;
END LOOP;
END
$BODY$ LANGUAGE plpgsql;
Is this correct?... or any other...
Please suggest...
What I know dblink doesn't support cursors - cursors cannot be parameter of query (cursors in PostgreSQL are very careful, usually limited to transaction). So this idea is wrong. You can generate row updates - but it will be slow.
What you can do. You can use FDW API (foreign data wrappers API) and create permanent link (foreign table to second database). Then you can send a usual UPDATE statement to second database.
I have two databases db1 and db2. db1 is source database, db2 is target database:
-- all is executed on db2
CREATE EXTENSION postgres_fdw;
CREATE SERVER db1 FOREIGN DATA WRAPPER postgres_fdw OPTIONS (dbname 'db1');
CREATE USER MAPPING FOR pavel
SERVER db1 OPTIONS (user 'pavel');
CREATE FOREIGN TABLE db1_source(a int, b int)
SERVER db1 OPTIONS (table_name 'source');
Now I have a FDW table db1_source and I can do UPDATE:
db2=# UPDATE target SET b = db1_source.b
FROM db1_source WHERE target.a = db1_source.a;
This is most effective way how to do UPDATE based on data in other database

How to save query errors in plpgsql to a table?

I need to save in a table the error code (SQLSTATE) and the error message (SQLERRM) returned by an INSERT or an UPDATE. My procedure must execute an INSERT, and if an error occurs, it must be saved into an apposite table.
But the problem is that if I use an EXCEPTION block, when an error occurs the transaction is aborted and any command after cannot execute.
How can I save the error returned by a query in a table using PLPGSQL?
There are two possible solutions:
use a CSV format of PostgreSQL log. Later you can import pg log to table by \copy statement. This way is preferred if it is possible, because it has minimal negative impact on performance.
You can emulate autonomous transactions via more techniques
PostgreSQL dblink
PostgreSQL FDW driver
Example of dblink based emulation (by Jon Roberts and me):
CREATE OR REPLACE FUNCTION fn_log_error(_function varchar,
_location int, _error varchar)
RETURNS void AS $$
DECLARE
_sql varchar;
_exec_error varchar;
BEGIN
PERFORM dblink_connect('autonom_connection', 'dbname=...');
_sql := format('INSERT INTO error_log (function_name, location,
error_message, error_time) VALUES (%L, %s, %L, %L)',
_function, _location, _error, clock_timestamp());
PERFORM dblink_exec('autonom_connection', _sql, false);
_exec_error := dblink_error_message('autonom_connection');
IF position('ERROR' in _exec_error) > 0
OR position('WARNING' in _exec_error) > 0 THEN
RAISE EXCEPTION '%', _exec_error;
END IF;
PERFORM dblink_disconnect('autonom_connection');
EXCEPTION
WHEN others THEN
PERFORM dblink_disconnect('autonom_connection');
RAISE EXCEPTION '(%)', SQLERRM;
END;
$$ LANGUAGE plpgsql;
Some other examples:
http://raghavt.blogspot.cz/2012/05/autonomous-transaction-in-postgresql-91.html
http://tapoueh.org/blog/2013/10/14-autonomous-transactions