This question already has answers here:
Committing transactions while executing a postgreql Function
(4 answers)
Closed 8 years ago.
I have written a function for automation, mentioned below, which calls some other functions based on some rules. The function is giving me the desired results, but the problem that I am facing is that it does not commit the data after each of the function is processed internally. Once the main function gets completed only then it commits the entire data. I want to do a internal transaction which should commit the data as and when the internal function execution get completed. I tried giving a COMMIT statement after each of the PERFORM statements, but I got an error saying 'cannot begin/end transactions in PL/pgSQL'.
Can anyone suggest how do I go about doing a transaction inside a function.
CREATE OR REPLACE FUNCTION ccdb.fn_automation_for_updation()
RETURNS void AS
$BODY$
DECLARE
sec_col refcursor;
cnt integer;
sec_code ccdb.update_qtable%ROWTYPE;
new_cnt integer;
BEGIN
SELECT COUNT(*)
INTO cnt
FROM ccdb.update_qtable
WHERE status_flag IN (-1,1);
OPEN sec_col FOR
SELECT * FROM ccdb.update_qtable WHERE status_flag IN (-1,1);
FOR i IN 1..cnt
LOOP
FETCH sec_col INTO sec_code;
PERFORM ccdb.o_dtr_update(sec_code.section_code);
PERFORM ccdb.o_consumer_update_for_update(sec_code.section_code);
PERFORM ccdb.o_consumer_update_for_insert(sec_code.section_code);
PERFORM ccdb.o_bills_update_for_update(sec_code.section_code);
PERFORM ccdb.o_bills_update_for_insert(sec_code.section_code);
PERFORM ccdb.o_payments_update_for_update_new(sec_code.section_code);
PERFORM ccdb.o_payments_update_for_insert(sec_code.section_code);
PERFORM ccdb.o_payments_map_update_for_update(sec_code.section_code);
PERFORM ccdb.o_payments_map_update_for_insert(sec_code.section_code);
SELECT COUNT(*) INTO new_cnt FROM ccdb.update_qtable WHERE status_flag IN (-1,1);
IF new_cnt > cnt
THEN
CLOSE sec_col;
OPEN sec_col FOR
SELECT * FROM ccdb.update_table WHERE status_flag IN (-1,1);
cnt := new_cnt;
END IF;
END LOOP;
CLOSE sec_col;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
You cannot perform autonomous transactions in PostgreSQL - its functions don't support it.
You must use DBLink.
See:
Committing Records into the table while executing a postgreql Function
Are PostgreSQL functions transactional?
COMMIT in PostgreSQL stored procedure
(Marked CW because I closed the post)
Related
I have many stored procedure in my postgresql db,
and for some reason i need to run many procedure in transaction so if there is a error it will rollback.
is there any way to do this?
edit 1
i run this through java and for some reason i cant make transaction from java and i cant run query string, just store procedure only.
I actually thinking making procedure like this
CREATE OR REPLACE FUNCTION ldt_pricing_rule_v1_api.start()
RETURNS VOID
LANGUAGE PLPGSQL
SECURITY DEFINER
AS $$
BEGIN
EXECUTE 'begin transaction'
RETURN;
END
$$;
select ldt_pricing_rule_v1_api.start();
but it's will display this
ERROR: cannot begin/end transactions in PL/pgSQL
HINT: Use a BEGIN block with an EXCEPTION clause instead.
BEGIN ... COMMIT should to work.
BEGIN
SELECT func1();
SELECT func2();
COMMIT;
PostgreSQL 11 (it is not released yet) has procedures where you can control transactions explicitly. Procedures are started by CALL statement like any other databases. Now, PostgreSQL functions doesn't allow control transactions (explicitly).
Any PostgreSQL function is executed under transaction - explicitly started by user (like my example), or implicitly started by system (by autocommit mode).
So outer BEGIN starts explicit transaction:
BEGIN
SELECT func1();
SELECT func2();
COMMIT;
and if there is any unhandled fail, then only ROLLBACK command is available.
or implicit transaction:
CREATE OR REPLACE FUNCTION outerfx()
RETURNS void AS $$
BEGIN
PERFORM func1();
PERFORM func2();
END;
$$ LANGUAGE plpgsql;
SELECT outerfx(); -- starts outer transaction implicitly.
Now, functions func1, func2 are executed under transaction too.
I have Postgresql Function which has to INSERT about 1.5 million data into a table. What I want is I want to see the table getting populated with every one records insertion. Currently what is happening when I am trying with say about 1000 records, the get gets populated only after the complete function gets executed. If I stop the function half way through, no data gets populated. How can I make the record committed even if I stop after certain number of records have been inserted?
This can be done using dblink. I showed an example with one insert being committed you will need to add your while loop logic and commit every loop. You can http://www.postgresql.org/docs/9.3/static/contrib-dblink-connect.html
CREATE OR REPLACE FUNCTION log_the_dancing(ip_dance_entry text)
RETURNS INT AS
$BODY$
DECLARE
BEGIN
PERFORM dblink_connect('dblink_trans','dbname=sandbox port=5433 user=postgres');
PERFORM dblink('dblink_trans','INSERT INTO dance_log(dance_entry) SELECT ' || '''' || ip_dance_entry || '''');
PERFORM dblink('dblink_trans','COMMIT;');
PERFORM dblink_disconnect('dblink_trans');
RETURN 0;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION log_the_dancing(ip_dance_entry text)
OWNER TO postgres;
BEGIN TRANSACTION;
select log_the_dancing('The Flamingo');
select log_the_dancing('Break Dance');
select log_the_dancing('Cha Cha');
ROLLBACK TRANSACTION;
--Show records committed even though we rolled back outer transaction
select *
from dance_log;
What you're asking for is generally called an autonomous transaction.
PostgreSQL does not support autonomous transactions at this time (9.4).
To properly support them it really needs stored procedures, not just the user-defined functions it currently supports. It's also very complicated to implement autonomous tx's in PostgreSQL for a variety of internal reasons related to its session and process model.
For now, use dblink as suggested by Bob.
If you have the flexibility to change from function to procedure, from PostgreSQL 12 onwards you can do internal commits if you use procedures instead of functions, invoked by CALL command. Therefore your function will be changed to a procedure and invoked with CALL command: e.g:
CREATE PROCEDURE transaction_test2()
LANGUAGE plpgsql
AS $$
DECLARE
r RECORD;
BEGIN
FOR r IN SELECT * FROM test2 ORDER BY x LOOP
INSERT INTO test1 (a) VALUES (r.x);
COMMIT;
END LOOP;
END;
$$;
CALL transaction_test2();
More details about transaction management regarding Postgres are available here: https://www.postgresql.org/docs/12/plpgsql-transactions.html
For Postgresql 9.5 or newer you can use dynamic background workers provided by pg_background extension. It creates autonomous transaction. Please, refer the github page of the extension. The sollution is better then db_link. There is a complete guide on Autonomous transaction support in PostgreSQL. There is a third way to start autonomous transaction in Postgres, but some patching neede. Please see Peter's Eisentraut patch proposal for OracleDB-style transactions.
I want to measure the performance of postgresql code I wrote. In the code tables get created, selfwritten functions get called etc.
Looking around, I found EXPLAIN ANALYSE is the way to go.
However, as far as I understand it, the code only gets executed once. For a more realistic analysis I want to execute the code many many times and have the results of each iteration written somewhere, ideally in a table (for statistics later).
Is there a way to do this with a native postgresql function? If there is no native postgresql function, would I accomplish this with a simple loop? Further, how would I write out the information of every EXPLAIN ANALYZE iteration?
One way to do this is to write a function that runs an explain and then spool the output of that into a file (or insert that into a table).
E.g.:
create or replace function show_plan(to_explain text)
returns table (line_nr integer, line text)
as
$$
declare
l_plan_line record;
l_line integer;
begin
l_line := 1;
for l_plan_line in execute 'explain (analyze, verbose)'||to_explain loop
return query select l_line, l_plan_line."QUERY PLAN";
l_line := l_line + 1;
end loop;
end;
$$
language plpgsql;
Then you can use generate_series() to run a statement multiple times:
select g.i as run_nr, e.*
from show_plan('select * from foo') e
cross join generate_series(1,10) as g(i)
order by g.i, e.line_nr;
This will run the function 10 times with the passed SQL statement. The result can either be spooled to a file (how you do that depends on the SQL client you are using) or inserted into a table.
For an automatic analysis it's probably easer to use a more "parseable" explain format, e.g. XML or JSON. This is also easier to handle in the output as the plan is a single XML (or JSON) value instead of multiple text lines:
create or replace function show_plan_xml(to_explain text)
returns xml
as
$$
begin
return execut 'explain (analyze, verbose, format xml)'||to_explain;
end;
$$
language plpgsql;
Then use:
select g.i as run_nr, show_plan_xml('select * from foo')
from join generate_series(1,10) as g(i)
order by g.i;
I am seeking clarification of how to ensure an atomic transaction in a plpgsql function, and where the isolation level is set for this particular change to the database.
In the plpgsql function shown below, I want to make sure that BOTH the deletion AND the insertion succeed. I am getting an error when I try to wrap them in a single transaction:
ERROR: cannot begin/end transactions in PL/pgSQL
What happens during execution of the function below if another user has added a default behavior for circumstances ('RAIN', 'NIGHT', '45MPH') after this function has deleted the custom row but before it has had a chance to insert the custom row? Is there an implicit transaction wrapping the insert and delete so that both are rolled back if another user has changed either of the rows referenced by this function? Can I set the isolation level for this function?
create function foo(v_weather varchar(10), v_timeofday varchar(10), v_speed varchar(10),
v_behavior varchar(10))
returns setof CUSTOMBEHAVIOR
as $body$
begin
-- run-time error if either of these lines is un-commented
-- start transaction ISOLATION LEVEL READ COMMITTED;
-- or, alternatively, set transaction ISOLATION LEVEL READ COMMITTED;
delete from CUSTOMBEHAVIOR
where weather = 'RAIN' and timeofday = 'NIGHT' and speed= '45MPH' ;
-- if there is no default behavior insert a custom behavior
if not exists
(select id from DEFAULTBEHAVIOR where a = 'RAIN' and b = 'NIGHT' and c= '45MPH') then
insert into CUSTOMBEHAVIOR
(weather, timeofday, speed, behavior)
values
(v_weather, v_timeofday, v_speed, v_behavior);
end if;
return QUERY
select * from CUSTOMBEHAVIOR where ... ;
-- commit;
end
$body$ LANGUAGE plpgsql;
A plpgsql function automatically runs inside a transaction. It all succeeds or it all fails. The manual:
Functions and trigger procedures are always executed within a
transaction established by an outer query — they cannot start or
commit that transaction, since there would be no context for them to
execute in. However, a block containing an EXCEPTION clause
effectively forms a subtransaction that can be rolled back without
affecting the outer transaction. For more about that see Section 42.6.6.
So, if you need to, you can catch an exception that theoretically might occur (but is very unlikely).
Details on trapping errors in the manual.
Your function reviewed and simplified:
CREATE FUNCTION foo(v_weather text
, v_timeofday text
, v_speed text
, v_behavior text)
RETURNS SETOF custombehavior
LANGUAGE plpgsql AS
$func$
BEGIN
DELETE FROM custombehavior
WHERE weather = 'RAIN'
AND timeofday = 'NIGHT'
AND speed = '45MPH';
INSERT INTO custombehavior (weather, timeofday, speed, behavior)
SELECT v_weather, v_timeofday, v_speed, v_behavior
WHERE NOT EXISTS (
SELECT FROM defaultbehavior
WHERE a = 'RAIN'
AND b = 'NIGHT'
AND c = '45MPH'
);
RETURN QUERY
SELECT * FROM custombehavior WHERE ... ;
END
$func$;
If you actually need to begin/end transactions like indicated in the title look to SQL procedures in Postgres 11 or later (CREATE PROCEDURE). See:
In PostgreSQL, what is the difference between a “Stored Procedure” and other types of functions?
Update: after PostgreSQL version 11. you can control transaction inside Store Procedure.
=====
Before Version 10:
START TRANSACTION;
select foo() ;
COMMIT;
"Unfortunately Postgres has no stored procedures, so you always need to manage the transaction in the calling code" – a_horse_with_no_name
Transaction in an exception block - how?
Let's say I have a function show_files(IN file text, IN suffix text, OUT statement text). In next step the function is called:
SELECT * FROM show_files(file := 'example', suffix := '.png');
My question is: Is there any solution that I could get statement that has called this function from inside that function?
I mean, after running the SELECT the output of function (OUT statement text) should be: 'SELECT * FROM show_files(file := 'example', suffix := '.png');', or is it possible to assign this statement to the variable inside the function?
I need the functionality like those with TG_NAME, TG_OP, etc. in trigger procedures.
Maybe is it possible to retrieve this statement from SELECT current_query FROM pg_stat_activity ?
When I'm trying to use it inside a function I've got an empty record:
CREATE OR REPLACE FUNCTION f_snitch(text)
RETURNS text AS
$BODY$
declare
rr text;
BEGIN
RAISE NOTICE '.. from f_snitch.';
-- do stuff
SELECT current_query into rr FROM pg_stat_activity
WHERE current_query ilike 'f_snitch';
RETURN rr;
END
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
Any help and suggestions would be happily welcome!
TG_NAME and friends are special variables that only exist for trigger functions. Regular plpgsql functions don't have anything like that. I am fresh out of ideas how you could possibly get this inside the called function in plpgsql.
You could add RAISE NOTICE to your function so you get the desired information
CREATE OR REPLACE FUNCTION f_snitch(text)
RETURNS text LANGUAGE plpgsql AS
$func$
BEGIN
RAISE NOTICE '.. from f_snitch.';
-- do stuff
RETURN 'Snitch says hi!';
END
$func$;
Call:
SELECT f_snitch('foo')
In addition to the result, this returns a notice:
NOTICE: .. from f_snitch.
Fails to please in two respects:
Calling statement is not in the notice.
No CONTEXT in the notice.
For 1. you can use RAISE LOG instead (or set your cluster up to log NOTICES, too - which I usually don't, too verbose for me). With standard settings, you get an additional line with the STATEMENT in the database log:
LOG: .. from f_snitch.
STATEMENT: SELECT f_snitch('foo')
For 2., have a look at this related question at dba.SE. CONTEXT would look like:
CONTEXT: SQL statement "SELECT f_raise('LOG', 'My message')"
PL/pgSQL function "f_snitch" line 5 at PERFORM
Ok, I've got it!
CREATE OR REPLACE FUNCTION f_snitch(text)
RETURNS setof record AS
$BODY$
BEGIN
RETURN QUERY
SELECT current_query
FROM pg_stat_activity
<strike>ORDER BY length(current_query) DESC LIMIT 1;</strike>
where current_query ilike 'select * from f_snitch%';
-- much more reliable solution
END
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
select * from f_snitch('koper') AS (tt text);
And here is the result:
It's probably not 100% reliable solution but for small systems (for few users) it's quite ok.