I have some procedure which when i execute in dbeaver works fine without issue, however when i call it from outside program i am getting error below. I do not want to copy/paste full procedure here because it's pretty big and it works in db tool. I just copy/paste top and bottom. What could be the cause of it?
Procedure:
CREATE OR REPLACE PROCEDURE MyProcedure(lot of args..)
LANGUAGE plpgsql
AS $procedure$
DECLARE
.....
.....
COMMIT;
END;
$procedure$
;
Error:
ERROR: invalid transaction termination
Where: PL/pgSQL function MyFunction line 185 at COMMIT Call getNextException to see other errors in the batch. Line 185 at COMMIT Call getNextException to see other errors in the batch.
The documentation says:
Transaction control is only possible in CALL or DO invocations from the top level or nested CALL or DO invocations without any other intervening command. For example, if the call stack is CALL proc1() → CALL proc2() → CALL proc3(), then the second and third procedures can perform transaction control actions. But if the call stack is CALL proc1() → SELECT func2() → CALL proc3(), then the last procedure cannot do transaction control, because of the SELECT in between.
There are some other, undocumented, restrictions:
You cannot start a transaction explicitly with BEGIN and commit it inside a transaction. So the following will fail:
START TRANSACTION;
CALL procedure_with_commit();
This may be improved in future releases.
All procedures in the call stack must be written in PL/pgSQL:
CREATE PROCEDURE b() LANGUAGE plpgsql
AS 'BEGIN PERFORM 42; COMMIT; END;';
CREATE PROCEDURE a() LANGUAGE sql
AS 'CALL b()';
CALL a();
ERROR: invalid transaction termination
CONTEXT: PL/pgSQL function b() line 1 at COMMIT
SQL function "a" statement 1
As it is, transaction control inside PostgreSQL procedures is somewhat limited.
If you violate any of these rules, you will get the error message you describe in your question. You will probably have to handle transactions in the application rather than in the procedure — perhaps splitting the procedure into smaller parts makes this possible.
Related
I'm trying to translate an SQL Server transaction from a textbook into PostgreSQL. The original transaction is
BEGIN TRANSACTION
INSERT INTO Customers(cust_id,cust_name) VALUES('1000000010','Toys Emporium');
SAVE TRANSACTION StartOrder;
INSERT INTO Orders(order_num,order_date,cust_id) VALUES(20100,'1999/12/1','1000000010');
IF ##ERROR <> 0 ROLLBACK TRANSACTION StartOrder;
[additional inserts with same rollback omitted]
COMMIT TRANSACTION
But I keep getting response
ERROR: syntax error at or near "IF"
LINE 1: IF ##ERROR <> 0 THEN
^
PostgreSQL has the best architecture for developing SQL codes. So, inside the function, you can not use transaction commit or start. Transactions work only inside the procedures. But, you can create your best ACID structure only by using functions. Because the function begin keyword is equivalent to start transaction command and the end keyword of the function is equivalent to commit transaction command. One function = One transaction. But if you need to use one transaction for inserting data into two tables, you can use your insert functions for the same tables inside the one function. In this variant, your main function will be your main transaction. Inside the main function, if one of the internal functions will have an exception so, other internal functions will be rollbacked. For example, we use sub-functions and sub-sub functions inside the main function. If one of the sub-sub functions will have exceptions then your main function will be rollbacked. Using this mechanism you will control your transactions fully, without any problems.
I wrote your query in PostgreSQL, Example:
begin transaction;
INSERT INTO Customers(cust_id,cust_name) VALUES('1000000010','Toys Emporium');
savepoint StartOrder;
commit;
INSERT INTO Orders(order_num,order_date,cust_id) VALUES(20100,'1999/12/1','1000000010');
commit;
exception when others then
rollback to StartOrder;
commit;
Database is postgressql.
I want to call five stored procedure in on stored procedure that is master_call_all() and need to handle exception after every call. So if any error occur it raise the exception and not terminate the process.
Let say error occur in 3rd call scratch.sp_insert3() then after raising error next stored procedure will call. This thing I want to achieve after every call.
create or replace procedure scratch.master_call_all()
LANGUAGE plpgsql
AS $$
DECLARE
BEGIN
call scratch.sp_insert1();
call scratch.sp_insert2();
call scratch.sp_insert3();
call scratch.sp_insert4();
call scratch.sp_insert5();
commit;
END;
$$;
First off: In general this is an extremely bad plan. It is the anti-thesis of creating a Transaction (begin ... end).
Transactions are a fundamental concept of all database systems. The essential point of a transaction is that it bundles multiple steps
into a single, all-or-nothing operation. The intermediate states
between the steps are not visible to other concurrent transactions,
and if some failure occurs that prevents the transaction from
completing, then none of the steps affect the database at all.
Ref: Database Transactions
Now with that out of the way, what you are asking can be done, but it is not done cleanly. A code block in Postgres contains 3 sections declaration, execution, exception with declaration and exception optional. Further a block can be nested inside another and the nested block retains all 3 sections. So what you need is to place each ca;; statement into a nested block. Like:
create or replace procedure master_call_all()
language plpgsql
as $$
declare
begin
begin
call sp_insert1();
exception
when others then
raise notice 'Exception occurred calling sp_insert1()';
end ;
begin
call sp_insert2();
exception
when others then
raise notice 'Exception occurred calling sp_insert2()';
end ;
begin
call sp_insert3();
exception
when others then
raise notice 'Exception occurred calling sp_insert3()';
end ;
begin
call sp_insert4();
exception
when others then
raise notice 'Exception occured calling sp_insert4()';
end ;
begin
call sp_insert5();
exception
when others then
raise notice 'Exception occured calling sp_insert5()';
end ;
commit;
end;
$$;
See demo here. Unfortunately db<>fiddle does not show the messages from raise notice .... Clearly in the demo sp_insert1, and sp_insert3 will raise an exception. (To see the messages run in your environment.)
I have some procedure which when i execute in dbeaver works fine without issue, however when i call it from outside program i am getting error below. I do not want to copy/paste full procedure here because it's pretty big and it works in db tool. I just copy/paste top and bottom. What could be the cause of it?
Procedure:
CREATE OR REPLACE PROCEDURE MyProcedure(lot of args..)
LANGUAGE plpgsql
AS $procedure$
DECLARE
.....
.....
COMMIT;
END;
$procedure$
;
Error:
ERROR: invalid transaction termination
Where: PL/pgSQL function MyFunction line 185 at COMMIT Call getNextException to see other errors in the batch. Line 185 at COMMIT Call getNextException to see other errors in the batch.
The documentation says:
Transaction control is only possible in CALL or DO invocations from the top level or nested CALL or DO invocations without any other intervening command. For example, if the call stack is CALL proc1() → CALL proc2() → CALL proc3(), then the second and third procedures can perform transaction control actions. But if the call stack is CALL proc1() → SELECT func2() → CALL proc3(), then the last procedure cannot do transaction control, because of the SELECT in between.
There are some other, undocumented, restrictions:
You cannot start a transaction explicitly with BEGIN and commit it inside a transaction. So the following will fail:
START TRANSACTION;
CALL procedure_with_commit();
This may be improved in future releases.
All procedures in the call stack must be written in PL/pgSQL:
CREATE PROCEDURE b() LANGUAGE plpgsql
AS 'BEGIN PERFORM 42; COMMIT; END;';
CREATE PROCEDURE a() LANGUAGE sql
AS 'CALL b()';
CALL a();
ERROR: invalid transaction termination
CONTEXT: PL/pgSQL function b() line 1 at COMMIT
SQL function "a" statement 1
As it is, transaction control inside PostgreSQL procedures is somewhat limited.
If you violate any of these rules, you will get the error message you describe in your question. You will probably have to handle transactions in the application rather than in the procedure — perhaps splitting the procedure into smaller parts makes this possible.
According to the documentation (https://www.postgresql.org/docs/current/app-psql.html), even with AUTOCOMMIT set to off, PSQL issues an implicit BEGIN just before any command that is not already in a transaction block and is not itself a BEGIN or other transaction-control command, nor a command that cannot be executed inside a transaction block such as VACUUM. (Unfortunately CALL is not treated in the same way as VACCUM). And, according to Shaun Thomas (https://blog.2ndquadrant.com/pg-phriday-stored-procedures-postgres-11/), the invalid transaction termination error happens because it is not possible to close the current transaction (in this case the one initiated by PSQL) from within the procedure. I have tried with all the PSQL settings related to transaction control, but the invalid transaction termination error occurs with all of them; even if the commands file processed by PSQL contains only the CALL statement.
This is the procedure I'm calling:
create or replace procedure producto$cargar_imagenes(_super$ bigint, _archivo$ character varying) as $$
declare
_msg character varying;
_log rastro_proceso%ROWTYPE;
begin
perform rastro_proceso_temporal$insert(_super$);
perform producto$cargar_imagenes$biz(_super$, _archivo$);
if (_super$ is not null and _super$ > 0) then
perform producto$cargar_imagenes$log(_super$, _archivo$);
else
perform tarea_usuario$private$update(6519204281880642486, null);
end if;
commit;
end;
$$ language plpgsql set search_path = public;
It fails at the commit statement; it works if I comment it out.
Remove the SET clause. Per the documentation:
If a SET clause is attached to a procedure, then that procedure cannot execute transaction control statements (for example, COMMIT and ROLLBACK, depending on the language).
It appears that in pg11 (tested in version 11.6) you have the same problem if you include the "SECURITY DEFINER" clause in the procedure definition. So I guess SECURITY DEFINER qualifies as a "SET" clause.
When I remove SECURITY DEFINER I can include a COMMIT statement within the procedure definition without getting the ERROR:invalid transaction termination on the COMMIT statement.
It is also unfortunate that this is a run time error, and NOT a compilation error.
Is a PostgreSQL function such as the following automatically transactional?
CREATE OR REPLACE FUNCTION refresh_materialized_view(name)
RETURNS integer AS
$BODY$
DECLARE
_table_name ALIAS FOR $1;
_entry materialized_views%ROWTYPE;
_result INT;
BEGIN
EXECUTE 'TRUNCATE TABLE ' || _table_name;
UPDATE materialized_views
SET last_refresh = CURRENT_TIMESTAMP
WHERE table_name = _table_name;
RETURN 1;
END
$BODY$
LANGUAGE plpgsql VOLATILE SECURITY DEFINER;
In other words, if an error occurs during the execution of the function, will any changes be rolled back? If this isn't the default behavior, how can I make the function transactional?
PostgreSQL 12 update: there is limited support for top-level PROCEDUREs that can do transaction control. You still cannot manage transactions in regular SQL-callable functions, so the below remains true except when using the new top-level procedures.
Functions are part of the transaction they're called from. Their effects are rolled back if the transaction rolls back. Their work commits if the transaction commits. Any BEGIN ... EXCEPT blocks within the function operate like (and under the hood use) savepoints like the SAVEPOINT and ROLLBACK TO SAVEPOINT SQL statements.
The function either succeeds in its entirety or fails in its entirety, barring BEGIN ... EXCEPT error handling. If an error is raised within the function and not handled, the transaction calling the function is aborted. Aborted transactions cannot commit, and if they try to commit the COMMIT is treated as ROLLBACK, same as for any other transaction in error. Observe:
regress=# BEGIN;
BEGIN
regress=# SELECT 1/0;
ERROR: division by zero
regress=# COMMIT;
ROLLBACK
See how the transaction, which is in the error state due to the zero division, rolls back on COMMIT?
If you call a function without an explicit surounding transaction the rules are exactly the same as for any other Pg statement:
BEGIN;
SELECT refresh_materialized_view(name);
COMMIT;
(where COMMIT will fail if the SELECT raised an error).
PostgreSQL does not (yet) support autonomous transactions in functions, where the procedure/function could commit/rollback independently of the calling transaction. This can be simulated using a new session via dblink.
BUT, things that aren't transactional or are imperfectly transactional exist in PostgreSQL. If it has non-transactional behaviour in a normal BEGIN; do stuff; COMMIT; block, it has non-transactional behaviour in a function too. For example, nextval and setval, TRUNCATE, etc.
As my knowledge of PostgreSQL is less deeper than Craig Ringer´s I will try to give a shorter answer: Yes.
If you execute a function that has an error in it, none of the steps will impact in the database.
Also, if you execute a query in PgAdmin the same happen.
For example, if you execute in a query:
update your_table yt set column1 = 10 where yt.id=20;
select anything_that_do_not_exists;
The update in the row, id = 20 of your_table will not be saved in the database.
UPDATE Sep - 2018
To clarify the concept I have made a little example with non-transactional function nextval.
First, let´s create a sequence:
create sequence test_sequence start 100;
Then, let´s execute:
update your_table yt set column1 = 10 where yt.id=20;
select nextval('test_sequence');
select anything_that_do_not_exists;
Now, if we open another query and execute
select nextval('test_sequence');
We will get 101 because the first value (100) was used in the latter query (that is because the sequences are not transactional) although the update was not committed.
https://www.postgresql.org/docs/current/static/plpgsql-structure.html
It is important not to confuse the use of BEGIN/END for grouping statements in PL/pgSQL with the similarly-named SQL commands for transaction control. PL/pgSQL's BEGIN/END are only for grouping; they do not start or end a transaction. Functions and trigger procedures are always executed within a transaction established by an outer query — they cannot start or commit that transaction, since there would be no context for them to execute in. However, a block containing an EXCEPTION clause effectively forms a subtransaction that can be rolled back without affecting the outer transaction. For more about that see Section 39.6.6.
In the function level, it is not transnational. In other words, each statement in the function belongs to a single transaction, which is the default db auto commit value. Auto commit is true by default. But anyway, you have to call the function using
select schemaName.functionName()
The above statement 'select schemaName.functionName()' is a single transaction, let's name the transaction T1, and so the all the statements in the function belong to the transaction T1. In this way, the function is in a single transaction.
Postgres 14 update: All statements written in between the BEGIN and END block of a Procedure/Function is executed in a single transaction. Thus, any errors arising while execution of this block will cause automatic roll back of the transaction.
Additionally, the ATOMIC Transaction including triggers as well.