Is there a best practice for unit testing a PostgreSQL 11+ PROCEDURE (NOT a FUNCTION) using pgTap.
For example, how would one recommend unit testing a stored procedure like this:
CREATE OR REPLACE PROCEDURE foo.do_something(IN i_value INT)
AS
$$
BEGIN
PERFORM foo.call_function_1(i_value);
COMMIT;
PERFORM foo.call_function_2(i_value);
COMMIT;
CALL foo.another_procedure(i_value);
END;
$$
LANGUAGE plpgsql;
This becomes difficult since pgTap unit tests run via a stored function like this:
SELECT * FROM runtests('foo'::NAME);
This executes in a transaction, making it impossible to execute stored procedures that modify transaction state by calling COMMIT or ROLLBACK.
Here is an approach I came up with inspired by using interfaces along with mocking frameworks in other languages.
First we move the COMMIT operation to a stored procedure like this:
CREATE PROCEDURE foo.do_commit()
AS
$$
BEGIN
COMMIT;
END;
$$
LANGUAGE plpgsql;
Then we alter the actual stored procedure to call do_commit instead of using COMMIT command directly. For example:
CREATE OR REPLACE PROCEDURE foo.do_something(IN i_value INT)
AS
$$
BEGIN
PERFORM foo.call_function_1(i_value);
CALL foo.do_commit();
CALL foo.another_procedure(i_value);
END;
$$
LANGUAGE plpgsql;
Since the unit tests are executed in a transaction that gets rolled back, we can replace the do_commit call temporarily to something mocked out for testing. A test could look something like this:
CREATE FUNCTION test.test_do_something()
RETURNS SETOF TEXT
AS
$$
BEGIN
CREATE TEMPORARY TABLE commit_calls
(
commit_call BOOLEAN NOT NULL DEFAULT TRUE
)
ON COMMIT DROP;
CREATE TEMPORARY TABLE function_calls
(
the_value INT NOT NULL
)
ON COMMIT DROP;
CREATE OR REPLACE PROCEDURE foo.do_commit()
AS
$mock_do_commit$
BEGIN
INSERT INTO commit_calls (commit_call)
VALUES (DEFAULT);
END;
$mock_do_commit$
LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION foo.call_function_1(i_value INT)
RETURNS VOID
AS
$mock_call_function_1$
INSERT INTO function_calls (the_value)
VALUES (i_value);
$mock_call_function_1$
LANGUAGE sql;
-- EXECUTE
CALL foo.do_something(9);
CALL foo.do_something(100);
-- VERIFY
RETURN NEXT assert.is((SELECT COUNT(*) FROM commit_calls)::INT, 2, 'verify transaction commits');
RETURN NEXT assert.bag_eq(
'SELECT the_value FROM function_calls',
'VALUES (9), (100)',
'verify function call values');
END;
$$
LANGUAGE plpgsql;
The idea is to temporarily mock out actual function calls for testing.
This way one can unit test a stored procedure without committing real transactions.
When the test ends it rolls back the transaction and the all of the changes are discarded.
Related
I'm new to Postgres, but with experience from Oracle. Trying to create a stored procedure which is going to:
Insert a row
Handle exceptions and in case of an exception insert a row into a log table by calling dedicated procedure
Emit an audit log record into a log table in case the whole procedure ran successfully
By pseudo code:
CREATE OR REPLACE PROCEDURE test.p_insert(IN p_test_param character varying)
LANGUAGE 'plpgsql'
SECURITY DEFINER
AS $BODY$
DECLARE
-- some declarations
BEGIN
BEGIN
INSERT INTO test.a(a) VALUES (p_test_param);
EXCEPTION
WHEN OTHERS THEN
-- GET STACKED DIAGNOSTICS
CALL test.p_insert_log(...); -- Inserts a row into a log table, another COMMIT may be required?
RAISE;
END;
COMMIT; -- CAN'T DO
BEGIN
IF (SELECT test.f_debug()) THEN
CALL test.p_insert_log(...); -- Audit the execution
END IF;
END;
COMMIT; -- CAN'T DO EITHER
END;
$$BODY$$;
However when I try to test the procedure out from an anonymous block in PgAdmin such as:
BEGIN;
DO
LANGUAGE plpgsql
$$
BEGIN
CALL test.p_insert(
p_test_param => 'test'
);
END;
$$
I'm getting an error ERROR: invalid transaction termination. How can I get rid of it? My objective is to let the procedure carry out the transaction control, I don't want the caller to COMMIT or ROLLBACK anything. If I remove both COMMIT commands from the code of the procedure, it executes well, however the invoker must explicitly COMMIT or REVOKE the transaction afterwards, which is not desired. In Oracle the pseudo code with COMMIT statements would work, in Postgres it doesn't seem to work as I would like to. Could you please help me out? Thanks
Your code will work as intended. Perhaps you made some mistake in calling the code:
you cannot call the procedure from a function
you cannot call the procedure in an explicitly started transaction:
BEGIN;
CALL p_insert('something); -- will fail
COMMIT;
I need that when inserting data into a table, a trigger function returns a JSON. But, from what I saw, a trigger function can only return a trigger. What alternatives are there for what I want to do?
-- Trigger Function
CREATE OR REPLACE FUNCTION fn_save_gps_data()
RETURNS json
LANGUAGE 'plpgsql'
AS
$$
DECLARE "gpsData" json;
BEGIN
-- PERFORM fn_return_gps_data();
SELECT json_agg(row_to_json(gps)) FROM tbl_admon_gps_data gps INTO "gpsData";
RETURN "gpsData";
END;
$$;
------------------------------------------------------------------------------
-- Trigger that executes the previous Function
CREATE TRIGGER trg_save_gps_data AFTER INSERT ON tbl_admon_gps_data
FOR EACH ROW
EXECUTE PROCEDURE fn_save_gps_data();
The value returned by an AFTER trigger is ignored. The trigger is only useful for its side effects.
I would like to create a temp table in a stored procedure which has a STABLE volatility category setted to store the result of a select for later usage in the stored procedure. At the end of the stored procedure this temp table is deallocated and i am sure that this temp table does not have any affect on the database, because as far as i know with this volatility category i ensure the optimizer that this stored procedure will not affect the database.
So i would like to do something like this:
Create a stored procedure which returns with a query:
CREATE OR REPLACE FUNCTION storedproc()
RETURNS TABLE
(Egy TEXT,
Ketto TEXT)
AS $$
BEGIN
RETURN QUERY SELECT * FROM temptable;
END;
$$ LANGUAGE plpgsql;
Create a stored procedure which is using the previous query:
CREATE OR REPLACE FUNCTION stablefunction()
RETURNS TABLE
(Egy TEXT,
Ketto TEXT)
AS $$
BEGIN
-- I would like to store the results here for later usage
CREATE TEMP TABLE buba AS select * from storedproc();
-- Do other stuff
-- ...
-- Reuse the results here which was stored before
END;
$$ LANGUAGE plpgsql
STABLE;
But when i want to execute this stored procedure as this:
DO
$$
BEGIN
perform stablefunction() ;
END;
$$ LANGUAGE plpgsql;
i get the following error message:
ERROR: CREATE TABLE AS is not allowed in a non-volatile function
Maybe this is not the intended usage of the stored procedures, but then is there a way for store the result of a query inside of the stored procedure for later usage in the same stored procedure, maybe like a handle or somethings?
The documentation states clearly: A stable function cannot modify the database. A temporary table is a part of a database as well, so you cannot create it, insert into, delete from etc. Your concept seems a bit strange but I don't want to judge it. There is a trick that allows what you want to do. Perfom all actions on the temp table using other functions that do not have to be stable. Example:
create or replace function create_my_temp_table()
returns void language plpgsql volatile as $$
begin
create temp table temp_table(id int);
insert into temp_table values (123);
end $$;
create or replace function stable_function()
returns text language plpgsql stable as $$
begin
perform create_my_temp_table();
return 'ok';
end $$;
Test:
select stable_function();
stable_function
-------------
ok
(1 row)
select * from temp_table;
id
-----
123
(1 row)
I'm trying to call a function from a trigger function and don't understand what control structure to use. Here's the situation:
I have 3 tables (table1, table2, table3) and two functions (Fct1 and Fct2).
Fct1 is a trigger function triggered after an insert in table1 and which makes insert in table2:
CREATE OR REPLACE FUNCTION Fct1()
RETURNS TRIGGER AS
$BODY$
BEGIN
TRUNCATE "table2";
INSERT INTO "table2"
SELECT ... FROM "table1";
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
The trigger is:
CREATE TRIGGER trig_fct1
AFTER INSERT
ON table1
FOR EACH ROW
WHEN ((pg_trigger_depth() < 1))
EXECUTE PROCEDURE Fct1();
If I do after that a SELECT "Fct2"(); everything works fine, but if I add in Fct1 a PERFORM "Fct2"(); , like this:
CREATE OR REPLACE FUNCTION Fct1()
RETURNS TRIGGER AS
$BODY$
BEGIN
TRUNCATE "table2";
INSERT INTO "table2"
SELECT ... FROM "table1";
TRUNCATE "table3";
PERFORM "Fct2"(); -- will insert into table3
RETURN NEW;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
It takes much more time to run (I never waited for the end, it's too long).
Fct2 looks like this
CREATE OR REPLACE FUNCTION "Fct2"()
RETURNS void AS
$BODY$
BEGIN
INSERT INTO "table3" ...;
RETURN;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
So, there is something I don't understand and I think it is related to these 'RETURNs' which are not clear to me. I have tried different 'solutions' but I always got errors mentioning some 'return' mismatches. Any suggestions ?
I'm using PostgreSQL 9.6
To capture long running SQL statements from functions in the log, you can use auto_explain with auto_explain.log_nested_statements set to on. But if the query doesn't even finish, that won't help a lot.
My bet is that you are blocked by a database lock. Set log_lock_waits to on and see if something is reported in the log. You should also query pg_locks to see if there are locks requested but not granted.
I have a sql UPDATE statement in a plpgsql function. I now want to call the pg_notify function for each updated row and am uncertain if my solution is the best possibility.
I am not aware of any position in the UPDATE statement itself where I could apply the function. I don't think it is possible in the SET part and if I would apply the function in the WHERE part, it would be applied to each row as it is checked and not only the updated rows, correct?
I therefore thought I could use the RETURNING part for my purposes and designed the function like this:
CREATE OR REPLACE FUNCTION function_name() RETURNS VOID AS $BODY$
BEGIN
UPDATE table1
SET a = TRUE
FROM table2
WHERE table1.b = table2.c
AND <more conditions>
RETURNING pg_notify('notification_name', table1.pk);
END;
$BODY$ LANGUAGE 'plpgsql' VOLATILE;
Unfortunately this gave me an error saying that I am not using or storing the return value of the query anywhere. I therefore tried putting PERFORM in front of the query but this seemed to be syntactically incorrect.
After trying different combinations with PERFORM my ultimate solution is this:
CREATE OR REPLACE FUNCTION function_name() RETURNS VOID AS $BODY$
DECLARE
dev_null INTEGER;
BEGIN
WITH updated AS (
UPDATE table1
SET a = TRUE
FROM table2
WHERE table1.b = table2.c
AND <more conditions>
RETURNING pg_notify('notification_name', table1.pk)
)
SELECT 1 INTO dev_null;
END;
$BODY$ LANGUAGE 'plpgsql' VOLATILE;
This works as it is supposed to, but I feel like there should be a better solution which does not temporarily store a useless result and does not use a useless variable.
Thank you for your help.
** EDIT 1 **
As can be seen in #pnorton 's answer, a trigger would do the trick in most cases. For me, however, it is not applicable as the receiver of the notifications also sometimes updates the table and I do not want to generate notifications in such a case
"I have a sql UPDATE statement in a plpgsql function. I now want to
call the pg_notify function for each updated row "
Ok I might be tempted to use a trigger Eg
CREATE TABLE foobar (id serial primary key, name varchar);
CREATE OR REPLACE FUNCTION notify_trigger() RETURNS trigger AS $$
DECLARE
BEGIN
PERFORM pg_notify('watch_tb_update', TG_TABLE_NAME || ',id,' || NEW.id );
RETURN new;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER foobar_trigger AFTER INSERT ON foobar
FOR EACH ROW EXECUTE PROCEDURE notify_trigger();
LISTEN watch_tb_update;
INSERT into foobar(id, name) values(1,'test_name');
I've tested this and it works fine