I'm trying to write a PL/pgSQL function in which I first validate parameters (mostly check whether the supplied ids exist).
When one of this validations fails, I raise an exception stating the reason so the client code can try again.
The problem I'm facing is that, for safety reasons (I can provide more context if needed, but basically I want to leave the app in a non-functional state until specialized intervention), I'd like to write some values to a table before raising the exception and rolling back the changes. It's only some of these changes that I'd like persisted (not rolled back).
I understand transactions cannot be used inside the function because there's no context, and I found that I could probably do what I want to do using dblink (which I just found out).
The thing is it really feels hackish, so I'd like to ask if this is a reasonable idea or not.
Here's some pseudocode to illustrate:
CREATE FUNCTION func(x_id INT) RETURNS INT AS $$
DECLARE
BEGIN
PERFORM * FROM x_table WHERE id = x_id;
IF NOT FOUND THEN
-- write persisting values that will prevent further
-- use, probably using dblink
RAISE EXCEPTION 'Invalid x_id: %', x_id
END IF;
-- function logic
END;
$$ LANGUAGE plpgsql;
Related
I would love to be able to validate objects representing table rows using the database's existing constraints (triggers that raise exceptions and checks) without actually inserting them into the database.
Is there currently a way one could do this in postgres? At least with BEFORE INSERT triggers and CHECK, I assume it makes no sense with AFTER INSERT triggers.
The easiest way I can think or right now would be to:
Lock the table
Insert a new row
If exception raise to the API / else DELETE the row and call it valid
Unlock
But I can see several issues with this.
A simpler way is to insert within a transaction and not commit:
BEGIN;
INSERT INTO tbl(...) VALUES (...);
-- see effects ...
ROLLBACK;
No need for additional locking. The row is never visible to any other transaction with default transacton isolation level READ COMMITTED. (You might be stalling concurrent writes that confict with the tested row.)
Notable side-effect: Sequences of serial or IDENTITY columns are advanced even if the INSERT is never committed. But gaps in sequential numbers are to be expected anyway and nothing to worry about.
Be wary of triggers with side-effects. All "transactional" SQL effects are rolled back, even most DDL commands. But some special operations (like advancing sequences) are never rolled back.
Also, DEFERRED constraints do not kick in. The manual:
DEFERRED constraints are not checked until transaction commit.
If you need this a lot, work with a copy of your table, or even your database.
Strictly speaking, while any trigger / constraint / concurrent event is allowed, there is no other way to "validate objects" than to insert them into the actual target table in the actual target database at the actual point in time. Triggers, constraints, even default values, can interact with the current state of the whole DB. The more possibilities are ruled out and requirements are reduced, the more options we might have to emulate the test.
CREATE FUNCTION validate_function ( )
RETURNS trigger LANGUAGE plpgsql
AS $function$
DECLARE
valid_flag boolean := 't';
BEGIN
--Validation code
if valid_flag = 'f' then
RAISE EXCEPTION 'This record is not valid id %', id
USING HINT = 'Please enter valid record';
RETURN NULL;
else
RETURN NEW;
end if;
END;
$function$
CREATE TRIGGER validate_rec BEFORE INSERT OR UPDATE ON some_tbl
FOR EACH ROW EXECUTE FUNCTION validate_function();
With this function and trigger you validate inside the trigger. If the new record fails validation you set the valid_flag to false and then use that to raise exception. The RETURN NULL; is probably redundant and I am not sure it will be reached, but if it is it will also abort the insert or update. If the record is valid then you RETURN NEW and the insert/update completes.
I have an exception in Oracle PL/SQL that needs to be ported to PostgreSQL pl/pgsql. Below is the oracle variant
EXCEPTION
WHEN OTHERS THEN
NULL;
What is the PL/PGSQL variant?
It's the same syntax. Compare the 2 following executions:
DO
$$
BEGIN
RAISE division_by_zero;
EXCEPTION
WHEN OTHERS THEN
NULL;
END;
$$
DO
And:
DO
$$
BEGIN
RAISE division_by_zero;
END;
$$
ERROR: division_by_zero
CONTEXT: PL/pgSQL function inline_code_block line 3 at RAISE
DO NOT port this as it is. It is a BUG ALWAYS. If you do nothing else at least log the error. If where it occurs is interactive, or data loading, or report generating, or a whole host of others then create some kind of message indicating the process failed. If users are depending on this data and it is not there it is your application that is broken, not the users expectations. I understand your are migrating a system but while doing so you should not carry forward obvious bugs. This is one.
First, avoid trapping "any exception"s like that with no specific behavior at all. Except for very simple functions, consider logging it somewhere, or rewrite the code in a more elegant manner.
I found it here: apparently, you don't need to put anything at all.
You should try the following:
EXCEPTION
WHEN OTHERS THEN
-- Do nothing
Hope it helps.
I have a general function that can manipulate the sequence of any table (why is irrelevant to my question). It reads the current value, works out the new value, sets it, and returns its calculation, which is what's inserted. This is obviously a multi-step process.
I call it from a BEFORE INSERT trigger on tables where I need it.
All I need to know is am I guaranteed that the function will be called by only one caller at a time in a multi-user environment?
Specifically, does the BEFORE INSERT trigger have to complete before it is called again by another caller?
Logically, I would assume yes, but one never knows what may be going on under the hood.
If the answer is no, what minimal locking would I need on the function to guarantee I can read and write the sequence in a "thread-safe" manner?
I'm using PG 10.
EDIT
Here is the function updated with a lock:
CREATE OR REPLACE FUNCTION public.uts_set()
RETURNS TRIGGER AS
$$
DECLARE
sv int8;
seq text := format('%I.%I_uts_seq', tg_table_schema, tg_table_name);
BEGIN
EXECUTE format('LOCK TABLE %I IN ROW EXCLUSIVE MODE;', tg_table_name);
EXECUTE 'SELECT last_value+1 FROM ' || seq INTO sv; -- currval(seq) isn't useable
PERFORM setval(seq, GREATEST(sv, (EXTRACT(epoch FROM localtimestamp) * 1000000)::int8), false);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
However, a SELECT already acquires ROW EXCLUSIVE, so this statement may be redundant and a stronger lock may be needed. Or, conversely, it may mean no lock is needed.
UPDATE
If I am reading this SO question correctly, my original version without the LOCK should work since the trigger acquires the same lock my updated function is redundantly taking.
All I need to know is am I guaranteed that the function will be called by only one caller at a time in a multi-user environment?
No. Not related to calling functions itself, but you can achieve this behaviour with SERIALIZABLE transaction isolation level:
This level emulates serial transaction execution for all committed
transactions; as if transactions had been executed one after another,
serially, rather than concurrently
But this approach would introduce several tradeoffs, such preparing your application to retry transactions with serialization failure.
Maybe a missed something, but I really believe that you just need NEXTVAL, something like below:
CREATE OR REPLACE FUNCTION public.uts_set()
RETURNS TRIGGER AS
$$
DECLARE
sv int8;
-- First, use %I wildcard for identifiers instead of %s
seq text := format('%I.%I', tg_table_schema, tg_table_name || '_uts_seq');
BEGIN
-- Second, you couldn't call CURRVAL on a session
-- that you didn't issued NEXTVAL before
sv := NEXTVAL(seq);
-- Do your logic here...
-- Result is ignored since this is an STATEMENT trigger
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
Remember that CURRVAL acts on session local scope and NEXTVAL on global scope, so you have a reliable thread-safe mechanism in hands.
The sequence itself handles thread safety with concurrent sessions. So it real comes down to the code that is interacting with the sequence. The following code is thread safe:
SELECT nextval('myseq');
If the sequence is doing much fancier things like setval and currval, I would be more worried about that being done in a high transaction/multi-user environment. Even so, the sequence itself should be locked from other queries while the sequence is being manipulated.
This is a followup question from this one so I know I can use (blocking) LOCKs but I want to use predicate locks and serializable transaction isolation.
What I'd like to have is a generic handler of serialization failures that would retry the function/query X number of times.
As example, I have this:
CREATE SEQUENCE account_id_seq;
CREATE TABLE account
(
id integer NOT NULL DEFAULT nextval('account_id_seq'),
title character varying(40) NOT NULL,
balance integer NOT NULL DEFAULT 0,
CONSTRAINT account_pkey PRIMARY KEY (id)
);
INSERT INTO account (title) VALUES ('Test Account');
CREATE OR REPLACE FUNCTION mytest() RETURNS integer AS $$
DECLARE
cc integer;
BEGIN
cc := balance from account where id=1;
RAISE NOTICE 'Balance: %', cc;
perform pg_sleep(3);
update account set balance = cc+10 where id=1 RETURNING balance INTO cc;
return cc;
END
$$
LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION myretest() RETURNS integer AS $$
DECLARE
tries integer := 5;
BEGIN
WHILE TRUE LOOP
BEGIN -- nested block for exception
RETURN mytest();
EXCEPTION
WHEN SQLSTATE '40001' THEN
IF tries > 0 THEN
tries := tries - 1;
RAISE NOTICE 'Restart! % left', tries;
ELSE
RAISE EXCEPTION 'NO RESTARTS LEFT';
END IF;
END;
END LOOP;
END
$$
LANGUAGE plpgsql;
So if call mytest() directly concurrently I get a serialization failure on the last commit:
4SO$ psql -c "select mytest()" & PIDA=$! && psql -c "select mytest()" && wait $PIDA
[1] 4909
NOTICE: Balance: 0
NOTICE: Balance: 0
mytest
--------
10
(1 row)
ERROR: could not serialize access due to concurrent update
CONTEXT: SQL statement "update account set balance = cc+10 where id=1 RETURNING balance"
PL/pgSQL function mytest() line 10 at SQL statement
If I call myretest() it should try to execute mytest() up until the 5th try where it would raise the exception.
So I have two points here (where maybe point 2 also invalidates point 1):
myretest() does not work as expected, every iteration results in serialiation_failure exception even after the concurrent thread finishes: is there something I should add to "reset" the transaction?
how could I make this (myretest() logic) generic so that it would apply to every called function in the system without the need for "wrapper" functions as such?
Serializable transactions provide exactly what you are looking for as long as you use some framework that starts the transaction over when it receives an error with a SQLSTATE of 40001 or 40P01.
In PostgreSQL a function always runs in the context of a transaction. You can't start a new transaction within the context of a "wrapper" function. That would require a slightly different feature, which is commonly called a "stored procedure" -- something which doesn't exist in PostgreSQL. Therefore, you need to put the logic to manage the restart into code which submits the transaction to the database. Fortunately, there are many connectors for that -- Java, perl, python, tcl, ODBC, etc. There is even a connector for making a separate connection to a PostgreSQL database within a PostgreSQL procedural language, which might allow you to do something like what you want:
http://www.postgresql.org/docs/current/static/dblink.html
I have seen this done in various "client" frameworks. Clearly it is a bad idea to spread this around to all locations where the application is logically dealing with the database, but there are many good reasons to route all database requests through one "accessor" method (or at least a very small number of them), and most frameworks provide a way to deal with this at that layer. (For example, in Spring you would want to create a transaction manager using dependency injection.) That probably belongs in some language you are using for your application logic, but if you really wanted to you could probably use plpgsql and dblink; that's probably not going to be your easiest path, though.
Is it possible for a trigger to be defined in such a way that the row that was to be inserted is not inserted, without raising an exception? My use case is that I want to simplify the exception handling for the client library: the client library will just execute a statement to insert a row in a table and I was hoping that the trigger could be defined, more or less using the below syntax:
CREATE TRIGGER control_tr AFTER INSERT ON tableFoo
FOR EACH ROW
EXECUTE PROCEDURE control_tr_fun();
CREATE OR REPLACE FUNCTION control_tr_fun() RETURNS TRIGGER AS $$
BEGIN
IF (NOT condition_is_met(NEW.a, NEW.b, NEW.c)) THEN
DO NOTHING INSTEAD OF INSERT // syntax I am hoping for instead of RAISE EXCEPTION
ELSE
RETURN NEW;
END IF;
END
$$ LANGUAGE plpgsql;
I appreciate that I can ask the client library to call a PL/pgSQL function or make a trigger that RAISEs an exception and ask the client library to catch the exception (if raised) and just ignore it, but I am looking for a way to implement this as transparently for the client as possible.
If you RETURN NULL then nothing will happen, the INSERT will fail silently. But you need to define the trigger as BEFORE INSERT, and not AFTER INSERT as it is in your example.
If you RAISE EXCEPTION, then the entire transaction will fail.
You can also RAISE NOTICE without failing the transaction and catch it in the client library, but only if it was the last notice produced. You cannot stack notices.