I have 2 tables, in one I create an element, in the other I save the details of said element in different languages.
The first table:
CREATE TABLE public.element
(
idelement integer NOT NULL DEFAULT nextval('element_idelement_seq'::regclass),
image_location text,
price numeric(6,2) NOT NULL DEFAULT 0,
CONSTRAINT element_pkey PRIMARY KEY (idelement)
)
the second table:
CREATE TABLE public.elementdetails
(
idelement integer NOT NULL,
title text,
description text,
notes text,
language text NOT NULL,
CONSTRAINT elementdetails_pkey PRIMARY KEY (idelement, language)
)
I use a function to insert the new elements in plpgsql
IF _idelement = 0 THEN
INSERT INTO element (price) VALUES (0.0);
SELECT lastval() INTO _idelement;
EXECUTE FORMAT('INSERT INTO elementdetails(idelement, %I, language) VALUES (%L, %L, %L))', _fieldname, _idelement, _value, _language);
END IF;
But it raises an error saying the key constraints (idelement, language) already exists.
Simply copying the SQL appearing in the error message and executing it in another window would work with no issues and the problem appears only while within the pgsql function.
EDIT: I'll add that this worked until I noticed an issue where I let people insert the language iso both in uppercase or lowercase, so I forced the language to always be uppercase, since then I get the error message.
I found the problem:
right after that statement I had another one
IF _idelement > 0 THEN
EXECUTE FORMAT('INSERT INTO elementdetails(idelement, %I, language) VALUES (%L, %L, %L))', _fieldname, _idelement, _value, _language);
END IF;
but I forgot to put it after an ELSE so both INSERTs were executed which raised the error.
Related
Postgres has a great RETURNING clause for INSERT, DELETE and UPDATE...and it's made me a bit greedy. In a few cases, what I'd like to get is not only the current value, but the previous value:
UPDATE analytic_productivity
SET points = 1000
WHERE points > 1000
RETURNING id,
points,
OLD.points;
I don't believe there's any way to access previous values outside of the lifespan and context of a trigger. So, I'll guess what I'd like isn't possible as such. If that's right, can anyone suggest an alternative? I'm overwriting outliers with some set values, and would like to record the modified values in another table. This is why I don't know the current value in advance. This is a rare (and clearly suspect) operation, and I don't want to record the change on normal inserts and updates.
As an alternative, I'm thinking that I can select the outliers, revise them, and then write back the modifications. So, do most of the work on the client side with a couple of requests to Postgres. If so, can someone suggest the right locking level to apply between my initial SELECT and my following UPDATE? I believe that the FOR UPDATE lock is right.
Any suggestions on a smart way to capture previous values, during an update, without a trigger would be great to hear about.
Follow-up
Thanks to comments here, I experimented a bit and came up with a solution that works in my case. To make my objectives clearer:
I've got a table named outlier_rule that defines values that are too high for a specific column.
The goal is to loop over the table, and apply the rules to set outliers to a fixed value.
Stomping on outliers like this is...questionable. There must be leaks in the app's UI that allow for unreasonable values. To help track these down, I'm recording the large values in a table named outlier_change.
I'd like to push this behavior into server-side function so that any of our servers, regardless of their codebase version, can invoke the current logic.
The client servers compose and send an email with a result summary, when outliers are found and corrected.
So, a server-side function to do everything, log some data, and return a result. I've got that working, but it's got the smell of You Don't Know What You're Doing So Just Keep Adding Code Until it Works. I've at least got a better handle on using FORMAT and think I understand now that a single function can do many things, and that you can choose what to return with the RETURN clause. For reference, the various bits of code:
CREATE TABLE IF NOT EXISTS data.outlier_rule (
id uuid NOT NULL DEFAULT extensions.gen_random_uuid(),
schema_name text NOT NULL DEFAULT NULL,
table_name text NOT NULL DEFAULT NULL,
column_name text NOT NULL DEFAULT NULL,
threshold integer,
set_to integer,
CONSTRAINT outlier_rule_id_pkey
PRIMARY KEY (schema_name,table_name,column_name)
);
For tracking the modifications, I've got a second table named outlier_change:
------------------------------
-- Table
------------------------------
DROP TABLE IF EXISTS data.outlier_change CASCADE;
CREATE TABLE IF NOT EXISTS data.outlier_change (
id uuid NOT NULL DEFAULT NULL,
outlier_rule_id uuid NOT NULL DEFAULT NULL,
value_was integer NOT NULL DEFAULT NULL,
set_to integer NOT NULL DEFAULT NULL,
change_count integer NOT NULL DEFAULT 0,
last_changed_dts timestamptz NOT NULL DEFAULT NOW(),
CONSTRAINT outlier_change_id_pkey
PRIMARY KEY (id,outlier_rule_id)
);
ALTER TABLE data.outlier_change OWNER TO user_change_structure;
------------------------------
-- Trigger Function
------------------------------
CREATE OR REPLACE FUNCTION data.on_outlier_change_upsert()
RETURNS pg_catalog.trigger AS $BODY$
BEGIN
NEW.last_changed_dts := NOW();
NEW.change_count := OLD.change_count + 1;
RETURN NEW; -- important!
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
------------------------------
-- Trigger
------------------------------
CREATE TRIGGER outlier_change_upsert BEFORE INSERT OR UPDATE ON data.outlier_change
FOR EACH ROW
EXECUTE PROCEDURE data.on_outlier_change_upsert();
DROP FUNCTION IF EXISTS data.outlier_fix ();
CREATE OR REPLACE FUNCTION data.outlier_fix ()
RETURNS TABLE (
schema_name text,
table_name text,
column_name text,
id uuid,
value_was integer,
set_to integer,
change_count integer
)
AS $$
DECLARE
rule record;
now_ timestamptz = NOW();
BEGIN
FOR rule IN SELECT * FROM data.outlier_rule LOOP
EXECUTE FORMAT (
'INSERT INTO outlier_change (
outlier_rule_id,
set_to,
id,
value_was)
SELECT %6$L,
%5$s,
%2$I.id,
%2$I.%3$I
FROM %1$I.%2$I
WHERE %3$I > %4$s
ON CONFLICT(id,outlier_rule_id) DO UPDATE SET
value_was = EXCLUDED.value_was,
set_to = EXCLUDED.set_to
RETURNING outlier_rule_id,
id,
value_was,
set_to
change_count;
UPDATE %1$I.%2$I
SET %3$I = %5$s
WHERE %3$I > %4$s;',
rule.schema_name,
rule.table_name,
rule.column_name,
rule.threshold,
rule.set_to,
rule.id);
END LOOP;
RETURN QUERY EXECUTE ('
SELECT outlier_rule.schema_name,
outlier_rule.table_name,
outlier_rule.column_name,
outlier_change.id,
outlier_change.value_was,
outlier_change.set_to,
outlier_change.change_count
FROM outlier_change
JOIN outlier_rule ON (outlier_rule.id = outlier_change.outlier_rule_id)
WHERE last_changed_dts = $1')
USING now_;
END;
$$ LANGUAGE plpgsql;
ALTER FUNCTION data.outlier_fix() OWNER TO user_bender;
You could achieve that with a bit of a hack. You can self join the table in your update query like this:
UPDATE analytic_productivity NEW
SET points = 1000
FROM analytic_productivity OLD
WHERE NEW.points > 1000
and NEW.id = OLD.id
RETURNING NEW.id,
NEW.points,
OLD.points as old_points;
I would like to record the id of a user in the session/transaction, using SET, so I could be able to access it later in a trigger function, using current_setting. Basically, I'm trying option n2 from a very similar ticket posted previously, with the difference that I'm using PG 10.1 .
I've been trying 3 approaches to setting the variable:
SET local myvars.user_id = 4, thereby setting it locally in the transaction;
SET myvars.user_id = 4, thereby setting it in the session;
SELECT set_config('myvars.user_id', '4', false), which depending of the last argument, will be a shortcut for the previous 2 options.
None of them is usable in the trigger, which receives NULL when getting the variable through current_setting. Here is a script I've devised to troubleshoot it (can be easily used with the postgres docker image):
database=$POSTGRES_DB
user=$POSTGRES_USER
[ -z "$user" ] && user="postgres"
psql -v ON_ERROR_STOP=1 --username "$user" $database <<-EOSQL
DROP TRIGGER IF EXISTS add_transition1 ON houses;
CREATE TABLE IF NOT EXISTS houses (
id SERIAL NOT NULL,
name VARCHAR(80),
created_at TIMESTAMP WITHOUT TIME ZONE DEFAULT now(),
PRIMARY KEY(id)
);
CREATE TABLE IF NOT EXISTS transitions1 (
id SERIAL NOT NULL,
house_id INTEGER,
user_id INTEGER,
created_at TIMESTAMP WITHOUT TIME ZONE DEFAULT now(),
PRIMARY KEY(id),
FOREIGN KEY(house_id) REFERENCES houses (id) ON DELETE CASCADE
);
CREATE OR REPLACE FUNCTION add_transition1() RETURNS TRIGGER AS \$\$
DECLARE
user_id integer;
BEGIN
user_id := current_setting('myvars.user_id')::integer || NULL;
INSERT INTO transitions1 (user_id, house_id) VALUES (user_id, NEW.id);
RETURN NULL;
END;
\$\$ LANGUAGE plpgsql;
CREATE TRIGGER add_transition1 AFTER INSERT OR UPDATE ON houses FOR EACH ROW EXECUTE PROCEDURE add_transition1();
BEGIN;
%1% SELECT current_setting('myvars.user_id');
%2% SELECT set_config('myvars.user_id', '55', false);
%3% SELECT current_setting('myvars.user_id');
INSERT INTO houses (name) VALUES ('HOUSE PARTY') RETURNING houses.id;
SELECT * from houses;
SELECT * from transitions1;
COMMIT;
DROP TRIGGER IF EXISTS add_transition1 ON houses;
DROP FUNCTION IF EXISTS add_transition1;
DROP TABLE transitions1;
DROP TABLE houses;
EOSQL
The conclusion I came to was that the function is triggered in a different transaction and a different (?) session. Is this something that one can configure, so that all happens within the same context?
Handle all possible cases for the customized option properly:
option not set yet
All references to it raise an exception, including current_setting() unless called with the second parameter missing_ok. The manual:
If there is no setting named setting_name, current_setting throws an error unless missing_ok is supplied and is true.
option set to a valid integer literal
option set to an invalid integer literal
option reset (which burns down to a special case of 3.)
For instance, if you set a customized option with SET LOCAL or set_config('myvars.user_id3', '55', true), the option value is reset at the end of the transaction. It still exists, can be referenced, but it returns an empty string now ('') - which cannot be cast to integer.
Obvious mistakes in your demo aside, you need to prepare for all 4 cases. So:
CREATE OR REPLACE FUNCTION add_transition1()
RETURNS trigger AS
$func$
DECLARE
_user_id text := current_setting('myvars.user_id', true); -- see 1.
BEGIN
IF _user_id ~ '^\d+$' THEN -- one or more digits?
INSERT INTO transitions1 (user_id, house_id)
VALUES (_user_id::int, NEW.id); -- valid int, cast is safe
ELSE
INSERT INTO transitions1 (user_id, house_id)
VALUES (NULL, NEW.id); -- use NULL instead
RAISE WARNING 'Invalid user_id % for house_id % was reset to NULL!'
, quote_literal(_user_id), NEW.id; -- optional
END IF;
RETURN NULL; -- OK for AFTER trigger
END
$func$ LANGUAGE plpgsql;
db<>fiddle here
Notes:
Avoid variable names that match column names. Very error prone. One popular naming convention is to prepend variable names with an underscore: _user_id.
Assign at declaration time to save one assignment. Note the data type text. We'll cast later, after sorting out invalid input.
Avoid raising / trapping an exception if possible. The manual:
A block containing an EXCEPTION clause is significantly more expensive
to enter and exit than a block without one. Therefore, don't use
EXCEPTION without need.
Test for valid integer strings. This simple regular expression allows only digits (no leading sign, no white space): _user_id ~ '^\d+$'. I reset to NULL for any invalid input. Adapt to your needs.
I added an optional WARNING for your debugging convenience.
Cases 3. and 4. only arise because customized options are string literals (type text), valid data types cannot be enforced automatically.
Related:
User defined variables in PostgreSQL
Is there a way to define a named constant in a PostgreSQL query?
All that aside, there may be more elegant solutions for what you are trying to do without customized options, depending on your exact requirements. Maybe this:
Fastest way to get current user's OID in Postgres?
It is not clear why you are trying to concat NULL to user_id but it is obviously the cause of the problem. Get rid of it:
CREATE OR REPLACE FUNCTION add_transition1() RETURNS TRIGGER AS $$
DECLARE
user_id integer;
BEGIN
user_id := current_setting('myvars.user_id')::integer;
INSERT INTO transitions1 (user_id, house_id) VALUES (user_id, NEW.id);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
Note that
SELECT 55 || NULL
always gives NULL.
You can catch the exception when the value doesn't exist - here's the changes I made to get this to work:
CREATE OR REPLACE FUNCTION add_transition1() RETURNS TRIGGER AS $$
DECLARE
user_id integer;
BEGIN
BEGIN
user_id := current_setting('myvars.user_id')::integer;
EXCEPTION WHEN OTHERS THEN
user_id := 0;
END;
INSERT INTO transitions1 (user_id, house_id) VALUES (user_id, NEW.id);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION insert_house() RETURNS void as $$
DECLARE
user_id integer;
BEGIN
PERFORM set_config('myvars.user_id', '55', false);
INSERT INTO houses (name) VALUES ('HOUSE PARTY');
END; $$ LANGUAGE plpgsql;
I have two tables vendor_service_place and places with places.id as a foreign key in vendor_service_table. I have created a procedure which first inserts an entry into places table and then takes that id from LASTVAL() and insert an entry into vendor_service_table. But when I am executing this function I am getting
insert or update on table "vendor_service_place" violates foreign key
constraint "fk_places"
DETAIL: Key (place_id)=(2057) is not present in table "places".
CONTEXT: SQL function "insert_data" statement 2
This is my insert procedure:
CREATE FUNCTION insert_data(vendorid integer,
serviceid integer,
name text,
address text,
latitude text,
longitude text,
contact_info text,
rating numeric,
description text) RETURNS bigint AS $$
INSERT INTO places(name,address,latitude,longitude,contact_info,created_at)
VALUES (name,address,latitude,longitude,contact_info,current_timestamp);
INSERT INTO vendor_service_place(vendor_id,service_id,place_id,description,rating,created_at)
VALUES (vendorid,serviceid,LASTVAL(),description,rating,current_timestamp);
SELECT LASTVAL() as result;
$$ LANGUAGE SQL;
I am suspecting that Postgres performs some kind of batching where it executes both these statements together, that's probably why Its not able to find the id in places table. Any ideas on how to do it properly?
Seems using lastval() to get last insert id is not recommend if you are doing multiple inserts. Postgres not returning lastval() properly.
Procedure is working fine after replacing LastVal() with return id statement.
DECLARE
insert_id bigint;
BEGIN
INSERT INTO places(name,address,latitude,
longitude,contact_info,
created_at,coordinates)
VALUES (name,address,latitude,
longitude,contact_info,
current_timestamp,
ST_SetSRID(ST_MakePoint(cast (longitude as numeric),cast (latitude as numeric)),4326))
returning id into insert_id;
INSERT INTO vendor_service_place(vendor_id,service_id,place_id,
description,rating,created_at)
VALUES (vendorid,serviceid,insert_id,
description,rating,current_timestamp);
return insert_id;
END
create table test(
t_id SERIAL primary key,
t_date CONSTANT date default CURRENT_DATE
);
ERROR: syntax error at or near "date"
LINE 3: t_date CONSTANT date default CURRENT_DATE
^
********** Error **********
ERROR: syntax error at or near "date"
SQL state: 42601
For a default value you can use a function,
CREATE TABLE test(
t_id SERIAL primary key,
t_date date DEFAULT now()
);
about constant, I never used, even other SQL (!), only in a PL/SQL context ...
If you need a "no update" constraint, you can use a trigger. Example:
CREATE FUNCTION correct_update() RETURNS trigger AS $$
BEGIN
NEW.t_date=OLD.t_date;
RETURN NEW;
END
$$ LANGUAGE plpgsql;
CREATE TRIGGER no_date_update
BEFORE BEFORE UPDATE ON test
FOR EACH ROW
WHEN (OLD.t_date IS DISTINCT FROM NEW.t_date)
EXECUTE PROCEDURE correct_update();
For a complete control, you need also trigg the INSERT event, (and does not need a default value anymore because insert trigger will do):
create table test(
t_id SERIAL primary key,
t_date date -- a default will be redundant
);
CREATE FUNCTION correct_date() RETURNS trigger AS $$
BEGIN
IF TG_OP = 'INSERT' THEN
NEW.t_date=now(); -- default value
ELSIF TG_OP = 'UPDATE' THEN -- optional AND OLD.t_date != NEW.t_date
NEW.t_date=OLD.t_date; -- "constant" behaviour
END IF;
RETURN NEW;
END
$$ LANGUAGE plpgsql;
CREATE TRIGGER constant_date
BEFORE INSERT OR UPDATE ON test
FOR EACH ROW
EXECUTE PROCEDURE correct_date();
The OLD.t_date != NEW.t_date comparison is optional, because not affects performance... But is a good practice to use it. Another way is to check in the trigger, by WHEN, but only update triggers can use OLD... So, the best create-triggers for the same correct_date() function (with no old/new comparison) are:
CREATE TRIGGER constant_date_ins
BEFORE INSERT ON test
FOR EACH ROW
EXECUTE PROCEDURE correct_date();
CREATE TRIGGER constant_date_upd
BEFORE UPDATE ON test
FOR EACH ROW
WHEN (OLD.t_date IS DISTINCT FROM NEW.t_date)
EXECUTE PROCEDURE correct_date();
Contextualizing in a scenario
As commented above in the question, there are a lack of contextualization , ex. explaining "why you think this should work and what it should do".
Scenario-1: the db-master need to block careless programmers
We can imagine a framework like CakePHP with a "created" field and a database-master that wants that this field have a "constant behaviour", preventing that careless programmers affects this "expected constraint".
That kind of scenario was used in the anwser.
Scenario-2: the project decision is to alert by error
This is the suggestion #IgorRomanchenko ...
... now here as a Wiki, you can EDIT and add new solution/example ...
You want a check constraint
create table test(
t_id SERIAL primary key,
t_date date default CURRENT_DATE check(t_date = current_date)
);
insert into test(t_date) values (default);
INSERT 0 1
insert into test(t_date) values ('2014-01-01');
ERROR: new row for relation "test" violates check constraint "test_t_date_check"
DETAIL: Failing row contains (2, 2014-01-01).
Or may be a foreign key constraint which allows multiple possible values and can be updated without altering the table's schema
create table date_constraint (
date_constraint date primary key
);
insert into date_constraint (date_constraint) values (current_date);
create table test(
t_id SERIAL primary key,
t_date date
default CURRENT_DATE
references date_constraint(date_constraint)
);
insert into test(t_date) values (default);
INSERT 0 1
insert into test(t_date) values ('2014-01-01');
ERROR: insert or update on table "test" violates foreign key constraint "test_t_date_fkey"
DETAIL: Key (t_date)=(2014-01-01) is not present in table "date_constraint".
http://www.postgresql.org/docs/current/static/ddl-constraints.html
I have this table
CREATE TABLE "UserCouponSentMail"
(
"IdUser" integer NOT NULL,
"IdCoupon" integer NOT NULL,
"SendType" character varying(100),
"Date" timestamp without time zone NOT NULL DEFAULT ('now'::text)::timestamp without time zone,
CONSTRAINT "pk_UserCouponSentMail" PRIMARY KEY ("IdUser" , "IdCoupon" ),
CONSTRAINT "pk_UserCouponSentMail_GroceryCoupon" FOREIGN KEY ("IdCoupon")
REFERENCES "GroceryCoupon" ("IdGroceryCoupon") MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
WITH ( OIDS=FALSE );
and this function
CREATE OR REPLACE FUNCTION "UserCouponSentMailInsertOrUpdate"(integer, integer, character varying, timestamp without time zone)
RETURNS void AS
$BODY$
BEGIN
IF (EXISTS(SELECT * FROM "UserCouponSentMail" WHERE "IdUser" = $1 AND "IdCoupon" = $2)) THEN
UPDATE "UserCouponSentMail" SET
"SendType" = $3,
"Date" = $4
WHERE
"IdUser" = $1 AND "IdCoupon" = $2;
ELSE
INSERT INTO "UserCouponSentMail" VALUES ($1, $2, $3, $4);
END IF;
RETURN;
END
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
And I don't know why, but somehow I sometimes receive this error while running that function
Unique violation: 7 ERROR: duplicate key value violates unique constraint "pk_UserCouponSentMail" CONTEXT: SQL statement "INSERT INTO "UserCouponSentMail" VALUES ( $1 , $2 , $3 , $4 )" PL/pgSQL function "UserCouponSentMailInsertOrUpdate" line 9 at SQL statement
Any idea on how could this happen?
The possibility of two scripts running this function with the same parameters at the very same time is almost impossible.
I'm using PostgreSQL 8.4.11 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.4.6 20110731 (Red Hat 4.4.6-3), 64-bit.
Thanks.
Concurrent executions with identical values for (IdUser,IdCoupon) seem like the only plausible explanation.
The question says it's almost impossible but multiple executions can't always be foresighted. For example, in the context of a web application, a double-post of a form with the exact same data may occur.
To avoid the error at the plpgsql level, you can use an exception block:
BEGIN
IF (EXISTS...) THEN
UPDATE the unique corresponding row
ELSE
INSERT new row
END IF;
EXCEPTION WHEN unique_violation THEN
UPDATE the unique row
END;
Also you may want to add some log in the exception block about the context and the time of the event in the hope of understanding why the concurrent executions happen.