Calling now() in a function - postgresql

One of our Postgres tables, called rep_event, has a timestamp column that indicates when each row was inserted. But all of the rows have a timestamp value of 2000-01-01 00:00:00, so something isn't set up right.
There is a function that inserts rows into the table, and it is the only code that inserts rows into that table - no other code inserts into that table. (There also isn't any code that updates the rows in that table.) Here is the definition of the function:
CREATE FUNCTION handle_event() RETURNS "trigger"
AS $$
BEGIN
IF (TG_OP = 'DELETE') THEN
INSERT INTO rep_event SELECT 'D', TG_RELNAME, OLD.object_id, now();
RETURN OLD;
ELSIF (TG_OP = 'UPDATE') THEN
INSERT INTO rep_event SELECT 'U', TG_RELNAME, NEW.object_id, now();
RETURN NEW;
ELSIF (TG_OP = 'INSERT') THEN
INSERT INTO rep_event SELECT 'I', TG_RELNAME, NEW.object_id, now();
RETURN NEW;
END IF;
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
Here is the table definition:
CREATE TABLE rep_event
(
operation character(1) NOT NULL,
table_name text NOT NULL,
object_id bigint NOT NULL,
time_stamp timestamp without time zone NOT NULL
)
As you can see, the now() function is called to get the current time. Doing a "select now()" on the database returns the correct time, so is there an issue with calling now() from within a function?

A simpler solution is to just modify your table definition to have NOW() be the default value:
CREATE TABLE rep_event (
operation character(1) NOT NULL,
table_name text NOT NULL,
object_id bigint NOT NULL,
time_stamp timestamp without time zone NOT NULL DEFAULT NOW()
);
Then you can get rid of the now() calls in your trigger.
Also as a side note, I strongly suggest including the column ordering in your function... IOW;
INSERT INTO rep_event (operation,table_name,object_id,time_stamp) SELECT ...
This way if you ever add a new column or make other table changes that change the internal ordering of the tables, your function won't suddenly break.

Your problem has to be elsewhere, as your function works well. Create test database, paste the code you cited and run:
create table events (object_id bigserial, data text);
create trigger rep_event
before insert or update or delete on events
for each row execute procedure handle_event();
insert into events (data) values ('v1'),('v2'),('v3');
delete from events where data='v2';
update events set data='v4' where data='v3';
select * from events;
object_id | data
-----------+------
1 | v1
3 | v4
select * from rep_event;
operation | table_name | object_id | time_stamp
-----------+------------+-----------+----------------------------
I | events | 1 | 2011-07-08 10:31:50.489947
I | events | 2 | 2011-07-08 10:31:50.489947
I | events | 3 | 2011-07-08 10:31:50.489947
D | events | 2 | 2011-07-08 10:32:12.65699
U | events | 3 | 2011-07-08 10:32:33.662936
(5 rows)
Check other triggers, trigger creation command etc. And change this timestamp without timezone to timestamp with timezone.

Related

postgresql: How do i add a column which stores the time of creation of the row with timestamp automatically

In postgresql I have a table already created with millions of rows.
Currently I have the following table
id | name | t1 | t2 | t3 |
--------------------------
I want to have
id | name | t1 | t2 | t3 | creation_time |
------------------------------------------
And the creation_time should be time_stamp with timezone. and automatically added whenever a new row is inserted.
HOw can I do this
Also since there are so many existing rows. I would like to add NULL for them. and for the new rows I want to use the current creation time.
You can either use a default value:
-- will default to NULL
ALTER TABLE mytab ADD creation_time timestamp with time zone;
ALTER TABLE mytab ALTER creation_time SET DEFAULT current_timestamp;
You don't set the default when you create the column, otherwise the default value will be applied to all existing rows.
The alternative is a trigger (which will not perform as well):
-- will default to NULL
ALTER TABLE mytab ADD creation_time timestamp with time zone;
CREATE FUNCTION set_creation_time() RETURNS trigger
LANGUAGE plpgsql AS
$$BEGIN
NEW.creation_time = current_timestamp;
RETURN NEW;
END;$$;
CREATE TRIGGER set_creation_time BEFORE INSERT ON mytab FOR EACH ROW
EXECUTE PROCEDURE set_creation_time();

Postgres auto insert with previous values if some data misses

I have data with date range, some date wont come for few days, during the missed window I just want to insert the previous data.
Is there way to take care of this during insert of data it self
For example
create table foo (ID VARCHAR(10), foo_count int, actual_date date);
insert into foo ( values('234534', 100, '2017-01-01'),('234534', 200, '2017-01-02'));
insert into foo ( values('234534', 300, '2017-01-03') );
insert into foo ( values('234534', 300, '2017-01-08') );
After the last insert I can make sure previous data gets generated
So it should look something like this
ID | foo_count | actual_date
-----------+-----------------+------------
234534 | 100 | 2017-01-01
234534 | 200 | 2017-02-01
234534 | 300 | 2017-03-01
234534 | 300 | 2017-04-01
234534 | 300 | 2017-05-01
234534 | 300 | 2017-06-01
234534 | 180 | 2017-07-01
I am using JPA to insert it, currently I query the table and see the current date in the table and populate the missing data
I would think about a better INSERT statement. Inserting from a SELECT statement would make things easier. The SELECT statement could be used to generate the requested date series.
INSERT INTO foo
SELECT
--<advanced query>
However, I guess, that's not simply possible since you are not using the JDBC directly or want to use native queries for inserting your data.
In that case, you could install a trigger to your database which could do the magic:
demo:db<>fiddle
Trigger function:
CREATE FUNCTION insert_missing()
RETURNS TRIGGER AS
$$
DECLARE
max_record record;
BEGIN
SELECT -- 1
id,
foo_count,
actual_date
FROM
foo
ORDER BY actual_date DESC
LIMIT 1
INTO max_record;
IF (NEW.actual_date - 1 > max_record.actual_date) THEN -- 2
INSERT INTO foo
SELECT
max_record.id,
max_record.foo_count,
generate_series(max_record.actual_date + 1, NEW.actual_date - 1, interval '1 day'); -- 3
END IF;
RETURN NEW;
END;
$$ language 'plpgsql';
Query the record with the current maximum date.
If the maximum date is more than one day before the new date...
... Insert a date series (from day after current max date until the date before the new one). This can be generated with generate_series().
Afterwards create the ON BEFORE INSERT trigger:
CREATE TRIGGER insert_missing
BEFORE INSERT
ON foo
FOR EACH ROW
EXECUTE PROCEDURE insert_missing();

Weird now() time difference with Postgres triggers

In a Postgres 10.10 database, I have a table table1 , and an AFTER INSERT trigger on table1 for table2:
CREATE TABLE table1 (
id SERIAL PRIMARY KEY,
-- other cols
created_at timestamp with time zone NOT NULL,
updated_at timestamp with time zone NOT NULL
);
CREATE UNIQUE INDEX table1_pkey ON table1(id int4_ops);
CREATE TABLE table2 (
id SERIAL PRIMARY KEY,
table1_id integer NOT NULL REFERENCES table1(id) ON UPDATE CASCADE,
-- other cols (not used in query)
created_at timestamp with time zone NOT NULL,
updated_at timestamp with time zone NOT NULL
);
CREATE UNIQUE INDEX table2_pkey ON table2(id int4_ops);
This query is executed on application start:
CREATE OR REPLACE FUNCTION after_insert_table1()
RETURNS trigger AS
$$
BEGIN
INSERT INTO table2 (table1_id, ..., created_at, updated_at)
VALUES (NEW.id, ..., 'now', 'now');
RETURN NEW;
END;
$$
LANGUAGE 'plpgsql';
DROP TRIGGER IF EXISTS after_insert_table1 ON "table1";
CREATE TRIGGER after_insert_table1
AFTER INSERT ON "table1"
FOR EACH ROW
EXECUTE PROCEDURE after_insert_table1();
I noticed some created_at and updated_at values on table2 are different to table1. In fact, table2 has mostly older values.
Here are 10 sequential entries, which show the difference jumping around a huge amount within a few minutes:
|table1_id|table1_created |table2_created |diff |
|---------|--------------------------|-----------------------------|----------------|
|2000 |2019-11-07 22:29:47.245+00|2019-11-07 19:51:09.727021+00|-02:38:37.517979|
|2001 |2019-11-07 22:30:02.256+00|2019-11-07 13:18:29.45962+00 |-09:11:32.79638 |
|2002 |2019-11-07 22:30:43.021+00|2019-11-07 13:44:12.099577+00|-08:46:30.921423|
|2003 |2019-11-07 22:31:00.794+00|2019-11-07 19:51:09.727021+00|-02:39:51.066979|
|2004 |2019-11-07 22:31:11.315+00|2019-11-07 13:18:29.45962+00 |-09:12:41.85538 |
|2005 |2019-11-07 22:31:27.234+00|2019-11-07 13:44:12.099577+00|-08:47:15.134423|
|2006 |2019-11-07 22:31:47.436+00|2019-11-07 13:18:29.45962+00 |-09:13:17.97638 |
|2007 |2019-11-07 22:33:19.484+00|2019-11-07 17:22:48.129063+00|-05:10:31.354937|
|2008 |2019-11-07 22:33:51.607+00|2019-11-07 19:51:09.727021+00|-02:42:41.879979|
|2009 |2019-11-07 22:34:28.786+00|2019-11-07 13:18:29.45962+00 |-09:15:59.32638 |
|2010 |2019-11-07 22:36:50.242+00|2019-11-07 13:18:29.45962+00 |-09:18:20.78238 |
Sequential entries have similar differences (mostly negative/mostly positive), and similar orders of magnitude (mostly minutes vs mostly hours) within the sequence, though there are exceptions
Here are the top 5 largest positive differences:
|table1_id|table1_created |table2_created |diff |
|---------|--------------------------|-----------------------------|----------------|
|1630 |2019-10-25 21:12:14.971+00|2019-10-26 00:52:09.376+00 |03:39:54.405 |
|950 |2019-09-16 12:36:07.185+00|2019-09-16 14:07:35.504+00 |01:31:28.319 |
|1677 |2019-10-26 22:19:12.087+00|2019-10-26 23:38:34.102+00 |01:19:22.015 |
|58 |2018-12-08 20:11:20.306+00|2018-12-08 21:06:42.246+00 |00:55:21.94 |
|171 |2018-12-17 22:24:57.691+00|2018-12-17 23:16:05.992+00 |00:51:08.301 |
Here are the top 5 largest negative differences:
|table1_id|table1_created |table2_created |diff |
|---------|--------------------------|-----------------------------|----------------|
|1427 |2019-10-15 16:03:43.641+00|2019-10-14 17:59:41.57749+00 |-22:04:02.06351 |
|1426 |2019-10-15 13:26:07.314+00|2019-10-14 18:00:50.930513+00|-19:25:16.383487|
|1424 |2019-10-15 13:13:44.092+00|2019-10-14 18:00:50.930513+00|-19:12:53.161487|
|4416 |2020-01-11 00:15:03.751+00|2020-01-10 08:43:19.668399+00|-15:31:44.082601|
|4420 |2020-01-11 01:58:32.541+00|2020-01-10 11:04:19.288023+00|-14:54:13.252977|
Negative differences outnumber positive differences 10x. The database timezone is UTC.
table2.table1_id is a foreign key, so it should be impossible to insert before insert on table1 completes.
table1.created_at is set by Sequelize, using option timestamps: true on the model.
When a row is inserted into table1, it's done inside a transaction. From the documentation I can find, triggers are executed inside the same transaction, so I can't think of a reason for this.
I can fix the issue by changing my trigger to use NEW.created_at instead of 'now', but I'm curious if anyone has any idea what the cause of this bug is?
Here is the query used to produce the above difference tables:
SELECT
table1.id AS table1_id,
table1.created_at AS table1_created,
table2.created_at AS table2_created,
(table2.created_at - table1.created_at) AS diff
FROM table1
INNER JOIN table2 ON
table2.table1_id = table1.id AND (
(table2.created_at - table1.created_at) > '2 min' OR
(table1.created_at - table2.created_at) > '2 min')
ORDER BY diff;
While 'now' is not a plain string, it is also not a function in this context, but a special date/time input. The manual:
... simply notational shorthands that will be converted to ordinary date/time values when read. (In particular, now and related strings are converted to a specific time value as soon as they are read.)
The body of a PL/pgSQL function is stored as string, each nested SQL command is parsed and prepared when control reaches it the first time per session. The manual:
The PL/pgSQL interpreter parses the function's source text and
produces an internal binary instruction tree the first time the
function is called (within each session). The instruction tree fully
translates the PL/pgSQL statement structure, but individual SQL
expressions and SQL commands used in the function are not translated
immediately.
As each expression and SQL command is first executed in the function,
the PL/pgSQL interpreter parses and analyzes the command to create a
prepared statement, using the SPI manager's SPI_prepare function.
Subsequent visits to that expression or command reuse the prepared statement.
There is more. Read on. But that's enough for our case:
The first time the trigger is executed per session, 'now' is translated to the current timestamp (the transaction timestamp). While doing more inserts in that same transaction, there won't be any difference to transaction_timestamp() because that is stable within a transaction by design.
But every subsequent transaction in the same session will insert the same, constant timestamp in table2, while values for table1 may be anything (not sure what Sequelize does there). If new values in table1 are the then current timestamp, that results in a "negative" diff in your test. (Timestamps in table2 will be older.)
Solution
Situations where you actually want 'now' are few and far between. Typically, you want the function now() (without single quotes!) - which is equivalent to CURRENT_TIMESTAMP (standard SQL) and transaction_timestamp(). Related (recommended reading!):
Difference between now() and current_timestamp
In your particular case I suggest column defaults instead of doing additional work in triggers. If you set the same default now() in table1 and table2, you also eliminate any nonsense the INSERT to table1 might add. And you never have to even mention these columns in inserts any more:
CREATE TABLE table1 (
id SERIAL PRIMARY KEY,
-- other cols
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now() -- or leave this one NULL?
);
CREATE TABLE table2 (
id SERIAL PRIMARY KEY,
table1_id integer NOT NULL REFERENCES table1(id) ON UPDATE CASCADE,
-- other cols (not used in query)
created_at timestamptz NOT NULL DEFAULT now(), -- not 'now'!
updated_at timestamptz NOT NULL DEFAULT now() -- or leave this one NULL?
);
CREATE OR REPLACE FUNCTION after_insert_table1()
RETURNS trigger LANGUAGE plpgsql AS
$$
BEGIN
INSERT INTO table2 (table1_id) -- more columns? but not: created_at, updated_at
VALUES (NEW.id); -- more columns?
RETURN NULL; -- can be NULL for AFTER trigger
END
$$;

postgres default values are applying on update, not just at create

In the process of writing this question, I found the answer and will post it below. If there are already duplicates of this question, please notify me and I will remove it, I was unable to find any.
I've got two columns tracking changes made to a table in postgres:
created_at timestamp default now()
updated_at timestamp
the updated_at column is being updated by a trigger:
united_states_congress=> \d congressional_bill_summaries;
Table "public.congressional_bill_summaries"
Column | Type | Modifiers
-------------+-----------------------------+---------------------------------------------------------------------------
id | bigint | not null default nextval('congressional_bill_summaries_id_seq'::regclass)
text | text |
created_at | timestamp without time zone | default now()
updated_at | timestamp without time zone |
bill_kid | integer | not null
date | date | not null
description | character varying(255) | not null
text_hash | uuid |
Indexes:
"congressional_bill_summaries_pkey" PRIMARY KEY, btree (id)
"congressional_bill_summaries_bill_kid_date_description_key" UNIQUE CONSTRAINT, btree (bill_kid, date, description)
Triggers:
hash_all_the_things BEFORE INSERT ON congressional_bill_summaries FOR EACH ROW EXECUTE PROCEDURE hash_this_foo()
update_iz_yoo BEFORE UPDATE ON congressional_bill_summaries FOR EACH ROW EXECUTE PROCEDURE update_iz_now()
as is one other column of the table, text_hash
My expected behavior is that when a line is first inserted, the created_at column will update to default value (which I understand to be the time at which the current transaction began, not the time of the specific query).
My expected behavior is that when a line is updated, the updated_at line will be updated by this function:
CREATE OR REPLACE FUNCTION public.update_iz_now()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
BEGIN
NEW.updated_at = now();
RETURN NEW;
END;
$function$
but that created_at will remain unchanged, because a value is already in the column, so it should not override with the default.
created_at is initially functioning correctly:
united_states_congress=> select created_at, updated_at from congressional_bill_actions limit 5;
created_at | updated_at
----------------------------+------------
2017-01-28 00:08:11.238773 |
2017-01-28 00:08:11.255533 |
2017-01-28 00:08:15.036168 |
2017-01-28 00:08:15.047991 |
2017-01-28 00:08:15.071715 |
(5 rows)
But then when a line is updated, created_at is being changed to match the insert value of updated_at, leaving me with:
united_states_congress=> select created_at, updated_at from congressional_bill_actions where updated_at is not null limit 5;
created_at | updated_at
----------------------------+----------------------------
2017-01-28 07:55:34.078783 | 2017-01-28 07:55:34.078783
2017-02-01 18:47:50.673996 | 2017-02-01 18:47:50.673996
2017-02-02 14:50:33.066341 | 2017-02-02 14:50:33.066341
2017-02-02 14:50:33.083343 | 2017-02-02 14:50:33.083343
2017-02-03 13:58:34.950716 | 2017-02-03 13:58:34.950716
(5 rows)
I have been all over the internet trying to figure this one out, but the internet keeps helpfully routing me to questions about "how to create default values" and "how to make triggers."
This obviously must be a usage problem somewhere on my end, but I'm having trouble identifying it. Just in case, here is the other trigger being run on the table (on insert):
CREATE OR REPLACE FUNCTION public.hash_this_foo()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
BEGIN
NEW.text_hash = md5(NEW.text)::uuid;
RETURN NEW;
END;
$function$
In the process of writing this question, I found the answer and will post it below. If there are already duplicates of this question, please notify me and I will remove it, I was unable to find any.
The problem here was my UPSERT handling, during which the schema of the table was being pulled in, resulting in the dynamic creation of queries that included lines like this:
ON CONFLICT ON CONSTRAINT congressional_bill_actions_bill_kid_date_action_actor_key DO UPDATE SET created_at = EXCLUDED.created_at,
because created_at was being set automatically to the EXCLUDED.created_at, this was causing the default value to overwrite the existing one precisely because I was instructing it to do so.
So when writing UPSERT handlers, this is something to be aware of, it would seem.
(Note: the way to avoid this is simply not to pull in any columns where column_default is not null.)

insert for each element in jsonb array with 'instead of' trigger on a view

I'm using a view to report related rows as an array. It would be convenient to also insert those rows with a instead of insert trigger on the view. The rows to insert will need to construct a json object with keys from a related table.
My current incomplete implementation needs where task in JSONB_ARRAY. I don't know if that is possible.
in addition to a visit table with primary id colum vid, I have
task table to pull from.
| task | sections |
+--------+-------------------------+
|task_A | ["section1","section2"] |
|task_B | ["part1", "part2" ] |
and a visit_task table to populate
| vid | task | measures |
+-----+--------+------------------------------------------------+
| 1 | task_C | ["level1": "value added later","level2": null] |
| 1 | task_E | ["q1": null,"q2": null] |
want:
insert into vt_view (vid,tasks) values (1,'["task_A","task_B"]::jsonb)
to actually do:
insert into visit_task (vid,task,measures)
values (1,'task_A','{"section1": null, "section2": null}'::jsonb);
insert into visit_task (vid,task,measures)
values (1,'task_B','{"part1": null, "part2": null}'::jsonb);
current incomplete trigger solution excerpt:
insert into visit_task from
select
NEW.vid as vid,
NEW.task as task,
-- *MAGIC* to create json object; measures list becomes object with null vals
row_to_json(_) from (select json_object(t.measures, 'null')) as measures
-- /MAGIC
from task t
-- *MAGIC* where we only grab the tasks within the array NEW.tasks
where t.task in (select * from NEW.tasks)
-- /MAGIC
Unfortunately, the server doesn't know what to do with/in the MAGIC flags. Neither do I.
To make the task easier create an auxiliary function to expand sections to desired format:
create or replace function expand_sections(jsonb)
returns jsonb language sql as $$
select jsonb_object_agg(e, null)
from jsonb_array_elements_text($1) e
$$;
-- test the function:
select task, expand_sections(sections)
from task;
task | expand_sections
--------+--------------------------------------
task_A | {"section1": null, "section2": null}
task_B | {"part1": null, "part2": null}
(2 rows)
Use the function in the trigger function:
create or replace function trigger_instead_on_vt_view()
returns trigger language plpgsql as $$
begin
insert into visit_task
select vid, task, expand_sections(sections)
from (
select new.vid, task
from jsonb_array_elements_text(new.tasks) task
) sub
join task using(task);
return null;
end $$;
create trigger trigger_instead_on_vt_view
instead of insert on vt_view
for each row execute procedure trigger_instead_on_vt_view();
insert into vt_view (vid,tasks) values (1,'["task_A","task_B"]'::jsonb);
select * from visit_task;
vid | task | measures
-----+--------+--------------------------------------
1 | task_A | {"section1": null, "section2": null}
1 | task_B | {"part1": null, "part2": null}
(2 rows)
If you do not want to create the auxiliary function, use this variant:
create or replace function trigger_instead_on_vt_view()
returns trigger language plpgsql as $$
begin
insert into visit_task
select vid, task, measures
from (
select new.vid, task
from jsonb_array_elements_text(new.tasks) task
) s
join (
select task, jsonb_object_agg(e, null) measures
from task, jsonb_array_elements_text(sections) e
group by 1
) t
using (task);
return null;
end $$;