So, i have a couple of functions inside my db.
One function needs to run when the data in a specific table is older than 5 minutes.
I've tried doing it with:
PERFORM case when now() - '5 minutes'::interval > (select end_time from x order by end_route desc limit 1) then update_x() else null end;
When i run the command as a regular select query, it runs OK. But when i put that inside another function (The one being called, returns updated table that is no older than 5 minutes), it never runs. Also, if i just put update_x(), then it runs OK (but every time the function is called).
Does anyone have any idea on how i could fix this?
One idea is to just set a cron to run the function every 5 mins independently, but i'd rather the server is idle since the function is quite resource intensive, and it doesn't get called that often.
I'm on version 8.4(Due to my ISP, so can't change, though i am considering moving to VPS, so if this is something that would work on 9.5 and newer, i can wait).
The function now() gives the start time of the current transaction and is invariable inside it. Use clock_timestamp(), example:
do $$
begin
for i in 1..3 loop
perform pg_sleep(1);
raise notice 'now(): % clock_timestamp(): %', now(), clock_timestamp();
end loop;
end $$;
NOTICE: now(): 2017-12-06 10:22:40.422683+01 clock_timestamp(): 2017-12-06 10:22:41.437099+01
NOTICE: now(): 2017-12-06 10:22:40.422683+01 clock_timestamp(): 2017-12-06 10:22:42.452456+01
NOTICE: now(): 2017-12-06 10:22:40.422683+01 clock_timestamp(): 2017-12-06 10:22:43.468124+01
Per the documentation:
clock_timestamp() returns the actual current time, and therefore its value changes even within a single SQL command (...)
now() is a traditional PostgreSQL equivalent to transaction_timestamp().
I'm not sure why, but once i moved the boolean up one level it started working.
So now instead of having perform case query inside function, i'm sending boolean to the function, and performing the check in a view above this function.
CREATE VIEW x_view AS select * from get_x((clock_timestamp() - '5 minutes'::interval)::timestamp > (select end_route from gps_skole2 order by end_route desc limit 1));
And inside the function:
PERFORM case when $1 then update_x() else null end;
Related
To aid me with debugging another issue in my database, I've written the following function as a trigger in postgresql:
CREATE OR REPLACE FUNCTION stage.triggerlogfunction()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
begin
insert into stage.temptriggerlog_complete
values
(current_timestamp, 'Hello');
perform pg_sleep(5);
insert into stage.temptriggerlog_complete
values
(current_timestamp, 'Did');
perform pg_sleep(5);
insert into stage.temptriggerlog_complete
values
(current_timestamp, 'This Work?');
return null;
END;
$function$
;
When a new row is inserted into a table in my database, this trigger is set to fire. This works as expected, but what I was expecting to see when this ran was three rows with timestamps that are 5 seconds apart (running in sequence with a 5 second delay between each), but the actual result I had returned was three rows all with the same timestamp.
Why is this happening? Is there a way I can force the behaviour that I was expecting?
is there any way I can get the current timestamp at the time each insert statement runs?
You're looking for statement_timestamp() (which would be the timestamp of the statement causing your trigger to fire) or even clock_timestamp() then.
See the documentation for what they do and the alternatives.
I am having problems with the following function. The purpose of this function is to return a set of records that will not be returned again if called within 60 seconds (Almost like a queue).
It seems to work fine when I run this one a time, however when I am using it in my threaded application, I see duplicates that show up. Am I locking the rows correctly? What is the correct way to use FOR UPDATE when inserting into a temp table?
CREATE OR REPLACE FUNCTION needs_quantities(computer TEXT)
RETURNS TABLE(id BIGINT, listing_id CHARACTER VARYING, asin CHARACTER VARYING, retry_count INT)
LANGUAGE plpgsql
AS $$
BEGIN
CREATE TEMP TABLE temp_needs_quantity ON COMMIT DROP
AS
SELECT
listing.id,
listing.listing_id,
listing.asin,
listing.retry_count
FROM listing
WHERE listing.id IN (
SELECT min(listing.id) AS id
FROM listing
WHERE (listing.quantity_assigned_to IS NULL
--quantity is null
-- and quantity assigned date is at least 60 seconds ago
-- and quantity date is within 2 hours
OR (
quantity IS NULL AND listing.quantity_assigned_date < now_utc() - INTERVAL '60 second'
AND (listing.quantity_date IS NULL OR listing.quantity_date > now_utc() - INTERVAL '2 hour')
)
)
AND listing.retry_count < 10
GROUP BY listing.asin
ORDER BY min(listing.retry_count), min(listing_date)
LIMIT 10
)
FOR UPDATE;
UPDATE listing
SET quantity_assigned_date = now_utc(), quantity_assigned_to = computer
WHERE listing.id IN (SELECT temp_needs_quantity.id
FROM temp_needs_quantity);
RETURN QUERY
SELECT *
FROM temp_needs_quantity
ORDER BY id;
END
$$
Your function should lock the rows in listing like you intended in the first thread that comes along.
The problem in the second thread is that this subselect:
...
WHERE listing.id IN (
SELECT min(listing.id) AS id
FROM listing
...
LIMIT 10
)
is not blocked by the lock on the rows, even though the enclosing SELECT ... FOR UPDATE is.
So the subselect will happily see the old row versions from before the UPDATE of the first thread, then blocks in the enclosing SELECT ... FOR UPDATE until the first thread is done. Then it proceeds to update the same rows again.
I am not sure if that is to be considered a bug or not – you may want to ask at the pgsql-general mailing list.
There has been a similar problem with CTEs recently, see this commit message fixing this bug. One could argue that this is a similar case.
Unfortunately with a complicated query like yours I can't think of a better solution than to
LOCK TABLE listing IN EXCLUSIVE MODE;
before you begin processing, which is not very satisfactory.
I'd like to compare two functions in pgAdmin's SQL editor. Here's the script. However, when I run it, seems the start_time and end_time have the same value no matter how many iterations. (But, the Query returned successfully with no result in nnn ms. message does go higher with every increase in the loop size.) Why?
DO
$$
DECLARE
start_time timestamp;
end_time timestamp;
diff interval;
BEGIN
SELECT now() INTO start_time;
FOR i IN 1..1000 LOOP
--PERFORM uuid_generate_v1mc();
PERFORM id_generator();
END LOOP;
SELECT now() INTO end_time;
SELECT end_time - start_time INTO diff;
RAISE NOTICE '%', start_time;
RAISE NOTICE '%', end_time;
RAISE NOTICE '%', diff;
END
$$
From the manual
Since these functions [CURRENT_TIMESTAMP, CURRENT_TIME, CURRENT_DATE] return the start time of the current transaction, their values do not change during the transaction.
And then further down:
transaction_timestamp() is equivalent to CURRENT_TIMESTAMP, but is named to clearly reflect what it returns. statement_timestamp() returns the start time of the current statement (more specifically, the time of receipt of the latest command message from the client). statement_timestamp() and transaction_timestamp() return the same value during the first command of a transaction, but might differ during subsequent commands. clock_timestamp() returns the actual current time, and therefore its value changes even within a single SQL command
So you probably want to use clock_timestamp()
I am writing a function in PostgreSQL and wondering if I can do following:
I have an insert statement in every if loop. Can I pass values like this for formatdate1 and formatdate2?
I am also updating a table. Is it how we do it PostgreSQL?
CREATE OR REPLACE FUNCTION Check returns void AS $$
DECLARE
startDate=date;
formateDate1=date; formatdate2=date;newDate=date;
BEGIN
startDate:= SELECT to_date(lastdate::date, 'MM-DD-YYYY') FROM setup;
for i in 1..3 LOOP
IF i = 1 THEN
formateDate1 := select (startDate - INTERVAL '11 months');
formatdate2:= to_date(formatdate2::date,'YYYYMM');
insert into warehouse.memcnts1 (select distinct source,
formatdate2
as yearmo, to_date(formateDate1, 'MM-DD-YYYY')
where effdt <= formateDate1 and enddt >= formateDate1);
ELSIF i = 2 THEN -- this is todays date
--insert query here
insert into warehouse.memcnts1 (select distinct source,formatdate2 as yearmo, to_date(formateDate1, 'MM-DD-YYYY') where effdt <= formateDate1
and enddt >= formateDate1);
ELSIF i = 3 THEN
formateDate1 := select (startDate + INTERVAL '1 months');
newDate=formateDate1;
update dwset SET lastdate := newDate; -- wonder if this is right?
formatdate2:=startDate;
END IF;
END LOOP;
END
$$ language 'plpgsql';
In general any sort of insert, insert, update loop is going to run into problems if the loop gets large enough. I assume you want to do something like
FOR 1 .. n
LOOP
INSERT INTO foo VALUES (...);
INSERT INTO foo VALUES (...);
UPDATE BAR set ....;
END LOOP;
I make that assumption because there is no reason to do a loop and then run different logic for each one.
The problem with the above is that as n gets large (let's say, over a hundred or so) you can start to run into cache management issues which can suddenly start to cause a lot of random disk I/O activity and very long queries. There is nothing like wondering why a given function call runs through 50 iterations of a loop in, say, 100ms, but runs through 1000 iterations of a loop in 1800000 ms. (been there, done that).
In general you want to try to run your logic avoiding loops and instead perform set operations instead.
I hope this helps. It is really hard to see what you are asking.
I've got a function in a postgres database that does a lot of analysis; it consists of a succession of update and insert statements and eventually throws back some output. I'd like to figure out which statements execute slowly, without looking through the log files. (I'm much more comfortable with SQL than I am with, say, perl, to write date / time arithmetic queries in order to spot problems.)
I have a table, activity_log:
CREATE TABLE activity_log
(
action character varying(250),
action_date date,
action_tune time without time zone
);
then throughout my function, after each INSERT / UPDATE I write statements like
INSERT INTO activity_log (action_date, action_tune, action)
VALUES (current_date, current_timestamp, 'INSERT to base_model');
So the function looks something like this:
CREATE FUNCTION rebucket(pos_control character varying, absolute_max_cpc numeric, absolute_max_bucket character varying)
RETURNS integer AS
$BODY$
DECLARE qty INT;
BEGIN
INSERT INTO activity_log (action_date, action_tune, action)
VALUES (current_date, current_timestamp, 'Off we go');
-- Do something that takes 5 minutes
INSERT INTO activity_log (action_date, action_tune, action)
VALUES (current_date, current_timestamp, 'INSERT to base_model');
-- Then do something else that also takes about 5 minutes ...
INSERT INTO activity_log (action_date, action_tune, action)
VALUES (current_date, current_timestamp, 'INSERT to diagnostics');
END
$BODY$
LANGUAGE plpgsql VOLATILE
I've got away with this in other databases in the past, but when I try this approach in Postgres (9.1 on Windows 7), then whenever I run the whole function the date and time in activity_log is exactly the same for every statement within the function: in the example above,
SELECT * FROM activity_log
gets me
Off we go 2013-05-13 12:33:23:386
INSERT to base_model 2013-05-13 12:33:23:386
INSERT to diagnostics 2013-05-13 12:33:23:386
(The function takes from 5 minutes to an hour to run, depending on what parameters we feed it, and it has upwards of 20 different statements within there, so it seems highly unlikely that every statement completed within the same 1/100th of a second.)
Why is that?
The timestamp you are using always gives the start of the current transaction. If you look in the manuals you will see that you want clock_timestamp().