Wrote and tested a pretty simple script in SQL. Now I need to get it working in Postgres (which I'm just learning). Can't figure out the latest error. The script is reading a file to be a variable. Perhaps I'm not using DBeaver correctly (which I'm also trying to learn). Basically, when data is over 90 days old, move date from transactions to archive_transactions and delete change_log records. The error and code is below:
SQL Error [42601]: ERROR: "archived_transactions_prerun" is not a known variable
Position: 325
CODE:
CREATE OR REPLACE FUNCTION Archiving ()
RETURNS void AS $$
declare
BEGIN
DROP TABLE IF EXISTS Archived_Transactions_PreRun;
DROP TABLE IF EXISTS Change_Log_PreRun;
DROP TABLE IF EXISTS Transactions_PreRun;
--COMMIT;
SELECT *
INTO Archived_Transactions_PreRun
FROM Archived_Transactions;
SELECT *
INTO Change_Log_PreRun
FROM Change_Log;
SELECT *
INTO Transactions_PreRun
FROM Transactions;
COMMIT;
-- Create reporting table entries
----------------------------------
DECLARE YYYY_MM_DD DATE = (SELECT CONVERT (DATE, GETDATE())) -- Run Date
, Report_Date DATE = (SELECT DATEADD (DAY, -90, GETDATE())) -- 90 Days ago
, To_Archive FLOAT
, Chg_Log FLOAT;
-- Count records to be archived
-------------------------------
SET Chg_Log = (SELECT COUNT(*) FROM Change_Log WHERE date_updated < Report_Date);
SET To_Archive = (SELECT COUNT(*) FROM transactions WHERE date < Report_Date);
-- If nothing to archive, exit
------------------------------
IF Chg_Log > 0
OR To_Archive > 0;
BEGIN
-- Remove 90+ records from change_log
-------------------------------------
DELETE
FROM change_log
WHERE date_updated < Report_Date;
-- Copy 90+ records to Archived_Transactions
--------------------------------------------
INSERT INTO Archived_Transactions
SELECT *
FROM Transactions
WHERE [date] < Report_Date;
-- Remove 90+ records from transactions
---------------------------------------
DELETE
FROM Transactions
WHERE date < Report_Date;
COMMIT;
END;
END;
$$
LANGUAGE 'plpgsql' VOLATILE
COST 100
Related
I have a database of orders and each order has a time_slot (of type TIME).
select id, time_slot from orders limit 5;
10 | 13:00:00
11 | 12:00:00
13 | 11:00:00
14 | 12:30:00
15 | 11:30:00
I want to make sure that only a certain number of orders can be placed in a time slot; for example, let's say I only want 8 orders per time slot.
I am using Supabase, so I would like to implement using RLS policies.
I've been through several iterations, but none of them have worked. Some complain of infinite recursion. My current approach is the following: I have created a view of time slot load.
create or replace view time_slot_load as
select time_slot, count(*)
from orders
group by time_slot;
select * from time_slot_load limit 5;
11:00:00 | 1
12:30:00 | 1
11:30:00 | 1
13:00:00 | 1
12:00:00 | 7
I can then create a policy that checks against this view.
ALTER POLICY "Only 8 orders per time slot"
ON public.orders
WITH CHECK (
(SELECT (load.count <= 8)
FROM time_slot_load load
WHERE (load.time_slot = orders.time_slot))
);
But this is not working.
Is there some way I can do this using constraints or RLS policies?
Any help is appreciated.
demo
table structure:
begin;
create table orders_count(
time_slot tstzrange ,
order_count integer default 0,
constraint order_ct_max1000 check(order_count <=4));
create table
orders(orderid int primary key, realtimestamp timestamptz , order_info text);
with a as(
SELECT x as begin_,x + interval '1 hour' as end_
FROM generate_series(
timestamp '2022-01-01',
timestamp '2022-01-02',
interval '60 min') t(x))
insert into orders_count(time_slot)
select tstzrange(a.begin_, a.end_,'[]') from a;
commit;
two table, one table for all the orders info, another table is track the timeslot's count, also make one constraint, within one timeslot no more than 4 orderid.
Then create trigger for delete/update/insert action on table orders.
for table orders_count, 20 years only 175200 rows (one hour one row).
main trigger function:
begin;
create or replace function f_trigger_orders_in()
returns trigger as
$$
begin
update orders_count set order_count = order_count + 1
where time_slot #> NEW.realtimestamp;
return null;
end;
$$ language plpgsql;
create or replace function f_trigger_orders_up()
returns trigger as
$$
begin
if OLD.realtimestamp <> NEW.realtimestamp THEN
update orders_count
set order_count = order_count -1
where time_slot #> OLD.realtimestamp;
update orders_count
set order_count = order_count + 1
where time_slot #> NEW.realtimestamp;
end if;
return null;
end
$$ language plpgsql;
create or replace function f_trigger_order_delaft()
returns trigger as
$$
BEGIN
update orders_count set order_count = order_count -1 where time_slot #> OLD.realtimestamp;
return null;
end;
$$
language plpgsql;
triggers action:
create trigger trigger_in_orders
AFTER INSERT ON public.orders FOR EACH ROW execute procedure f_trigger_orders_in();
create trigger trigger_up_orders
after update on public.orders for each row execute procedure f_trigger_orders_up();
create trigger trigger_del_orders
AFTER DELETE ON public.orders FOR EACH ROW execute procedure f_trigger_order_delaft();
I want to make sure that only a certain number of orders can be placed
in a time slot; for example, let's say I only want 8 orders per time
slot.
You cannot do that in PostgreSQL.
https://www.postgresql.org/docs/current/sql-createpolicy.html
check_expression:
Any SQL conditional expression (returning boolean). The conditional
expression cannot contain any aggregate or window functions. This
expression will be used in INSERT and UPDATE queries against the table
if row-level security is enabled. Only rows for which the expression
evaluates to true will be allowed. An error will be thrown if the
expression evaluates to false or null for any of the records inserted
or any of the records that result from the update. Note that the
check_expression is evaluated against the proposed new contents of the
row, not the original contents.
Why view won't work:
https://www.postgresql.org/docs/current/sql-createview.html
see Updatable Views section:
A view is automatically updatable if it satisfies all of the following
conditions:
The view's select list must not contain any aggregates, window functions or set-returning functions.
I have a huge table with 25 million records, and a system that has a scheduled task that can execute a query. The query needs to quickly pick up latest records by create date(timestamp) column and apply some calculations. The problem with this is that the date is also kept in a table and with each execution it is updated to the latest execution date. It does work but it is very slow:
select * from request_history
where createdate > (select startdate from request_history_config)
limit 10;
it takes about 20 seconds to complete, which is rediculously slow compared to this:
set custom.startDate = '2019-06-13T18:02:04';
select * from request_history
where createdate > current_setting('custom.startDate')::timestamp
limit 10;
and this query finishes well within 100 miliseconds. The problem with this is that I can't update and save the date for the next execution! I was looking for SET variable TO statement that would allow me to grab some value from a table but all these attempts are not working:
select set_config('custom.startDate', startDate, false) from request_history_config;
// ERROR: function set_config(unknown, timestamp without time zone, boolean) does not exist
set custom.startDate to (select startDate from request_history_config);
// ERROR: syntax error at or near "("
You can do this using function like this:
CREATE OR REPLACE FUNCTION get_request_history()
RETURNS TABLE(createdate timestamp)
LANGUAGE plpgsql
AS $function$
DECLARE
start_date timestamp;
BEGIN
SELECT startdate INTO start_date FROM request_history_config;
RETURN QUERY
SELECT *
FROM request_history h
WHERE h.createdate > start_date
LIMIT 10;
END
$function$
And then use the function to get values:
select * from get_request_history()
I receive this error to begin with:
ERROR: syntax error at or near "conference"
LINE 19: FOR conference IN conferenceset
Here's the function:
CREATE OR REPLACE FUNCTION due_payments_to_suppliers_previous_month()
RETURNS TABLE(supplier varchar,due_amount numeric)
AS $$
DECLARE
BEGIN
CREATE TABLE conferenceset AS -- temporary table, so I can store the result set
SELECT
conference.conference_supplier_id,
conference.id AS conferenceid,
conference.price_per_person,
0 AS participants_count,
400 AS deduction_per_participant,
0 AS total_amount
FROM Conference WHERE --- date_start has to be from the month before
date_start >= date_trunc('month', current_date - interval '1' month)
AND
date_start < date_trunc('month', current_date);
FOR conference IN conferenceset
LOOP
---fill up the count_participants column for the conference
conference.participants_count :=
SELECT COUNT(*)
FROM participant_conference JOIN conferenceset
ON participant_conference.conference_id = conferenceset.conferenceid;
---calculate the total amount for that conference
conference.total_amount := somerec.participants_count*(conference.price_per_person-conference.deduction_per_participant);
END LOOP;
----we still don't have the name of the suppliers of these conferences
CREATE TABLE finalresultset AS -- temporary table again
SELECT conference_supplier.name, conferenceset.total_amount
FROM conferenceset JOIN conference_supplier
ON conferenceset.conference_supplier_id = conference_supplier.id
----we have conference records with their amounts and suppliers' names scattered all over this set
----return the result with the suppliers' names extracted and their total amounts calculated
FOR finalrecord IN (SELECT name,SUM(total_amount) AS amount FROM finalresultset GROUP BY name)
LOOP
supplier:=finalrecord.name;
due_amount:=finalrecord.amount;
RETURN NEXT;
END LOOP;
END; $$
LANGUAGE 'plpgsql';
I don't know how and where to declare the variables that I need for the two FOR loops that I have: conference as type conferenceset and finalrecord whose type I'm not even sure of.
I guess nested blocks will be needed as well. It's my first stored procedure and I need help.
Thank you.
CREATE OR REPLACE FUNCTION due_payments_to_suppliers_previous_month()
RETURNS TABLE(supplier varchar,due_amount numeric)
AS $$
DECLARE
conference record;
finalrecord record;
BEGIN
CREATE TABLE conferenceset AS -- temporary table, so I can store the result set
SELECT
conference.conference_supplier_id,
conference.id AS conferenceid,
conference.price_per_person,
0 AS participants_count,
400 AS deduction_per_participant,
0 AS total_amount
FROM Conference WHERE --- date_start has to be from the month before
date_start >= date_trunc('month', current_date - interval '1' month)
AND
date_start < date_trunc('month', current_date);
FOR conference IN (select * from conferenceset)
LOOP
---fill up the count_participants column for the conference
conference.participants_count = (
SELECT COUNT(*)
FROM participant_conference JOIN conferenceset
ON participant_conference.conference_id = conferenceset.conferenceid
);
---calculate the total amount for that conference
conference.total_amount = somerec.participants_count*(conference.price_per_person-conference.deduction_per_participant);
END LOOP;
----we still don't have the name of the suppliers of these conferences
CREATE TABLE finalresultset AS -- temporary table again
SELECT conference_supplier.name, conferenceset.total_amount
FROM conferenceset JOIN conference_supplier
ON conferenceset.conference_supplier_id = conference_supplier.id
----we have conference records with their amounts and suppliers' names scattered all over this set
----return the result with the suppliers' names extracted and their total amounts calculated
FOR finalrecord IN (SELECT name,SUM(total_amount) AS amount FROM finalresultset GROUP BY name)
LOOP
supplier = finalrecord.name;
due_amount = finalrecord.amount;
RETURN NEXT;
END LOOP;
END; $$
LANGUAGE 'plpgsql';
Questions about transpose are asked many times before, but I cannot find any good answer when using generate_series and dates, because the columns may vary.
WITH range AS
(SELECT to_char(generate_series('2015-01-01','2015-01-05', interval '1 day'),'YYYY-MM-DD'))
SELECT * FROM range;
The normal output from generate series is:
2015-12-01
2015-12-02
2015-12-03
... and so on
http://sqlfiddle.com/#!15/9eecb7db59d16c80417c72d1e1f4fbf1/5478
But I want it to be columns instead
2015-12-01 2015-12-02 2015-12-03 ...and so on
It seems that crosstab maybe should do the trick, but I only get errors:
select * from crosstab('(SELECT to_char(generate_series('2015-01-01','2015-01-05', interval '1 day'),'YYYY-MM-DD'))')
as ct (dynamic columns?)
How do I get crosstab to work with generate_series(date-date) and different intervals dynamically?
TIA
Taking Reference from link PostgreSQL query with generated columns.
you can generate columns dynamically:
create or replace function sp_test()
returns void as
$$
declare cases character varying;
declare sql_statement text;
begin
drop table if exists temp_series;
create temporary table temp_series as
SELECT to_char(generate_series('2015-01-01','2015-01-02', interval '1 day'),'YYYY-MM-DD') as series;
select string_agg(concat('max(case when t1.series=','''',series,'''',' then t1.series else ''0000-00-00'' end) as ','"', series,'"'),',') into cases from temp_series;
drop table if exists temp_data;
sql_statement=concat('create temporary table temp_data as select ',cases ,'
from temp_series t1');
raise notice '%',sql_statement;
execute sql_statement;
end;
$$
language 'plpgsql';
Call function in following way to get output:
select sp_test(); select * from temp_data;
Updated Function which takes two date paramaeters:
create or replace function sp_test(start_date timestamp without time zone,end_date timestamp without time zone)
returns void as
$$
declare cases character varying;
declare sql_statement text;
begin
drop table if exists temp_series;
create temporary table temp_series as
SELECT to_char(generate_series(start_date,end_date, interval '1 day'),'YYYY-MM-DD') as series;
select string_agg(concat('max(case when t1.series=','''',series,'''',' then t1.series else ''0000-00-00'' end) as ','"', series,'"'),',') into cases from temp_series;
drop table if exists temp_data;
sql_statement=concat('create temporary table temp_data as select ',cases ,'
from temp_series t1');
raise notice '%',sql_statement;
execute sql_statement;
end;
$$
language 'plpgsql';
Function call:
select sp_test('2015-01-01','2015-01-10'); select * from temp_data;
I'm trying to retrieve player statistics for the last 20 weeks:
# select yw, money
from pref_money where id='OK122471020773'
order by yw desc limit 20;
yw | money
---------+-------
2010-52 | 1130
2010-51 | 3848
2010-50 | 4238
2010-49 | 2494
2010-48 | 936
2010-47 | 3453
2010-46 | 3923
2010-45 | 1110
2010-44 | 185
(9 rows)
But I would like to have the result as a string, where all values are concatenated by colons and semicolons like this:
"2010-44:185;2010-45:1110; .... ;2010-52:1130"
So I'm trying to create the following PL/pgSQL procedure:
create or replace function pref_money_stats(_id varchar)
returns varchar as $BODY$
begin
declare stats varchar;
for row in select yw, money from pref_money
where id=_id order by yw desc limit 20 loop
stats := row.id || ':' || row.money || ';' stats;
end loop;
return stats;
end;
$BODY$ language plpgsql;
But I get the syntax error:
ERROR: syntax error at or near "for"
LINE 7: for row in select yw, money from pref_money where id...
Using PostgreSQL 8.4.6 with CentOS 5.5 Linux.
UPDATE:
I'm trying to perform all this string concatenation with PL/pgSQL and not in PHP script, because I already have a main SQL select statement, which returns user information and that information is printed row by row as XML for my mobile app:
select u.id,
u.first_name,
u.female,
u.city,
u.avatar,
m.money,
u.login > u.logout as online
from pref_users u, pref_money m where
m.yw=to_char(current_timestamp, 'YYYY-IW')
and u.id=m.id
order by m.money desc
limit 20 offset ?
Here is the screenshot of the mobile app:
And here is an XML excerpt:
<?xml version="1.0"?>
<pref>
<user id="OK510352632290" name="ирина" money="2067" pos="1" medals="1" female="1" avatar="http://i221.odnoklassniki.ru/getImage?photoId=259607761026&photoType=0" city="староконстантинов" />
<user id="OK19895063121" name="Александр" money="1912" pos="2" online="1" avatar="http://i69.odnoklassniki.ru/getImage?photoId=244173589553&photoType=0" city="Сызрань" />
<user id="OK501875102516" name="Исмаил" money="1608" pos="3" online="1" avatar="http://i102.odnoklassniki.ru/res/stub_128x96.gif" city="Москва" />
.....
</pref>
But my problem is that I have 3 other tables, from which I need that statistics for the last 20 weeks. So I'm hoping to create 3 procedures returning varchars as in my original post and integrate them in this SQL select statement. So that I can add further attributes to the XML data:
<user id="OK12345" .... money_stats="2010-44:185;2010-45:1110; .... ;2010-52:1130" ..... />
Thank you!
Alex
Aggregate functions are good for concatenating values:
create or replace function test
(text, text, text)
returns text as
$$
select $1 || ':' || $2 || ';' || $3
$$
language sql;
drop function test(text, text);
drop aggregate test(text, text) cascade;
create aggregate test(text, text)
(
sfunc = test,
stype = text,
initcond = ''
);
test=# select test(a::text, b::text) from (select generate_series(1,3) as a, generate_series(4,5) a
s b) t;
:1;4:2;5:3;4:1;5:2;4:3;5
(I'll leave it to you to deal with the leading colin :-)
You probably have already found the answer to your problem. Even so, the problem was indeed syntax.
The problem was that the declare statement was misplaced: it should appear before the begin (docs):
create or replace function pref_money_stats(_id varchar)
returns varchar as $BODY$
declare stats varchar;
begin
...
Another detail to take notice of is that you need to declare row as a record:
declare
stats varchar;
row record;
Then this statement will run properly:
for row in select yw, money from pref_money where id=_id order by yw desc limit 20 loop
This is not exactly JSON but pretty close:
SELECT ARRAY
(
SELECT ROW(yw, money)
FROM pref_money
WHERE id = 'OK122471020773'
ORDER BY
yw DESC
LIMIT 20
)::TEXT
This will output this string:
{"(2010-44:185)","(2010-45:1110)",…,"(2010-52:1130)"}
which can later be cast back into the appropriate types.