I have a function that used dblink_exec inside. Sometimes this execution takes a long time and in some cases produces deadlocks.
I know that I can set local lock_timeout and local statement_timeout but when I do this inside a function, It doesn't take any effect over dblink_exec.
Is that any way to set those parameters on dblink_connect? I know that I can set these two parameters on .conf file but I want to do this locally (per connection).
Is this possible?
Thanks!
Julio
It is quite ugly, but you can specify the statement timeout (to be implemented on the remote side) in the connection string, using the options construct.
create or replace function dblink_timeout() returns int language plpgsql as $$
declare xx int;
begin
select x into xx from
dblink(
'dbname=jjanes options = ''-c statement_timeout=500 -c lock_timeout=100''',
'select count(*) from pgbench_accounts'
) as (x int);
return xx;
end
$$;
Related
How can I identify slow queries in a Postgres function?
For example:
CREATE OR REPLACE FUNCTION my_function ()
RETURNS void AS $$
BEGIN
query#1;
query#2; --> slow query (durration 4 sec)
query#3;
query#4;
END
$$ LANGUAGE plpgsql;
After executing my_function() I get something like this in my Postgres log file:
duration: 4.904 ms statement: select my_function ();",,,,,,,,,"psql"
So I can't identify slow queries in my function.
By default, PL/pgSQL functions are black boxes to the Postgres query planner and logging.
The additional module auto_explain allows for more insights. It can be loaded dynamically, but you have to be a superuser. (Does not have to be installed like most other modules.)
To load it an individual session:
LOAD 'auto_explain';
-- SET auto_explain.log_min_duration = 1; -- exclude very fast trivial queries?
SET auto_explain.log_nested_statements = ON; -- statements inside functions
-- SET auto_explain.log_analyze = ON; -- get actual times, too?
Any query running in the same session will get extra logging. Just:
SELECT my_function(); -- your function
See:
Postgres query plan of a function invocation written in plpgsql
This covers most use cases How do you use variables in a simple PostgreSQL script? but not the select clause.
This code produces an error column "ct" does not exist"
DO
$$
declare CT timestamp := '2020-09-04 23:59:59';
select CT,5 from job;
$$;
I can see why it would interpret CT as a column name. What's the Postgres syntax required to refer to a variable in the context of the select clause?
I would expect that query to return
'2020-09-04 23:59:59',5
for each row in the job table.
Addendum to the accepted answer
My use case doesn't return rows. Instead, the result of the select is consumed by an insert statement. I'm transforming rows from staging tables into other tables and adding value like the import date and the identity owning the inserts. It's these values that are provided by the variables - they are used in several such transforms and the point of the variable is to let me set each value once up the top of the script.
Because the rows are consumed like this, it turns out that I don't need a function wrapping this code. It's a bit inconvenient to test since I can't run the select and look at the outcome without copying it and pasting in literals, but at least it's possible to use variables. My working script looks like this:
do
$$
declare ct timestamp := '2020-09-04 23:59:59';
declare cb int := 2;
declare iso8601 varchar(50) := 'YYYY-MM-DD HH24:MI:SS';
declare USAdate varchar(50) := 'MM-DD-YYYY HH24:MI:SS';
begin
delete from dozer_wheel_loader_equipment_movement where created = ct;
INSERT INTO dozer_wheel_loader_equipment_movement
(site, primary_category_id, machine, machine_class, x, y, z, timestamp_local, created, created_by)
select site ,mc.id ,machine , machineclass ,x,y,z,to_timestamp(timestamplocal, iso8601), ct, cb
from stage_dozer_csv d join machine_category mc on d.primarycategory = mc.short_code;
...
end
$$
There is a lot of worthwhile related reading at How to declare a variable in a PostgreSQL query
There are few things about variables in PostgreSQL.
Variable can not be used in Plain SQL in Postgres. So you have to use any pl language i.e. plpgsql to use this. You have tried the same in your example.
In your DO block you have missed the Begin and End, So you have to write it like below
DO
$$
begin
declare CT timestamp := '2020-09-04 23:59:59';
select CT,5 from job;
end
$$;
But when you read the official documentation of DO Statement, it says DO will allow to run the anonymous code but it returns void, that's why above code will throw following error:
ERROR: query has no destination for result data
HINT: If you want to discard the results of a SELECT, use PERFORM instead.
CONTEXT: PL/pgSQL function inline_code_block line 4 at SQL statement
So there is only one way - wrap this code block in a Function like below:
create or replace function func() returns table(col1 timestamp, col2 int )
AS
$$
declare ct timestamp := '2020-09-04 23:59:59';
begin
return query
select CT,5 from job;
end;
$$
language plpgsql
and you can call it like below:
select * from func()
DEMO
Conclusion
You can not use variable in normal SQL statement in Postgres.
You have to use any Procedural Language i.e. plpgsql to use variable.
DO Block doesn't return any value so you can not use select statement like above in DO block. It is good for non-returning queries i.e. insert, update, delete or grant etc.
Only way to return a value from procedural language code block is - you have to wrap it in a suitable PostgreSQL Function.
I'm trying to make a trigger function to create a time stamp based on a base date stored in a variable plus an interval in seconds.
This base date is given to the psql script with the -v option, e.g. "-v start_time='2013-10-10 13:48:00'".
I want to access this variable from within a trigger function a do something like:
NEW.mytimestamp = timestamp :start_time + interval NEW.elapsed_seconds ' s';
Unfortunately I cannot figure out the right syntax for that. Any ideas?
It is impossible. psql variables (accessed via :varname) are client side variables. Trigger functions are executed on the server and cannot access these variables.
There is a way around this, but a little difficult (one cannot simple initialize values via command line). You can use custom configuration setting variables:
postgres=# select set_config('public.xxx', '10', false);
set_config
------------
10
(1 row)
create or replace function foo_trg()
returns trigger as $$
begin
raise notice '%', current_setting('public.xxx');
return new;
end;
$$ language plpgsql;
create table foo(a int);
create trigger hh before insert on foo for each row execute procedure foo_trg();
postgres=# insert into foo values(200);
NOTICE: 10
INSERT 0 1
Another (more established) technique would be to use an auxiliary table.
On second thought, trigger parametrization (based on some global value) is usually a terrible idea. It indicates you are doing some wrong. Use a function instead.
Let's say I have a function show_files(IN file text, IN suffix text, OUT statement text). In next step the function is called:
SELECT * FROM show_files(file := 'example', suffix := '.png');
My question is: Is there any solution that I could get statement that has called this function from inside that function?
I mean, after running the SELECT the output of function (OUT statement text) should be: 'SELECT * FROM show_files(file := 'example', suffix := '.png');', or is it possible to assign this statement to the variable inside the function?
I need the functionality like those with TG_NAME, TG_OP, etc. in trigger procedures.
Maybe is it possible to retrieve this statement from SELECT current_query FROM pg_stat_activity ?
When I'm trying to use it inside a function I've got an empty record:
CREATE OR REPLACE FUNCTION f_snitch(text)
RETURNS text AS
$BODY$
declare
rr text;
BEGIN
RAISE NOTICE '.. from f_snitch.';
-- do stuff
SELECT current_query into rr FROM pg_stat_activity
WHERE current_query ilike 'f_snitch';
RETURN rr;
END
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
Any help and suggestions would be happily welcome!
TG_NAME and friends are special variables that only exist for trigger functions. Regular plpgsql functions don't have anything like that. I am fresh out of ideas how you could possibly get this inside the called function in plpgsql.
You could add RAISE NOTICE to your function so you get the desired information
CREATE OR REPLACE FUNCTION f_snitch(text)
RETURNS text LANGUAGE plpgsql AS
$func$
BEGIN
RAISE NOTICE '.. from f_snitch.';
-- do stuff
RETURN 'Snitch says hi!';
END
$func$;
Call:
SELECT f_snitch('foo')
In addition to the result, this returns a notice:
NOTICE: .. from f_snitch.
Fails to please in two respects:
Calling statement is not in the notice.
No CONTEXT in the notice.
For 1. you can use RAISE LOG instead (or set your cluster up to log NOTICES, too - which I usually don't, too verbose for me). With standard settings, you get an additional line with the STATEMENT in the database log:
LOG: .. from f_snitch.
STATEMENT: SELECT f_snitch('foo')
For 2., have a look at this related question at dba.SE. CONTEXT would look like:
CONTEXT: SQL statement "SELECT f_raise('LOG', 'My message')"
PL/pgSQL function "f_snitch" line 5 at PERFORM
Ok, I've got it!
CREATE OR REPLACE FUNCTION f_snitch(text)
RETURNS setof record AS
$BODY$
BEGIN
RETURN QUERY
SELECT current_query
FROM pg_stat_activity
<strike>ORDER BY length(current_query) DESC LIMIT 1;</strike>
where current_query ilike 'select * from f_snitch%';
-- much more reliable solution
END
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
select * from f_snitch('koper') AS (tt text);
And here is the result:
It's probably not 100% reliable solution but for small systems (for few users) it's quite ok.
In Postgresql, I have an UPDATE rule on a table which only needs to call a dctUpdate function without doing a whole SQL statement, since the SQL statement is actually done in the function. The only way I know of calling the function is through SELECT dctUpdate(windowId):
create or replace function infoUpdate(windowId in numeric) returns void as $$
begin
if windowId is null then
update info_timestamp set timestamp = now();
else
update info_timestamp set timestamp = now() where window_id = windowId;
end if;
end;
$$ LANGUAGE plpgsql;
create or replace rule info_update_rule as on update to some_table do also select infoUpdate(NEW.window_id);
However, on the command line, when that rule gets triggered because I updated a row in some_table, I get useless output from the SELECT clause that calls the function :
db=# update some_table set name = 'foobar' where window_id = 1;
infoupdate
-----------
(1 row)
UPDATE 1
Is there a way to have info_update_rule call the infoUpdate function without it displaying dummy output?
I've found no options to implement this using rules, but there is an alternative way of implementing this usign triggers.
So, you define trigger function as following:
CREATE OR REPLACE FUNCTION ur_wrapper_trg()
RETURNS trigger AS
$BODY$
begin
perform infoUpdate(NEW.window_id);
RETURN NEW;
end;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION ur_wrapper_trg() OWNER TO postgres;
Note PERFORM syntax is used. This syntax is identical to SELECT syntax except it supresses all output.
Than you define a trigger
CREATE TRIGGER some_table_utrg
BEFORE UPDATE
ON some_table
FOR EACH ROW
EXECUTE PROCEDURE ur_wrapper_trg();
In the end, you remve your rule.
Haven't tested with null, but with actual windos_ids works as expected, without any unwanted output.
Consult with Triggers and Rules vs triggers for detailed description.
The closes solution to which I came is to call \t \a before select function() and right after it. The only remaining thing is a new line for each call.