I'm writing a function that wraps a fairly expensive query in a PostgreSQL function:
CREATE OR REPLACE FUNCTION
myschema.expensive_fn(search_term text)
RETURNS TABLE(
objectid integer,
geom geometry
)
AS
$$
BEGIN
RETURN QUERY
SELECT [... slow query here...];
END;
$$
LANGUAGE 'plpgsql' STABLE PARALLEL SAFE;
I tried invoke this function with a LIMIT to make it more responsive in the UI:
SELECT myschema.expensive_fn('user input') LIMIT 100;
But this still runs pretty slowly, which I interpret as being because the expensive query inside the function is being run to completion without a limit. So what I've ended up doing is:
CREATE OR REPLACE FUNCTION
myschema.expensive_fn(
search_term text,
lmt integer DEFAULT 100
)
RETURNS TABLE(
objectid integer,
geom geometry
)
AS
$$
BEGIN
RETURN QUERY
SELECT [... slow query here...]
LIMIT lmt;
END;
$$
LANGUAGE 'plpgsql' STABLE PARALLEL SAFE;
And the query becomes
SELECT myschema.expensive_fn('user input', 200);
This works, but it feels inelegant, especially as some services (e.g. pg_featureserv) will set the query limit anyway, which makes the query look really ugly:
SELECT myschema.expensive_fn('user input', 200) limit 200;
So I just wanted to ask whether there's a better way? Is there, for example, some way for a PostgreSQL function to automagically determine whether it is being invoked in a query with a given LIMIT?
Related
As I am newbie to plpgSQL,
I stuck while migrating a Oracle query into PostgreSQL.
Oracle query:
create or replace FUNCTION employee_all_case(
p_ugr_id IN integer,
p_case_type_id IN integer
)
RETURN number_tab_t PIPELINED
-- LANGUAGE 'plpgsql'
-- COST 100
-- VOLATILE
-- AS $$
-- DECLARE
is
l_user_id NUMBER;
l_account_id NUMBER;
BEGIN
l_user_id := p_ugr_id;
l_account_id := p_case_type_id;
FOR cases IN
(SELECT ccase.case_id, ccase.employee_id
FROM ct_case ccase
INNER JOIN ct_case_type ctype
ON (ccase.case_type_id=ctype.case_type_id)
WHERE ccase.employee_id = l_user_id)
LOOP
IF cases.employee_id IS NOT NULL THEN
PIPE ROW (cases.case_id);
END IF;
END LOOP;
RETURN;
END;
--$$
When I execute this function then I get the following result
select * from table(select employee_all_case(14533,1190) from dual)
My question here is: I really do not understand the pipelined function and how can I obtain the same result in PostgreSQL as Oracle query ?
Please help.
Thank you guys, your solution was very helpful.
I found the desire result:
-- select * from employee_all_case(14533,1190);
-- drop function employee_all_case
create or replace FUNCTION employee_all_case(p_ugr_id IN integer ,p_case_type_id IN integer)
returns table (case_id double precision)
-- PIPELINED
LANGUAGE 'plpgsql'
COST 100
VOLATILE
AS $$
DECLARE
-- is
l_user_id integer;
l_account_id integer;
BEGIN
l_user_id := cp_lookup$get_user_id_from_ugr_id(p_ugr_id);
l_account_id := cp_lookup$acctid_from_ugr(p_ugr_id);
RETURN QUERY SELECT ccase.case_id
FROM ct_case ccase
INNER JOIN ct_case_type ctype ON ccase.case_type_id = ctype.case_type_id
WHERE ccase.employee_id = p_ugr_id
and ccase.employee_id IS NOT NULL;
--return NEXT;
END;
$$
You would rewrite that to a set returning function:
Change the return type to
RETURNS SETOF integer
and do away with the PIPELINED.
Change the PIPE ROW statement to
RETURN NEXT cases.case_id;
Of course, you will have to do the obvious syntactic changes, like using integer instead of NUMBER and putting the IN before the parameter name.
But actually, it is quite unnecessary to write a function for that. Doing it in a single SELECT statement would be both simpler and faster.
Pipelined functions are best translated to a simple SQL function returning a table.
Something like this:
create or replace function employee_all_case(p_ugr_id integer, p_case_type_IN integer)
returns table (case_id integer)
as
$$
SELECT ccase.case_id
FROM ct_case ccase
INNER JOIN ct_case_type ctype ON ccase.case_type_id = ctype.case_type_id
WHERE ccase.employee_id = p_ugr_id
and cases.employee_id IS NOT NULL;
$$
language sql;
Note that your sample code did not use the second parameter p_case_type_id.
Usage is also more straightforward:
select *
from employee_all_case(14533,1190);
Before diving into the solution, I will provide some information which will help you to understand better.
So basically PIPELINED came into picture for improving memory allocation at run time.
As you all know collections will occupy space when ever they got created. So the more you use, the more memory will get allocated.
Pipelining negates the need to build huge collections by piping rows out of the function.
saving memory and allowing subsequent processing to start before all the rows are generated.
Pipelined table functions include the PIPELINED clause and use the PIPE ROW call to push rows out of the function as soon as they are created, rather than building up a table collection.
By using Pipelined how memory usage will be optimized?
Well, it's very simple. instead of storing data into an array, just process the data by using pipe row(desired type). This actually returns the row and process the next row.
coming to solution in plpgsql
simple but not preferred while storing large data.
Remove PIPELINED from return declaration and return an array of desired type. something like RETURNS typrec2[].
Where ever you are using pipe row(), add that entry to array and finally return that array.
create a temp table like
CREATE TEMPORARY TABLE temp_table (required fields) ON COMMIT DROP;
and insert data into it. Replace pipe row with insert statement and finally return statement like
return query select * from temp_table
**The best link for understanding PIPELINED in oracle [https://oracle-base.com/articles/misc/pipelined-table-functions]
pretty ordinary for postgres reference [http://manojadinesh.blogspot.com/2011/11/pipelined-in-oracle-as-well-in.html]
Hope this helps some one conceptually.
I'm trying to store a simple SELECT query with the new CREATE PROCEDURE method in PostgreSQL 11. My idea is to store the queries in the DB, because I can have a much simple code in my API server and maybe I don't need to develop a query builder if I can use if/else in an sql function with enforced type safety. I have this minimal example:
First I tried this plpgsql function:
CREATE OR REPLACE PROCEDURE test_proc() AS $$
BEGIN
SELECT * FROM my_db
LIMIT 1;
END;
$$ LANGUAGE plpgsql;
CALL test_proc();
However throws this error:
ERROR: query has no destination for result data
HINT: If you want to discard the results of a SELECT, use PERFORM instead.
CONTEXT: PL/pgSQL function test_proc() line 3 at SQL statement SQL state: 42601
If I trying to use RETURN QUERY:
CREATE OR REPLACE PROCEDURE test_proc() AS $$
BEGIN
RETURN QUERY;
SELECT * FROM my_db
LIMIT 1;
END;
$$ LANGUAGE plpgsql;
I'm getting this error:
ERROR: cannot use RETURN QUERY in a non-SETOF function
LINE 17: RETURN QUERY; ^
SQL state: 42804
Character: 310
I'm also getting error when I try to use RETURNS void AS $$ or RETURNS table(...) AS $$. Seems like RETURNS not supported in CREATE PROCEDURE? So, is it possible to return a table with the new stored procedure method? Or if it's not, maybe JSON?
Procedures in PostgreSQL (Oracle, DB2) are not same like procedures in MS-SQL. It has different target, and you cannot use it. Usually, the best what you can do, forgot all what you know from MSSQL. The procedural part is really different.
Only functions can returns some data - so you need to use functions. Functions can returns scalar value, composite value or array value, or table. You want function that returns table.
CREATE OR REPLACE FUNCTION fx()
RETURNS SETOF mytab AS $$
BEGIN
RETURN QUERY SELECT * FROM mytab;
END
$$ LANGUAGE plpgsql;
SELECT * FROM fx();
For record:
You can use SQL function, that can have better (or worse) performance (depends on context). These functions are sometimes named as parametrized views.
CREATE OR REPLACE FUNCTION fx()
RETURNS SETOF mytab AS $$
SELECT * FROM mytab;
$$ LANGUAGE sql;
Attention: this technique is antipattern!!! Don't do it. It is really not good idea. The functions should not to wrap queries. If you want to hide some complexity of queries, then use a views. Don't use a functions. Functions are effective barier for query optimizer, and when you use this antipattern, then optimizer cannot to well optimize any non trivial queries that use in this form evaluated subqueries.
Use it - if you want very very slow applications - or if your data model or queries are primitive. In other cases, don't do it.
Don't afraid of SQL - it is great language designed for manual usage. It is good to place all data access to one module (model), to don't access database everywhere in your code, but it is bad too hide SQL in your code.
First of all Procedure was introduced in PostgreSQL 11, If you are using below 11th version, you cannot use Procedures. Instead to Procedure you can use functions.
Syntax to create function
CREATE or replace function function_name(_parameter varchar)
returns table(col1 varchar, col2 varchar, col3 varchar)
language 'plpgsql'
as $BODY$
BEGIN
return query select a.col1, a.col2, b.col3 from table a
join table2 as b on a.col1 = b.col1;
END;
$BODY$;
you can call a function same a like table
select * From function_name('sample data');
syntax to create Procedure.
CREATE OR REPLACE PROCEDURE procedure_name(_parameter varcar,INOUT result refcursor)
LANGUAGE 'plpgsql'
AS $BODY$
BEGIN
open result for SELECT , * from sampletable where a = _parameter;
END;
$BODY$;
you can execute a Procedure using call keyword, within a transaction
BEGIN;
CALL public.procedure_name( 'sample data', 'test');
fetch all in "test";
COMMIT;
The postgreSql 11. we have to create a stored procedure
there is the solution :
Create procedure to execute query in PostgreSQL
i'm trying to use the right modifier to a particular function that checks if a count is bigger or equal 2, but i'm not sure which one to use, here is the function:
CREATE FUNCTION check_table_ids()
RETURNS trigger AS
$$
DECLARE
counter integer := (SELECT count(*) FROM table WHERE fk_id = NEW.fk_id AND status <> 'CANCELED');
BEGIN
IF counter >= 2 THEN
RAISE EXCEPTION 'The number of entries for "%" is greater or equal than 2', NEW.fk_id;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql VOLATILE;
BTW this function will be called by a trigger on insert.
According to the manual (https://www.postgresql.org/docs/current/static/sql-createfunction.html)
STABLE indicates that the function cannot modify the database, and that within a single table scan it will consistently return the same result for the same argument values, but that its result could change across SQL statements.
So if you modify the database or give different results without the database changing then use VOLATILE otherwise use STABLE.
For the code in your question STABLE should be fine.
The fact that this is called by a trigger doesn't make a difference as it's the content of the function you are declaring as STABLE not its usage.
I need to show the Tree_Nodes table data
CREATE OR REPLACE FUNCTION sample()
RETURNS TABLE() AS
$BODY$
BEGIN
select * from "Tree_Nodes";
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100
ROWS 1000;
ALTER FUNCTION sample()
OWNER TO postgres;
It fails with:
ERROR: query has no destination for result data
Avoid the error and i will get the table column format in all data
To return the result of a SELECT, a pure SQL function is much more suitable:
CREATE OR REPLACE FUNCTION sample()
RETURNS TABLE ( .... ) AS
$BODY$
select * from "Tree_Nodes";
$BODY$
LANGUAGE sql;
Or if you really need PL/pgSQL, you need to use return query (which is clearly documented in the manual)
CREATE OR REPLACE FUNCTION sample()
RETURNS TABLE (....)
AS
$BODY$
BEGIN
return query select * from "Tree_Nodes";
END;
$BODY$
LANGUAGE plpgsql;
But you cannot just specify returns table() you have to also define the structure of the result:
CREATE OR REPLACE FUNCTION sample()
RETURNS TABLE(id integer, some_column text, other_column decimal(10,2), ...)
AS
The exact error you quote is caused by using SELECT without an INTO clause in PL/PgSQL. You must either use SELECT INTO somevariable, use RETURN QUERY, if you want to discard the data, use the PERFORM statement instead of SELECT, as covered by the PL/PgSQL manual.
Once you fix that by using RETURN QUERY SELECT .... you'll find that the function still doesn't work, because RETURNS TABLE() doesn't make sense. You're returning an empty result set. It'll fail, complaining that the statement is returning a result set that doesn't match the function.
It makes no sense to do this anyway, since you can just write it as a trivial SQL function like:
CREATE OR REPLACE FUNCTION sample()
RETURNS SETOF "Tree_Nodes"
AS $$
SELECT * FROM "Tree_Nodes";
$$ LANGUAGE sql;
This function appears to serve no purpose. What are you trying to achieve with it?
(By the way, you should generally avoid SELECT * in production code. List the columns. That way, if you add a column later, things that use the table won't suddenly stop working.)
I want a plpgsql function that returns the content of any table, given the name. The function below, although not working because of many reasons will gove you the general idea. Safety and coding practice aside, what's the easiest way to accomplish this?
In the end I want to get these results trough a Java CallableStatement.
CREATE OR REPLACE FUNCTION get_table(tablename VARCHAR)
RETURNS SETOF RECORD AS $PROC$
BEGIN
RETURN QUERY SELECT * FROM tablename;
END;
$PROC$ LANGUAGE 'plpgsql';
You can get your function working like this:
CREATE OR REPLACE FUNCTION get_table(tablename VARCHAR)
RETURNS SETOF RECORD AS $PROC$
BEGIN
RETURN QUERY EXECUTE 'SELECT * FROM ' || quote_ident(tablename);
END;
$PROC$ LANGUAGE 'plpgsql';
In order to call it, you must specify names and data types for all returned columns. If you want to list table "t" which has two columns, you could use your function like this:
SELECT * FROM get_table('t') x(id int, val text);
Which of course, is longer and a lot more trouble than either:
SELECT * FROM t;
or the equivalent:
TABLE t;
I really can't imagine a use-case where such a function makes anything better.