How to pass multiple rows to PostgreSQL function? - postgresql

How can we pass an array of (an unlimited amount of) rows (ie, a constant table) as the parameter/argument of a PostgreSQL function?
Here's an idea:
CREATE TYPE foo AS (
x bigint,
y smallint,
z varchar(64)
);
CREATE OR REPLACE FUNCTION bar(bigint, foo[]) RETURNS TABLE(a bigint, x bigint, y smallint, z varchar(64)) AS
$$
SELECT $1, x, y, z FROM unnest($2);
$$
LANGUAGE SQL;
The below function call works, but is there a way to make it shorter?
SELECT * FROM bar(1, ARRAY[(1,2,'body1'),(2,1,'body2')]::foo[]);
For example, we can't remove the ::foo[] cast, but is there a way to rewrite things so that we can omit it?
Should we be using a variatic argument?

My Google searches kept leading me here, so I'm going to post an answer that may not match exactly the needs of the OP, but might be helpful to others who see the title How to pass multiple rows to PostgreSQL function?
The OPs original request was for a type:
CREATE TYPE foo AS (
x bigint,
y smallint,
z varchar(64)
);
If you are like me, you may want to pass in the results of a standard SELECT query to a function. So imagine I have a table (rather than a type) created as:
CREATE TABLE foo AS (
x bigint,
y smallint,
z varchar(64)
);
I want to pass to a function the results of:
SELECT * from foo WHERE x = 12345;
The results may be zero or many rows.
According to the postgres docs at https://www.postgresql.org/docs/9.5/static/rowtypes.html creating a table also leads to the creation of a composite type with the same name. Which is helpful, since this automatically handles the CREATE TYPE foo in the original question, which I can now pass in to a function as an array.
Now I can create a function that accepts an array of foo typed values (simplified to focus on what is passed in, and how the records are used, rather than what is returned):
CREATE OR REPLACE FUNCTION bar(someint bigint, foos foo[]) RETURNS ...
LANGUAGE plpgsql
AS $$
DECLARE
foo_record record;
begin
-- We are going to loop through each composite type value in the array
-- The elements of the composite value are referenced just like
-- the columns in the original table row
FOREACH foo_record IN ARRAY foos LOOP
-- do something, maybe like:
INSERT INTO new_foo (
x, y, z
)
VALUES (
foo_record.x,
foo_record.y,
foo_record.z
);
END LOOP;
RETURN...
END;
$$;
This function bar(bigint, foo[]) can then be called quite simply with:
SELECT bar(4126521, ARRAY(SELECT * from foo WHERE x = 12345));
which passes in all the rows of a query on the foo table as a foo typed array. The function as we have seen then performs some action against each of those rows.
Although the example is contrived, and perhaps not exactly what the OP was asking, it fits the title of the question and might save others from having to search more to find what they need.
EDIT naming the function arguments makes things easier

PostgreSQL doesn't have table-valued variables (yet), so nothing's going to be pretty. Passing arrays is inefficient but will work for reasonable-sized inputs.
For bigger inputs, what often works is to pass a refcursor. It's clumsy, but can be practical for larger data sets, sometimes combined with temp tables.
e.g.
CREATE OR REPLACE FUNCTION bar(i bigint, c refcursor) RETURNS TABLE(a bigint, x bigint, y smallint, z varchar(64)) AS
$$
DECLARE
cursrow foo;
BEGIN
LOOP
FETCH NEXT FROM c INTO cursrow;
a := i;
x := cursrow.x;
y := cursrow.y;
z := cursrow.z;
RETURN NEXT;
IF NOT FOUND THEN
EXIT;
END IF;
END LOOP;
RETURN;
END;
$$;
usage:
demo=> BEGIN;
BEGIN
demo=> DECLARE "curs1" CURSOR FOR VALUES (1,2,'body1'), (2,1,'body2');
DECLARE CURSOR
craig=> SELECT bar(1, 'curs1');
bar
---------------
(1,1,2,body1)
(1,2,1,body2)
(1,,,)
(3 rows)
demo=> COMMIT;
COMMIT
Not beautiful. But then, plpgsql never is. It's a pity it doesn't have row-valued lvalues, as being able to write something like (x, y, z) := cursrow or ROW(x, y, z) := cursrow would make it a bit less ugly.
RETURN NEXT works, but only if you return record not named out parameters or TABLE.
And sadly, you can't use SQL (not plpgsql) FETCH ALL in a subexpression so you cannot write
RETURN QUERY NEXT i, cursrow.* FROM (FETCH ALL FROM c) AS cursrow;

It seems that one of the problems is the using of smallint type which can not be converted implicitly from an int constants. And consider the following:
-- drop function if exists bar(bigint, variadic foo[]);
-- drop type if exists foo;
CREATE TYPE foo AS (
x bigint,
y int, -- change type to integer
z varchar(64)
);
CREATE OR REPLACE FUNCTION bar(bigint, variadic foo[]) RETURNS TABLE(
a bigint,
x bigint,
y int, -- and here
z varchar(64)) AS
$$
SELECT $1, x, y, z FROM unnest($2);
$$
LANGUAGE SQL;
-- Voila! It is even simpler then the using of the ARRAY constructor
SELECT * FROM bar(1, (1,2,'body1'), (2,1,'body2'), (3,4,'taddy bear'));
dbfiddle
About variadic parameters

Related

Cannot run Postgres Stored Procedure [duplicate]

I have a Postgres function which is returning a table:
CREATE OR REPLACE FUNCTION testFunction() RETURNS TABLE(a int, b int) AS
$BODY$
DECLARE a int DEFAULT 0;
DECLARE b int DEFAULT 0;
BEGIN
CREATE TABLE tempTable AS SELECT a, b;
RETURN QUERY SELECT * FROM tempTable;
DROP TABLE tempTable;
END;
$BODY$
LANGUAGE plpgsql;
This function is not returning data in row and column form. Instead it returns data as:
(0,0)
That is causing a problem in Coldfusion cfquery block in extracting data. How do I get data in rows and columns when a table is returned from this function? In other words: Why does the PL/pgSQL function not return data as columns?
To get individual columns instead of the row type, call the function with:
SELECT * FROM testfunction();
Just like you would select all columns from a table.
Also consider this reviewed form of your test function:
CREATE OR REPLACE FUNCTION testfunction()
RETURNS TABLE(a int, b int)
LANGUAGE plpgsql AS
$func$
DECLARE
_a int := 0;
_b int := 0;
BEGIN
CREATE TABLE tempTable AS SELECT _a, _b;
RETURN QUERY SELECT * FROM tempTable;
DROP TABLE tempTable;
END
$func$;
In particular:
The DECLARE key word is only needed once.
Avoid declaring parameters that are already (implicitly) declared as OUT parameters in the RETURNS TABLE (...) clause.
Don't use unquoted CaMeL-case identifiers in Postgres. It works, unquoted identifiers are cast to lower case, but it can lead to confusing errors. See:
Are PostgreSQL column names case-sensitive?
The temporary table in the example is completely useless (probably over-simplified). The example as given boils down to:
CREATE OR REPLACE FUNCTION testfunction(OUT a int, OUT b int)
LANGUAGE plpgsql AS
$func$
BEGIN
a := 0;
b := 0;
END
$func$;
Of course you can do this by putting the function call in the FROM clause, like Eric Brandstetter correctly answered.
However, this is sometimes complicating in a query that already has other things in the FROM clause.
To get the individual columns that the function returns, you can use this syntax:
SELECT (testfunction()).*
Or to get only the column called "a":
SELECT (testfunction()).a
Place the whole function, including the input value(s) in parenteses, followed by a dot and the desired column name, or an asterisk.
To get the column names that the function returns, you'll have to either:
check the source code
inspect the result of the function first, like so : SELECT * FROM testfunction() .
The input values can still come out of a FROM clause.
Just to illustrate this, consider this function and test data:
CREATE FUNCTION funky(a integer, b integer)
RETURNS TABLE(x double precision, y double precision) AS $$
SELECT a*random(), b*random();
$$ LANGUAGE SQL;
CREATE TABLE mytable(a integer, b integer);
INSERT INTO mytable
SELECT generate_series(1,100), generate_series(101,200);
You could call the function "funky(a,b)", without the need to put it in the FROM clause:
SELECT (funky(mytable.a, mytable.b)).*
FROM mytable;
Which would result in 2 columns:
x | y
-------------------+-------------------
0.202419687062502 | 55.417385618668
1.97231830470264 | 63.3628275180236
1.89781916560605 | 1.98870931006968
(...)

How to return uncertain number columns of a table from a postgresql function?

As we know, plpgsql functions can return a table like this:
RETURNS table(int, char(1), ...)
But how to write this function, when the list of columns is uncertain at the time of creating the function.
When a function returns anonymous records
RETURNS SETOF record
you have to provide a column definition list when calling it with SELECT * FROM. SQL demands to know column names and types to interpret *. For registered tables and types this is provided by the system catalog. For functions you need to declare it yourself one way or the other. Either in the function definition or in the call. The call could look like #Craig already provided. You probably didn't read his answer carefully enough.
Depending on what you need exactly, there are a number of ways around this, though:
1) Return a single anonymous record
Example:
CREATE OR REPLACE FUNCTION myfunc_single() -- return a single anon rec
RETURNS record AS
$func$
DECLARE
rec record;
BEGIN
SELECT into rec 1, 'foo'; -- note missing type for 'foo'
RETURN rec;
END
$func$ LANGUAGE plpgsql;
This is a very limited niche. Only works for a single anonymous record from a function defined with:
RETURNS record
Call without * FROM:
SELECT myfunc_single();
Won't work for a SRF (set-returning function) and only returns a string representation of the whole record (type record). Rarely useful.
To get individual cols from a single anonymous record, you need to provide a column definition list again:
SELECT * FROM myfunc_single() AS (id int, txt unknown); -- note "unknown" type
2) Return well known row type with a super-set of columns
Example:
CREATE TABLE t (id int, txt text, the_date date);
INSERT INTO t VALUES (3, 'foz', '2014-01-13'), (4, 'baz', '2014-01-14');
CREATE OR REPLACE FUNCTION myfunc_tbl() -- return well known table
RETURNS SETOF t AS
$func$
BEGIN
RETURN QUERY
TABLE t;
-- SELECT * FROM t; -- equivalent
END
$func$ LANGUAGE plpgsql;
The function returns all columns of the table. This is short and simple and performance won't suffer as long as your table doesn't hold a huge number of columns or huge columns.
Select individual columns on call:
SELECT id, txt FROM myfunc_tbl();
SELECT id, the_date FROM myfunc_tbl();
-> SQLfiddle demonstrating all.
3) Advanced solutions
This answer is long enough already. And this closely related answer has it all:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Look to the last chapter in particular: Various complete table types
If the result is of uncertain/undefined format you must use RETURNS record or (for a multi-row result) RETURNS SETOF record.
The calling function must then specify the table format, eg:
SELECT my_func() AS result(a integer, b char(1));
BTW, char is an awful data type with insane space-padding rules that date back to the days of fixed-width file formats. Don't use it. Always just use text or varchar.
Given comments, let's make this really explicit:
regress=> CREATE OR REPLACE FUNCTION f_something() RETURNS SETOF record AS $$
SELECT 1, 2, TEXT 'a';
$$ LANGUAGE SQL;
CREATE FUNCTION
regress=> SELECT * FROM f_something();
ERROR: a column definition list is required for functions returning "record"
LINE 1: SELECT * FROM f_something();
regress=> SELECT * FROM f_something() AS x(a integer, b integer, c text);
a | b | c
---+---+---
1 | 2 | a
(1 row)

Easy way to have return type be SETOF table plus additional fields?

I'm writing a PL/pgSQL stored procedure that will return a set of records; each record contains all the fields of an existing table (call it Retailer, which has two fields: retailer_key and retailer_name). This, of course, works:
CREATE FUNCTION proc_Find_retailers
(IN p_Store_key INT)
RETURNS SETOF Retailer
AS $$ ...`
Now I want to update the sp so that it returns an additional two fields to the 'end' of each returned record. I can do something such as:
CREATE FUNCTION proc_Find_store
(IN p_Store_key INT)
RETURNS TABLE (
retailer_key int,
retailer_name varchar(50),
addl_field_1 int,
addl_field_2 double precision)
AS $$ ...
In the real world, my Retailer table has 50 fields (not the two in my example), so enumerating all those fields in the RETURNS TABLE clause is tedious. Is there any shortcut to this, so that I might say something such as (I realize I'm making stuff up here that's syntactically illegal, but I'm doing it to give you the flavor of what I'm looking for):
CREATE FUNCTION proc_Find_store
(IN p_Store_key INT)
RETURNS (SETOF Store,
addl_field_1 int,
addl_field_2 double precision)
AS $$ ...
You could return a whole row as composite type and add some more:
CREATE OR REPLACE FUNCTION f_rowplus()
RETURNS TABLE (rec demo, add_int int, add_txt text) AS
$func$
SELECT d, 5, 'baz'::text FROM demo d;
$func$ LANGUAGE sql;
But then, when you use the simple call:
SELECT * FROM f_rowplus();
You get the row from table demo as separate composite type. You'd have to call:
SELECT (rec).*, add_int, add_txt FROM f_rowplus();
to get all individual columns. Parentheses required.
Postgres is a bit inconsistent here. If you create a function with:
CREATE OR REPLACE FUNCTION f_row2()
RETURNS TABLE (rec demo) AS
...
then the composite type demo is silently converted into individual columns (decomposed). No link to the original composite type remains. You cannot reference the declared output column rec at all, since that has been replaced with the columns of the decomposed type. This call would result in an error message:
SELECT rec FROM f_row2(); -- error!
Same here:
CREATE OR REPLACE FUNCTION f_row3(OUT rec demo)
RETURNS SETOF demo AS
...
However, as soon as you add any more OUT columns, the composite type is preserved as declared (not decomposed) and you can:
SELECT rec FROM f_rowplus();
with the first function.
db<>fiddle here - demonstrating all variants
Old sqlfiddle
Asides
When using a function returning multiple columns in the FROM list (as table function) and decomposing in the SELECT list like this:
SELECT (rec).* FROM f_rowplus();
... the function is still evaluated once only - while calling and decomposing in the SELECT list directly like this:
SELECT (f_rowplus()).*; -- also: different result
... would evaluate once for every column in the return type. See:
How to avoid multiple function evals with the (func()).* syntax in an SQL query?
In Postgres 14 or later, you can also use standard-SQL syntax:
CREATE OR REPLACE FUNCTION f_rowplus_std()
RETURNS TABLE (rec demo, add_int int, add_txt text)
LANGUAGE sql PARALLEL SAFE
BEGIN ATOMIC
SELECT d, 5, 'baz'::text FROM demo d;
END;
See:
What does BEGIN ATOMIC ... END mean in a PostgreSQL SQL function / procedure?

Declaring the tuple structure of a record in PL/pgSQL

I can't find anything in the PostgreSQL documentation that shows how to declare a record, or row, while declaring the tuple structure at the same time. If you don't define you tuple structure you get the error "The tuple structure of a not-yet-assigned record is indeterminate".
This is what I'm doing now, which works fine, but there must be a better way to do it.
CREATE OR REPLACE FUNCTION my_func()
RETURNS TABLE (
"a" integer,
"b" varchar
) AS $$
DECLARE r record;
BEGIN
CREATE TEMP TABLE tmp_t (
"a" integer,
"b" varchar
);
-- Define the tuple structure of r by SELECTing an empty row into it.
-- Is there a more straight-forward way of doing this?
SELECT * INTO r
FROM tmp_t;
-- Now I can assign values to the record.
r.a := at.something FROM "another_table" at
WHERE at.some_id = 1;
-- A related question is - how do I return the single record 'r' from
-- this function?
-- This works:
RETURN QUERY
SELECT * FROM tmp_t;
-- But this doesn't:
RETURN r;
-- ERROR: RETURN cannot have a parameter in function returning set
END; $$ LANGUAGE plpgsql;
You are mixing the syntax for returning SETOF values with syntax for returning a single row or value.
-- A related question is - how do I return the single record 'r' from
When you declare a function with RETURNS TABLE, you have to use RETURN NEXT in the body to return a row (or scalar value). And if you want to use a record variable with that it has to match the return type. Refer to the code examples further down.
Return a single value or row
If you just want to return a single row, there is no need for a record of undefined type. #Kevin already demonstrated two ways. I'll add a simplified version with OUT parameters:
CREATE OR REPLACE FUNCTION my_func(OUT a integer, OUT b text)
AS
$func$
BEGIN
a := ...;
b := ...;
END
$func$ LANGUAGE plpgsql;
You don't even need to add RETURN; in the function body, the value of the declared OUT parameters will be returned automatically at the end of the function - NULL for any parameter that has not been assigned.
And you don't need to declare RETURNS RECORD because that's already clear from the OUT parameters.
Return a set of rows
If you actually want to return multiple rows (including the possibility for 0 or 1 row), you can define the return type as RETURNS ...
SETOF some_type, where some_type can be any registered scalar or composite type.
TABLE (col1 type1, col2 type2) - an ad-hoc row type definition.
SETOF record plus OUT parameters to define column names andtypes.
100% equivalent to RETURNS TABLE.
SETOF record without further definition. But then the returned rows are undefined and you need to include a column definition list with every call (see example).
The manual about the record type:
Record variables are similar to row-type variables, but they have no
predefined structure. They take on the actual row structure of the
row they are assigned during a SELECT or FOR command.
There is more, read the manual.
You can use a record variable without assigning a defined type, you can even return such undefined records:
CREATE OR REPLACE FUNCTION my_func()
RETURNS SETOF record AS
$func$
DECLARE
r record;
BEGIN
r := (1::int, 'foo'::text); RETURN NEXT r; -- works with undefined record
r := (2::int, 'bar'::text); RETURN NEXT r;
END
$func$ LANGUAGE plpgsql;
Call:
SELECT * FROM my_func() AS x(a int, b text);
But this is very unwieldy as you have to provide the column definition list with every call. It can generally be replaced with something more elegant:
If you know the type at time of function creation, declare it right away (RETURNS TABLE or friends).
CREATE OR REPLACE FUNCTION my_func()
RETURNS SETOF tbl_or_type AS
$func$
DECLARE
r tbl_or_type;
BEGIN
SELECT INTO tbl_or_type * FROM tbl WHERE id = 10;
RETURN NEXT r; -- type matches
SELECT INTO tbl_or_type * FROM tbl WHERE id = 12;
RETURN NEXT r;
-- Or simpler:
RETURN QUERY
SELECT * FROM tbl WHERE id = 14;
END
$func$ LANGUAGE plpgsql;
If you know the type at time of the function call, there are more elegant ways using polymorphic types:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Your question is unclear as to what you need exactly.
There might be some way that avoids the explicit type declaration, but offhand the best I can come up with is:
CREATE TYPE my_func_return AS (
a integer,
b varchar
);
CREATE OR REPLACE FUNCTION my_func()
RETURNS my_func_return AS $$
DECLARE
r my_func_return;
BEGIN
SELECT 1, 'one' INTO r.a, r.b;
RETURN r;
END; $$ LANGUAGE plpgsql;
Oh, I almost forgot the simplest way to do this:
CREATE OR REPLACE FUNCTION my_func2(out a int, out b text)
RETURNS RECORD AS $$
BEGIN
SELECT 1, 'one' INTO a, b;
RETURN;
END; $$ LANGUAGE plpgsql;
It is much easier to use OUT parameters rather than a record. If iteratively building a set of records (a table) use RETURN NEXT. If generating from a query, use RETURN QUERY. See:
https://stackoverflow.com/a/955289/398670
and:
http://www.postgresql.org/docs/current/static/plpgsql-declarations.html
http://www.postgresql.org/docs/current/static/sql-createfunction.html
http://www.postgresonline.com/journal/archives/129-Use-of-OUT-and-INOUT-Parameters.html
Think:
CREATE OR REPLACE FUNCTION my_func(OUT a integer, OUT b varchar) RETURNS SETOF RECORD AS $$
BEGIN
-- Assign a and b, RETURN NEXT, repeat. when done, RETURN.
END;
$$ LANGUAGE 'plpgsql';

PostgreSQL: store function in column as value

Can functions be stored as anonymous functions directly in column as its value?
Let's say I want this function be stored in column.
Example (pseudocode):
Table my_table: pk (int), my_function (func)
func ( x ) { return x * 100 }
And later use it as:
select
t.my_function(some_input) AS output
from
my_table as t
where t.pk = 1999
Function may vary for each pk.
Your title asks something else than your example.
A function has to be created before you can call it. (title)
An expression has to be evaluated. You would need a meta-function for that. (example)
Here are solutions for both:
1. Evaluate expressions dynamically
You have to take into account that the resulting type can vary. I use polymorphic types for that.
CREATE OR REPLACE FUNCTION f1(int)
RETURNS int
LANGUAGE sql IMMUTABLE AS
'SELECT $1 * 100;';
CREATE OR REPLACE FUNCTION f2(text)
RETURNS text
LANGUAGE sql IMMUTABLE AS
$$SELECT $1 || '_foo';$$;
CREATE TABLE my_expr (
expr text PRIMARY KEY
, def text
, rettype regtype
);
INSERT INTO my_expr VALUES
('x', 'f1(3)' , 'int')
, ('y', $$f2('bar')$$, 'text')
, ('z', 'now()' , 'timestamptz')
;
CREATE OR REPLACE FUNCTION f_eval(text, _type anyelement = 'NULL'::text, OUT _result anyelement)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE
'SELECT ' || (SELECT def FROM my_expr WHERE expr = $1)
INTO _result;
END
$func$;
Related:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Call:
SQL is strictly typed, the same result column can only have one data type. For multiple rows with possibly heterogeneous data types, you might settle for type text, as every data type can be cast to and from text:
SELECT *, f_eval(expr) AS result -- default to type text
FROM my_expr;
Or return multplce columns like:
SELECT *
, CASE WHEN rettype = 'text'::regtype THEN f_eval(expr) END AS text_result -- default to type text
, CASE WHEN rettype = 'int'::regtype THEN f_eval(expr, NULL::int) END AS int_result
, CASE WHEN rettype = 'timestamptz'::regtype THEN f_eval(expr, NULL::timestamptz) END AS tstz_result
-- , more?
FROM my_expr;
db<>fiddle here
2. Create and use functions dynamically
It is possible to create functions dynamically and then use them. You cannot do that with plain SQL, however. You will have to use another function to do that or at least an anonymous code block (DO statement), introduced in PostgreSQL 9.0.
It can work like this:
CREATE TABLE my_func (func text PRIMARY KEY, def text);
INSERT INTO my_func VALUES
('f'
, $$CREATE OR REPLACE FUNCTION f(int)
RETURNS int
LANGUAGE sql IMMUTABLE AS
'SELECT $1 * 100;'$$);
CREATE OR REPLACE FUNCTION f_create_func(text)
RETURNS void
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE (SELECT def FROM my_func WHERE func = $1);
END
$func$;
Call:
SELECT f_create_func('f');
SELECT f(3);
db<>fiddle here
You may want to drop the function afterwards.
In most cases you should just create the functions instead and be done with it. Use separate schemas if you have problems with multiple versions or privileges.
For more information on the features I used here, see my related answer on dba.stackexchange.com.