I have a generic key/value table that I would like to preprocess in a function to filter by the key, and then name the value according to the key.
The table is like this where id points into another table (but really is don't care for the purposes of this question):
CREATE TABLE keyed_values (id INTEGER, key TEXT, value TEXT, PRIMARY KEY(id, key));
INSERT INTO keyed_values(id, key, value) VALUES
(1, 'x', 'Value for x for 1'),
(1, 'y', 'Value for y for 1'),
(2, 'x', 'Value for x for 2'),
(2, 'z', 'Value for z for 2');
and so forth. The key values can vary and not all IDs will use the same keys.
Specifically, what I'd like to do is to create a function that would produce rows matching a given key. Then it would return a row for each matching key with the value named with the key.
So the output of the function given the key name argument 'x' would be the same as if I executed this:
SELECT id, value x FROM keyed_values WHERE key = 'x';
id | x
----+-------------------
1 | Value for x for 1
2 | Value for x for 2
(2 rows)
Here's my stab, put together from looking at some other SO answers:
CREATE FUNCTION value_for(key TEXT) RETURNS SETOF RECORD AS $$
BEGIN
RETURN QUERY EXECUTE format('SELECT id, value %I FROM keyed_values WHERE key = %I', key, key);
END;
$$ LANGUAGE 'plpgsql';
The function is accepted. But when I execute it, I get this error:
SELECT * FROM value_for('x');
ERROR: a column definition list is required for functions returning "record"
LINE 1: SELECT * FROM value_for('x');
I get what it's telling me, but not how to fix that. How can I dynamically define the output column name?
The key used in the WHERE clause is not an identifier but a literal, so the function should looks like:
CREATE FUNCTION value_for(key TEXT) RETURNS SETOF RECORD AS $$
BEGIN
RETURN QUERY EXECUTE format('SELECT id, value %I FROM keyed_values WHERE key = %L', key, key);
END;
$$ LANGUAGE plpgsql;
From the documentation:
If the function has been defined as returning the record data type, then an alias or the key word AS must be present, followed by a column definition list in the form ( column_name data_type [, ... ]). The column definition list must match the actual number and types of columns returned by the function.
Use a column definition list:
SELECT * FROM value_for('x') AS (id int, x text);
Note that you do not need dynamic SQL nor plpgsql, as the column name is defined in the above list. The simple SQL function should be a bit faster:
CREATE OR REPLACE FUNCTION value_for(_key text)
RETURNS SETOF RECORD AS $$
SELECT id, value
FROM keyed_values
WHERE key = _key
$$ LANGUAGE SQL;
If you do not like being forced to append a column definition list to every function call, you can return a table from the function:
CREATE OR REPLACE FUNCTION value_for_2(_key text)
RETURNS TABLE(id int, x text) AS $$
SELECT id, value
FROM keyed_values
WHERE key = _key
$$ LANGUAGE SQL;
SELECT * FROM value_for_2('x');
id | x
----+-------------------
1 | Value for x for 1
2 | Value for x for 2
(2 rows)
However this does not solve the issue. There is no way to dynamically define a column name returned from a function. The names and types of returned columns must be specified before the query is executed.
Related
CREATE FUNCTION test(VARIADIC arr text[])
RETURNS TABLE (data_time TIMESTAMPTZ, id text, data jsonb)
AS 'SELECT data_timestamp, key, value
FROM hit_count
CROSS JOIN jsonb_each(data)
WHERE key = ALL ($1)'
LANGUAGE SQL;
If I call the function above with select test('123') it works fine. If I call it with select test('123','234') it returns nothing ie
test
------
(0 rows)
However if I define it as
CREATE FUNCTION test(VARIADIC arr text[])
RETURNS TABLE (data_time TIMESTAMPTZ, id text, data jsonb)
AS 'SELECT data_timestamp, key, value
FROM hit_count
CROSS JOIN jsonb_each(data)
WHERE key != ALL ($1)'
LANGUAGE SQL;
then the function returns all the data but those that fit the condition
Any ideas??
ALL was used as opposed to ANY the corrected function looks like this
CREATE FUNCTION test(VARIADIC arr text[])
RETURNS TABLE (data_time TIMESTAMPTZ, id text, data jsonb)
AS 'SELECT data_timestamp, key, value
FROM hit_count
CROSS JOIN jsonb_each(data)
WHERE key = ANY ($1)'
LANGUAGE SQL;
I'm working with Postgres and PostGIS. Trying to write a function that that selects specific columns according to the given argument.
I'm using a WITH statement to create the result table before converting it to bytea to return.
The part I need help with is the $4 part. I tried it is demonstrated below and $4::text and both give me back the text value of the input and not the column value in the table if cols=name so I get back from the query name and not the actual names in the table. I also try data($4) and got type error.
The code is like this:
CREATE OR REPLACE FUNCTION select_by_txt(z integer,x integer,y integer, cols text)
RETURNS bytea
LANGUAGE 'plpgsql'
AS $BODY$
declare
res bytea;
begin
WITH bounds AS (
SELECT ST_TileEnvelope(z, x, y) AS geom
),
mvtgeom AS (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom, $4
FROM table1 t, bounds
WHERE ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
)
SELECT ST_AsMVT(mvtgeom, 'public.select_by_txt')
INTO res
FROM mvtgeom;
RETURN res;
end;
$BODY$;
Example for calling the function:
select_by_txt(10,32,33,"col1,col2")
The argument cols can be multiple column names from 1 and not limited from above. The names of the columns inside cols will be checked before calling the function that they are valid columns.
Passing multiple column names as concatenated string for dynamic execution urgently requires decontamination. I suggest a VARIADIC function parameter instead, with properly quoted identifiers (using quote_ident() in this case):
CREATE OR REPLACE FUNCTION select_by_txt(z int, x int, y int, VARIADIC cols text[] = NULL, OUT res text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format(
$$
SELECT ST_AsMVT(mvtgeom, 'public.select_by_txt')
FROM (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom%s
FROM table1 t
JOIN (SELECT ST_TileEnvelope($1, $2, $3)) AS bounds(geom)
ON ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
) mvtgeom
$$, (SELECT ', ' || string_agg(quote_ident (col), ', ') FROM unnest(cols) col)
)
INTO res
USING z, x, y;
END
$func$;
db<>fiddle here
The format specifier %I for format() deals with a single identifier. You have to put in more work for multiple identifiers, especially for a variable number of 0-n identifiers. This implementation quotes every single column name, and only add a , if any column names have been passed. So it works for every possible input, even no input at all. Note VARIADIC cols text[] = NULL as last input parameter with NULL as default value:
Optional argument in PL/pgSQL function
Related:
quote_ident() does not add quotes to column name "first"
Column names are case sensitive in this context!
Call for your example (important!):
SELECT select_by_txt(10,32,33,'col1', 'col2');
Alternative syntax:
SELECT select_by_txt(10,32,33, VARIADIC '{col1,col2}');
More revealing call, with a third column name and malicious (though futile) intent:
SELECT select_by_txt(10,32,33,'col1', 'col2', $$col3'); DROP TABLE table1;--$$);
About that odd third column name and SQL injection:
https://www.explainxkcd.com/wiki/index.php/Little_Bobby_Tables
About VAIRADIC parameters:
Return rows matching elements of input array in plpgsql function
Pass multiple values in single parameter
Using an OUT parameter for simplicity. That's totally optional. See:
Returning from a function with OUT parameter
What I would not do
If you really, really trust the input to be a properly formatted list of 1 or more valid column names at all times - and you asserted that ...
the names of the columns inside cols will be checked before calling the function that they are valid columns
You could simplify:
CREATE OR REPLACE FUNCTION select_by_txt(z int, x int, y int, cols text, OUT res text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format(
$$
SELECT ST_AsMVT(mvtgeom, 'public.select_by_txt')
FROM (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom, %s
FROM table1 t
JOIN (SELECT ST_TileEnvelope($1, $2, $3)) AS bounds(geom)
ON ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
) mvtgeom
$$, cols
)
INTO res
USING z, x, y;
END
$func$;
(How can you be so sure that the input will always be reliable?)
You would need to use a dynamic query:
CREATE OR REPLACE FUNCTION select_by_txt(z integer,x integer,y integer, cols text)
RETURNS bytea
LANGUAGE 'plpgsql'
AS $BODY$
declare
res bytea;
begin
EXECUTE format('
WITH bounds AS (
SELECT ST_TileEnvelope($1, $2, $3) AS geom
),
mvtgeom AS (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom, %I
FROM table1 t, bounds
WHERE ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
)
SELECT ST_AsMVT(mvtgeom, ''public.select_by_txt'')
FROM mvtgeom', cols)
INTO res
USING z,x,y;
RETURN res;
end;
$BODY$;
I have a Postgres function which is returning a table:
CREATE OR REPLACE FUNCTION testFunction() RETURNS TABLE(a int, b int) AS
$BODY$
DECLARE a int DEFAULT 0;
DECLARE b int DEFAULT 0;
BEGIN
CREATE TABLE tempTable AS SELECT a, b;
RETURN QUERY SELECT * FROM tempTable;
DROP TABLE tempTable;
END;
$BODY$
LANGUAGE plpgsql;
This function is not returning data in row and column form. Instead it returns data as:
(0,0)
That is causing a problem in Coldfusion cfquery block in extracting data. How do I get data in rows and columns when a table is returned from this function? In other words: Why does the PL/pgSQL function not return data as columns?
To get individual columns instead of the row type, call the function with:
SELECT * FROM testfunction();
Just like you would select all columns from a table.
Also consider this reviewed form of your test function:
CREATE OR REPLACE FUNCTION testfunction()
RETURNS TABLE(a int, b int)
LANGUAGE plpgsql AS
$func$
DECLARE
_a int := 0;
_b int := 0;
BEGIN
CREATE TABLE tempTable AS SELECT _a, _b;
RETURN QUERY SELECT * FROM tempTable;
DROP TABLE tempTable;
END
$func$;
In particular:
The DECLARE key word is only needed once.
Avoid declaring parameters that are already (implicitly) declared as OUT parameters in the RETURNS TABLE (...) clause.
Don't use unquoted CaMeL-case identifiers in Postgres. It works, unquoted identifiers are cast to lower case, but it can lead to confusing errors. See:
Are PostgreSQL column names case-sensitive?
The temporary table in the example is completely useless (probably over-simplified). The example as given boils down to:
CREATE OR REPLACE FUNCTION testfunction(OUT a int, OUT b int)
LANGUAGE plpgsql AS
$func$
BEGIN
a := 0;
b := 0;
END
$func$;
Of course you can do this by putting the function call in the FROM clause, like Eric Brandstetter correctly answered.
However, this is sometimes complicating in a query that already has other things in the FROM clause.
To get the individual columns that the function returns, you can use this syntax:
SELECT (testfunction()).*
Or to get only the column called "a":
SELECT (testfunction()).a
Place the whole function, including the input value(s) in parenteses, followed by a dot and the desired column name, or an asterisk.
To get the column names that the function returns, you'll have to either:
check the source code
inspect the result of the function first, like so : SELECT * FROM testfunction() .
The input values can still come out of a FROM clause.
Just to illustrate this, consider this function and test data:
CREATE FUNCTION funky(a integer, b integer)
RETURNS TABLE(x double precision, y double precision) AS $$
SELECT a*random(), b*random();
$$ LANGUAGE SQL;
CREATE TABLE mytable(a integer, b integer);
INSERT INTO mytable
SELECT generate_series(1,100), generate_series(101,200);
You could call the function "funky(a,b)", without the need to put it in the FROM clause:
SELECT (funky(mytable.a, mytable.b)).*
FROM mytable;
Which would result in 2 columns:
x | y
-------------------+-------------------
0.202419687062502 | 55.417385618668
1.97231830470264 | 63.3628275180236
1.89781916560605 | 1.98870931006968
(...)
I have a function that has the return type of record.
Below is the function am using, basically what the function does is , it takes in input paramter and queries the database to return the columns:
drop function if exists test_proc(sr_number1 bigint);
create or replace function test_proc(sr_number1 bigint) RETURNS record /*SETOF tbl*/ AS $$
declare
i integer default 0;
record_type record;
begin
select sr_num,product_number,phone,addr into record_type from my_table where sr_num=sr_number1;
return record_type;
end
$$ LANGUAGE plpgsql;
Unfortunately, when I execute the function as
select test_proc(12345); I get the result as a comma separated list in just one column like (sr_num,product_number,phone,addr). But what I was hoping to have it return was a table row with the column values and their respective column names.
I also tried executing the function as
select * from test_proc(12345); but get the following error
ERROR: a column definition list is required for functions returning "record"
When querying a function that returns a record you must specify the type of record you want to get in the result
select * from test_proc(12345) f(b bigint, t text);
This should work.
A better solution is to declare the type of record in the function
CREATE OR REPLACE FUNCTION test_proc(sr_number1 bigint)
RETURNS TABLE(b bigint, t text) AS $$ $$
I have dynamicly generated SELECT. I try to return result as SETOF RECORD. Sth like that:
CREATE FUNCTION test(column_name text) RETURNS SETOF RECORD AS $$
DECLARE
row RECORD;
BEGIN
FOR row IN EXECUTE 'SELECT ' || quote_ident(column_name) || ' FROM dates'
LOOP
RETURN NEXT row;
END LOOP;
RETURN;
END;
$$ LANGUAGE 'plpgsql';
When I try:
SELECT * FROM test('column1');
I get this:
ERROR: a column definition list is required for functions returning "record"
I know that column1 is integer type:
SELECT * FROM test('column1') f(a int);
result is correct, because I know that this is going to be Integer type.
When I try:
SELECT * FROM test('column1') f(a varchar);
I get error:
ERROR: wrong record type supplied in RETURN NEXT
DETAIL: Returned type integer does not match expected type character varying in column 1.
Now my question:
What to do to get rid of part of querty where I define types 'f(a int)'. It should by feasible because Postgres knowns what is returned type. I tried with IMMUTABLE options, but unsuccessfully.
You could cast the value to text inside the function, and declare that the function RETURNS SETOF text. You can also return the whole result set at once; no need to iterate explicitly.
CREATE TABLE dates (column1 int, column2 date);
INSERT INTO dates VALUES (1, date '2012-12-22'), (2, date '2013-01-01');
CREATE FUNCTION test(column_name text) RETURNS SETOF text AS $$
BEGIN
RETURN QUERY EXECUTE 'SELECT '
|| quote_ident(column_name) || '::text FROM dates';
END;
$$ LANGUAGE 'plpgsql';
Now SELECT test('column1'); yields:
test
------
1
2
(2 rows)
... and (with my locale settings) SELECT test('column2'); yields:
test
------------
2012-12-22
2013-01-01
(2 rows)
You need to specify OUT parameters corresponding to the columns you want to return.