I want to query a table using the 'with' clause inside a function using PostgreSQL. Can someone please tell me how to do it?
I tried returning the table but it's saying that the column names are ambiguous.
PostgreSQL requires knowing the exact structure of the returned data.
Example:
CREATE OR REPLACE FUNCTION test()
RETURNS TABLE(id integer, caption text)
LANGUAGE plpgsql
AS $function$
begin
return query
with tb (a1, a2) as (
values
(1, 'Test 1'),
(2, 'Test 2'),
(3, 'Test 3'),
(4, 'Test 4')
)
select tb.a1, tb.a2 from tb;
end ;
$function$
;
And fixing - column names are ambiguous - problem recommended use table name aliases inside the function and select columns with aliases.
Related
I'm working with Postgres and PostGIS. Trying to write a function that that selects specific columns according to the given argument.
I'm using a WITH statement to create the result table before converting it to bytea to return.
The part I need help with is the $4 part. I tried it is demonstrated below and $4::text and both give me back the text value of the input and not the column value in the table if cols=name so I get back from the query name and not the actual names in the table. I also try data($4) and got type error.
The code is like this:
CREATE OR REPLACE FUNCTION select_by_txt(z integer,x integer,y integer, cols text)
RETURNS bytea
LANGUAGE 'plpgsql'
AS $BODY$
declare
res bytea;
begin
WITH bounds AS (
SELECT ST_TileEnvelope(z, x, y) AS geom
),
mvtgeom AS (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom, $4
FROM table1 t, bounds
WHERE ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
)
SELECT ST_AsMVT(mvtgeom, 'public.select_by_txt')
INTO res
FROM mvtgeom;
RETURN res;
end;
$BODY$;
Example for calling the function:
select_by_txt(10,32,33,"col1,col2")
The argument cols can be multiple column names from 1 and not limited from above. The names of the columns inside cols will be checked before calling the function that they are valid columns.
Passing multiple column names as concatenated string for dynamic execution urgently requires decontamination. I suggest a VARIADIC function parameter instead, with properly quoted identifiers (using quote_ident() in this case):
CREATE OR REPLACE FUNCTION select_by_txt(z int, x int, y int, VARIADIC cols text[] = NULL, OUT res text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format(
$$
SELECT ST_AsMVT(mvtgeom, 'public.select_by_txt')
FROM (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom%s
FROM table1 t
JOIN (SELECT ST_TileEnvelope($1, $2, $3)) AS bounds(geom)
ON ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
) mvtgeom
$$, (SELECT ', ' || string_agg(quote_ident (col), ', ') FROM unnest(cols) col)
)
INTO res
USING z, x, y;
END
$func$;
db<>fiddle here
The format specifier %I for format() deals with a single identifier. You have to put in more work for multiple identifiers, especially for a variable number of 0-n identifiers. This implementation quotes every single column name, and only add a , if any column names have been passed. So it works for every possible input, even no input at all. Note VARIADIC cols text[] = NULL as last input parameter with NULL as default value:
Optional argument in PL/pgSQL function
Related:
quote_ident() does not add quotes to column name "first"
Column names are case sensitive in this context!
Call for your example (important!):
SELECT select_by_txt(10,32,33,'col1', 'col2');
Alternative syntax:
SELECT select_by_txt(10,32,33, VARIADIC '{col1,col2}');
More revealing call, with a third column name and malicious (though futile) intent:
SELECT select_by_txt(10,32,33,'col1', 'col2', $$col3'); DROP TABLE table1;--$$);
About that odd third column name and SQL injection:
https://www.explainxkcd.com/wiki/index.php/Little_Bobby_Tables
About VAIRADIC parameters:
Return rows matching elements of input array in plpgsql function
Pass multiple values in single parameter
Using an OUT parameter for simplicity. That's totally optional. See:
Returning from a function with OUT parameter
What I would not do
If you really, really trust the input to be a properly formatted list of 1 or more valid column names at all times - and you asserted that ...
the names of the columns inside cols will be checked before calling the function that they are valid columns
You could simplify:
CREATE OR REPLACE FUNCTION select_by_txt(z int, x int, y int, cols text, OUT res text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format(
$$
SELECT ST_AsMVT(mvtgeom, 'public.select_by_txt')
FROM (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom, %s
FROM table1 t
JOIN (SELECT ST_TileEnvelope($1, $2, $3)) AS bounds(geom)
ON ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
) mvtgeom
$$, cols
)
INTO res
USING z, x, y;
END
$func$;
(How can you be so sure that the input will always be reliable?)
You would need to use a dynamic query:
CREATE OR REPLACE FUNCTION select_by_txt(z integer,x integer,y integer, cols text)
RETURNS bytea
LANGUAGE 'plpgsql'
AS $BODY$
declare
res bytea;
begin
EXECUTE format('
WITH bounds AS (
SELECT ST_TileEnvelope($1, $2, $3) AS geom
),
mvtgeom AS (
SELECT ST_AsMVTGeom(ST_Transform(t.geom, 3857), bounds.geom) AS geom, %I
FROM table1 t, bounds
WHERE ST_Intersects(t.geom, ST_Transform(bounds.geom, 4326))
)
SELECT ST_AsMVT(mvtgeom, ''public.select_by_txt'')
FROM mvtgeom', cols)
INTO res
USING z,x,y;
RETURN res;
end;
$BODY$;
An example what I am attempting to do,
SELECT count(*) as "count"
FROM (
SELECT overlay('db_xx.company_summary' placing 'US' from 4)
) as s
This returns count to be 1, I would expect it to count all rows in db_us.company_summary 1*10^6
I would expect this to result in a query to similar to this;
SELECT count(*) as "count"
FROM (db_us.company_summary')
I attempted the overlay function, to be similar to the above query. I am not sure if its possible to do this in SQL.
Normally in python you would do something like this;
"hello {}".format("world")
So I would like the inputted string to act as a SQL syntax command.
Plain SQL does not allow to parameterize identifiers (or anything but values). You need dynamic SQL. Examples:
Truncating all tables in a Postgres database
How to use variable as table name in plpgsql
How do I use variables in a postgresql function for loop query
Something like this:
Minimal test setup (should be in the question):
CREATE TABLE country_code (iso2 text PRIMARY KEY);
INSERT INTO country_code VALUES ('US'), ('GB'), ('AT');
CREATE SCHEMA db_us;
CREATE SCHEMA db_gb;
CREATE SCHEMA db_at;
CREATE TABLE db_us.company_summary (foo int);
CREATE TABLE db_gb.company_summary (foo int);
CREATE TABLE db_at.company_summary (foo int);
INSERT INTO db_us.company_summary VALUES (1), (2);
INSERT INTO db_gb.company_summary VALUES (1), (2), (3);
INSERT INTO db_at.company_summary VALUES (1), (2), (3), (4);
PL/pgSQL function with dynamic SQL:
CREATE OR REPLACE FUNCTION f_counts()
RETURNS SETOF bigint
LANGUAGE plpgsql AS
$func$
DECLARE
_lower_iso2 text;
_ct bigint;
BEGIN
FOR _lower_iso2 IN
SELECT lower(iso2) FROM country_code
LOOP
RETURN QUERY EXECUTE
format ('SELECT count(*) AS count FROM %I.company_summary'
, overlay('db_xx' PLACING _lower_iso2 FROM 4)
);
END LOOP;
END
$func$;
Call:
SELECT * FROM f_counts();
Result:
f_counts
---------
2
3
4
db<>fiddle here
Be aware that Postgres identifiers are case sensitive. See:
Are PostgreSQL column names case-sensitive?
I have a generic key/value table that I would like to preprocess in a function to filter by the key, and then name the value according to the key.
The table is like this where id points into another table (but really is don't care for the purposes of this question):
CREATE TABLE keyed_values (id INTEGER, key TEXT, value TEXT, PRIMARY KEY(id, key));
INSERT INTO keyed_values(id, key, value) VALUES
(1, 'x', 'Value for x for 1'),
(1, 'y', 'Value for y for 1'),
(2, 'x', 'Value for x for 2'),
(2, 'z', 'Value for z for 2');
and so forth. The key values can vary and not all IDs will use the same keys.
Specifically, what I'd like to do is to create a function that would produce rows matching a given key. Then it would return a row for each matching key with the value named with the key.
So the output of the function given the key name argument 'x' would be the same as if I executed this:
SELECT id, value x FROM keyed_values WHERE key = 'x';
id | x
----+-------------------
1 | Value for x for 1
2 | Value for x for 2
(2 rows)
Here's my stab, put together from looking at some other SO answers:
CREATE FUNCTION value_for(key TEXT) RETURNS SETOF RECORD AS $$
BEGIN
RETURN QUERY EXECUTE format('SELECT id, value %I FROM keyed_values WHERE key = %I', key, key);
END;
$$ LANGUAGE 'plpgsql';
The function is accepted. But when I execute it, I get this error:
SELECT * FROM value_for('x');
ERROR: a column definition list is required for functions returning "record"
LINE 1: SELECT * FROM value_for('x');
I get what it's telling me, but not how to fix that. How can I dynamically define the output column name?
The key used in the WHERE clause is not an identifier but a literal, so the function should looks like:
CREATE FUNCTION value_for(key TEXT) RETURNS SETOF RECORD AS $$
BEGIN
RETURN QUERY EXECUTE format('SELECT id, value %I FROM keyed_values WHERE key = %L', key, key);
END;
$$ LANGUAGE plpgsql;
From the documentation:
If the function has been defined as returning the record data type, then an alias or the key word AS must be present, followed by a column definition list in the form ( column_name data_type [, ... ]). The column definition list must match the actual number and types of columns returned by the function.
Use a column definition list:
SELECT * FROM value_for('x') AS (id int, x text);
Note that you do not need dynamic SQL nor plpgsql, as the column name is defined in the above list. The simple SQL function should be a bit faster:
CREATE OR REPLACE FUNCTION value_for(_key text)
RETURNS SETOF RECORD AS $$
SELECT id, value
FROM keyed_values
WHERE key = _key
$$ LANGUAGE SQL;
If you do not like being forced to append a column definition list to every function call, you can return a table from the function:
CREATE OR REPLACE FUNCTION value_for_2(_key text)
RETURNS TABLE(id int, x text) AS $$
SELECT id, value
FROM keyed_values
WHERE key = _key
$$ LANGUAGE SQL;
SELECT * FROM value_for_2('x');
id | x
----+-------------------
1 | Value for x for 1
2 | Value for x for 2
(2 rows)
However this does not solve the issue. There is no way to dynamically define a column name returned from a function. The names and types of returned columns must be specified before the query is executed.
I have record type variable new from trigger.
Can I get variables from this record, not text representation.
Example:
create table test(x int, y varchar);
insert into test values (1, 'test_var');
if i get new.*
i get this
(1, test_var)
Is there an option get record like this
(1, 'test_var')
Thanks.
Sorry for mistakes, english not my base language.
you should to use point notation recordvar.field. In PostgreSQL documentation is lot of examples.
create table test(x int, y varchar);
create or replace function trig_fx()
returns trigger as $$
begin
raise notice '<<%>> <<%>>', new.x, new.y;
return new;
end;
$$ language plpgsql;
create trigger trig before insert on test
for each row execute procedure trig_fx();
postgres=# insert into test values(1, 'test var');
NOTICE: <<1>> <<test var>>
INSERT 0 1
I have the following table called as test_type which contains two columns namely cola and colb.
Table: test_type
create table test_type
(
cola int,
colb varchar(50)
);
Now I want to create a type with same columns.
Type: type1
create type type1 as
(
cola int,
colb varchar(50)
);
Here I have created function in which I am passing type name type1 to insert the data to the
table test_type.
--Creating Function
create or replace function fun_test ( p_Type type1 )
returns void
as
$$
begin
insert into test_type(cola,colb)
select cola,colb from p_type
EXCEPT
select cola,colb from test_type;
end
$$
language plpgsql;
---Calling Function
SELECT fun_test(1,'Xyz');
Error Details:
ERROR: function fun_test(integer, unknown) does not exist
SQL state: 42883
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Character: 8
You need to "pack" the arguments together: (1,'xs'), so that postgres recognise them as single argument of type type1:
SELECT fun_test((1,'xs'));
For a better readability you can cast the argument to type1 (not really necessary):
SELECT fun_test((1,'xs')::type1);
If the purpose of the function is to to insert the values only if they are not already contained in the table, you could change your code so:
create or replace function fun_test ( p_Type type1 )
returns void AS $$
BEGIN
INSERT INTO test_type(cola,colb)
SELECT p_Type.cola,p_Type.colb
EXCEPT
SELECT cola,colb FROM test_type;
END;
$$ language plpgsql;
But this syntax is my opinion not good readable. This statement looks better:
...
BEGIN
PERFORM 0 FROM test_type WHERE (cola, colb) = p_Type;
IF NOT FOUND THEN
INSERT INTO test_type(cola,colb) VALUES (p_Type.cola,p_Type.colb);
END IF;
END;
...