I have a table (named VGI_table) that contains a column (named match_tabl) which contains the names of other tables in the same database along with object_ids for those tables. I am trying to create a plpgsql function that loops through each row in the VGI_table and performs a query to retrieve an object from another table as shown below.
The function takes 4 parameters (all varchar), the first two are names of columns in the VGI_table, the third is the name of the VGI_table and the last parameter is the output.
vgi_match_id_col, vgi_match_table_col, vgi_table, output_table
The code for the function is shown below, ro is used to hold the first table query, match_row holds the output of the queried external table. Distance is an output created using the PostGIS st_distance function.
DECLARE
ro record;
match_row record;
distance float;
BEGIN
for ro in EXECUTE 'select gid, geom, '||vgi_match_id_col||' as match_id, '||vgi_match_table_col||' as match_table from '||vgi_table
LOOP
--raise notice '(%)', 'select geom from public.'||ro.match_table||' where gid = '||ro.match_id;
execute 'select geom from public.'||ro.match_table||' where gid = '||ro.match_id into match_row;
distance := st_distance(ro.geom, st_transform(match_row.geom,st_srid(ro.geom)));
EXECUTE 'INSERT INTO '||output_table||' VALUES('||ro.gid||', '||distance||')';
END LOOP;
The table being queried has no null values in the match_tabl column or the object_id colum. The code identifies ro.match_table and ro.match_id as null values when attempting to perform the EXECUTE statement. I even used the RAISE NOTICE function with the same string that is used in the EXECUTE statement and the correct query is returned. If I hard code the execute string with a predefined table_name and object id the script works fine. The link below is similar but I don't think it addresses my question. Thanks for the help.
Similar Question
Well, clearly something you're concatenating is null.
Use the format function instead, that way you'll get more useful info.
format('select geom from public.%I ....', ro.match_table);
Use EXECUTE ... USING ... to insert literals.
e.g.
EXECUTE format('INSERT INTO %I VALUES($1, $2)', output_table) USING (ro.gid, distance);
In postgres , If anything null is being passed in Dynamic DML , we are bound to get this issue."query string argument of EXECUTE is null"
You can use below sample steps for insert and update .
CREATE OR REPLACE FUNCTION samplefunc(
col1param character varying,
col2param character varying,
col3param character varying,
col4param character varying,
col5param character varying,
col6param character varying,
col7param character varying,
col8param character varying
RETURNS boolean AS
$BODY$
declare
begin
EXECUTE format( 'insert into '||tablename||' (id, col1, col2, col3, col4, col5)values($1,$2,$3,$4,$5)') USING col1param ,col2param,col3param,col4param,col5param;
EXECUTE format('update '||tablename||' set col1 =$1,col2 = $2,col3=$3,col4=$4,col5=$5
where col6=$6 and col7=$7 and col8=$8 ') using col1param,col2param,,col3param,col4param,col5param,col6param,col7param,col8param;
end
Related
I am currently working a stored procedure capable of detecting continuity on a specific set of entries..
The specific set of entries is extracted from a sql query
The function takes in two input parameter, first being the table that should be investigated, and the other being the list of ids which should be evaluated.
For every Id I need to investigate every row provided by the select statement.
DROP FUNCTION IF EXISTS GapAndOverlapDetection(table_name text, entity_ids bigint[]);
create or replace function GapAndOverlapDetection ( table_name text, enteity_ids bigint[] )
returns table ( entity_id bigint, valid tsrange, causes_overlap boolean, causes_gap boolean)
as $$
declare
x bigint;
var_r record;
begin
FOREACH x in array $2
loop
EXECUTE format('select entity_id, valid from' ||table_name|| '
where entity_id = '||x||'
and registration #> now()::timestamp
order by valid ASC') INTO result;
for var_r in result
loop
end loop;
end loop ;
end
$$ language plpgsql;
select * from GapAndOverlapDetection('temp_country_registration', '{1,2,3,4}')
I currently get an error in the for statement saying
ERROR: syntax error at or near "$1"
LINE 12: for var_r in select entity_id, valid from $1
You can iterate over the result of the dynamic query directly:
create or replace function gapandoverlapdetection ( table_name text, entity_ids bigint[])
returns table (entity_id bigint, valid tsrange, causes_overlap boolean, causes_gap boolean)
as $$
declare
var_r record;
begin
for var_r in EXECUTE format('select entity_id, valid
from %I
where entity_id = any($1)
and registration > now()::timestamp
order by valid ASC', table_name)
using entity_ids
loop
... do something with var_r
-- return a row for the result
-- this does not end the function
-- it just appends this row to the result
return query
select entity_id, true, false;
end loop;
end
$$ language plpgsql;
The %I injects an identifier into a string and the $1 inside the dynamic SQL is then populated through passing the argument with the using keyword
Firstly, decide whether you want to pass the table's name or oid. If you want to identify the table by name, then the parameter should be of text type and not regclass.
Secondly, if you want the table name to change between executions then you need to execute the SQL statement dynamically with the EXECUTE statement.
I have a table with multiple columns in PostgreSQL. I try to make a function returning a table with a few default columns and a variable column. The column name should be passed as function parameter. Example:
SELECT * FROM get_gas('temperature');
This is my code right now:
CREATE OR REPLACE FUNCTION get_gas(gas text)
RETURNS TABLE (id INTEGER, node_id INTEGER,
gas_name DOUBLE PRECISION,
measurement_timestamp timestamp without time zone )
AS
$$
BEGIN
SELECT measurements_lora.id, measurements_lora.node_id, gas, measurements_lora.measurement_timestamp
AS measure
FROM public.measurements_lora;
END
$$ LANGUAGE plpgsql;
When passing, for example, 'temperature' as column name (gas), I want to get a table with these columns from the function call.
id - node_id - temperature - measurement_timestamp
How would I achieve this?
You can use EXECUTE statement.
CREATE OR REPLACE FUNCTION get_gas(gas text) RETURNS TABLE (f1 INTEGER, f2 INTEGER, f3 DOUBLE PRECISION, f4 timestamp without time zone ) AS
$$
DECLARE
sql_to_execute TEXT;
BEGIN
SELECT 'SELECT measurements_lora.id,
measurements_lora.node_id, '
|| gas ||',
measurements_lora.measurement_timestamp AS measure
FROM public.measurements_lora '
INTO sql_to_execute;
RETURN QUERY EXECUTE sql_to_execute;
END
$$ LANGUAGE plpgsql;
This will create a variable sql_to_execute with your field and. QUERY EXECUTE will execute your interpreted query.
EDIT 1: Look at the another answer the concernings about security issues.
If you really need dynamic SQL in a PL/pgSQL function (which you don't), be sure to defend against SQL injection! Like:
CREATE OR REPLACE FUNCTION get_gas(gas text)
RETURNS TABLE (id integer
, node_id integer
, gas_name double precision
, measurement_timestamp timestamp)
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE format(
'SELECT m.id, m.node_id, m.%I, m.measurement_timestamp
FROM public.measurements_lora m'
, gas
);
END
$func$;
The format specifier %I in format() double-quotes identifiers where needed,
See:
SQL injection in Postgres functions vs prepared queries
Insert text with single quotes in PostgreSQL
I have two table Documents and RegCard (name uses a name of current_user)
Table Documents
CREATE TABLE public.document_dimauser
(
documentid uuid NOT NULL,
documentname character varying(100),
author character varying(100),
contents bytea,
CONSTRAINT document_dimauser_pkey PRIMARY KEY (documentid)
)
Table RegCard
CREATE TABLE public.regcard_dimauser
(
regcardid uuid NOT NULL,
documentid uuid,
documentintronumber character varying(100),
documentexternnumber character varying(100),
dateintro date,
dateextern date,
CONSTRAINT regcard_dimauser_pkey PRIMARY KEY (regcardid),
CONSTRAINT regcard_dimauser_documentid_fkey FOREIGN KEY (documentid)
REFERENCES public.document_dimauser (documentid) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
This two tables are connected one row 'documentid'
Also, i have a function, which inserting data in this two tables. Here i try selecting last rec of documentid from first table and transer him in second table
CREATE OR REPLACE FUNCTION public.addrecuserdocuments(
documentname character varying,
contents bytea,
documentintronumber character varying)
RETURNS void AS
$BODY$
DECLARE
comm VARCHAR;
nameuser VARCHAR;
currdate DATE;
iddoc uuid;
BEGIN
SELECT CURRENT_USER INTO STRICT nameuser;
SELECT CURRENT_DATE INTO STRICT currdate;
comm:='INSERT INTO Document_'||nameuser||' VALUES ('||quote_literal(uuid_generate_v4())||', '||quote_literal(documentname)||','||quote_literal(nameuser)||','||quote_literal(contents)||');
SELECT documentid INTO STRICT '||quote_literal(iddoc)||' FROM Document_'||nameuser||' order by documentid DESC LIMIT 1;
INSERT INTO Regcard_'||nameuser||' (regcardid, documentid, documentintronumber, dateintro) VALUES ('||quote_literal(uuid_generate_v4())||' , '||quote_literal(iddoc)||', '||quote_literal(documentintronumber)||', '||quote_literal(currdate)||');';
EXECUTE comm;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
But i have a error
ERROR: query string argument of EXECUTE is null
SQL-состояние: 22004
Контекст: PL/pgSQL function addrecuserdocuments(character varying,bytea,character varying) line 13 at EXECUTE
Please, tell me, what i do wrong?
you concat with || operator - if any value is null the whole string is null...
you can instead do smth like :
comm := format('INSERT INTO %I VALUES(%L)',concat(Document_',nameuser),uuid_generate_v4());
and so on...
Don't concatenate input values if you don't need that, use parameters in the string and use format() to build it. You also don't need variables to store the results of built-in functions and you don't need to "select" the UUID you generated inside your code.
Putting all that together you can simplify your code to:
BEGIN
iddoc := uuid_generate_v4();
comm : = format('INSERT INTO Document_%s VALUES ($1, $2, $3, $4)', current_user);
execute comm
using iddoc, documentname, current_user, contents;
comm := format('insert into regcard_%s (regcardid, documentid, documentintronumber, dateintro) values ($1, $2, $3, $4)', current_user);
execute comm
using uuid_generate_v4(), iddoc, documentintronumber, current_date;
END;
This not only makes the code much more readable it is also a way to prevent SQL injection through that function.
Unrelated, but: I find a design that uses a table with the user name appended highly questionable - especially if that user name is also stored inside the table. If you don't do that you can avoid all that dynamic SQL.
I have a fields table to store column information for other tables:
CREATE TABLE public.fields (
schema_name varchar(100),
table_name varchar(100),
column_text varchar(100),
column_name varchar(100),
column_type varchar(100) default 'varchar(100)',
column_visible boolean
);
And I'd like to create a function to fetch data for a specific table.
Just tried sth like this:
create or replace function public.get_table(schema_name text,
table_name text,
active boolean default true)
returns setof record as $$
declare
entity_name text default schema_name || '.' || table_name;
r record;
begin
for r in EXECUTE 'select * from ' || entity_name loop
return next r;
end loop;
return;
end
$$
language plpgsql;
With this function I have to specify columns when I call it!
select * from public.get_table('public', 'users') as dept(id int, uname text);
I want to pass schema_name and table_name as parameters to function and get record list, according to column_visible field in public.fields table.
Solution for the simple case
As explained in the referenced answers below, you can use registered (row) types, and thus implicitly declare the return type of a polymorphic function:
CREATE OR REPLACE FUNCTION public.get_table(_tbl_type anyelement)
RETURNS SETOF anyelement AS
$func$
BEGIN
RETURN QUERY EXECUTE format('TABLE %s', pg_typeof(_tbl_type));
END
$func$ LANGUAGE plpgsql;
Call:
SELECT * FROM public.get_table(NULL::public.users); -- note the syntax!
Returns the complete table (with all user columns).
Wait! How?
Detailed explanation in this related answer, chapter
"Various complete table types":
Refactor a PL/pgSQL function to return the output of various SELECT queries
TABLE foo is just short for SELECT * FROM foo:
Is there a shortcut for SELECT * FROM?
2 steps for completely dynamic return type
But what you are trying to do is strictly impossible in a single SQL command.
I want to pass schema_name and table_name as parameters to function and get record list, according to column_visible field in
public.fields table.
There is no direct way to return an arbitrary selection of columns (return type not known at call time) from a function - or any SQL command. SQL demands to know number, names and types of resulting columns at call time. More in the 2nd chapter of this related answer:
How do I generate a pivoted CROSS JOIN where the resulting table definition is unknown?
There are various workarounds. You could wrap the result in one of the standard document types (json, jsonb, hstore, xml).
Or you generate the query with one function call and execute the result with the next:
CREATE OR REPLACE FUNCTION public.generate_get_table(_schema_name text, _table_name text)
RETURNS text AS
$func$
SELECT format('SELECT %s FROM %I.%I'
, string_agg(quote_ident(column_name), ', ')
, schema_name
, table_name)
FROM fields
WHERE column_visible
AND schema_name = _schema_name
AND table_name = _table_name
GROUP BY schema_name, table_name
ORDER BY schema_name, table_name;
$func$ LANGUAGE sql;
Call:
SELECT public.generate_get_table('public', 'users');
This create a query of the form:
SELECT usr_id, usr FROM public.users;
Execute it in the 2nd step. (You might want to add column numbers and order columns.)
Or append \gexec in psql to execute the return value immediately. See:
How to force evaluation of subquery before joining / pushing down to foreign server
Be sure to defend against SQL injection:
INSERT with dynamic table name in trigger function
Define table and column names as arguments in a plpgsql function?
Asides
varchar(100) does not make much sense for identifiers, which are limited to 63 characters in standard Postgres:
Maximum characters in labels (table names, columns etc)
If you understand how the object identifier type regclass works, you might replace schema and table name with a singe regclass column.
I think you just need another query to get the list of columns you want.
Maybe something like (this is untested):
create or replace function public.get_table(_schema_name text, _table_name text, active boolean default true) returns setof record as $$
declare
entity_name text default schema_name || '.' || table_name;
r record;
columns varchar;
begin
-- Get the list of columns
SELECT string_agg(column_name, ', ')
INTO columns
FROM public.fields
WHERE fields.schema_name = _schema_name
AND fields.table_name = _table_name
AND fields.column_visible = TRUE;
-- Return rows from the specified table
RETURN QUERY EXECUTE 'select ' || columns || ' from ' || entity_name;
RETURN;
end
$$
language plpgsql;
Keep in mind that column/table references may need to be surrounded by double quotes if they have certain characters in them.
I have created the following stored procedure, which basically receives a name of table, and a prefix. The function then finds all columns that share this prefix and returns as an output a 'select' query command ('myoneliner').
as follows:
CREATE OR REPLACE FUNCTION mytext (mytable text, myprefix text)
RETURNS text AS $myoneliner$
declare
myoneliner text;
BEGIN
SELECT 'SELECT ' || substr(cols,2,length(cols)-2) ||' FROM '||mytable
INTO myoneliner
FROM (
SELECT array(
SELECT DISTINCT quote_ident(column_name::text)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
order by quote_ident
)::text cols
) sub;
RETURN myoneliner;
END;
$myoneliner$ LANGUAGE plpgsql;
Call:
select mytext('dkj_p_k27ac','enri');
As a result of running this stored procedure and the 'select' that is following it, I get the following output at the Data Output window (all within one cell, named "mytext text"):
'SELECT enrich_d_dkj_p_k27ac,enrich_lr_dkj_p_k27ac,enrich_r_dkj_p_k27ac
FROM dkj_p_k27ac'
I would like to basically be able to take the output command line that I received as an output and execute it. In other words, I would like to be able and execute the output of my stored procedure.
How can I do so?
I tried the following:
CREATE OR REPLACE FUNCTION mytext (mytable text, myprefix text)
RETURNS SETOF RECORD AS $$
declare
smalltext text;
myoneliner text;
BEGIN
SELECT 'SELECT ' || substr(cols,2,length(cols)-2) ||' FROM '||mytable
INTO myoneliner
FROM (
SELECT array(
SELECT DISTINCT quote_ident(column_name::text)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
order by quote_ident
)::text cols
) sub;
smalltext=lower(myoneliner);
raise notice '%','my additional text '||smalltext;
RETURN QUERY EXECUTE smalltext;
END;
$$ LANGUAGE plpgsql;
Call function:
SELECT * from mytext('dkj_p_k27ac','enri');
But I'm getting the following error message, could you please advise what should I change in order for it to execute?:
ERROR: a column definition list is required for functions returning "record"
LINE 26: SELECT * from mytext('dkj_p_k27ac','enri');
********** Error **********
ERROR: a column definition list is required for functions returning "record"
SQL state: 42601
Character: 728
Your first problem was solved by using dynamic SQL with EXECUTE like Craig advised.
But the rabbit hole goes deeper:
CREATE OR REPLACE FUNCTION myresult(mytable text, myprefix text)
RETURNS SETOF RECORD AS
$func$
DECLARE
smalltext text;
myoneliner text;
BEGIN
SELECT INTO myoneliner
'SELECT '
|| string_agg(quote_ident(column_name::text), ',' ORDER BY column_name)
|| ' FROM ' || quote_ident(mytable)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
AND table_schema = 'public'; -- schema name; might be another param
smalltext := lower(myoneliner); -- nonsense
RAISE NOTICE 'My additional text: %', myoneliner;
RETURN QUERY EXECUTE myoneliner;
END
$func$ LANGUAGE plpgsql;
Major points
Don't cast the whole statement to lower case. Column names might be double-quoted with upper case letters, which are case-sensitive in this case (no pun intended).
You don't need DISTINCT in the query on information_schema.columns. Column names are unique per table.
You do need to specify the schema, though (or use another way to single out one schema), or you might be mixing column names from multiple tables of the same name in multiple schemas, resulting in nonsense.
You must sanitize all identifiers in dynamic code - including table names: quote_ident(mytable). Be aware that your text parameter to the function is case sensitive! The query on information_schema.columns requires that, too.
I untangled your whole construct to build the list of column names with string_agg() instead of the array constructor. Related answer:
Update multiple columns that start with a specific string
The assignment operator in plpgsql is :=.
Simplified syntax of RAISE NOTICE.
Core problem impossible to solve
All of this still doesn't solve your main problem: SQL demands a definition of the columns to be returned. You can circumvent this by returning anonymous records like you tried. But that's just postponing the inevitable. Now you have to provide a column definition list at call time, just like your error message tells you. But you just don't know which columns are going to be returned. Catch 22.
Your call would work like this:
SELECT *
FROM myresult('dkj_p_k27ac','enri') AS f (
enrich_d_dkj_p_k27ac text -- replace with actual column types
, enrich_lr_dkj_p_k27ac text
, enrich_r_dkj_p_k27ac text);
But you don't know number, names (optional) and data types of returned columns, not at creation time of the function and not even at call time. It's impossible to do exactly that in a single call. You need two separate queries to the database.
You could return all columns of any given table dynamically with a function using polymorphic types, because there is a well defined type for the whole table. Last chapter of this related answer:
Refactor a PL/pgSQL function to return the output of various SELECT queries