I am trying to run the below query in a stored procedure and its not working.
We tried to print the query using NOTICE and we saw E gets appended to the regex and thats the reason the query doesnt show any output.
Not working
select order,version from mytable
where substring(version from quote_literal('[0-9]+\.[0-9]{1}'))
IN ('6.2') and order= 'ABC';
But the same query if i run from pgadmin query tool, it works fine.
Working
select order,version from mytable
where substring(version from '[0-9]+\.[0-9]{1}')
IN ('6.2') and order= 'ABC';
My requirement is to form the reqex dynamically in the stored procedure. Please guide on how to achieve this.
Below is the line of code in my stored procedure,
sql_query = sql_query || ' AND substring(version from ' || quote_literal( '[0-9]+\.[0-9]{1}' ) || ') IN (' || quote_literal(compatibleVersions) || ')';
raise notice 'Value: %', sql_query;
EXECUTE sql_query INTO query_result ;
and from notice i am getting the below output,
AND substring(version from E'[0-9]+\\.[0-9]{1}') IN ('{6.2}')
My requirement is to make this regex work.
I narrowed down to this query,
working
select substring(version from '[0-9]+\.[0-9]{1}') from mytable ;
not working
select substring(version from quote_literal('[0-9]+\.[0-9]{1}')) from mytable ;
Now i think its easy to fix it. You can try at your end also running this above queries.
Since your problem is not really the extended string literal syntax using E, but the string representation of the array in the IN list, your PL/pgSQL should look somewhat like this:
sql_query = sql_query ||
' AND substring(version from ' || quote_literal( '[0-9]+\.[0-9]{1}' ) ||
') IN (' || (SELECT string_agg(quote_literal(x), ', ')
FROM unnest(compatibleVersions
) AS x(x)) || ')';
quote_literal should be used in situations where u want to dynamically construct queries. In such situation quote_literal will be replaced by E in the final constructed query.
right way to use
select * from config_support_module where substring(firmware from '[0-9]+\.[0-9]{1}') IN ('6.2');
select * from config_support_module where substring(firmware from E'[0-9]+\.[0-9]{1}') IN ('6.2') ;
wrong usage of quote_literal in static queries
select * from config_support_module where substring(firmware from quote_literal('[0-9]+\.[0-9]{1}')) IN ('6.2') ;
This doesnt give you any errors/output.
quote_literal usage in dynamic queries
sql_query = sql_query || ' AND substring(version from ' || quote_literal( '[0-9]+\.[0-9]{1}' ) || ') ... .. ...
Related
PostgreSQL 11.4, compiled by Visual C++ build 1914, 64-bit
Reviewed dozens of articles in Stackoverflow, no real match. The need: pass a comma separated string (id values), and use that list with the "ANY" postgresql clause.
Code
return query execute
'select aa.id, aa.course_id, aa.comments, aa.curr_cont_t_id, aa.date_done, ' ||
'aa.unit_id, aa.time_seq, aa.week_num, bb.module_id, bb.expected_hrs, ' ||
'bb.title unit_title, cc.module_name, cc.tally_hours, cc.time_of_day, ' ||
'bb.file_upload_expected, aa.app_files_id, xx.facility_id ' ||
'from course_content aa ' ||
'left outer join units bb on aa.unit_id = bb.id ' ||
'left outer join module_categories cc on bb.module_id = cc.id ' ||
'left outer join courses xx on aa.course_id = xx.id ' ||
'where xx.facility_id = any(''{' || $1 || '}'') '
using p_facilities;
I have checked p_facilities to ensure it is not empty or null. I have even specifically set p_facilities to a value inside the function like this:
p_facilities text = '3';
The returned error is consistently: 'query string argument of EXECUTE is null (SQL State 22004)'
The problem is that you are not referring to the using parameter anywhere in your query. Instead, you are concatenating $1 directly into your query, and this $1 refers to the first argument of the pl/pgsql function you are in (and apparently is NULL).
To use parameters in dynamically executed sql and pass them through using, you need to hardcode the text $1 into the query string:
EXECUTE 'SELECT … WHERE xx.facility_id = any($1)' USING some_array;
To interpolate a string into the query, you don't need any using clause, just refer to the string directly:
EXECUTE 'SELECT … WHERE xx.facility_id = any(''{' || p_facilities || '}'')';
However, notice that you don't need (and shouldn't use) dynamic sql here at all. You're constructing a value, not sql structure. You can just refer to that directly in a normal query:
SELECT … WHERE xx.facility_id = any( ('{' || p_facilities || '}')::int[] );
-- or better
SELECT … WHERE xx.facility_id = any( string_to_array(p_facilities, ',')::int[] );
I'm struggling with a stored procedure which heavily uses dynamic queries. Among others I need to store maximum value of an existing column into a variable.
Postgres documents state "if you want to use dynamically determined table or column names, you must insert them into the command string textually". Based on that I've come up with following statement:
EXECUTE 'SELECT MAX(' || pkColumn::regclass || ') FROM ' ||
tableName::regclass INTO maxValue;
Table name seems to be OK, column name triggers error.
What am I doing wrong ?
Pavel
EXECUTE 'SELECT MAX(' || pkColumn ||'::regclass) FROM ' || ...
::regclass is a cast done inside query. You can also skip it, or put " - which in PG works the same. So please try one of:
EXECUTE 'SELECT MAX(' || pkColumn || ') FROM ' || ...
or
EXECUTE 'SELECT MAX("' || pkColumn || '") FROM ' || ...
All tree should work. If not - just let me know. In that case it is my fault, postgresql simply works.
There is no reason to cast parameters as they are just identifiers. For better control and readability use the function format(), e.g.:
declare
pkcolumn text = 'my_column';
tablename text = 'my_table';
...
execute format('select max(%I) from %I', pkcolumn, tablename)
into maxvalue;
I've got a plpgsql function that needs to prepare data from 3 tables based on user input, and export the data using COPY TO. The data are road accidents, so the 3 tables are accident, casualty and vehicle, each accident links to zero or more records in the vehicle and casualty tables via an accidentid column that exists in all three tables. severity and local_authorities are input parameters (both text []).
sql_query = 'SELECT COUNT(*) FROM accident WHERE severity = ANY(' || quote_literal(severity)
|| ') AND local_auth = ANY (' || quote_literal(local_authorities) || ')';
EXECUTE sql_query INTO result_count;
IF result_count > 0 THEN
-- replace Select Count(*) With Select *
sql_query = Overlay(sql_query placing '*' from 8 for 8);
-- copy the accident data first
EXECUTE 'COPY (' || sql_query || ') TO ' || quote_literal(file_path || file_name_a) ||
' CSV';
This first bit will get the relevant accidents, so I'm now looking for the most efficient way to use the accidentid's from the first query to download the related vehicle and casualty data.
I thought I'd be able to use a WITH block like this:
-- replace * with accidentid
sql_query = Overlay(sql_query placing 'accidentid' from 8 for 1);
WITH acc_ids AS (sql_query)
EXECUTE 'COPY (SELECT * FROM vehicle WHERE accidentid IN (SELECT accidentid FROM
acc_ids)) TO ' || out_path_and_vfilename || ' CSV';
EXECUTE 'COPY (SELECT * FROM casualty WHERE accidentid IN (SELECT accidentid FROM
acc_ids)) TO ' || out_path_and_cfilename || ' CSV';
but get an error:
ERROR: syntax error at or near "$1"
LINE 1: WITH acc_ids AS ( $1 ) EXECUTE 'COPY (SELECT * FROM accident....
I have tried the above in a non-dynamic test case e.g.
WITH acc_ids AS (
SELECT accidentid FROM accident
WHERE severity = ANY ('{3,2}')
AND local_auth = ANY ('{E09000001,E09000002}')
)
SELECT * FROM vehicle
WHERE accidentid IN (
SELECT accidentid FROM acc_ids);
which works. Unfortunately the server is still running Postgres 8.4 so I can't use format() for the time being.
Perhaps this isn't possible with a WITH block, but I hope it at least illustrates what I'm trying to achieve.
Edit/Update
The main goal is to get the relevant data from the 3 tables in 3 separate csv files, ideally without having to run the selection on the accident table 3 times
If you want to run a query (part) that is stored in a string variable, you need a dynamic query like
EXECUTE 'WITH acc_ids AS (' || sql_query || ')'
'SELECT ... ';
Either the whole query is a string executed by EXECUTE, or the whole query is static SQL. You cannot mix them.
Do you need a CTE? If you can express the query as a join, the optimizer has more options.
This does what I need to do without CTE but I can't see this being the most efficient way of solving this since I have to perform the same query on the accident table 3 times:
sql_query = sql_query || which_tab || ' WHERE severity = ANY ('||
quote_literal(severity) ||') AND ' || date_start || ' AND ' ||
date_end || ' AND local_auth = ANY (' ||
quote_literal(local_authorities) || ')';
-- replace * with COUNT(*)
sql_query = Overlay(sql_query placing 'COUNT(*)' from 8 for 1);
EXECUTE sql_query INTO result_count;
IF result_count > 0 THEN
-- replace COUNT(*) with *
sql_query = Overlay(sql_query placing '*' from 8 for 8);
-- copy the accident data first
EXECUTE 'COPY (' || sql_query || ') TO ' || quote_literal(file_path ||
file_name_a) || ' CSV';
sql_query = Overlay(sql_query placing 'accidentid' from 8 for 1);
-- vehicles
EXECUTE 'COPY (SELECT * FROM vehicle WHERE accidentid IN (
SELECT accidentid FROM accident
WHERE severity = ANY (' || quote_literal(severity) || ')
AND local_auth = ANY (' || quote_literal(local_authorities) ||')))
TO ' || quote_literal(file_path || file_name_v) || ' CSV';
-- casualties
EXECUTE 'COPY (SELECT * FROM casualty WHERE accidentid IN (
SELECT accidentid FROM accident
WHERE severity = ANY (' || quote_literal(severity) || ')
AND local_auth = ANY (' || quote_literal(local_authorities) ||')))
TO ' || quote_literal(file_path || file_name_c) || ' CSV';
END IF;
So my problem is simple. I have a schema prod with many tables, and another one log with the exact same tables and structure (primary keys change that's it).
When I do UPDATE or DELETE in the schema prod, I want to record old data in the log schema.
I have the following function called after a update or delete:
CREATE FUNCTION prod.log_data() RETURNS trigger
LANGUAGE plpgsql AS $$
DECLARE
v RECORD;
column_names text;
value_names text;
BEGIN
-- get column names of current table and store the list in a text var
column_names = '';
value_names = '';
FOR v IN SELECT * FROM information_schema.columns WHERE table_name = quote_ident(TG_TABLE_NAME) AND table_schema = quote_ident(TG_TABLE_SCHEMA) LOOP
column_names = column_names || ',' || v.column_name;
value_names = value_names || ',$1.' || v.column_name;
END LOOP;
-- remove first char ','
column_names = substring( column_names FROM 2);
value_names = substring( value_names FROM 2);
-- execute the insert into log schema
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' ( ' || column_names || ' ) VALUES ( ' || value_names || ' )' USING OLD;
RETURN NULL; -- no need to return, it is executed after update
END;$$;
The annoying part is that I have to get column names from information_schema for each row.
I would rather use this:
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' SELECT ' || OLD;
But some values can be NULL so this will execute:
INSERT INTO log.user SELECT 2,,,"2015-10-28 13:52:44.785947"
instead of
INSERT INTO log.user SELECT 2,NULL,NULL,"2015-10-28 13:52:44.785947"
Any idea to convert ",," to ",NULL,"?
Thanks
-Quentin
First of all I must say that in my opinion using PostgreSQL system tables (like information_schema) is the proper way for such a usecase. Especially that you must write it once: you create the function prod.log_data() and your done. Moreover it may be dangerous to use OLD in that context (just like *) as always because of not specified elements order.
But,
to answer your exact question the only way I know is to do some operations on OLD. Just observe that you cast OLD to text by doing concatenation ... ' SELECT ' || OLD. The default casting create that ugly double-commas. So, next you can play with that text. In the end I propose:
DECLARE
tmp TEXT
...
BEGIN
...
/*to make OLD -> text like (2,,3,4,,)*/
SELECT '' || OLD INTO tmp; /*step 1*/
/*take care of commas at the begining and end: '(,' ',)'*/
tmp := replace(replace(tmp, '(,', '(NULL,'), ',)', ',NULL)'); /*step 2*/
/* replace rest of commas to commas with NULL between them */
SELECT array_to_string(string_to_array(tmp, ',', ''), ',', 'NULL') INTO tmp; /*step 3*/
/* Now we can do EXECUTE*/
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' SELECT ' || tmp;
Of course you can do steps 1-3 in one big step
SELECT array_to_string(string_to_array(replace(replace('' || NEW, '(,', '(NULL,'), ',)', ',NULL)'), ',', ''), ',', 'NULL') INTO tmp;
In my opinion this approach isn't any better from using information_schema, but it's your call.
When I run the following command from a function I defined, I get the error "EXECUTE of SELECT ... INTO is not implemented". Does this mean the specific command is not allowed (i.e. "SELECT ...INTO")? Or does it just mean I'm doing something wrong? The actual code causing the error is below. I apologize if the answer is already out here, however I looked and could not find this specific error. Thanks in advance... For whatever it's worth I'm running 8.4.7
vCommand = 'select ' || stmt.column_name || ' as id ' ||
', count(*) as nCount
INTO tmpResults
from ' || stmt.table_name || '
WHERE ' || stmt.column_name || ' IN (select distinct primary_id from anyTable
WHERE primary_id = ' || stmt.column_name || ')
group by ' || stmt.column_name || ';';
EXECUTE vCommand;
INTO is ambiguous in this use case and then is prohibited there.
You can use a CREATE TABLE AS SELECT instead.
CREATE OR REPLACE FUNCTION public.f1(tablename character varying)
RETURNS integer
LANGUAGE plpgsql
AS $function$
begin
execute 'create temp table xx on commit drop as select * from '
|| quote_ident(tablename);
return (select count(*) from xx);
end;
$function$
postgres=# select f1('omega');
f1
────
2
(1 row)