I'm struggling with a stored procedure which heavily uses dynamic queries. Among others I need to store maximum value of an existing column into a variable.
Postgres documents state "if you want to use dynamically determined table or column names, you must insert them into the command string textually". Based on that I've come up with following statement:
EXECUTE 'SELECT MAX(' || pkColumn::regclass || ') FROM ' ||
tableName::regclass INTO maxValue;
Table name seems to be OK, column name triggers error.
What am I doing wrong ?
Pavel
EXECUTE 'SELECT MAX(' || pkColumn ||'::regclass) FROM ' || ...
::regclass is a cast done inside query. You can also skip it, or put " - which in PG works the same. So please try one of:
EXECUTE 'SELECT MAX(' || pkColumn || ') FROM ' || ...
or
EXECUTE 'SELECT MAX("' || pkColumn || '") FROM ' || ...
All tree should work. If not - just let me know. In that case it is my fault, postgresql simply works.
There is no reason to cast parameters as they are just identifiers. For better control and readability use the function format(), e.g.:
declare
pkcolumn text = 'my_column';
tablename text = 'my_table';
...
execute format('select max(%I) from %I', pkcolumn, tablename)
into maxvalue;
Related
PostgreSQL 11.4, compiled by Visual C++ build 1914, 64-bit
Reviewed dozens of articles in Stackoverflow, no real match. The need: pass a comma separated string (id values), and use that list with the "ANY" postgresql clause.
Code
return query execute
'select aa.id, aa.course_id, aa.comments, aa.curr_cont_t_id, aa.date_done, ' ||
'aa.unit_id, aa.time_seq, aa.week_num, bb.module_id, bb.expected_hrs, ' ||
'bb.title unit_title, cc.module_name, cc.tally_hours, cc.time_of_day, ' ||
'bb.file_upload_expected, aa.app_files_id, xx.facility_id ' ||
'from course_content aa ' ||
'left outer join units bb on aa.unit_id = bb.id ' ||
'left outer join module_categories cc on bb.module_id = cc.id ' ||
'left outer join courses xx on aa.course_id = xx.id ' ||
'where xx.facility_id = any(''{' || $1 || '}'') '
using p_facilities;
I have checked p_facilities to ensure it is not empty or null. I have even specifically set p_facilities to a value inside the function like this:
p_facilities text = '3';
The returned error is consistently: 'query string argument of EXECUTE is null (SQL State 22004)'
The problem is that you are not referring to the using parameter anywhere in your query. Instead, you are concatenating $1 directly into your query, and this $1 refers to the first argument of the pl/pgsql function you are in (and apparently is NULL).
To use parameters in dynamically executed sql and pass them through using, you need to hardcode the text $1 into the query string:
EXECUTE 'SELECT … WHERE xx.facility_id = any($1)' USING some_array;
To interpolate a string into the query, you don't need any using clause, just refer to the string directly:
EXECUTE 'SELECT … WHERE xx.facility_id = any(''{' || p_facilities || '}'')';
However, notice that you don't need (and shouldn't use) dynamic sql here at all. You're constructing a value, not sql structure. You can just refer to that directly in a normal query:
SELECT … WHERE xx.facility_id = any( ('{' || p_facilities || '}')::int[] );
-- or better
SELECT … WHERE xx.facility_id = any( string_to_array(p_facilities, ',')::int[] );
I am trying to run the below query in a stored procedure and its not working.
We tried to print the query using NOTICE and we saw E gets appended to the regex and thats the reason the query doesnt show any output.
Not working
select order,version from mytable
where substring(version from quote_literal('[0-9]+\.[0-9]{1}'))
IN ('6.2') and order= 'ABC';
But the same query if i run from pgadmin query tool, it works fine.
Working
select order,version from mytable
where substring(version from '[0-9]+\.[0-9]{1}')
IN ('6.2') and order= 'ABC';
My requirement is to form the reqex dynamically in the stored procedure. Please guide on how to achieve this.
Below is the line of code in my stored procedure,
sql_query = sql_query || ' AND substring(version from ' || quote_literal( '[0-9]+\.[0-9]{1}' ) || ') IN (' || quote_literal(compatibleVersions) || ')';
raise notice 'Value: %', sql_query;
EXECUTE sql_query INTO query_result ;
and from notice i am getting the below output,
AND substring(version from E'[0-9]+\\.[0-9]{1}') IN ('{6.2}')
My requirement is to make this regex work.
I narrowed down to this query,
working
select substring(version from '[0-9]+\.[0-9]{1}') from mytable ;
not working
select substring(version from quote_literal('[0-9]+\.[0-9]{1}')) from mytable ;
Now i think its easy to fix it. You can try at your end also running this above queries.
Since your problem is not really the extended string literal syntax using E, but the string representation of the array in the IN list, your PL/pgSQL should look somewhat like this:
sql_query = sql_query ||
' AND substring(version from ' || quote_literal( '[0-9]+\.[0-9]{1}' ) ||
') IN (' || (SELECT string_agg(quote_literal(x), ', ')
FROM unnest(compatibleVersions
) AS x(x)) || ')';
quote_literal should be used in situations where u want to dynamically construct queries. In such situation quote_literal will be replaced by E in the final constructed query.
right way to use
select * from config_support_module where substring(firmware from '[0-9]+\.[0-9]{1}') IN ('6.2');
select * from config_support_module where substring(firmware from E'[0-9]+\.[0-9]{1}') IN ('6.2') ;
wrong usage of quote_literal in static queries
select * from config_support_module where substring(firmware from quote_literal('[0-9]+\.[0-9]{1}')) IN ('6.2') ;
This doesnt give you any errors/output.
quote_literal usage in dynamic queries
sql_query = sql_query || ' AND substring(version from ' || quote_literal( '[0-9]+\.[0-9]{1}' ) || ') ... .. ...
I have a PL/pgSQL function that takes table name as dynamic parameter. As I am updating an existing query to take table name as dynamic parameter, this is what I have as my function:
DECLARE rec RECORD;
BEGIN
EXECUTE 'insert into stat_300_8_0(ts, target, data)
select distinct timestamp-(timestamp%3600) as wide_row_ts,
target, array[]::real[] as data
from ' || temp_table_name || ' as temp
where class_id=8
and subclass_id=0
and not exists (select ts from stat_300_8_0
where ts=temp.timestamp-(temp.timestamp%3600)
and target=temp.target)';
FOR rec IN EXECUTE 'SELECT DISTINCT timestamp AS ts
FROM ' || temp_table_name ||
' WHERE class_id=8'
LOOP
EXECUTE 'update stat_300_8_0 as disk_table
set data[new_data.start_idx:new_data.end_idx] = array[data_0,data_1]
from (select timestamp-(timestamp%3600) as wide_row_ts,
(timestamp%3600)/300 * 2 + 1 as start_idx,
((timestamp%3600 / 300) + 1) * 2 as end_idx,
target, data_0, data_1
from ' || temp_table_name ||
' where class_id=8 and subclass_id=0
and timestamp=rec.ts) as new_data
where disk_table.ts=new_data.wide_row_ts
and disk_table.target=new_data.target';
END LOOP;
END;
However, when this function is executed I get an error saying
ERROR: missing FROM-clause entry for table "rec"
However, rec is declared in the first line of the above code. I am not able to figure what is wrong with my queries. Any help is appreciated.
Supplemental to Eelke's answer:
Assuming temp_table_name is an argument, you really, really want to run it through quote_ident() because otherwise someone could create a table with a name that could inject sql into your function.
Instead of the change suggested there, you are better off using EXECUTE...USING since that gives you parameterization regarding values (and hence protection against SQL injection). You would change rec.ts to $1 and then add to the end USING ts.rec (outside the quoted execute string). This gives you a parameterized statement inside your execute which is safer. However parameters cannot include table names, so it doesn't spare you from the first point above.
Is it possible to write a query that automatically selects all database names and column names from dbc.Columns table in Teradata, and searches a particular set of values?
Set of values:
WHERE abc in (1,2,3)
Selecting dbc.columns:
SELECT DatabaseName, TableName FROM dbc.COLUMNS
WHERE ColumnName LIKE '%abc%'
How can I combine this and make a query that will return only those combinations of DatabaseName and TableName where ColumnName has specific subset of values?
UPDATE:
This query finds all database - column combinations:
SELECT TRIM(BOTH FROM a.DatabaseName) || '.' || TRIM( BOTH FROM a.TableName)
FROM dbc.COLUMNS AS a
WHERE ColumnName LIKE '%abc%'
is it possible to define some variables or sthg. else?
You need to write Dynamic SQL statements like
SELECT
'SELECT ''' || DatabaseName || '.' || TableName || '.' || ColumnName || ''''
' WHERE EXISTS (SELECT * FROM ' || DatabaseName || '.' || TableName ||
' WHERE ' || ColumnName || ' IN (1,2,3));'
FROM dbc.ColumnsVX
WHERE ColumnName LIKE '%abc%';
Running the resulting queries will return one result set with zero or one row for each table.
To get a single result set you need to write a Stored Procedure with a cursor on the dbc.columnsVX result (adding an INSERT INTO temptable), EXECTE IMMEDIATE each row. Finally return the rows of the temptable.
Unless you're an experienced SQL programmer your DBA will not grant you the right to create SPs.
But why do you actually need this kind of info? Looking for a needle in a haystack?
I have a number of tables that use the Postgres "Partitioning" feature. I want to define a common BEFORE INSERT OF ROW trigger on each table that will 1) dynamically create the partition should the insert occur against the parent table and 2) re-execute the insert against the partition.
Something like:
CREATE OR REPLACE FUNCTION partition_insert_redirect( )
RETURNS trigger AS $BODY$
BEGIN
... create the new partition and set up the redirect Rules ...
/* Redo the INSERT dynamically. The new RULE will redirect it to the child table */
EXECUTE 'INSERT INTO ' || quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(TG_TABLE_NAME) ||
' SELECT NEW.*'
END
But the "NEW" record isn't visible inside the EXECUTE SQL. How can I make this work as simply as possible?
As an alternative, can I iterate over the fields in the NEW record somehow?
I've thought of using a temp-table:
EXECUTE 'CREATE TEMPORARY TABLE new_row (LIKE ' ||
quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(TG_TABLE_NAME) ||
') ON COMMIT DROP';
INSERT INTO new_row SELECT NEW.*;
EXECUTE 'INSERT INTO ' || quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(TG_TABLE_NAME) ||
' SELECT * FROM new_row';
DROP TABLE new_row;
But this also doesn't work because of the cached reference to a temp-table: Why do I get "relation with OID ##### does not exist" errors when accessing temporary tables in PL/PgSQL functions?
I'm using Postgres 8.2 and I can't change to any other version.
EDIT:
As #alvherre pointed out, this can probably be done in Postgres 8.4 with the EXECUTE ... USING syntax. See an example at http://wiki.postgresql.org/wiki/PL/pgSQL_Dynamic_Triggers
You can use EXECUTE USING to pass NEW to it. Your example would be
EXECUTE 'INSERT INTO ' || TG_RELID || '::regclass SELECT $1' USING NEW;
(Note that I use TG_RELID casted to regclass instead of fiddling with TG_TABLE_SCHEMA and TABLE_NAME because it is easier to use, if nonstandard. But then, plpgsql is nonstandard anyway.)
Yes, you can use EXECUTE ... USING in 8.4. For example:
EXECUTE 'INSERT INTO ' || table_name || ' SELECT $1.*' USING NEW;
In lower versions (I've only tested in 8.3), you can use:
EXECUTE 'INSERT INTO ' || table_name ||
' SELECT (' || quote_literal(NEW) || '::' || TG_RELID::regclass || ').*';
I've managed to get this to work by dynamically compiling a function that accepts the NEW row as a parameter:
EXECUTE 'create or replace function partition_insert(r ' || TG_TABLE_NAME || ') RETURNS void AS $FUNC$' ||
'BEGIN ' ||
'insert into ' || TG_TABLE_NAME || ' SELECT r.*; ' ||
'END $FUNC$ LANGUAGE plpgsql VOLATILE';
PERFORM partition_insert(NEW);
As Postgres functions are polymorphic, this will generate a different function for each table that uses this trigger.
Despite being an ugly kludge, this seems to do the job.
Although it looks like I could define each polymorphic variation up front when I build the system, because of caching, I must recompile the function whenever I create or drop a child table so that the function uses the latest insert RULE.
EDIT: Additional wrinkles
There's a little gotcha with this technique: If this EXECUTE/PERFORM action is rolled-back on the first attempt due to another error (for example, in my case a CHECK constraint failure) then the function containing this code seems to cache a reference to the rolled-back partition_insert() function it created using the EXECUTE and subsequent calls fail due to a cached object not being found.
I resolved this by pre-creating stub versions of the function for each required table-type parameter when I define the database.