I have a table test, which have columns listed as x0,x1,x2,x3
I have an UDA which takes in two columns as arguments and does some computation
I am trying to call the UDA from my nzplsq
when I directly call UDA like:
create table newtable as select ncorrFactor(x0,x2) from test;
It works
but when I try to do this:
p varchar;
p := X || 0 || '';
create table newtable as select ncorrFactor(p,x2) from test;
It gives me this error:
ERROR: pg_atoi: error in "x0": can't parse "x0"
What do I need to fix?
Assuming the first snippet is a store procedure written in NZPLSQL for p to be treated as 'X0' you need to build your query dynamically and use "execute immediate" on that query.
eg:
declare
query varchar;
begin
query:='create table newtable as select ncorrFactor('|| 0 ||',x2) from test';
execute immediate query;
end;
Related
I have a function that uses RECORD to temporarily store the data. I can use it - it's fine. My problem is that I can't hardcode columns I need to get from the RECORD. I must do it dynamically. Something line:
DECLARE
r1 RECORD;
r2 RECORD;
BEGIN
for r1 in Select column_name
from columns_to_process
where process_now = True
loop
for r2 in Select *
from my_data_table
where whatever
loop
-----------------------------
here I must call column by its name that is unknown at design time
-----------------------------
... do something with
r2.(r1.column_name)
end loop;
end loop;
END;
Does anyone know how to do it?
best regards
M
There is no need to select the all the qualifying rows and compute the total in a loop. Actually when working with SQL try to drop the word loop for your vocabulary; instead just use sum(column_name) in the select. The issue here is that you do not know what column to sum when the query is written, and all structural components(table names, columns names, operators, etc) must be known before submitting. You cannot use a variable for a structural component - in this case a column name. To do that you must use dynamic sql - i.e. SQL statement built by the process. The following accomplishes that: See example here.
create or replace function sum_something(
the_something text -- column name
, for_id my_table.id%type -- my_table.id
)
returns numeric
language plpgsql
as $$
declare
k_query_base constant text :=
$STMT$ Select sum(%I) from my_table where id = %s; $STMT$;
l_query text;
l_sum numeric;
begin
l_query = format(k_query_base, the_something, for_id);
raise notice E'Rumming Statememt:\n %',l_query; -- for prod raise Log
execute l_query into l_sum;
return l_sum;
end;
$$;
Well, after some time I figured out that I could use temporary table instead of RECORD. Doing so gives me all advantages of using dynamic queries so I can call any column by its name.
DECLARE
_my_var bigint;
BEGIN
create temporary table _my_temp_table as
Select _any, _column, _you, _need
from _my_table
where whatever = something;
execute 'Select ' || _any || ' from _my_temp_table' into _my_var;
... do whatever
END;
However I still believe that there should be a way to call records field by it's name.
Given an arbitrary PostgreSQL query, e.g. SELECT * FROM (...) AS T, how can I get resultset column names and types WITHOUT actually executing the query using psycopg2 Python3 library?
I saw JDBC solution using getMetaData(), but I cannot figure out how to get that same information in psycopg2.
PreparedStatement pstmt = con.prepareStatement("select ... ");
ResultSetMetaData meta = pstmt.getMetaData();
for (int i=1; i <= meta.getColumnCount(); i++) { ... }
As far as I understand, that's not possible without executing(running cur.execute())
But, If you want a Postgres solution using a function that can be used by Psycopg2 as a query, you may use this solution. As you were expecting, this will not execute your query, it simply creates a temporary View which allows us to query it's metadata using the catalog information_schema.columns
CREATE OR REPLACE function define_query(query text)
RETURNS TABLE( column_name text,data_type text)
LANGUAGE plpgsql AS
$$
DECLARE
v_view_n TEXT := 'temp_view$';
BEGIN
EXECUTE format( 'CREATE OR REPLACE TEMP VIEW %I AS %s', v_view_n,query);
RETURN QUERY select i.column_name::text, i.data_type ::text
from information_schema.columns i where i.table_name = v_view_n;
END $$;
Once you've got this function, you can get the definition of any query by simply calling this function and not executing it.
knayak=# select * from define_query('select 1::int as a,''TWO''::text as b');
column_name | data_type
-------------+-----------
a | integer
b | text
(2 rows)
I think you have to execute something at least.
If you don't want any rows returned, you can query like select * from XXX where false. This query will return types of columns to client with 0 rows.
I tried to run this script with the parameters in place but it keeps throwing syntax errors.Is there anything wrong with the syntax.Also what is the correct way to call this function.I require an output that tells me the update statement was executed successfully. I tried "select function_name(schema_name.TABLE_NAME);".Let me add that I am a beginner and am open to any kind of feedback. will also provide more details if necessary.
CREATE OR REPLACE FUNCTION function_name (TABLE_NAME IN character varying)
RETURNS text AS $SQLQuery$
DECLARE SQLQuery text;
BEGIN
SQLQuery =
' UPDATE '|| TABLE_NAME || ' SET column1=''0''
WHERE column1 is null;' ||
' UPDATE '|| TABLE_NAME || ' SET column2='value'
WHERE column2=''different value'';' ||
--multiple update statements later
Execute SQLQuery;
Return SQLQuery;
END;
$SQLQuery$
LANGUAGE plpgsql;
Update:
this is the error i am getting when i call the test function
ERROR: missing FROM-clause entry for table "schema_name"
LINE 2: select test_function(schema_name.TABLE_NAME);
^
********** Error **********
ERROR: missing FROM-clause entry for table "schema_name"
SQL state: 42P01
it is reading the function as a table?
I have also received syntax errors saying
EXECUTE column does not exist or that the function does not exist
even though i just declared it.
To use single quotes inside a siting constant, you must escape them by doubling them.
Instead of
' SET column1='0''
you'll have to write
' SET column1=''0'''
smth like:
CREATE OR REPLACE FUNCTION function_name (schema_name text,TABLE_NAME IN character varying)
RETURNS text AS $SQLQuery$
DECLARE
c int;
rtn text :='';
BEGIN
execute format(' UPDATE %I.%I SET column1=''0'' WHERE column1 is null;',schema_name,TABLE_NAME);
get diagnostics c = row_count;
raise info '%', 'affected: '||c;
rtn = rtn + 'affected: '||c||chr(10);
--repeat above construct for multiple update statement
return rtn;
END;
$SQLQuery$
LANGUAGE plpgsql;
and advises. I'm novice like you, but I learned to follow several rules, that help me:
with dynamic sql use format to avoid sql injection
don't overcomplicate things (eg the functionality you are looking for is inside UPDATE statement already - check the output. If you want to check the resulting row use, UPDATE ... RETURNING * construct.
practice is good, but reading concepts is precious.
In your POST select function_name(schema_name.TABLE_NAME); would not work, because you use schema_name.TABLE_NAME without quotes, but even if you put them, your function is vulnerable - what will happen if you run select function_name(';drop sometable;--');?..
You are trying to pass SQL Identifier, but your function takes string as parameter instead. You should change it to something like:
select test_function('schema_name.TABLE_NAME');
You can try that function below as base for whatever you are trying to do.
/* You need to split table and schema name
or you might get errors when using names that aren't lower case.
This: 'public.TEST1' would be translated to: "public.TEST1"
that is different table from public.test1
*/
CREATE OR REPLACE FUNCTION multi_update_stuff(schema_name varchar, table_name varchar)
/* We will return set of multiple columns. One possible method is to return table.
First column shows executed query, second if it returned no errors (true)
*/
RETURNS TABLE(SQLQuery text, result boolean)
AS $body$
DECLARE
/* Declare arroy of queries that we will iterate and execute later.
We use format() to build query from template and fill it with values.
%1$I can be described as "put first value here and treat it as object identifier"
%3$L can be described as "put third value here and treat it as SQL literal"
*/
SQLQueries text[] := array[
/* First query */
format('UPDATE %1$I.%2$I SET column1 = %3$L WHERE column1 is null;',
schema_name, table_name, '0'),
/* Second query */
format('UPDATE %1$I.%2$I SET column2 = %3$L WHERE column2 = %4$L;',
schema_name, table_name, 'value', 'different value'),
/* Third query, to see error free result */
'SELECT 1'];
BEGIN
/* Iterate our array */
FOREACH SQLQuery IN ARRAY SQLQueries
LOOP
/* Start transaction block */
BEGIN
EXECUTE SQLQuery;
result := true;
/* Catch error if any */
EXCEPTION
WHEN others THEN
result := false;
END;
/* Return row with whatever is assigned to variables listed in RETURNS.
In this case SQLQuery was already assigned by FOREACH.
*/
RETURN NEXT;
END LOOP;
END;
$body$
LANGUAGE plpgsql;
SELECT * FROM multi_update_stuff('schema_name', 'TABLE_NAME')
I have created the following stored procedure, which basically receives a name of table, and a prefix. The function then finds all columns that share this prefix and returns as an output a 'select' query command ('myoneliner').
as follows:
CREATE OR REPLACE FUNCTION mytext (mytable text, myprefix text)
RETURNS text AS $myoneliner$
declare
myoneliner text;
BEGIN
SELECT 'SELECT ' || substr(cols,2,length(cols)-2) ||' FROM '||mytable
INTO myoneliner
FROM (
SELECT array(
SELECT DISTINCT quote_ident(column_name::text)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
order by quote_ident
)::text cols
) sub;
RETURN myoneliner;
END;
$myoneliner$ LANGUAGE plpgsql;
Call:
select mytext('dkj_p_k27ac','enri');
As a result of running this stored procedure and the 'select' that is following it, I get the following output at the Data Output window (all within one cell, named "mytext text"):
'SELECT enrich_d_dkj_p_k27ac,enrich_lr_dkj_p_k27ac,enrich_r_dkj_p_k27ac
FROM dkj_p_k27ac'
I would like to basically be able to take the output command line that I received as an output and execute it. In other words, I would like to be able and execute the output of my stored procedure.
How can I do so?
I tried the following:
CREATE OR REPLACE FUNCTION mytext (mytable text, myprefix text)
RETURNS SETOF RECORD AS $$
declare
smalltext text;
myoneliner text;
BEGIN
SELECT 'SELECT ' || substr(cols,2,length(cols)-2) ||' FROM '||mytable
INTO myoneliner
FROM (
SELECT array(
SELECT DISTINCT quote_ident(column_name::text)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
order by quote_ident
)::text cols
) sub;
smalltext=lower(myoneliner);
raise notice '%','my additional text '||smalltext;
RETURN QUERY EXECUTE smalltext;
END;
$$ LANGUAGE plpgsql;
Call function:
SELECT * from mytext('dkj_p_k27ac','enri');
But I'm getting the following error message, could you please advise what should I change in order for it to execute?:
ERROR: a column definition list is required for functions returning "record"
LINE 26: SELECT * from mytext('dkj_p_k27ac','enri');
********** Error **********
ERROR: a column definition list is required for functions returning "record"
SQL state: 42601
Character: 728
Your first problem was solved by using dynamic SQL with EXECUTE like Craig advised.
But the rabbit hole goes deeper:
CREATE OR REPLACE FUNCTION myresult(mytable text, myprefix text)
RETURNS SETOF RECORD AS
$func$
DECLARE
smalltext text;
myoneliner text;
BEGIN
SELECT INTO myoneliner
'SELECT '
|| string_agg(quote_ident(column_name::text), ',' ORDER BY column_name)
|| ' FROM ' || quote_ident(mytable)
FROM information_schema.columns
WHERE table_name = mytable
AND column_name LIKE myprefix||'%'
AND table_schema = 'public'; -- schema name; might be another param
smalltext := lower(myoneliner); -- nonsense
RAISE NOTICE 'My additional text: %', myoneliner;
RETURN QUERY EXECUTE myoneliner;
END
$func$ LANGUAGE plpgsql;
Major points
Don't cast the whole statement to lower case. Column names might be double-quoted with upper case letters, which are case-sensitive in this case (no pun intended).
You don't need DISTINCT in the query on information_schema.columns. Column names are unique per table.
You do need to specify the schema, though (or use another way to single out one schema), or you might be mixing column names from multiple tables of the same name in multiple schemas, resulting in nonsense.
You must sanitize all identifiers in dynamic code - including table names: quote_ident(mytable). Be aware that your text parameter to the function is case sensitive! The query on information_schema.columns requires that, too.
I untangled your whole construct to build the list of column names with string_agg() instead of the array constructor. Related answer:
Update multiple columns that start with a specific string
The assignment operator in plpgsql is :=.
Simplified syntax of RAISE NOTICE.
Core problem impossible to solve
All of this still doesn't solve your main problem: SQL demands a definition of the columns to be returned. You can circumvent this by returning anonymous records like you tried. But that's just postponing the inevitable. Now you have to provide a column definition list at call time, just like your error message tells you. But you just don't know which columns are going to be returned. Catch 22.
Your call would work like this:
SELECT *
FROM myresult('dkj_p_k27ac','enri') AS f (
enrich_d_dkj_p_k27ac text -- replace with actual column types
, enrich_lr_dkj_p_k27ac text
, enrich_r_dkj_p_k27ac text);
But you don't know number, names (optional) and data types of returned columns, not at creation time of the function and not even at call time. It's impossible to do exactly that in a single call. You need two separate queries to the database.
You could return all columns of any given table dynamically with a function using polymorphic types, because there is a well defined type for the whole table. Last chapter of this related answer:
Refactor a PL/pgSQL function to return the output of various SELECT queries
I'm trying to write a function in PL/PgSQL that have to work with a table it receives as a parameter.
I use EXECUTE..INTO..USING statements within the function definition to build dynamic queries (it's the only way I know to do this) but ... I encountered a problem with RECORD data types.
Let's consider the follow (extremely simplified) example.
-- A table with some values.
DROP TABLE IF EXISTS table1;
CREATE TABLE table1 (
code INT,
descr TEXT
);
INSERT INTO table1 VALUES ('1','a');
INSERT INTO table1 VALUES ('2','b');
-- The function code.
DROP FUNCTION IF EXISTS foo (TEXT);
CREATE FUNCTION foo (tbl_name TEXT) RETURNS VOID AS $$
DECLARE
r RECORD;
d TEXT;
BEGIN
FOR r IN
EXECUTE 'SELECT * FROM ' || tbl_name
LOOP
--SELECT r.descr INTO d; --IT WORK
EXECUTE 'SELECT ($1)' || '.descr' INTO d USING r; --IT DOES NOT WORK
RAISE NOTICE '%', d;
END LOOP;
END;
$$ LANGUAGE plpgsql STRICT;
-- Call foo function on table1
SELECT foo('table1');
It output the following error:
ERROR: could not identify column "descr" in record data type
although the syntax I used seems valid to me. I can't use the static select (commented in the example) because I want to dinamically refer the columns names.
So..someone know what's wrong with the above code?
It's true. You cannot to use type record outside PL/pgSQL space.
RECORD value is valid only in plpgsql.
you can do
EXECUTE 'SELECT $1.descr' INTO d USING r::text::xx;
$1 should be inside the || ,like || $1 || and give spaces properly then it will work.
BEGIN
EXECUTE ' delete from ' || quote_ident($1) || ' where condition ';
END;