CREATE OR REPLACE FUNCTION my_function2()
RETURNS VOID
LANGUAGE plpgsql AS
$func$
BEGIN
PREPARE "my-statement7" (text, text) AS
INSERT INTO "table" (key, a_id)
VALUES ($1, $2);
EXECUTE format('EXECUTE "my-statement7" (%1, %2)', 'value1', 'value2');
END;
$func$;
SELECT my_function2();
However this will return error.
I have run successfully when I do execute prepare statement in the command line but when I do it in a function it will prompt error. Somebody said the execute statement will be ambiguous in execute statement or function, so it need put execute into execute format. However, it doesn't work too.
I want to execute the prepare statement for insert because I got a lot of data got to insert in same format, but the data may need to insert to different table (also data are different), so I cannot just do insert multi or COPY for it. I think the prepare statement is the solution for me, but it cannot work in a function. This will bother me if I want to do a lot of "execute prepare statement" in a function.
In PL/SQL, queries are already prepared and cached, so you don't have to.
As each expression and SQL command is first executed in the function, the PL/pgSQL interpreter parses and analyzes the command to create a prepared statement, using the SPI manager's SPI_prepare function. Subsequent visits to that expression or command reuse the prepared statement.
Here's how you'd write it if you tried. Note the use of _ instead of - in the name to avoid quoting issues; no need to format.
CREATE OR REPLACE FUNCTION my_function2()
RETURNS VOID
LANGUAGE plpgsql AS
$func$
BEGIN
PREPARE my_statement7 (text, text) AS
INSERT INTO example (key, a_id)
VALUES ($1, $2);
EXECUTE my_statement7( 'value1', 'value2');
END;
$func$;
However, the first time it will fail (for reasons I don't entirely understand): ERROR: function my_statement7(unknown, unknown) does not exist. The second time it is called it will also fail: ERROR: prepared statement "my_statement7" already exists.
Related
I am replacing a legacy function get_data in our database which takes some entity_id and returns a refcursor.
I am writing a new function get_data_new which is using different data sources, but the outputs are expected to be the same as get_data for the same input.
I'd like to verify this with pgtap, and am doing so as follows within a test (with _expected and _actual being the names of the returned cursors):
SELECT schema.get_data('_expected', 123);
SELECT schema.get_data_new('_actual', 123);
SELECT results_eq(
'FETCH ALL FROM _actual',
'FETCH ALL FROM _expected',
'get_data_new should return identical results to the legacy version'
);
This works as expected for other functions, but the query in get_data happens to return some json columns meaning that comparison expectedly fails with ERROR: could not identify an equality operator for type json.
I'd rather leave the legacy function untouched so refactoring to jsonb isn't possible. I'm imagining a workaround to be transforming the data before comparison, hypothetically with something like SELECT entity_id, json_column::jsonb FROM (FETCH ALL FROM _actual), but this specific attempt obviously isn't valid.
What would be a suggested approach here? Write a helper function to insert data from the cursors into a couple of temporary tables? I'm hoping there's a cleaner solution I haven't discovered.
Using postgres 11.14, pgtap11
Have solved it by creating a function to loop over the cursor and return the results as a table. Unfortunately this isn't a generic solution - it only works for the cursors with specific data.
In this specific case, json_column can be implicitly converted to type jsonb so this is all that is needed. However, we can now SELECT * FROM cursor_to_table('_actual') meaning we can do whatever transformations we require on the result.
CREATE OR REPLACE FUNCTION cursor_to_table(_cursor refcursor)
RETURNS TABLE (entity_id bigint, json_column jsonb)
AS $func$
BEGIN
LOOP
FETCH _cursor INTO entity_id, json_column
EXIT WHEN NOT FOUND;
RETURN NEXT;
END LOOP;
RETURN;
END
$func$ LANGUAGE plpgsql;
SELECT results_eq(
'SELECT * FROM cursor_to_table(''_actual'')',
'SELECT * FROM cursor_to_table(''_expected'')',
'get_data_new should return identical results to the legacy version'
);
I am trying to get the return from a QUERY EXEUTE in a plpgsql function to be able to check how many rows were affected from a dynamic update query. My use case is adding an event (with a custom payload) to a separate table on insert or update to a dynamically set table. Because my event has a custom payload, I have not been able to use a database trigger (e.g. trigger before insert). As a simplified example, assume I have this table:
CREATE TABLE users (user_id text primary key, name text)
Here is my simplified events table:
CREATE TABLE events(event_id text primary key, payload json)
Here is my simplified function:
CREATE OR REPLACE FUNCTION my_function(_rowtype anyelement, q text, payload jsonb)
RETURNS SETOF anyelement AS
$func$
DECLARE
event_id text;
BEGIN
SELECT jsonb_object_field_text (payload, 'id')::text INTO STRICT event_id;
execute format('insert into event(event_id, payload) values ($1, $2)') using event_id, payload;
RETURN QUERY EXECUTE format('%s', q);
END
$func$ LANGUAGE plpgsql;
The goal is to have this work exactly the same as if someone had created these in a transaction. In pseucode for insert:
BEGIN
insert into events(id, payload) values($1, $2)
insert into users(columns) values(<any values>)
COMMIT
and similarly for update:
BEGIN
insert into events(id, payload) values($1, $2)
result, error := query(`update users set name = 'hello' where id = 'Not Exists Thus No Rows Modified'`);
if result.rowsAffected() == 0 {
ROLLBACK
}
COMMIT
The function my_function almost works except for one edge case: when an update actually doesn't affect any rows.
For example, this works:
select * from my_function(NULL::users,
'insert into users(id,name) values('u1', ''a2'') returning *',
payload => '{"id": "e1", "custom": "s1", "field": "2019-10-12T07:20:50.52Z"}')
As expected, after this is done both a row in the users table and the events table is created.
What fails is the following:
select * from my_function(NULL::users,
'update users set name = ''hello'' where user_id = ''NotExists'' returning *',
payload => '{"id": "e2", "custom": "s3", "field": "2019-10-12T07:20:50.52Z"}')
Here, a row is created in the events table (my goal is that it should not be created).
I know this approach is not elegant, and I know this is vulnerable to SQL injection. I'd love suggestions on better ways to solve this (including scrapping what we're doing now). But to answer the question directly, I'm looking to store the result of QUERY EXECUTE, check if any rows were affected, and raise an error so that there is never a case where a row in the events table is created when there is not real corresponding change in the users table. Users table is just an example, in general, it could be any dynamically set table.
A RETURN QUERY doesn't need to go to the end of the function, it only says: "the results of this query are part of the resulting set".
So you can use the RETURN QUERY, ask for FOUND and act accordingly. Here is your function modified for working this way:
CREATE OR REPLACE FUNCTION public.my_function(_rowtype anyelement, q text, payload jsonb)
RETURNS SETOF anyelement
LANGUAGE plpgsql
AS $function$
DECLARE
event_id text;
BEGIN
SELECT jsonb_object_field_text (payload, 'id')::text INTO STRICT event_id;
RETURN QUERY EXECUTE format('%s', q);
IF FOUND THEN
execute format('insert into events(event_id, payload) values ($1, $2)') using event_id, payload;
END IF;
RETURN;
END
$function$
PD: Maybe you can also solve your problem with triggers FOR EACH STATEMENT using the transition tables OLD and NEW (which are available since v10, https://www.postgresql.org/docs/10/sql-createtrigger.html)
CREATE OR REPLACE FUNCTION my_function(_rowtype anyelement, q text, payload jsonb)
RETURNS SETOF anyelement
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE q;
IF NOT FOUND THEN
RETURN; -- nothing happened yet, we can exit silently.
-- Or you WANT an error for this case. Then do this instead:
-- RAISE EXCEPTION 'Query passed in parameter "q" did not affect any rows. Doing nothing!';
END IF;
INSERT INTO event(event_id, payload)
VALUES (payload->>'id', payload);
END
$func$;
As has been commented, RETURN QUERY does not return from the function. The manual:
RETURN NEXT and RETURN QUERY do not actually return from the
function — they simply append zero or more rows to the function's
result set. Execution then continues with the next statement in the
PL/pgSQL function. As successive RETURN NEXT or RETURN QUERY
commands are executed, the result set is built up. A final RETURN,
which should have no argument, causes control to exit the function (or
you can just let control reach the end of the function).
There's a code example for your case exactly at the bottom of that chapter in the manual. From me, actually. Originating here:
FUNCTION syntax error
It was suggested to use GET DIAGNOSTICS instead of the simpler FOUND. It's true that EXECUTE does not set the state of FOUND. But RETURN QUERY does. So keep using the simpler FOUND. Related:
Dynamic SQL (EXECUTE) as condition for IF statement
You have format() in your original twice. And while that's typically very useful for dynamic SQL, it's useless in your case. EXECUTE format('%s', q) is exactly the same as just EXECUTE q, with added cost. Both are open doors for SQL injection when passing user input.
While there is a good chance that the transaction might be rolled back, start with the critical step, and do the rest later. Avoid wasting the work. So I moved executing q to the top. Assuming it does not depend on the "payload" row, now inserted later.
Also, INSERT INTO events can be plain SQL. Nothing dynamic there. No need for format() or EXECUTE.
Finally, assuming your jsonb_object_field_text (payload, 'id')::text is just a fancy way of saying payload->>'id'. No need for an additional variable and another SELECT INTO.
Warning against SQL injection
Converting user input (parameter q in the example) to code to execute dynamically is the most direct SQL injection vulnerability of all. I wouldn't want to be caught in my underwear doing that.
I have a stored procedure that performs inserts and updates in the tables. The need to create it was to try to centralize all the scan functions before inserting or updating records. Today the need arose to return the value of the field ID of the table so that my application can locate the registry and perform other stored procedures.
Stored procedure
SET TERM ^ ;
CREATE OR ALTER procedure sp_insupd (
iaction varchar(3),
iusuario varchar(20),
iip varchar(15),
imodulo varchar(30),
ifieldsvalues varchar(2000),
iwhere varchar(1000),
idesclogs varchar(200))
returns (
oid integer)
as
declare variable vdesc varchar(10000);
begin
if (iaction = 'ins') then
begin
vdesc = idesclogs;
/*** the error is on the line below ***/
execute statement 'insert into '||:imodulo||' '||:ifieldsvalues||' returning ID into '||:oid||';';
end else
if (iaction = 'upd') then
begin
execute statement 'select '||:idesclogs||' from '||:imodulo||' where '||:iwhere into :vdesc;
execute statement 'execute procedure SP_CREATE_AUDIT('''||:imodulo||''');';
execute statement 'update '||:imodulo||' set '||:ifieldsvalues||' where '||:iwhere||';';
end
insert into LOGS(USUARIO, IP, MODULO, TIPO, DESCRICAO) values (
:iusuario, :iip, :imodulo, (case :iaction when 'ins' then 1 when 'upd' then 2 end), :vdesc);
end^
SET TERM ; ^
The error in the above line is occurring due to syntax error. The procedure is compiled normally, that is, the error does not happen in the compilation, since the line in question is executed through the "execute statement". When there was no need to return the value of the ID field, the procedure worked normally with the line like this:
...
execute statement 'insert into '||:imodulo||' '||:ifieldsvalues||';';
...
What would be the correct way for the value of the ID field to be stored in the OID variable?
What is REAL VALUE in ifieldsvalues ?
you can not have BOTH
'insert into '||:imodulo||' '||:ifieldsvalues
'update '||:imodulo||' set '||:ifieldsvalues
because methods to specify column names and column values in INSERT and UPDATE statements is fundamentally different!!! You either would have broken update-stmt or broken insert-stmt!
The error in the above line is occurring due to syntax error
This is not enough. Show the real error text, all of it.
It includes the actual command you generate and it seems you had generated it really wrong way.
all the scan functions before inserting or updating records
Move those functions out of the SQL server and into your application server.
Then you would not have to make insert/update in that "strings splicing" way, which is VERY fragile and "SQL injection" friendly. You stepped into the road to hell here.
the error does not happen in the compilation
Exactly. And that is only for starters. You are removing all the safety checks that should had helped you in applications development.
http://searchsoftwarequality.techtarget.com/definition/3-tier-application
https://en.wikipedia.org/wiki/Multitier_architecture#Three-tier_architecture
http://bobby-tables.com
On modern Firebird versions EXECUTE STATEMENT command can have the same INTO clause as PSQL SELECT command.
https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-psql-coding.html#fblangref25-psql-execstmt
Use http://translate.ru to read http://www.firebirdsql.su/doku.php?id=execute_statement
Or just see SQL examples there. Notice, however, those examples all use SELECT dynamic command, not INSERT. So I am not sure it would work that way.
This works in Firebird 2.5 (but not in Firebird 2.1) PSQL blocks.
execute statement 'insert into Z(payload) values(2) returning id' into :i;
To run it from IBExpert/FlameRobin/iSQL interactive shell add that obvious boilerplate:
execute block returns (i integer) as
begin
execute statement 'insert into Z(payload) values(2) returning id' into :i;
suspend;
end
What are the difference between perform and execute on PL/pgSQL?
From the manual:
Sometimes it is useful to evaluate an expression or SELECT query but discard the result, for example when calling a function that has side-effects but no useful result value. To do this in PL/pgSQL, use the PERFORM statement.
But, when I'm trying something like:
perform 'create table foo as (select 1)';
Nothing happens. Although this query should have side effects (creating table), and the result can be discarded.
I think I get 1 thing right: in order to run functions I can use perform:
perform pg_temp.addInheritance(foo);
PERFORM is plpgsql command used for calls of void functions. PLpgSQL is careful about useless SELECT statements - the SELECT without INTO clause is not allowed. But sometimes you need to call a function and you don't need to store result (or functions has no result). The function in SQL is called with SELECT statement. But it is not possible in PLpgSQL - so the command PERFORM was introduced.
CREATE OR REPLACE FUNCTION foo()
RETURNS void AS $$
BEGIN
RAISE NOTICE 'Hello from void function';
END;
$$ LANGUAGE plpgsql;
-- direct call from SQL
SELECT foo();
-- in PLpgSQL
DO $$
BEGIN
SELECT foo(); -- is not allowed
PERFORM foo(); -- is ok
END;
$$;
The PERFORM statements execute a parameter and forgot result.
Your example perform 'create table foo as (select 1)';
is same like SELECT 'create table foo as (select 1)'. It returns a string "create table foo as (select 1)" and this string is discarded.
The EXECUTE statement evaluate a expression to get string. In next step this string is executed.
So EXECUTE 'create table ' || some_var || '(a int)'; has two steps
evaluate expression 'create table ' || some_var || '(a int)'
if some_var is mytab for example, then execute a command create table mytab(a int)
The PERFORM statement is used for function calls, when functions are not used in assignment statement. The EXECUTE is used for evaluation of dynamic SQL - when a form of SQL command is known in runtime.
Further next line in docs you quote:
This executes query and discards the result. Write the query the same
way you would write an SQL SELECT command, but replace the initial
keyword SELECT with PERFORM.
Emphasis mine
execute in its turn executes dynamic query (same docs above)
I recently had a case where I needed to set specific constraints to DEFERRED and found something interesting.
- EXECUTE 'SET CONSTRAINTS fk_a DEFERRED';
- PERFORM 'SET CONSTRAINTS fk_a DEFERRED';
Both EXECUTE and PERFORM executed the statements without error but only EXECUTE persisted the action for the rest of the code.
Somehow, it looks like PERFORM runs in it's own transaction "bubble".
In my case I had two tables a and b and a FK (the real stuation is much more complex). There was a need to insert the data out of the parent/child order and for that we needed the constraint DEFERRED.
Using the PERFORM we had a foreign key violation, with EXECUTE we didn't.
According to docs of PostgreSQL it is possible to copy data to csv file right from a query without using an intermediate table. I am curious how to do that.
CREATE OR REPLACE FUNCTION m_tbl(my_var integer)
RETURNS void AS
$BODY$
DECLARE
BEGIN
COPY (
select my_var
)
TO 'c:/temp/out.csv';
END;
$$ LANGUAGE plpgsql;
I get an error: no such column 'my_var'.
Yes, it is possible to COPY from any query, whether or not it refers to a table.
However, COPY is a non-plannable statement, a utility statement. It doesn't support query parameters - and query parameters are how PL/PgSQL implements the insertion of variables into statements.
So you can't use PL/PgSQL variables with COPY.
You must instead use dynamic SQL with EXECUTE. See the Pl/PgSQL documentation for examples. There are lots of examples here on Stack Overflow and on https://dba.stackexchange.com/ too.
Something like:
EXECUTE format('
COPY (
select %L
)
TO ''c:/temp/out.csv'';
', my_var);
The same applies if you want the file path to be dynamic - you'd use:
EXECUTE format('
COPY (
select %L
)
TO %L;
', my_var, 'file_name.csv');
It also works for dynamic column names but you would use %I (for identifier, like "my_name") instead of %L for literal like 'my_value'. For details on %I and %L, see the documentation for format.