In a plpgsql function, I have a variable of type record:
my_rec RECORD;
This record contains a row from an arbitrary table, so I do not know the columns before it is executed.
However, I do have the name of at least one of the columns available as a varchar.
The question is: How do I retrieve the value for a given column from my_rec?
Use hstore to work with records with dynamic columns in PL/PgSQL functions:
CREATE EXTENSION hstore;
CREATE OR REPLACE FUNCTION test_fn(col_name text) RETURNS text AS $$
DECLARE
input_row record;
col_value text;
BEGIN
SELECT INTO input_row
*
FROM ( VALUES ('a','b','c',1) ) AS dummyrow(col1,col2,col3,intcol);
SELECT INTO col_value
hstore(input_row) -> col_name;
RETURN col_value;
END;
$$ LANGUAGE 'plpgsql';
hstore is an extension, but it's an extension that's been bundled with PostgreSQL since 8.3 and has been installable using CREATE EXTENSION since 9.1. The record-to-hstore conversion has been supported since something like 8.4 or 9.0.
I don't know of a way to do this in plpgsql. I did a bit of testing for you and tried to make a "EXECUTE SELECT" solution work, such as:
EXECUTE 'select $1.' || quote_ident(the_param) USING my_rec INTO my_var;
This does not work for me and I get:
could not identify column "{{param_value here}}" in record data type
Here is a very similar question from a few years ago saying that it is not possible with plpgsql. Per it's suggestion, it appears that it should be possible with some other languages. Quoting Tom Lane's answer:
There is no way to do that in plpgsql. You could do it in the other PLs
(eg plperl, pltcl) since they are not as strongly typed as plpgsql.
Related
So I have a function in PostgreSQL that dynamically selects columns from a dynamic table. I got this solution from this post and it works great other than one thing.
This is inside of a file that is connected to a Node server, and so the $1 and $2 in the second SELECT * FROM represent values passed from there. The issue right now is that I am getting a syntax error that I don't understand (I am newer to SQL so that may be why).
$2 represents the name of the table to be selected from as a string, so for example it could be 'goals'. The error is syntax error at or near "'goals'". I realize that it cannot be a string with single quotes (I believe) and so I am wondering how to convert that variable to be a table name? using "goals" there as well as goals, for example works as expected, though I'm not sure how to do that outside of a function.
CREATE OR REPLACE FUNCTION get_data(user_id INT, table_name anyelement)
RETURNS SETOF ANYELEMENT AS $$
BEGIN
RETURN QUERY EXECUTE
format('SELECT * FROM %s WHERE user_id = $1', pg_typeof(table_name)) USING user_id;
END;
$$ LANGUAGE plpgsql;
SELECT * FROM get_data($1, NULL::$2);
$1 is 5 and $2 is 'goals' for example
After many hours of trying to figure it out, thanks to Adrian's comment, I found MassiveJS (how I'm connecting to my PostgreSQL server) has inline functions to do queries. In my controller file in my server I was able to create a one line function as such:
const data = await db[tableName].where("user_id=$1", [userId])
Didn't know inline SQL existed in MassiveJS, so that was great to find out!
I need to create a function, which returns results of a SELECT query. This SELECT query is a JOIN of few temporary tables created inside this function. Is there any way to create such function? Here is an example (it is very simplified, in reality there are multiple temp tables with long queries):
CREATE OR REPLACE FUNCTION myfunction () RETURNS TABLE (column_a TEXT, column_b TEXT) AS $$
BEGIN
CREATE TEMPORARY TABLE raw_data ON COMMIT DROP
AS
SELECT d.column_a, d2.column_b FROM dummy_data d JOIN dummy_data_2 d2 using (id);
RETURN QUERY (select distinct column_a, column_b from raw_data limit 100);
END;
$$
LANGUAGE 'plpgsql' SECURITY DEFINER
I get error:
[Error] Script lines: 1-19 -------------------------
ERROR: RETURN cannot have a parameter in function returning set;
use RETURN NEXT at or near "QUERY"Position: 237
I apologize in advance for any obvious mistakes, I'm new to this.
Psql version is PostgreSQL 8.2.15 (Greenplum Database 4.3.12.0 build 1)
The most recent version of Greenplum Database (5.0) is based on PostgreSQL 8.3, and it supports the RETURN QUERY syntax. Just tested your function on:
PostgreSQL 8.4devel (Greenplum Database 5.0.0-beta.10+dev.726.gd4a707c762 build dev)
The most probable error this could raise in Postgres:
ERROR: column "foo" specified more than once
Meaning, there is at least one more column name (other than id which is folded to one instance with the USING clause) included in both tables. This would not raise an exception in a plain SQL SELECT which tolerates duplicate output column names. But you cannot create a table with duplicate names.
The problem also applies for Greenplum (like you later declared), which is not Postgres. It was forked from PostgreSQL in 2005 and developed separately. The current Postgres manual hardly applies at all any more. Look to the Greenplum documentation.
And psql is just the standard PostgreSQL interactive terminal program. Obviously you are using the one shipped with PostgreSQL 8.2.15, but the RDBMS is still Greenplum, not Postgres.
Syntax fix (for Postgres, like you first tagged, still relevant):
CREATE OR REPLACE FUNCTION myfunction()
RETURNS TABLE (column_a text, column_b text) AS
$func$
BEGIN
CREATE TEMPORARY TABLE raw_data ON COMMIT DROP AS
SELECT d.column_a, d2.column_b -- explicit SELECT list avoids duplicate column names
FROM dummy_data d
JOIN dummy_data_2 d2 using (id);
RETURN QUERY
SELECT DISTINCT column_a, column_b
FROM raw_data
LIMIT 100;
END
$func$ LANGUAGE plpgsql SECURITY DEFINER;
The example wouldn't need a temp table - unless you access the temp table after the function call in the same transaction (ON COMMIT DROP). Else, a plain SQL function is better in every way. Syntax for Postgres and Greenplum:
CREATE OR REPLACE FUNCTION myfunction(OUT column_a text, OUT column_b text)
RETURNS SETOF record AS
$func$
SELECT DISTINCT d.column_a, d2.column_b
FROM dummy_data d
JOIN dummy_data_2 d2 using (id)
LIMIT 100;
$func$ LANGUAGE plpgsql SECURITY DEFINER;
Not least, it should also work for Greenplum.
The only remaining reason for this function is SECURITY DEFINER. Else you could just use the simple SQL statement (possibly as prepared statement) instead.
RETURN QUERY was added to PL/pgSQL with version 8.3 in 2008, some years after the fork of Greenplum. Might explain your error msg:
ERROR: RETURN cannot have a parameter in function returning set;
use RETURN NEXT at or near "QUERY" Position: 237
Aside: LIMIT without ORDER BY produces arbitrary results. I assume you are aware of that.
If for some reason you actually need temp tables and cannot upgrade to Greenplum 5.0 like A. Scherbaum suggested, you can still make it work in Greenplum 4.3.x (like in Postgres 8.2). Use a FOR loop in combination with RETURN NEXT.
Examples:
plpgsql error "RETURN NEXT cannot have a parameter in function with OUT parameters" in table-returning function
How to use `RETURN NEXT`in PL/pgSQL correctly?
Use of custom return types in a FOR loop in plpgsql
I would love to exploit the SQL output formatting of PostgreSQL inside my PL/pgSQL functions, but I'm starting to feel I have to give up the idea.
I have my PL/pgSQL function query_result:
CREATE OR REPLACE FUNCTION query_result(
this_query text
) RETURNS SETOF record AS
$$
BEGIN
RETURN QUERY EXECUTE this_query;
END;
$$ LANGUAGE plpgsql;
..merrily returning a SETOF records from an input text query, and which I can use for my SQL scripting with dynamic queries:
mydb=# SELECT * FROM query_result('SELECT ' || :MYVAR || ' FROM Alice') AS t (id int);
id
----
1
2
3
So my hope was to find a way to deliver this same nicely formatted output from inside a PL/pgSQL function instead, but RAISE does not support SETOF types, and there's no magic predefined cast from SETOF records to text (I know I could create my own CAST..)
If I create a dummy print_result function:
CREATE OR REPLACE FUNCTION print_result(
this_query text
) RETURNS void AS
$$
BEGIN
SELECT query_result(this_query);
END;
$$ LANGUAGE plpgsql;
..I cannot print the formatted output:
mydb=# SELECT print_result('SELECT ' || :MYVAR || ' FROM Alice');
ERROR: set-valued function called in context that cannot accept a set
...
Thanks for any suggestion (which preferably works with PostgreSQL 8.4).
Ok, to do anything with your result set in print_result you'll have to loop over it. That'll look something like this -
Here result_record is defined as a record variable. For the sake of explanation, we'll also assume that you have a formatted_results variable that is defined as text and defaulted to a blank string to hold the formatted results.
FOR result_record IN SELECT * FROM query_result(this_query) AS t (id int) LOOP
-- With all this, you can do something like this
formatted_results := formatted_results ||','|| result_record.id;
END LOOP;
RETURN formatted_results;
So, if you change print_results to return text, declare the variables as I've described and add this in, your function will return a comma-separated list of all your results (with an extra comma at the end, I'm sure you can make use of PostgreSQL's string functions to trim that). I'm not sure this is exactly what you want, but this should give you a good idea about how to manipulate your result set. You can get more information here about control structures, which should let you do pretty much whatever you want.
EDIT TO ANSWER THE REAL QUESTION:
The ability to format data tuples as readable text is a feature of the psql client, not the PostgreSQL server. To make this feature available in the server would require extracting relevant code or modules from the psql utility and recompiling them as a database function. This seems possible (and it is also possible that someone has already done this), but I am not familiar enough with the process to provide a good description of how to do that. Most likely, the best solution for formatting query results as text will be to make use of PostgreSQL's string formatting functions to implement the features you need for your application.
Edit: After posting I found Erwin Brandstetter's answer to a similar question. It sounds like in 9.2+ I could use the last option he listed, but none of the other alternatives sound workable for my situation. However, the comment from Jakub Kania and reiterated by Craig Ringer suggesting I use COPY, or \copy, in psql appears to solve my problem.
My goal is to get the results of executing a dynamically created query into a text file.
The names and number of columns are unknown; the query generated at run time is a 'pivot' one, and the names of columns in the SELECT list are taken from values stored in the database.
What I envision is being able, from the command line to run:
$ psql -o "myfile.txt" -c "EXECUTE mySQLGeneratingFuntion(param1, param2)"
But what I'm finding is that I can't get results from an EXECUTEd query unless I know the number of columns and their types that are in the results of the query.
create or replace function carrier_eligibility.createSQL() returns varchar AS
$$
begin
return 'SELECT * FROM carrier_eligibility.rule_result';
-- actual procedure writes a pivot query whose columns aren't known til run time
end
$$ language plpgsql
create or replace function carrier_eligibility.RunSQL() returns setof record AS
$$
begin
return query EXECUTE carrier_eligibility.createSQL();
end
$$ language plpgsql
-- this works, but I want to be able to get the results into a text file without knowing
-- the number of columns
select * from carrier_eligibility.RunSQL() AS (id int, uh varchar, duh varchar, what varchar)
Using psql isn't a requirement. I just want to get the results of the query into a text file, with the column names in the first row.
What format of a text file do you want? Something like csv?
How about something like this:
CREATE OR REPLACE FUNCTION sql_to_csv(in_sql text) returns setof text
SECURITY INVOKER -- CRITICAL DO NOT CHANGE THIS TO SECURITY DEFINER
LANGUAGE PLPGSQL AS
$$
DECLARE t_row RECORD;
t_out text;
BEGIN
FOR t_row IN EXECUTE in_sql LOOP
t_out := t_row::text;
t_out := regexp_replace(regexp_replace(t_out, E'^\\(', ''), E'\\)$', '');
return next t_out;
END LOOP;
END;
$$;
This should create properly quoted csv strings without the header. Embedded newlines may be a problem but you could write a quick Perl script to connect and write the data or something.
Note this presumes that the tuple structure (parenthesized csv) does not change with future versions, but it currently should work with 8.4 at least through 9.2.
First I am really new to pl/pgsql. Need it for a project.
I am stuck with this (simplified) problem.
My db schema has a n to m relationship (author, books, author_books)
Now I want to have a pl/psgsql function insert_book. (I do know that all authors are definitely already in the author table, so I just want to pass their primary keys).
This function outline is what I have in mind.
create or replace function insert_book(book_to_insert book, authors integer[])
returns void as $$
begin
-- insert book into table books
-- for each author add an entry to author_books table
end;
$$ language plpgsql;
As arguments I thought to pass a record of type book and the authors that wrote it. But how exactly would this work? I googled quite a bit and can't seem to figure this out...
Question 1: Is the function outline "correct"/does it make sense?
Question 2: How to insert record book into table book? Do I have to go over all fields of book (title, isbn, publisher,...) and add them to an INSERT INTO statement or is there a "smarter" way?
Question 3: How would I call my function insert_book? I found this example here (http://dbaspot.com/postgresql/206142-passing-record-function-argument-pl-pgsql.html), but that doesn't really help me. For testing purposes I am using the shell, but later on we will use Java with JDBC.
Thank you very much for your help.
Using unnest() and a data-modifying CTE (requires Postgres 9.1 or later), this can be a simple SQL query:
WITH x AS (SELECT '(1,foo_book)'::book AS _book
, '{1,2,3}'::int[] AS _authors)
, y AS (
INSERT INTO book -- no column list, correct due to composite type
SELECT (x._book).*
FROM x
RETURNING book_id
)
INSERT INTO author_book (book_id, author_id)
SELECT y.book_id, unnest(x._authors)
FROM x,y; -- CROSS JOIN ok, only 1 row for x and y
The first CTE x is just for simplified data input and not strictly needed.
SQL Fiddle.
As to your questions:
Question 1: Is the function outline "correct"/does it make sense?
Might be easier to pass base types instead of the composite type book, but it is a perfectly valid approach. You have to know your way around the syntax for complex types, though. For instance, note the parenthesis around the name in my example: (x._book).*.
A plpgsql function could look like this:
CREATE OR REPLACE FUNCTION f_insert_book(_book book, _authors integer[])
RETURNS void AS
$func$
BEGIN
WITH y AS (
INSERT INTO book b
SELECT (_book).*
RETURNING b.book_id
)
INSERT INTO author_book (book_id, author_id)
SELECT y.book_id, unnest(_authors)
FROM y;
END
$func$ LANGUAGE plpgsql;
Question 2: How to insert record book into table book? (...) or is there a "smarter" way?
The smarter way is to decompose the composite type with (variable_name).*.
As the type is guaranteed to match the table (being derived from it), this is one of the rare cases, where it is perfectly ok, not to provide a column list for the INSERT command in persisted code.
Question 3: How would I call my function insert_book? ...
SELECT f_insert_book('(1,foo_book)'::book, '{1,2,3}'::int[]);
Within other plpgsql functions, use PERFORM instead of SELECT if you don't provide a target (INTO foo) for the (non-existing) results.
Passing JSON datatype (Postgresql 9.2 or higher):
CREATE OR REPLACE FUNCTION f_insert_book(_book json, _authors json)
RETURNS void AS
$$
BEGIN
-- insert book into table books
Insert into books values select * from json_populate_recordset(null:book, _book);
-- for each author add an entry to author_books table
Insert into authors values select * from json_populate_recordset(null:authors, _authors);
end;
$$ language plpgsql;