I have a simple table "id, name, content" and I would like to export all the records in files named "id_name.txt" with content from the "content" column (text type). This will create as many files as necessary. How to do that in psql?
I was thinking something like this, but parsed does not like "arow.id" and "TO filename" syntax.
do $$
declare
arow record;
filename varchar;
begin
for arow in
select id, name, template from config_templates
loop
filename := '/tmp/' || arow.name || '.txt';
COPY (select template from config_templates where id = arow.id) TO filename (FORMAT CSV);
end loop;
end;
$$;
May be it is late for you but I found the solution in your question, so wanted to share my solution here. Your looped solution didn't work because COPY command waits filename as a string constant. My solution is using dynamic query with EXECUTE. Please pay attention to double single quotes in EXECUTE params.
do $$
declare
arow record;
filename varchar;
begin
for arow in
select id, name, template from config_templates
loop
filename := '/tmp/' || arow.name::character varying(100) || '.txt';
EXECUTE format('COPY (select template from config_templates where id = ''%s'' ) TO ''%s'' (FORMAT CSV)', arow.id::character varying(100), filename);
end loop;
end;
$$;
Take psql console and type
\COPY table_name(column_name) TO 'path_of_text_file';
Related
How to delete a row in a stored procedure in PostgreSQL with dynamic table and column names?
delete_word will be the one that should be deleted.
CREATE OR REPLACE PROCEDURE delete_row(tablename VARCHAR(255),columnname VARCHAR(255), delete_word VARCHAR(255))
language plpgsql
as $$
BEGIN
EXECUTE 'DELETE FROM ' || quote_ident(tablename) || ' WHERE ' || columnname || ' = ' || quote_ident(delete_word);
END $$;
CALL delete_row('sales_2019','orderid', 'Order ID');
Handling dynamic SQL properly:
CREATE OR REPLACE PROCEDURE delete_row(tablename text, columnname text, delete_word text)
LANGUAGE plpgsql AS
$proc$
DECLARE
_sql text := format('DELETE FROM %I WHERE %I = $1', $1, $2);
BEGIN
-- RAISE NOTICE '%', _sql; -- debug first?!
EXECUTE _sql
USING delete_word;
END
$proc$;
Call:
CALL delete_row('sales_2019', 'orderid', 'Order ID');
See:
Format specifier for integer variables in format() for EXECUTE?
Table name as a PostgreSQL function parameter
How to use variable as table name in plpgsql
Dynamic SQL with format() and EXECUTE is only needed if table and column name must indeed be variable. The answer is a proof of concept for more complex tasks. A simple command like in the example, I would rather just concatenate and execute as plain SQL.
Also using data type text for arguments. varchar(255) does not do anything useful there, and generally tends to be a misunderstanding carried over from other RDBMS. See:
Refactor foreign key to fields
I am trying to write function in postgresql, that creates temp_table with columns table_name text, table_rec jsonb and fill it through for loop with table names from my table containing names of tables and records in json. I have the for loop in string and I want to execute it. But it doesnt work.
I have variable rec record, sql_query text and tab_name text and I want to do this:
CREATE OR REPLACE FUNCTION public.test51(
)
RETURNS TABLE(tabel_name text, record_json jsonb)
LANGUAGE 'plpgsql'
COST 100
VOLATILE
ROWS 1000
AS $BODY$
declare
rec record;
tabel_name text;
tabel_names text[];
counter integer := 1;
sql_query text;
limit_for_sending integer;
rec_count integer;
begin
select into tabel_names array(select "TABLE_NAME" from public."TABLES");
create temp table temp_tab(tab_nam text, recik jsonb);
while array_length(tabel_names, 1) >= counter loop
tabel_name := '"' || tabel_names[counter] || '"';
select into limit_for_sending "TABLE_LIMIT_FOR_SENDING_DATA" from public."TABLES" where "TABLE_NAME" = tabel_name;
sql_query := 'select count(*) from public.' || tabel_name;
execute sql_query into rec_count;
if (rec_count >= limit_for_sending and limit_for_sending is not null) then
sql_query := 'for rec in select * from public.' || tabel_name || '
loop
insert into temp_tab
select ' || tabel_name || ', to_jsonb(rec);
end loop';
execute sql_query;
end if;
counter := counter + 1;
end loop;
return query
select * from temp_tabik;
drop table temp_tabik;
end;
$BODY$;
Thank you for response.
It seems you have some table that contains the information for which tables you want to return all rows as JSONB. And that meta-table also contains a column that sets a threshold under which the rows should not be returned.
You don't need the temp table or an array to store the table names. You can iterate through the query on the TABLES table and run the dynamic SQL directly in that loop.
return query in PL/pgSQL doesn't terminate the function, it just appends the result of the query to the result of the function.
Dynamic SQL is best created using the format() function because it is easier to read and using the %I placeholder will properly deal with quoted identifiers (which is really important as you are using those dreaded upper case table names)
As far as I can tell, your function can be simplified to:
CREATE OR REPLACE FUNCTION public.test51()
RETURNS TABLE(tabel_name text, record_json jsonb)
LANGUAGE plpgsql
AS
$BODY$
declare
rec record;
sql_query text;
rec_count bigint;
begin
for rec in
select "TABLE_NAME" as table_name, "TABLE_LIMIT_FOR_SENDING_DATA" as rec_limit
from public."TABLES"
loop
if rec.rec_limit is not null then
execute format('select count(*) from %I', rec.table_name)
into rec_count;
end if;
if (rec.rec_limit is not null and rec_count >= rec.rec_limit) then
sql_query := format('select %L, to_jsonb(t) from %I as t', rec.table_name, rec.table_name);
return query execute sql_query;
end if;
end loop;
end;
$BODY$;
Some notes
the language name is an identifier and should not be enclosed in single quotes. This syntax is deprecated and might be removed in a future version so don't get used to it.
you should really avoid those dreaded quoted identifiers. They are much more trouble than they are worth it. See the Postgres wiki for details.
I have a users table that has an id, first_name, and last_name.
I also have a messages table that has a user_id, and text.
How do I export a user's messages to a CSV file for each user?
I can do it for one user:
COPY (SELECT text FROM messages WHERE user_id = "my user id") to "Firstname_Lastname.csv" DELIMITER ',' CSV HEADER;
but postgres doesn't seem to have a for loop or anything? I did some googling and stumbled into LATERAL but could not get that to work...
Using Adrian's link and a lot of reading and cursing at my screen I hacked something together:
CREATE OR REPLACE FUNCTION export()
RETURNS integer
LANGUAGE plpgsql
AS $$
DECLARE
myuser RECORD;
filename TEXT;
BEGIN
FOR myuser IN SELECT id,first_name,last_name FROM users LOOP
RAISE NOTICE 'user %', myuser;
EXECUTE $$SELECT '/home/me/' || $1 || '_' || $2 || '.csv'$$ INTO filename USING myuser.first_name, myuser.last_name;
RAISE NOTICE 'filename %', filename;
EXECUTE $$COPY (SELECT text FROM messages WHERE user_id = '$$ || myuser.id || $$') TO '$$ || filename || $$' DELIMITER ',' CSV HEADER$$;
END LOOP;
RETURN 1;
END;
$$
It's quite a mess and is not idiomatic at all I'm sure. But it worked for me.
I have a database with multiple identical schemas. There is a number of tables all named 'tran_...' in each schema. I want to loop through all 'tran_' tables in all schemas and pull out records that fall within a specific date range. This is the code I have so far:
CREATE OR REPLACE FUNCTION public."configChanges"(starttime timestamp, endtime timestamp)
RETURNS SETOF character varying AS
$BODY$DECLARE
tbl_row RECORD;
tbl_name VARCHAR(50);
tran_row RECORD;
out_record VARCHAR(200);
BEGIN
FOR tbl_row IN
SELECT * FROM pg_tables WHERE schemaname LIKE 'ivr%' AND tablename LIKE 'tran_%'
LOOP
tbl_name := tbl_row.schemaname || '.' || tbl_row.tablename;
FOR tran_row IN
SELECT * FROM tbl_name
WHERE ch_edit_date >= starttime AND ch_edit_date <= endtime
LOOP
out_record := tbl_name || ' ' || tran_row.ch_field_name;
RETURN NEXT out_record;
END LOOP;
END LOOP;
RETURN;
END;
$BODY$
LANGUAGE plpgsql;
When I attempt to run this, I get:
ERROR: relation "tbl_name" does not exist
LINE 1: SELECT * FROM tbl_name WHERE ch_edit_date >= starttime AND c...
#Pavel already provided a fix for your basic error.
However, since your tbl_name is actually schema-qualified (two separate identifiers in : schema.table), it cannot be escaped as a whole with %I in format(). You have to escape each identifier individually.
Aside from that, I suggest a different approach. The outer loop is necessary, but the inner loop can be replaced with a simpler and more efficient set-based approach:
CREATE OR REPLACE FUNCTION public.config_changes(_start timestamp, _end timestamp)
RETURNS SETOF text AS
$func$
DECLARE
_tbl text;
BEGIN
FOR _tbl IN
SELECT quote_ident(schemaname) || '.' || quote_ident(tablename)
FROM pg_tables
WHERE schemaname LIKE 'ivr%'
AND tablename LIKE 'tran_%'
LOOP
RETURN QUERY EXECUTE format (
$$
SELECT %1$L || ' ' || ch_field_name
FROM %1$s
WHERE ch_edit_date BETWEEN $1 AND $2
$$, _tbl
)
USING _start, _end;
END LOOP;
RETURN;
END
$func$ LANGUAGE plpgsql;
You have to use dynamic SQL to parametrize identifiers (or code), like #Pavel already told you. With RETURN QUERY EXECUTE you can return the result of a dynamic query directly. Examples:
Return SETOF rows from PostgreSQL function
Refactor a PL/pgSQL function to return the output of various SELECT queries
Remember that identifiers have to be treated as unsafe user input in dynamic SQL and must always be sanitized to avoid syntax errors and SQL injection:
Table name as a PostgreSQL function parameter
Note how I escape table and schema separately:
quote_ident(schemaname) || '.' || quote_ident(tablename)
Consequently I just use %s to insert the already escaped table name in the later query. And %L to escape it a string literal for output.
I like to prepend parameter and variable names with _ to avoid naming conflicts with column names. No other special meaning.
There is a slight difference compared to your original function. This one returns an escaped identifier (double-quoted only where necessary) as table name, e.g.:
"WeIRD name"
instead of
WeIRD name
Much simpler yet
If possible, use inheritance to obviate the need for above function altogether. Complete example:
Select (retrieve) all records from multiple schemas using Postgres
You cannot use a plpgsql variable as SQL table name or SQL column name. In this case you have to use dynamic SQL:
FOR tran_row IN
EXECUTE format('SELECT * FROM %I
WHERE ch_edit_date >= starttime AND ch_edit_date <= endtime', tbl_name)
LOOP
out_record := tbl_name || ' ' || tran_row.ch_field_name;
RETURN NEXT out_record;
END LOOP;
I need to create a stored procedure which can take in the schema name and table name as the parameter and make an update on the same table.
CREATE OR REPLACE FUNCTION garscratch.update_table(schema_name text, table_name text ) RETURNS void as $$
DECLARE
table TEXT;
BEGIN
execute 'update '||schema||'.'||table_name|| 'set id = substring(id from 1 for 2) where name = "test"';
END;
$$ LANGUAGE plpgsql;
When I execute the procedure above as:
select update_table(my,my_table);
I get the error:
Column "my" does not exist.
It does not treat "my" as the schema name.
You'd need a space between tablename and set.
You need sinlge quotes around the value 'test'.
Or if it's supposed to be a column name, you don't need quotes at all.
You need to sanitize identifiers to avoid SQL injection.
Use instead (complete rewrite):
CREATE OR REPLACE FUNCTION garscratch.update_table(_tbl regclass)
RETURNS void AS
$func$
BEGIN
EXECUTE 'UPDATE ' || _tbl || $$ SET id = left(id, 2) WHERE name = 'test'$$;
END
$func$ LANGUAGE plpgsql;
Call:
SELECT garscratch.update_table('myschema.my_table');
Detailed explanation:
Table name as a PostgreSQL function parameter
It seems to me you may have problem with the table_name|| 'set id= part - specifically I think you should add a space between ' and set. Try to print what you're executing and you may found out where the problem may be.
Execute Procedure as below:
select update_table('my','my_table');
Also change line
execute 'update '||schema||'.'||table_name|| 'set id = substring(id from 1 for 2) where name = "test"';
to
execute 'update '||schema_name||'.'||table_name|| 'set id = substring(id from 1 for 2) where name = "test"';