I use below command to export table to csv
COPY (
SELECT
*
FROM
table_name
TO '/data/test/test.csv' WITH CSV DELIMITER ',' HEADER;
I used it in Postgres function but i cannot pass string to file path
CREATE OR REPLACE FUNCTION backup (filePath CHARACTER VARYING)
RETURNS void
LANGUAGE SQL
AS $$
-- Export data to csv
COPY (
SELECT
*
FROM
table_name)
TO filePath WITH CSV DELIMITER ',' HEADER;
$$
err: Query 1 ERROR: ERROR: syntax error at or near "filePath" LINE 16: TO filePath WITH CSV DELIMITER ',' HEADER;
See here SQL Functions:
SQL function arguments can only be used as data values, not as identifiers. Thus for example this is reasonable: ...
I'm going to say the filePath is being seen as an identifier. You will need to use plpgsql like so:
CREATE OR REPLACE FUNCTION public.backup(filepath character varying)
RETURNS void
LANGUAGE plpgsql
AS $function$
BEGIN
-- Export data to csv
EXECUTE 'COPY ( SELECT * FROM table_name) TO ' || quote_literal(filePath) || ' WITH CSV DELIMITER '','' HEADER';
RETURN;
END;
$function$
Related
I am generating a few CSV files using COPY statements from postgres in a stored function.
CREATE OR REPLACE FUNCTION "public"."create_csv" () RETURNS INT AS
$$
BEGIN
EXECUTE format('COPY (SELECT * FROM test) TO %L DELIMITER '','' CSV HEADER', '/tmp/test.csv');
EXECUTE format('COPY (SELECT * FROM test2) TO %L DELIMITER '','' CSV HEADER', '/tmp/test2.csv');
RETURN 0;
END;
$$
LANGUAGE 'plpgsql' SECURITY DEFINER
This function is called from a parent one, after the execution of that I want to zip the generated CSV files into a zip file, how can I accomplish this using stored functions in the data base? Is there a way to do so using plpgsql? Or should i use plpythonu language?
Postgres version 9.6.
Thanks!
#a_horse_with_no_name gave you an alert about what you should do use copy ... PROGRAM, just add the code at the end of your function, in this case in tar format:
CREATE OR REPLACE FUNCTION "public"."create_csv" () RETURNS INT AS
$$
BEGIN
EXECUTE format('COPY (SELECT * FROM test) TO %L DELIMITER '','' CSV HEADER', '/tmp/test.csv');
EXECUTE format('COPY (SELECT * FROM test2) TO %L DELIMITER '','' CSV HEADER', '/tmp/test2.csv');
--add copy commando with PROGRAM
COPY (SELECT 1) TO PROGRAM 'tar cvf /tmp/mytarfile.tar --remove-files /tmp/*.csv';
RETURN 0;
END;
$$
LANGUAGE 'plpgsql' SECURITY DEFINER
I have two functions. One that copies variable (uuid) to a binary file on one database.
CREATE OR REPLACE FUNCTION ECRS."MIGRATION.DBF_COPY_TO"(file_name VARCHAR(500))
RETURNS INTEGER AS $$
DECLARE
iniPath varchar(500) = file_name || '/Ini.dat';
researchIdToCopy uuid;
BEGIN
SELECT R.RESEARCHID FROM ECRS.RESEARCH R WHERE R.NAME = 'BADANIE_TESTOWE' INTO researchIdToCopy;
EXECUTE('COPY (SELECT '''|| researchIdToCopy ||''') TO ' || quote_literal(iniPath)|| ' WITH BINARY');
RETURN 1;
END; $$
LANGUAGE plpgsql;
The secound one is responsible for copying this data from file and store it into variable on another postgress database.
CREATE OR REPLACE FUNCTION ECRS."MIGRATION.DBF_COPY_FROM"(file_name VARCHAR(500))
RETURNS INTEGER AS $$
DECLARE
iniPath varchar(500) = file_name || '/Ini.dat';
researchIdToInsert text;
tmp text;
BEGIN
-- get research id (store ini file in tmp table and select from it)
tmp := quote_ident(uuid_generate_v4()::text);
EXECUTE 'CREATE TEMP TABLE ' || tmp || ' (researchid text)';
EXECUTE 'COPY ' || tmp || ' FROM ' || quote_literal(iniPath);
EXECUTE 'SELECT researchid FROM ' || tmp INTO researchIdToInsert;
EXECUTE 'DROP TABLE ' || tmp;
-- DO SOMETHING MORE ...
RETURN 1;
END; $$
LANGUAGE plpgsql;
I get an error executing function DBF_COPY_FROM:
ERROR: literal carriage return found in data
HINT: Use "\r" to represent carriage return.
CONTEXT: COPY f4c96770-f2f3-45e9-8678-1626b9bef843, line 2
SQL statement "COPY "f4c96770-f2f3-45e9-8678-1626b9bef843" FROM 'C://Test/Ini.dat'"
PL/pgSQL function ecrs."MIGRATION.DBF_COPY_FROM"(character varying) line 29 at EXECUTE
Both databases have the same properties :
ENCODING = 'UTF8'
TABLESPACE = pg_default
LC_COLLATE ='English_United Kingdom.1252'
LC_CTYPE = 'English_United Kingdom.1252'
CONNECTION LIMIT = -1;
I have tried to use
set client_encoding to 'WIN1252';
or
set client_encoding to 'UTF8'; but it doesn't work.
Do I understand correctly that you write a file with MIGRATION.DBF_COPY_TO and then read it with MIGRATION.DBF_COPY_FROM?
If yes, that's not going to work, because the first function uses the binary format, while the second function uses the text format.
Carriage return characters without following line feed are not allowed in text mode COPY, but in this case it is a consequence of you using different formats for input and output.
I am using postgresql query to copy data from csv file and insert that data into another table..the query works fine but i want that query to be put into a function so that i can use this function in my c# application.
Here is the query which works fine...
copy emp_temp from 'd:\\temp.csv' WITH DELIMITER AS ',' CSV QUOTE AS '"';
here emp_temp is the table and temp.csv is my file.
but
when i am using the above function like
create or replace function CopyToTable(file_path text,table_name text) returns void as
$body$
begin
copy quote_ident(table_name) from file_path WITH DELIMITER AS ',' CSV QUOTE AS '"';
end;
$body$
language plpgsql volatile
It shows error like
ERROR: syntax error at or near "file_path"...
From some of the suggestions i also tried using dynamic sql
create or replace function CopyToTable(file_path text,table_name text) returns void as
$body$
begin
execute 'copy quote_ident(' || table_name || ')' || ' from ' ||
file_path || 'WITH DELIMITER AS ' || ',' || ' CSV QUOTE AS ' || '"';
end;
$body$
language plpgsql volatile
The function compiles successfully...but when i am giving parameters by running the function
select * from CopyToTable('d:\\temp.csv','emp_temp')
it shows error..
ERROR: syntax error at or near "d"
LINE 1: copy quote_ident(emp_temp) from d:\\temp.csvWITH DELIMITER ...
^
I'm trying to compile a postgresql function
CREATE OR REPLACE FUNCTION LoadData(tablename varchar(25), filepath varchar(35))
RETURNS void AS $$
declare
BEGIN
RAISE NOTICE 'Data is being loaded from an external file, please wait...';
COPY tablename FROM filepath DELIMITER ',' CSV HEADER;
RAISE NOTICE 'Data loaded successfully!!';
END;
$$ LANGUAGE plpgsql;
but it's giving an error like this
ERROR: syntax error at or near "filepath"
LINE 9: COPY tablename FROM filepath DELIMITER ',' CSV HEADER;
How to use both the in parameters in the COPY command?
To execute dynamically built SQL queries use execute. format makes it easy and safe
create or replace function loaddata(
tablename varchar(25), filepath varchar(35)
) returns void as $$
begin
raise notice 'data is being loaded from an external file, please wait...';
execute format($copy$
copy %I from %L delimiter ',' csv header
$copy$,
tablename, filepath
);
raise notice 'data loaded successfully!!';
end;
$$ language plpgsql;
http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN
I would like to repeat the following query 8760 times, replacing ‘2’ with 1 to 8760 for every hour in the year. The idea is to create a separate CSV file for each hour for further processing.
COPY
(SELECT *
FROM
public.completedsolarirad2012
WHERE
completedsolarirad2012."UniquetmstmpID" = 2)
TO 'C:\temp\2012hour2.csv' WITH DELIMITER ','
CSV HEADER
I have put together the following function (testing with only a few hours):
CREATE OR REPLACE FUNCTION everyhour()
RETURNS void AS
$BODY$BEGIN
FOR i IN 0..5 LOOP
EXECUTE $x$
COPY (
SELECT *
FROM
public.completedsolarirad2012
WHERE
completedsolarirad2012."UniquetmstmpID" = i
)
TO $concat$ 'C:\temp.' || i::text
|| '.out.csv' WITH DELIMITER ',' CSV HEADER $concat$
$x$;
END LOOP;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION everyhour()
OWNER TO postgres;
I seem to be having two separate problems:
Firstly, I’m getting:
Error: column "i" does not exist.
Secondly, when I test the concatenation statement only by replacing “i” with e.g. “2”, I get:
Error: relative path not allowed for COPY to file
I am the postgres superuser, so I do not understand why I am having this problem.
Note: Removing the $concat$ double-quoting around the concatenation statement gives the following error:
ERROR: syntax error at or near "||"
LINE 9: TO 'C:\temp.' || i::text
I would be very grateful for any help.
Assuming your server OS is Windows.
CREATE OR REPLACE FUNCTION everyhour()
RETURNS void AS
$func$
BEGIN
FOR i IN 0..5 LOOP
EXECUTE '
COPY (
SELECT *
FROM public.completedsolarirad2012 c
WHERE c."UniquetmstmpID" = ' || i || $x$)
TO 'C:/temp.'$x$ || i || $x$'.out.csv' WITH DELIMITER ',' CSV HEADER$x$;
END LOOP;
END
$func$ LANGUAGE plpgsql;
You had one layer of dollar-quoting too many.
You also accidentally concatenated the letter "i" instead of its value.
Use forward-slashes, even with windows. Or you may have to double up the backslashes, depending on your settings. More in this related answer:
PostgreSQL: export resulting data from SQL query to Excel/CSV
Simpler with format()
Since Postgres 9.1 you can use format() to simplify complex concatenations:
CREATE OR REPLACE FUNCTION everyhour()
RETURNS void AS
$func$
BEGIN
FOR i IN 0..5 LOOP
EXECUTE format($x$COPY (
SELECT *
FROM public.completedsolarirad2012 c
WHERE c."UniquetmstmpID" = %1$s)
TO 'C:/temp.%1$s.out.csv' WITH DELIMITER ',' CSV HEADER$x$, i);
END LOOP;
END
$func$ LANGUAGE plpgsql;