I am using postgresql query to copy data from csv file and insert that data into another table..the query works fine but i want that query to be put into a function so that i can use this function in my c# application.
Here is the query which works fine...
copy emp_temp from 'd:\\temp.csv' WITH DELIMITER AS ',' CSV QUOTE AS '"';
here emp_temp is the table and temp.csv is my file.
but
when i am using the above function like
create or replace function CopyToTable(file_path text,table_name text) returns void as
$body$
begin
copy quote_ident(table_name) from file_path WITH DELIMITER AS ',' CSV QUOTE AS '"';
end;
$body$
language plpgsql volatile
It shows error like
ERROR: syntax error at or near "file_path"...
From some of the suggestions i also tried using dynamic sql
create or replace function CopyToTable(file_path text,table_name text) returns void as
$body$
begin
execute 'copy quote_ident(' || table_name || ')' || ' from ' ||
file_path || 'WITH DELIMITER AS ' || ',' || ' CSV QUOTE AS ' || '"';
end;
$body$
language plpgsql volatile
The function compiles successfully...but when i am giving parameters by running the function
select * from CopyToTable('d:\\temp.csv','emp_temp')
it shows error..
ERROR: syntax error at or near "d"
LINE 1: copy quote_ident(emp_temp) from d:\\temp.csvWITH DELIMITER ...
^
Related
I use below command to export table to csv
COPY (
SELECT
*
FROM
table_name
TO '/data/test/test.csv' WITH CSV DELIMITER ',' HEADER;
I used it in Postgres function but i cannot pass string to file path
CREATE OR REPLACE FUNCTION backup (filePath CHARACTER VARYING)
RETURNS void
LANGUAGE SQL
AS $$
-- Export data to csv
COPY (
SELECT
*
FROM
table_name)
TO filePath WITH CSV DELIMITER ',' HEADER;
$$
err: Query 1 ERROR: ERROR: syntax error at or near "filePath" LINE 16: TO filePath WITH CSV DELIMITER ',' HEADER;
See here SQL Functions:
SQL function arguments can only be used as data values, not as identifiers. Thus for example this is reasonable: ...
I'm going to say the filePath is being seen as an identifier. You will need to use plpgsql like so:
CREATE OR REPLACE FUNCTION public.backup(filepath character varying)
RETURNS void
LANGUAGE plpgsql
AS $function$
BEGIN
-- Export data to csv
EXECUTE 'COPY ( SELECT * FROM table_name) TO ' || quote_literal(filePath) || ' WITH CSV DELIMITER '','' HEADER';
RETURN;
END;
$function$
I am a newbie wrt functions and I am struggling with using the name of a table in the function body. I get an error "SQL Error [42703]: ERROR: column "tname" does not exist" when I call the function using
select "JsonToView"('data_import.import_360xero_report');
My code is below
create or replace
function data_import."JsonToView"(tname text) returns numeric
language plpgsql
as $function$
begin
do
$$
declare
l_keys text;
begin
drop view if exists v_json_view cascade;
select
string_agg(distinct format('import_data ->> %L as %I', jkey, jkey), ', ')
into
l_keys
from
import_360xero_report,
json_object_keys(import_data) as t(jkey);
execute 'create view v_json_view as select ' || l_keys || ' from ' || tname;
end;
$$;
return 0;
end $function$ ;
I have modified the code and the second create view query works with the table name but the first one does not.
Below if my modified code
create or replace
function data_import."JsonToView"(tname text) returns numeric
language plpgsql
as $function$
declare
l_keys text;
begin
drop view if exists v_json_view cascade;
execute $a$select
string_agg(distinct format('import_data ->> %L as %I', jkey, jkey), ', ')
into
l_keys
from $a$ ||
tname || $b$,
json_object_keys(import_data) as t(jkey)$b$;
execute 'create view v_json_view as select ' || l_keys || ' from ' || tname;
return 0;
end $function$ ;
The error I am getting is
SQL Error [0A000]: ERROR: EXECUTE of SELECT ... INTO is not implemented
Hint: You might want to use EXECUTE ... INTO or EXECUTE CREATE TABLE ... AS instead.
Where: PL/pgSQL function "JsonToView"(text) line 10 at EXECUTE
The problem is the superfluous nested DO statement.
The variable tname exists only in the scope of the function, not in the nested DO statement. DO is an SQL statement, not a PL/pgSQL statement, and there are no variables in SQL. Also, DO does not allow parameters.
Get rid of the DO and you will be fine.
I have two functions. One that copies variable (uuid) to a binary file on one database.
CREATE OR REPLACE FUNCTION ECRS."MIGRATION.DBF_COPY_TO"(file_name VARCHAR(500))
RETURNS INTEGER AS $$
DECLARE
iniPath varchar(500) = file_name || '/Ini.dat';
researchIdToCopy uuid;
BEGIN
SELECT R.RESEARCHID FROM ECRS.RESEARCH R WHERE R.NAME = 'BADANIE_TESTOWE' INTO researchIdToCopy;
EXECUTE('COPY (SELECT '''|| researchIdToCopy ||''') TO ' || quote_literal(iniPath)|| ' WITH BINARY');
RETURN 1;
END; $$
LANGUAGE plpgsql;
The secound one is responsible for copying this data from file and store it into variable on another postgress database.
CREATE OR REPLACE FUNCTION ECRS."MIGRATION.DBF_COPY_FROM"(file_name VARCHAR(500))
RETURNS INTEGER AS $$
DECLARE
iniPath varchar(500) = file_name || '/Ini.dat';
researchIdToInsert text;
tmp text;
BEGIN
-- get research id (store ini file in tmp table and select from it)
tmp := quote_ident(uuid_generate_v4()::text);
EXECUTE 'CREATE TEMP TABLE ' || tmp || ' (researchid text)';
EXECUTE 'COPY ' || tmp || ' FROM ' || quote_literal(iniPath);
EXECUTE 'SELECT researchid FROM ' || tmp INTO researchIdToInsert;
EXECUTE 'DROP TABLE ' || tmp;
-- DO SOMETHING MORE ...
RETURN 1;
END; $$
LANGUAGE plpgsql;
I get an error executing function DBF_COPY_FROM:
ERROR: literal carriage return found in data
HINT: Use "\r" to represent carriage return.
CONTEXT: COPY f4c96770-f2f3-45e9-8678-1626b9bef843, line 2
SQL statement "COPY "f4c96770-f2f3-45e9-8678-1626b9bef843" FROM 'C://Test/Ini.dat'"
PL/pgSQL function ecrs."MIGRATION.DBF_COPY_FROM"(character varying) line 29 at EXECUTE
Both databases have the same properties :
ENCODING = 'UTF8'
TABLESPACE = pg_default
LC_COLLATE ='English_United Kingdom.1252'
LC_CTYPE = 'English_United Kingdom.1252'
CONNECTION LIMIT = -1;
I have tried to use
set client_encoding to 'WIN1252';
or
set client_encoding to 'UTF8'; but it doesn't work.
Do I understand correctly that you write a file with MIGRATION.DBF_COPY_TO and then read it with MIGRATION.DBF_COPY_FROM?
If yes, that's not going to work, because the first function uses the binary format, while the second function uses the text format.
Carriage return characters without following line feed are not allowed in text mode COPY, but in this case it is a consequence of you using different formats for input and output.
I'm trying to compile a postgresql function
CREATE OR REPLACE FUNCTION LoadData(tablename varchar(25), filepath varchar(35))
RETURNS void AS $$
declare
BEGIN
RAISE NOTICE 'Data is being loaded from an external file, please wait...';
COPY tablename FROM filepath DELIMITER ',' CSV HEADER;
RAISE NOTICE 'Data loaded successfully!!';
END;
$$ LANGUAGE plpgsql;
but it's giving an error like this
ERROR: syntax error at or near "filepath"
LINE 9: COPY tablename FROM filepath DELIMITER ',' CSV HEADER;
How to use both the in parameters in the COPY command?
To execute dynamically built SQL queries use execute. format makes it easy and safe
create or replace function loaddata(
tablename varchar(25), filepath varchar(35)
) returns void as $$
begin
raise notice 'data is being loaded from an external file, please wait...';
execute format($copy$
copy %I from %L delimiter ',' csv header
$copy$,
tablename, filepath
);
raise notice 'data loaded successfully!!';
end;
$$ language plpgsql;
http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN
I would like to repeat the following query 8760 times, replacing ‘2’ with 1 to 8760 for every hour in the year. The idea is to create a separate CSV file for each hour for further processing.
COPY
(SELECT *
FROM
public.completedsolarirad2012
WHERE
completedsolarirad2012."UniquetmstmpID" = 2)
TO 'C:\temp\2012hour2.csv' WITH DELIMITER ','
CSV HEADER
I have put together the following function (testing with only a few hours):
CREATE OR REPLACE FUNCTION everyhour()
RETURNS void AS
$BODY$BEGIN
FOR i IN 0..5 LOOP
EXECUTE $x$
COPY (
SELECT *
FROM
public.completedsolarirad2012
WHERE
completedsolarirad2012."UniquetmstmpID" = i
)
TO $concat$ 'C:\temp.' || i::text
|| '.out.csv' WITH DELIMITER ',' CSV HEADER $concat$
$x$;
END LOOP;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION everyhour()
OWNER TO postgres;
I seem to be having two separate problems:
Firstly, I’m getting:
Error: column "i" does not exist.
Secondly, when I test the concatenation statement only by replacing “i” with e.g. “2”, I get:
Error: relative path not allowed for COPY to file
I am the postgres superuser, so I do not understand why I am having this problem.
Note: Removing the $concat$ double-quoting around the concatenation statement gives the following error:
ERROR: syntax error at or near "||"
LINE 9: TO 'C:\temp.' || i::text
I would be very grateful for any help.
Assuming your server OS is Windows.
CREATE OR REPLACE FUNCTION everyhour()
RETURNS void AS
$func$
BEGIN
FOR i IN 0..5 LOOP
EXECUTE '
COPY (
SELECT *
FROM public.completedsolarirad2012 c
WHERE c."UniquetmstmpID" = ' || i || $x$)
TO 'C:/temp.'$x$ || i || $x$'.out.csv' WITH DELIMITER ',' CSV HEADER$x$;
END LOOP;
END
$func$ LANGUAGE plpgsql;
You had one layer of dollar-quoting too many.
You also accidentally concatenated the letter "i" instead of its value.
Use forward-slashes, even with windows. Or you may have to double up the backslashes, depending on your settings. More in this related answer:
PostgreSQL: export resulting data from SQL query to Excel/CSV
Simpler with format()
Since Postgres 9.1 you can use format() to simplify complex concatenations:
CREATE OR REPLACE FUNCTION everyhour()
RETURNS void AS
$func$
BEGIN
FOR i IN 0..5 LOOP
EXECUTE format($x$COPY (
SELECT *
FROM public.completedsolarirad2012 c
WHERE c."UniquetmstmpID" = %1$s)
TO 'C:/temp.%1$s.out.csv' WITH DELIMITER ',' CSV HEADER$x$, i);
END LOOP;
END
$func$ LANGUAGE plpgsql;