Using COPY statement of PostgreSQL, we can load data from a text file into data base's table as below:
COPY CME_ERROR_CODES FROM E'C:\\Program Files\\ERROR_CODES\\errcodes.txt' DELIMITER AS '~'
The above statement is run from a machine which has postgresql client where as the server is in another windows machine. Running the above statement is complaining me that ERROR: could not open file "C:\Program Files\ERROR_CODES\errcodes.txt" for reading: No such file or directory.
After some research, i observed that COPY statement is looking for the loader file(errcodes.txt) in the postgresql server's machine at the same path (C:\Program Files\ERROR_CODES). To test this , i have create the same folder structure in the postgresql server's machine and kept the errcodes.txt file in there. Then the COPY statement worked well. It looks very tough constraint for me with COPY statement.
Is there any setting needed to avoid this? or it is the behavior of COPY statement? I didn't find any information on PostgreSQL documents.
here's the standard solution:
COPY foo (i, j, k) FROM stdin;
1<TAB>2<TAB>3
\.
The data must be properly escaped and tab-separated.
Actually, it is in the docs, even in grammar definition you have STDIN... See: http://www.postgresql.org/docs/9.1/static/sql-copy.html
If you're using some programming language with COPY support, you will have pg_putcopy or similar function. So you don't have to worry about escaping and concatenation.
Hints how to do this manually in Python -> Recreating Postgres COPY directly in Python?
The Perl way -> http://search.cpan.org/dist/DBD-Pg/Pg.pm#COPY_support
Hope this helps.
From the documentation
Quote:
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server.
If you want to copy from a local machine file to a server use \copy command.
Related
I'm familiarizing myself with the standalone version of Datagrip and having a bit of trouble understanding the different approaches to composing SQL via console, external files, scratch files, etc.
I'm managing, referencing the documentation, and am happy to figure things out as such.
However, I'm trying to ingest CSV data into tables via batch files using the Postgres \copy command. Datagrip will execute this command without error but no data is being populated.
This is my syntax, composed and ran in the console view:
\copy tablename from 'C:\Users\username\data_file.txt' WITH DELIMITER E'\t' csv;
Note that the data is tab-separated and stored in a .txt file.
I'm able to use the import functions of Datagrip (via context menu) just fine but I'd like to understand how to issue commands to do similarly.
\copy is a command of the command-line PostgreSQL client psql.
I doubt that Datagrip invokes psql, so it won't be able to use \copy or any other “backslash command”.
You probably have to use Datagrip's import facilities. Or you start using psql.
Ok, but what about the SQL COPY command https://www.postgresql.org/docs/12/sql-copy.html ?
How can I run something like that with datagrip ?
BEGIN;
CREATE TEMPORARY TABLE temp_json(values text) ON COMMIT DROP;
COPY temp_json FROM 'MY_FILE.JSON';
SELECT values->>'aJsonField' as f
FROM (select values::json AS values FROM temp_json) AS a;
COMMIT;
I try to replace 'MY_FILE.JSON' with full path, parameter (?), I put it in sql directory etc.
The data grip answer is :
[2021-05-05 10:30:45] [58P01] ERROR: could not open file '...' for reading : No such file or directory
EDIT :
I know why. RTFM! -_-
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible by the PostgreSQL user (the user ID the server runs as) and the name must be specified from the viewpoint of the server.
Sorry.....
I am trying to import a CSV file into postgresSQL, however, I keep getting the error that no such file exists or directory.
this is the line of code I execute copy mu_data from
copy mu_data from 'users/mysurname/Desktop/FILE.CSV' DELIMITER ',' CSV
HEADER;
Can anyone suggest how to fix this?
copy is a command run on the server side. So unless your Postgres server happens to be on your localhost, the file very likely doesn't exist from the view of the server.
So one solution is you to transfer the file to the servers filesystem somehow. Or, if you're using the psql command line tool (or at least can use it for this task), you can use the \copy command there.
I try this:
COPY gemeenten
FROM 'D:\CBS_woningcijfers_2014.csv'
DELIMITER ';' CSV
and get this:
ERROR: could not open file "D:\CBS_woningcijfers_2014.csv" for reading: No such file or directory
I doubled the backslashes, tried an E string, replaced \ by /, used " instead of ' but now I've run out of options. I am sure the file exists. Anybody any idea?
If the file and the PostgreSQL database are on the same machine, then the path and/or name of the file are not correct.
If the file is on your local machine and the database is on another, you cannot use the COPY command in SQL. You have two main choices to make this work:
1) Use psql \copy from your local machine. The syntax is similar, but it will transfer from your local to the remote. The docs are pretty helpful: https://www.postgresql.org/docs/9.5/static/app-psql.html#APP-PSQL-META-COMMANDS-COPY
2) Upload the file to the remote machine and then execute your command. Just make sure you are referencing the correct path and filename.
I'm getting this error:
ERROR: syntax error at or near "\"
LINE 1: \COPY "Staging_Budget" FROM 'C:\Users\My.Name\Desktop\RD - F...
^
********** Error **********
when I try to execute this simple command:
\COPY "Staging_Budget" FROM 'C:\Users\My.Name\Desktop\RD - Facilities Management (001321).csv';
Can anyone tell me why that is?
Also, can anyone tell me why none of the examples online have the drive (C:) or the file type (.csv) listed in the filepath like I do?
Postgres version is 9.5 and my OS is Windows 7.
Thank you!
UPDATED:
I'm trying to run this statement instead:
COPY "Staging_Budget" FROM STDIN 'C:\Program Files (x86)\PostgreSQL\9.5\data\csv\RD - Facilities Management (001321).csv';
I read here that I needed to move the CSV file to the postgres CSV file directory. I didn't have one, so I created it in the filepath in the statement above. Now I get this error message:
ERROR: syntax error at or near "'C:\Program Files (x86)\PostgreSQL\9.5\data\csv\RD - Facilities Management (001321).csv'"
LINE 1: COPY "Staging_Budget" FROM STDIN 'C:\Program Files (x86)\Pos...
^
I don't understand why postgres doesn't like the apostrophe when it's given in every example I've found online and in the reference guide.
The problem is that you can either COPY "Staging_Budget" FROM STDIN or COPY "Staging_Budget" FROM 'C:\etc', but not both. Since you're on Windows, you can ignore everything people are saying about cat. You don't want to copy from STDIN, you want to copy from a file.
Also \copy and COPY are not the same thing. In your case it looks like you want COPY (no backslash), which is the source of your original error.
Note that you can only use 'COPY .. FROM' by referencing an external file whenever you're in the server. I mean, the file must be accessible to the postmaster instance.
One technique I always apply in Linux is 'COPY .. FROM STDIN', by issuing a cat before psql.
Take a look at the reference guide:
http://www.postgresql.org/docs/current/static/sql-copy.html
I was able to get this statement to work:
COPY "Staging_Budget" FROM 'C:\Program Files (x86)\PostgreSQL\9.5\data\csv\RD - Facilities Management (001321).csv' (DELIMITER ',');
I guess postgres had a problem with the apostrophe used with STDIN? Also, having the CSV file in a folder called CSV within the Data folder was needed. Lastly, I needed to make sure that "(DELIMITER ',')" was at the end of the statement - even though the reference guide says that a ',' is the default when using CSV files...I'm not sure why I had to explicitly state it as the delimiter.
I'm trying to import a csv file using the COPY FROM command with postgres.
The db is stored on a linux server, and my data is stored locally, i.e. C:\test.csv
I keep getting the error:
ERROR: could not open file "C:\test.csv" for reading: No such file or directory
SQL state: 58P01
I know that I need to use the absolute path for the filename that the server can see, but everything I try brings up the same error
Can anyone help please?
Thanks
Quote from the PostgreSQL manual:
The file must be accessible to the server and the name must be specified from the viewpoint of the server
So you need to copy the file to the server before you can use COPY FROM.
If you don't have access to the server, you can use psql's \copy command which is very similar to COPY FROM but works with local files. See the manual for details.