I am trying to copy a table's data to a file. I have a small database table in the local windows machine and this code has no problems. When I use this in the development environment (still windows, different database), I get an error.
My standard_conforming_strings is currently off.
This query:
COPY some_table TO 'C:\\temp\\test.csv' WITH CSV HEADER;
Gives this error:
WARNING: nonstandard use of \\ in a string literal
LINE 1: COPY t_table TO 'C:\\temp\\test.csv' WITH CSV HEADER;
^
HINT: Use the escape string syntax for backslashes, e.g., E'\\'.
ERROR: relative path not allowed for COPY to file
********** Error **********
ERROR: relative path not allowed for COPY to file
SQL state: 42602
I have tried:
'C:\temp\test.csv'
'C:\\temp\\test.csv'
'C:/temp/test.csv'
'C:\/temp\/test.csv'
And in all my test, the caret ^ in the error message points at ^'C:.
You are using PostgreSQL on linux not Windows. Just using a Windows client.
COPY is a server side command. It expects a path on the server. That file is on your Windows client PC so the server cannot access it.
Use \copy from psql instead. Or use PgAdmin's CSV importer.
(Newer PostgreSQL versions would give you a HINT about this in the error message.)
Related
I'm familiarizing myself with the standalone version of Datagrip and having a bit of trouble understanding the different approaches to composing SQL via console, external files, scratch files, etc.
I'm managing, referencing the documentation, and am happy to figure things out as such.
However, I'm trying to ingest CSV data into tables via batch files using the Postgres \copy command. Datagrip will execute this command without error but no data is being populated.
This is my syntax, composed and ran in the console view:
\copy tablename from 'C:\Users\username\data_file.txt' WITH DELIMITER E'\t' csv;
Note that the data is tab-separated and stored in a .txt file.
I'm able to use the import functions of Datagrip (via context menu) just fine but I'd like to understand how to issue commands to do similarly.
\copy is a command of the command-line PostgreSQL client psql.
I doubt that Datagrip invokes psql, so it won't be able to use \copy or any other “backslash command”.
You probably have to use Datagrip's import facilities. Or you start using psql.
Ok, but what about the SQL COPY command https://www.postgresql.org/docs/12/sql-copy.html ?
How can I run something like that with datagrip ?
BEGIN;
CREATE TEMPORARY TABLE temp_json(values text) ON COMMIT DROP;
COPY temp_json FROM 'MY_FILE.JSON';
SELECT values->>'aJsonField' as f
FROM (select values::json AS values FROM temp_json) AS a;
COMMIT;
I try to replace 'MY_FILE.JSON' with full path, parameter (?), I put it in sql directory etc.
The data grip answer is :
[2021-05-05 10:30:45] [58P01] ERROR: could not open file '...' for reading : No such file or directory
EDIT :
I know why. RTFM! -_-
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible by the PostgreSQL user (the user ID the server runs as) and the name must be specified from the viewpoint of the server.
Sorry.....
I am trying to import data from a CSV file into PostgreSQL using pgAdmin. However, I am getting an error message when attempting to perform a COPY command.
ERROR: could not open file "/Users/brandoncruz/Desktop/Test File.csv" for reading: Permission denied
HINT: COPY FROM instructs the PostgreSQL server process to read a file. You may want a client-side facility such as psql's \copy.
SQL state: 42501
Below is the code I have attempted to use for the import.
CREATE TABLE roles(
role_id serial PRIMARY KEY,
role_name VARCHAR (255)
);
COPY roles FROM '/Users/brandoncruz/Desktop/Test File.csv' DELIMITER ',' CSV HEADER;
It looks about your permissions. You can change permissions of 'Test File.csv'. I mean Postgres user cannot read your file. After change permissions it must be copied successfully.
You might need to check following path if you are using pgAdmin:
pgAdmin > File > Preferences
Paths > Binary Paths
For postgreSQL binary path, you should find the location of PostgreSQL from your mac and save that:
i.e. C:\Program Files\PostgreSQL\12\bin (Windows)
For mac, check under Applications, Program Files, that depends and changes from Mac to Mac.
I tried to make a variable in SQL statement in Postgresql, but it did not work.
There are many csv files stored under the path. I want to set path in Postgresql that can tell copy command where can find csv files.
SQL statement sample:
\set outpath '/home/clients/ats-dev/'
\COPY licenses (_id, name,number_seats ) FROM :outpath + 'licenses.csv' CSV HEADER DELIMITER ',';
\COPY uploaded_files (_id, added_date ) FROM :outpath + 'files.csv' CSV HEADER DELIMITER ',';
It did not work. I got error: no such files. The two files licneses.csv and files.csv are stored under /home/cilents/ats-dev on Ubuntu. I found some sultion that use "\set file 'license.csv'". It did not work for me becacuse I have many csv files. also I tried to use "from : outpath || 'licenses.csv'". it did not work ether. Appreciate for any helps.
Using 9.3.
It looks like psql does not support :variable substitution withinpsql backslash commands.
test=> \set somevar fred
test=> \copy z from :somevar
:somevar: No such file or directory
so you will need to do this via an external tool like the unix shell. e.g.
for f in *.sql; do
psql -c "\\copy $(basename $f) FROM '$f'"
done
You can try COPY command
\set outpath '\'/home/clients/ats-dev/'
COPY licenses (_id, name,number_seats ) FROM :outpath/licenses.csv' WITH CSV HEADER DELIMITER ',';
COPY uploaded_files (_id, added_date ) FROM :outpath/files.csv' WITH CSV HEADER DELIMITER ',';
Note: Files named in a COPY command are read or written directly by the server, not by the client application. Therefore, they must reside on or be accessible to the database server machine, not the client. They must be accessible to and readable or writable by the PostgreSQL user (the user ID the server runs as), not the client. Similarly, the command specified with PROGRAM is executed directly by the server, not by the client application, must be executable by the PostgreSQL user. COPY naming a file or command is only allowed to database superusers, since it allows reading or writing any file that the server has privileges to access.
Documentation: Postgresql 9.3 COPY
It may have been true when this was originally asked, that psql backslash commands didn't support variable interpolation, but in my PostgreSQL 14 instance that's no longer the case. However, the psql manpage is clear that \copy specifically does not support variable interpolation.
I'm getting this error:
ERROR: syntax error at or near "\"
LINE 1: \COPY "Staging_Budget" FROM 'C:\Users\My.Name\Desktop\RD - F...
^
********** Error **********
when I try to execute this simple command:
\COPY "Staging_Budget" FROM 'C:\Users\My.Name\Desktop\RD - Facilities Management (001321).csv';
Can anyone tell me why that is?
Also, can anyone tell me why none of the examples online have the drive (C:) or the file type (.csv) listed in the filepath like I do?
Postgres version is 9.5 and my OS is Windows 7.
Thank you!
UPDATED:
I'm trying to run this statement instead:
COPY "Staging_Budget" FROM STDIN 'C:\Program Files (x86)\PostgreSQL\9.5\data\csv\RD - Facilities Management (001321).csv';
I read here that I needed to move the CSV file to the postgres CSV file directory. I didn't have one, so I created it in the filepath in the statement above. Now I get this error message:
ERROR: syntax error at or near "'C:\Program Files (x86)\PostgreSQL\9.5\data\csv\RD - Facilities Management (001321).csv'"
LINE 1: COPY "Staging_Budget" FROM STDIN 'C:\Program Files (x86)\Pos...
^
I don't understand why postgres doesn't like the apostrophe when it's given in every example I've found online and in the reference guide.
The problem is that you can either COPY "Staging_Budget" FROM STDIN or COPY "Staging_Budget" FROM 'C:\etc', but not both. Since you're on Windows, you can ignore everything people are saying about cat. You don't want to copy from STDIN, you want to copy from a file.
Also \copy and COPY are not the same thing. In your case it looks like you want COPY (no backslash), which is the source of your original error.
Note that you can only use 'COPY .. FROM' by referencing an external file whenever you're in the server. I mean, the file must be accessible to the postmaster instance.
One technique I always apply in Linux is 'COPY .. FROM STDIN', by issuing a cat before psql.
Take a look at the reference guide:
http://www.postgresql.org/docs/current/static/sql-copy.html
I was able to get this statement to work:
COPY "Staging_Budget" FROM 'C:\Program Files (x86)\PostgreSQL\9.5\data\csv\RD - Facilities Management (001321).csv' (DELIMITER ',');
I guess postgres had a problem with the apostrophe used with STDIN? Also, having the CSV file in a folder called CSV within the Data folder was needed. Lastly, I needed to make sure that "(DELIMITER ',')" was at the end of the statement - even though the reference guide says that a ',' is the default when using CSV files...I'm not sure why I had to explicitly state it as the delimiter.
Using COPY statement of PostgreSQL, we can load data from a text file into data base's table as below:
COPY CME_ERROR_CODES FROM E'C:\\Program Files\\ERROR_CODES\\errcodes.txt' DELIMITER AS '~'
The above statement is run from a machine which has postgresql client where as the server is in another windows machine. Running the above statement is complaining me that ERROR: could not open file "C:\Program Files\ERROR_CODES\errcodes.txt" for reading: No such file or directory.
After some research, i observed that COPY statement is looking for the loader file(errcodes.txt) in the postgresql server's machine at the same path (C:\Program Files\ERROR_CODES). To test this , i have create the same folder structure in the postgresql server's machine and kept the errcodes.txt file in there. Then the COPY statement worked well. It looks very tough constraint for me with COPY statement.
Is there any setting needed to avoid this? or it is the behavior of COPY statement? I didn't find any information on PostgreSQL documents.
here's the standard solution:
COPY foo (i, j, k) FROM stdin;
1<TAB>2<TAB>3
\.
The data must be properly escaped and tab-separated.
Actually, it is in the docs, even in grammar definition you have STDIN... See: http://www.postgresql.org/docs/9.1/static/sql-copy.html
If you're using some programming language with COPY support, you will have pg_putcopy or similar function. So you don't have to worry about escaping and concatenation.
Hints how to do this manually in Python -> Recreating Postgres COPY directly in Python?
The Perl way -> http://search.cpan.org/dist/DBD-Pg/Pg.pm#COPY_support
Hope this helps.
From the documentation
Quote:
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server.
If you want to copy from a local machine file to a server use \copy command.