How to export the resulting data in PostgreSQL to .CSV? - postgresql

I use PostgreSQL 9.4.1
My query:
copy(select * from city) to 'C:\\temp\\city.csv'
copy(select * from city) to E'C:\\temp\\city.csv'
ERROR: relative path not allowed for COPY to file
********** Error **********
ERROR: relative path not allowed for COPY to file SQL state: 42602

As with this case, it seems likely that you are attempting to use copy from a computer other than the one which hosts your database. copy does I/O from the database host machine's local file system only. If you have access to that filesystem, you can adjust your attempt accordingly. Otherwise, you can use the \copy command in psql.

I am using pgAdmin v1.5 . The first query is
select table_name from information_schema.tables where table_catalog = 'ofbiz' order by table_name
Then I press button download, pgAdmin will return a csv file, is result set of first query.

It could be late but i think it can be helpful.
On Windows, make sure the output directory has gain the read/write right for Everyone (or you can specific user name).
Using slash(/) instead of backslash(), example
COPY DT1111 TO 'D:/TEST/DT1111_POST.CSV' DELIMITER ',' CSV HEADER;

TLDR: Make sure you also have write permissions in your copy-to location!
I had the exact same first error, ERROR: relative path not allowed for COPY to file, even though I used '/tmp/db.csv' (which is not a relative path).
In my case, the error message was quite misleading, since I was on the host machine, had an absolute filepath and the location existed. My problem was that I used the bitnami postgres:12 docker image, and the tmp folder in the container belongs to root there, while postgres and psql use the postgres user. My solution was to create an export folder there and transform the ownership to the postgres user:
mkdir /tmp/export
chown postgres:postgres /tmp/export
Then I was able to use COPY tablename TO '/tmp/export/db.csv'; successfully.

Related

Export to CSV file with windows path [duplicate]

I use PostgreSQL 9.4.1
My query:
copy(select * from city) to 'C:\\temp\\city.csv'
copy(select * from city) to E'C:\\temp\\city.csv'
ERROR: relative path not allowed for COPY to file
********** Error **********
ERROR: relative path not allowed for COPY to file SQL state: 42602
As with this case, it seems likely that you are attempting to use copy from a computer other than the one which hosts your database. copy does I/O from the database host machine's local file system only. If you have access to that filesystem, you can adjust your attempt accordingly. Otherwise, you can use the \copy command in psql.
I am using pgAdmin v1.5 . The first query is
select table_name from information_schema.tables where table_catalog = 'ofbiz' order by table_name
Then I press button download, pgAdmin will return a csv file, is result set of first query.
It could be late but i think it can be helpful.
On Windows, make sure the output directory has gain the read/write right for Everyone (or you can specific user name).
Using slash(/) instead of backslash(), example
COPY DT1111 TO 'D:/TEST/DT1111_POST.CSV' DELIMITER ',' CSV HEADER;
TLDR: Make sure you also have write permissions in your copy-to location!
I had the exact same first error, ERROR: relative path not allowed for COPY to file, even though I used '/tmp/db.csv' (which is not a relative path).
In my case, the error message was quite misleading, since I was on the host machine, had an absolute filepath and the location existed. My problem was that I used the bitnami postgres:12 docker image, and the tmp folder in the container belongs to root there, while postgres and psql use the postgres user. My solution was to create an export folder there and transform the ownership to the postgres user:
mkdir /tmp/export
chown postgres:postgres /tmp/export
Then I was able to use COPY tablename TO '/tmp/export/db.csv'; successfully.

Postgresql cannot find file name specified in copy command

I try this:
COPY gemeenten
FROM 'D:\CBS_woningcijfers_2014.csv'
DELIMITER ';' CSV
and get this:
ERROR: could not open file "D:\CBS_woningcijfers_2014.csv" for reading: No such file or directory
I doubled the backslashes, tried an E string, replaced \ by /, used " instead of ' but now I've run out of options. I am sure the file exists. Anybody any idea?
If the file and the PostgreSQL database are on the same machine, then the path and/or name of the file are not correct.
If the file is on your local machine and the database is on another, you cannot use the COPY command in SQL. You have two main choices to make this work:
1) Use psql \copy from your local machine. The syntax is similar, but it will transfer from your local to the remote. The docs are pretty helpful: https://www.postgresql.org/docs/9.5/static/app-psql.html#APP-PSQL-META-COMMANDS-COPY
2) Upload the file to the remote machine and then execute your command. Just make sure you are referencing the correct path and filename.

Postgres ERROR: could not open file for reading: Permission denied

Computer: Mac OS X, version 10.8
Database: Postgres
Trying to import csv file into postgres.
pg> copy items_ordered from '/users/darchcruise/desktop/items_ordered.csv' with CSV;
ERROR: could not open file "/users/darchcruise/desktop/items_ordered.csv" for reading: Permission denied
Then I tried
$> chown postgres /users/darchcruise/desktop/items_ordered.csv
chown: /users/darchcruise/desktop/items_ordered.csv: Operation not permitted
Lastly, I tried
$> ls -l
-rw-r--r-- 1 darchcruise staff 1016 Oct 18 21:04 items_ordered.csv
Any help is much appreciated!
Assuming the psql command-line tool, you may use \copy instead of copy.
\copy opens the file and feeds the contents to the server, whereas copy tells the server the open the file itself and read it, which may be problematic permission-wise, or even impossible if client and server run on different machines with no file sharing in-between.
Under the hood, \copy is implemented as COPY FROM stdin and accepts the same options than the server-side COPY.
Copy the CSV file to /tmp
For me this solved the issue.
chmod a+rX /users/darchcruise/ /users/darchcruise/desktop /users/darchcruise/desktop/items_ordered.csv
This will change access rights for your folder. Note that everyone will be able to read your file.
You can't use chown being a user without administrative rights.
Also consider learning umask to ease creation of shared files.
Copy your CSV file into the /tmp folder
Files named in a COPY command are read or written directly by the server, not by the client application. Therefore, they must reside on or be accessible to the database server machine, not the client. They must be accessible to and readable or writable by the PostgreSQL user (the user ID the server runs as), not the client. COPY naming a file is only allowed to database superusers, since it allows reading or writing any file that the server has privileges to access.
I had the issue when I was trying to export data from a remote server into the local disk. I hadn't realised that SQL copy actually is executed on the server and that it tries to write to a server folder. Instead the correct thing to do was to use \copy which is the psql command and it writes to the local file system as I expected. http://www.postgresql.org/message-id/CAFjNrYsE4Za_KWzmfgN1_-MG7GTw_vpMRxPk=OEjAiLqLskxdA#mail.gmail.com
Perhaps that might be useful to someone else too.
Another way to do this, if you have pgAdmin and are comfortable using the GUI is to go the table in the schema and right click on the table you wish to import the file to and select "Import" browse your computer for the file, select the type your file is, the columns you want the data to be imputed into, and then select import.
That was done using pgAdmin III and the 9.4 version of PostgreSQL
I resolved the same issue with a recursive chown on the parent folder:
sudo chown -R postgres:postgres /home/my_user/export_folder
(my export being in /home/my_user/export_folder/export_1.csv)
for macbook first i opened terminal then type
open /tmp
or in finder directory you directly enter command+shift+g then type /tmp in go to the folder.
it opens temp folder in finder. then i paste copied csv file into this folder.then again i go to postgres terminal and typed below command and then it is copied my csv data into db table
\copy recharge_operator FROM '/private/tmp/operator.csv' DELIMITER ',' CSV;
COPY your table (Name, Latitude, Longitude) FROM 'C:\Temp\your file.csv' DELIMITERS ',' CSV HEADER;
Use c:\Temp\"Your File"\.
For me it worked to simply to add sudo (or run as root) for the chown command:
sudo chown postgres /users/darchcruise/desktop/items_ordered.csv
You must grant the pg_read_server_files permission to the user if you are not using postgres superuser.
Example:
GRANT pg_read_server_files TO my_user WITH ADMIN OPTION;
just in case you're facing this problem under windows 10 , add the group of users "youcomputer\Users" on the security Tab and grant it full control , that solved my issue
I had the same error message but was using psycopg2 to communicate with PostgreSQL. I fixed the permission issues by using the functions copy_from and copy_expert that will open the file on the client side as the user running the python script and feed the data to the database over STDIN.
Refer to this link for further information.
This answer is only for Linux Beginners.
Assuming initially the DB user didn't have file/folder(directory) permission on the client side.
Let's constrain ourselves to the following:
User: postgres
Purpose: You wanted to (write to / read from) a specific folder
Tool: psql
Connected to a specific database: YES
FILE_PATH: /home/user/training/sql/csv_example.csv
Query: \copy (SELECT * FROM table_name TO FILE_PATH, DELIMITER ',' CSV HEADER;
Actual Results: After running the query you got an error : Permission Denied
Expected Results: COPY COUNT_OF_ROWS_COPIED
Here are the steps I'd follow to try and resolve it.
Confirm the FILE_PATH permissions on your File system.
Inside a terminal to view the permissions for a file/folder you need to long list them by entering the command ls -l.
The output has a section that shows sth like this -> drwxrwxr-x
Which is interpreted in the following way:
TYPE | OWNER RIGHTS | GROUP RIGHTS | USER RIGHTS
rwx (r: Read, W: Write, X: Execute)
TYPE (1 Char) = d: directory, -: file
OWNER RIGHTS (3 Chars after TYPE)
GROUP RIGHTS (3 Chars after OWNER)
USER RIGHTS (3 Chars after GROUP)
If permissions are not enough (Ensure that a user can at least enter all folders in the path you wanted path) - x.
This means for FILE_PATH, All the directories (home , user, training, sql) should have at least an x in the USER RIGHTS.
Change permissions for all parent folders that you need to enter to have a x. You can use chmod rights_you_want parent_folder
Assuming /training/ didn't have an execute permission.
I'd go the user folder and enter chmod a+x training
Change the destination folder/directory to have a w if you want to write to it. or at least a r if you want to read from it
Assuming /sql didn't have a write permission.
I would now chmod a+w sql
Restart the postgresql server sudo systemctl restart postgresql
Try again.
This would most probably help you now get a successful expected result.
On Linux you can fix this by giving the postgres user read/write/execute permissions on the target directory. Eg:
setfacl -m u:postgres:rwx /home/hi
I just copied the source csv file to another folder in which you have more permissions (C:/temp), and it worked fine.
May be You are using pgadmin by connecting remote host then U are trying to update there from your system but it searches for that file in remote system's file system... its the error wat I faced May be its also for u check it

Postgres COPY FROM csv file- No such file or directory

I'm trying to import a (rather large) .txt file into a table geonames in PostgreSQL 9.1. I'm in the /~ directory of my server, with a file named US.txt placed in that directory. I set the search_path variable to geochat, the name of the database I'm working in. I then enter this query:
COPY geonames
FROM 'US.txt',
DELIMITER E'\t',
NULL 'NULL');
I then receive this error:
ERROR: could not open file "US.txt" for reading: No such file or directory.
Do I have to type in \i US.txt or something similar first, or should it just get it from the present working directory?
Maybe a bit late, but hopefully useful:
Use \copy instead
https://wiki.postgresql.org/wiki/COPY
jvdw
A couple of misconceptions:
1.
I'm in the /~ directory of my server
There is no directory /~. It's either / (root directory) or ~ (home directory of current user). It's also irrelevant to the problem.
2.
I set the search_path variable to geochat, the name of the database I'm working in
The search_path has nothing to do with the name of the database. It's for schemas inside the current database. You probably need to reset this.
3.
You are required to use the absolute path for your file. As documented in the manual here:
filename
The absolute path name of the input or output file.
4.
DELIMITER: just noise.
The default is a tab character in text format
5.
NULL: It's rather uncommon to use the actual string 'NULL' for a NULL value. Are you sure?
The default is \N (backslash-N) in text format, and an unquoted empty string in CSV format.
My guess (after resetting search_path - or you schema-qualify the table name):
COPY geonames FROM '/path/to/file/US.txt';
The paths are relative to the PostgreSQL server, not the psql client.
Assuming you are running PostgreSQL 9.4, you can put US.txt in the directory /var/lib/postgresql/9.4/main/.
Another option is to pipe it in from stdin:
cat US.txt | psql -c "copy geonames from STDIN WITH (FORMAT csv);"
if you're running your COPY command from a script, you can have a step in the script that creates the COPY command with the correct absolute path.
MYPWD=$(pwd)
echo "COPY geonames FROM '$MYPWD/US.txt', DELIMITER E'\t';"
MYPWD=
you can then run this portion into a file and execute it
./step_to_create_COPY_with_abs_path.sh >COPY_abs_path.sql
psql -f COPY_abs_path.sql -d your_db_name

PostgreSQL: export resulting data from SQL query to Excel/CSV

I need to export the resulting data from a query in PostgreSQL to Excel/CSV.
I use PostgreSQL 8.2.11.
SQL error:
ERROR: relative path not allowed for COPY to file
In statement:
COPY (select distinct(m_price) from m_product)TO '"c:\auto_new.txt"';
Example with Unix-style file name:
COPY (SELECT * FROM tbl) TO '/var/lib/postgres/myfile1.csv' format csv;
Read the manual about COPY (link to version 8.2).
You have to use an absolute path for the target file. Be sure to double quote file names with spaces. Example for MS Windows:
COPY (SELECT * FROM tbl)
TO E'"C:\\Documents and Settings\\Tech\Desktop\\myfile1.csv"' format csv;
In PostgreSQL 8.2, with standard_conforming_strings = off per default, you need to double backslashes, because \ is a special character and interpreted by PostgreSQL. Works in any version. It's all in the fine manual:
filename
 The absolute path name of the input or output file. Windows users might need to use an E'' string and double backslashes used as path separators.
Or the modern syntax with standard_conforming_strings = on (default since Postgres 9.1):
COPY tbl -- short for (SELECT * FROM tbl)
TO '"C:\Documents and Settings\Tech\Desktop\myfile1.csv"' (format csv);
Or you can also use forward slashes for filenames under Windows.
An alternative is to use the meta-command \copy of the default terminal client psql.
You can also use a GUI like pgadmin and copy / paste from the result grid to Excel for small queries.
Closely related answer:
Copy results from a PostgreSQL view in one DB to a table in another
Similar solution for MySQL:
Exporting MYSQL data into Excel/CSV via php
In PostgreSQL 9.4 to create to file CSV with the header in Ubuntu:
COPY (SELECT * FROM tbl) TO '/home/user/Desktop/result_sql.csv' WITH CSV HEADER;
Note: The folder must be writable.
This worked for me:
COPY (SELECT * FROM table)
TO E'C:\\Program Files (x86)\\PostgreSQL\\8.4\\data\\try.csv';
In my case the problem was with the writing permission to a special folder (though I work as administrator), after changing the path to the original data folder under PostgreSQL I had success.
Several GUI tools like Squirrel, SQL Workbench/J, AnySQL, ExecuteQuery can export to Excel files.
Most of those tools are listed in the PostgreSQL wiki:
http://wiki.postgresql.org/wiki/Community_Guide_to_PostgreSQL_GUI_Tools
If you have error like "ERROR: could not open server file "/file": Permission denied" you can fix it that:
Ran through the same problem, and this is the solution I found:
Create a new folder (for instance, tmp) under /home
$ cd /home
make postgres the owner of that folder
$ chown -R postgres:postgres tmp
copy in tmp the files you want to write into the database, and make sure they also are owned by postgres.
That's it. You should be in business after that.
The correct script for postgres (Ubuntu) is:
COPY (SELECT * FROM tbl) TO '/var/lib/postgres/myfile1.csv';