I would like to export a single Postgres table's data into a .csv file. Can anyone give me an example of how to do that?
In psql:
\copy tablename to 'filename' csv;
First, log into the PostgreSQL console via the command line with the psql command.
To export:
\connect database_name;
\copy my_table TO 'my_table.csv' CSV;
\q
To import:
\connect database_name;
\copy my_table FROM 'my_table.csv' DELIMITER ',' CSV;
\q
Done!
Or, from a shell script!
export PGPASSWORD=dbpass
psql --dbname=mydb --username=dbuser --host=127.0.0.1 -c "COPY (SELECT * FROM widget) TO stdout DELIMITER ',' CSV HEADER" > export.csv
Bonus Advice
Use pgcli, it's way better than psql
When logged into psql:
COPY tablename TO 'filename';
For more details, see this: http://www.postgresql.org/docs/current/static/sql-copy.html
Related
I am a beginner in PostgreSQL. I have connected to postgres using
sudo -u postgres psql
Then I connected to 'test' database as 'postgres' user using the following command:
postgres=# \c test
Now when I try to export the results to home directory using the following command;
test=# \copy (select * from person left join car on
person.car_id=car.id) to '/home/navdeep/Downloads/data.csv' delimiter
',' csv header;
I get the following error;
/home/navdeep/Downloads/data.csv: Permission denied
What could be the reason. Please advise. Thanks.
Could you try saving to the temp folder? I don't think you'll have any permission issues there.
I have a table tbl_a in schema sch_a in a database db_a
I want to copy this table tbl_a to schema sch_b in database db_b. Any suggestion?
You can use copy :
sudo -u postgres psql --dbname=db_a -c "\copy (SELECT * FROM sch_a.tbl_a ) TO '/path/`date +%Y-%m-%d`_sch_a_tbl_a.csv' CSV DELIMITER '|' HEADER"
Then import it like so :
sudo -u postgres psql --dbname=db_b -c "\copy sch_b.tbl_b FROM '/path/`date +%Y-%m-%d`_sch_a_tbl_a.csv' DELIMITER '|' CSV"
You can also use dblink
I'm trying to copy the table before deleting it using pgAdmin 4 v4. Here is the query I used.
copy table_name to 'C:\tmp\backup.csv' DELIMITER ',' CSV HEADER;
But I got error message.
ERROR: must be superuser or a member of the pg_write_server_files role to COPY to a file
HINT: Anyone can COPY to stdout or from stdin. psql's \copy command also works for anyone.
SQL state: 42501
I used
\copy table_name to 'C:\tmp\backup.csv' DELIMITER ',' CSV HEADER;
query, but there was error at \copy.
Does anyone know how to copy table as a file (csv/sql)?
You can't use \copy in pgAdmin, because \copy is a psql command, whereas COPY is a SQL command.
I'd like to execute a query from the shell (not in the interactive psql client) and have it print the CSV or TSV representation of the output to STDOUT. How do you do that with psql or one of the PostgreSQL command-line tools?
If you are using PostgreSQL 8.2 or newer, use this for CSV:
psql -c "COPY (<select query>) TO STDOUT WITH CSV"
and this of TSV, with proper NULLs:
psql -c "COPY (<select query>) TO STDOUT WITH NULL AS ''"
The CSV form will properly quote any fields that contain the double-quote character. See the PostgreSQL documentation of your specific version for more details and options for COPY.
Starting from Bohemian's answer, I found these flags useful:
psql my_database -U myuser -A -F , -X -t -f /path/to/query.sql -o /path/to/output.csv
Unaligned output mode: -A
Use comma as field delimiter: -F ,
Do not read psqlrc: -X
Tuples only (no header/footer): -t
File containing SQL query: -f
Output file: -o
EDITED: Using -F
Use commas via -F and use "unaligned table output mode" -A:
psql my_database -U myuser -A -F , -c "select * from mytable"
To specify a tsv use the delimiter '\t'
psql my_database -U myuser -F'\t' --no-align -f mysqlfile.sql -o outputfile.tsv
To specify a csv use the delimiter ','
psql my_database -U myuser -F',' --no-align -f mysqlfile.sql -o outputfile.csv
The simplest way (using psql) seems to be by using --csv flag:
psql --csv -c "SELECT * FROM sometable"
Also possible is the copy command which allows you to specify header, delimiters and quoting options
psql my_database -U my_user -c "copy (select a.id,b.id from my_table_a as a inner join my_table_b as b on b.id = a.id) to STDOUT"
You can specify the field separator with the -F command line parameter to psql
Export AS TSV WITH HEADER
You can include the HEADER as follows:
\COPY (SELECT * FROM tca) TO '/.../metab/tca.tsv' WITH DELIMITER E'\t' CSV HEADER;
\COPY (SELECT * FROM tca) TO '/...a/metab/tca.tsv' WITH NULL AS '' DELIMITER E'\t' CSV HEADER;
E.g. (PSQL):
[metabolism]# \COPY (SELECT * FROM tca) TO '/mnt/Vancouver/programming/data/metabolism/tca.tsv' WITH NULL AS '' DELIMITER E'\t' CSV HEADER;
COPY 22
BASH:
[victoria#victoria tsv]$ pwd
/mnt/Vancouver/programming/data/metabolism/tsv
[victoria#victoria tsv]$ head -n3 tca.tsv
uuid src tgt rel rel_type
878b87de-0ca8-49a8-9f77-a24353e251d2 oxalosuccinic acid oxoglutaric acid 1.1.1.42 2
7fd9cf88-495b-491b-956e-294f19097923 isocitric acid oxoglutaric acid 1.1.1.41 2
[victoria#victoria csv]$
I'm using psql's \dt to list all tables in a database and I need to save the results.
What is the syntax to export the results of a psql command to a file?
From psql's help (\?):
\o [FILE] send all query results to file or |pipe
The sequence of commands will look like this:
[wist#scifres ~]$ psql db
Welcome to psql 8.3.6, the PostgreSQL interactive terminal
db=>\o out.txt
db=>\dt
Then any db operation output will be written to out.txt.
Enter '\o' to revert the output back to console.
db=>\o
The psql \o command was already described by jhwist.
An alternative approach is using the COPY TO command to write directly to a file on the server. This has the advantage that it's dumped in an easy-to-parse format of your choice -- rather than psql's tabulated format. It's also very easy to import to another table/database using COPY FROM.
NB! This requires superuser or pg_write_server_files privileges and will write to a file on the server.
Example: COPY (SELECT foo, bar FROM baz) TO '/tmp/query.csv' (format csv, delimiter ';')
Creates a CSV file with ';' as the field separator.
As always, see the documentation for details
Use o parameter of pgsql command.
-o, --output=FILENAME send query results to file (or |pipe)
psql -d DatabaseName -U UserName -c "SELECT * FROM TABLE" -o /root/Desktop/file.txt
\copy which is a postgres command can work for any user. Don't know if it works for \dt or not, but general syntax is reproduced from the following link Postgres SQL copy syntax
\copy (select * from tempTable limit 100) to 'filenameinquotes' with header delimiter as ','
The above will save the output of the select query in the filename provided as a csv file
EDIT:
For my psql server the following command works this is an older version v8.5
copy (select * from table1) to 'full_path_filename' csv header;
Use the below query to store the result in a CSV file
\copy (your query) to 'file path' csv header;
Example
\copy (select name,date_order from purchase_order) to '/home/ankit/Desktop/result.csv' cvs header;
Hope this helps you.
If you got the following error
ufgtoolspg=> COPY (SELECT foo, bar FROM baz) TO '/tmp/query.csv' (format csv, delimiter ';');
ERROR: must be superuser to COPY to or from a file
HINT: Anyone can COPY to stdout or from stdin. psql's \copy command also works for anyone.
you can run it in this way:
psql somepsqllink_or_credentials -c "COPY (SELECT foo, bar FROM baz) TO STDOUT (format csv, delimiter ';')" > baz.csv
COPY tablename TO '/tmp/output.csv' DELIMITER ',' CSV HEADER;
this command is used to store the entire table as csv
I assume that there exist some internal psql command for this, but you could also run the script command from util-linux-ng package:
DESCRIPTION
Script makes a typescript of everything printed on your terminal.
This approach will work with any psql command from the simplest to the most complex without requiring any changes or adjustments to the original command.
NOTE: For Linux servers.
Save the contents of your command to a file
MODEL
read -r -d '' FILE_CONTENT << 'HEREDOC'
[COMMAND_CONTENT]
HEREDOC
echo -n "$FILE_CONTENT" > sqlcmd
EXAMPLE
read -r -d '' FILE_CONTENT << 'HEREDOC'
DO $f$
declare
curid INT := 0;
vdata BYTEA;
badid VARCHAR;
loc VARCHAR;
begin
FOR badid IN SELECT some_field FROM public.some_base LOOP
begin
select 'ctid - '||ctid||'pagenumber - '||(ctid::text::point) [0]::bigint
into loc
from public.some_base where some_field = badid;
SELECT file||' '
INTO vdata
FROM public.some_base where some_field = badid;
exception
when others then
raise notice 'Block/PageNumber - % ',loc;
raise notice 'Corrupted id - % ', badid;
--return;
end;
end loop;
end;
$f$;
HEREDOC
echo -n "$FILE_CONTENT" > sqlcmd
Run the command
MODEL
sudo -u postgres psql [some_db] -c "$(cat sqlcmd)" >>sqlop 2>&1
EXAMPLE
sudo -u postgres psql some_db -c "$(cat sqlcmd)" >>sqlop 2>&1
View/track your command output
cat sqlop
Done! Thanks! =D
Approach for docker
via psql command
docker exec -i %containerid% psql -U %user% -c '\dt' > tables.txt
or query from sql file
docker exec -i %containerid% psql -U %user% < file.sql > data.txt