I want to send data from my local postgresql to a remote server in csv form..
something similar to
copy (select * from table) to /home/ubuntu/a.csv with csv
But in place of local direcory, I want to take this csv dump in other server
Use the psql client's \copy feature; this does exactly what you want.
As far as I know copy command reads/writes from local path/location. To generate the output file on the remote server, you need to use a script(bash/python) to execute copy command on remote server.
Related
I exported a database to a folder in a local ftp server (using pg_dump) in order to save some space on my computer. Now I want to connect to the generated db.sql file but without importing it again.
In PgAdmin I've created a new database, opened the psql tool and executed \i /path/todb/db.sql, which imports the database into my computer. However, I just need a remote connection over ftp.
I'll appreciate any help.
Thanks.
I would like to import a table from the server computer into a Client computer using the copy command. I know this is a recurring issue for users, but I have not been able to get an answer to this particular one and it's also a different scenario, and I believe this to be common.
I used a copy command to copy a Table from the server to the client computer using the code below:
COPY (Select * from Table_Name) TO 'C:\somedirectory\file.csv' DELIMITER ',' CSV HEADER;
However, I got the following
ERROR: relative path not allowed for COPY to file
My question is: How do I use the correct COPY command to copy from the server computer to the client computer in Postgres.
Thank you in anticipation
Please check if your user has read/write access to the destination folder.
This is one thread I found, see if it helps
https://dba.stackexchange.com/questions/158466/relative-path-for-psql-copy-file
https://postgrespro.com/list/thread-id/1116997
Try with network through access using client public IP.
How do I use the correct COPY command to copy from the server computer to the client computer in Postgres
You simply can't.
Which is clearly stated in the manual
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible by the PostgreSQL user (the user ID the server runs as) and the name must be specified from the viewpoint of the server
(emphasis mine)
You need to use psql's \copy command or any other export tool that works on the client side.
I'm wondering how I would UNLOAD a file from a SQL Anywhere v10 database onto a client computer. I have multiple servers (30+) at different locations, this is the query I have been using to UNLOAD on a local server:
UNLOAD
SELECT tran, id, amount, date, collection, impacts, type
FROM transactions
ORDER BY tran_num
TO 'C:\Users\administrator\Desktop\Clinic.txt' DELIMITED BY '|'
APPEND ON
QUOTES OFF
I'm looking to modify this to be able to unload from the remote servers but the file to generate at my local/client location where I am sending the query. I am using DtSQL to connect to the database remotely. Anyone have a solution?
share a folder on the client machine and then use the full network path to that client's folder in the unload statement. like '\\targetPC\SharedFolder\Clinic.txt'
or
share a folder on the DB server and then copy the file to your target machine after the unload
I am trying to create a query that will copy data from a CSV file that is located on my computer to a Postgres DB that is on a different computer.
Our Postgres DB is located on another computer, and I work on my own to import and query data. I have successfully copied data from the CSV file on MY computer TO the DB in PSQL Console using the following:
\COPY table_name FROM 'c:\path\to\file.csv' CSV DELIMITER E'\t' HEADER;
But when writing a query using the SQL Editor, I use the same code above without the '\' in the beginning. I get the following error:
ERROR: could not open file "c:\pgres\dmi_vehinventory.csv" for reading: No such file or directory
********** Error **********
ERROR: could not open file "c:\pgres\dmi_vehinventory.csv" for reading: No such file or directory
SQL state: 58P01
I assume the query is actually trying to find the file on the DB's computer rather than my own.
How do I write a query that tells Postgres to look for the file on MY computer rather than the DB's computer?
Any help will be much appreciated !
\COPY is a correct way if you want to upload file from local computer (computer where you've stared psql)
COPY is correct when you want to upload on remote host from remote directory
here is an example, i've connected with psql to remote server:
test=# COPY test(i, i1, i3)
FROM './test.csv' WITH DELIMITER ',';
ERROR: could not open file "./test.csv" for reading: No such file
test=# \COPY test(i, i1, i3)
FROM './test.csv' WITH DELIMITER ',';
test=# select * from test;
i | i1 | i3
---+----+----
1 | 2 | 3
(1 row)
There are several common misconceptions when dealing with PostgreSQL's COPY command.
Even though psql's \COPY FROM '/path/to/file/on/client' command has identical syntax (other than the backslash) to the backend's COPY FROM '/path/to/file/on/server' command, they are totally different. When you include a backslash, psql actually rewrites it to a COPY FROM STDIN command instead, and then reads the file itself and transfers it over the connection.
Executing a COPY FROM 'file' command tells the backend to itself open the given path and load it into a given table. As such, the file must be mapped in the server's filesystem and the backend process must have the correct permissions to read it. However, the upside of this variant is that it is supported by any postgresql client that supports raw sql.
Successfully executing a COPY FROM STDIN places the connection into a special COPY_IN state during which an entirely different (and much simpler) sub-protocol is spoken between the client and server, which allows for data (which may or may not come from a file) to be transferred from the client to the server. As such, this command is not well supported outside of libpq, the official client library for C. If you aren't using libpq, you may or may not be able to use this command, but you'll have to do your own research.
COPY FROM STDIN/COPY TO STDOUT doesn't really have anything to do with standard input or standard output; rather the client needs to speak the sub-protocol on the database connection. In the COPY IN case, libpq provides two commands, one to send data to the backend, and another to either commit or roll back the operation. In the COPY OUT case, libpq provides one function that receives either a row of data or an end of data marker.
I don't know anything about SQL Editor, but it's likely that issuing a COPY FROM STDIN command will leave the connection in an unusable state from its point of view, especially if it's connecting via an ODBC driver. As far as I know, ODBC drivers for PostgreSQL do not support COPY IN.
I've been using JDBC with a local Postgres DB to copy data from CSV files into the database with the Postgres COPY command. I use Java to parse the existing CSV file into a CSV format matches the tables in the DB. I then save this parsed CSV to my local disk. I then have JDBC execute a COPY command using the parsed CSV to my local DB. Everything works as expected.
Now I'm trying to perform the same process on a Postgres DB on a remote server using JDBC. However, when JDBC tries to execute the COPY I get
org.postgresql.util.PSQLException: ERROR: could not open file "C:\data\datafile.csv" for reading: No such file or directory
Am I correct in understanding that the COPY command tells the DB to look locally for this file? I.E. the remote server is looking on its C: drive (doesn't exist).
If this is the case, is there anyway to indicate to the copy command to look on my computer rather than "locally" on the remote machine? Reading through the copy documentation I didn't find anything that indicated this functionality.
If the functionality doesn't exist, I'm thinking of just populating the whole database locally then copying to database to the remote server but just wanted to check that I wasn't missing anything.
Thanks for your help.
Create your sql file as follows on your client machine
COPY testtable (column1, c2, c3) FROM STDIN WITH CSV;
1,2,3
4,5,6
\.
Then execute, on your client
psql -U postgres -f /mylocaldrive/copy.sql -h remoteserver.example.com
If you use JDBC, the best solution for you is to use the PostgreSQL COPY API
http://jdbc.postgresql.org/documentation/publicapi/org/postgresql/copy/CopyManager.html
Otherwise (as already noted by others) you can use \copy from psql which allows accessing the local files on the client machine
To my knowledge, the COPY command can only be used to read locally (either from stdin or from file) from the machine where the database is running.
You could make a shell script where you run you the java conversion, then use psql to do a \copy command, which reads from a file on the client machine.