lease=# COPY dhcpd_data (ip_address, start_time, end_time, mac_address, machine_name) FROM '~/outputcsvre.csv' DELIMITER ',' CSV HEADER;
ERROR: could not open file "~/outputcsvre.csv" for reading: No such file or directory
if i define the path as '/home/rihiraj12/outputcsvre.csv', it works fine.
Yes, that's normal.
You don't really have a directory called ~. When you execute a command on the command line, the shell will expand ~ to /home/rihiraj12 before running the program. But here you're not using the shell, so ~ is interpreted literally.
As a workaround you could say
COPY dhcpd_data (...) FROM PROGRAM 'cat ~/outputcsvre.csv' ...
But note that the COPY command is executed by the server, so this will make the server spawn a cat command and use the home directory of the PostgreSQL server.
To specify the file from your own point of view, you can (in psql) use the \copy meta-command (which has the same syntax as COPY):
\copy dhcpd_data (...) FROM PROGRAM 'cat ~/outputcsvre.csv' ...
This will use your own home directory as ~.
~ is a shortcut that unix-like shells can expand to be the home directory of your user.
i.e. if you use ~/outputcsvre.csv , the shell converts this to /home/rihiraj12/outputcsvre.csv before doing anything else with it.
Outside a shell, applications rarely implement this expansion - and neither does postgresql, so you have to provide real path to the file.
In the case of the COPY command in postgresql, it is executed by the server - so in this case you will have to provide a filename that the server can resolve and read directly. (i.e. a relative path would be relative to wherever the postgresql server is located - so use an absolute path for the file.)
Related
when running:
~/fidelity/releases/20220907033831$ ls -a
.
..
.browserslistrc
221005_users_all.csv
_private
the presence of a file is confirmed.
However, when launching a postgresql command
psql fidelity_development
COPY users (id,migrated_id,[...]) FROM '~/fidelity/releases/20220907033831/221005_users_all.csv' DELIMITER ';' CSV HEADER;
The response is unexpected:
ERROR: could not open file "~/fidelity/releases/20220907033831/221005_users_all.csv" for reading: No such file or directory
What am I missing to determine why postgresql cannot see this file?
note this directory was also simlinked as fidelity/current and the same result was obtained when referring to that directory for the file, whereas bash sees it.
Use \COPY command as this one is client based and handles the local path correctly.
While COPY is server based and this could cause issues finding your file.
I try this:
COPY gemeenten
FROM 'D:\CBS_woningcijfers_2014.csv'
DELIMITER ';' CSV
and get this:
ERROR: could not open file "D:\CBS_woningcijfers_2014.csv" for reading: No such file or directory
I doubled the backslashes, tried an E string, replaced \ by /, used " instead of ' but now I've run out of options. I am sure the file exists. Anybody any idea?
If the file and the PostgreSQL database are on the same machine, then the path and/or name of the file are not correct.
If the file is on your local machine and the database is on another, you cannot use the COPY command in SQL. You have two main choices to make this work:
1) Use psql \copy from your local machine. The syntax is similar, but it will transfer from your local to the remote. The docs are pretty helpful: https://www.postgresql.org/docs/9.5/static/app-psql.html#APP-PSQL-META-COMMANDS-COPY
2) Upload the file to the remote machine and then execute your command. Just make sure you are referencing the correct path and filename.
I'm trying to import a (rather large) .txt file into a table geonames in PostgreSQL 9.1. I'm in the /~ directory of my server, with a file named US.txt placed in that directory. I set the search_path variable to geochat, the name of the database I'm working in. I then enter this query:
COPY geonames
FROM 'US.txt',
DELIMITER E'\t',
NULL 'NULL');
I then receive this error:
ERROR: could not open file "US.txt" for reading: No such file or directory.
Do I have to type in \i US.txt or something similar first, or should it just get it from the present working directory?
Maybe a bit late, but hopefully useful:
Use \copy instead
https://wiki.postgresql.org/wiki/COPY
jvdw
A couple of misconceptions:
1.
I'm in the /~ directory of my server
There is no directory /~. It's either / (root directory) or ~ (home directory of current user). It's also irrelevant to the problem.
2.
I set the search_path variable to geochat, the name of the database I'm working in
The search_path has nothing to do with the name of the database. It's for schemas inside the current database. You probably need to reset this.
3.
You are required to use the absolute path for your file. As documented in the manual here:
filename
The absolute path name of the input or output file.
4.
DELIMITER: just noise.
The default is a tab character in text format
5.
NULL: It's rather uncommon to use the actual string 'NULL' for a NULL value. Are you sure?
The default is \N (backslash-N) in text format, and an unquoted empty string in CSV format.
My guess (after resetting search_path - or you schema-qualify the table name):
COPY geonames FROM '/path/to/file/US.txt';
The paths are relative to the PostgreSQL server, not the psql client.
Assuming you are running PostgreSQL 9.4, you can put US.txt in the directory /var/lib/postgresql/9.4/main/.
Another option is to pipe it in from stdin:
cat US.txt | psql -c "copy geonames from STDIN WITH (FORMAT csv);"
if you're running your COPY command from a script, you can have a step in the script that creates the COPY command with the correct absolute path.
MYPWD=$(pwd)
echo "COPY geonames FROM '$MYPWD/US.txt', DELIMITER E'\t';"
MYPWD=
you can then run this portion into a file and execute it
./step_to_create_COPY_with_abs_path.sh >COPY_abs_path.sql
psql -f COPY_abs_path.sql -d your_db_name
I would like to use the COPY function in PostgreSQL to import a CSV file into a PostgreSQL database.
Where it says the filename in the documentation, does the CSV file have to be stored in a specific location or can it be stored in any location.
For example, copy data_table from '/tmp/outputdata.csv' WITH DELIMITER AS ',' CSV QUOTE AS '"';. Where it says tmp, does that mean the tmp folder in the C: drive. Can it be change to another folder name?
It looks like you are confused by Linux vs. Windows file-path notation. What you have there is a Linux path anchored to root. Windows uses drive letters, which you can specify just as well when you are running on Windows.
If you use Windows notation, take care that you have to escape backslashes if you are not using standard_conforming_strings = on - which is the default in Postgres 9.1 or later, but not in older versions. Like:
COPY data_table from E'C:\\tmp\\outputdata.csv' WITH ...
With standard_conforming_strings = on you can simply write:
COPY data_table from 'C:\tmp\outputdata.csv' WITH ...
Note that a PostgreSQL Windows server also understands default path notation with slashes instead of backslashes.
For SQL COPY FROM / TO you can use any path that the owner of server process (postgres by default) has permission to read / write.
For the \copy meta command of the psql client the permissions of current local user apply.
Yes, of course you can specify whatever location where you have read access. There's no problem changing the path of the file.
Keep attention only on the fact that on windows you have to escape the backslash in this way :
copy data_table from 'c:\\Temp\\outputdata.csv' WITH DELIMITER AS ',' CSV QUOTE AS '"';
Using COPY statement of PostgreSQL, we can load data from a text file into data base's table as below:
COPY CME_ERROR_CODES FROM E'C:\\Program Files\\ERROR_CODES\\errcodes.txt' DELIMITER AS '~'
The above statement is run from a machine which has postgresql client where as the server is in another windows machine. Running the above statement is complaining me that ERROR: could not open file "C:\Program Files\ERROR_CODES\errcodes.txt" for reading: No such file or directory.
After some research, i observed that COPY statement is looking for the loader file(errcodes.txt) in the postgresql server's machine at the same path (C:\Program Files\ERROR_CODES). To test this , i have create the same folder structure in the postgresql server's machine and kept the errcodes.txt file in there. Then the COPY statement worked well. It looks very tough constraint for me with COPY statement.
Is there any setting needed to avoid this? or it is the behavior of COPY statement? I didn't find any information on PostgreSQL documents.
here's the standard solution:
COPY foo (i, j, k) FROM stdin;
1<TAB>2<TAB>3
\.
The data must be properly escaped and tab-separated.
Actually, it is in the docs, even in grammar definition you have STDIN... See: http://www.postgresql.org/docs/9.1/static/sql-copy.html
If you're using some programming language with COPY support, you will have pg_putcopy or similar function. So you don't have to worry about escaping and concatenation.
Hints how to do this manually in Python -> Recreating Postgres COPY directly in Python?
The Perl way -> http://search.cpan.org/dist/DBD-Pg/Pg.pm#COPY_support
Hope this helps.
From the documentation
Quote:
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible to the server and the name must be specified from the viewpoint of the server. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server.
If you want to copy from a local machine file to a server use \copy command.