I'm importing data from csv file into postgres db using pgadmin 4
Everything is ok but I get an issue when I try to import a file which contain some data like this
“‘t Zand, Vlotbrug”
“Dussen, `t Middeltje”
as you can see, the data contains
`
and
'
I also tried to import the file with utf-8 encoding but could not.
Anyone knows how to solve this issue?
Updated
Structure:
stop_id,stop_code,stop_name,stop_lat,stop_lon,location_type,parent_station,stop_timezone,wheelchair_boarding,platform_code,zone_id
Data:
stoparea:123953,,"De Zande, 'Koelucht'",52.5184475,5.956368,1,,,0,,
Error:
ERROR: unterminated CSV quoted field
CONTEXT: COPY stops, line 69400: "stoparea:123953,,"De Zande, 'Koelucht'",52.5184475,5.956368,1,,,0,,
stoparea:120536,,"Poortvliet, Zu..."
Updated 2
Command:
"/Applications/pgAdmin 4.app/Contents/SharedSupport/psql" --command "
"\copy transit.stops (stop_id, stop_code, stop_name, stop_lat,
stop_lon, location_type, parent_station, stop_timezone,
wheelchair_boarding, platform_code, zone_id) FROM
'/Users/tvtan/Desktop/gtfs-nl/stops.txt' DELIMITER ',' CSV HEADER
QUOTE '\"' ESCAPE '''';""
UI:
From the command line it looks like you have defined the ESCAPE character as a single quote. The single quote appears in your data but is not escaped.
The default ESCAPE character is the same as the QUOTE character.
More information here
Related
I tried the below query to import data
COPY table_name(id, name, create_uid, create_date, write_uid, write_date)
FROM '/path/filename.csv'
DELIMITER ',' CSV HEADER ENCODING 'UTF8' QUOTE '"' ESCAPE '''';
The name field data type is JSONB. I tried the below formats in my CSV file.
These 2 formats both produced the same error:
"{"en_US": "Test record"}" or {"en_US": "Test record"}
Error: ERROR: invalid input syntax for type json
DETAIL: Token "en_US" is invalid.
and this format produced a different error:
'{"en_US": "Test record"}'
ERROR: invalid input syntax for type json
DETAIL: Token "'" is invalid.
How can I fix the above errors?
Postgres uses " as escaping characters. Since importing from CSV file, postgres will escape the quotes.
Try something like,
"{""en_us"": ""Test record""}"
or
"{\"en_us\": \"Test record\"}"
I am trying to import the following csv file (named dummy.csv):
"F1", "F2", "F3"
1,2,"a"
3,4, "b"
3.4,2.4,"c"
to PostgreSQL using the pgAdmin GUI with the following command:
"C:\\Program Files\\PostgreSQL\\12\\bin\\psql.exe"
--command " "\\copy public.\"Result\"
FROM 'dummy.csv'
DELIMITER ','
CSV HEADER ENCODING 'UTF8'
QUOTE '\"'
ESCAPE '''';""
but keep getting the following error:
ERROR: extra data after the last expected column
CONTEXT: COPY Result, line 2: «1,2,"a"
What am I doing wrong?
The csv is not very 'clean':
"F1", "F2", "F3"
1,2,"a"
3,4, "b"
3.4,2.4,"c"
There are spaces between the separator and the quotes which could give problems with some parsers.
Maybe try with the following first and see what this gives:
"F1","F2","F3"
1,2,"a"
3,4,"b"
3.4,2.4,"c"
You could even remove the quotes in this example.
Hi I did this command on my original DB :
\COPY (SELECT * FROM cms_title WHERE title = 'Migration-test') TO '/Users/JayCee/cms_title_dump.csv' WITH CSV HEADER DELIMITER ';' NULL AS '';
It worked fine.
But when I try to copy it to an other DB I have this :
\COPY cms_title FROM 'cms_title_dump.csv' DELIMITER ';' CSV;
ERROR: invalid input syntax for integer: "id"
CONTEXT: COPY cms_title, line 1, column id: "id"
I don't understand, here is what I have when I do a cat -e on my file :
id;language;title;page_title;menu_title;meta_description;slug;path;has_url_overwrite;redirect;creation_date;published;publisher_is_draft;publisher_state;page_id;publisher_public_id$
217;en;Migration-test;"";"";"";migration-test;migration-test;f;"";2015-11-24 13:01:52.184969+00;t;t;0;99;218$
wow I am so lost even after reading tons of subject.. But contrary to those, I don't see any mistake here!
Your \COPY command generates a header line which is being read by the import command as data.
Either you do not generate the header line or tell the import command to not read the first line.
To avoid the header line generation omit the HEADER option.
To ignore the header line insert the HEADER option in the import command.
I try to import data from csv to my database postgresql, i use COPY but i have this error message:
ERROR: Missing data for the column « inventaire »
this is my instruction:
COPY calcul FROM 'C:/vente.csv' DELIMITER ',' CSV;
Do not get it wrong because
COPY is used for SQL
while
\copy is used for psql
i.e
\copy calcul FROM 'C:/vente.csv' (FORMAT csv, HEADER, DELIMITER ',');
I hope this helps you.
I have a CSV file from which I am trying to use Postgres COPY command in order to populate a table from that CSV file. One of the table columns NEXT_VISIT is of a date data_type. Some of the corresponding fields in the CSV file which are supposed to go into this date column have null values.
The Copy command am running is like so:
COPY "VISIT_STAGING_TABLE" from E'C:\\Users\\Sir Codealot\\Desktop\\rufijihdss-2007-2010\\rufijihdss\\VISIT_TEST.CSV' CSV HEADER
When I run this command I get the error:
ERROR: invalid input syntax for type date: ""
CONTEXT: COPY VISIT_STAGING_TABLE, line 2, column NEXT_VISIT: ""
********** Error **********
ERROR: invalid input syntax for type date: ""
SQL state: 22007
Context: COPY VISIT_STAGING_TABLE, line 2, column NEXT_VISIT: ""
How can I run the copy command and get Postgres to accept that some of the fields in the CSV file corresponding to NEXT_VISIT have values ""?
Add WITH NULL AS '' to your command (COPY expects NULLs to be represented as "\N" (backslash-N) by default).
COPY "VISIT_STAGING_TABLE" from E'C:\\Users\\Sir Codealot\\Desktop\\rufijihdss-2007-2010\\rufijihdss\\VISIT_TEST.CSV' WITH CSV HEADER NULL AS ''
More details here: postgresql COPY
I was having the exact same problem, and what solved it for me was to use the statement WITH NULL ''. It is important not to have a space between the apostrophes.
I originally used the statement WITH NULL ' ' and got the same error message you did (ERROR: syntax error at or near "WITH NULL").
But when I eliminated the space between the apostrophes it worked.