I want to use COPY CSV method to insert a csv file with array of json. I tried to look around in stack overflow but most of the methods are done through language like Python, I havent found the one using COPY CSV.
My Postgres table is this
CREATE TABLE individuals
(
arr jsonb[]
)
My CSV is something like this
arr
-------
{{"y":0, "x"; 0}}
And here is how I do the copy command:
cat 'file.csv' | psql -h localhost -p 5432 -d testdb -U user -c "COPY individuals(arr) FROM STDIN DELIMITER ',' CSV HEADER;"
I got this error:
ERROR: malformed array literal: "{{"y": 0, "x": 0}}"
DETAIL: Unexpected array element.
Related
I am a beginner with PostgreSQL, I am trying to import a csv file into pgAdmin4, but seem to be having some trouble. I have the CSV file saved on my desktop and there is no header in the CSV file. Here is what my query currently looks like,
COPY opioid_csv(Substance, Source, Specific_Measure, Type_event, Region, PRUID, Time_Period, Year_Quarter, Aggregator, Disaggregator, Unit, Value)
FROM 'Users/myusername/Desktop/opioid_csv.csv'
DELIMITER ','
I have created a table with all the column names as well but I am getting the following error:
ERROR: relation "opioid_csv" does not exist SQL state: 42P01
Try, using psql:
postgres=# \dt *.opioid_csv
List of relations
Schema | Name | Type | Owner
--------------+------------+-------+-------
marcothesane | opioid_csv | table | marco
(1 row)
Very probably, if you did succeed to create your opioid_csv table, you created it in another schema than public. You would have to put the schema (marcothesane in my example) before the table name: use marcothesane.opioid_csv instead of just opioid.csv
Let me try to do it with a CSV file of mine:
marco ~/1/Vertica/supp $ head test.csv
id,text
1,1VIAgFg
2,IfPLHbT
3,EWOmXAx
4,zl8paoh
5,9EpQ9Kx
6,ZpagcCh
7,6xoVoit
8,mCniu1U
9,euieQZa
Matching table DDL - using my own d2l DDL inferrer:
marco ~/1/Vertica/supp $ d2lm -coldelcomma test.csv | tee test.ddl.sql
CREATE TABLE IF NOT EXISTS test (
id INTEGER NOT NULL
, text CHAR(7) NOT NULL
);
Then, I run psql with that newly generated script:
marco ~/1/Vertica/supp $ psql -af test.ddl.sql
CREATE TABLE IF NOT EXISTS test (
id INTEGER NOT NULL
, text CHAR(7) NOT NULL
);
CREATE TABLE
Then, as I have a header line in my file, I COPY with csv header:
marco ~/1/Vertica/supp $ psql -c "copy test from '/Users/marco/1/Vertica/supp/test.csv' delimiter ',' csv header"
COPY 50000
marco ~/1/Vertica/supp $ psql -c "select count(*) from test"
count
-------
50000
What is different from what you do?
I am trying to export a column type jsonb using the copy command as follow:
payload
-------
{"test": "testing"}
sudo -u postgres psql test_database -c "COPY (SELECT payload FROM test_table) TO STDIN CSV"
The output gives me quoted text which is not a correct json format:
"{""test"":""testing""}"
How can I get a correct json format ?
You've chosen CSV output format, which escapes quotes that way. COPY does not produce JSON.
Do not use COPY to get the output, rather see store postgresql result in bash variable or How to return a value from psql to bash and use it?:
psql -U postgres -d test_database -AXqtc "SELECT payload FROM test_table;"
I am trying to run the below command on the unix console :
env 'PGOPTIONS=-c search_path=admin -c client_min_messages=error' psql -h hostname -U user -p 1111 -d platform -c "CREATE TEMP TABLE admin.tmp_213 AS SELECT * FROM admin.file_status limit 0;\copy admin.tmp_213(feed_id,run_id,extracted_date,file_name,start_time,file_size,status,crypt_flag) FROM '/opt/temp/213/list.up' delimiter ',' csv;UPDATE admin.file_status SET file_size = admin.tmp_213.file_size FROM admin.tmp_213 A WHERE admin.file_status.feed_id = A.feed_id and admin.file_status.file_name = A.file_name;"
I am getting the below error:
ERROR: syntax error at or near "\"
LINE 1: ...* FROM admin.file_status limit 0;\copy admi...
If I use the above command without a backslash before COPY, it gives me the below error:
ERROR: cannot create temporary relation in non-temporary schema
I am doing the above to implement the solution as mentioned here:
How to update selected rows with values from a CSV file in Postgres?
Your first error is because metacommands like \copy cannot be combined in the same line as regular commands when given with -c. You can give two -c options, with one command in each instead.
The second error is self-explanatory. You don't get to decide what schema your temp table goes to. Just omit the schema.
I am using timescaledb.
The doumentation I am following is Using PostgreSQL's COPY to migrate data from a csv file to timescale db. The name of csv file is test.csv.
I created the db named test , the name of table is test1. Table is a hypertable as per the timescaledb documentation.
The table's structure and csv files structure are the same.
While executing the following command in cmd I am not getting a result other than an addition of - symbol in the console command test-#
psql -d test -c "\COPY test1 FROM C:\Users\DEGEJOS\Downloads\test.csv CSV"
If I put ; after the command
psql -d test -c "\COPY test1 FROM C:\Users\DEGEJOS\Downloads\test.csv CSV"; I am getting a syntax error at Line 1.
How can I solve this error and insert data from csv file to db.?
You are trying to run psql with \COPY inside psql session, thus you get an error in the second call, since psql keyword does not exist in SQL. psql is an executable.
To follow the instructions from Timescale, you need to call the command directly in CMD. I.e, call:
psql -d test -c "\COPY test1 FROM C:\Users\DEGEJOS\Downloads\test.csv CSV"
If you are in C:\Users\DEGEJOS as in your screenshoot, it will look like:
C:\Users\DEGEJOS\psql -d test -c "\COPY test1 FROM C:\Users\DEGEJOS\Downloads\test.csv CSV"
I need to execute a query from a file and write the output of it to a CSV file. I want to keep the delimiter as ;. I have tried below queries.
psql -h localhost -p 5432 -U postgres -d postgres -f \path\to\sqlQuery.sql -o \path\to\result\result.csv
This query puts the result into a file but it is in a postgres result tabular format.
psql -h localhost -p 5432 -U postgres -d postgres -f copy(\path\to\sqlQuery.sql) to \path\to\result.csv csv header;
Above query is giving me a syntax error.
I'm looking for a way to use COPY command and \f or -f together so I'll be able to exceute the query from a file and also write the output to another CSV file with specified delimiter.
Change your SQL script to use COPY:
COPY (/* your query */) TO STDOUT
(FORMAT 'csv', DELIMITER ';');