I am trying to create a CSV from table data.
column1 column2
this,is"part|of"the,:long,//data :"hello,
Expected output:
column1,column2
this\,is"part|of"the\,:long\,//data,:"hello\,
My end goal is to load this CSV delimited by ',' into a hive table
i tried this but didn't work:
\COPY (select * from table) TO /tmp/file.csv WITH (FORMAT CSV, DELIMITER ',', ESCAPE '\', HEADER TRUE);
CSV i got:
column1,column2
this,is"part|of"the,:long,//data,:"hello,
The commas weren't prefixed by backslash.
Any help is appreciated.
Related
There are many standalone double quote '"' in a tab delimited text file, need to be loaded into PostgreSQL with psql \copy command.
If I use FORMAT CSV option, I have to specify the QUOTE, and QUOTE char needs to be paired.
Here is the code, and output,
create table t1(
c1 varchar(20),
n1 numeric
);
echo 'Alf_7" 5.12' > m.csv
psql> \copy t1 FROM 'm.csv' (FORMAT CSV, delimiter E'\t', NULL 'NULL', HEADER false);
ERROR: unterminated CSV quoted field
CONTEXT: COPY t1, line 1: "Alfa_7" 5.1
Use FORMAT text option. then you do not have to specify the QUOTE.
psql=> \copy t1 FROM 'm.csv' (FORMAT text, delimiter E'\t', NULL 'NULL', HEADER false);
COPY 1
Let say you have a SELECT id from table query (the real case is a complex query) that does return you several results.
The problem is how to get all id return in a single row, comma separated?
SELECT string_agg(id::text, ',') FROM table
Requires PostgreSQL 9.0 but that's not a problem.
You can use the array() and array_to_string() functions togetter with your query.
With SELECT array( SELECT id FROM table ); you will get a result like: {1,2,3,4,5,6}
Then, if you wish to remove the {} signs, you can just use the array_to_string() function and use comma as separator, so: SELECT array_to_string( array( SELECT id FROM table ), ',' ) will get a result like: 1,2,3,4,5,6
You can generate a CSV from any SQL query using psql:
$ psql
> \o myfile.csv
> \f ','
> \a
> SELECT col1 AS column1, col2 AS column2 ... FROM ...
The resulting myfile.csv will have the SQL resultset column names as CSV column headers, and the query tuples as CSV rows.
h/t http://pookey.co.uk/wordpress/archives/51-outputting-from-postgres-to-csv
use array_to_string() & array() function for the same.
select array_to_string(array(select column_name from table_name where id=5), ', ');
Use this below query it will work and gives the exact result.
SELECT array_to_string(array_agg(id), ',') FROM table
Output : {1,2,3,4,5}
SELECT array_agg(id, ',') FROM table
{1,2,3,4}
I am using Postgres 11 and EntityFramework is fetching it as array of integers.
I have an array (text) (sample row) in my PostgreSQL 9.5 database like follows:
my_array(text)
1,112,292,19.7
I am exporting this text array using Postgres COPY command to a custom text file like this:
Copy
(
Select
my_array
from
my_table
Order by my_table_id
) to '~my_path/output.str' With DELIMITER ',';
I get the output:
1\,112\,292\,19.7\
How can I avoid these unwanted \ in my copy command output?
if the delimiter character (, in your case) is present in a string, it will be escaped (normally by prefixing it with a \)
If you use a different separator (from ,), the , separator doesn't have to be escaped.
If you quote the string in the output, the , separator doesn't have to be escaped.
-- CREATE TABLE my_table(my_table_id SERIAL PRIMARY KEY, my_array text);
-- INSERT INTO my_table(my_array )VALUES ('1,112,292,19.7') ;
COPY ( SELECT my_array FROM my_table ORDER BY my_table_id)
TO '/tmp/output.str'
WITH CSV DELIMITER ',' QUOTE '"'
;
I am exporting table using COPY command in CSV file with pipe delimiter. Few columns are null and few others are blank. In the CSV file blank string is exported as "" which I don't want.
Is there any option in COPY command to treat blank string as NULL?
For example I have five columns where 1st is blank while rest are null my output is coming as:
""||||
Thanks in advance.
If you're already using a SELECT statement in the COPY, just replace the values you don't like.
copy (select case when col_1 = '' then null else col_1 end as col_1,
case when col_2 = '' then null else col_2 end as col_2,
...)
...
I want to copy a csv to postgres. And some value is string like this "{\"foo\": 123}"
If I use the COPY in the postgres directly, it will escape the string. when I select from postgres, it will become "{foo: 123}", but it's hard for me to handle, so how to not to escape the ". That is to say, I hope I can get the origin string "{\"foo\": 123}" when I select it from postgres
CREATE TABLE meuk
( bagger varchar
);
COPY meuk(bagger) FROM stdin WITH CSV QUOTE '"' ESCAPE E'\\' ;
"{\"foo\": 123}"
\.
SELECT * from meuk;
Result:
CREATE TABLE
bagger
--------------
{"foo": 123}
(1 row)