I m trying to run a query in HeidiSQL. If I run only one SELECT statement, everything works fine. If I want to run multiple SELECT statements I keep getting SQL error 1064
Any ideas what to change to the code?
SELECT id, date, customer_id
INTO OUTFILE 'orders.txt'
CHARACTER SET utf8
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM orders;
SELECT order_id, line_number, quantity, discount, price
INTO OUTFILE 'order_line_items.txt'
CHARACTER SET utf8
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM order_line_items;
Thanks!
Related
I have following row in AWS Redshift warehouse table.
name
-------------------
tokenauthserver2018
This I queried via simple SELECT query
SELECT name
FROM tablename
When I am trying to unload it using UNLOAD query from AWS Redshift, it is successfully finishing but giving weird quoting.
"name"
"tokenauthserver2018\
Here is my query
UNLOAD ($TABLE_QUERY$
SELECT name
FROM tablename
$TABLE_QUERY$)
TO 's3://bucket/folder'
MANIFEST VERBOSE HEADER DELIMITER AS ','
NULL AS '' ESCAPE GZIP ADDQUOTES ALLOWOVERWRITE PARALLEL OFF;
I tried unloading without ADDQUOTES as well, but got following data
name
"tokenauthserver2018
This is the query for above.
UNLOAD ($TABLE_QUERY$
SELECT name
FROM tablename
$TABLE_QUERY$)
TO 's3://bucket/folder'
MANIFEST VERBOSE HEADER CSV NULL AS '' GZIP ALLOWOVERWRITE PARALLEL OFF;
Amazon support was able to resolve this, I am posting answer here for anyone interested.
This was due to presence of NULL character \0 in my data. As I don't have control over source data, I used TRANSLATE function to replace \0 character.
SELECT
TRANSLATE("name", CHR(0), '') AS "name"
FROM <tablename>
Reference: https://docs.aws.amazon.com/redshift/latest/dg/r_TRANSLATE.html
I am trying to upload a CSV which may/may not contain empty value for a column in a row.
I want to discard the rows that contain empty value from uploading to the DB through SQL Loader.
How can this be handled in ctrl file:
I have tried below conditions in the ctl file :
when String_Value is not null
when String_Value <> ''
but the rows are still getting inserted
This worked for me using either '<>' or '!='. I suspect the order of the clauses was incorrect for you. Note colc (also the third column in the data file) matches the column name in the table.
load data
infile 'c:\temp\x_test.dat'
TRUNCATE
into table x_test
when colc <> ''
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
cola char,
colb char,
colc char,
cold integer external
)
This is a follow-up question from this answer for "Save PL/pgSQL output from PostgreSQL to a CSV file".
I need to write a client-side CSV file using psql's \copy command. A one liner works:
db=> \copy (select 1 AS foo) to 'bar.csv' csv header
COPY 1
However, I have long queries that span several lines. I don't need to show the query, as I can't seem to extend this past one line without a parse error:
db=> \copy (
\copy: parse error at end of line
db=> \copy ( \\
\copy: parse error at end of line
db=> \copy ("
\copy: parse error at end of line
db=> \copy "(
\copy: parse error at end of line
db=> \copy \\
\copy: parse error at end of line
Is it possible to use \copy with a query that spans multiple lines? I'm using psql on Windows.
The working solution I have right now is to create a temporary view, which can be declared over multiple lines, then select from it in the \copy command, which fits comfortably on one line.
db=> CREATE TEMP VIEW v1 AS
db-> SELECT i
db-> FROM generate_series(1, 2) AS i;
CREATE VIEW
db=> \cd /path/to/a/really/deep/directory/structure/on/client
db=> \copy (SELECT * FROM v1) TO 'out.csv' csv header
COPY 2
db=> DROP VIEW v1;
DROP VIEW
We may use HEREDOC to feed multiline SQL to psql and use
# Putting the SQL using a HEREDOC
tr '\n' ' ' << SQL| \psql mydatabase
\COPY (
SELECT
provider_id,
provider_name,
...
) TO './out.tsv' WITH( DELIMITER E'\t', NULL '', )
SQL
Source: https://minhajuddin.com/2017/05/18/how-to-pass-a-multi-line-copy-sql-to-psql/
You can combine the server side COPY command with the \g psql command to produce a multi-line query to local file:
db=# COPY (
SELECT department, count(*) AS employees
FROM emp
WHERE role = 'dba'
GROUP BY department
ORDER BY employees
) TO STDOUT WITH CSV HEADER \g department_dbas.csv
COPY 5
I describe this technique in detial here https://hakibenita.com/postgresql-unknown-features#use-copy-with-multi-line-sql
I'm trying to use COPY with HEADER option but my header line in file is in different order than the column order specified in database.
Is the column name order necessary in my file ??
My code is as below:
COPY table_name (
SELECT column_name
FROM information_schema.columns
WHERE table_schema = 'schema_name'
AND table_name = 'table_name'
)
FROM 'file.csv'
WITH DELIMITER ',' CSV HEADER;
My database table has got a different order from file.csv and i wanted to select the table order and copy data from csv to table.
You can't issue an SQL query in copy from. You can only list the columns.
If the CSV columns are in the b, a, c order then list that in the copy from command:
copy target_table (b, a, c)
from file.csv
with (delimiter ',', format csv, header)
Assuming the order of the columns we need is the one of the table from which we are copying the results, the next logical step would be to simulate a sub-query through a Bash script.
psql schema_origin -c 'COPY table_origin TO stdout' | \
psql schema_destination -c \
"$(echo 'COPY table_destination (' \
$(psql schema_origin -t -c "select string_agg(column_name, ',') \
from information_schema.columns where table_name = 'table_origin'") \
') FROM stdin')"
StackOverflow answer on COPY command
StackExchange answer on fetching column names
StackOverflow answer on fetching results as tuples
I came up with the following setup for making COPY TO/FROM successful even for quite sophisticated JSON columns:
COPY "your_schema_name.yor_table_name" (
SELECT string_agg(
quote_ident(column_name),
','
) FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'yuour_table_name'
AND TABLE_SCHEMA = 'your_schema_name'
) FROM STDIN WITH CSV DELIMITER E'\t' QUOTE '\b' ESCAPE '\';
--here rows data
\.
the most important parts:
be explicit in filtering from information_schema.columns and user also the table_schema. Otherwise, you may end up with unexpected columns when one table name occurs in multiple schemas.
use quote_ident to make sure your command does not crash if someone made poor naming of table columns using Postgres registred keywords like user or unique. Thanks to quote_ident you will get them wrapped in double-quotes what makes them safe for importing.
I also found the following setup:
QUOTE '\b' - quote with backspace
DELIMITER E'\t' - delimiter with tabs
ESCAPE '\' - and escape with a backslash
for making both COPY to and from most reliable also for dealing with sophisticated/nested JSON columns.
I am facing a strange problem while using postgresql. I have a table with columns as id, data, day_data. I am firing a simple query
select * from tablename where id = 'someid';
However, when I am modifying the query to
select * from tablename where day_data = 'somedata';
Both the columns are primary key of the table and both have a data type of chracter varying (255). This, is a very strange behavior and I am not able to make any head or tail out of it. Any help will be appreciated.
My guess is that you've got trailing spaces in your data values. TRIM() removes trailing whitespace. One way to find out would be SELECT '"' || day_data || '"' FROM tablename;, which will enclose each value including leading and trailing whitespace with quotes.