The first SQL statement results in an "ERROR: syntax error at or near \ "
This refers to s3_uri\gset.
psql> SELECT aws_commons.create_s3_uri(
'test-bucket',
'animals.csv',
'us-east-1'
) AS s3_uri \gset
psql> \echo :s3_uri
(test-bucket,animals.csv,us-east-1)
psql> SELECT aws_commons.create_aws_credentials(
'<my_access_id>',
'<my_secret_key>',
'<session_token>'
) AS credentials \gset
psql> \echo :credentials
(<my_access_id>,<my_secret_key>,<session_token>)
psql> SELECT aws_s3.table_import_from_s3(
'animals',
'',
'(FORMAT CSV, DELIMITER '','', HEADER true)',
:'s3_uri',
:'credentials'
);
Related
I have an Athena table in text format. I looks like this in S3:
2022-06-05,55389e3b-3730-4cbb-85f2-d6f4de5123f4,{""05b7ede6-c9b7-4919-86d3-015dc5e77d40"":2",""1008b57c-fe53-4e3b-b84e-257eef70ce73"":2",""886e6dce-c40d-4c58-b87d-956b61382f18"":1",""e7c67b9b-3b01-4c3b-8411-f36659600bc3"":9}
2022-06-05,04e5b51e-8e16-4827-80c1-b50776bfb71b,{""04112c3e-0923-4c33-b92e-1183c06580b9"":1",""0f930559-0e66-45c0-bf9e-56edce16448d"":1",""1008b57c-fe53-4e3b-b84e-257eef70ce73"":70",""11e2e1cd-3078-4272-8b8b-55d62d2c0894"":2018",""19109d21-6150-4dd2-82e1-8bc5eee1d55c"":8",""1e58bb5f-cb5b-48d9-b752-f41dd5bd75bc"":32",""28276ff9-5221-4e41-be15-b7f9edee1162"":23",""2b70946f-1004-456b-9786-0c0274d31d1b"":1",""350b04d8-7910-4f19-b14b-d84e046f0bd6"":1",""3d4b0cb7-b701-4086-8fc8-22f957336670"":4",""3ed395b6-b354-4905-8d70-53174d68e293"":1",""41d99562-fd0b-4c1b-9e5b-66b82615b587"":1",""41e778fd-f6b9-4d71-8053-e2f2446b495e"":23",""44760b78-f700-4e4f-bb5b-cfe1de2b3771"":4",""4b01c168-e16d-499c-9e0e-483d7d86f679"":10",""5050d32f-6b4e-493b-bf37-876dc4cf7d4f"":5}
The columns are: DATE, UUID, JSONB
I have escaped the " and , characters, but Postgres seems unable to import it?
SELECT aws_s3.table_import_from_s3(
'my_table',
'd, id, j',
'(format csv)',
aws_commons.create_s3_uri(
'my-bucket',
'abc/20220608_172519_00015_d684z_740d0f86-1df0-4058-9d2c-7354a328dfcb.gz',
'us-west-2'
)
);
ERROR: extra data after last expected column
JSON should be escaped like this:
SELECT '"' || replace(json_format(j), '"', '""') || '"' AS escaped_json
FROM my_table
I have following with statement and copy command
with output01 as
(select * from (
select name,
case
when column1 is not null and lower(column1) in ('point1','point2','point3','point4') then 3456
else null end column1Desc,
case
when column2 is not null and lower(column2) in ('point1','point2','point3','point4') then 2456
else null end column2Desc,
column3, column4),
output02 as
(select * from (
select name,
case
when column1 is not null and lower(column1) in ('point1','point2','point3','point4') then 3456
else null end column1Desc,
case
when column2 is not null and lower(column2) in ('point1','point2','point3','point4') then 2456
else null end column2Desc,
column3, column4),
output3 as (SELECT * FROM output01 UNION ALL SELECT * FROM output02)
\copy (select * from output3) to '/usr/share/output.csv' with CSV ENCODING 'UTF-8' DELIMITER ',' HEADER;
I am getting following ERROR
ERROR: relation "tab3" does not exist
All psql backslash commands need to be written on a single line, so you can't have a multi-line query together with \copy. The only workaround is to create a (temporary) view with that query, then use that in the \copy command.
Something along the lines:
create temporary view data_to_export
as
with cte as (..)
select *
from cte
;
\copy (select * data_to_export) to ...
You are getting this error because you are running your CTE query and copy command in different statements. Considering your with query is working fine, you should write your copy statement like below:
\copy (WITH tab1 as (Your SQL statement),
tab2 as ( SELECT ... FROM tab1 WHERE your filter),
tab3 as ( SELECT ... FROM tab2 WHERE your filter)
SELECT * FROM tab3) to '/usr/share/results.csv' with CSV ENCODING 'UTF-8' DELIMITER ',' HEADER;
I have a task is change the DB2 sql to Postgres sql. It has a problem in XML handle.
I tried to used this sql
SELECT xmlagg( ', <foo>abc</foo>') as YM
FROM TEST_Table"
It's working fine.
But
SELECT xmlagg( concat(', ', TRIM(CATEGORY))) as YM
from TEST_Table
is not working.
Error msg : SQL Error [42883]: ERROR: function xmlagg(text) does not exist
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
My question is how to change the below DB2sql to Postgres sql ?
**SELECT substr( xmlserialize( xmlagg( xmltext( concat( ', ', TRIM(NM_TEXT) ) ) ) as varchar( 1024 ) ), 3, 128 ) as YM
FROM TEST_Table**
how do I remove trailing blanks in a select on a char(32)?
db2 "create table test (col1 char(32))"
db2 "insert into test values ('one')"
I tried
db2 "select cast(trim(t ' ' from col1 ) as varchar(32)) from test"
db2 "select trim(t ' ' from cast(col1 as varchar(32)) from test"
db2 "select cast(trim(t ' ' from cast(col1 as varchar(32))) as varchar(32)) from test"
... but I always have trailing blanks
Try
select trim(cast(col1 as varchar(32))) from test
proof it with
select length(trim(cast(col1 as varchar(32))) ) from test
when making dump file with pg_dump command this performace is very fine but in dump is only create alter and inserts with syntax
COPY table1 (id, name) FROM stdin;
1 developer's portal
...
.
I want to make with this syntax sql dump where table will updated
for example
update table1 set name ='hello' where id=1
with pg_dump syntax
This script will do, although, for simplicity's sake, it assumes the table's primary key is the first field:
table=table1
psql -Atc "with a as (select attrelid::regclass::text rel, attnum i, quote_ident(attname) col from pg_attribute where attrelid='$table'::regclass)
select 'select format(\$\$update '||rel||' set '||string_agg(col||'=%s', ', ' order by i)||'\$\$, '||string_agg('quote_nullable('||col||')', ',' order by i)||')||\$\$ where '||
(select col||'=\$\$||quote_nullable('||col||')||\$\$;\$\$ from '||rel from a where i = 1) from a where i > 1 group by rel" |\
psql -At
If you're using PostgreSQL 9.5+, then you could use INSERT ... ON CONFLICT UPDATE ... instead:
table=table1
psql -Atc "with a as (select attrelid::regclass::text rel, attnum i, quote_ident(attname) col from pg_attribute where attrelid='$table'::regclass)
select 'select format(\$\$insert into '||rel||'('||string_agg(col, ',' order by i)||
') values ('||string_agg('%s', ',')||
') on conflict ('||(select col from a where i=1)||') do update set '||
string_agg(case when i > 1 then col||'=excluded.'||col end, ', ' order by i)||';\$\$, '||
string_agg('quote_nullable('||col||')', ',' order by i)||') from '||rel from a where i > 0 group by rel" |\
psql -At