Insert multiple records from xml - postgresql

I want bulk insert into a PostgreSQL database: backend code would create XML like
<items>
<i dt="2014-08-01" name="vvv" count="12" />
<i dt="2014-08-02" name="zzz" count="6" />
</items>
which I want to pass to a function and save all the values in one go.
But I'm stuck at xpath: each "column" seems to be an array of values, and I'm not sure how to insert them into a table then. Here's a test example:
CREATE TEMP TABLE temp_values (dt date, name varchar, count int);
WITH x AS (SELECT '
<items>
<i dt="2014-08-01" name="vvv" count="12" />
<i dt="2014-08-02" name="zzz" count="6" />
<i dt="2014-08-03" name="bbd" count="10" />
</items>'::xml AS t
)
INSERT INTO temp_values
SELECT
xpath('/items/i/#dt', t),
xpath('/items/i/#name', t),
xpath('/items/i/#count', t)
FROM x;
Now, in the end I want temp_values to have 3 records, as in the XML, but the table is empty.
If you comment out the "insert into" line, you'd see that the values are parsed correctly. It's just that it returns a single record where each column is an array, instead of returning multiple records.
What am I missing?

Ok, apparently unnest and array indexing is the solution:
unnest(array) | expand an array to a set of rows
So the code would be
CREATE TEMP TABLE temp_values (dt date, name varchar, count int) ON COMMIT DROP;
WITH x AS (SELECT '
<items>
<i dt="2014-08-01" name="vvv" count="12" />
<i dt="2014-08-02" name="zzz" count="6" />
<i dt="2014-08-03" name="bbd" count="10" />
</items>'::xml AS t
)
INSERT INTO temp_values
SELECT
CAST(CAST((xpath('//#dt', node))[1] as varchar) as date),
CAST((xpath('//#name', node))[1] as varchar),
CAST(CAST((xpath('//#count', node))[1] as varchar) as int)
FROM (SELECT unnest(xpath('/items/i', t)) AS node FROM x) sub;
SELECT * FROM temp_values

I'd prefer to delete some extra slashes before #:
CREATE TEMP TABLE temp_values (dt date, name varchar, count int) ON COMMIT DROP;
WITH x AS (SELECT '
<items>
<i dt="2014-08-01" name="vvv" count="12" />
<i dt="2014-08-02" name="zzz" count="6" />
<i dt="2014-08-03" name="bbd" count="10" />
</items>'::xml AS t
)
INSERT INTO temp_values
SELECT
CAST(CAST((xpath('#dt', node))[1] as varchar) as date),
CAST((xpath('#name', node))[1] as varchar),
CAST(CAST((xpath('#count', node))[1] as varchar) as int)
FROM (SELECT unnest(xpath('/items/i', t)) AS node FROM x) sub;
SELECT * FROM temp_values

Related

Extract value from an xmlstring in db2

I have an XML string in a column called RawData from table Inbound. I have to read value Success from an element called status.
xmlstring:
<InboundMessage>
<Transaction>
<Status>Success</Status>
</Transaction>
</InboundMessage>
SELECT X.STATUS
FROM (VALUES XMLPARSE(DOCUMENT '
<InboundMessage>
<Transaction>
<Status>Success</Status>
</Transaction>
</InboundMessage>
')) T (DOC)
, XMLTABLE
(
'$D/InboundMessage/Transaction/Status' PASSING T.DOC AS "D" COLUMNS
STATUS VARCHAR(20) PATH '.'
) X;
Refer to the XMLTABLE function overview link for more details.

Postgres inserting value issue - column "insert" does not exists

I have a table in DB:
<createTable tableName="api_consumer">
<column name="id"
type="INTEGER">
<constraints primaryKey="true"
primaryKeyName="PK_api_consumer_id"/>
</column>
<column name="business_id"
type="VARCHAR(50)"/>
<column name="type"
type="VARCHAR(32)"/>
<column name="status"
type="VARCHAR(32)"/>
<column name="description"
type="VARCHAR(256)"/>
<column name="created_by"
type="VARCHAR(64)"/>
<column name="created_date"
type="DATETIME"/>
<column name="modified_by"
type="VARCHAR(64)"/>
<column name="modified_date"
type="DATETIME"/>
<!--TTP below-->
<column name="name"
type="VARCHAR(64)">
</column>
<column name="origin_country"
type="VARCHAR(64)"/>
<column name="license_authority_name"
type="VARCHAR(64)"/>
<column name="registration_date"
type="DATETIME"/>
<column name="api_callback_url"
type="VARCHAR(256)"/>
<column name="application_base_url"
type="VARCHAR(256)"/>
<column name="authorization_callback_url"
type="VARCHAR(256)"/>
<column name="address"
type="INTEGER">
<constraints foreignKeyName="FK_api_consumer_address_id"
references="address(id)"/>
</column>
<column name="local_address"
type="INTEGER">
<constraints foreignKeyName="FK_api_consumer_local_address_id"
references="address(id)"/>
</column>
<column name="license"
type="INTEGER">
<constraints foreignKeyName="FK_api_consumer_license_id"
references="license(id)"/>
</column>
</createTable>
Generated in SQL dialect:
CREATE TABLE api_consumer
(
id INTEGER NOT NULL
CONSTRAINT "PK_api_consumer_id"
PRIMARY KEY,
business_id VARCHAR(50),
type VARCHAR(32),
status VARCHAR(32),
description VARCHAR(256),
created_by VARCHAR(64),
created_date TIMESTAMP,
modified_by VARCHAR(64),
modified_date TIMESTAMP,
name VARCHAR(64),
origin_country VARCHAR(64),
license_authority_name VARCHAR(64),
registration_date TIMESTAMP,
api_callback_url VARCHAR(256),
application_base_url VARCHAR(256),
authorization_callback_url VARCHAR(256),
address INTEGER
CONSTRAINT FK_api_consumer_address_id
REFERENCES address,
local_address INTEGER
CONSTRAINT FK_api_consumer_local_address_id
REFERENCES address,
license INTEGER
CONST
RAINT FK_api_consumer_license_id
REFERENCES license
);
and from the IntelliJ DB console I am trying to insert value to this table and test my triggers on it.
My insert is like:
INSERT INTO api_consumer VALUES ('11', '101', 'PSD2', 'NEW', 'TEST OF TRIGGER ON INSERT', 'kkwiatkowski', CURRENT_TIMESTAMP, null, null, 'SPRAWDZAM_COS', 'PL', null, null, null, null, null, 1, 1, 1);
And I am getting an error like:
sql> INSERT INTO api_consumer VALUES ('11', '101', 'PSD2', 'NEW', 'TEST OF TRIGGER ON INSERT', 'kkwiatkowski', CURRENT_TIMESTAMP, null, null, 'SPRAWDZAM_COS', 'PL', null, null, null, null, null, 1, 1, 1)
[2017-10-16 10:30:43] [42703] ERROR: column "insert" does not exist
[2017-10-16 10:30:43] Gdzie: PL/pgSQL function proc_subscription_audit_insert() line 3 at SQL statement
Word: "Gdzie" means Where in eng.
If I delete this id value: '11', then the insert query is invalid, cause the argument scope is moving on the right. Understandable.
[2017-10-16 10:39:08] [22007] ERROR: invalid input syntax for type timestamp: "SPRAWDZAM_COS"
[2017-10-16 10:39:08] Pozycja: 133
Word: "Pozycja" means Position in eng.
1) So how can I fix that?
2) How can I implement auto-increment value in here?
I have tried adding an autoIncrement="true" in the xml structure and then invoke this insert query without id argument. Neither this nor adding nextval(id) to the insert query works.
Edit:
procedures.xml
<changeSet id="1.0-procedures" author="blab">
<sql>
DROP FUNCTION IF EXISTS proc_api_consumer_audit_insert();
DROP FUNCTION IF EXISTS proc_api_consumer_audit_update();
DROP FUNCTION IF EXISTS proc_subscription_audit_insert();
DROP FUNCTION IF EXISTS proc_subscription_audit_update();
</sql>
<createProcedure>
CREATE FUNCTION proc_api_consumer_audit_insert()
RETURNS TRIGGER AS $api_consumer$
BEGIN
INSERT INTO api_consumer_audit(api_consumer_id, change_type, changed_by, changed_date, business_id_old, business_id_new, name_old, name_new, api_callback_url_old, api_callback_url_new, application_base_url_old, application_base_url_new, authorization_callback_url_old, authorization_callback_url_new, status_old, status_new) VALUES(NEW.id, 'INSERT', CURRENT_USER, CURRENT_TIMESTAMP, null, NEW.business_id, null, NEW.name, null, NEW.api_callback_url, null, NEW.application_base_url, null, NEW.authorization_callback_url, null, NEW.status);
RETURN NEW;
END;
$api_consumer$ LANGUAGE plpgsql;
</createProcedure>
<createProcedure>
CREATE FUNCTION proc_api_consumer_audit_update()
RETURNS TRIGGER AS $api_consumer$
BEGIN
INSERT INTO api_consumer_audit(api_consumer_id, change_type, changed_by, changed_date, business_id_old, business_id_new, name_old, name_new, api_callback_url_old, api_callback_url_new, application_base_url_old, application_base_url_new, authorization_callback_url_old, authorization_callback_url_new, status_old, status_new) VALUES(NEW.id, 'UPDATE', CURRENT_USER, CURRENT_TIMESTAMP, OLD.business_id, NEW.business_id, OLD.name, NEW.name, OLD.api_callback_url, NEW.api_callback_url, OLD.application_base_url, NEW.application_base_url, OLD.authorization_callback_url, NEW.authorization_callback_url, OLD.status, NEW.status);
RETURN NEW;
END;
$api_consumer$ LANGUAGE plpgsql;
</createProcedure>
<createProcedure>
CREATE FUNCTION proc_subscription_audit_insert()
RETURNS TRIGGER AS $subscription_audit$
BEGIN
INSERT INTO subscription_audit(subscription_id, change_type, status_old, status_new, valid_from_old, valid_from_new, valid_through_old, valid_through_new, changed_by, changed_date) VALUES (NEW.id, INSERT, null, NEW.status, null, NEW.valid_from, null, NEW.valid_through, CURRENT_USER, CURRENT_TIMESTAMP);
RETURN NEW;
END;
$subscription_audit$ LANGUAGE plpgsql;
</createProcedure>
<createProcedure>
CREATE FUNCTION proc_subscription_audit_update()
RETURNS TRIGGER AS $subscription_audit$
BEGIN
INSERT INTO subscription_audit(subscription_id, change_type, status_old, status_new, valid_from_old, valid_from_new, valid_through_old, valid_through_new, changed_by, changed_date) VALUES (NEW.id, 'UPDATE', OLD.status, NEW.status, OLD.valid_from, NEW.valid_from, OLD.valid_through, NEW.valid_through, CURRENT_USER, CURRENT_TIMESTAMP);
RETURN NEW;
END;
$subscription_audit$ LANGUAGE plpgsql;
</createProcedure>
<rollback>
DROP FUNCTION IF EXISTS proc_api_consumer_audit_insert();
DROP FUNCTION IF EXISTS proc_api_consumer_audit_update();
DROP FUNCTION IF EXISTS proc_subscription_audit_insert();
DROP FUNCTION IF EXISTS proc_subscription_audit_update();
</rollback>
</changeSet>
triggers.xml
<changeSet id="1.0-triggers" author="blab">
<sql>
DROP TRIGGER IF EXISTS trg_api_consumer_audit_insert ON api_consumer;
DROP TRIGGER IF EXISTS trg_api_consumer_audit_update ON api_consumer;
<!-- DROP TRIGGER IF EXISTS trg_api_consumer_audit_delete ON api_consumer-->
DROP TRIGGER IF EXISTS trg_subscription_audit_insert ON subscription;
DROP TRIGGER IF EXISTS trg_subscription_audit_update ON subscription;
<!-- DROP TRIGGER IF EXISTS trg_subscription_audit_delete ON subscription-->
</sql>
<createProcedure>
CREATE TRIGGER trg_api_consumer_audit_insert
AFTER INSERT ON api_consumer
FOR EACH ROW EXECUTE PROCEDURE proc_api_consumer_audit_insert();
</createProcedure>
<createProcedure>
CREATE TRIGGER trg_api_consumer_audit_update
AFTER UPDATE ON api_consumer
FOR EACH ROW EXECUTE PROCEDURE proc_api_consumer_audit_update();
</createProcedure>
<createProcedure>
CREATE TRIGGER trg_subscription_audit_insert
AFTER INSERT ON api_consumer
FOR EACH ROW EXECUTE PROCEDURE proc_subscription_audit_insert();
</createProcedure>
<createProcedure>
CREATE TRIGGER trg_subscription_audit_update
AFTER UPDATE ON api_consumer
FOR EACH ROW EXECUTE PROCEDURE proc_subscription_audit_update();
</createProcedure>
<rollback>
DROP TRIGGER IF EXISTS trg_api_consumer_audit_insert;
DROP TRIGGER IF EXISTS trg_api_consumer_audit_update;
DROP TRIGGER IF EXISTS trg_subscription_audit_insert;
DROP TRIGGER IF EXISTS trg_subscription_audit_update;
</rollback>
</changeSet>

How to import XML data into postgres database table?

I have XML format file for account table created using query as:
SELECT table_to_xml('account', true, FALSE, '');
-----The table structure is as:
CREATE TABLE public.account (
account_id INTEGER NOT NULL,
name VARCHAR(1) NOT NULL,
type VARCHAR(20),
group_name VARCHAR(50),
CONSTRAINT account_pkey PRIMARY KEY(account_id));
Question: How can I directly load data into account table using XML file in PostgreSQL?
I had to use varchar(2) due to the conversion from xml.
I've used a select into (creates public.account)
select account_id::text::int, account_name::varchar(2),
account_type::varchar(20) , account_group::varchar(50) INTO
public.account from(
WITH x AS ( SELECT
'<accounts>
<account>
<account_id>1</account_id>
<account_name> A </account_name>
<account_type> off shore</account_type>
<account_group> slush fund </account_group>
</account>
<account>
<account_id>3</account_id>
<account_name> C </account_name>
<account_type> off shore</account_type>
<account_group> slush fund </account_group>
</account>
</accounts> '::xml AS t)
SELECT unnest(xpath('/accounts/account/account_id/text()', t))
AS account_id,
unnest(xpath('/accounts/account/account_name/text()', t))
AS account_name,
unnest(xpath('/accounts/account/account_type/text()', t))
AS account_type,
unnest(xpath('/accounts/account/account_group/text()', t))
AS account_group
FROM x) as accounts
If you're interested in reading the xml file in then this may be useful.
Ref stackexchange sql to read xml from file into postgresql
I hope this helps

Prevent lines appearing in XML

I have the following table and content:
CREATE TABLE [dbo].[MyTable](
[PID] [int] NOT NULL,
[CID] [int] NOT NULL
)
INSERT INTO MyTable values (17344,17345)
INSERT INTO MyTable values (17344,17346)
INSERT INTO MyTable values (17272,17273)
INSERT INTO MyTable values (17272,17255)
INSERT INTO MyTable values (17272,17260)
INSERT INTO MyTable values (17272,17274)
INSERT INTO MyTable values (17272,17252)
From this I need to create the following XML layout:
<Item code="17344">
<BOMs>
<BOM code="17344">
<BOMLine type="17345"/>
<BOMLine type="17346"/>
</BOM>
</BOMs>
</Item>
<Item code="17272">
<BOMs>
<BOM code="17272">
<BOMLine type="17273"/>
<BOMLine type="17255"/>
<BOMLine type="17260"/>
<BOMLine type="17274"/>
<BOMLine type="17252"/>
</BOM>
</BOMs>
</Item>
I'm trying to achieve this with the following statement which gives me far too much lines and duplicates:
DECLARE #test XML
SELECT #test =
(SELECT PID '#code',
(SELECT PID as '#code',
(SELECT CID as '#type'
FROM MyTable
FOR XML PATH('BOMLine'), TYPE)
FROM MyTable GROUP BY PID
FOR XML PATH('BOM'), TYPE, ROOT('BOMs'))
FROM MyTable
FOR XML PATH('Item'), TYPE)
select #test
Can anyone help me with this? I'm using SQL Server 2008 by the way.
It would be greatly appreciated.
Best regards,
Wes
You need the group by in the outermost query and you need to make your sub-query correlated with the outer query on PID.
The extra PID in BOM does not need a from clause.
select T1.PID as '#Code',
(
select T1.PID as '#code',
(
select T2.CID as '#type'
from dbo.MyTable as T2
where T1.PID = T2.PID
for xml path('BOMLine'), type
)
for xml path('BOM'), root('BOMs'), type
)
from dbo.MyTable as T1
group by T1.PID
for xml path('Item')

Quotation mark incorrect when using crosstab() in PostgreSQL

I have a table t1 as below:
create table t1 (
person_id int,
item_name varchar(30),
item_value varchar(100)
);
There are five records in this table:
person_id | item_name | item_value
1 'NAME' 'john'
1 'GENDER' 'M'
1 'DOB' '1970/02/01'
1 'M_PHONE' '1234567890'
1 'ADDRESS' 'Some Addresses unknown'
Now I want to use crosstab function to extract NAME, GENDER data, so I write a SQL as:
select * from crosstab(
'select person_id, item_name, item_value from t1
where person_id=1 and item_name in ('NAME', 'GENDER') ')
as virtual_table (person_id int, NAME varchar, GENDER varchar)
My problem is, as you see the SQL in crosstab() contains condition of item_name, which will cause the quotation marks to be incorrect.
How do I solve the problem?
To avoid any confusion about how to escape single quotes and generally simplify the syntax, use dollar-quoting for the query string:
SELECT *
FROM crosstab(
$$
SELECT person_id, item_name, item_value
FROM t1
WHERE person_id = 1
AND item_name IN ('NAME', 'GENDER')
$$
) AS virtual_table (person_id int, name varchar, gender varchar);
See:
Insert text with single quotes in PostgreSQL
And you should add ORDER BY to your query string. I quote the manual for the tablefunc module:
In practice the SQL query should always specify ORDER BY 1,2 to ensure
that the input rows are properly ordered, that is, values with the
same row_name are brought together and correctly ordered within the
row. Notice that crosstab itself does not pay any attention to the
second column of the query result; it's just there to be ordered by,
to control the order in which the third-column values appear across the page.
See:
PostgreSQL Crosstab Query
Double your single quotes to escape them:
select * from crosstab(
'select person_id, item_name, item_value from t1
where person_id=1 and item_name in (''NAME'', ''GENDER'') ')
as virtual_table (person_id int, NAME varchar, GENDER varchar)