I have the following tables:
create table WorkPendingSummary
(
WorkPendingID int not null,
WorkPendingDate date not null,
Status varchar(20) not null,
EndDate date null
)
create table WorkPendingSummaryStage
(
WorkPendingID int not null,
WorkPendingDate date not null,
Status varchar(20) not null
)
I then have the following merge statement:
MERGE INTO WorkPendingSummary w USING WorkPendingSummaryStage
AS vals(WorkPendingID, WorkPendingDate, Status)
ON w.WorkPendingID = vals.WorkPendingID
WHEN MATCHED AND vals.status = 'CLOSED'
THEN UPDATE SET w.workpendingdate = vals.workpendingdate, w.status = vals.status, w.enddate = current_time
The documentation at: http://hsqldb.org/doc/guide/dataaccess-chapt.html#dac_merge_statement states that the "WHEN MATCHED" statement can have an additional "AND" clause as I have above, however that fails with:
unexpected token: AND required: THEN : line: 4 [SQL State=42581, DB Errorcode=-5581]
Does this feature work or am I just missing something?
Using HSQLDB 2.3.1.
Thanks!
The documentation is for version 2.3.3 and forthcoming 2.3.4. The AND clause is supported in these latest versions.
Related
By given table creation statement and query it's necessary to get old values before update:
CREATE TABLE IF NOT EXISTS products(
id INT GENERATED BY DEFAULT AS IDENTITY NOT NULL PRIMARY KEY,
product_id INT UNIQUE,
image_link CHARACTER VARYING NOT NULL,
additional_image_links CHARACTER VARYING[] NOT NULL
);
WITH temp AS (
INSERT INTO products(product_id, image_link, additional_image_links)
VALUES(1, 'http://www.e1xazm1ple1k113.com',ARRAY['http://www.examkple1113.com','http://www.example2.com'])
ON CONFLICT (product_id) DO UPDATE SET image_link = EXCLUDED.image_link, additional_image_links = EXCLUDED.additional_image_links
WHERE products.image_link != EXCLUDED.image_link OR products.additional_image_links != EXCLUDED.additional_image_links OR products.image_link != EXCLUDED.image_link
RETURNING id, image_link, additional_image_links
)
SELECT image_link, additional_image_links FROM products WHERE id IN (SELECT id FROM temp);
If conflict happens and new values conform criteria result is generated, however I need to use sqlalchemy machinery for it. Approximate but not working example:
def upsert(table, rows, constraint, update_cols):
query = insert(table).values(rows)
return query.on_conflict_do_update(
constraint=constraint,
set_={c: getattr(query.excluded, c) for c in update_cols},
where=getattr(table.c, "additional_image_link") != getattr(query.excluded, "additional_image_link"),
).cte("upsert")
Calling which produces the exception:
sesh = session(autocommit=False, autoflush=False, engine=DEFAULT)
sesh.execute(upsert(*args))
sqlalchemy.exc.ArgumentError: Executable SQL or text() construct expected, got <sqlalchemy.sql.selectable.CTE at 0x1042c3f10; upsert>.
I've created below code to implement SCD type 2 using merge, when i run the code i'm getting primary key violations on csname field. I have the below values as part of primary key, not sure whether merge SQL does support for varchar or not.
if I run the normal inner join SQL on the same key then i'm getting the matching records as well.
Any help much appreciated
csname
ER - Building Complaints
TR - Building Applications
CREATE PROCEDURE dbo.load_target
AS
BEGIN
INSERT INTO [TR_DW].[enum].[Rt]([csname],[enddatetime],[EffectiveToDate],[EffectiveFromDate],[CurrentRecord])
SELECT[csname],[enddatetime],[EffectiveToDate],[EffectiveFromDate],[CurrentRecord]
FROM
(
MERGE [TR_DW].[enum].[Rt] RtCSQSuTT
USING [TR].[enum].[Rt] RtCSQSuST
ON (RtCSQSuTT.csname = RtCSQSuST.csname)
WHEN NOT MATCHED THEN
INSERT ([csname],[enddatetime],[EffectiveToDate],[EffectiveFromDate],[CurrentRecord])
VALUES ([csname],[enddatetime],'12/31/9999', getdate(), 'Y')
WHEN MATCHED AND RtCSQSuTT.[CurrentRecord] = 'Y' AND
(ISNULL(RtCSQSuTT.[enddatetime], '') != ISNULL(RtCSQSuST.[enddatetime], ''))THEN
UPDATE SET
RtCSQSuTT.[CurrentRecord] = 'N',
RtCSQSuTT.[EffectiveFromDate] = GETDATE() - 1,
RtCSQSuTT.[EffectiveToDate] = GETDATE()
OUTPUT $Action Action_Taken,RtCSQSuST.[csqname],RtCSQSuST.[enddatetime],'12/31/9999' AS[EffectiveToDate],GETDATE() AS[EffectiveFromDate],'Y' AS[CurrentRecord]
)AS MERGE_OUT21
WHERE MERGE_OUT21.Action_Taken = 'UPDATE';
END
GO
I have a table that I created in postgresql:
> CREATE TABLE issuer(
> cik char(10) NOT null ,issuer_name char(150) NOT NULL ,trading_symbol char(10) NOT
> NULL ,SIC char(6) NOt NULL
> ,date_added timestamp NULL DEFAULT
> CURRENT_TIMESTAMP ,CONSTRAINT issuer_pk PRIMARY key (cik) );
I am trying to either update a row if it exists or insert it if it doesn't.
I have searched the documentation on how to make this work, but I am baffled by the errors I get.
I have a function that I call
io = postgres_update_issuer(con,cur,cik,coname,ticker,'')
When I call this function, python calls threading and then quits.
Here is the function I call:
def postgres_update_issuer(conn,cur,issuer_cik,name,ticker,sic):
sql = """
INSERT INTO issuer ( cik,issuer_name,trading_symbol,SIC)
VALUES (%s,%s,%s,%s)
ON CONFLICT (cik)
DO UPDATE SET
(issuer_name,trading_symbol,SIC )
= (EXCLUDED.issuer_name, EXCLUDED.trading_symbol, EXCLUDED.SIC)
;"""
try:
# data = (issuer_cik,name,ticker,sic)
cur.execute(sql,(issuer_cik,name,ticker,sic) )
return True
except (Exception, psycopg2.DatabaseError) as error:
print(error)
When I change the function to this, I get the couldn't move all fields error message:
def postgres_update_issuer(conn,cur,issuer_cik,name,ticker,sic):
sql = """
INSERT INTO issuer ( cik,issuer_name,trading_symbol,SIC)
VALUES (%s)
ON CONFLICT (cik)
DO UPDATE SET
(issuer_name,trading_symbol,SIC )
= (EXCLUDED.issuer_name, EXCLUDED.trading_symbol, EXCLUDED.SIC)
;"""
try:
data = (issuer_cik,name,ticker,sic)
cur.execute(sql,(data )
return True
except (Exception, psycopg2.DatabaseError) as error:
print(error)
What is the correct way to do this. I am using python 3.6 psycopg2, and postgresql 10
I'm trying to migrate a Tomcat app from using Postgres 9.5 to SQL Server 2016 and I've got a problem statement I can't seem to duplicate.
It's basically an upsert but one of the complications is the request supplies arguments to do the update, but when there is conflict I need to use some of the existing values from conflicting rows to insert/update.
The primary keys in the table can sometimes cause a conflict, which requires updating rows and deleting the old ones.
The table schema in MS SQL looks like:
CREATE TABLE [dbo].[signup](
[site_key] [varchar](32) NOT NULL,
[list_id] [bigint] NOT NULL,
[email_address] [varchar](256) NOT NULL,
[customer_id] [bigint] NULL,
[attribute1] [varchar](64) NULL,
[date1] [datetime] NOT NULL,
[date2] [datetime] NULL,
CONSTRAINT [pk_signup] PRIMARY KEY CLUSTERED
(
[site_key] ASC,
[list_id] ASC,
[email_address] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
The old Postgres SQL looked like this:
WITH updated_rows AS (
INSERT INTO signup
(site_key, list_id, email_address, customer_id, attribute1, date1, date2)
SELECT site_key, list_id, :emailAddress, customer_id, attribute1, date1, date2
FROM signup WHERE customer_id = :customerId and email_address <> :emailAddress
ON CONFLICT (site_key, list_id, email_address) DO UPDATE SET customer_id = excluded.customer_id
RETURNING site_key, customer_id, email_address, list_id
)
DELETE FROM signup AS signup_delete USING updated_rows
WHERE
signup_delete.site_key = updated_rows.site_key
AND signup_delete.customer_id = updated_rows.customer_id
AND signup_delete.list_id = updated_rows.list_id
AND signup_delete.email_address <> :emailAddress;
Two arguments are supplied, customer id and email address, shown here as Spring NamedParameterJdbcTemplate values :customerId and :emailAddress
It's trying to change the email address of the customer id to be the supplied one, but sometimes the supplied email address already exists in the primary key constraint.
In which case it needs to change the existing customer id to be supplied one, and remove the rows with that don't match the new email address.
I also need to try and maintain isolation so that nothing can change the data whilst I'm updating.
I'm trying to do it with a MERGE statement but I can't seem to get it to work, it's complaining I cant use values that aren't in the clause scope, but I think I've probably got other issues here too.
This is what I had so far. It doesn't even address the deleting part - only the upserting, but I can't even get this part to work. I was planning to use the OUTPUT from this as input to something to delete the rows similar to the postgres version.
WITH source AS (
SELECT cs.[site_key] as existing_site_key,
cs.list_id as existing_list_id,
cs.email_address as existing_email,
cs.customer_id as existing_customer_id,
cs.attribute1 as existing_attribute1,
cs.date1 as existing_date1,
cs.date2 as existing_date2,
cs2.email_address as conflicting_email,
cs2.customer_id AS conflicting_customer_id
FROM [dbo].[signup] cs
LEFT JOIN [dbo].[signup] cs2 ON cs2.email_address = :emailAddress
AND cs.site_key = cs2.site_key
AND cs.list_id = cs2.list_id
WHERE cs.customer_id = :customerId
)
MERGE signup WITH (HOLDLOCK) AS target
USING source
ON ( source.conflicting_customer_id is not null )
WHEN MATCHED AND source.existing_site_key = target.site_key AND source.existing_list_id = target.list_id AND source.conflicting_email = target.email_address THEN UPDATE
SET customer_id = :customerId
WHEN NOT MATCHED BY target AND source.existing_site_key = target.site_key AND source.existing_list_id = target.list_id AND source.conflicting_customer_id = :customerId THEN INSERT
(site_key, list_id, email_address, customer_id, attribute1, date1, date2) VALUES
(source.existing_site_key, source.existing_list_id, :emailAddress, source.customer_id, source.existing_attribute1, source.existing_date1, source.existing_date2)
Thanks,
mikee
I'm trying to insert records on my trying to implement an SCD2 on Redshift
but get an error.
The target table's DDL is
CREATE TABLE ditemp.ts_scd2_test (
id INT
,md5 CHAR(32)
,record_id BIGINT IDENTITY
,from_timestamp TIMESTAMP
,to_timestamp TIMESTAMP
,file_id BIGINT
,party_id BIGINT
)
This is the insert statement:
INSERT
INTO ditemp.TS_SCD2_TEST(id, md5, from_timestamp, to_timestamp)
SELECT TS_SCD2_TEST_STAGING.id
,TS_SCD2_TEST_STAGING.md5
,from_timestamp
,to_timestamp
FROM (
SELECT '20150901 16:34:02' AS from_timestamp
,CASE
WHEN last_record IS NULL
THEN '20150901 16:34:02'
ELSE '39991231 11:11:11.000'
END AS to_timestamp
,CASE
WHEN rownum != 1
AND atom.id IS NOT NULL
THEN 1
WHEN atom.id IS NULL
THEN 1
ELSE 0
END AS transfer
,stage.*
FROM (
SELECT id
FROM ditemp.TS_SCD2_TEST_STAGING
WHERE file_id = 2
GROUP BY id
HAVING count(*) > 1
) AS scd2_count_ge_1
INNER JOIN (
SELECT row_number() OVER (
PARTITION BY id ORDER BY record_id
) AS rownum
,stage.*
FROM ditemp.TS_SCD2_TEST_STAGING AS stage
WHERE file_id IN (2)
) AS stage
ON (scd2_count_ge_1.id = stage.id)
LEFT JOIN (
SELECT max(rownum) AS last_record
,id
FROM (
SELECT row_number() OVER (
PARTITION BY id ORDER BY record_id
) AS rownum
,stage.*
FROM ditemp.TS_SCD2_TEST_STAGING AS stage
)
GROUP BY id
) AS last_record
ON (
stage.id = last_record.id
AND stage.rownum = last_record.last_record
)
LEFT JOIN ditemp.TS_SCD2_TEST AS atom
ON (
stage.id = atom.id
AND stage.md5 = atom.md5
AND atom.to_timestamp > '20150901 16:34:02'
)
) AS TS_SCD2_TEST_STAGING
WHERE transfer = 1
and to short things up, I am trying to insert 20150901 16:34:02 to from_timestamp and 39991231 11:11:11.000 to to_timestamp.
and get
ERROR: 42804: column "from_timestamp" is of type timestamp without time zone but expression is of type character varying
Can anyone please suggest how to solve this issue?
Postgres isn't recognizing 20150901 16:34:02 (your input) as a valid time/date format, so it assumes it's a string.
Use a standard date format instead, preferably ISO-8601. 2015-09-01T16:34:02
SQLFiddle example
Just in case someone ends up here trying to insert into a postgresql a timestamp or a timestampz from a variable in groovy or Java from a prepared statement and getting the same error (as I did), I managed to do it by setting the property stringtype to "unspecified". According to the documentation:
Specify the type to use when binding PreparedStatement parameters set
via setString(). If stringtype is set to VARCHAR (the default), such
parameters will be sent to the server as varchar parameters. If
stringtype is set to unspecified, parameters will be sent to the
server as untyped values, and the server will attempt to infer an
appropriate type. This is useful if you have an existing application
that uses setString() to set parameters that are actually some other
type, such as integers, and you are unable to change the application
to use an appropriate method such as setInt().
Properties props = [user : "user", password: "password",
driver:"org.postgresql.Driver", stringtype:"unspecified"]
def sql = Sql.newInstance("url", props)
With this property set, you can insert a timestamp as a string variable without the error raised in the question title. For instance:
String myTimestamp= Instant.now().toString()
sql.execute("""INSERT INTO MyTable (MyTimestamp) VALUES (?)""",
[myTimestamp.toString()]
This way, the type of the timestamp (from a String) is inferred correctly by postgresql. I hope this helps.
Inside apache-tomcat-9.0.7/conf/server.xml
Add "?stringtype=unspecified" to the end of url address.
For example:
<GlobalNamingResources>
<Resource name="jdbc/??" auth="Container" type="javax.sql.DataSource"
...
url="jdbc:postgresql://127.0.0.1:5432/Local_DB?stringtype=unspecified"/>
</GlobalNamingResources>