I have a table that looks like this:
CREATE TABLE SCHEDULE (
RCDNO INTEGER NOT NULL,
MASTCONNO INTEGER NOT NULL,
ROWNO INTEGER DEFAULT 0 NOT NULL,
EDITTIMESTAMP TIMESTAMP
)
ALTER TABLE SCHEDULE ADD CONSTRAINT PK_SCHEDULE PRIMARY KEY (RCDNO);
CREATE UNIQUE INDEX IDX_SCHEDULE_3 ON SCHEDULE (MASTCONNO, ROWNO);
I can run a query like this:
select mastconno, rowno, rcdno, edittimestamp
from schedule
where mastconno = 12
order by rowno desc
and I get this:
Due to a bug in the app code, the time portion of the edittimestamp is missing but that's of no consequence here. The records are nonetheless listed in desc order of entry by ROWNO. That's what the design of this table is meant to facilitate.
What I tried to do is this...
update SCHEDULE
set ROWNO = (ROWNO + 1)
where MASTCONNO = 12
... in preparation for the insert of a new ROWNO=0 record, I get this error:
attempt to store duplicate value (visible to active transactions) in unique index "IDX_SCHEDULE_3".
Problematic key value is ("MASTCONNO" = 12, "ROWNO" = 3).
Incidently I have an exact copy of the table in MS SQL Server, and I didn't have this problem there. This seems to be specific to the way Firebird works.
So then I tried this, hoping that Firebird would "feed" values from the IN predicate to the UPDATE in a non-offending order.
update SCHEDULE
set ROWNO = (ROWNO + 1)
where MASTCONNO = 12 and
ROWNO in (select ROWNO from SCHEDULE
where MASTCONNO = 12 order by ROWNO DESC)
Sadly the response was the same as before. ROWNO 3 is being duplicated by the update statement.
With the unique index (MASTCONNO, ROWNO) in place, I have tested the following:
update SCHEDULE
set ROWNO = (ROWNO + 1)
where MASTCONNO = 12
order by ROWNO DESC
Works correctly!
update SCHEDULE
set ROWNO = (ROWNO - 1)
where MASTCONNO = 12
order by ROWNO ASC
Also works correctly!
Thanks very much Mark Rotteveel.
Indeed, constraint checks and triggers Firebird runs per every row, not per transaction or per table.
So you would have to use the same loop-based approach you use in applications dealing with lists and arrays.
You have to use EXECUTE BLOCK and a properly directed loop.
http://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-dml-execblock.html
You take MAXimum ID that is of interest, and increment it.
Then you take previous value.
Then previous...
EXECUTE BLOCK
AS
DECLARE ID INTEGER;
BEGIN
ID = ( SELECT MAX(ROWNO) FROM SCHEDULE WHERE MASTCONNO = 12 );
/*
ID = NULL;
SELECT MAX(ROWNO) FROM SCHEDULE WHERE MASTCONNO = 12 INTO :ID;
IF (ID IS NULL) raise-some-error-somehow
*/
While (ID >= 0)
Begin
update SCHEDULE set ROWNO = (ID + 1)
where MASTCONNO = 12 and :ID = ROWNO;
ID = ID - 1;
End
END
DANGER!
If while you started are doing it some another transaction (maybe from another program working at another computer) inserts some new row with MASTCONNO = 12 and commits it - you have problems.
Firebird is multi-versions server, not table-blocking server, so there will be nothing in server, that prohibits this insert because your procedure is working on the table. Then you have race conditions, the fastest transaction would commit itself, and the slower one would fail with unique index violation.
You may also use FOR-SELECT loop instead of WHILE loop.
https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-psql-coding.html#fblangref25-psql-forselect
Like this
EXECUTE BLOCK
AS
BEGIN
FOR
SELECT * FROM SCHEDULE
WHERE MASTCONNO = 12
ORDER BY ROWNO DESCENDING -- proper loop direction: ordering is key!
AS CURSOR PTR
DO
UPDATE SCHEDULE SET ROWNO = ROWNO + 1
WHERE CURRENT OF PTR;
END
However, in Firebird 3 cursors-based positioning became rather slow.
Related
I have a table cusers with a primary key:
primary key(uid, lid, cnt)
And I try to insert some values into the table:
insert into cusers (uid, lid, cnt, dyn, ts)
values
(A, B, C, (
select C - cnt
from cusers
where uid = A and lid = B
order by ts desc
limit 1
), now())
on conflict do nothing
Quite often (with the possibility of 98%) a row cannot be inserted to cusers because it violates the primary key constraint, so hard select queries do not need to be executed at all. But as I can see PostgreSQL first counts the select query as a result of dyn column and only then rejects row because of uid, lid, cnt violation.
What is the best way to insert rows quickly in such situation?
Another explanation
I have a system where one row depends on another. Here is an example:
(x, x, 2, 2, <timestamp>)
(x, x, 5, 3, <timestamp>)
Two columns contain an absolute value (2 and 5) and relative value (2, 5 - 2). Each time I insert new row it should:
avoid same rows (see primary key constraint)
if new row differs, it should count a difference and put it into the dyn column (so I take the last inserted row for the user according to the timestamp and subtract values).
Another solution I've found is to use returning uid, lid, ts for inserts and get user ids which were really inserted - this is how I know they have differences from existing rows. Then I update inserted values:
update cusers
set dyn = (
select max(cnt) - min(cnt)
from (
select cnt
from cusers
where uid = A and lid = B
order by ts desc
limit 2) Table
)
where uid = A and lid = B and ts = TS
But it is not a fast approach either, as it seeks all over the ts column to find the two last inserted rows for each user. I need a fast insert query as I insert millions of rows at a time (but I do not write duplicates).
What the solution can be? May be I need a new index for this? Thanks in advance.
In my table I have the following scheme:
id - integer | date - text | name - text | count - integer
I want just to count some actions.
I want put 1 when date = '30-04-2019' not exist yet.
I want put +1 when is row already exist.
My idea is:
UPDATE "call" SET count = (1 + (SELECT count
FROM "call"
WHERE date = '30-04-2019'))
WHERE date = '30-04-2019'
But it is not working when row doesn't exist.
It is possible without some extra triggers, etc...
You can use a writeable CTE to achieve this. Additionally the UPDATE statement can be simplified to a simple set count = count + 1 there is no need for a sub-select.
with updated as (
update "call"
set count = count + 1
where date = '30-04-2019'
returning id
)
insert into "call" (date, count)
select '30-04-2019', 1
where not exists (select *
form updated);
If the update did not find a row, the where not exists condition will be true and the insert will be executed.
Note that the above is not safe for concurrent execution from multiple transactions. If you want to make this safe, create a unique index on the date column. Then use an INSERT ... ON CONFLICT instead:
insert into "call" (date, count)
values ('30-04-2019', 1)
on conflict (date)
do update
set count = "call".count + 1;
Again: the above requires a unique index (or constraint) on the date column.
Unrelated to the immediate problem, but: storing dates in a text column is a really, really bad idea. You should change your table definition and change the data type for the "date" column to date.
I have two tables: tire_life and transaction on my database.
It works fine when the subquery returns a value, but when no row is returned it does not update the tire_life table.
That's the query I'm running.
UPDATE tire_life as life SET covered_distance = initial_durability + transactions.durability
FROM (SELECT tire_life_id, SUM(tire_covered_distance) as durability
FROM transaction WHERE tire_life_id = 24 AND deleted_at IS NULL
GROUP BY (tire_life_id)) as transactions
WHERE life.id = 24
I have tried to use the COALESCE() function with no success at all.
You can use coalesce() for a subquery only if it returns single column. As you do not use tire_life_id outside the subquery, you can skip it:
UPDATE tire_life as life
SET covered_distance = initial_durability + transactions.durability
FROM (
SELECT coalesce(
(
SELECT SUM(tire_covered_distance)
FROM transaction
WHERE tire_life_id = 24 AND deleted_at IS NULL
GROUP BY (tire_life_id)
), 0) as durability
) as transactions
WHERE life.id = 24;
I guess you want to get 0 as durability if the subquery returns no rows.
First off, I'm using SQL Server 2008 R2
I am moving data from one source to another. In this particular case there is a field called SiteID. In the source it's not a required field, but in the destination it is. So it was my thought, when the SiteID from the source is NULL, to sort of create a SiteID "on the fly" during the query of the source data. Something like a combination of the state plus the first 8 characters of a description field plus a ten digit number incremented.
At first I thought it might be easy to use a combination of date/time + nanoseconds but it turns out that several records can be retrieved within a nanosecond leading to duplicate SiteIDs.
My second idea was to create a table that contained an identity field plus a function that would add a record to increment the identity field and then return it (the function would also delete all records where the identity field is less than the latest saving space). Unfortunately after I got it written, when trying to "CREATE" the function I got a notice that INSERTs are not allowed in functions.
I could (and did) convert it to a stored procedure, but stored procedures are not allowed in select queries.
So now I'm stuck.
Is there any way to accomplish what I'm trying to do?
This script may take time to execute depending on the data present in the table, so first execute on a small sample dataset.
DECLARE #TotalMissingSiteID INT = 0,
#Counter INT = 0,
#NewID BIGINT;
DECLARE #NewSiteIDs TABLE
(
SiteID BIGINT-- Check the datatype
);
SELECT #TotalMissingSiteID = COUNT(*)
FROM SourceTable
WHERE SiteID IS NULL;
WHILE(#Counter < #TotalMissingSiteID )
BEGIN
WHILE(1 = 1)
BEGIN
SELECT #NewID = RAND()* 1000000000000000;-- Add your formula to generate new SiteIDs here
-- To check if the generated SiteID is already present in the table
IF ( ISNULL(( SELECT 1
FROM SourceTable
WHERE SiteID = #NewID),0) = 0 )
BREAK;
END
INSERT INTO #NewSiteIDs (SiteID)
VALUES (#NewID);
SET #Counter = #Counter + 1;
END
INSERT INTO DestinationTable (SiteID)-- Add the extra columns here
SELECT ISNULL(MainTable.SiteID,NewIDs.SiteID) SiteID
FROM (
SELECT SiteID,-- Add the extra columns here
ROW_NUMBER() OVER(PARTITION BY SiteID
ORDER BY SiteID) SerialNumber
FROM SourceTable
) MainTable
LEFT JOIN ( SELECT SiteID,
ROW_NUMBER() OVER(ORDER BY SiteID) SerialNumber
FROM #NewSiteIDs
) NewIDs
ON MainTable.SiteID IS NULL
AND MainTable.SerialNumber = NewIDs.SerialNumber
In DB2, I need to do an insert, then, using results/data from that insert, update a related table. I need to do it on a million plus records and would prefer not to lock the entire database. So, 1) how do I 'couple' the insert and update statements? 2) how can I ensure the integrity of the transaction (without locking the whole she-bang)?
some pseudo-code should help clarify
STEP 1
insert into table1 (neededId, id) select DYNAMICVALUE, id from tableX where needed value is null
STEP 2
update table2 set neededId = (GET THE DYNAMIC VALUE JUST INSERTED) where id = (THE ID JUST INSERTED)
note: in table1, the ID col is not unique, so i can't just filter on that to find the new DYNAMICVALUE
This should be more clear (FTR, this works, but I don't like it, because I'd have to lock the tables to maintain integrity. Would be great it I could run these statements together, and allow the update to refer to the newAddressNumber value.)
/****RUNNING TOP INSERT FIRST****/*
--insert a new address for each order that does not have a address id
insert into addresses
(customerId, addressNumber, address)
select
cust.Id,
--get next available addressNumber
ifNull((select max(addy2.addressNumber) from addresses addy2 where addy2.customerId = cust.id),0) + 1 as newAddressNumber,
cust.address
from customers cust
where exists (
--find all customers with at least 1 order where addressNumber is null
select 1 from orders ord
where 1=1
and ord.customerId = cust.id
and ord.addressNumber is null
)
/*****RUNNING THIS UPDATE SECOND*****/
update orders ord1
set addressNumber = (
select max(addressNumber) from addresses addy3
where addy3.customerId = ord1.customerId
)
where 1=1
and ord1.addressNumber is null
The IDENTITY_VAL_LOCAL function is a non-deterministic function that returns the most recently assigned value for an identity column, where the assignment occurred as a result of a single INSERT statement using a VALUES clause