I have a sproc
CREATE PROCEDURE [dbo].[GetNextImageRequest]
(...) AS
DECLARE #ReturnValue BIT
SET #ReturnValue = 1
-- Set paranoid level isolation: only one access at a time!
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRANSACTION
BEGIN TRY
...
UPDATE STATEMENT THAT THROWS FOREIGN KEY EXCEPTION
IF ##trancount > 0
BEGIN
COMMIT TRANSACTION
END
SET #ReturnValue = 0
END TRY
BEGIN CATCH
IF ##trancount > 0
BEGIN
ROLLBACK TRANSACTION
END
SET #ReturnValue = 1
END CATCH
SET TRANSACTION ISOLATION LEVEL READ COMMITTED
SET NOCOUNT ON
RETURN #ReturnValue -- 0=success
GO
When i call this manually from Sql Server Management studio, i don't get any exception.
When I Call this through Entity Framework 6, i get
Transaction count after EXECUTE indicates a mismatching number of
BEGIN and COMMIT statements. Previous count = 1, current count = 0.
What am I doing wrong? The foreign key constraint is doing roll back but i am checking ##TRANCOUNT.
Entity Framework 6 (EF6)
Add this line before ExecuteSqlCommand
db.Configuration.EnsureTransactionsForFunctionsAndCommands = false;
It turns out EF6 wraps sproc call in its own transaction.
The workaround is to check if transaction is already open or not and only do BEGIN|COMMIT|ROLLBACK TRANSACTION if it isn't already open
Related
When using Flyway in combination with a Microsoft SQL Server, we are observing the issue described on this question.
Basically, a migration script like this one does not rollback the successful GO-delimited batches when another part failed:
BEGIN TRANSACTION
-- Create a table with two nullable columns
CREATE TABLE [dbo].[t1](
[id] [nvarchar](36) NULL,
[name] [nvarchar](36) NULL
)
-- add one row having one NULL column
INSERT INTO [dbo].[t1] VALUES(NEWID(), NULL)
-- set one column as NOT NULLABLE
-- this fails because of the previous insert
ALTER TABLE [dbo].[t1] ALTER COLUMN [name] [nvarchar](36) NOT NULL
GO
-- create a table as next action, so that we can test whether the rollback happened properly
CREATE TABLE [dbo].[t2](
[id] [nvarchar](36) NOT NULL
)
GO
COMMIT TRANSACTION
In the above example, the table t2 is being created even though the preceding ALTER TABLE statement fails.
On the linked question, the following approaches (outside of the flyway context) were suggested:
A multi-batch script should have a single error handler scope that rolls back the transaction on error, and commits at the end. In TSQL you can do this with dynamic sql
Dynamic SQL makes for hard-to-read script and would be very inconvenient
With SQLCMD you can use the -b option to abort the script on error
Is this available in flyway?
Or roll your own script runner
Is this maybe the case in flyway? Is there a flyway-specific configuration to enable proper failing on errors?
EDIT: alternative example
Given: simple database
BEGIN TRANSACTION
CREATE TABLE [a] (
[a_id] [nvarchar](36) NOT NULL,
[a_name] [nvarchar](100) NOT NULL
);
CREATE TABLE [b] (
[b_id] [nvarchar](36) NOT NULL,
[a_name] [nvarchar](100) NOT NULL
);
INSERT INTO [a] VALUES (NEWID(), 'name-1');
INSERT INTO [b] VALUES (NEWID(), 'name-1'), (NEWID(), 'name-2');
COMMIT TRANSACTION
Migration Script 1 (failing, without GO)
BEGIN TRANSACTION
ALTER TABLE [b] ADD [a_id] [nvarchar](36) NULL;
UPDATE [b] SET [a_id] = [a].[a_id] FROM [a] WHERE [a].[a_name] = [b].[a_name];
ALTER TABLE [b] ALTER COLUMN [a_id] [nvarchar](36) NOT NULL;
ALTER TABLE [b] DROP COLUMN [a_name];
COMMIT TRANSACTION
This results in the error message Invalid column name 'a_id'. for the UPDATE statement.
Possible solution: introduce GO between statements
Migration Script 2 (with GO: working for "happy case" but only partial rollback when there's an error)
BEGIN TRANSACTION
SET XACT_ABORT ON
GO
ALTER TABLE [b] ADD [a_id] [nvarchar](36) NULL;
GO
UPDATE [b] SET [a_id] = [a].[a_id] FROM [a] WHERE [a].[a_name] = [b].[a_name];
GO
ALTER TABLE [b] ALTER COLUMN [a_id] [nvarchar](36) NOT NULL;
GO
ALTER TABLE [b] DROP COLUMN [a_name];
GO
COMMIT TRANSACTION
This performs the desired migration as long as all values in table [b] have a matching entry in table [a].
In the given example, that's not the case. I.e. we get two errors:
expected: Cannot insert the value NULL into column 'a_id', table 'test.dbo.b'; column does not allow nulls. UPDATE fails.
unexpected: The COMMIT TRANSACTION request has no corresponding BEGIN TRANSACTION.
Horrifyingly: the last ALTER TABLE [b] DROP COLUMN [a_name] statement was actually executed, committed and not rolled back. I.e. one cannot fix this up afterwards as the linking column is lost.
This behaviour is actually independent of flyway and can be reproduced directly via SSMS.
The problem is fundamental to the GO command. It's not a part of the T-SQL language. It's a construct in use within SQL Server Management Studio, sqlcmd, and Azure Data Studio. Flyway is simply passing the commands on to your SQL Server instance through the JDBC connection. It's not going to be dealing with those GO commands like the Microsoft tools do, separating them into independent batches. That's why you won't see individual rollbacks on errors, but instead see a total rollback.
The only way to get around this that I'm aware of would be to break apart the batches into individual migration scripts. Name them in such a way so it's clear, V3.1.1, V3.1.2, etc. so that everything is under the V3.1* version (or something similar). Then, each individual migration will pass or fail instead of all going or all failing.
Edited 20201102 -- learned a lot more about this and largely rewrote it! So far have been testing in SSMS, do plan to test in Flyway as well and write up a blog post. For brevity in migrations, I believe you could put the ##trancount check / error handling into a stored procedure if you prefer, that's also on my list to test.
Ingredients in the fix
For error handling and transaction management in SQL Server, there are three things which may be of great help:
Set XACT_ABORT to ON (it is off by default). This setting "specifies whether SQL Server automatically rolls back the current transaction when a Transact-SQL statement raises a runtime error" docs
Check ##TRANCOUNT state after each batch delimiter you send and using this to "bail out" with RAISERROR / RETURN if needed
Try/catch/throw -- I'm using RAISERROR in these examples, Microsoft recommends you use THROW if it's available to you (it's available SQL Server 2016+ I think) - docs
Working on the original sample code
Two changes:
Set XACT_ABORT ON;
Perform a check on ##TRANCOUNT after each batch delimiter is sent to see if the next batch should be run. The key here is that if an error has occurred, ##TRANCOUNT will be 0. If an error hasn't occurred, it will be 1. (Note: if you explicitly open multiple "nested" transactions you'd need to adjust trancount checks as it can be higher than 1)
In this case the ##TRANCOUNT check clause will work even if XACT_ABORT is off, but I believe you want it on for other cases. (Need to read up more on this, but I haven't come across a downside to having it ON yet.)
BEGIN TRANSACTION;
SET XACT_ABORT ON;
GO
-- Create a table with two nullable columns
CREATE TABLE [dbo].[t1](
[id] [nvarchar](36) NULL,
[name] [nvarchar](36) NULL
)
-- add one row having one NULL column
INSERT INTO [dbo].[t1] VALUES(NEWID(), NULL)
-- set one column as NOT NULLABLE
-- this fails because of the previous insert
ALTER TABLE [dbo].[t1] ALTER COLUMN [name] [nvarchar](36) NOT NULL
GO
IF ##TRANCOUNT <> 1
BEGIN
DECLARE #ErrorMessage AS NVARCHAR(4000);
SET #ErrorMessage
= N'Transaction in an invalid or closed state (##TRANCOUNT=' + CAST(##TRANCOUNT AS NVARCHAR(10))
+ N'). Exactly 1 transaction should be open at this point. Rolling-back any pending transactions.';
RAISERROR(#ErrorMessage, 16, 127);
RETURN;
END;
-- create a table as next action, so that we can test whether the rollback happened properly
CREATE TABLE [dbo].[t2](
[id] [nvarchar](36) NOT NULL
)
GO
COMMIT TRANSACTION;
Alternative example
I added a bit of code at the top to be able to reset the test database. I repeated the pattern of using XACT_ABORT ON and checking ##TRANCOUNT after each batch terminator (GO) is sent.
/* Reset database */
USE master;
GO
IF DB_ID('transactionlearning') IS NOT NULL
BEGIN
ALTER DATABASE transactionlearning
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE;
DROP DATABASE transactionlearning;
END;
GO
CREATE DATABASE transactionlearning;
GO
/* set up simple schema */
USE transactionlearning;
GO
BEGIN TRANSACTION;
CREATE TABLE [a]
(
[a_id] [NVARCHAR](36) NOT NULL,
[a_name] [NVARCHAR](100) NOT NULL
);
CREATE TABLE [b]
(
[b_id] [NVARCHAR](36) NOT NULL,
[a_name] [NVARCHAR](100) NOT NULL
);
INSERT INTO [a]
VALUES
(NEWID(), 'name-1');
INSERT INTO [b]
VALUES
(NEWID(), 'name-1'),
(NEWID(), 'name-2');
COMMIT TRANSACTION;
GO
/*******************************************************/
/* Test transaction error handling starts here */
/*******************************************************/
USE transactionlearning;
GO
BEGIN TRANSACTION;
SET XACT_ABORT ON;
GO
IF ##TRANCOUNT <> 1
BEGIN
DECLARE #ErrorMessage AS NVARCHAR(4000);
SET #ErrorMessage
= N'Check 1: Transaction in an invalid or closed state (##TRANCOUNT=' + CAST(##TRANCOUNT AS NVARCHAR(10))
+ N'). Exactly 1 transaction should be open at this point. Rolling-back any pending transactions.';
RAISERROR(#ErrorMessage, 16, 127);
RETURN;
END;
ALTER TABLE [b] ADD [a_id] [NVARCHAR](36) NULL;
GO
IF ##TRANCOUNT <> 1
BEGIN
DECLARE #ErrorMessage AS NVARCHAR(4000);
SET #ErrorMessage
= N'Check 2: Transaction in an invalid or closed state (##TRANCOUNT=' + CAST(##TRANCOUNT AS NVARCHAR(10))
+ N'). Exactly 1 transaction should be open at this point. Rolling-back any pending transactions.';
RAISERROR(#ErrorMessage, 16, 127);
RETURN;
END;
UPDATE [b]
SET [a_id] = [a].[a_id]
FROM [a]
WHERE [a].[a_name] = [b].[a_name];
GO
IF ##TRANCOUNT <> 1
BEGIN
DECLARE #ErrorMessage AS NVARCHAR(4000);
SET #ErrorMessage
= N'Check 3: Transaction in an invalid or closed state (##TRANCOUNT=' + CAST(##TRANCOUNT AS NVARCHAR(10))
+ N'). Exactly 1 transaction should be open at this point. Rolling-back any pending transactions.';
RAISERROR(#ErrorMessage, 16, 127);
RETURN;
END;
ALTER TABLE [b] ALTER COLUMN [a_id] [NVARCHAR](36) NOT NULL;
GO
IF ##TRANCOUNT <> 1
BEGIN
DECLARE #ErrorMessage AS NVARCHAR(4000);
SET #ErrorMessage
= N'Check 4: Transaction in an invalid or closed state (##TRANCOUNT=' + CAST(##TRANCOUNT AS NVARCHAR(10))
+ N'). Exactly 1 transaction should be open at this point. Rolling-back any pending transactions.';
RAISERROR(#ErrorMessage, 16, 127);
RETURN;
END;
ALTER TABLE [b] DROP COLUMN [a_name];
GO
COMMIT TRANSACTION;
My fave references on this topic
There is a wonderful free resource online which digs into error and transaction handling in great detail. It is written and maintained by Erland Sommarskog:
Part One – Jumpstart Error Handling
Part Two – Commands and Mechanisms
Part Three – Implementation
One common question is why XACT_ABORT is still needed/ if it is entirely replaced by TRY/CATCH. Unfortunately it is not entirely replaced, and Erland has some examples of this in his paper, this is a good place to start on that.
I'm working on an ETL project, actively populating tables with data. Sometimes if something is missing, a whole block of data modifications must be undone, so I'm using a transaction. When something goes wrong, a rollback applies, if not, an UPDATE STATISTICS could help efficiency.
So my question is what would be more efficient, to UPDATE STATISTICS inside the TRANSACTION or after the COMMIT?
Currently working fine as:
BEGIN TRY
BEGIN TRAN
UPDATE stuffs SET ...
INSERT things VALUES(...
UPDATE STATISTICS stuffs
UPDATE STATISTICS things
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0 ROLLBACK TRAN
RAISERROR( ... -- RAISERROR prevents from executing past this point
END CATCH
IF ##TRANCOUNT > 0 COMMIT TRAN
But maybe this is better
BEGIN TRY
BEGIN TRAN
UPDATE stuffs SET ...
INSERT things VALUES(...
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0 ROLLBACK TRAN
RAISERROR( ... -- RAISERROR prevents from executing past this point
END CATCH
IF ##TRANCOUNT > 0 COMMIT TRAN
UPDATE STATISTICS stuffs
UPDATE STATISTICS things
I've tried both with virtually the same results, but with more data or more rollbacks could be different.
I will use the second one.
The transaction will be rolled back when it raises the error(s) if you put them in the try block.
I believe you still need 'update statistics' to improve efficiency even after a rollback applies to your transaction. If so, it is better to keep 'update statistics' after the transaction commits.
Postgres version:
PostgreSQL 11.0 on x86_64-pc-linux-gnu,
compiled by gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-9), 64-bit
I have this stored procedure as shown below. This is just a test.
In that procedure I have 2 transactions:
The first one is supposed to finish OK (i.e. I have written the code
so that it does not encounter any errors and so it will reach the COMMIT statement).
The second transaction is supposed to fail as I intentionally
introduce an error in it (either via this cast there, or via an INSERT which causes PK violation).
Also, yb.print_now is a simple function which is just logging (inserting) messages to another table.
When I run this stored procedure I am expecting the updates and the logging of messages done by
the first transaction to be persisted in the database, even though the 2nd transaction failed.
But this is not what happens, both transactions seem to be rolled back.
Why is this? Am I doing something wrong?
And 2 more questions which are very important to me. :
When an error occurs (say like on line marked ***) and when control reaches/jumps to the EXCEPTION block, I have the feeling that the transaction I was in is already rolled back before I even reach the EXCEPTION block.
So in the exception block I cannot do ROLLBACK or COMMIT or anything
related to the transaction. Is that feeling correct?
Say I want to commit all the stuff done, despite of the error, is there a way I can do that?
That's exactly what I want here. The error is an error... OK, but I want everything
which happened before I got the error to get committed.
How do I do this in Postgres 11?
CREATE OR REPLACE PROCEDURE yb.test123()
LANGUAGE plpgsql
AS $procedure$
DECLARE
var_cnt int;
c int;
BEGIN
START TRANSACTION; --- 1 ---
raise notice '001.';
PERFORM yb.print_now('===> 0010.');
var_cnt = 0;
update yb.mbb
set the_price = the_price + 1
where
the_id = 23164;
raise notice '002.';
PERFORM yb.print_now('===> 0020.');
raise notice '003.';
PERFORM yb.print_now('===> 0030.');
update yb.mbb
set the_price = the_price + 1
where
the_id = 23164;
COMMIT; --- 1 ---
START TRANSACTION; --- 2 ---
c = cast('###a1e3Z' as int); --- *** ---
raise notice '004.';
PERFORM yb.print_now('===> 0040.');
update yb.mbb
set the_price = the_price + 1
where
the_id = 23164;
-- insert into yb.mbb(the_id)
-- values (23164); -- this will throw duplicate PK error
raise notice '005.';
PERFORM yb.print_now('===> 0050.');
COMMIT; --- 2 ---
EXCEPTION
WHEN OTHERS THEN
raise notice 'We are in the exception block now.';
-- ROLLBACK;
-- COMMIT;
RETURN;
END
$procedure$;
The error happens right at the start of your procedure, in the statement
START TRANSACTION;
As the documentation says:
A new transaction is started automatically after a transaction is ended using these commands, so there is no separate START TRANSACTION command.
That should answer your first question.
As to the second, when you are in the exception branch, you have effectively rolled back the subtransaction that started with the BEGIN that belongs to the EXCEPTION clause (or after the last COMMIT). You are still in the transaction though, so you can issue COMMIT and ROLLBACK.
To your third question: No, there is no way to commit “everything up to the last exception”. You could only have that by wrapping every statement in a BEGIN ... EXCEPTION ... END block, but that would seriously hurt your performance (apart from making your code unreadable).
Use BEGIN ... EXCEPTION ... END blocks judiciously whenever you expect that a statement could fail.
I have tried to wrap my brain around this, but can't make it work, so I present a little testcase here, and hopefully someone can explain it to me:
First a little test database:
CREATE DATABASE test;
USE test;
CREATE TABLE testA (nr INT)
GO
CREATE TRIGGER triggerTestA
ON testA
FOR INSERT AS BEGIN
SET NOCOUNT ON;
IF EXISTS (SELECT nr FROM Inserted WHERE nr > 10)
RAISERROR('Too high number!', 16, 1);
END;
And here is a tSQL test, to test the behaviour:
ALTER PROCEDURE [mytests].[test1] AS
BEGIN
EXEC tSQLt.FakeTable #TableName = N'testA'
EXEC tSQLt.ApplyTrigger
#TableName = N'testA',
#TriggerName ='triggerTestA'
EXEC tSQLt.ExpectException
INSERT INTO dbo.testA VALUES (12)
END;
This test will run ok - but the trigger doesn't do what I want: prevent user from entering values > 10. This version of the trigger does what I want:
CREATE TRIGGER triggerTestA
ON testA FOR INSERT AS BEGIN
SET NOCOUNT ON;
BEGIN TRANSACTION;
BEGIN TRY
IF EXISTS (SELECT nr FROM Inserted WHERE nr > 10)
RAISERROR('Too high number!', 16, 1);
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
THROW;
END CATCH;
END;
But now the test fails, stating A) there is an error (which was expected!) and B) that there is no BEGIN TRANSACTION to match a ROLLBACK TRANSACTION. I guess this last error is with the tSQLt surrounding transaction, and that my trigger somehow interferes with that, but it is sure not what I expect.
Could someone explain, and maybe help me do it right?
tSQLt is currently restricted to run tests in its own transaction and reacts, as you have seen, unwelcoming when you fiddle with its transaction.
So, to make this test work, you need to skip the rollback within the test but not outside.
I suggest this approach:
Remove all transaction handling statements from the trigger. You don't need to begin a transaction anyway as triggers are always executed inside of one.
If you find a violating row, call a procedure that does the rollback and the raiserror
Spy that procedure for your test
To test that procedure itself, you could use tSQLt.NewConnection
In T-Sql, rollback transaction rolls back all transactions except specified a save point name. To roll back only part of modifications, we use rollback transaction #save_point_name. That is a save transaction #save_point_name has to be earlier stated. If the referred save point has already been rolled back (removed form the transaction log), an error is raised. Likewise, if there isn't an active transaction, a begin transaction #transaction_name needs to be stated, and can be rolled back in the same manner. This enables fast discoveries of bugs or using try...catch mechanism.
Unlike rollback, commit transaction #transaction_name ignores its #transaction_name part completely and just performs a commit, either decreasing ##trancount or ending the transaction. So there is no way in knowing, or specifying, which (nested) transaction, or for that matter - a save point, is being (pseudo) committed. I know transactions weren't meant to be nested in the first place, hence the existence of save points.
A typical approach is to, in each procedure, check ##trancount to determine whether to create a save point or to begin a new transaction. Then later, confirm the determination, check transaction state and commit or roll back (or do nothing) accordingly.
This checking is a lot of boilerplate especially when you have a lot of procedures calling (multiple) procedures all of which only rollback their own actions if some thing went wrong. So my attempt was to abstract the transactioning so that one could simply write something like this.
create procedure RollBackOnlyMyActions
as
declare #SavePointName nvarchar(32) = N'RollBackToHere';
begin try
exec CreateSavePoint #SavePointName;
--do stuff
--call other procedures that may roll back only its actions
--do other stuff
exec CommitSavePoint #SavePointName;
end try
begin catch
exec RollBackSavePoint #SavePointName;
end catch
where (spare me for the print-outs)
create procedure dbo.CreateSavePoint
#SavePointName nvarchar(32)
as
declare #Msg nvarchar(255);
print N'Preparing to create save point ['+#SavePointName+N'].';
print N'Checking not in an uncommitable transaction.';
if xact_state() != -1
begin
print N'Not in an uncommitable transaction. Checking for transaction existence.';
if ##TranCount = 0
begin
print N'No active transaction. Starting a new one.';
begin transaction #SavePointName;
end
else
begin
print N'In active transaction. Saving transaction point.';
save transaction #SavePointName;
end
end
else
begin
print N'In an uncommitable transaction. No use saving. Throwing exeption.';
set #Msg = N'Uncommitable transaction.';
throw 50000,#Msg,1;
end
go
create procedure dbo.CommitSavePoint
#SavePointName nvarchar(32)
as
declare #Msg nvarchar(255);
print N'Preparing to commit save point ['+#SavePointName+N'].';
print N'Checking not in an uncommitable transaction.';
if xact_state() != -1
begin
print N'Not in an uncommitable transaction. Checking transaction count.';
if ##trancount > 1
begin
print N'In nested transaction of '+convert(nvarchar(255),##trancount)+N'. Committing once.';
end
else if ##trancount = 1
begin
print N'In outter transaction. Committing.';
end
else
begin
print N'No active transaction. Throw exception.';
set #Msg = N'No transaction to commit.';
throw 50000,#Msg,1;
end
commit transaction;
end
else
begin
print N'In an uncommitable transaction. Cannot commit. Throwing exeption.';
set #Msg = N'Uncommitable transaction.';
throw 50000,#Msg,1;
end
go
create procedure dbo.RollbackSavePoint
#SavePointName nvarchar(32)
as
declare #Msg nvarchar(255);
print N'Prepare to rollback savepoint ['+#SavePointName+N']';
print N'Checking not in an uncommitable transaction.';
if xact_state() != -1
begin
print N'Not in an uncommitable transaction. Trying rollback';
begin try
rollback transaction #SavePointName;
end try
begin catch
print N'Something went wrong. Rethrowing exception.';
throw;
end catch
end
else
begin
print N'In an uncommitable transaction. No use rolling back. Throwing exeption.';
set #Msg = N'Uncommitable transaction.';
throw 50000,#Msg,1;
end
go
Well this didn't work. As I get a
Msg 266, Level 16, State 2, Procedure CreateSavePoint, Line 0
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 0, current count = 1.
after the first call to CreateSavePoint. That is, it seems sql server doesn't like managing transactions across multiple procedures.
So. Is there any workaround for this, such as, a way to suppress this error? Or am I missing an important concept here?
The rollback issue can be resolved by using SET XACT_ABORT ON which is "auto rollback" (simply) and suppresses error 266.