I am getting this error, and I have not been able to resolve:
System.Data.SqlClient.SqlException: 'The transaction operation cannot be performed because there are pending requests working on this transaction.'
What is going on is that a usual data operation is taking place as part of a Controller Action.
At the same time, there is a Filter that is running that logs the action to a database.
this._orderEntryContext.ServerLog.Add(serverLog);
return this._orderEntryContext.SaveChanges() > 0;
This is where the error occurs.
So it seems to me that there is two SaveChanges going on at the same time, and so the transaction gets fouled up.
Not sure how to resolve. They are both using the same context that is gotten through DI. A workaround was to create a second context manually, but I would rather stick to the DI pattern. But I don't know how to create a second Db Context in DI, or even if that is a good idea.
Perhaps I should be using SaveChangesAsync() on both calls to ensure that they do not step on each other?
Turns out the answer to this was to make the Context a transient service:
services.AddDbContext<OrderEntryContext>(options =>
options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection")), ServiceLifetime.Transient);
Then, I changed all repositories to also be transient:
services.AddTransient<AssociateRepository, AssociateRepository>();
Related
I have been looking for a way to design my API so it will be idempotent, meaning that some of that is to make my POST request routes idempotent, and I stumbled upon this article.
(If I have understood something not the way it is, please correct me!)
In it, there is a good explanation of the general idea. but what is lacking are some examples of the way that he implemented it by himself.
Someone asked the writer of the article, how would he guarantee atomicity? so the writer added a code example.
Essentially, in his code example there are two cases,
the flow if everything goes well:
Open a transaction on the db that holds the data that needs to change by the POST request
Inside this transaction, execute the needed change
Set the Idempotency-key key and the value, which is the response to the client, inside the Redis store
Set expire time to that key
Commit the transaction
the flow if something inside the code goes wrong:
and exception inside the flow of the function occurs.
a rollback to the transaction is performed
Notice that the transaction that is opened is for a certain DB, lets call him A.
However, it is not relevant for the redis store that he also uses, meaning that the rollback of the transaction will only affect DB A.
So it covers the case when something happends inside the code that make it impossible to complete the transaction.
But what will happend if the machine, which the code runs on, will crash, while it is in a state when it has already executed the Set expire time to that key and it is now about to run the committing of the transaction?
In that case, the key will be available in the redis store, but the transaction has not been committed.
This will result in a situation where the service is sure that the needed changes have already happen, but they didn't, the machine failed before it could finish it.
I need to design the API in such a way that if the change to the data or setting of the key and value in redis fail, that they will both roll back.
What is the solution to this problem?
How can I guarantee the atomicity of a changing the needed data in one database, and in the same time setting the key and the needed response in redis, and if any of them fails, rollback them both? (Including in a case that a machine crashes in the middle of the actions)
Please add a code example when answering! I'm using the same technologies as in the article (nodejs, redis, mongo - for the data itself)
Thanks :)
Per the code example you shared in your question, the behavior you want is to make sure there was no crash on the server between the moment where the idempotency key was set into the Redis saying this transaction already happened and the moment when the transaction is, in fact, persisted in your database.
However, when using Redis and another database together you have two independent points of failure, and two actions being executed sequentially in different moments (and even if they are executed asynchronously at the same time there is no guarantee the server won’t crash before any of them completed).
What you can do instead is include in your transaction an insert statement to a table holding relevant information on this request, including the idempotent key. As the ACID properties ensure atomicity, it guarantees either all the statements on the transaction to be executed successfully or none of them, which means your idempotency key will be available in your database if the transaction succeeded.
You can still use Redis as it’s gonna provide faster results than your database.
A code example is provided below, but it might be good to think about how relevant is the failure between insert to Redis and database to your business (could it be treated with another strategy?) to avoid over-engineering.
async function execute(idempotentKey) {
try {
// append to the query statement an insert into executions table.
// this will be persisted with the transaction
query = ```
UPDATE firsttable SET ...;
UPDATE secondtable SET ...;
INSERT INTO executions (idempotent_key, success) VALUES (:idempotent_key, true);
```;
const db = await dbConnection();
await db.beginTransaction();
await db.execute(query);
// we're setting a key on redis with a value: "false".
await redisClient.setAsync(idempotentKey, false, 'EX', process.env.KEY_EXPIRE_TIME);
/*
if server crashes exactly here, idempotent key will be on redis with false as value.
in this case, there are two possibilities: commit to database suceeded or not.
if on next request redis provides a false value, query database to verify if transaction was executed.
*/
await db.commit();
// you can now set key value to true, meaning commit suceeded and you won't need to query database to verify that.
await redis.setAsync(idempotentKey, true);
} catch (err) {
await db.rollback();
throw err;
}
}
There is a question on stackoverflow, but in my case I run the nodes from console: deployNodes, runnodes. So there is no StartedMockNode class to use transaction{} function
What’s wrong with it and how can I fix it?
Here is the method throwing the exception
serviceHub.withEntityManager {
persist(callbackData)
}
Debugged this through with Hayk on Slack.
DB transactions are handled by corda. These transactions are only generated at two points. During node startup so corda services can invoke database queries and inserts and inside of flows.
In this scenario, the database was being accessed from outside of node startup and a not during a flow's invocation.
To circumvent this, a new flow needs to be created that handles the db operations that were causing the error. The db operation can still be kept inside of a corda service but it must be called from a flow.
This flow does not need a responder. It should be annotated with #StartableByService and should not need #InitiatingFlow (need to double check that one). The contents of call simply call the db operation and return the result back to the caller.
TLDR - all db operations must be called from inside a flow or during node startup.
I have a PUT request that is too long to run. I'd like to make it async, using continuations (await/promise feature).
I create a job (LongJobThatUpdatesThePassedEntity) that modifies my entity
public static void myLongPut(#required Long id, String someData) {
MyJpaModel myJpaModel = MyJpaModel.findById(id);
//straightforward modifications
updateMyJpaModel(someData);
myJpaModel.save();
//long processing modifications to entity, involving WS calls
Promise<String> delayedResult = new LongJobThatUpdatesThePassedEntity(id).now();
await(delayedResult);
render(myJpaModel.refresh());
}
How are the DB transactions managed?
is there a commit before the job's call?
the job has it's own DB transaction?
if there is an issue in the LongJobThatUpdatesThePassedEntity that rollsback, the modifications done in updateMyJpaModel are persisted?
can I do render(myJpaModel.refresh()) at the end?
will it contain the straighforward modifications and the long ones?
thank's
I can answer most of your question for Play 1.4.3, which is the version I'm currently using. I don't expect that much has changed since Play 1.2.
How are the DB transactions managed?
Play! handles the transactions for jobs and controller actions using an "invocation", which is a Play-specific concept. In short, for any invocation, each plugin gets a chance to do some setup and cleanup before and after the invoked method runs. For database access, the JPAPlugin.withinFilter method starts and closes the transaction using the JPA class's helper methods.
is there a commit before the job's call?
When you call await(Future<T>), it has the effect of closing the current transaction and starting a new one. The specific mechanism is that it throws a "Suspend" exception, which bubbles up to PlayHandler$NettyInvocation and causes the afterInvocation callbacks to be called. This causes JPAPlugin.afterInvocation to call
JPA.closeTx() which either commits or rollsback the transaction, as appropriate.
When the Job exits and the await() continuation is resumed. This is also handled as an invocation, so the transaction is started in the same way as before, using JPAPlugin.withinFilter(). However, unlike before, the controller action is not the target of the invocation, but instead ActionInvoker.invoke() calls invokeWithContinuation, which restores the saved continuation state and resumes execution by returning from await().
JPA.withTransaction looks like it has some special logic to retain the same entity manager across the continuation suspend/resume. I think without this, you wouldn't be able to call refresh().
In your code, I think there's a race condition between when await() closes the transaction and the Job starts its transaction. That is, it's possible that the Job's transaction begins before the controller commits the "before await" transaction. To avoid this, you can explicitly call JPA.closeTx() before calling Job.now().
Based on code inspection, it looks like the way Play! is implemented, it so happens that the Job will exit and the Job's transaction will be closed before the "after await()" transaction is opened. I don't know if any
documentation that says this is an intended part of the await() contract, so if this is essential for your appliaction, you can avoid using undocumented behavior by committing the transaction just before your Job.doJobWithResult() method returns.
the job has it's own DB transaction?
Yes, unless its annotated to not have a transaction.
if there is an issue in the LongJobThatUpdatesThePassedEntity that rollsback, the modifications done in updateMyJpaModel are persisted?
Based on the explanation above, each of the three transactions are independent. If one is rolled back, I don't see how it would affect the others.
I have a message driven bean which is communicating with a database via EntityManager. The EM is injected via #PersistenceContext, like normal. I want to flush changes to an Entity immediately without waiting for the MDB to fully complete its processing of the given Message.
For example:
MDB's onMessage() {
Foo f = em.find(Foo.class, 123);
f.setNewStatus("Performing work!");
em.merge(f);
em.flush();
...
// Continue doing a lot of work...
...
f.setNewStatus("Done!");
em.merge(f);
em.flush();
}
The problem is that I never see the "Performing Work!" status from outside the context of the MDB (e.g. by logging into the DB directly and checking the tuple's value).
This appears to be related to transactions. From online material, it sounds like a transaction is started within the context of onMessage() and not committed until the method is complete. Hence, the intermediate status is never committed since we eventually write "Done!" which overwrites the Foo's value within the PersistentContext.
Is there a solution to this type of problem? Some way to control the context of the transaction?
I think what you want to achieve is to see changes from outside of the transaction before this transaction commits. Well this is only possible when transaction isolation is set to Read uncommitted which I dont think is default in your DB.
What you can do is to add method, that will log your data, with attribute: #TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
In this case, container will have to pause current transaction, create new one that will executed in this method, and when it finishes, main transaction will be resumed.
I've seen a fair few articles/posts that recommend not having more than one context per request when using EF.
Is it valid to have a second context for logging purposes such as 'user x did y', 'failed login from z' etc.
The rationale behind this is that I'd like these errors to be logged even if there is an error while using the "main" context, ie. foreign key issues etc.
Is there another way to do this or if I head down this road is there any things to try and avoid?
You can always have more context instances if your application logic really needs them and ability to persist log to database even with invalid data in the main context can be considered as such situation. You just need to ensure that your updates do not run in the same transaction (they must use different DB connection as well) - that should be a default behavior unless you use TransactionScope.