I am trying to replace database pessimistic lock with Curator interprocess lock. With a database lock, I take a select for update lock on an entity at the beginning of a transaction and it is released when the transaction is committed. The transaction could span over multiple methods and if any method in the chain tries to acquire this lock again, it gets it as it is running in the same transaction/thread.
Now with zookeeper lock, the examples that I've seen look like this:
InterProcessMutex lock = new InterProcessMutex(client, lockPath);
if ( lock.acquire(maxWait, waitUnit) )
{
try
{
// do some work inside of the critical section here
}
finally
{
lock.release();
}
}
This would create a problem if I service call spans over multiple methods and any of those methods individually tries to acquire this lock. I don't want the lock to be passed around as these are independent business methods. Could there be some service/factory implementation which could return this same lock object if I am running in same thread?
Related
I have a PUT request that is too long to run. I'd like to make it async, using continuations (await/promise feature).
I create a job (LongJobThatUpdatesThePassedEntity) that modifies my entity
public static void myLongPut(#required Long id, String someData) {
MyJpaModel myJpaModel = MyJpaModel.findById(id);
//straightforward modifications
updateMyJpaModel(someData);
myJpaModel.save();
//long processing modifications to entity, involving WS calls
Promise<String> delayedResult = new LongJobThatUpdatesThePassedEntity(id).now();
await(delayedResult);
render(myJpaModel.refresh());
}
How are the DB transactions managed?
is there a commit before the job's call?
the job has it's own DB transaction?
if there is an issue in the LongJobThatUpdatesThePassedEntity that rollsback, the modifications done in updateMyJpaModel are persisted?
can I do render(myJpaModel.refresh()) at the end?
will it contain the straighforward modifications and the long ones?
thank's
I can answer most of your question for Play 1.4.3, which is the version I'm currently using. I don't expect that much has changed since Play 1.2.
How are the DB transactions managed?
Play! handles the transactions for jobs and controller actions using an "invocation", which is a Play-specific concept. In short, for any invocation, each plugin gets a chance to do some setup and cleanup before and after the invoked method runs. For database access, the JPAPlugin.withinFilter method starts and closes the transaction using the JPA class's helper methods.
is there a commit before the job's call?
When you call await(Future<T>), it has the effect of closing the current transaction and starting a new one. The specific mechanism is that it throws a "Suspend" exception, which bubbles up to PlayHandler$NettyInvocation and causes the afterInvocation callbacks to be called. This causes JPAPlugin.afterInvocation to call
JPA.closeTx() which either commits or rollsback the transaction, as appropriate.
When the Job exits and the await() continuation is resumed. This is also handled as an invocation, so the transaction is started in the same way as before, using JPAPlugin.withinFilter(). However, unlike before, the controller action is not the target of the invocation, but instead ActionInvoker.invoke() calls invokeWithContinuation, which restores the saved continuation state and resumes execution by returning from await().
JPA.withTransaction looks like it has some special logic to retain the same entity manager across the continuation suspend/resume. I think without this, you wouldn't be able to call refresh().
In your code, I think there's a race condition between when await() closes the transaction and the Job starts its transaction. That is, it's possible that the Job's transaction begins before the controller commits the "before await" transaction. To avoid this, you can explicitly call JPA.closeTx() before calling Job.now().
Based on code inspection, it looks like the way Play! is implemented, it so happens that the Job will exit and the Job's transaction will be closed before the "after await()" transaction is opened. I don't know if any
documentation that says this is an intended part of the await() contract, so if this is essential for your appliaction, you can avoid using undocumented behavior by committing the transaction just before your Job.doJobWithResult() method returns.
the job has it's own DB transaction?
Yes, unless its annotated to not have a transaction.
if there is an issue in the LongJobThatUpdatesThePassedEntity that rollsback, the modifications done in updateMyJpaModel are persisted?
Based on the explanation above, each of the three transactions are independent. If one is rolled back, I don't see how it would affect the others.
I have a message driven bean which is communicating with a database via EntityManager. The EM is injected via #PersistenceContext, like normal. I want to flush changes to an Entity immediately without waiting for the MDB to fully complete its processing of the given Message.
For example:
MDB's onMessage() {
Foo f = em.find(Foo.class, 123);
f.setNewStatus("Performing work!");
em.merge(f);
em.flush();
...
// Continue doing a lot of work...
...
f.setNewStatus("Done!");
em.merge(f);
em.flush();
}
The problem is that I never see the "Performing Work!" status from outside the context of the MDB (e.g. by logging into the DB directly and checking the tuple's value).
This appears to be related to transactions. From online material, it sounds like a transaction is started within the context of onMessage() and not committed until the method is complete. Hence, the intermediate status is never committed since we eventually write "Done!" which overwrites the Foo's value within the PersistentContext.
Is there a solution to this type of problem? Some way to control the context of the transaction?
I think what you want to achieve is to see changes from outside of the transaction before this transaction commits. Well this is only possible when transaction isolation is set to Read uncommitted which I dont think is default in your DB.
What you can do is to add method, that will log your data, with attribute: #TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
In this case, container will have to pause current transaction, create new one that will executed in this method, and when it finishes, main transaction will be resumed.
If I've got an environment with multiple instances of the same client connecting to a MongoDB server and I want a simple locking mechanism to ensure single client access for a short time, can I safely use my own lock object?
Say I have one object with a lockState that can be "locked" or "unlocked" and the plan is everyone checks that it is "unlocked" before doing "stuff". To lock the system I say:
db.collection.update( { "lockState": "unlocked" }, { "lockState": "locked" })
(aka UPDATE lockObj SET lockState = 'locked' WHERE lockState = 'unlocked')
If two clients try to lock the system at the same time, is it possible that both clients can end up thinking they "have the lock"?
Both clients find the record by the query parameter of the update
Client 1 updates the record (which is an atomic operation)
update returns success
Client 2 updates the document (it's already found it before client 1 modified it)
update returns success
I realize this is probably a very contrived case that would be very hard to reproduce, but is it possible or does mongo somehow make client 2's update fail?
Alternative approach
Use insert instead of update. insert is atomic and will fail if the document already exists.
To lock the system: db.locks.insert({someId: 27, state: “locked”}).
If the insert succeeds - I've got the lock and since the update was atomic, no one else can have it.
If the insert fails - someone else must have the lock.
If two clients try to lock the system at the same time, is it possible that both clients can end up thinking they "have the lock"?
No, only one client at a time writes to the lock space (Global, Database, Collection or Document depending on your version and configuration) and the operations on that lock space are sequential and one or the other (read or write, not both) per document so that other connections will not mistakenly pick up a document in a inbetween state and think that it is not locked by another client.
All operations on a single document are atomic, whether update or insert.
Background
I have both an MVC app and a windows service that access the same data access library which utilizes EntityFramework. The windows service monitors certain activity on several tables and performs some calculations.
We are using the DAL project against several hundred databases, generating the connection string for the context at runtime.
We have a number of functions (both stored procedures and .NET methods which call on EF entities) which because of the scope of data we are using are VERY db intensive which have the potential to block one another.
The problem
The windows service is not so important that it can't wait. If something must be blocked, the windows service can. Earlier I found a number of SO questions that stated that System.Transactions is the way to go when setting your transaction isolation level to READ UNCOMMITTED to minimize locks.
I tried this, and I may be misunderstanding what is going on, so I need some clarification.
The method in the windows service is structured like so:
private bool _stopMe = false;
public void Update()
{
EntContext context = new EntContext();
do
{
var transactionOptions = new System.Transactions.TransactionOptions();
transactionOptions.IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted;
using (var transactionScope = new System.Transactions.TransactionScope( System.Transactions.TransactionScopeOption.Required, transactionOptions))
{
List<Ent1> myEnts = (from e....Complicated query here).ToList();
SomeCalculations(myEnts);
List<Ent2> myOtherEnts = (from e... Complicated query using entities from previous query here).ToList();
MakeSomeChanges(myOtherEnts);
context.SaveChanges();
}
Thread.Sleep(5000); //wait 5 seconds before allow do block to continue
}while (! _stopMe)
}
When I execute my second query, an exception gets thrown:
The underlying provider failed on Open.
Network access for Distributed Transaction Manager (MSDTC) has been disabled. Please
enable DTC for network access in the security configuration for MSDTC using the
Component Services Administrative tool.
The transaction manager has disabled its support for remote/network
transactions. (Exception from HRESULT: 0x8004D024)
I can assume that I should not be calling more than one query in that using block? The first query returned just fine. This is being performed on one database at a time (really other instances are being run in different threads and nothing from this thread touches the others).
My question is, is this how it should be used or is there more to this that I should know?
Of Note: This is a monitoring function, so it must be run repeatedly.
In your code you are using transaction scope. It looks like the first query uses a light weight db transaction. When the second query comes the transaction scope upgrades the transaction to a distributed transaction.
The distributed transaction uses MSDTC.
Here is where the error comes, by default MSDTC is not enabled. Even if it is enabled and started, it needs to be configured to allow a remote client to create a distributed transaction.
I want confirm if the Transaction that Entity Framework's ObjectContext.Connection.BeginTransaction() method returns uses the support of MSDTC (Microsoft Distributed Transaction Coordinator) or not?
Is there any way to use transactions without support of MSDTC?
It will automatically promote to a transaction coordinated by MSDTC under certain conditions. Each time you call ObjectContext.SaveChanges() a new transaction is created if there isn't already one in scope (if there is already an active transaction in scope, it will enlist in that transaction). However, by default, the connection will also be opened and closed each time you call ObjectContext.SaveChanges(). So if you're calling ObjectContext.Connection.BeginTransaction() at the beginning of a method, then calling ObjectContext.SaveChanges() multiple times while holding onto the original transaction, with some versions of SQL Server and Entity Framework, this can cause the transaction to get promoted to MSDTC because it's now using different connections within a single transaction. If you're trying to avoid your transaction getting promoted to MSDTC, then explicitly open your connection at the beginning and close it when you're done:
using(MyEntities context = new MyEntities())
using(DbConnection conn = context.Connection)
{
conn.Open();
DbTransaction transaction = conn.BeginTransaction();
// now do stuff within the transaction scope
transaction.Commit();
}
However, it's recommended that you use a TransactionScope, as it's more flexible, less platform-dependent, and will make it easier on you if in the future you decide you do actually need to something that requires MSDTC. EntityFramework will automatically enlist in the transaction if there's an active TransactionScope:
using(TransactionScope transaction = new TransactionScope())
using(MyEntities context = new MyEntities())
using(DbConnection conn = context.Connection)
{
conn.Open();
// now do stuff within the transaction scope
transaction.Complete();
}