I have created several custom activities that update tables in my DB (in this case SQL Server Compact), using Entity Framework 4 with POCOs.
If I put more than one of these inside a WF4 TransactionScope activity, I'm running into problems: EF disposes the DB connection after the first activity has finished, and when the next DB activity tries to do a DB update a new connection is built up. At this moment an exception is thrown.
System.Activities.WorkflowApplicationAbortedException : The workflow has been aborted.
----> System.Data.EntityException : The underlying provider failed on Open.
----> System.InvalidOperationException : The connection object can not be enlisted in transaction scope.
Do I have to keep the EF connection open during the whole transaction scope? How can I do that? Create an explicit custom activity for that, or is there a standard way?
My current workaround goes like this: I created a new code activity that creates our ObjectContext and explicitely calls dbContext.Connection.Open(). It returns the ObjectContext, which is then saved in a workflow variable. That one is passed to all the DB related activities as an InArgument<>. Inside my DB activities, I use this ObjectContext if it is passed in, otherwise I create a new one.
This does work, but I'm not satisfied with this solution: It needs the new InArgument for every DB related activity. In the workflow designer, I have to insert that special OpenDatabaseConnection activity inside the transaction scope, and then make sure that the correct variable is passed into all DB activities. This seems to be very inelegant and error prone, especially if other team members have to use these DB activities.
What would be a better way to handle this?
The problem is that when you open a second connection in the same transaction scope, an attempt is made to promote the transaction to a distributed transaction (even though there's nothing distributed about it since you connect to the same database). SQL Server CE doesn't support this scenario.
What I would do is create a custom 'container' activity that opens (and closes) the connection and makes it available to child activities. This is still not optimal but at least you no longer need to pass InArgument's around. You get the following activity tree:
TransactionScope
InitializeConnection
Sequence
CustomDataActivity1
CustomDataActivity2
CustomDataActivity3
InitializeConnection is a NativeActivity that uses NativeActivityContext.Properties to expose the connection (or the ObjectContext) to child activities.
Make sure you implement proper error handling to ensure you close the connection at all times.
NOTE: Distributed transactions are supported by the full SQL Server only through a Windows service called MSDTC (Microsoft Distributed Transaction Coordinator). You can find this one in your 'Local Services'. Since SQL Server CE is a database that should be able to operate completely standalone, it makes sense that it has no dependency on MSDTC. Therefore it has no support for distributed transactions.
Related
I´m getting PK Violation Exception when using EF Core 2.1 DbContext in an Azure QueueTrigger function. Guess is due to the nature of DbContext not being thread-safe, and the Azure Function running different instances in parallel. I have read quite a few, but I can´t find a good approach to solve this.
Here is my scenario (producer-consumer pattern):
I have a Scheduled Azure Function that is calling an API to get Projects from different external systems. To get all the required info for a project, I need to run different Queries to other external services, so I´m decoupling this to another Azure function, so the Scheduled function just queues a message per Project, as “Sync Project ID 101”.
Another QueueTrigger Function fires every time a message is queued, so, it means different instances running in parallel. This function must gather all the data of a specific Project, and that means more calls to other external services / APIs, to (some kind of) aggregate all the info about a Project. IMHO it´s good to do it that way, as I can process multiple Projects in parallel, and I can scale the Function if I need it.
Once I have all this Project info, I want to persist it in a SQL DB using EF Core (and here comes the issue)
Project data includes Users in the Project, and each user have a specific GUID as PK (coming from the external system). That means I can have repeated Users IDs in different Function instances, and here is the problem, as when I try to persist User info in a SQL Table, I can get PK Duplication exception, as multiple Function instances can try to Insert the same User at the same time (when the instance A check if user exists, it gets False, but another instance B is actually adding this User, so when instance A tries the Insert, it fails).
Guess I can lock DbContext somehow, but not sure if is good, as I also have a website doing Queries to the SQL DB (read-only queries for now, but could be updates in future too).
Another idea could be to send the entire Project info to another Queue / Blob file, and have another function in Singleton mode that Insert the data into SQL.
I´ve created this project simplifying my scenario, but enough to reproduce the issue and understand the problem.
https://github.com/luismanez/queuetrigger-efcore-multithreading
Any other ideas or recommended approaches? (open to change the architecture if find something better)
Many thanks!
A "more easy" way could be to do some kind of upsert in the database. There is a sample of how to do that with EF Core: https://www.flexlabs.org/2018/02/adding-upsert-support-for-entity-framework-core
I am working on a very large application with over 100 modules, and almost 500 tables in the database. We are converting this application to WPF/WCF using Entity Framework 4.2 Code First. Our database is SQL Anywhere 11. Because of the size of the database, we are using an approach similar to Bounded DbContexts, as described here http://msdn.microsoft.com/en-us/magazine/jj883952.aspx by Julie Lerman.
Each of our modules creates its own DbContext, modeling only the subset of the database that it needs.
However, we have run into a serious problem with the way DbContexts are created. Our modules are not neatly self-contained, nor can they be. Some contain operations that are called from several other modules. And when they are, they need to participate in transactions started by the calling modules. (And for architectural reasons, DTC is not an option for us.) In our old ADO architecture, there was no problem passing an open connection from module to module in order to support transactions.
I've looked at various DbContext constructor overloads, and tried managing the transaction from EntityConnection vs. the StoreConnection, and as far as I can tell, there is no combination that allows ModuleA to begin a transaction, call a function in ModuleB, and have ModuleB's DbContext participate in the transaction.
It comes down to two simple things:
Case 1. If I construct DbContextB with DbContextA's EntityConnection, DbContextB is not built with its own model metadata; it reuses DbContextA's metadata. Since the Contexts have different collections of DbSets, all ModuleB's queries fail. (The entity type is not a part of the current context.)
Case 2. If I construct DbContextB with ModuleA's StoreConnection, DbContextB does not recognize the StoreConnection's open transaction at the EntityConnection level, so EF tries to start a new transaction when ModuleB calls SaveChanges(). Since the database connection in fact has an open transaction, this generates a database exception. (Connection does not support parallel transactions.)
Is there any way to 1) force DbContextB to build its own model in Case 1, or 2) get DbContextB's ObjectContext to respect its StoreConnection's transaction state in Case 2?
(By the way, I saw some encouraging things in the EF6 alpha, but after testing it out, found the only difference was that I could create DbContextB on an open connection. But even then, the above 2 problems still exist.)
I suggest you try use the TransactionScope object to manage this task for you. As long as all of your DbContexts use the same connection string (not connection object) the transaction should not try to enlist MS-DTC.
I've deployed to Azure and Azure SQL, which doesn't support MSDTC and I'm having trouble understanding how to rework my code to prevent what I assume is nested connections. I'm fairly new to EF and my knowledge of TransactionScope is not wonderful, so I'm not sure that I have the right pattern.
I am trying to use repos, which call on a shared instance of the ObjectContext (I tried to dispose on EndRequest but had issues, so this is another problem for me).
I have a transaction which calls SaveChanges on the ObjectContext instance several times, but at some point it becomes disposed. What is governing this and can you recommend what I can do to get it working correctly?
If you want to avoid issues with distributed transaction you must handle connection manually because you need only one opened connection per TransactionScope = one context instance with one connection used for all queries and database updates. The code should look like:
using (var context = new YourObjectContext()) {
context.Connection.Open();
...
}
I am trying to use repos, which call on a shared instance of the
ObjectContext (I tried to dispose on EndRequest but had issues, so
this is another problem for me).
If you share your context instance among multiple request or even worse if you use just single context instance to handle all your requests you should stop now and completely redesign your application. Otherwise it will not work correctly.
I'm using EF code-first to manage my DB connection, with an explicit connection string declared in web.config. I would like to schedule some DB cleaning process (like deleting test transactions every day), so I'm spawning a thread from Role.OnStart() with proper concurrency management among the instances.
But I'm getting DB database exceptions, like the DB not matching my model, whereas I'm sure it does (the same code used from "inside" the app works well). So my guess is that web.config is not used from the thread, so EF probably uses the default connection string.
What would be the best way to use my connection string from there ?
Thanks
The OnStart method doesn't run in the same process as your web application meaning it doesn't make use of the web.config. I suggest you store the connection string in the service configuration and read it from here when initializing your context.
An other advantage is that you change the setting without re-deploying the application.
This problem is not readily reproducible in a simple example here but was wondering if anyone has any experience and tips, here is the issue:
using Entity Framework
have many points in application where (1) data is written to some entity table e.g. Customer, (2) data is written to history table
both of these actions use Entity Framework, HOWEVER, they use different contexts
these actions need to be both in one transaction: i.e. if one fails to write, the other should not write, etc.
I can wrap them with a TransactionScope,
like this:
using (TransactionScope txScope = new TransactionScope()) {
...
}
but this gives me:
Microsoft Distributed Transaction Coordinator (MSDTC) is disabled for
network transactions.
Our database admin has told me that MSDTC is disabled by choice and can not be installed.
Hence I am making changes trying to create my own EntityConnection with a MetadataWorkspace with the idea that each context will use the same EntityConnection. However, this is proving near impossible trying to get it to work, e.g. currently I continue to get the above error even though theoretically both contexts are using EntityConnection. It's difficult to understand where/why Entity Framework is requiring the MSDTC for example.
Has anyone gone down this road before, have experience or code examples to share?
Well, the problem is quite easy.
If you are using sql server 2008 you should not have that problem because you have promotable transaction, and as .NET knows that you are using the same persistence store (the database) it wont promote it to DTC and commit it as local. look into promotable transaction with sql server 2008.
As far as I know Oracle is working in its driver to support promotable transactions, but I do not know the state, MS oracle driver does not support it.
http://www.oracle.com/technology/tech/windows/odpnet/col/odp.net_11.1.0.7.20_twp.pdf
If you are using a driver that do not support promotable transactions it is impossible for .NET to use local transaction doing two connections. You should change your architecture or convince the database admin for installing MSDTC.
I had a similar problem with SQL 2008, Entity Framework.
I had two frameworks defined (EF1, and EF2) but using identical connection strings to a sql 2008 database.
I got the MSDTC error above, when using nested "usings" across both.
eg the code was like this:
using (TransactionScope dbContext = new TransactionScope())
{
using (EF1 context = new EF1())
{
// do some EF1 db call
using (EF2 context2 = new EF2())
{
// do some EF2 db call
}
}
dbContext.Complete();
}
It wasnt as simple as this, because it was split across several methods, but this was the basic structure of "usings".
The fix was to only open one using at a time. No MTDSC error, No need to open distributed transactions on db.
using (TransactionScope dbContext = new TransactionScope())
{
using (EF1 context = new EF1())
{
// do some EF1 db call
}
using (EF2 context2 = new EF2())
{
// do some EF2 db call
}
dbContext.Complete();
}
I think that what you need to do is to force your contexts to share single database connection. You will be able then to perform these two operations against two different contexts in single transaction. You can achieve this by passing one EntityConnection object to both of your context's constructors. Of course this approach will require you to pass this object to methods which update DB.
I have recently blogged about creating database context scope which will make using multiple EF contexts and transactions easier.