We have kind of an interesting situation here. We are using the repository pattern with Entity Framework so each database table has its own Repository class that accepts an instance of a DbContext in its constructor. We are also using Ninject for dependency injection. We have defined a single context to be instantiated during a given request so that when a multiple of repositories ask for a DbContext, the same instance is used throughout. This allows us to follow a UnitOfWork pattern so multiple things can occur on multiple repositories and a single commit will commit all changes to the database.
Here is the issue, we are also using SQL Azure Federations which splits up our client data into multiple databases (sharding). We need to be able to jump from one federation member (database) to another within the same request but want to be able to use the same service/repository methods that are dependent upon the injected context. Our first thought was to just execute the USE FEDERATION sql command on the existing context to move to the next database but it seems to work sometimes and not others. After executing the statement we see that the underlying connection on the context really is pointing to the new server but for some reason queries executed on that context end up returning results from the previous connection. I presume using the same instance of a context on multiple database is not something that is really natively supported as you would normally spin up a new context when connecting to a new database. Unfortunately we have a bunch of repositories that are created dynamically using Ninject which then take in the same instance of the context so even if we do spin up a new context for the new database, we have no way to making all the existing repositories suddenly become dependent upon the new context we just spun up instead of the one that was created at the initial request.
Here are a few solutions we can think of but are not sure how to get any of them to work:
Change the database used on an existing context (as explained above but didn't seem to work)
Swap out an injected context for all repositories with a new one so that all existing repositories now become dependent upon a different context
Somehow request from Ninject a new instance of all the repositories passing in the parameters needed for the context once it is requested.
Again, bottom line here is we have a set of repositories and services that are dependent upon a single Context and we want to be able to reuse those services and repositories but swap out or change the context, on which all are dependent, to point to a new server.
Solution was to just completely drop Ninject out of the scenario. Not the best solution but the tools we are using were not really designed to do what we wanted in the environment we are working.
Related
I´m getting PK Violation Exception when using EF Core 2.1 DbContext in an Azure QueueTrigger function. Guess is due to the nature of DbContext not being thread-safe, and the Azure Function running different instances in parallel. I have read quite a few, but I can´t find a good approach to solve this.
Here is my scenario (producer-consumer pattern):
I have a Scheduled Azure Function that is calling an API to get Projects from different external systems. To get all the required info for a project, I need to run different Queries to other external services, so I´m decoupling this to another Azure function, so the Scheduled function just queues a message per Project, as “Sync Project ID 101”.
Another QueueTrigger Function fires every time a message is queued, so, it means different instances running in parallel. This function must gather all the data of a specific Project, and that means more calls to other external services / APIs, to (some kind of) aggregate all the info about a Project. IMHO it´s good to do it that way, as I can process multiple Projects in parallel, and I can scale the Function if I need it.
Once I have all this Project info, I want to persist it in a SQL DB using EF Core (and here comes the issue)
Project data includes Users in the Project, and each user have a specific GUID as PK (coming from the external system). That means I can have repeated Users IDs in different Function instances, and here is the problem, as when I try to persist User info in a SQL Table, I can get PK Duplication exception, as multiple Function instances can try to Insert the same User at the same time (when the instance A check if user exists, it gets False, but another instance B is actually adding this User, so when instance A tries the Insert, it fails).
Guess I can lock DbContext somehow, but not sure if is good, as I also have a website doing Queries to the SQL DB (read-only queries for now, but could be updates in future too).
Another idea could be to send the entire Project info to another Queue / Blob file, and have another function in Singleton mode that Insert the data into SQL.
I´ve created this project simplifying my scenario, but enough to reproduce the issue and understand the problem.
https://github.com/luismanez/queuetrigger-efcore-multithreading
Any other ideas or recommended approaches? (open to change the architecture if find something better)
Many thanks!
A "more easy" way could be to do some kind of upsert in the database. There is a sample of how to do that with EF Core: https://www.flexlabs.org/2018/02/adding-upsert-support-for-entity-framework-core
I'm using Prism framework with EF in WPF application.
ViewModel:
keeps service references (passed by unity container).
Services:
are providing "high level" operations with data
keeps reference of Repository, which provides basic CRUD operations with database (single table per repository).
Repository:
every method in repository uses the "using" pattern, where I work with short lived object context.
This is where i got stuck: after object context is disposed, I can not work with mapped properties no longer. My database model is complex (many related tables) and many .Include() calls when retrieving data keeps code dirty.
After reading several threads, I discovered that "unit of work" pattern is probably what I need.
Here comes my question:
Who keeps reference of unit of work (and therefore context)?
If I choose context per view approach, viewModel should have context reference.
How can I inject unit of work to my services then? Or should I create new instance of Service in ViewModel and pass context in constructor parameter?
We're using a similar architecture in a project:
Each ViewModel gets its own Service object which gets injected in the constructor (at least the top level ones that directly correspond to a View. Some hierarchical ViewModels may reuse their parent's Service, but let's keep it simple here).
By default, each Service operation creates a new context, but...
Services have BeginContext and EndContext methods which can be called
by the ViewModels to keep the context open over multiple operations.
This worked pretty well for us. Most of the time we call BeginContext when a View is opened and EndContext when it is closed.
I've deployed to Azure and Azure SQL, which doesn't support MSDTC and I'm having trouble understanding how to rework my code to prevent what I assume is nested connections. I'm fairly new to EF and my knowledge of TransactionScope is not wonderful, so I'm not sure that I have the right pattern.
I am trying to use repos, which call on a shared instance of the ObjectContext (I tried to dispose on EndRequest but had issues, so this is another problem for me).
I have a transaction which calls SaveChanges on the ObjectContext instance several times, but at some point it becomes disposed. What is governing this and can you recommend what I can do to get it working correctly?
If you want to avoid issues with distributed transaction you must handle connection manually because you need only one opened connection per TransactionScope = one context instance with one connection used for all queries and database updates. The code should look like:
using (var context = new YourObjectContext()) {
context.Connection.Open();
...
}
I am trying to use repos, which call on a shared instance of the
ObjectContext (I tried to dispose on EndRequest but had issues, so
this is another problem for me).
If you share your context instance among multiple request or even worse if you use just single context instance to handle all your requests you should stop now and completely redesign your application. Otherwise it will not work correctly.
In my MVC controllers, I'm using an IoC container (Ninject), but am not sure how to best use when it comes to the Entity Framework ObjectContext.
Currently, I'm doing something like:
using(var context = new MyObjectContext())
{
var stuff = m_repository.GetStuff(context);
}
This is the best way to manage from the point of view of keeping the database connection open for the shortest time possible.
If I were to create the ObjectContext via Ninject on a per request basis, this obviously keeps the database connection open for too long.
Also the above code would become...
var stuff = m_repository.GetStuff(m_myObjectContext);
(And when would I dispose of the context...?)
Should I be creating a factory for the ObjectContext and pass that in via DI? This would loosen the coupling, but does this really help with testability if there is no easy means of maintaining an interface for the ObjectContext (that I know of)?.
Is there a better way? Thanks
This is the best way to manage from the point of view of keeping the
database connection open for the shortest time possible.
If I were to create the ObjectContext via Ninject on a per request
basis, this obviously keeps the database connection open for too long.
Entity Framework will close the connection directly after the execution of each query (except when supplying an open connection from the outside), so your argument for doing things like this does not hold.
In the past I used to have by business logic (my command handlers to be precise) have control over the context (create, commit, and dispose it), but the downside is that you need to pass on this context to all other methods and all dependencies. When the application logic gets more complex, this results in less readable, less maintainable code.
For that reason I moved to a model where the unit of work (your MyObjectContext) is created, committed, and disposed outside the control of the business logic. This allows you to inject the unit of work into all dependencies and reuse the same unit of work for all objects. Downside is that this makes your DI configuration a bit harder. Some things your need to make sure of:
The unit of work must be created as per web request or within a certain scope.
The unit of work must be disposed at the end of the request or scope (although it is probably not a problem when the DbContext is not disposed, since the underlighing connection is closed and DbContext does not implemente a finalizer).
You need to explicitly commit the unit of work, but you can't do this at the end of the web request, since at that point you have no idea whether it is safe to commit (since you don't want to commit when your business logic threw an exception, but at the end of the request there is no way to correctly detect if this actually happened).
One tip I can give you is to model the business logic in the system around command handlers, since this allows you do define a single decorator that handles the transactional behavior (committing the unit of work and perhaps even running everything in a database transaction) at a single point. This decorator can be wrapped around each handler in the system.
I must admit that I have no idea how to register generic types and generic decorators with Ninject, but you'll probably get an answer quickly when asking here at Stackoverflow.
Is it better to create and dispose a context in the repository class? or is it better to create the context in the service and pass it to each repository through dependency injection?
If it is only created in the repository, the methods cannot do lazy loading (no use returning IQueryable) in the service/business layer.
Most definitally create it outside the repository. If you create it inside the service the code is strongly coupled to the instance type of the context. This means that you cant have one of your repositories without also having EF. what you really want to do is layer your system so that a repository can use EF as the data source or possibly something else (such as an in memory table).
This matters a lot when it comes to unit testing as you dont want to have to instantiate your database in order to run a unit test.
Take a look at http://blog.staticvoid.co.nz/2011/10/staticvoid-repository-pattern-nuget.html which is my take on how a repository should be implemented with EF4, also i've created a nuget package to implement this pattern in such a way that the code to get a nicely decoupled repository system is about 4 lines.