I'm trying to unit test EventStore persistence using JSON serialization and Sqlite running in inmemory-mode. I'm getting a "No such table: Commits" exception during EventStore initialization. I'm assuming this is because EventStore somehow closes the connection and opens a new one, causing it to see a new in-memory Sqlite instance (sans Commits table). Is there any way to make this work?
The SQLite implementation is interesting, especially when you're using the ":memory:" connection string. The acceptance tests surrounding the implementation depend upon the the database not "disappearing" between operations.
The fundamental design of the EventStore separates each discrete action into a separate operation which:
Depending upon the configuration, enlists in the ambient TransactionScope (if any)
Opens the connection
Builds the IDbCommand
Executes the IDbCommand
Evaluates the results
Disposes the connection (releasing it back to the pool)
Completes the TransactionScope (when no exceptions are thrown)
Disposes the TransactionScope
The issue that you're running into is that each invocation against the EventStore opens and closes the connection.
Now, there is a way around this because I wanted to support explicitly utilizing the same IDbConnection without releasing it back to the pool. EventStore v3.0 (which is in release candidate stage) has a method call that utilizes the same connection and which avoids connection tear down after each operation:
ConfigurationConnectionFactory.OpenScope("SQLite"); // SQLite = app.config connection key
Simply add this to the "using_the_persistence_engine" acceptance tests class and you're all set:
private static IDisposable scope;
Add this as the first line to "Establish context":
scope = ConfigurationConnectionFactory.OpenScope("SQLite");
Finally, the lastline in "Cleanup everything" should be:
scope.Dispose();
Related
This question already has answers here:
What is the best practice in EF Core for using parallel async calls with an Injected DbContext?
(2 answers)
Closed last year.
I have a problem with the concept of scope in dependency injection. I have registered my db context as a scope and And I save the user activity in a table using an asynchronous method without using "await".
// In Startup:
services.AddScoped<IDbContext, StorageSystemDbContext>();
services.AddScoped<IUserActivityService,UserActivityService>();
// In UserActivityService:
public async void LogUserActivityAsync(string controllerName, string actionName, ActionType actionType = ActionType.View, string data = "", string description = "")
{
await InsertAsync(new UserActivity
{
ControllerName = controllerName,
ActionName = actionName,
ActionType = actionType,
CreatedDateTime = DateTime.Now,
Description = description,
UserId = (await _workContext.CurrentUserAsync())?.Id
});
}
//In Controller:
_userActivityService.LogUserActivityAsync(CurrentControllerName, CurrentActionName,data);
I get the following error when I call same action twice immediately:
InvalidOperationException: A second operation was started on this context before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
I expected a new db context to be created with the second request, depending on the type of db context dependency registration, but according to this error, a new context was not created for the second request and used the previous one.
What is the reason for this?
I'm using Asp Net.Core MVC and EF in .Net Core 5
An injected DbContext into a service regardless of scoping will be one single reference when constructor injected. Calling multiple methods in that service will always use the same instance. AddedScoped with ASP.Net will scope the services (and DbContext) to the web request. This is the recommended scoping for a DbContext to ensure any entities loaded during a request can ensure that they are all tracked by the same DbContext instance and that DbContext should be alive for the life of that request. (i.e. to provided lazy loading support if needed) A Transient scoped dependency would mean the DbContext passed to 2 different services would be distinct references. This leads to problems where Service A calls another service to retrieve entities that it wants to associate with an entity it loaded and is trying to update. These entities are associated to a different DbContext resulting in errors or issues like duplicate data being created.
Even with a transient scope DbContext you would still have the exact same problem trying to run two calls from the same service in parallel, and there are many good reasons referenced in the comments not to use un-awaited async calls to do so. Even if your intention is to await multiple calls together, the only way to enable something like would be to internally scope the DbContext within the method call itself. This would typically involve injecting a DbContextFactory type class rather than a DbContext into the service, where the DbContextFactory is a dependency that can initialize and provide a new DbContext; Then:
using (var context = _contextFactory.Create())
{
// operations with DbContext. (context)
}
Even then you need to consider the DB synchronization guards like row and table locks / deadlocks which could rear their heads if you have a significant number of operations happening in parallel. Keep in mind with web applications the web server can be responding to a significant number of requests in parallel, each of which could be kicking off these processes at any time. (Works fine during development with 1 client, crawls/dies out in the real world.)
I found the answer here:
https://stackoverflow.com/a/44121808/4604557
If for some reason you want to run parallel database operations (and think you can avoid deadlocks, concurrency conflicts etc.), make sure each one has its own DbContext instance. Note however, that parallelization is mainly useful for CPU-bound processes, not IO-bound processes like database interaction. Maybe you can benefit from parallel independent read operations but I would certainly never execute parallel write processes. Apart from deadlocks etc. it also makes it much harder to run all operations in one transaction.
I've deployed to Azure and Azure SQL, which doesn't support MSDTC and I'm having trouble understanding how to rework my code to prevent what I assume is nested connections. I'm fairly new to EF and my knowledge of TransactionScope is not wonderful, so I'm not sure that I have the right pattern.
I am trying to use repos, which call on a shared instance of the ObjectContext (I tried to dispose on EndRequest but had issues, so this is another problem for me).
I have a transaction which calls SaveChanges on the ObjectContext instance several times, but at some point it becomes disposed. What is governing this and can you recommend what I can do to get it working correctly?
If you want to avoid issues with distributed transaction you must handle connection manually because you need only one opened connection per TransactionScope = one context instance with one connection used for all queries and database updates. The code should look like:
using (var context = new YourObjectContext()) {
context.Connection.Open();
...
}
I am trying to use repos, which call on a shared instance of the
ObjectContext (I tried to dispose on EndRequest but had issues, so
this is another problem for me).
If you share your context instance among multiple request or even worse if you use just single context instance to handle all your requests you should stop now and completely redesign your application. Otherwise it will not work correctly.
I have created several custom activities that update tables in my DB (in this case SQL Server Compact), using Entity Framework 4 with POCOs.
If I put more than one of these inside a WF4 TransactionScope activity, I'm running into problems: EF disposes the DB connection after the first activity has finished, and when the next DB activity tries to do a DB update a new connection is built up. At this moment an exception is thrown.
System.Activities.WorkflowApplicationAbortedException : The workflow has been aborted.
----> System.Data.EntityException : The underlying provider failed on Open.
----> System.InvalidOperationException : The connection object can not be enlisted in transaction scope.
Do I have to keep the EF connection open during the whole transaction scope? How can I do that? Create an explicit custom activity for that, or is there a standard way?
My current workaround goes like this: I created a new code activity that creates our ObjectContext and explicitely calls dbContext.Connection.Open(). It returns the ObjectContext, which is then saved in a workflow variable. That one is passed to all the DB related activities as an InArgument<>. Inside my DB activities, I use this ObjectContext if it is passed in, otherwise I create a new one.
This does work, but I'm not satisfied with this solution: It needs the new InArgument for every DB related activity. In the workflow designer, I have to insert that special OpenDatabaseConnection activity inside the transaction scope, and then make sure that the correct variable is passed into all DB activities. This seems to be very inelegant and error prone, especially if other team members have to use these DB activities.
What would be a better way to handle this?
The problem is that when you open a second connection in the same transaction scope, an attempt is made to promote the transaction to a distributed transaction (even though there's nothing distributed about it since you connect to the same database). SQL Server CE doesn't support this scenario.
What I would do is create a custom 'container' activity that opens (and closes) the connection and makes it available to child activities. This is still not optimal but at least you no longer need to pass InArgument's around. You get the following activity tree:
TransactionScope
InitializeConnection
Sequence
CustomDataActivity1
CustomDataActivity2
CustomDataActivity3
InitializeConnection is a NativeActivity that uses NativeActivityContext.Properties to expose the connection (or the ObjectContext) to child activities.
Make sure you implement proper error handling to ensure you close the connection at all times.
NOTE: Distributed transactions are supported by the full SQL Server only through a Windows service called MSDTC (Microsoft Distributed Transaction Coordinator). You can find this one in your 'Local Services'. Since SQL Server CE is a database that should be able to operate completely standalone, it makes sense that it has no dependency on MSDTC. Therefore it has no support for distributed transactions.
I'm using EF code-first to manage my DB connection, with an explicit connection string declared in web.config. I would like to schedule some DB cleaning process (like deleting test transactions every day), so I'm spawning a thread from Role.OnStart() with proper concurrency management among the instances.
But I'm getting DB database exceptions, like the DB not matching my model, whereas I'm sure it does (the same code used from "inside" the app works well). So my guess is that web.config is not used from the thread, so EF probably uses the default connection string.
What would be the best way to use my connection string from there ?
Thanks
The OnStart method doesn't run in the same process as your web application meaning it doesn't make use of the web.config. I suggest you store the connection string in the service configuration and read it from here when initializing your context.
An other advantage is that you change the setting without re-deploying the application.
Considering this class
public class XQueries
{
public IQueryable Query1()
{
using (XEntities context = new XEntities())
{
return something;
}
}
public IQueryable Query2()
{
using (XEntities context = new XEntities())
{
return somethingElse;
}
}
}
Is a connection to the database created for every (XEntities context = new XEntities()) {...} ? If so what is the correct way to create a static UnitOfWork class so that only 1 connection to exist?
You can't create a static unit of work, because by definition a unit of work is a short lived object. Because the EF ObjectContext is designed around the unit of work pattern it is a bad idea to have a single ObjectContext instance during the life time of the application. There are several reasons for this.
First of all, the ObjectContext class is not thread-safe. This means that during the unit of work of one user (in a web app for instance), another user can commit his unit of work. When they share the same ObjectContext, it means that in that situation just half of the changes are persisted and changes are not transactional. When you are lucky the ObjectContext fails and throws an exception. When you are unlucky, you corrupt the ObjectContext and safe and load crap from and to your database and find out when your application is running in production (of course, during testing and staging everything always seems to work).
Second, the ObjectContext has a caching mechanism that is designed for it to be short lived. When an entity is retrieved from the database it stays in the ObjectContext’s cache until that instance is garbage collected. When you keep that instance alive for a long period of time, entities get stale. Especially if that particular ObjectContext instance is not the only one writing to that database.
The Entity Framework opens connections only when required, for example to execute a query or to call SaveChanges, and then closes the connection when the operation is complete.
From Martin Fowler’s book Patterns of Enterprise Application Architecture in respect to Unit Of Work.
When you're pulling data in and out of
a database, it's important to keep
track of what you've changed;
otherwise, that data won't be written
back into the database. Similarly you
have to insert new objects you create
and remove any objects you delete.
You can change the database with each
change to your object model, but this
can lead to lots of very small
database calls, which ends up being
very slow. Furthermore it requires you
to have a transaction open for the
whole interaction, which is
impractical if you have a business
transaction that spans multiple
requests. The situation is even worse
if you need to keep track of the
objects you've read so you can avoid
inconsistent reads.
A Unit of Work keeps track of
everything you do during a business
transaction that can affect the
database. When you're done, it figures
out everything that needs to be done
to alter the database as a result of
your work.
Whenever I use Entity Framework for a clients (which I'd admit is rare) the ObjectContext object is the Unit Of Work implementation for the system. That is the ObjectContext will somewhat meet the three statements above. Rather than concentrating too much on the absolutely correct definition using the ObjectContext makes things a little easier for you.
Do some research on DI/IoC and Repository patterns this will give you more flexibility in handling your problem.