I am making a data extraction tool (.NET Core console app) that will process tens of thousands of files and write extracted data to a database. There are only insertions to the DB, no reads or updates. I intend to run multiple instances, all writing to the same table.
I am using Autofac for DI, and injecting the EF DB Context into a class that loops through the files and writes to the DB.
I want to keep the DB Context lifetime short, but am not sure how to do that with DI. Is there some kind of "refresh" operation to effectively renew the DBContext? Or should I inject a DBContextFactory and get a new DBContext for each file processed? (And if so, what does the factory look like? Does it explicitly construct a new DBContext?)
If you're holding on to the injected instance of DbContext and using it in a loop, then clearly the lifetime of DbContext cannot be short.
As you suggest, use a Context Builder in the DI. Here's an example that creates a context of type T using SQLServer:
public class DbContextBuilder<T> where T : DbContext
{
public readonly T Context;
public DbContextBuilder(string connectionStringName)
{
IConfigurationBuilder cfgBuilder = new ConfigurationBuilder();
cfgBuilder.AddJsonFile("appsettings.json");
IConfiguration cfg = cfgBuilder.Build();
DbContextOptionsBuilder<T> optsBuilders = new DbContextOptionsBuilder<T>();
optsBuilders.UseSqlServer(cfg.GetConnectionString(connectionStringName));
Context = (T)Activator.CreateInstance(typeof(T), optsBuilders.Options);
}
}
and in each loop:
foreach (var file in files)
{
using (DbContext ctx = new DbContextBuilder<DbContext>("{name of conn string in appsettings}").Context)
{
// Do you writes for the current file
}
}
Related
I am working on a .net core project where the requirement is to maintain an SQLite DB and an SQL Server DB simultaneously. I created two DbContext files SqlServerContext and SqliteContext and separate migration folders for them. These files are derived from a WorkerContext file that's derived from DbContext. The migration is working properly, as tables are created in both databases. But I could not make simultaneous data operation work.
This is the IKeyboardMouseActivityRepository. There are separate parts for using SqliteContext and SqlServerContext. I have to comment out one part when using the other. So I can do data entry in one DB at a time now.
public interface IKeyboardMouseActivityRepository :
IRepository<KeyboardMouseActivity, Guid, SqlServerContext>
// IRepository<KeyboardMouseActivity, Guid, SqliteContext>
{
}
public class KeyboardMouseActivityRepository :
IKeyboardMouseActivityRepository,
Repository<KeyboardMouseActivity, Guid, SqlServerContext>
// Repository<KeyboardMouseActivity, Guid, SqliteContext>
{
public KeyboardMouseActivityRepository(SqlServerContext dbContext)
: base(dbContext)
{
}
// public KeyboardMouseActivityRepository(SqliteContext dbContext)
// : base(dbContext)
// {
// }
}
This is the main Repository class.
public abstract class Repository<TEntity, TKey, TContext>
: IRepository<TEntity, TKey, TContext>
where TEntity : class, IEntity<TKey>
where TContext : DbContext
{
protected TContext _dbContext;
protected DbSet<TEntity> _dbSet;
public Repository(TContext context)
{
_dbContext = context;
_dbSet = _dbContext.Set<TEntity>();
}
// other methods such as Add, Remove etc.
}
My understanding is that since the context parameter is specified in KeyboardMouseActivityRepository, it only works for that specified context. How can I modify it so it works for both DbContext files and I can do data operation in both DB at the same time?
The repository you have defined is typed per-DbContext. If you want to have a repository that can update two known DbContext implementations then you can back off the Generic approach for the DbContexts and implement the repository to accept one of each in the constructor:
public abstract class Repository<TEntity, TKey>
: IRepository<TEntity, TKey>
where TEntity : class, IEntity<TKey>
{
protected SqlAppDbContext _sqlContext;
protected SqlLiteAppDbContext _sqlLiteContext;
protected DbSet<TEntity> _sqlDbSet;
protected DbSet<TEntity> _sqlLiteDbSet;
public Repository(SqlAppDbContext sqlContext, SqlLiteAppDbContext sqlLiteContext)
{
_sqlContext = sqlContext ?? throw new ArgumentNullException("sqlContext");
_sqlLiteContext = sqlLiteContext ?? throw new ArgumentNullException("sqlLiteContext");
_sqlDbSet = _sqlContext.Set<TEntity>();
_sqlLiteDbSet = _sqlLiteContext.Set<TEntity>();
}
// other methods such as Add, Remove etc.
}
Note that you will want to investigate and implement something like TransactionScope to help ensure that operations done via the repository are mutually committed or rolled back. For instance if you have code that attempts to update data in both DbSets and SaveChanges, if one succeeds and the other fails for any reason, usually the expectation would be they both roll back. Reads I expect would prioritize one DbSet over the other, but expect if you were to want to support something like a fail-over or situational load from one server or the other you will run into issues if it is at all possible that entities fetched from one DbContext are ever married up with entities fetched from the other. (entities loaded by _sqlContext cannot be associated with entities loaded by _sqlLiteContext) When updating entities and associating them via navigation properties you will be loading everything twice or playing a very dangerously error prone game of detaching and reattaching entities betewen DbContexts.
I would advise against using a Generic Repository pattern /w EF. This will paint you into various corners that will limit many of the capabilities that EF can provide for optimizing queries, working with projections, and performing operations like pagination, filtering, sorting, etc. efficiently without a lot of extra code or introducing pretty complex code into the repository.
Overall I wish you luck with the project, however a requirement and design like this will be a nest of hungry dragons for your time and sanity. :)
I have a project created from the ASP.NET Core Web Application template in VS. When run, the project creates a database to support the Identity package.
The Identity package is a Razor Class Library. I have scaffolded it and the models can be seen. The models are sub-classed from Microsoft.AspNetCore.Mvc.RazorPages.PageModel.
I am tracing the code to try and get a better understanding of how it all works. I am trying to find the path from the models to the physical database.
In the file appsettings.json, I see the connection string DefaultConnection pointing to the physical database.
In startup.cs, I see a reference to the connection string DefaultConnection:
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(
Configuration.GetConnectionString("DefaultConnection")));
After this, I lost the trail. I can't find the link from a model in code to a table in the database. What is the code needed to perform a query like select * from AspNetUsers?
As #Daniel Schmid suggested , you should firstly learn the Dependency injection in ASP.NET Core.
ASP.NET Core has an excellent Dependency Injection feature through which this framework provides you with an object of any class that you want. So you don’t have to manually create the class object in your code.
EF Core supports using DbContext with a dependency injection container. Your DbContext type can be added to the service container by using the AddDbContext<TContext> method.
Then you can use the instance like :
public class MyController
{
private readonly ApplicationDbContext _context;
public MyController(ApplicationDbContext context)
{
_context = context;
}
...
}
or using ServiceProvider directly, less common :
using (var context = serviceProvider.GetService<ApplicationDbContext>())
{
// do stuff
}
var options = serviceProvider.GetService<DbContextOptions<ApplicationDbContext>>();
And get users by directly querying the database :
var users = _context.Users.ToList();
Please also read this article .
Using autofac as my IoC framework.
I'd like to be able to set up my DbContext instance in my application's startup.
In my ASP.NET MVC 3 project, I register DbContext instance in Global.asax (PerLifetimeScope). But when I fire up my site on multiple browsers (or multiple tabs) at once, sometimes I get Object reference not set to an instance of an object. or New transaction is not allowed because there are other threads running in the session when I try to save changes back to database. Also I get
ExecuteReader requires an open and available Connection. The connection's current state: Broken. sometimes when I want to read data from database.
the errors seem to pop up randomly and I suspect it has something to do with my context's lifetime scope. here's my DbContext's overriden SaveChange method.
public class MyContext : DbContext
{
public override int SaveChanges()
{
var result = base.SaveChanges(); // Exception here
}
}
Here's how I register my context:
builder.Register(c => new MyContext("SomeConnectionString"))
.InstancePerLifetimeScope();
If I just have one open tab of my site in the browser everything works ok.
Also, It's worth mentioning I have CRUD operations with db every 5-10 seconds in my website by calling a controller method using Ajax.
StackTrace for New transaction is not allowed because there are other threads running in the session:
at System.Data.EntityClient.EntityConnection.BeginDbTransaction(IsolationLevel isolationLevel)
at System.Data.EntityClient.EntityConnection.BeginTransaction()
at System.Data.Objects.ObjectContext.SaveChanges(SaveOptions options)
at System.Data.Entity.Internal.InternalContext.SaveChanges()
at System.Data.Entity.Internal.LazyInternalContext.SaveChanges()
at System.Data.Entity.DbContext.SaveChanges()
at MyProject.Data.MyContext.SaveChanges() in D:\Test.cs
StackTrace for Object reference not set to an instance of an object.:
at System.Data.Objects.ObjectStateManager.DetectConflicts(IList`1 entries)
at System.Data.Objects.ObjectStateManager.DetectChanges()
at System.Data.Entity.Internal.InternalContext.DetectChanges(Boolean force)
at System.Data.Entity.Internal.InternalContext.GetStateEntries(Func`2 predicate)
at System.Data.Entity.Internal.InternalContext.GetStateEntries()
at System.Data.Entity.Infrastructure.DbChangeTracker.Entries()
at System.Data.Entity.DbContext.GetValidationErrors()
at System.Data.Entity.Internal.InternalContext.SaveChanges()
at System.Data.Entity.Internal.LazyInternalContext.SaveChanges()
at System.Data.Entity.DbContext.SaveChanges()
at MyProject.Data.MyContext.SaveChanges() in D:\Test.cs at
Registration of MyContext looks ok. Is it possible that some other service that takes a MyContext is registered as a singleton and being shared across threads?
I had the same issue, sporadic errors related to the DbContext while using Autofac to resolve the DbContext.
{System.Data.EntityCommandExecutionException: An error occurred while executing the command definition. See the inner exception for details.
etc.
{System.NullReferenceException: Object reference not set to an instance of an object.
at System.Data.Objects.ObjectStateManager.DetectConflicts(IList`1 entries)
etc.
I found a class resembling the following in my code. The dependency resolution was occurring within a static method inside of the singleton. The object being resolved had a dependency on the DbContext. I haven't had any additional issues after I found a way to restructure this class so that it was no longer a singleton.
Perhaps you have a similar situation? Another thing to try might be to make your DbContext InstancePerHttpRequest. That could help identify whether this is the issue.
public class Singleton
{
private static Singleton _instance = new Singleton();
private Singleton()
{
}
public static void DoSomething<TSource>(TSource source) where TSource : ISource
{
var items = DependencyResolver.Current.Resolve<IEnumerable<IDbContextConsumer<TSource>>>();
foreach (var item in items)
{
item.Execute(source);
}
}
}
I have an ASP.NET mvc3 application which is using ninject for DI. The app is making use of Entity Framework 4 and a database initializer that also seeds custom data into the db.
The database initializer runs in HttpApplication.Application_Start.
However I get an error because it seems that the child request is called before the database initializer finished creating and seeding the db:
The ObjectContext instance has been disposed and can no longer be used for operations that require a connection.
My _layout.cshtml has some child request:
#Html.Action("Present", "Time")
That it:
[ChildActionOnly]
public ActionResult Present()
{
//HttpContext.Trace.Write("Inside [ChildActionOnly]\npublic ActionResult Present() (child requests that is called by _layout.cshtml)");
System.Diagnostics.Debugger.Launch();
Employee employee = this.employeeService.CurrentEmployee;
However once the database is successfully created and seeded with my database initializer called "EFZeiterfassungDataContextInitializer" (it inherits from DropCreateDatabaseIfModelChanges) the app is working.
I'm only having this problem with the first (child) request!
In my ninject module I configured my datacontext (called EFZeiterfassungContext) like this:
Bind<EFZeiterfassungContext>().ToSelf().InRequestScope();
Some more info:
Here is some code of my HttpApplication:
public void SetupDependencyInjection()
{
// Create Ninject DI Kernel
IKernel kernel = new StandardKernel(new ZeiterfassungNinjectModule());
//Tell asp.net mvc to use our ninject di container
DependencyResolver.SetResolver(new NinjectDependencyResolver(kernel));
}
protected void Application_Start()
{
SetupDependencyInjection();
AreaRegistration.RegisterAllAreas();
RegisterGlobalFilters(GlobalFilters.Filters);
RegisterRoutes(RouteTable.Routes);
InitializeDatabase();
}
private void InitializeDatabase()
{
IActiveDirectoryService adService = DependencyResolver.Current.GetService<IActiveDirectoryService>();
IEmployeeService employeeService = DependencyResolver.Current.GetService<IEmployeeService>();
EFZeiterfassungDataContextInitializer d = new EFZeiterfassungDataContextInitializer(
Server.MapPath("~/Content/UserImages/"),
adService,
employeeService);
Database.SetInitializer<EFZeiterfassungContext>(d);
}
EDIT + workaround:
Obviously Database.SetInitializer(d); returns immediately. the initializer isn't run immediately - not even async. EF first checks the DB for changes if you have some interaction with the datacontext i.e. by querying some value from the not yet existing db. I could workaround it by calling var currentEm = employeeService.GetById(1); after Database.SetInitializer(d);. so the DB really gets initialized on application_start and not later.
I would like to see some overloaded method of SetInitializer so you can immediately trigger that process.
I'm still interested in a better solution for this until SetInitializer has such an overloaded method.
Code First is a terrific feature. But I could not figure out how to persist an object without creating a specialized DbContext first. Is it possible at all? I'd like to see something along these lines:
var dbConn = new SqlCeConnection("Data Source=Test.sdf");
using (var db = new DbContext(dbConn, true))
{
var cmp = new Company { CompanyName = "Microsoft" };
db.Set(typeof(Company)).Add(cmp);
var recordsAffected = db.SaveChanges();
Console.WriteLine(
"Saved {0} entities to the database, press any key to exit.",
recordsAffected);
Console.ReadKey();
}
Is it possible to dynamically register a class for model creation? There must be a way!
No, as I know there is not any possibility to create DbContext without any information about mapping. Specialized context with predefined DbSets is necessary to define mappings and database initialization. You can probably use base DbContext only if you provide informations about mapping through its constructor by passing DbCompiledModel created manually before using DbContext but I haven't tried this yet.
The problem is that DbSets and overriden OnModelCreating are used to infer needed mapping. If you don't have DbSets or OnModelCreating defined DbContext can't infer and cache mapping. Mapping metadata are created and compiled only once for each context type (until application restarts). This operation is considered as very slow. If you don't have specific context type EF can't probably infer mapping and even if it can it will probably need to create and compile metadata for each instance of the context (like for anonymous types).
If you use DbCompiledModel created manually it will be your responsibility to reuse it.