I use in my work Entity Framework Code first (6.2). One of the biggest problem with such approach is slow performance at first query. I use InteractiveViews nuget package to workaround this issue, but unfortunately, it doesn't give expected productivity growth.
When I was experimenting with source code, i noticed that
**( dbContext as IObjectContextAdapter).ObjectContext**
took i very long time ( 8 sec. in my case).
Q: What is the reason for the such big delay? Is it normal or it possible somehow to solve?
new DbContext
Creating a new context such as var ctx = new DbContext() doesn't make the model to be compiled or do any database verification with the Migration History table.
It could take 500ms because it JIT compiles some method within Entity Framework or another third library that you may call in your constructors.
It's hard to know why it takes 500ms because they might have several reasons.
(dbContext as IObjectContextAdapter).ObjectContext
Doing it mean the OnModelCreating method will be invoked, and the model will be compiled for the first time. Some query will be executed on the database as well.
The best way to find out is making a simple test, adding some break point, and enabling SQL Profiler to see it by yourself:
var ctx = new CurrentContext1();
var objectContext = (ctx as IObjectContextAdapter).ObjectContext;
You will suffer from the same performance problem as people usually report when they ask Why Entity Framework First Load is Slow
Related
I am using EF6
I have a generic table which holds data for different types of class objects using the "Table Per Hierarchy" Approach. In addition these class objects use complex types for defining types for their properties.
So using a made up example,
Table = Person
"Mike the Teacher" is a "Teacher" instance of Person with a personType of "Teacher"
The "Teacher" instance has 2 properties, complextypePersonalDetails and complextypeAddress.
complextypePersonalDetails contains
First Name, Surname and Age.
complextypeAddress contains
HouseName, Street, Town, City, County.
I admit that this design may be over the top, and the problem may be of my making, but that aside I wanted to check to see whether I could do anymore with EF6 before I rewrite it.
I am performance profiling the code with JetBrains DotTrace.
On first call, say on
personTeacher = db.person.OfType().First()
I get a massive delay of around 150,000ms
around:
SerializedGeneratedViewOfType (150,000ms)
TryGenerateQueryViewOfType
GenerateTypeSpecificQueryView
GenerateQueryViewForSingleExtent
GenerateQueryViewForExtentAndType
GenerateViewsForExtentAndType
GenerateViewComponents
EnsureExtentIsFullyMapped (90,000ms)
GenerateCaseStatements (60,000ms)
I have created a pregenerated View using the "InteractivePreGeneratedViews" nuget package which creates the SQL. However even with this I still need to incur my first hit. Also this hit seems to happen every time the Webserver/Website/AppPool is restarted.
I am not totally sure of the EF process, but I guess there is some further form of runtime compilation or caching which happens when web app starts. Where could this be happening and is there a proactive method that I could use to pregenerate/precompile/precache this problem away.
In the medium term, we will rewrite this code in Dapper or EF.Core. So for now, any thoughts on what can be done?
Thanks.
I had commented on this before, but retracted it, but just agreeing with "this design may be over the top, and the problem may be of my making", but I thought I'd see if anyone else jumped in.
The initial spin-up cost is due to EF needing to resolve the mapping for your schema. This happens once, the first time a DBSet on the context is accessed. You can mitigate this by executing a query on your application start, I.e.
void Application_Start(object sender, EventArgs e)
{
// Initialization stuff...
using (var context = new MyContext())
{
var result = context.MyTable.Any(); // Spin up will happen here, not when the first user attempts to access a query.
}
}
You actually need to run a query for the the DbContext to resolve the mapping, just new-ing one up won't do it.
For larger, or more complex schemas you can also look to utilize bounded contexts where each context maps a particular set of relationships for a specific area of the application. The less complex/comprehensive a context is, the faster it initializes.
As far as the design goes, TPH is for representing inheritance, which is where you need to establish an "is-a" relation between like entities. Relational models, and ORMs by definition can support this, but they're geared more towards "has-a" relationships. Rather than having a model where you go "is-a person with an address", the relation is best mapped out that a person may "have-an" address. I've worked on a system that was designed by a team of engineers where an entire reporting system with dynamic rules was represented by 6 tables. Honestly, those designs are a nightmare to maintain.
I don't know why OfType() is so slow, but I found a fast and easy workaround by replacing it with a cast; EntityFramework seems to support that just fine and without the performance penalty.
var stopwatch = Stopwatch.StartNew();
using (var db = new MyDbContext())
{
// warn up
Console.WriteLine(db.People.Count());
Console.WriteLine($"{stopwatch.ElapsedMilliseconds} ms");
// fast
Console.WriteLine(db.People.Select(p => p as Teacher).Where(p => p != null).Count());
Console.WriteLine($"{stopwatch.ElapsedMilliseconds} ms");
// slow
Console.WriteLine(db.People.OfType<Teacher>().Count());
Console.WriteLine($"{stopwatch.ElapsedMilliseconds} ms");
}
20
3308 ms
2
3796 ms
2
10026 ms
Our company has been using Telerik Open Access for years. We have multiple projects using it including some in development and some in production that need updated. Because Telerik no longer updates or supports Open Access, we are having a variety of problems. We've got users that have to go to another work station because we can't get Open Access on their computers and we've got projects where we can't add or update tables because the visual designer doesn't work in modern Visual Studio versions. So my question is, how do we convert these and what do we convert these to?
I've heard of Microsoft Entities Framework and we used to just call stored procedures instead of having a separate data project. Obviously our clients aren't going to pay us for hours to switch so we need something that works quick. How do we convert our existing Telerik Open Access project to Microsoft Entities Framework, straight SQL queries, or some other data layer option?
Here's an example of what we have currently.
A separate Visual Studio project that acts as our data layer where all the code was created by Telerik Open Access's visual designer:
We then have a DataAccess.cs class in our main project that creates the instance of the data layer:
Then we call it by using linq statements in the main project:
I don't know OA hands-on, so I can only put my two-cents in.
I'm afraid this isn't going to be be an easy transition. I've yet to see the first seamless transition from one data layer implementation to another (and I've seen a few). The main cause for this is that IQueryable is a leaky abstraction. That is, the data layer exposes IQueryables, but
it doesn't support all features of the interface, and
it adds its own features, and
it's got its own interpretation of how to implement the features that are supported.
This means that if you're going to port your data layer to EF, you may notice that some LINQ queries throw runtime errors because they contain unsupported .Net methods/properties (for instance DateTime.Date), or perform worse -- or better, or return data in subtly different shapes or sorting orders.
Some important OA features that are missing in EF:
Runtime mappings (EF's mapping is static)
Bulk update/delete functions (with EF: only by using third-party libraries)
Second-leve cache
Profiler and Tuning Advisor
Streaming of large objects
Mixing database-side and client-side evaluation of LINQ queries (EF6: only db-evaluation)
On the other hand, the basic architectures of OA and EF don't seem to be too different. They both -
support POCOs
work with simple navigation properties
have a fluent mapping API
support LINQ through IQueryable<T>, where T is an entity class.
most importantly: both have revolve around the Unit of Work and Repository patterns. (For EF: DbContext and DbSet, respectively)
All-in-all it's goinig to be a delicate process of converting, trying and testing. One good thing is that your current DAL is already abstracted to a certain extent. Another is that the syntax doesn't even look too different. Where you have ...
dbContext.Add(newDockReport);
dbContext.SaveChanges();
... using EF this would become ...
dbContext.DockReports.Add(newDockReport);
dbContext.SaveChanges();
With EF-core it wouldn't even have to change one bit.
But that's another important choice to make: EF6 or EF-core? EF6 is stable, mature, feature-rich, but at the end of its life cycle (a phrase you've probably come to hate by now). EF-core, on the other hand, is the future, but is presently struggling to get bug-free in its major functions, not yet as feature-rich as EF6 (and some features will never return, although other new features are great improvements compared to EF6). At the moment, I'd be wary of using EF-core for enterprise applications, but I'm pretty sure that a year from now this is not an issue any more.
Whichever way you go, I'd start the process by writing large amounts of integration tests, if you didn't do so already. The advantage of integration tests is that you can avoid the hassle of mocking either framework first (which isn't trivial).
I have never heard of a tool that can do that.Expect it to take time.
You have to figure how to do it by yourself :
( for the exact safer way to migrate )
1rst you must have a simple page that use your DataLayer it will be your test page. A simple one that you can adapt the LinQ logic .
Add a LinQ to SQL Class, Right click> Add > LinQ to SQL Class.
Drop your table for this page only the usefull one, put the link if needed.
In the test page create a new data context of the linQtoSql.
Use it fixing the type and rename what have to be rename.
Fix error till it compile.
Stock coffee and anything that can boost your brain.
Add table to your context to match the one you had in telerik data access.
Debug for days.
Come back with new question on how to fix a new issue twice a day.
To help with the dbContext.Add() difference between the 2 frameworks you could use this extension in the EF 6.x :
public static void Add<T>(this DbContext db, T entityToCreate) where T : class
{
db.Set<T>().Add(entityToCreate);
db.SaveChanges();
}
then you could do :
db.Add(obj);
See Gert Arnold answer : In Entity Framework, how do I add a generic entity to its corresponding DbSet without a switch statement that enumerates all the possible DbSets?
I'm looking at an existing system that has a significant performance problem when inserting data using LINQ to Entities. The relationships are complex.
I am familiar with common Entity Framework performance factors such as how often SaveChanges is called, etc. There are two problems with this, though. First, it's not the problem here (the performance problem occurs before SaveChanges even happens). Second, it's just guessing. It often works but it's not a process, it's a hope.
I'd like to know the best way to deal with these big problems the real way, where hope doesn't work. If this were ADO using SQL commands, the tool would be SQL Server Performance Profiler to see how many queries were happening, what they were, what their execution plans are, etc., and then adjusting the commands to mitigate those problems. In Entity Framework land, these tools aren't effective because there is not a 1-1 mapping between what you tell it and the commands it generates, and in this particular case it's not even SQL being slow.
Although I'd like the answer here to be a generic approach, here is the relevant code specific to my current problem:
foreach( var thing in objectsToImport ) {
var newEntityFrameworkManagedObject = new EntityFrameworkManagedObject { createdOn = DateTime.UtcNow };
newEntityFrameworkManagedObject.OtherExistingEntity = getExistingEntity( thing.OtherEntityMatchingInfo );
... /*Many more lines associating new object to a bunch of existing items selected before loop.*/
modelRepository.Insert( newEntityFrameworkManagedObject );
}
...
modelRepository.SaveChanges();
The performance hit is on modelRepository.Insert(). We're importing a CSV of several-hundred rows and each call to Insert takes a full minute. Notice that it never gets to SaveChanges. Even the first item to be inserted is slow.
I am using MVC3, ASP.NET 4.5, LINQ to Entities, EF5 and SQL Server 2008 R2 and Azure(for live).
I am creating csdl, edml, msl and ssdl files for the Model in a seperate Model Project.
These files are produced by Entity developer from Devart, where I manage my entity model.
Target: Entity Framework 5
Lazy Loading : Enabled
View Generation : True
Validation on build : True
Metadata Artifact Processing: Embed in Output Assembly
ObjectContext is used
ObjectContext Proxy Creation Enabled : true.
I have implemented TPH inheritance in my entity model, whereby the child class is also made up of upto 10 complex types(CTn).
Animal<-Cat(CT1,CT2,CT3 etc) (for example)
Each complex type maps to column in the generic Animal Table.
My initial LINQ is:
if (db.Animal.OfType<Cat>().Any(a => a.OwnerId == myOwnerId))
When this is run for the first time, it can take around 40 secs to complete. Successive runs take about 200ms.
When I analyse this further using an ORM Profiler, it gives me the LINQ code as:
Cat.MergeAs(0).Any(a => a.OwnerId == ((Nullable<int>)myOwnerId))
I discovered a wonderful SO question at: Related SO Question, but it does not go deep enough. While it recommends an upgrade to EF6, it does not mention the new EF6 issue of having to JIT the EF runtime on first use due to it being external to the .NET runtime now. Perhaps in a later version of EF6 ie 6.1.2 this is solved.
Once the T-SQL is created by EF, it runs, in its own right, very quickly. I have tested this in SSMS.
So my question, as of November 2014, is made up of:
1) How can I resolve initial load delays for my TPH/Complex Type scenario, having tried pregenerated Views. I have seen references to "Compiled Queries".
2) Perhaps I should upgrade to EF6? However if I do, is there now a JIT penalty for the EF runtime itself, and how should I resolve this. I deploy into Azure Websites.
3) Finally I spotted that other more straightforward queries benefited from the pregenerated views, so I am using them. It is just for this TPH/Complex Type scenerario it has no impact. Are there situations where pregen views have no impact?
4) Could 3) be due to the time taken to "autocompile a query" which EF5 can now do. Having pregenerated the View, I guess this is the next bottleneck. Perhaps this "autocompile" feature for complex entities like mine takes a long time, so could one do a proactive manual compile? Quess this is what one calls a "CompiledQuery". Is it relevant to write this extra code, or would EF6x help me here? I have a strong hunch that this query compilation step is the bottleneck, but also realise that writing compiled queries is also not necessarily the most easy and maintainable solution. At present we have a periodic startup job that just warms up all of these complex entities, so the user goes straight into "warm query execution mode".
Any help with the above would be hugely appreciated.
EDIT1
I have just used JetBrains's DotTrace Profile which goes much deeper, and it seems to confirm that my bottleneck is happening with :
System.Data.Query.PlanCompiler.PreProcessor.Process(Dictionary[EdmFunctionEdmProperty[]]&)
On first hit, it spends 99% of the time here, which confirms my view that it was something to do with the Query Plan generation. How I resolve this issue is another matter.
EDIT2
I am going to test out on EF 6.1.2 after some good advice and having followed Julie Lerman's excellent course on Pluralsight.
I had a similar problem and tried multiple approaches to improve 'first hit' performance.
Have you thought about implementing IProcessHostPreloadClient to pre-warm your cache and making an initial call, via your Entity Framework model, to your database?
This solution worked for me and I've created a sample PreWarmCache class below:
namespace MyNamespace
{
public class PreWarmCache : IProcessHostPreloadClient
{
private static readonly NLog.Logger Logger = NLog.LogManager.GetCurrentClassLogger();
private readonly SystemRepository _systemRepository = new SystemRepository();
public void Preload(string[] parameters)
{
// Perform initialization and cache loading logic
Logger.Debug("Application Started: {0}", _systemRepository.InitAppliation() ? "DB response OK" : "DB response failed");
}
}
}
Check out this blog for details on how to hook this up to your AutoStart provider on IIS.
Considering this class
public class XQueries
{
public IQueryable Query1()
{
using (XEntities context = new XEntities())
{
return something;
}
}
public IQueryable Query2()
{
using (XEntities context = new XEntities())
{
return somethingElse;
}
}
}
Is a connection to the database created for every (XEntities context = new XEntities()) {...} ? If so what is the correct way to create a static UnitOfWork class so that only 1 connection to exist?
You can't create a static unit of work, because by definition a unit of work is a short lived object. Because the EF ObjectContext is designed around the unit of work pattern it is a bad idea to have a single ObjectContext instance during the life time of the application. There are several reasons for this.
First of all, the ObjectContext class is not thread-safe. This means that during the unit of work of one user (in a web app for instance), another user can commit his unit of work. When they share the same ObjectContext, it means that in that situation just half of the changes are persisted and changes are not transactional. When you are lucky the ObjectContext fails and throws an exception. When you are unlucky, you corrupt the ObjectContext and safe and load crap from and to your database and find out when your application is running in production (of course, during testing and staging everything always seems to work).
Second, the ObjectContext has a caching mechanism that is designed for it to be short lived. When an entity is retrieved from the database it stays in the ObjectContext’s cache until that instance is garbage collected. When you keep that instance alive for a long period of time, entities get stale. Especially if that particular ObjectContext instance is not the only one writing to that database.
The Entity Framework opens connections only when required, for example to execute a query or to call SaveChanges, and then closes the connection when the operation is complete.
From Martin Fowler’s book Patterns of Enterprise Application Architecture in respect to Unit Of Work.
When you're pulling data in and out of
a database, it's important to keep
track of what you've changed;
otherwise, that data won't be written
back into the database. Similarly you
have to insert new objects you create
and remove any objects you delete.
You can change the database with each
change to your object model, but this
can lead to lots of very small
database calls, which ends up being
very slow. Furthermore it requires you
to have a transaction open for the
whole interaction, which is
impractical if you have a business
transaction that spans multiple
requests. The situation is even worse
if you need to keep track of the
objects you've read so you can avoid
inconsistent reads.
A Unit of Work keeps track of
everything you do during a business
transaction that can affect the
database. When you're done, it figures
out everything that needs to be done
to alter the database as a result of
your work.
Whenever I use Entity Framework for a clients (which I'd admit is rare) the ObjectContext object is the Unit Of Work implementation for the system. That is the ObjectContext will somewhat meet the three statements above. Rather than concentrating too much on the absolutely correct definition using the ObjectContext makes things a little easier for you.
Do some research on DI/IoC and Repository patterns this will give you more flexibility in handling your problem.