I am using MVC3, ASP.NET 4.5, LINQ to Entities, EF5 and SQL Server 2008 R2 and Azure(for live).
I am creating csdl, edml, msl and ssdl files for the Model in a seperate Model Project.
These files are produced by Entity developer from Devart, where I manage my entity model.
Target: Entity Framework 5
Lazy Loading : Enabled
View Generation : True
Validation on build : True
Metadata Artifact Processing: Embed in Output Assembly
ObjectContext is used
ObjectContext Proxy Creation Enabled : true.
I have implemented TPH inheritance in my entity model, whereby the child class is also made up of upto 10 complex types(CTn).
Animal<-Cat(CT1,CT2,CT3 etc) (for example)
Each complex type maps to column in the generic Animal Table.
My initial LINQ is:
if (db.Animal.OfType<Cat>().Any(a => a.OwnerId == myOwnerId))
When this is run for the first time, it can take around 40 secs to complete. Successive runs take about 200ms.
When I analyse this further using an ORM Profiler, it gives me the LINQ code as:
Cat.MergeAs(0).Any(a => a.OwnerId == ((Nullable<int>)myOwnerId))
I discovered a wonderful SO question at: Related SO Question, but it does not go deep enough. While it recommends an upgrade to EF6, it does not mention the new EF6 issue of having to JIT the EF runtime on first use due to it being external to the .NET runtime now. Perhaps in a later version of EF6 ie 6.1.2 this is solved.
Once the T-SQL is created by EF, it runs, in its own right, very quickly. I have tested this in SSMS.
So my question, as of November 2014, is made up of:
1) How can I resolve initial load delays for my TPH/Complex Type scenario, having tried pregenerated Views. I have seen references to "Compiled Queries".
2) Perhaps I should upgrade to EF6? However if I do, is there now a JIT penalty for the EF runtime itself, and how should I resolve this. I deploy into Azure Websites.
3) Finally I spotted that other more straightforward queries benefited from the pregenerated views, so I am using them. It is just for this TPH/Complex Type scenerario it has no impact. Are there situations where pregen views have no impact?
4) Could 3) be due to the time taken to "autocompile a query" which EF5 can now do. Having pregenerated the View, I guess this is the next bottleneck. Perhaps this "autocompile" feature for complex entities like mine takes a long time, so could one do a proactive manual compile? Quess this is what one calls a "CompiledQuery". Is it relevant to write this extra code, or would EF6x help me here? I have a strong hunch that this query compilation step is the bottleneck, but also realise that writing compiled queries is also not necessarily the most easy and maintainable solution. At present we have a periodic startup job that just warms up all of these complex entities, so the user goes straight into "warm query execution mode".
Any help with the above would be hugely appreciated.
EDIT1
I have just used JetBrains's DotTrace Profile which goes much deeper, and it seems to confirm that my bottleneck is happening with :
System.Data.Query.PlanCompiler.PreProcessor.Process(Dictionary[EdmFunctionEdmProperty[]]&)
On first hit, it spends 99% of the time here, which confirms my view that it was something to do with the Query Plan generation. How I resolve this issue is another matter.
EDIT2
I am going to test out on EF 6.1.2 after some good advice and having followed Julie Lerman's excellent course on Pluralsight.
I had a similar problem and tried multiple approaches to improve 'first hit' performance.
Have you thought about implementing IProcessHostPreloadClient to pre-warm your cache and making an initial call, via your Entity Framework model, to your database?
This solution worked for me and I've created a sample PreWarmCache class below:
namespace MyNamespace
{
public class PreWarmCache : IProcessHostPreloadClient
{
private static readonly NLog.Logger Logger = NLog.LogManager.GetCurrentClassLogger();
private readonly SystemRepository _systemRepository = new SystemRepository();
public void Preload(string[] parameters)
{
// Perform initialization and cache loading logic
Logger.Debug("Application Started: {0}", _systemRepository.InitAppliation() ? "DB response OK" : "DB response failed");
}
}
}
Check out this blog for details on how to hook this up to your AutoStart provider on IIS.
Related
I use in my work Entity Framework Code first (6.2). One of the biggest problem with such approach is slow performance at first query. I use InteractiveViews nuget package to workaround this issue, but unfortunately, it doesn't give expected productivity growth.
When I was experimenting with source code, i noticed that
**( dbContext as IObjectContextAdapter).ObjectContext**
took i very long time ( 8 sec. in my case).
Q: What is the reason for the such big delay? Is it normal or it possible somehow to solve?
new DbContext
Creating a new context such as var ctx = new DbContext() doesn't make the model to be compiled or do any database verification with the Migration History table.
It could take 500ms because it JIT compiles some method within Entity Framework or another third library that you may call in your constructors.
It's hard to know why it takes 500ms because they might have several reasons.
(dbContext as IObjectContextAdapter).ObjectContext
Doing it mean the OnModelCreating method will be invoked, and the model will be compiled for the first time. Some query will be executed on the database as well.
The best way to find out is making a simple test, adding some break point, and enabling SQL Profiler to see it by yourself:
var ctx = new CurrentContext1();
var objectContext = (ctx as IObjectContextAdapter).ObjectContext;
You will suffer from the same performance problem as people usually report when they ask Why Entity Framework First Load is Slow
Our company has been using Telerik Open Access for years. We have multiple projects using it including some in development and some in production that need updated. Because Telerik no longer updates or supports Open Access, we are having a variety of problems. We've got users that have to go to another work station because we can't get Open Access on their computers and we've got projects where we can't add or update tables because the visual designer doesn't work in modern Visual Studio versions. So my question is, how do we convert these and what do we convert these to?
I've heard of Microsoft Entities Framework and we used to just call stored procedures instead of having a separate data project. Obviously our clients aren't going to pay us for hours to switch so we need something that works quick. How do we convert our existing Telerik Open Access project to Microsoft Entities Framework, straight SQL queries, or some other data layer option?
Here's an example of what we have currently.
A separate Visual Studio project that acts as our data layer where all the code was created by Telerik Open Access's visual designer:
We then have a DataAccess.cs class in our main project that creates the instance of the data layer:
Then we call it by using linq statements in the main project:
I don't know OA hands-on, so I can only put my two-cents in.
I'm afraid this isn't going to be be an easy transition. I've yet to see the first seamless transition from one data layer implementation to another (and I've seen a few). The main cause for this is that IQueryable is a leaky abstraction. That is, the data layer exposes IQueryables, but
it doesn't support all features of the interface, and
it adds its own features, and
it's got its own interpretation of how to implement the features that are supported.
This means that if you're going to port your data layer to EF, you may notice that some LINQ queries throw runtime errors because they contain unsupported .Net methods/properties (for instance DateTime.Date), or perform worse -- or better, or return data in subtly different shapes or sorting orders.
Some important OA features that are missing in EF:
Runtime mappings (EF's mapping is static)
Bulk update/delete functions (with EF: only by using third-party libraries)
Second-leve cache
Profiler and Tuning Advisor
Streaming of large objects
Mixing database-side and client-side evaluation of LINQ queries (EF6: only db-evaluation)
On the other hand, the basic architectures of OA and EF don't seem to be too different. They both -
support POCOs
work with simple navigation properties
have a fluent mapping API
support LINQ through IQueryable<T>, where T is an entity class.
most importantly: both have revolve around the Unit of Work and Repository patterns. (For EF: DbContext and DbSet, respectively)
All-in-all it's goinig to be a delicate process of converting, trying and testing. One good thing is that your current DAL is already abstracted to a certain extent. Another is that the syntax doesn't even look too different. Where you have ...
dbContext.Add(newDockReport);
dbContext.SaveChanges();
... using EF this would become ...
dbContext.DockReports.Add(newDockReport);
dbContext.SaveChanges();
With EF-core it wouldn't even have to change one bit.
But that's another important choice to make: EF6 or EF-core? EF6 is stable, mature, feature-rich, but at the end of its life cycle (a phrase you've probably come to hate by now). EF-core, on the other hand, is the future, but is presently struggling to get bug-free in its major functions, not yet as feature-rich as EF6 (and some features will never return, although other new features are great improvements compared to EF6). At the moment, I'd be wary of using EF-core for enterprise applications, but I'm pretty sure that a year from now this is not an issue any more.
Whichever way you go, I'd start the process by writing large amounts of integration tests, if you didn't do so already. The advantage of integration tests is that you can avoid the hassle of mocking either framework first (which isn't trivial).
I have never heard of a tool that can do that.Expect it to take time.
You have to figure how to do it by yourself :
( for the exact safer way to migrate )
1rst you must have a simple page that use your DataLayer it will be your test page. A simple one that you can adapt the LinQ logic .
Add a LinQ to SQL Class, Right click> Add > LinQ to SQL Class.
Drop your table for this page only the usefull one, put the link if needed.
In the test page create a new data context of the linQtoSql.
Use it fixing the type and rename what have to be rename.
Fix error till it compile.
Stock coffee and anything that can boost your brain.
Add table to your context to match the one you had in telerik data access.
Debug for days.
Come back with new question on how to fix a new issue twice a day.
To help with the dbContext.Add() difference between the 2 frameworks you could use this extension in the EF 6.x :
public static void Add<T>(this DbContext db, T entityToCreate) where T : class
{
db.Set<T>().Add(entityToCreate);
db.SaveChanges();
}
then you could do :
db.Add(obj);
See Gert Arnold answer : In Entity Framework, how do I add a generic entity to its corresponding DbSet without a switch statement that enumerates all the possible DbSets?
The first call to our API is always extremely slow. For example, below demonstrates the CPU usage and time it takes for the first call to complete:
The first call can take up to 30 seconds and eats almost 100% CPU. Call 2 and 3 take 200ms (as they should). After recycling the application pool, it will do the same thing with the first call.
I've read a bit about IIS "warm-up" and done the following, but nothing has changed:
IIS 8 Application Initialization is installed:
I have the following set in IIS:
Set Start Mode to AlwaysRunning:
Set the Recycling Timeout to 0:
Set the Idle Time-out to 0:
Set Preload Enabled to true on the site:
I am actually setting these in code in RoleEntryPoint.OnStart().
using (var serverManager = new ServerManager())
{
serverManager.ApplicationPoolDefaults.ProcessModel.IdleTimeout = TimeSpan.Zero;
foreach (var application in serverManager.Sites.SelectMany(x => x.Applications))
{
application["preloadEnabled"] = true;
}
foreach (var applicationPool in serverManager.ApplicationPools)
{
applicationPool.AutoStart = true;
applicationPool["startMode"] = "AlwaysRunning";
applicationPool.ProcessModel.IdleTimeout = TimeSpan.Zero;
applicationPool.Recycling.PeriodicRestart.Time = TimeSpan.Zero;
}
serverManager.CommitChanges();
}
I am almost certain that Entity Framework could be the culprit:
We are generating models from around 100 tables in a EDMX model "designer".
We are generating precompiled views generated by EF Power Tools.
The following initialization is running in Application_Start():
using (var context = new MyContext())
{
context.Database.Initialize(false);
}
I don't have these "initialization" problems when debugging.
The following tech is being used:
.NET 4.5.1
ASP.NET Web Api 2
Entity Framework 6.1.1
IIS 8 (Azure Web Role)
Unity 3.5
Can anyone provide me with any other ideas or suggestions?
Not sure if anyone has addressed this yet, but I've learned about some performance issues arising on the initial startup of Entity Framework. Julie Lerman discussed this in her Pluralsight course on Entity Framework, and is also alluded to in the following article excerpt from Microsoft:
One of the biggest drags on performance is the startup time involved with the first use of a context in an application process. You can do a lot to improve that startup time, though. Hopefully you’ve already learned these tricks from my own writing or other resources, such as the MSDN doc on performance considerations at bit.ly/3D6AiC.
A startup step that often hampers performance is the view generation of mapping views, where EF creates the relevant SQL to query against each of the entity sets in the model. These views get leveraged as your app runs so that for certain queries, EF doesn’t have to work out the SQL on the fly. View generation happens whether you created your model with the EF Designer or with Code First. You can pre-generate these views and compile them into the application to save time.
http://msdn.microsoft.com/en-us/magazine/dn532202.aspx
Here it seems she's not just talking about the 'initial load' but the actual first use of a context. I'd to a quick search for Julie Lerman and Entity Framework performance issues. I noticed similar slowness when making the initial calls to my Web API. Every call after the first was significantly faster. I personally haven't found it to be too awful, so I'm ignoring it (for now). However I do find it interesting that it doesn't occur in debug mode. Sorry if you've already explored these options, but I hope this helps a little.
EF 6.0.1, code first, two tables (parent / child relation). I am using the model to query data inside a WebAPI and return the data (approx. 3KB) in JSON format to a website (I also tested the most simple ".FirstOrDefault()" query - does not make a difference here).
The first hit to the WebAPI ist slow (approx 10 seconds). This has nothing to do with the web site starting, the web site was already warmed up at that point. I implemented the same query via ADO and that takes about 1 second for the first hit. Subsequent requests show a much smaller Entity Framework penalty (about 190ms with EF vs. 170ms with ADO).
I tested this with the local sql server and Azure SQL, similar results. I profiled the local database: not much happening there, all 0 Duration requests. The Entity Framework produces additional queries to check for migrations, but those are lean. Looking at the profile output I noticed that all the database action seems to be happening within about one second, while the complete EF initialization process takes 10 seconds. Why?
I already found some hints here, like using Release Mode and turning on "Generate serialization" on, but that did not do much. Before I start implementing the obvious workaround (initializing EF early or just drop EF) I was hoping there are some other caveats I should check.
There are known issues in Entity Framework 6.0.1 regarding startup performance, especially with the debugger attached. Details of this are discussed on the Entity Framework Blog, but since EF6.0.2 is marked stable, your first action should be to update to using that.
In general, there will always be some startup delay with EF6 vs querying directly in ADO, as it needs to load things related to the model, which you don't need to do when using ADO directly. However, it does appear that this is worse than normal on EF6.0.1.
I'm developing a data access layer for a database with over 700 tables. I created the model including all the tables, which generated a huge model. I then changed the model to use DBContext from 4.1 which seemed to improve how it compiled and worked. The designer didnt seem to work at all.
I then created a test app which just added two records to the table, but the processor went 100% in the db.SaveChanges method. Being a black box it was difficult to accertain what went wrong.
So my questions are
Is the entity framework the best approach to a large database
If so, should the model be broken down into logical areas. I did note that you cant have the same sql table in multiple models
I have read that the code only approach is best in these large cases. What is that.
Any guidance would be truly appreciated
Thanks
Large database is always something special. Any technology has some pros and cons when working with a large database.
The problem you have encountered is the most probably related to building the model. When you start the application and use EF related stuff for the first time EF must build the model description and compile it - this is the most time consuming operation you can find in EF. Complexity of this operation grows with number of entities in the model. Once the model is compiled it is reused for the whole lifetime of the application (if you restart the application or unload application domain the model must be compiled again). You can avoid this by precompiling the model. It is done at design time where you use some tool to generate code from the model and you include that code into your project (it must be done again after each change in the model). For EDMX based models you can use EdmGen.exe to generate views and for code first based models you can use EF Power Tools CTP1.
EDMX (the designer) was improved in VS 2010 SP1 to be able to work with large models but I still think the large in this case is around 100 entities / tables. In the same time you rarely need 715 tables in the same model. I believe that these 715 tables indeed model several domains so you can divide them into multiple models.
The same is true when you are using DbContext and code first. If you model a class do you think that it is correct design when the class exposes 715 properties? I don't think so but that is exactly what your derived DbContext looks like - it has a public property for each exposed entity set (in the simplest mapping it means one property per table).
Same entity can be used in multiple models but you should try to avoid it as much as possible because it can introduce some complexities when loading entity in one context type and using it in other context type.
Code only = code first = Entity framework when you define mapping in the code without using EDMX.
take a look this post.
http://blogs.msdn.com/b/adonet/archive/2008/11/24/working-with-large-models-in-entity-framework-part-1.aspx