Im using lazyloading and pre generated views.
I create the context.
I get all my objects about 6000 + navigations are filled in like 3sec. thats ok.
I update all my objects first time ( 4mins....)
I update all my objects second time ( 6sec )
I suspect lazyloading running on background and updating must be doing something that makes him reloop or EF startup is still loading..
I'm on EF 6.1 and the datas are hierarchical.
Database size is about 6000rows on 30 tables.
EF model is DatabaseFirst.
Any Workaround ?
If your concern is about the time difference between the two updates, I suspect that the second one runs faster because fewer objects have been modified.
Why do you need all the objects loaded into your context? What is the lifecycle of your context?
The general recommendation is that you create single use contexts - for a single web request or a single windows form.
You'll further see a lot of example with a using statement, where the life of the context is purposely kept very short. Letting the context live too long can increase memory usage and increase the possibility of concurrency problems. Your database and other layers also do their own caching.
Lastly, Lazy loading is fine, as long as you don't know if you're going to need things. You can consider explicitly loading data if you know you're going to need related records, and want to avoid multiple round trips.
http://msdn.microsoft.com/en-us/data/jj574232.aspx
Related
I have been tasked with fixing a number of bugs on existing application that makes use of JPA (EclipseLink).
In this application it common for there to be multiple entities that indirectly share the same table via SQL views. As an example, I might have a entity called TEmployeeInfo and another entity called VActiveEmployeeInfo. While the first maps to a table directly, the second maps to a view of the same table as the first.
This seems like it would cause issues with caching as any updates using the first entity (table) would not be immediately available in the second (view).
Are my suspicions correct?
If so, what are some possible steps to correct this short of refactoring a large portion of the application?
If the same rows are mapped twice, then the cache can become stale if the objects are updated. You can have both classes use the real table, or perhaps use inheritance.
Otherwise, you can disable the shared cache of this class.
http://wiki.eclipse.org/EclipseLink/FAQ/How_to_disable_the_shared_cache%3F
I'm researching data layer underpinnings for a new web-based reporting system and have spent a lot of time evaluating ORM's over the last few days. That said, I've never dealt with "lazy loading" before and am confused at why its the default setting for LINQ queries in the Entity Framework. It seems like it creates a lot of network traffic and unnecessarily tasks the database with additional queries that could otherwise be resolved with joins.
Can someone describe a scenario in which lazy loading would be beneficial?
Some meta:
The new system will be working against a database with hundreds of tables and many terabytes of data in a production environment with over 3,000 concurrent users on the system 24 hours a day. They will be retrieving large datasets continuously. Is it possible that an ORM just isn't the right solution for our needs, especially since the app will be web-based?
When we talk about lazy loading we are talking about Navigation Properties (how we follow foreign keys). What lazy loading will do for us is to populate the entity from a remote table as we attempt to access that entity. For example if we have a model like this
public class TestEntity
{
public int Id{get;set;}
public AnotherEntity RemoteEntity{get;set;}
}
And call the following
var something = WhateverContext.TestEntities.First().RemoteEntity;
We will get 2 database calls, one for WhateverContext.TestEntities.First() and one for loading the remote entity.
I'm a web guy, (and more specifically an MVC guy) and for web stuff I don't think there is ever a good reason for wanting to do this, One database call is always going to be quicker than two if we require the same set of data.
The situation where I think that lazy loading is actually worth considering is when you don't know when you do your first query if you will need the second entity at all. In my opinion this is much more relevant for windows applications where we have a user who is performing actions in real time (rather than stateless MVC where users are requesting whole pages at once). For example I think lazy loading shines when we have a list of data with a details link, then we don't load the details until the user decides they want to see them.
I don't feel this extends to paging, sorting and filtering, IMO there should be one specifically crafted database query per page of data you are displaying, which returns exactly the data set required to display that page.
In terms of your performance question, I feel that EF (or another ORM) can probably meet your needs here but you want to be careful with how you are retrieving large datasets due to the way EF tracks entities. Check out my EF performance tuning cheat sheet, and read up on DetectChanges and AsNoTracking if you do decide to use EF with large queries.
Most ORMs will give you the option, when you're building up your object selections, to say "don't be lazy, go ahead and join", so if you're worried about it from an efficiency perspective, don't be. You can make it work (usually).
There are 2 particular cases I know of where lazy loading helps:
Chaining commands
What if you want to create a basic select, but then you want to run it through a sort and a filter function that's based on user input. You can simply pass the ORM object in, and attach the sort and filtering functionality to it. Instead of evaluating it each time, it only evaluates when it's actually used.
Avoiding huge, deep, highly-relational queries
What if you just need the IDs of some related fields? If it loads lazily, you don't have to worry about it joining a whole bunch of data and tables that you don't need, potentially slowing down the query and overusing bandwidth. Of course, if you DID want everything else, then you'll need to be explicit, or you may run into a problem where it lazily runs a query for each detail record. Like I mentioned at the outset, that's easily overcome in any ORM worth using.
A simple case is a result set of N records which you do not want to bring to the client at once. The benefit is that you are able to lazily load only what is needed for the clients demands, such as sorting, filtering, etc... An example would be a paging view where one could page through records and sort them accordingly, thus the client only needs N amount at a given time.
When you perform the LINQ query it translates that to SQL commands on the server side to provide only what is needed in the given context. It boils down to offloading work to the database and minimizing what you need to send back to the client.
Some will argue that ORM based lazy loading is wrong however that starts to move to semantics fairly quick and should be more about approach to design versus what is right and wrong.
We develop the back office application with quite large Db.
It's not reasonable to load everything from DB to memory so when model's proprties are requested we read from DB (via EF)
But many of our UIs are just simple lists of entities with some (!) properties presented to the user.
For example, we just want to show Id, Title and Name.
And later when user select the item and want to perform some actions the whole object is needed. Now we have list of items stored in memory.
Some properties contain large textst, images or other data.
EF works with entities and reading a bunch of large objects degrades performance notably.
As far as I understand, the problem can be solved by creating lightweight entities and using them in appropriate context.
First.
I'm afraid that each view will make us create new LightweightEntity and we eventually will end with bloated object context.
Second. As the Model wraps EF we need to provide methods for various entities.
Third. ViewModels communicate and pass entities to each other.
So I'm stuck with all these considerations and need good architectural design advice.
Any ideas?
For images an large textst you may consider table splitting, which is commonly used to split a table in a lightweight entity and a "heavy" entity.
But I think what you call lightweight "entities" are data transfer objects (DTO's). These are not supplied by the context (so it won't get bloated) but by projection from entities, which is done in a repository or service.
For projection you can use AutoMapper, especially its newer feature that I describe here. This allows you to reduce the number of methods you need to provide "for various entities" (DTO's), because the type to project to can be given in a generic type parameter.
There is no Detach(object entity) on the DbContext.
Do I have the ability to detach objects on EF code first?
This is an option:
dbContext.Entry(entity).State = EntityState.Detached;
If you want to detach existing object follow #Slauma's advice. If you want to load objects without tracking changes use:
var data = context.MyEntities.AsNoTracking().Where(...).ToList();
As mentioned in comment this will not completely detach entities. They are still attached and lazy loading works but entities are not tracked. This should be used for example if you want to load entity only to read data and you don't plan to modify them.
Both previous answers provide good instructions, however, both might leave you with the entities still loaded into EF's context and/or its Change Tracker.
This is not a problem when you are changing small data sets, but it will become an issue when changing large ones. EF would have increased memory and resource usage, which in turn would reduce the procedure performance as it uses more data/entities.
Both other approaches are valid but, In this case, Microsoft recommends cleaning the Change tracker instead of detaching the entities individually
Clearing the Change tracker on the data changing loop (which changes a chunk of data for instance) can save you from this trouble.
context.ChangeTracker.Clear();
This would unload/detach all entities and its related changeTracker references from the context, so use with care after your context.SaveChanges().
I am new to CSLA, i just want to know what factors that could affect CSLA performance when loading records in a grid
Thanks
There is nothing inheritantly bad with the performance with CSLA and you should not see any additional issues by using CSLA.
A common trap that developers using CSLA can fall into is when it comes to parent/child relationships. The architecture may lead to more seperate database calls then would be necessary.
This is totally avoidable and can easily be avoided as long as you are vigilant of the issue.
I think, choose the right stereotypes. for example if your grid is for displaying records only, you can use readOnlyListBase, with this it will not include states property such as isDirty, isNew and etc..
The issue which may you may face is constant Daatabase calls bacause of Parent Child Relationsship.
But, It can be avoided by creating a method in your repository which will act as Cache by fetchng the data accordingly at its first call to DB and store the fecthed data and use the those data for further child classes and prevent from further DB calls.