I have read about lazy loading from this web site.
Enable or disable LazyLoading
"If we request a list of Students with LazyLoading enabled, the data provider will get all of our students from the DB but each StudentAddress property won’t be loaded until the property will be explicitly accessed."
This statement says that when I set Lazy Loading Enabled = true the related data won't be loaded. However
List<Students> stdList = Datacontext.Students.ToList();
if I set lazy loading enabled = true the above code returns all stundents with their teachers and address. What is the point that I am missing here? Please could someone explain it?
No matter what setting you have, if you use .ToList() it will "enumerate the enumerable". This is very significant, and this phrase should become common knowledge to you.
When .ToList() is used, many things occur. Enumerating the enumerable means that the previous set of how to enumerate the set is now being used to actually iterate through the set and populate the data. What that means is that the previous enumerator (which was stored internally as an Expression Tree) is now going to be sent from Entity Framework to your SQLProvider Factory. That will then convert the object graph from the Expression Tree into SQL and execute the query on the server, thus returning the data and populating your list.
Lazy loading instead of using ToList() would be if you had this IQueryable enumerable, and then iterated that manually loading each element in the set, or only partial elements in the set.
Once you have the list of elements returned, lazy loading will only come in to play if there are navigational properties. If there were related properties, for example if you have an Invoice and you want to get the related Customer information from the customer table. The relation will not explicitly be returned at first, only the invoices. So to get the customer data you could then (while the context was still open, i.e. not disposed) access that via the .Customer reference on your object and it would load. Conversely, to load all the customers during the original enumeration, you could use the .Include() functionality on your queryable, and that would then tell the sql provider factory to use a join when issuing the query.
In your specific example,
List<Students> stdList = Datacontext.Students.ToList();
This will actually not load all of the Teachers and Addresses regardless of if lazy loading is enabled or not. It will only load the students. If you want to lazy load a Teacher, while the Datacontext is still not disposed, you could then use
var firstStudent = stdList.First();
var teacher = firstStudent.Teacher;
//and at this point lazy loading will fetch the teacher
//by issuing **another** query (round trip) to the database
That would only be possible if lazy loading were enabled.
The alternative to this is to eager load, which would include the teachers and addresses. That would look like this
List<Students> stdList = Datacontext.Students
.Include( s => s.Teacher )
.Include( s => s.Address ).ToList();
And then later on if you were to try to access a teacher the context could be disposed and access would still be possible because the data was already loaded.
var firstStudent = stdList.First();
var teacher = firstStudent.Teacher;
//and at this point the teacher was already
//loaded and as a result no additional round trip is required
How was that you noticed that property was loaded, using the debugger? If that is the case, then you already have the answer. With the debugger you are accessing to that property too, so, that also triggers lazy loading.
How this works?
If your entities meets these requirements, then EF will create a proxy class for each of your entities that support change tracking or lazy loading. This way you can load the related entities only when these are accessed. As I explained earlier, the debugger will also trigger lazy loading.
Now, be careful with lazy loading, once you context has been disposed, you will get an exception when you try to get access to one of the related properties. So, I would suggest to use eager loading in that case.
Related
I have tried to implement lazy and eager loading in EF. Everything seems fine, but the .Include() method is available in both cases and returns the same results irrespective of lazy loading false or true.
I have 2 tables categories and product. Category can have multiple products and related using foreign key.
When I load categories using lazy loading by using .Include("Products"), it loads all products related to categories.
Same behavior is shown by eager loading.
var results = objDB.Categories.Include("Products").ToList(); // Enabled lazy load
var results = objDB.Categories.Include("Products").ToList(); // Disabled lazy load
Above both lines have same result. Please clarify this confusion.
.Include should not be available in case of lazy loading = true.
Please give your valuable opinion. Thanks in advance.
When you're explicitly using Include(), you're preforming an Eager Loading. Obviously, disabling/enabling Lazy Loading has no effect.
The difference will be reflected when you'll omit the Include and try to access the Products navigation-property in some of your Category instances. When Lazy-Loading is enabled, EF will load it from the database. When it's disabled, you'll get null.
The difference will be that if you have lazy loading on any entities which are not loaded by the original query will be accessable transparently, and when accessed will trigger an additional query. For example
var results = objDB.Categories.Include("Products").ToList(); // Enabled lazy load
var thing = results.First().Products.First().AnotherEntity;//This will trigger another query and thing will get a value
var results = objDB.Categories.Include("Products").ToList(); // Disabled lazy load
var thing = results.First().Products.First().AnotherEntity;//Thing will be null as its not included in the original result set
FWIW I think most people using lazy loading are causing themselves problems. The only place I would use lazy loading is in a forms/wpf application where people are essentially browsing data in the database and you want to fetch more data as the user clicks things. I don't think there's ever a case for lazy loading in a web application, you are better to be explicit about the data you want from the DB.
Include and ToList are both considered eager methods, so you wouldn't be invoking lazy loading. It's only when you try to access the property of a related entity would the lazy loading feature kick in.
I have an ASP.NET MVC application utilizing Entity Framework for the data layer.
In one of my methods I retrieve the seasonal availability data for a product, and afterwards, the best tax rate for the product.
public ProductList FetchProductSearchList(ProductSearchCriteria criteria)
{
...
var avail = ProductAvailabilityTemplate.Get(criteria.ProductID);
...
var tr = TaxRate.BestMatchFor(criteria.ProductID, criteria.TaxCode);
...
}
In the data layer for ProductAvailabilityTemplate.Get, I had been optimizing the performance of my LINQ code. In particular, I had set ctx.ObjectContext.ContextOptions.LazyLoadingEnabled = false; to prevent EF from loading some entities (via navigation properties) that I don't need in this scenario.
However, once this change was made I noticed that my TaxRates weren't loading fully, because ctx.ObjectContext.ContextOptions.LazyLoadingEnabled was still false in my Tax data layer code. This meant that an entity linked to TaxRate via a navigation property wasn't being loaded.
To overcome this problem I simply set ctx.ObjectContext.ContextOptions.LazyLoadingEnabled = true; in the Tax data layer method, but I am concerned that an unrelated change could cause a problem like this. It seems that you can't safely disable lazy loading for one feature without potentially affecting the operation of whatever is called afterwards. I am tempted to remove all navigation properties, disable lazy loading, and use good old fashioned joins to load exactly what I need for each data layer call, no more no less.
Would welcome any advice.
It's a trade off:
Lazy Loading
gives you the benefit of not needing to specify the depth of graph during loading
will need to return to the database to retrieve missing results
requires some pollution of the POCO's (e.g. virtual properties with proxies)
requires the DbContext to be longer lived, for the duration of all data accesses.
Eager Loading
requires a lot more thought into the depth of loading during each fetch
will typically generate fewer queries with wider joins to fetch the graph at once
does not require any alteration or ceremony around your entities
Allows much shorter lived connections and DbContexts
FWIW, I've generally done prototype work with Lazy Loading enabled, to get software to a demonstrable state, and once the data access patterns stabilize, then switch off Lazy Loading and move to explicitly Included references. A few Unit Tests checking for null references will also do wonders at this point. I am loathe to deliver a production system with Lazy Loading still enabled, as there is an element of non-determinism (e.g. difficult to fully test), and the need to return to the DB for further data will hurt performance.
Either way, I wouldn't switch off all Navigation and do explicit Joins - you are losing the power of navigability that an ORM provides. When you switch out of Lazy Loading, simply explicitly define the entities to be eager loaded with applicable Includes
I was fond of lazy loading when I started using EF, but after a while I realized that it was affecting performance since it effectively disables joins and pulls all subdata in separate queries even if you need to consume it all at once.
So now I'm rather using Includes to eagerly load the sub-entities that I'm interested in. You could also do this somewhat dynamic, for instance by providing a includeDetails parameter:
public IEnumerable<Customer> LoadCustomersStartingWithName(string name, bool includeDetails)
{
using (var db = new MyContext())
{
var customers = db.Customers;
if (includeDetails)
customers = customers.Include(x => x.Orders).Include(x => x.ContactPersons);
customers = customers.Where(x => x.Name.StartsWith(name));
return customers;
}
}
For the code to work in EF6, you would also need to include
using System.Data.Entity;
at the top of the class
I have a very deep relational tree in my model design, that is, the root entity contains a collection of entities that contains more collections of other entities that contains more collections and on an on ... I develop a business layer that other developers have to use to perform operations, including get/save data.
Then, I am thinking about what is the best strategy to cope with this situation. I cannot allow that when retrieving a entity, EF resolves all the dependency tree, since it will end in a lot of useless JOIN (useless because maybe I do not need that data in the next level).
If I disable lazy loading and enforce eager loading for what is needed, it works as expected, but if other developer calls child.Parent.Id instead of child.ParentId trying to do something new (like a new requirement or feature not considered at the beggining), it will get a NullReferenceException if that dependency was not included, which is bad... but it will be a "fast error", and it could be fixed straight away.
If I enable lazy loading, accessing child.Parent.Id instead of child.ParentId will end in a standalone query to the DB each time it is accessed. It won't fail, but it is worse because there is no error, only a decrement in the performance, and all the code should be reviewed.
I am not happy with any of these two solutions.
I am not happy having entities that contains null or empty collections, when in reality, it is not true.
I am not happy with letting EF perform arbitrary queries to the DB at any moment. I would like to get all the information in one shoot if possible.
So, I come up with several possible solutions that involve disabling lazy loading and enforcing eager loading, but not sure which is better:
I can create a EntityBase class, that contains the data in the table without the collections, so they cannot be accessed. And concrete implementations that contains the relationships, the problem is that you do not have much flexibility since C# does not allow multi-inheritance.
I can create interfaces that "mask" the objects hidding the properties that are not available at that method call. For example, if I have a User.Roles property, in order to show a grid will all users, I do not need to resolve the .Roles property, so I could create an interface 'IUserData' that does not contain such property.
But I do not if this additional work is worth, maybe a fast NullReferenceException indicating "This property has not been loaded" would be enough.
Would it be possible to throw a specific exception type if the property is virtual and it has not been overridden/set ?
What method do you use?
Thanks.
In my opinion you are trying to protect the developers from the need to understand what they are doing when they access data and what performance implications it can have - which might result in an unnecessary convoluted API with a lot of helper classes, base classes, interfaces, etc.
If a developer uses user.MiddleName.Trim() and MiddleName is null he gets a NullReferenceException and did something wrong, either didn't check for null or didn't make sure that the MiddleName is set to a value. The same when he accesses user.Roles and gets a NullReferenceException: He didn't check for null or didn't call the appropriate method of your API that loads the Roles of the user.
I would say: Explain how navigation properties work and that they have to be requested explicitly and let the application crash if a developer doesn't follow the rules. He needs to understand the mistake and fix it.
As a help you could make loading related data explicit somehow in the API, for example with methods like:
public User GetUser(int userId);
public User GetUserWithRoles(int userId);
Or:
public User GetUser(int userId, params Expression<Func<User,object>>[] includes);
which could be called with:
var userWithoutRoles = layer.GetUser(1);
var userWithRoles = layer.GetUser(2, u => u.Roles);
You could also leverage explicit loading instead of lazy loading to force the developers to call a method when they want to load a navigation property and not just access the property.
Two additional remarks:
...lazy loading ... will end in a standalone query to the DB each time
it is accessed.
"...and not yet loaded" to complete this. If the navigation property has already been loaded within the same context, accessing the property again won't trigger a query to the database.
I would like to get all the information in one shoot if possible.
Multiple queries do not necessarily result in worse performance than one query with a lot of Includes. In fact complex eager loading can lead to data multiplication on the wire and make entity materialization very time consuming and slower than multiple lazy or explicit loading queries. (Here is an example where a query's performance has been improved by a factor of 50 by changing it from a single query with Includes to more than 1000 queries without Include.) Quintessence is: You cannot reliably predict what's the best loading strategy in a specific situation without measuring the performance (if the performance matters in that situation).
I've been using Entity Framework Profiler to test my data access in an MVC project and have come accross several pages where I'm making far more db queries than I need to because of N+1 problems.
Here is a simple example to show my problem:
var club = this.ActiveClub; // ActiveClub uses code similar to context.Clubs.First()
var members = club.Members.ToList();
return View("MembersWithAddress", members);
The view loops through Members and then follows a navigion property on each member to also show their address. Each of the address requests results in an extra db query.
One way to solve this would be to use Include to make sure the extra tables I need are queried up front. However, I only seem to be able to do this on the ObjectSet of Clubs attached directly to the context. In this case the ActiveClub property is shared by lots of controllers and I don't always want to query the Member and address table up front.
I'd like to be able to use something like:
var members = club.Members.Include("Address").ToList();
But, Members is an EntityCollection and that doesn't have the Include method on it.
Is there a way to force a load on the Members EntityCollection and ask EF to also load their Addresses?
Or, is using EntityCollection navigation properties on an entity in this way, just a really bad idea; and you should know what you're loading when you get it from the context?
If your entities inherits from EntityObject try to use this:
var members = club.Members.CreateSourceQuery()
.Include("Address")
.ToList();
If you use POCOs with lazy loading proxies try to use this:
var members = ((EntityCollection<Club>)club.Members).CreateSourceQuery()
.Include("Address")
.ToList();
Obviously second version is not very nice because POCOs are used to remove dependency on EF but now you need to convert the collection to EF class. Another problem is that the query will be executed twice. Lazy loading will trigger for Members once to access the property and then second query will be executed when you call ToList. This can be solved by turning off lazy loading prior to running the query.
When you say ActiveClub is shared I believe it means something like it is property in base controller used in derived controllers. In such case you can still use different code in different controller to fill the property.
Is it possible to load an entity excluding some properties? One of this entity's properties is expensive to select. I would like to lazy load this property. Is that possible?
Now that you have read everyone's reply, I will give you the correct answer. EF does not support lazy loading of properties. However it does support a much powerful concept then this. It's called table splitting where you can map a table to two entities. Say a product table in the the database can be mapped to product entity and ProductDetail entity. You can then move the expensive fields to the ProductDetail entity and then create a 1..1 association between prodcut and productdetail entity. You can then lazy load the productdetail association only when you need it.
In my performance chapter of my book, I have a recipe called.
13-9. Moving an Expensive Property to Another Entity
Hope that helps!
Julie Lerman has an article on how to split a table
With a scalar property, the only way to selectively not load a certain property is to project in ESQL or L2E:
var q = from p in Context.People
select new
{
Id = p.Id,
Name = p.Name // note no Biography
};
+1 to Dan; doing this lazily is worse than loading it up-front. If you want to control loading, be explicit.
stimms is correct, but be careful while using lazy loading. You may have performance issues and not realize the property is getting loaded at a specific location in your code. This is because it loads the data when you use the property
I prefer to use explicit loading. This way you know when they get loaded and where. Here's a link that gives an example for the LoadProperty http://sankarsan.wordpress.com/2010/05/09/ado-net-entity-framework-data-loading-part-2/
You can also you Eager Loading by using the Include method. Example here:http://wildermuth.com/2008/12/28/Caution_when_Eager_Loading_in_the_Entity_Framework
Given a query over an EntityFramework DbSet, where the targeted entity contains a BigProperty and a SmallProperty,
When you're trying to only access the SmallProperty without loading the BigProperty in memory :
//this query loads the entire entity returned by FirstOrDefault() in memory
//the execution is deferred during Where; the execution happens at FirstOrDefault
db.BigEntities.Where(filter).FirstOrDefault()?.SmallProperty;
//this query only loads the SmallProperty in memory
//the execution is still deferred during Select; the execution happens at FirstOrDefault
//a subset of properties can be selected from the entity, and only those will be loaded in memory
db.BigEntities.Where(filter).Select(e=>e.SmallProperty).FirstOrDefault();
Therefore you could exploit this behaviour to only query the BigProperty where you actually need it, and use select statements to explicitly filter it out everywhere else.
I tested this with the Memory Usage functionality from the Visual Studio debug Diagnostic Tools.