EF Filtering a Child Table with Lazy Load - entity-framework

I'm using entity framework with POCOs and the repository pattern and am wondering if there is any way to filter a child list lazy load. Example:
class Person
{
public virtual Organisation organisation {set; get;}
}
class Organisation
{
public virtual ICollection<Product> products {set; get;}
}
class Product
{
public bool active {set; get;}
}
Currently I only have a person repository because I'm always starting from that point, so ideally I would like to do the following:
Person person = personRepo.GetById(Id);
var products = person.organisation.products;
And have it only load products where active = true from the database.
Is this possible and if so how?
EDIT My best guess would be either a filter can be added to the configuration of the entity. Or there might be a way to intercept/override the lazy load call and modify it. Obviously if I created an Organisation Repository I could manually load it as I please but I am trying to avoid that.

There's not a direct way to do this via lazy loading, but if you were willing to explicitly load the collection, you could follow whats in this blog, see the Applying filters when explicitly loading related entities section.
context.Entry(person)
.Collection(p => p.organisation.products)
.Query()
.Where(u => u.IsActive)
.Load();

You can do what Mark Oreta and luksan suggest while keeping all the query logic within the repository.
All you have to do is pass a Lazy<ICollection<Product>> into the organization constructor, and use the logic they provided. It will not evaluate until you access the value property of the lazy instance.
UPDATE
/*
First, here are your changes to the Organisation class:
Add a constructor dependency on the delegate to load the products to your
organization class. You will create this object in the repository method
and assign it to the Person.Organization property
*/
public class Organisation
{
private readonly Lazy<ICollection<Product>> lazyProducts;
public Organisation(Func<ICollection<Product>> loadProducts){
this.lazyProducts = new Lazy<ICollection<Product>>(loadProducts);
}
// The underlying lazy field will not invoke the load delegate until this property is accessed
public virtual ICollection<Product> Products { get { return this.lazyProducts.Value; } }
}
Now, in your repository method, when you construct the Person object you will assign the Organisation property with an Organisation object containing the lazy loading field.
So, without seeing your whole model, it will looks something like
public Person GetById(int id){
var person = context.People.Single(p => p.Id == id);
/* Now, I'm not sure about the cardinality of the person-organization or organisation
product relationships, but let's assume you have some way to access the PK of the
organization record from the Person and that the Product has a reference to
its Organisation. I may be misinterpreting your model, but hopefully you
will get the idea
*/
var organisationId = /* insert the aforementioned magic here */
Func<ICollection<Product>> loadProducts = () => context.Products.Where(product => product.IsActive && product.OrganisationId == organisationId).ToList();
person.Organisation = new Organisation( loadProducts );
return person;
}
By using this approach, the query for the products will not be loaded until you access the Products property on the Organisationinstance, and you can keep all your logic in the repository. There's a good chance that I made incorrect assumptions about your model (as the sample code is quite incomplete), but I think there is enough here for you to see how to use the pattern. Let me know if any of this is unclear.

This might be related:
Using CreateSourceQuery in CTP4 Code First
If you were to redefine your properties as ICollection<T> rather than IList<T> and enable change-tracking proxies, then you might be able to cast them to EntityCollection<T> and then call CreateSourceQuery() which would allow you to execute LINQ to Entities queries against them.
Example:
var productsCollection = (EntityCollection<Product>)person.organisation.products;
var productsQuery = productsCollection.CreateSourceQuery();
var activeProducts = products.Where(p => p.Active);

Is your repository using something like:
IQueryable<T> Find(System.Linq.Expressions.Expression<Func<T, bool>> expression)
If so you can do something like this:
var person = personRepo.Find(p => p.organisation.products.Any(e => e.active)).FirstOrDefault();

You could possibly use Query() method to achieve this. Something like:
context.Entry(person)
.Collection(p => p.organisation.products)
.Query()
.Where(pro=> pro.Active==true)
.Load();
Have a look at this page click here

Related

EF Core load references of unknown entity

DISCLAIMER: Since we are all familiar with it, i will be using contoso university design to explain my question. Also, i am using EF core and .net core 2.0 on a mvc code first design.
I am developing a very generic RESTful API that works on any model. It has one method for each of create, read, update and delete operation in only one controller, the route of this is
[Route("/api/{resource}")]
Resource is the entity that the client wants to work with, for example if someone wants to get all Courses using the api he has to do a GET request on http://www.example.com/api/course/ or http://www.example.com/api/course/2 to get one by id and the following code will do the job.
[HttpGet("{id:int:min(1)?}")]
public IActionResult Read([FromRoute] string resource, [FromRoute] int? id)
{
//find resourse in models
IEntityType entityType = _context.Model
.GetEntityTypes()
.FirstOrDefault(x => x.Name.EndsWith($".{resource}", StringComparison.OrdinalIgnoreCase));
if (entityType == null) return NotFound(resource);
Type type = entityType.ClrType;
if (id == null)//select all from table
{
var entityRows = context.GetType().GetMethod("Set").MakeGenericMethod(type).Invoke(context, null);
if (entityRows == null)
return NoContent();
//TODO: load references (1)
return Ok(entityRows);
}
else //select by id
{
var entityRow = _context.Find(type, id);
if (entityRow == null)
return NoContent();
//TODO: load references (2)
return Ok(entityRows);
}
}
This small piece of code will do the magic with one small exception, intermediate collections will not be loaded. Given our example, the fetched course or courses will have no info for CourseInstructor (the intermediate collection in between Course and Person). I am trying to find a way to Eager load the navigation properties only if it is a collection; or by any other condition that will ensure that only many-to-many relationships are loaded.
For //TODO: load reference (2) i could use
_context.Entry(entityRow).Collection("CourseInsructor").Load();
On runtime if i could find all the navigation properties (filtered by spoken condition) and foreach of them i did Load(), i should get the desired result. My problem is when i get all (when id is null) the entityRows is type 'InternalDbSet' which is an unknown model.
So for the two TODOs i need some help on doing the following steps
1: find navigation properties of many-to-many relationships only
2: load them
Any suggestions?
In general, this seems like a very bad idea to me. While the CRUD stuff is going to be identical for most resources, there will be variances (as you've now run into). There's also something to be said for having a self-documenting API: with individual controllers, you know which resources can be accessed by nature of having a controller associated with that resource. With they way you're doing it, it's a complete black box. This also will of course effect any sort of actual generated API documentation. For example, if you were to include Swagger in your project, it would not be able to determine what you're doing here. Finally, you're now having to use reflection for everything, which will effect your performance.
What I would suggest instead is creating a base abstract controller and then creating a controller for each unique resource that inherits from that, for example:
public abstract class BaseController<TEntity> : Controller
where TEntity : class, new()
{
protected readonly MyContext _context;
public BaseController(MyContext context)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
}
...
[HttpGet("create")]
public IActionResult Create()
{
var model = new TEntity();
return View(model);
}
[HttpPost("create")]
public async Task<IActionResult> Create(TEntity model)
{
if (ModelState.IsValid)
{
_context.Add(model);
await _context.SaveChangesAsync();
return RedirectToAction("Index");
}
return View(model);
}
...
}
I just wanted to give a quick example, but you'd build out all the rest of the CRUD methods in the same fashion, generically using TEntity. Then, for each actual resource, you simply do:
public class WidgetController : BaseController<Widget>
{
public WidgetController(MyContext context)
: base(context)
{
}
}
No duplication of code, but you've now got an actual real controller backing the resource, aiding both the innate and possibly explicit documentation of your API. And, no reflection anywhere.
Then, to solve problems like what you have here, you can add hooks to your base controller: essentially just virtual methods that are utilized in your base controller's CRUD actions and do nothing or just default things. However, you can then override these in your derived controllers to stub in additional functionality. For example, you can add something like:
public virtual IQueryable<TEntity> GetQueryable()
=> _context.Set<TEntity>();
Then, in your derived controller, you can do something like:
public class CourseController : BaseController<Course>
{
...
public override IQueryable<Course> GetQueryable()
=> base.GetQueryable().Include(x => x.CourseInstructors).ThenInclude(x => x.Instructor);
So, for example, you'd make your BaseController.Index action, perhaps, utilize GetQueryable() to get the list of entities to display. Simply by overriding this on the derived class, you can alter what happens based on the context of a particular type of resource.

Materializing an ICollection structure containing subclasses

I'm reviewing some code that was written in the EF 4 days because it stands out during performance benchmarking.
The purpose of the code is to materialize an ICollection<MyBaseClass> using Entity Framework (we're now on EF 6.1).
The code exists because references present in specific subclasses aren't materialized when retrieving
public Parent
{
public virtual ICollection<MyBaseClass>() Base { get; set; }
}
from the database, when the actual types stored are subclasses of MyBaseClass.
Example subclass:
public SubA : MyBaseClass
{
public virtual ICollection<Options> Ref1 { get; set; }
}
Currently, the code does something like this:
var parent = ctx.Parents.Include(p => p.Base).Where(...).Single();
LoadSubclasses(parent.Base);
...
private void LoadSubclasses(IEnumerable<MyBaseClass> myBase)
{
foreach (var my in myBase)
{
if (my is SubA)
{
this.Entry(my).Reference("Ref1").Load();
this.Entry((SubA)my).Ref1).Collection("Options").Load();
}
else... // Similar for other subclasses
}
}
Note that ICollection<MyBaseClass>() Base contains a mix of several concrete subclasses. There are generally a few hundred objects in the ICollection.
Is there a more efficient way to materialize Base?
It cannot be said in advance if the performance will be better (sometimes executing a single complex query, especially with sub collection includes may have actually negative impact), but you can minimize the number of database queries to K, where K is the number of subclass types that need additional includes.
You need to base the LoadSubclasses method on IQueryable<TBase> representing all base entities, and execute one query per each subclass type using OfType filter:
private void LoadSubclasses(IQueryable<MyBaseClass> baseQuery)
{
// SubA
baseQuery.OfType<SubA>()
.Include(x => x.Ref1.Options)
.Load();
// Similar for other subclasses
}
The usage with your sample would be:
var parent = ctx.Parents.Include(p => p.Base).Where(...).Single();
LoadSubclasses(ctx.Entry(parent).Collection(p => p.Base).Query());
or more generally:
var parentQuery = ctx.Parents.Where(...);
var parents = parentQuery.Include(p => p.Base).ToList();
LoadSubclasses(parentQuery.SelectMany(p => p.Base));
For those on EF Core 2.1 or later, this feature is now supported out-of-the-box.
Request from 2010:
When in an data model for entity framework has a navigation property
it is not posseble to eager load that navigation property besides when
using OfType<> or when eager loading the derived type itself by a
navigation property.
Response from 2018:
The feature is part of EF Core 2.1, which is currently in preview.
Please create issues in our issue tracker if you find any problems.

How to get navigation property of foreign key when calling SaveChanges()?

I'm using EF5 code first.
We have a method
LogHistoryTracking(DbEntityEntry entity)
to log changes when SaveChanges is called.
At SaveChanges, we get the changed entities and pass into LogHistoryTracking
var changedEntities = ChangeTracker.Entries().ToList();
But when I access
changedEntity.OriginalValues.PropertyNames
there is no properties for foreign keys object (only foreign key Id - but how can we get the data when there is only id here?).
I also tried to google for a solution, but this issue might be not so popular.
There is this article, but it does not work.
Appreciate any help. Thanks.
If you want to have your entity properties to be accessible you must 'Include' them prior to accessing them. Like in the following example which gets the orders of the first cutomer :
var orders = context.Customers
.Include("Orders")
.First().Orders;
In this example if you do not call .Include("Orders") you will not have Customer.Orders. The same goes if you have foreign key and forget to include the navigation property of the foreign key. This is because the key (the ID) is part of the object and the navigation property is not.
Let us see one real world example :
public class Employee : Entity
{
public virtual int CompanyUserId { get; set; }
public virtual CompanyUser CompanyUser { get; set; }
//... cut out for brevity
}
If you get the employees like this :
var employees = context.Employees;
You will not be able to access employees[0].CompanyUser after
context.SaveChanges() because of lazy loading. The connection is disposed after context.SaveChanges(), so no more data fetching.
But if you call :
var employees = context.Employees
.Include("CompanyUser")
.ToArray();
You will be able to access employees[0].CompanyUser.SomeProperty right away before context.SaveChanges regardless lazy loading because ToArray() will execute the query and fetch the entities with the "includes".
If you call :
var employees = context.Employees
.Include("CompanyUser");
Then you will have employee[0].CompanyUser.SomeProperty even after context.SaveChanges() with Lazy Loading because you have told EF to include "CompanyUser" property before executing the query. On execution EF will include the named property.
UPDATE
Intercepting DbContext can be done in at least two different ways.
First - override SaveChanges() or SaveChangesAsync because it is virtual:
public class MyDbContext : DbContext
{
public event Action<MyDbContext> SavingChanges = _ => { };
public override int SaveChanges()
{
this.SavingChanges(this);
return base.SaveChanges();
}
}
Second way without direct override is by hiding the DbContext inside interface like this one (this is from real project) :
public interface IUnitOfWork : IDisposable
{
void Commit();
}
Third way (somewhat different) is by intercepting the Db calls.
Fourth way exists but it depends on what IoC you use. If you use Castle Windsor you can use interceptors. I suppose that with every IoC there is its own way of intercepting this.

Wicket - Wrapped collection Model "transformation"

I have a domain object which has a collection of primitive values, which represent the primary keys of another domain object ("Person").
I have a Wicket component that takes IModel<List<Person>>, and allows you to view, remove, and add Persons to the list.
I would like to write a wrapper which implements IModel<List<Person>>, but which is backed by a PropertyModel<List<Long>> from the original domain object.
View-only is easy (Scala syntax for brevity):
class PersonModel(wrappedModel: IModel[List[Long]]) extends LoadableDetachableModel[List[Person]] {
#SpringBean dao: PersonDao =_
def load: List[Person] = {
// Returns a collection of Persons for each id
wrappedModel.getObject().map { id: Long =>
dao.getPerson(id)
}
}
}
But how might I write this to allow for adding and removing from the original List of Longs?
Or is a Model not the best place to do this translation?
Thanks!
You can do something like this:
class PersonModel extends Model<List<Person>> {
private transient List<Person> cache;
private IModel<List<String>> idModel;
public PersonModel( IModel<List<String>> idModel ) {
this.idModel = idModel;
}
public List<Person> getObject() {
if ( cache == null ) {
cache = convertIdsToPersons( idModel.getObject() );
return cache;
}
public void setObject( List<Person> ob ) {
cache = null;
idModel.setObject( convertPersonsToIds( ob ) );
}
}
This isn't very good code but it shows the general idea. One thing you need to consider is how this whole thing will be serialised between requests, you might be better off extending LoadableDetachableModel instead.
Another thing is the cache: it's there to avoid having to convert the list every time getObject() is called within a request. You may or may not need it in practice (depends on a lot of factors, including the speed of the conversion), but if you use it, it means that if something else is modifying the underlying collection, the changes may not be picked up by this model.
I'm not quite sure I understand your question and I don't understand the syntax of Scala.
But, to remove an entity from a list, you can provide a link that simply removes it using your dao. You must be using a repeater to populate your Person list so each repeater entry will have its own Model which can be passed to the deletion link.
Take a look at this Wicket example that uses a link with a repeater to select a contact. You just need to adapt it to delete your Person instead of selecting it.
As for modifying the original list of Longs, you can use the ListView.removeLink() method to get a link component that removes an entry from the backing list.

Why is my Entity Framework Code First proxy collection null and why can't I set it?

I am using DBContext and have two classes whose properties are all virtual. I can see in the debugger that I am getting a proxy object when I query the context. However, a collection property is still null when I try to add to it. I thought that the proxy would ensure that collection is initialized.
Because my Poco object can be used outside of its data context, I added a check for the collection being null in the constructor and create it if necessary:
public class DanceStyle
{
public DanceStyle()
{
if (DanceEvents == null)
{
DanceEvents = new Collection<DanceEvent>();
}
}
...
public virtual ICollection<DanceEvent> DanceEvents { get; set; }
}
That works outside the data context but if I retrieve an object using a query, although the test is true, when I try to set it, I get following exception: 'The property 'DanceEvents' on type 'DanceStyle_B6089AE40D178593955F1328A70EAA3D8F0F01DDE9F9FBD615F60A34F9178B94' cannot be set because the collection is already set to an EntityCollection.'
I can see it is null and I cannot add to it, but neither can I set it to a collection because the proxy says it is already set. Therefore I cannot use it. I'm confused.
Here is the DanceEvent class:
public class DanceEvent
{
public DanceEvent()
{
if (DanceStyles == null)
{
DanceStyles = new Collection<DanceStyle>();
}
}
...
public virtual ICollection<DanceStyle> DanceStyles { get; set; }
}
I have omitted the other value-type properties from the code above. I have no other mappings for those classes in the context class.
As you correctly observed in the answer to your own question, removing the "virtual" keyword from the collection properties works around the problem, by preventing the Entity Framework from creating a change tracking proxy. However, this is not a solution for many people, because change tracking proxies can be really convenient and can help prevent issues when you forget to detect changes at the right places in your code.
A better approach would be to modify your POCO classes, so that they instantiate the collection properties in their get accessor, rather than in the constructor. Here's your POCO class, modified to allow change tracking proxy creation:
public class DanceEvent
{
private ICollection<DanceStyle> _danceStyles;
public virtual ICollection<DanceStyle> DanceStyles
{
get { return _danceStyles ?? (_danceStyles = new Collection<DanceStyle>()); }
protected set { _danceStyles = value; }
}
}
In the above code the collection property is no longer automatic, but rather has a backing field. It's better if you leave the setter protected, preventing any code (other than the proxy) from subsequently modifying these properties. You will notice that the constructor was no longer necessary and was removed.
I found the solution to this problem here: Code First adding to collections? How to use Code First with repositories?
I removed 'virtual' from all properties except collections and lazy loaded objects, that is, all native types.
But I still don't understand how you can end up with the situation where you have a null collection that you cannot use and have no way to set it to a valid collection.
I also found this answer from Rowan Miller on an MSDN forum
Hi,
If you make all your properties virtual then EF will generate proxy classes at runtime that derives from your POCO classed, these proxies allow EF to find out about changes in real time rather than having to capture the original values of your object and then scan for changes when you save (this is obviously has performance and memory usage benefits but the difference will be negligible unless you have a large number of entities loaded into memory). These are known as 'change tracking proxies', if you make your navigation properties virtual then a proxy is still generated but it is much simpler and just includes some logic to perform lazy loading when you access a navigation property.
Because your original code was generating change tracking proxies, EF was replacing your collection property with a special collection type to help it find out about changes. Because you try and set the collection back to a simple list in the constructor you are getting the exception.
Unless you are seeing performance issues I would follow Terrence's suggestion and just remove 'virtual' from your non-navigation properties.
~Rowan
So it appears that I only have the problem with a full 'change tracking proxy' if all my properties are virtual. But given that, why can I still not use the virtual property on the change tracking proxy? This code blows up on line three because ds2.DanceEvents is null and cannot be set in the constructor:
DanceStyle ds2 = ctx.DanceStyles.Where(ds => ds.DanceStyleId == 1).Single();
DanceEvent evt = CreateDanceEvent();
ds2.DanceEvents.Add(evt);
I'm still confused, even though my code is now working because of the fix above.
Old question...
Poco class:
public partial class MyPOCO
{
public MyPOCO()
{
this.MyPocoSub = new HashSet<MyPocoSub>();
}
//VIRTUAL
public virtual ICollection<MyPocoSub> MyPocoSub { get; set; }
}
and proxy code:
public override ICollection<MyPocoSubSet> MyPocoSubSets
{
get
{
ICollection<MyPocoSubSet> myPocoSubSets = base.MyPocoSubSets;
if (!this.ef_proxy_interceptorForMyPocoSubSets(this, myPocoSubSets))
{
return base.MyPocoSubSets;
}
return myPocoSubSets;
}
set
{
if (value != this.RelationshipManager.GetRelatedEnd("WindowsFormsApplication.Models.MyPocoSubSet_MyPOCO", "MyPocoSubSet_MyPOCO_Source"))
{
// EXCEPTION
throw new InvalidOperationException("The property 'MyPocoSubSets' on type 'MyPOCO_A78FCE6C6A890855C68B368B750864E3136B589F9023C7B1D90BF7C83FD291AC' cannot be set because the collection is already set to an EntityCollection.");
}
base.MyPocoSubSets = value;
}
}
As you can see that exception raised in proxy class in ExtityFramework 5. This means that behavior still exist.