Entity Framework eager loading in many nested entities - entity-framework

I have six classes as shown below -
Class A{
int ValA;
ICollection<B> AllBs;
}
Class B{
int ValB;
ICollection<C> AllCs;
}
Class C{
int ValC;
ICollection<D> AllDs;
}
Class D{
int ValD;
ICollection<E> AllEs;
}
Class E{
int ValE;
ICollection<F> AllFs;
}
Class F{
int ValF;
}
I have to eager load an A entity with specific id and I want the result must load all the nested collections eagerly.
I wrote the following code to achieve the same -
var entity = _context.A.Where(a => a.Id == 1)
.Include(a => a.AllBs
.Select(b => b.AllCs
.Select(c => c.AllDs
.Select(d => d.AllEs
.Select(e => e.AllFs)))))
.FirstOrDefault();
Have I written the include statements correctly ? All the nested collections are not getting loaded properly. They are getting loaded upto AllBs, AllCs and further collections are not loaded.
Kindly help.

If this is EF Core you will want to use
_context.A
.Include(a => a.AllBs)
.ThenInclude(b => b.AllCs)
.ThenInclude(c => c.AllDs)
.ThenInclude(d => d.AllEs)
.ThenInclude(e => e.AllFs)
.Single(a => a.Id == 1)
Intellisense may give you a bit of grief diving through collections but it does work through.
For EF6, your statement should work, so if not I would look specifically at your mappings to ensure your FKs are resolving correctly. I'd also remove the .Where() & .FirstOrDefault() and instead use a .Single() at the end. It should not matter in the query generation, but Include statements are best placed before any filtering. Use an SQL Profiler to capture the SQL being sent to the server to see what tables are being requested and how they are joining. This might highlight some FK resolution errors.

Related

EF Core: Serializing composite objects with polymorphic associations

In my design, I have a Challenge aggregate root that maintains a list of strategies that collectively determine if the challenge is complete.
There are several types of strategies that examine the challenge submission in their own way. Each type of strategy inherits from the base ChallengeFullfilmentStrategy, as shown in the diagram.
When a submission is made, I load the challenge along with its strategies with the following code:
return _dbContext.Challenges
.Where(c => c.Id == challengeId)
.Include(c => c.Solution)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as BasicMetricChecker).ClassMetricRules)
.ThenInclude(r => r.Hint)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as BasicMetricChecker).MethodMetricRules)
.ThenInclude(r => r.Hint)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as BasicNameChecker).Hint)
.FirstOrDefault();
This setup is a consequence of the polymorphism introduced by the strategy hierarchy. Recently, I've tried to add a more sophisticated strategy (ProjectChecker, marked in red on the diagram). Through an intermediate class, it introduces a composite relationship, where now a Strategy has a list of Strategies (through the SnippetStrategies class).
This change severely complicates the data model, as now the code should look something like this:
return _dbContext.Challenges
.Where(c => c.Id == challengeId)
.Include(c => c.Solution)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as BasicMetricChecker).ClassMetricRules)
.ThenInclude(r => r.Hint)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as BasicMetricChecker).MethodMetricRules)
.ThenInclude(r => r.Hint)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as BasicNameChecker).Hint)
.Include(c => c.FulfillmentStrategies)
.ThenInclude(s => (s as ProjectChecker).SnippetStrategies)
.ThenInclude(snippetStrats => snippetStrats.Strategies)
.ThenInclude(s => (s as BasicMetricChecker).MethodMetricRules)
//More code here to include the other children, not sure if this can even work.
.FirstOrDefault();
I'm not sure if I've hit a limitation of EF Core or if I'm not aware of some mechanism that can solve this issue.
If it's the former, I was considering serializing my Challenge aggregate into a JSON object (I'm using postgresql) and removing this part of the relationship model. While it makes sense from a domain perspective (I either need only the challenge header or all of it - including all the strategies etc.), my research so far has revealed that System.Text.Json suffers from the same limitation and that I'll need to write a custom JSONConverter to enable serialization of this data structure.
The cleanest option I've found so far is the use of Newtonsoft along with the JsonSubtypes library, but I wanted to check if I'm missing something that would help solve my issue without introducing these dependencies (especially since JsonSubtypes seems to be less active).
This is a static helper method I created for when I had the same problem, utilizing reflection.
Imagine it like a beefed up .Find(). It loops through all the properties of a class and loads them in the same fashion as you would with the .Include() and .ThenInclude() pattern that you follow in your code.
This does have an extra content in the first if/else block where it finds the DbContext Model from an Interface being passed in for the Type parameter. As in our case we used quite a few interfaces to classify our Model classes (figured it could be useful). This has some hefty overhead, so be careful. We use the FullFindAsync for very specific use cases, and typically try to do .Includes().
You could also add further depth searching to the foreach loop at the end if you want to dive deeper in what gets loaded. Again, this has some hefty overhead as well, so make sure to test it on large datasets.
public static async Task<object> FullFindAsync(this DbContext context, Type entityType, params object[] keyValues)
{
object entity = null;
//handle an Interface as a passed in type
if (entityType.IsInterface)
{
//get all Types of Models in the DbContext that implement the Interface
var implementedModels = context.Model.GetEntityTypes()
.Select(et => et.ClrType)
.Where(t => entityType.IsAssignableFrom(t))
.ToList();
//loop through all Models and try to find the object via the parameters
foreach (var modelType in implementedModels)
{
entity = await context.FindAsync(modelType, keyValues);
if (entity != null) break;
}
}
else
//find the object, via regular Find
entity = await context.FindAsync(entityType, keyValues);
//didnt find the object
if (entity == null) return null;
//loop through all navigation properties
foreach (var navigation in entity.GetType().GetProperties())
{
//skip NotMapped properties (to avoid context collection error)
if (context.Entry(entity).Metadata.FindNavigation(navigation.Name) == null)
continue;
//check and load reference and collection properties
if (typeof(IDatabaseObject).IsAssignableFrom(navigation.PropertyType))
context.Entry(entity).Reference(navigation.Name).Load();
if (typeof(IEnumerable<object>).IsAssignableFrom(navigation.PropertyType))
context.Entry(entity).Collection(navigation.Name).Load();
}
return entity;
}
//***USAGES***
var fullModel = await _dbContext.FullFindAsync(typeof(MyModelObject), model.Id);
var correctModel = await _dbContext.FullFindAsync(typeof(IModelInterface), someRandomId);

What does Entity Framework Context Add do?

we are using Entity Framework 5.0 and have a use case for creating a preview.
So I though I'd see if I could use EF to build my preview in EF data model classes and not persist (no SaveChanges).
(I could of course do this without EF, but I'd thought I'd experiment.)
Now I see some strange things when adding to my context.
In the code snippet below, the Add calls do not effect the collections added to (the DbSets). before = after = 0.
When I do Child.Add, the relation between parent and child is set (parent.Child now contains child). It did not do that before calling Add.
private void Insert(Parent parent, Child child)
{
// Context.Parent does not get any items:
int before = this.Context.Parent.Count();
this.Context.Parent.Add(parent);
int after = this.Context.Parent.Count();
// This affects Parent.Child:
this.Context.Child.Add(child);
}
Here is the vital part from the Context class (removed a lot for clarity):
modelBuilder.Entity<Child>(entity =>
{
// Removed code here...
entity.HasOne(d => d.Parent)
.WithMany(p => p.Child)
.HasForeignKey(d => d.ChildId)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Child_Parent");
});
So what does Add really do?

How to convert a method of a ValueObject to SQL with Entity Framework Core 3.1

I have this value object
public class ProductReference : ValueObject
{
protected ProductReference(){}
public ProductReference(string value){}
public string Value{get; protected set;}
}
I use it in my entity as :
public class Product : Entity<long>
{
protected Product(){}
public ProductReference Reference{get; protected set;}
}
In the OnModelCreating of my DbContext I defined :
modelBuilder.Entity<Product>(entity => {
entity.Property(a => a.Reference)
.HasColumnName("Reference")
.HasConversion(
a => a.Value,
s => new ProductReference (s);
});
When I do :
await dbcontext.Products.Where(p=>p.Reference.Value.Contains("some text")).toArrayAsync();
I get an exception
Expression cannot be converted to a valid SQL statement
I know for sure there is a way to create a custom expression converter, but I cannot find a good, simple and EF Core 3.1 compatible example to deal with my issue and that explain me clearly the concepts I miss.
I found this very interesting project
https://github.com/StevenRasmussen/EFCore.SqlServer.NodaTime
but it is too advanced for me to reproduce it for only my use case.
[EDIT] the ValueObject ans Entity are from
CSharpFunctionalExtensions nuget package, I dont think they are really relevant in my question.
I am not completely sure if i understand correctly what you want to accomplish, but you could try to configure your ProductReference as an Owned Entity Type.
Here you would transform the following code from:
modelBuilder.Entity<Product>(entity => {
entity.Property(a => a.Reference)
.HasColumnName("Reference")
.HasConversion(
a => a.Value,
s => new ProductReference (s);
});
to
modelBuilder.Entity<Product>(entity => {
entity.OwnsOne(a => a.Reference, referenceBuilder => {
referenceBuilder.Property(p => p.Value).HasColumnName("Reference");
});
});
With that your select statement should work.
It could be that you have to play around with the properties of your class ProductReference, or use some modifiers of the fluent API.
So first for some context on what is happening here behind the scenes and why its not gonna work even for build in simple converters like BoolToZeroOneConverter.
The problem here is that you are calling when converting the new ProductReference(s). This is method where you can do whatever you want in it. For example if use it in a Select statement it will again fail. For example:
await dbcontext.Products
.Select(x=>new ProductReference(x.Value))
.toArrayAsync();
The reason is obvious, it won't be able to translate. But why it cant transform it to a query?
Because you are passing a constructor. Inside this constructor you could be doing API calls or using Reflections to set the variables to your object, pretty much anything. That of course is not able to be translated in an SQL query.
Converters are generally used for in memory but they can be used for databse operations as well. This would mean that you will need something like this:
await dbcontext.Products
.Select(x=>new ProductReference() // empty constructor that does nothing
{
Property1 = x.Property1 // I don't know how the constructor maps them
})
.toArrayAsync();
Using this type of expression allow you to actually transalte the expression to an SQL statement and not making the conversion on the SQL DB and not in memory.
Now in your specific case using:
.HasConversion(
a => a.Value,
s => new ProductReference (){};
});
Should fix your issues but I fail to understand why would you want to initialize or convert a ProductReference to a ProductReference.

What is the difference between creating a new object inside select LINQ clause and inside a method

I have an entity class which is mapped to an SQL table:
public class EntityItem {
public virtual ICollection<EntityItem2> SomeItems { get; set; }
}
And I have the following two snippets:
var items = _repository.Table.Where(x => x.Id == id)
.Select(x => new ItemModel {
Items = x.SomeItems.Select(y => new SomeItem { //mapping is here...}).ToList()
});
And
var items = _repository.Table.Where(x => x.Id == id).Select(x => someModelMapper.BuildModel(x));
//inside a mapper
public ItemModel BuildModel(EntityType entity){
var model = new ItemModel();
model.Items = entity.SomeItems.Select(x => anotherMapper.BuildModel(x));
return model;
}
As a result, I am getting different SQL queries in both cases. Moreover, the second snippet is working much slower than the first one. As I can see in SQL profiler the second snipper is generating many SQL queries.
So my questions:
Why is that happening?
How to create new objects like in the second snippet but to avoid
lots of SQL queries?
The likely reason you are seeing a difference in performance is due to EF Core materializing the query prematurely. When a Linq statement is compiled, EF attempts to translate it into SQL. If you call a function within the expression, EF6 would have raised an exception to the effect that the method cannot be converted to SQL. EF Core tries to be clever, and when it encounters a method it cannot convert, it executes the query up to the point it could get to, and then continues to execute the rest as Linq2Object where you method can run. IMO this is a pretty stupid feature and represents a huge performance landmine, and while it's fine to offer it as a possible option, it should be disabled by default.
You're probably seeing extra queries due to lazy loading after the main query runs, to populate the view models in the mapping method.
For instance if I execute:
var results = context.Parents.Select(x => new ParentViewModel
{
ParentId = x.ParentId,
Name = x.Name,
OldestChildName = x.Children.OrderByDescending(c => c.BirthDate).Select(c => c.Name).FirstOrDefault() ?? "No Child"
}).Single(x => x.ParentId == parentId);
That would execute as one statement. Calling a method to populate the view model:
var results = context.Parents
.Select(x => buildParentViewModel(x))
.Single(x => x.ParentId == parentId);
would execute something like:
var results = context.Parents
.ToList()
.Select(x => new ParentViewModel
{
ParentId = x.ParentId,
Name = x.Name,
OldestChildName = x.Children.OrderByDescending(c => c.BirthDate).Select(c =>
c.Name).FirstOrDefault() ?? "No Child"
}).Single(x => x.ParentId == parentId);
at worst or:
var results = context.Parents
.Where(x => x.ParentId == parentId)
.ToList()
.Select(x => new ParentViewModel
{
ParentId = x.ParentId,
Name = x.Name,
OldestChildName = x.Children.OrderByDescending(c => c.BirthDate).Select(c =>
c.Name).FirstOrDefault() ?? "No Child"
}).Single();
... at best. These are due to the Extra .ToList() call prior to the Select which is roughly what the premature execution will do automatically. The issue with these queries compared to the first one is that when it comes to loading the child's name. In the first query, the SQL generated pulls the parent and related child's details in one query. In the alternative cases the query will execute to pull the parent's details, but getting the child details will constitute a lazy load call to get more details since that will be executed as Linq2Object.
The solution would be to use Automapper and it's built in ProjectTo method to populate your view model. This will place the mapping code in automatically so that it works like the first scenario without you needing to write out all of the mapping code.

Eager loading of 2 levels in entity framework using Lambda Expression

I am working with MS entity-framework core and I am trying to do a eager loading query. For that I have the following method:
public FieldSet GetFieldSetById(int id)
{
return _context.FieldSet.Include("FieldSetFields.Field").SingleOrDefault(fs => fs.FieldSetId == id);
}
this code works fine, but I was wondering how can I achieve it using the lambda syntax (System.Func).
So far I have this one that works as well, but does not include the "Field".
return _context.FieldSet.Include(e => e.FieldSetFields).SingleOrDefault(fs => fs.FieldSetId == id);
FieldSetFields is a list, and then I cannot call .Field. How can I achieve this?
Use ThenInclude:
return _context.FieldSet
.Include(e => e.FieldSetFields)
.ThenInclude(fsf => fsf.Field)
.SingleOrDefault(fs => fs.FieldSetId == id);