Entity Framework Core 5.0 - Many to many select query - entity-framework-core

I am trying to get a single User, with a list of Items, mapped with a many-to-many entity UserItems. However, I am unable to retrieve the mapped Items due to to an error that I'm unable to solve (error at bottom of question). Here is my code:
public class User
{
public int Id { get; set; }
public ICollection<UserItem> UserItems { get; set; }
}
public class Item
{
public int Id { get; set; }
public ICollection<UserItem> UserItems { get; set; }
}
public class UserItem
{
public int Id { get; set; }
public int UserId { get; set; }
public User User { get; set; }
public int ItemId { get; set; }
public Item Item { get; set; }
public int Quantity { get; set; }
}
The UserItem class configuration has the following relationships defined:
builder.HasOne(x => x.User)
.WithMany(x => x.UserItems)
.HasForeignKey(x => x.UserId)
.OnDelete(DeleteBehavior.ClientCascade);
builder.HasOne(x => x.Item)
.WithMany(x => x.UserItems)
.HasForeignKey(x => x.ItemId)
.OnDelete(DeleteBehavior.ClientCascade);
I have the following generic repo with this method:
public class GenericRepository<T> : where T : class
{
private readonly DbContext _context;
public GenericRepository(DbContext context) => _context = context;
public T Get(Expression<Func<T, bool>> where, params Expression<Func<T, object>>[] navigationProperties)
{
IQueryable<T> query = _context.Set<T>();
query = navigationProperties.Aggregate(query, (current, property) => current.Include(property));
var entity = query.FirstOrDefault(where);
return entity;
}
}
However, when I try to run the code, I get an error on the Select(x => x.Item):
var user = repo.Get(x => x.Id == 1, x => x.UserItems.Select(y => y.Item));
Error:
System.InvalidOperationException: 'The expression 'x.UserItems.AsQueryable().Select(y => y.Item)' is invalid inside an 'Include' operation, since it does not represent a property access: 't => t.MyProperty'. To target navigations declared on derived types, use casting ('t => ((Derived)t).MyProperty') or the 'as' operator ('t => (t as Derived).MyProperty'). Collection navigation access can be filtered by composing Where, OrderBy(Descending), ThenBy(Descending), Skip or Take operations. For more information on including related data, see http://go.microsoft.com/fwlink/?LinkID=746393.'
What am I doing wrong, this seems to work for my other projects?

This error Occurs because you are not passing in a navigation property (x.UserItems would be a navigation property) but rather something you want to do with the navigation property. UserItems.Select(y => y.Item) is not a property of x because Select() is a function and therefore it cannot be included.
What you are trying to do (I assume it is including UserItems and also the corresponding Items) is not going to work with your current implementation of the repository. To include navigation properties of navigation properties .ThenInclude() must be used instead of .Include() which works only for navigation properties directly defined on the Entity the DbSet is created for.
But apart from your question I would suggest not to use such an generic implementation of Repository. The main benefit from using reposiories is to separarte code related to loading and storing of entities from the rest of your code. In your case if the consumer of repository knows that navigation properties must be included and that he has to provide them - then what is the point of having a repository at all? Then the consumer again cares about database specific code which makes having a repository unneccessary. I would recommend just making a conrete "UserRepository" which can only be used to retrieve users and explicitly includes the needed properties.

Related

EF Core 6 not loading child entities

I am working on a simple web API that is just supposed to parse a JSON tree, and save it to a database. I am working with EF Core 6.0.4 and my application shows a really weird behaviour: right after saving the tree, it loads from the context just fine. But when I call a different endpoint and load the data using a freshly initialized context, the children won't load. I believe it's due to the EF config, but I can't figure out how to load the children.
I tried using _context.Entity(returnValue).Collection(x => x.Children) but the x in the lambda only has ICollection extension methods, not the model fields like in the code examples I saw.
I also tried using .Include, but that seems to be a thing from regular EF.
Here's some of my code:
Controller
public class CategoryTreeManagerController : ControllerBase
{
private readonly CategoryTreeManagerService _service;
public CategoryTreeManagerController(CategoryTreeManagerService service)
{
_service = service;
}
[HttpGet]
public IEnumerable<CategoryTreeNode> GetTree()
{
return _service.GetTree(); //this only returns the root node without any children
}
[HttpPost]
public IEnumerable<CategoryTreeNode> SaveTree(CategoryTreeNode[] nodes)
{
_service.SaveTree(nodes[0]);
return _service.GetTree(); //this correctly returns the tree
}
}
Service
public class CategoryTreeManagerService
{
private readonly CategoryTreeManagerApiDbContext _context;
public CategoryTreeManagerService(CategoryTreeManagerApiDbContext context)
{
_context = context;
}
public IEnumerable<CategoryTreeNode> GetTree()
{
CategoryTreeNode[] returnValue = _context.CategoryTreeNodes
.Where(x => x.Parent == null) //just return the root node
.ToArray(); //easier for frontend
return returnValue;
}
public void SaveTree(CategoryTreeNode node)
{
if (_context.CategoryTreeNodes.Any(x => x.Id == node.Id))
{
_context.Update(node);
}
else
{
_context.CategoryTreeNodes.Add(node);
}
_context.SaveChanges();
}
}
DbContext
public class CategoryTreeManagerApiDbContext : DbContext
{
public CategoryTreeManagerApiDbContext() : base ()
{
Database.EnsureCreated();
}
public CategoryTreeManagerApiDbContext(DbContextOptions<CategoryTreeManagerApiDbContext> options) : base(options)
{
Database.EnsureCreated();
}
public DbSet<CategoryTreeNode> CategoryTreeNodes { get; set; }
public DbSet<TreeNodeDetail> TreeNodeDetails { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
base.OnConfiguring(optionsBuilder);
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<CategoryTreeNode>(entity =>
{
entity.HasKey(x => x.Id);
entity.HasOne(x => x.Parent)
.WithMany(x => x.Children)
.IsRequired(false)
.OnDelete(DeleteBehavior.Restrict);
});
}
Model classes
public class CategoryTreeNode
{
public int Id { get; set; }
public string Label { get; set; }
public string Story { get; set; }
public string Icon { get; set; }
public ICollection<TreeNodeDetail> Details { get; set; }
public ICollection<CategoryTreeNode> Children { get; set; }
[JsonIgnore]
public CategoryTreeNode? Parent { get; set; }
}
public class TreeNodeDetail
{
[JsonPropertyName("detailId")]
public string Id { get; set; }
[JsonPropertyName("detailTitle")]
public string Title { get; set; }
[JsonPropertyName("detailValue")]
public string Value { get; set; }
[JsonIgnore]
[ForeignKey("CategoryTreeNode")]
public int CategoryTreeNodeId { get; set; }
}
Include would work for one level, or more if you expand out the expression, but it's not an ideal solution for tree structures which could be variable depth. (I.e. child of a child of a child ...)
For instance:
CategoryTreeNode[] returnValue = _context.CategoryTreeNodes
.Include(x => x.Children)
.Where(x => x.Parent == null) //just return the root node
.ToArray(); //easier for frontend
would load all parents and their first level children. To load 2nd level children:
CategoryTreeNode[] returnValue = _context.CategoryTreeNodes
.Include(x => x.Children)
.ThenInclude(x => x.Children)
.Where(x => x.Parent == null) //just return the root node
.ToArray(); //easier for frontend
The issue is knowing how many levels to load, and each level produces a Cartesian Product for EF to work through, exponentially increasing the amount of data being loaded to build a tree. Loading an entire table once quickly becomes a much more efficient solution.
If you have a Single tree structure where you expect only one top-level entry, or want to load an entire reasonable set of top-level nodes then loading all entries into memory will work since EF will be tracking all of the entities and it can resolve all of the various references as it builds the entity structure. This has to load the entire set even if you only want one specific parent out of several possible parents.
If you have several top level parent nodes and a sizeable overall table size to work through, and do want to be able to load a single parent and it's children then one option is to add a de-normalized top-level ID reference to the tree node.
public int Id { get; set; }
public int? TopLevelId { get; set; }
This would be a null-able FK but does not need a navigation property. The current Parent reference would continue to use a shadow property. (I.e. ParentId) In this way once you have an ID for the top-level parent you want to load a tree for:
_context.CategoryTreeNodes.Where(x => x.TopLevelId == topLevelId).ToList();
var topLevelNode = _context.CategoryTreeNodes.Single(x => x.Id == topLevelId);
The first statement will have the DBContext load and track all nodes under that top level tree node. Then when you call the DbContext to get that top level node, the tracked related entities will all get filled in.
The caveat of this approach is that it is a denormalization, in that there is no DB-level assertion that a TopLevelId is set, or remains set correctly. For instance if nodes can be moved between top-level entities and you forget to update this value, this node would not be loaded and associated under the new parent using the above load.

Make reference table data read-only - EF Core

I have a table (Commodity) which has a one-to-one relationship with another table (CommodityMaterial), in my GET endpoint the Commodity returns it's own columns and also the columns (and values) of the referenced table which works perfectly. However, in the POST operation of the endpoint, a user should not be able to POST data of the reference table (CommodityMaterial), how can this be achieved? I used to disable this by using a DataContract, however, because I need the columns for my GET operator, this is not an option.
I already tried, following this post: https://csharp.christiannagel.com/2016/11/07/efcorefields/, removing the SET on the reference table and making a backing field but this does not seem to work (error that the backing field is read-only).
I also tried setting the SET to protected, but this is not working.
So the question is, how to make the reference table read-only (only available for my GET endpoint and not my POST endpoint).
The Commodity POCO class:
[DataContract]
public class Commodity
{
public Commodity()
{
}
public Commodity(CommodityMaterial commodityMaterial)
{
CommodityMaterial = commodityMaterial;
}
[DataMember]
public long CommodityID { get; set; }
[DataMember]
public long CommodityMaterialID { get; set; }
[DataMember]
public decimal? SpecficWeight { get; set; }
[DataMember]
public CommodityMaterial CommodityMaterial { get; }
}
Fluent part:
modelBuilder.Entity<Commodity>(entity =>
{
entity.Property(e => e.CommodityID)
.HasColumnName("CommodityID")
.ValueGeneratedOnAdd();
entity.Property(e => e.CommodityMaterialID)
.HasColumnName("CommodityMaterialID");
entity.Property(e => e.SpecficWeight)
.HasColumnName("SpecficWeight")
.HasColumnType("decimal(18, 2)");
entity.HasOne(a => a.CommodityMaterial)
.WithOne(b => b.Commodity)
.HasForeignKey<Commodity>(b => b.CommodityMaterialID);
});
The parameters your action accepts should represent what your action does/is allowed to do. If a client should not be able to update a related entity, then the class you bind the request body to, should not have that entity available. Use a view model, essentially:
public class CommodityRequest
{
// all properties you want editable
// exclude `CommodityMaterial` obviously
}
Then:
public IActionResult Update(CommodityRequest model)

Nested Tables Using a DTO

I need help getting my WebApi Controller to work.
I have a 3 table Models like this.
First Table
public class MainTable{
public int MainTableID { get; set; }
... Other Fields
public ICollection<SubTable> SubTables { get; set; }
}
Second Table
public class SubTable{
public int SubTableID { get; set; }
... Other Fields
public int MainTableID { get; set; }
[ForeignKey("MainTableID ")]
[JsonIgnore]
public virtual MainTable MainTable{ get; set; }
public ICollection<SubSubTable> SubSubTables { get; set; }
}
Third Table
public class SubSubTable{
public int SubSubTableID { get; set; }
... Other Fields
public int SubTableID { get; set; }
[ForeignKey("SubTableID")]
[JsonIgnore]
public virtual SubTable SubTable{ get; set; }
}
I need to flatten the first model because of other relationships not mentioned in this post so I am using a dto
DTO
public class TableDTO
{
public int MainTableID { get; set; }
... Other Fields (there is a lot of flattening happening here but I am going to skip it to keep this simple)
public ICollection<SubTable> SubTables { get; set; }
}
Now that I got all of that out of the way. To my question.. I am linking this all to a web api controller.
If I use the DTO and create a controller like this
Controller with DTO
public IQueryable<TableDTO> GetMainTable()
{
var mainTable = from b in db.MainTables
.Include(b => b.SubTable.Select(e => e.SubSubTable))
select new TableDTO()
{
MainTableID = b.MainTableID
eager mapping of all the fields,
SubTables = b.SubTables
};
return mainTable;
}
This works for everything except the SubSubTable which returns null. If I ditch the DTO and create a controller like this
Controller without DTO
public IQueryable<MainTable> GetMainTable()
{
return db.MainTables
.Include(c => c.SubTables)
.Include(c => c.SubTables.Select(b => b.SubSubTables));
}
This works perfect and the JSon returns everything I need, except that I lose the DTO which I desperately need for other aspects of my code. I have rewritten my code in every way I can think of but nothing works. I am pretty sure that this can be done with the DTO but I don't know what it would take to make it work, and as they say "You don't know what you don't know" so hopefully someone here knows.
In Entity Framework 6 (and lower), Include is always ignored when the query ends in a projection, because the Include path can't be applied to the end result. Stated differently, Include only works if it can be positioned at the very end of the LINQ statement. (EF-core is more versatile).
This doesn't help you, because you explicitly want to return DTOs. One way to achieve this is to do the projection after you materialize the entities into memory:
var mainTable = from b in db.MainTables
.Include(b => b.SubTable.Select(e => e.SubSubTable))
.AsEnumerable()
select new MessageDTO()
{
MainTableID = b.MainTableID ,
// eager mapping of all the fields,
SubTables = b.SubTables
};
The phrase, "eager mapping of all the fields" suggests that the projection isn't going to narrow down the SELECT clause anyway, so it won't make much of a difference.
Another way could be to load all SubSubTable objects into the context that you know will be in the MainTables you fetch from the database. EF will populate all SubTable.SubSubTables collections by relationship fixup.
If this works:
public IQueryable<MainTable> GetMainTable()
{
return db.MainTables
.Include(c => c.SubTables)
.Include(c => c.SubTables.Select(b => b.SubSubTables));
}
Then use this one and just add a Select() to the end with a ToList(). Note the IEnumerable in the return type:
public IEnumerable<MainTableDto> GetMainTable()
{
return db.MainTables
.Include(c => c.SubTables)
.Include(c => c.SubTables.Select(b => b.SubSubTables))
.Select(c=> new MainTableDto { SubTables=c.SubTables /* map your properties here */ })
.ToList();
}
Not sure about the types though (at one place you have MainTableDto, at another you mention MessageDto?).

Entity Framework Many To Many exception with inheritance

I am trying to create two many-to-many relationship maps on a Record object:
Record object that is inherited from
public class Record {
public virtual ICollection<Language> SourceLanguages { get; set; }
public virtual ICollection<Language> TargetLanguages { get; set; }
}
Second Object
public class Language
{
public int Language { get; set; }
public string Locale { get; set; }
public string LanguageName { get; set; }
public virtual ICollection<Record> Records { get; set; }
}
Map for Record
public class RecordMap : EntityTypeConfiguration<Record>
{
this.HasMany(r => r.SourceLanguages)
.WithMany(c => c.Records)
.Map(sl =>
{
sl.ToTable("SourceLanguageRecordMap", "dbo");
sl.MapLeftKey("RecordId");
sl.MapRightKey("LanguageId");
});
this.HasMany(r => r.TargetLanguages)
.WithMany(c => c.Records)
.Map(tl =>
{
tl.ToTable("TargetLanguageRecordMap", "dbo");
tl.MapLeftKey("RecordId");
tl.MapRightKey("LanguageId");
});
}
When I run migration on the object listed above I get the following error:
System.Data.Entity.Core.MetadataException: Schema specified is not
valid. Errors: The relationship
'Toolbox.EntityModel.Contexts.Record_SourceLanguages' was not loaded
because the type 'Toolbox.EntityModel.Contexts.Language' is not
available. ...
Schema specified is not valid. Errors: The relationship
'Toolbox.EntityModel.Contexts.Record_SourceLanguages' was not loaded
because the type 'Toolbox.EntityModel.Contexts.Language' is not
available.
If I comment the following line out, it will work with just one many to many map, however, it will add RecordId_Record to Language Table. Any idea why?
this.HasMany(r => r.TargetLanguages)
.WithMany(c => c.Records)
.Map(tl =>
{
tl.ToTable("TargetLanguageRecordMap", "dbo");
tl.MapLeftKey("RecordId");
tl.MapRightKey("LanguageId");
});
Any idea as to what I am doing wrong?
If you have 2 Many-to-Many relationships to the same table you need to create 2 separate ICollection properties in order for Entity Framework to fully pick up on what you're trying to do. You can't combine them into one, or else you'll get that lovely error that you're seeing there.

Multiple level of inheritance in EF Code First Configuration

I have an abstract base class for a few entities I'm defining. One of those derived entities is actually a non-abstract base class to another entity.
Following this code:
public abstract class BaseReportEntry {
public int ReportEntryId { get; set;}
public int ReportBundleId { get; set; } //FK
public virtual ReportBundle ReportBunde { get; set; }
}
//A few different simple pocos like this one
public PerformanceReportEntry : BaseReportEntry {
public int PerformanceAbsolute { get; set; }
public double PerformanceRelative { get; set; }
}
//And one with a second level of inheritance
public ByPeriodPerformanceReportEntry : PerformanceReportEntry {
public string Period { get; set; }
}
I'm using a base EntityTypeConfiguration:
public class BaseReportEntryMap<TReportEntry> : EntityTypeConfiguration<TReportEntry>
where TReportEntry : BaseReportEntry
{
public BaseReportEntryMap()
{
this.HasKey(e => e.ReportEntryId);
this.HasRequired(e => e.ReportsBundle)
.WithMany()
.HasForeignKey(e => e.ReportsBundleId);
}
}
Presumably this works fine for the one-level of inheritance but throw the following error for that one case where it has a second level:
The foreign key component 'ReportsBundleId' is not a declared property on type 'ByPeriodPerformanceReportEntry'
public class ByPeriodPerformanceReportEntryMap : BaseReportEntryMap<ByPeriodPerformanceReportEntry>
{
public ByPeriodPerformanceReportEntryMap ()
: base()
{
this.Property(e => e.Period).IsRequired();
this.Map(m =>
{
m.MapInheritedProperties();
m.ToTable("ByPeriodPerformanceReportEntries");
});
}
}
Here's ReportBundle class if needed
public class ReportsBundle
{
public int ReportsBundleId { get; set; }
public virtual ICollection<PerformanceReportEntry> PerformanceReportEntries{ get; set; }
public virtual ICollection<ByPeriodPerformanceReportEntry> ByPeriodPerformanceReportEntries{ get; set; }
}
The problem is not so much the second level of inheritance but that PerformanceReportEntry (the base of ByPeriodPerformanceReportEntry) is an entity while BaseReportEntry (the base of PerformanceReportEntry) is not.
Your mapping would work if PerformanceReportEntry would not be an entity - i.e. its mapping is not added to the model builder configuration and you have no DbSet for this type and it would not occur in a navigation collection in ReportsBundle.
Deriving the configuration from BaseReportEntryMap<ByPeriodPerformanceReportEntry> is not possible in this case - and it is not necessary because the mapping for the base properties already happened by the BaseReportEntryMap<PerformanceReportEntry>. Therefore you can use
public class ByPeriodPerformanceReportEntryMap
: EntityTypeConfiguration<ByPeriodPerformanceReportEntry>
But I have doubt that the resulting model is as you would expect it. I don't know what the PerformanceReportEntries and ByPeriodPerformanceReportEntries collections in ReportsBundle are supposed to express. Do you expect that ByPeriodPerformanceReportEntries is a collection filtered by the subtype? Do you expect that PerformanceReportEntries contains only the ReportsEntries that are PerformanceReportEntrys but not ByPeriodPerformanceReportEntrys? Do you expect that PerformanceReportEntries contains all entries including the ByPeriodPerformanceReportEntries?
Anyway, BaseReportEntry.ReportBundle is a navigation property mapped in PerformanceReportEntry (not in ByPeriodPerformanceReportEntry). That means that the inverse navigation property in class ReportsBundle must refer to PerformanceReportEntry which is the PerformanceReportEntries navigation collection. ByPeriodPerformanceReportEntries will introduce a second one-to-many relationship between ReportsBundle and ByPeriodPerformanceReportEntry (without a navigation property in ByPeriodPerformanceReportEntry). The inverse navigation property of ByPeriodPerformanceReportEntries will NOT be BaseReportEntry.ReportBundle.
My feeling is that you should not have the ReportsBundle.ByPeriodPerformanceReportEntries collection, but I am not sure what you want to achieve exactly.
Edit
Refering to your comment that you only have these two Report types your mapping is way too complicated in my opinion. I would do the following:
Remove the BaseReportEntry class and move its properties into PerformanceReportEntry. It makes no sense to have a base class that only one single other class derives from.
Remove the ByPeriodPerformanceReportEntries from ReportsBundle, so that ReportsBundle will be:
public class ReportsBundle
{
public int ReportsBundleId { get; set; }
public virtual ICollection<PerformanceReportEntry>
PerformanceReportEntries { get; set; }
}
Remove the BaseReportEntryMap and move the mapping into PerformanceReportEntryMap. Derive this map from EntityTypeConfiguration<PerformanceReportEntry>.
Correct the mapping. Currently it is wrong because you don't specify the inverse navigation property in WithMany. PerformanceReportEntryMap should look like this:
public class PerformanceReportEntryMap
: EntityTypeConfiguration<PerformanceReportEntry>
{
public PerformanceReportEntryMap()
{
this.HasKey(e => e.ReportEntryId);
this.HasRequired(e => e.ReportsBundle)
.WithMany(b => b.PerformanceReportEntries)
.HasForeignKey(e => e.ReportsBundleId);
}
}
Derive ByPeriodPerformanceReportEntryMap from EntityTypeConfiguration<ByPeriodPerformanceReportEntry> and specify only mappings for properties that are declared in ByPeriodPerformanceReportEntry, not again for the base properties. That already happened in PerformanceReportEntryMap. You don't need and can't specify it again because it will cause exactly the exception you had.
Use Table-Per-Hierarchy (TPH) inheritance instead of Table-Per-Concrete-Type (TPC), especially if you only have a few properties declared in ByPeriodPerformanceReportEntry. TPC is more difficult to use because it has problems with database-generated identities and with polymorphic associations (which you have in your relationship between PerformanceReportEntry and ReportsBundle). The problems are explained in more details here. TPH instead offers the best performance. ByPeriodPerformanceReportEntryMap would then look like this:
public class ByPeriodPerformanceReportEntryMap
: EntityTypeConfiguration<ByPeriodPerformanceReportEntry>
{
public ByPeriodPerformanceReportEntryMap()
{
this.Property(e => e.Period).IsRequired();
}
}
No explicit configuration for TPH is necessary because it is the default inheritance mapping.