Nested Tables Using a DTO - entity-framework

I need help getting my WebApi Controller to work.
I have a 3 table Models like this.
First Table
public class MainTable{
public int MainTableID { get; set; }
... Other Fields
public ICollection<SubTable> SubTables { get; set; }
}
Second Table
public class SubTable{
public int SubTableID { get; set; }
... Other Fields
public int MainTableID { get; set; }
[ForeignKey("MainTableID ")]
[JsonIgnore]
public virtual MainTable MainTable{ get; set; }
public ICollection<SubSubTable> SubSubTables { get; set; }
}
Third Table
public class SubSubTable{
public int SubSubTableID { get; set; }
... Other Fields
public int SubTableID { get; set; }
[ForeignKey("SubTableID")]
[JsonIgnore]
public virtual SubTable SubTable{ get; set; }
}
I need to flatten the first model because of other relationships not mentioned in this post so I am using a dto
DTO
public class TableDTO
{
public int MainTableID { get; set; }
... Other Fields (there is a lot of flattening happening here but I am going to skip it to keep this simple)
public ICollection<SubTable> SubTables { get; set; }
}
Now that I got all of that out of the way. To my question.. I am linking this all to a web api controller.
If I use the DTO and create a controller like this
Controller with DTO
public IQueryable<TableDTO> GetMainTable()
{
var mainTable = from b in db.MainTables
.Include(b => b.SubTable.Select(e => e.SubSubTable))
select new TableDTO()
{
MainTableID = b.MainTableID
eager mapping of all the fields,
SubTables = b.SubTables
};
return mainTable;
}
This works for everything except the SubSubTable which returns null. If I ditch the DTO and create a controller like this
Controller without DTO
public IQueryable<MainTable> GetMainTable()
{
return db.MainTables
.Include(c => c.SubTables)
.Include(c => c.SubTables.Select(b => b.SubSubTables));
}
This works perfect and the JSon returns everything I need, except that I lose the DTO which I desperately need for other aspects of my code. I have rewritten my code in every way I can think of but nothing works. I am pretty sure that this can be done with the DTO but I don't know what it would take to make it work, and as they say "You don't know what you don't know" so hopefully someone here knows.

In Entity Framework 6 (and lower), Include is always ignored when the query ends in a projection, because the Include path can't be applied to the end result. Stated differently, Include only works if it can be positioned at the very end of the LINQ statement. (EF-core is more versatile).
This doesn't help you, because you explicitly want to return DTOs. One way to achieve this is to do the projection after you materialize the entities into memory:
var mainTable = from b in db.MainTables
.Include(b => b.SubTable.Select(e => e.SubSubTable))
.AsEnumerable()
select new MessageDTO()
{
MainTableID = b.MainTableID ,
// eager mapping of all the fields,
SubTables = b.SubTables
};
The phrase, "eager mapping of all the fields" suggests that the projection isn't going to narrow down the SELECT clause anyway, so it won't make much of a difference.
Another way could be to load all SubSubTable objects into the context that you know will be in the MainTables you fetch from the database. EF will populate all SubTable.SubSubTables collections by relationship fixup.

If this works:
public IQueryable<MainTable> GetMainTable()
{
return db.MainTables
.Include(c => c.SubTables)
.Include(c => c.SubTables.Select(b => b.SubSubTables));
}
Then use this one and just add a Select() to the end with a ToList(). Note the IEnumerable in the return type:
public IEnumerable<MainTableDto> GetMainTable()
{
return db.MainTables
.Include(c => c.SubTables)
.Include(c => c.SubTables.Select(b => b.SubSubTables))
.Select(c=> new MainTableDto { SubTables=c.SubTables /* map your properties here */ })
.ToList();
}
Not sure about the types though (at one place you have MainTableDto, at another you mention MessageDto?).

Related

Entity Framework Core 5.0 - Many to many select query

I am trying to get a single User, with a list of Items, mapped with a many-to-many entity UserItems. However, I am unable to retrieve the mapped Items due to to an error that I'm unable to solve (error at bottom of question). Here is my code:
public class User
{
public int Id { get; set; }
public ICollection<UserItem> UserItems { get; set; }
}
public class Item
{
public int Id { get; set; }
public ICollection<UserItem> UserItems { get; set; }
}
public class UserItem
{
public int Id { get; set; }
public int UserId { get; set; }
public User User { get; set; }
public int ItemId { get; set; }
public Item Item { get; set; }
public int Quantity { get; set; }
}
The UserItem class configuration has the following relationships defined:
builder.HasOne(x => x.User)
.WithMany(x => x.UserItems)
.HasForeignKey(x => x.UserId)
.OnDelete(DeleteBehavior.ClientCascade);
builder.HasOne(x => x.Item)
.WithMany(x => x.UserItems)
.HasForeignKey(x => x.ItemId)
.OnDelete(DeleteBehavior.ClientCascade);
I have the following generic repo with this method:
public class GenericRepository<T> : where T : class
{
private readonly DbContext _context;
public GenericRepository(DbContext context) => _context = context;
public T Get(Expression<Func<T, bool>> where, params Expression<Func<T, object>>[] navigationProperties)
{
IQueryable<T> query = _context.Set<T>();
query = navigationProperties.Aggregate(query, (current, property) => current.Include(property));
var entity = query.FirstOrDefault(where);
return entity;
}
}
However, when I try to run the code, I get an error on the Select(x => x.Item):
var user = repo.Get(x => x.Id == 1, x => x.UserItems.Select(y => y.Item));
Error:
System.InvalidOperationException: 'The expression 'x.UserItems.AsQueryable().Select(y => y.Item)' is invalid inside an 'Include' operation, since it does not represent a property access: 't => t.MyProperty'. To target navigations declared on derived types, use casting ('t => ((Derived)t).MyProperty') or the 'as' operator ('t => (t as Derived).MyProperty'). Collection navigation access can be filtered by composing Where, OrderBy(Descending), ThenBy(Descending), Skip or Take operations. For more information on including related data, see http://go.microsoft.com/fwlink/?LinkID=746393.'
What am I doing wrong, this seems to work for my other projects?
This error Occurs because you are not passing in a navigation property (x.UserItems would be a navigation property) but rather something you want to do with the navigation property. UserItems.Select(y => y.Item) is not a property of x because Select() is a function and therefore it cannot be included.
What you are trying to do (I assume it is including UserItems and also the corresponding Items) is not going to work with your current implementation of the repository. To include navigation properties of navigation properties .ThenInclude() must be used instead of .Include() which works only for navigation properties directly defined on the Entity the DbSet is created for.
But apart from your question I would suggest not to use such an generic implementation of Repository. The main benefit from using reposiories is to separarte code related to loading and storing of entities from the rest of your code. In your case if the consumer of repository knows that navigation properties must be included and that he has to provide them - then what is the point of having a repository at all? Then the consumer again cares about database specific code which makes having a repository unneccessary. I would recommend just making a conrete "UserRepository" which can only be used to retrieve users and explicitly includes the needed properties.

GroupBy Expression failed to translate

//Model
public class Application
{
[Key]
public int ApplicationId { get; set; }
public DateTime CreatedAt { get; set; }
public DateTime ConfirmedDate { get; set; }
public DateTime IssuedDate { get; set; }
public int? AddedByUserId { get; set; }
public virtual User AddedByUser { get; set; }
public int? UpdatedByUserId { get; set; }
public virtual User UpdatedByuser { get; set; }
public string FirstName { get; set; }
public string MiddleName { get; set; }
public string LastName { get; set; }
public string TRN { get; set; }
public string EmailAddress { get; set; }
public string Address { get; set; }
public int ParishId { get; set; }
public Parish Parish { get; set; }
public int? BranchIssuedId { get; set; }
public BranchLocation BranchIssued { get; set; }
public int? BranchReceivedId { get; set; }
public BranchLocation BranchReceived {get; set; }
}
public async Task<List<Application>> GetApplicationsByNameAsync(string name)
{
if (string.IsNullOrEmpty(name))
return null;
return await _context.Application
.AsNoTracking()
.Include(app => app.BranchIssued)
.Include(app => app.BranchReceived)
.Include(app => app.Parish)
.Where(app => app.LastName.ToLower().Contains(name.ToLower()) || app.FirstName.ToLower()
.Contains(name.ToLower()))
.GroupBy(app => new { app.TRN, app })
.Select(x => x.Key.app)
.ToListAsync()
.ConfigureAwait(false);
}
The above GroupBy expression fails to compile in VS Studio. My objective is to run a query filtering results by name containing a user given string and then it should group the results by similar TRN numbers returning a list of those applications to return to the view. I think I am really close but just cant seem to figure out this last bit of the query. Any guidance is appreciated.
Error being presented
InvalidOperationException: The LINQ expression 'DbSet<Application>
.Where(a => a.LastName.ToLower().Contains(__ToLower_0) || a.FirstName.ToLower().Contains(__ToLower_0))
.GroupBy(
source: a => new {
TRN = a.TRN,
app = a
},
keySelector: a => a)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()
UPDATE
Seems it is definitely due to a change in how .net core 3.x and EF core play together since recent updates. I had to change it to client evaluation by using AsEnumerable() instead of ToListAsync(). The rest of the query given by Steve py works with this method. I was unaware even after reading docs how the groupby really worked in LINQ, so that has helped me a lot. Taking the query to client side eval may have performance issues however.
The GroupBy support in EF core is a joke.
This worked perfectly on the server in EF6
var nonUniqueGroups2 = db.Transactions.GroupBy(e => new { e.AccountId, e.OpeningDate })
.Where(grp => grp.Count() > 1).ToList();
In EF core it causes an exception "Unable to translate the given 'GroupBy' pattern. Call 'AsEnumerable' before 'GroupBy' to evaluate it client-side." The message is misleading, do not call AsEnumerable because this should be handled on the server.
I have found a workaround here. An additional Select will help.
var nonUniqueGroups = db.Transactions.GroupBy(e => new { e.AccountId, e.OpeningDate })
.Select(x => new { x.Key, Count = x.Count() })
.Where(x => x.Count > 1)
.ToList();
The drawback of the workaround is that the result set does not contain the items in the groups.
There is an EF Core issue. Please vote on it so they actually fix this.
Based on this:
I want to group by TRN which is a repeating set of numbers eg.12345, in the Application table there may be many records with that same sequence and I only want the very latest row within each set of TRN sequences.
I believe this should satisfy what you are looking for:
return await _context.Application
.AsNoTracking()
.Include(app => app.BranchIssued)
.Include(app => app.BranchReceived)
.Include(app => app.Parish)
.Where(app => app.LastName.ToLower().Contains(name.ToLower()) || app.FirstName.ToLower()
.Contains(name.ToLower()))
.GroupBy(app => app.TRN)
.Select(x => x.OrderByDescending(y => y.CreatedAt).First())
.ToListAsync()
.ConfigureAwait(false);
The GroupBy expression should represent what you want to group by. In your case, the TRN. From there when we do the select, x represents each "group" which contains the Enumarable set of Applications that fall under each TRN. So we order those by the descending CreatedAt date to select the newest one using First.
Give that a shot. If it's not quite what you're after, consider adding an example set to your question and the desired output vs. what output / error this here produces.
I experience a similar issue where I find it interesting and stupid at the same time. Seems like EF team prohibits doing a WHERE before GROUP BY hence it does not work. I don't understand why you cannot do it but this seems the way it is which is forcing me to implement procedures instead of nicely build code.
LMK if you find a way.
Note: They have group by only when you first group then do where (where on the grouped elements of the complete table => does not make any sense to me)

How to deep clone/copy in EF Core

What I would like to do is duplicate/copy my School object and all of its children/associations in EF Core
I have something like the following:
var item = await _db.School
.AsNoTracking()
.Include(x => x.Students)
.Include(x => x.Teachers)
.Include(x => x.StudentClasses)
.ThenInclude(x => x.Class)
.FirstOrDefaultAsync(x => x.Id == schoolId);
I have been reading up on deep cloning and it seems that I should be able to do just add the entity...so pretty much the next line.
await _db.AddAsync(item);
Then EF should be smart enough to add that entity as a NEW entity. However, right off the bat I get a conflict that says "the id {schoolId} already exists" and will not insert. Even if I reset the Id of the new item I am trying to add, I still get conflicts with the Ids of the associations/children of the school iteam.
Is anyone familiar with this and what I might be doing wrong?
I had the same problem too, but in my case EF core was throwing exception "the id already exists".
Following the answer of #Irikos so I have created method which clones my objects.
Here's example
public class Parent
{
public int Id { get; set; }
public string SomeProperty { get; set; }
public virtual List<Child> Templates { get; set; }
public Parent Clone()
{
var output = new Parent() { SomeProperty = SomeProperty };
CloneTemplates(output);
return output;
}
private void CloneTemplates(Parent parentTo, Child oldTemplate = null, Child newTemplate = null)
{
//find old related Child elements
var templates = Templates.Where(c => c.Template == oldTemplate);
foreach (var template in templates)
{
var newEntity = new Child()
{
SomeChildProperty = template.SomeChildProperty,
Template = newTemplate,
Parent = parentTo
};
//find recursivly all related Child elements
CloneTemplates(parentTo, template, newEntity);
parentTo.Templates.Add(newEntity);
}
}
}
public class Child
{
public int Id { get; set; }
public int ParentId { get; set; }
public virtual Parent Parent { get; set; }
public int? TemplateId { get; set; }
public virtual Child Template { get; set; }
public string SomeChildProperty { get; set; }
}
Then I just call DbContext.Parents.Add(newEntity) and DbContext.SaveChanges()
That worked for me. Maybe this will be useful for someone.
I had the same problem, but in my case, ef core was smart enough save them as new entities even with existing id. However, before realising that, I just made a copy constructor for all the items, created a local task variable containing only the desired properties and returned the copy.
Remove certain properties from object upon query EF Core 2.1

Many to Many between IdentityUser and other table in a separate context

I am having difficulty creating a join table relationship between my Identity Framework IdentityContext(the IdentityUser) and one of my other tables Let's call it Entry. The problem is, Entry is in an entirely separate context doing it's own thing as well.
What is the proper way to associate these two? Where do I define the Join Table in fluent api?
Right now, I am getting the following error.
The key {'ApplicationUserId'} contains properties in shadow state and is referenced by a relationship from 'ApplicationUser.ApplicationUserEntries' to 'ApplicationUserEntry.ApplicationUser'. Configure a non-shadow principal key for this relationship.
These are how my tables are defined.
public class ApplicationUser : IdentityUser
{
...
public virtual List<ApplicationUserEntry> ApplicationUserEntries { get; set; }
}
public class Entry
{
public int Id { get; set; }
...
public virtual List<ApplicationUserEntry> ApplicationUserEntries { get; set; }
}
And the join table as follows.
public class ApplicationUserEntry
{
public int ApplicationUserId { get; set; }
public ApplicationUser ApplicationUser { get; set; }
public int EntryId { get; set; }
public Entry Entry { get; set; }
}
For the IdentityContext I have just some generic setup for other properties
var users = modelBuilder.Entity<ApplicationUser>();
users.Property(u => u.Name).IsRequired().HasMaxLength(65);
users.Property(u => u.FirstName).HasMaxLength(32);
users.Property(u => u.LastName).HasMaxLength(32);
And in my GoalsContext I have some general setup for other unrelated stuff, and the join table defined for ApplicationUserEntry
// Entry Configuration
var entries = modelBuilder.Entity<Entry>();
entries.HasKey(e => e.Id);
entries.HasAlternateKey(e => new { e.MilestoneId, e.CategoryId, e.MetricId });
entries.Property(e => e.Value).IsRequired();
entries.Property(e => e.Locked).IsRequired().HasDefaultValue(false);
entries.ToTable("GoalsEntries");
// ApplicationUserEntry Join Table
modelBuilder.Entity<ApplicationUserEntry>()
.ToTable("GoalsApplicationUserEntry")
.HasKey(se => new { se.ApplicationUserId, se.EntryId });
modelBuilder.Entity<ApplicationUserEntry>()
.HasOne(se => se.ApplicationUser)
.WithMany(s => s.ApplicationUserEntries)
.HasForeignKey(se => se.ApplicationUserId);
modelBuilder.Entity<ApplicationUserEntry>()
.HasOne(se => se.Entry)
.WithMany(e => e.ApplicationUserEntries)
.HasForeignKey(se => se.EntryId);
Now I'm sure I'm obviously missing something but I can't figure out what. I've never attempted to create a many to many relationship between two tables that are defined in two different contexts... and not even sure if that's wise or not to do.
My ultimate goal is to be able to associate owners with Entry records, so they can only be modified by the owners, which I verify with Identity Framework.
Ideally I would just prefer a unidirectional relationship, so I can find the owner from the Entry, but I'm not intending to get a list of Entry by looking at the IdentityUser

Prevent AutoMapper Projections From Forcing Related Data Loading

Is there a way to configure AutoMapper to adhere to the .Include style loading instructions for Entity Framework?
I've disabled lazy loading for my context, and I want to conditionally load related data for particular entities. Ideally, I'd like to do this by using an include syntax. Something like:
if(loadAddreses)
{
query = query.Include(e => e.Addresses);
}
if(loadEmails)
{
query = query.Include(e => e.Emails);
}
The problem is, AutoMapper is seeing that the model I'm projecting to includes Addresses and E-mails, and is generating SQL that loads all that data regardless of what I've asked EF to include. In other words:
var model = query.Project.To<MyModel>();
If MyModel has an Addresses collection, it will load addresses, regardless of my Include statements.
Short of changing my model so that I have one that doesn't have an Addresses or Emails property, is there a way to fix this? I suppose I could change my mapping, but mappings are usually static and don't change after they're initially created.
This was kind of tricky to tease out, but see how this works for you. Note that I'm using version 3.3.0-ci1027 of AutoMapper (at the time of writing this was a pre-release).
Assume my data model looks like this:
public class Address
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int AddressId { get; set; }
public string Text { get; set; }
}
public class Email
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int EmailId { get; set; }
public string Text { get; set; }
}
public class User
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int UserId { get; set; }
public virtual ICollection<Address> Addresses { get; set; }
public virtual ICollection<Email> Emails { get; set; }
public User()
{
this.Addresses = new List<Address>();
this.Emails = new List<Email>();
}
}
My view models are not specified but they just contain the same properties as the entities.
My mapping from User to UserViewModel looks like this:
Mapper.CreateMap<User, UserViewModel>()
.ForMember(x => x.Emails, opt => opt.ExplicitExpansion())
.ForMember(x => x.Addresses, opt => opt.ExplicitExpansion());
And my projection looks like this:
var viewModels = context.Set<User>().Project()
.To<UserViewModel>(new { }, u => u.Emails).ToList();
With that mapping and projection, only the Emails collection is loaded. The important parts to this are the opt => opt.ExplicitExpansion() call in the mapping - which prevents a navigation property being followed unless explicitly expanded during projection, and the overloaded To method. This overload allows you to specify parameters (which I've left as an empty object), and the members you wish to expand (in this case just the Emails).
The one thing I'm not sure of at this stage is the precise mechanism to extract the details from the Include statements so you can in turn pass them into the To method, but hopefully this gives you something to work with.