I'm attempting to query 5 parent records, then a summary of all child categories and counts:
context.Parent
.Take(5)
.GroupJoin(inner: context.Child,
outerKeySelector: parent => parent.Id,
innerKeySelector: child => child.ParentId,
resultSelector: (parent, children) => new SummaryResult
{
Id = parent.Id,
Name = parent.Name,
Children = parent.Children
.GroupBy(c => c.Category)
.Select(group => new ChildCategorySummary
{
Category = group.Key,
Count = group.Count()
})
});
This works as expected from LinkPad---I get one query for the first 5 Parent records then five queries summarizing each of the groups of children.
However, in EF7, I get this query:
exec sp_executesql N'SELECT [child].[Id], [child].[Category], [t].[Id]
FROM (
SELECT TOP(#__p_0) [s0].*
FROM [Parent] AS [s0]
) AS [t]
LEFT JOIN [Child] AS [child] ON [t].[Id] = [child].[ParentId]
ORDER BY [t].[Id]',N'#__p_0 int',#__p_0=5
and then this, five times:
SELECT [p].[Id], [p].[Category]
FROM [Child] AS [p]
I didn't expect the left join in the first query, which gives me all the child records.
Is this a problem with my query?
Partially figured this out. The child class has both Parent and ParentId:
public class Child
{
[Key]
public virtual Guid Id { get; set; }
[Required]
public virtual Parent Parent { get; set; }
public virtual Guid ParentId { get; set; }
}
So the selector should use the required field Parent, not 'ParentId`:
outerKeySelector: parent => parent,
innerKeySelector: child => child.parent
However, I now have a different error, which seems to be an EF Core bug:
System.ArgumentException : Property 'MyApp.Models.Parent Parent' is not defined for type 'MyApp.Models.Parent'
Parameter name: property
at System.Linq.Expressions.Expression.Property(Expression expression, PropertyInfo property)
at System.Linq.Expressions.Expression.MakeMemberAccess(Expression expression, MemberInfo member)
...
I think this is fixed in a prerelease version of EF Core 2.
Related
The following syntax when migrated to EF Core has the following error
InvalidOperationException: The LINQ expression 'DbSet()
.Join(
inner: DbSet(),
outerKeySelector: ij => ij.ImportDefinitionId,
innerKeySelector: id => id.ImportDefinitionId,
resultSelector: (ij, id) => new {
ij = ij,
id = id
})
.Join(
inner: DbSet(),
outerKeySelector: <>h__TransparentIdentifier0 => <>h__TransparentIdentifier0.id.ImportTypeId,
innerKeySelector: it => it.ImportTypeId,
resultSelector: (<>h__TransparentIdentifier0, it) => new {
<>h__TransparentIdentifier0 = <>h__TransparentIdentifier0,
it = it
})
.GroupJoin(
inner: DbSet(),
outerKeySelector: <>h__TransparentIdentifier1 => <>h__TransparentIdentifier1.<>h__TransparentIdentifier0.ij.ImportJobId,
innerKeySelector: ijp => ijp.ImportJobId,
resultSelector: (<>h__TransparentIdentifier1, ijpGroup) => new {
<>h__TransparentIdentifier1 = <>h__TransparentIdentifier1,
ijpGroup = ijpGroup
})' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly
by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList',
or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038
for more information.
(from ij in ImportJobs
join id in ImportDefinitions
on ij.ImportDefinitionId equals id.ImportDefinitionId
join it in ImportTypes
on id.ImportTypeId equals it.ImportTypeId
join ijp in ImportJobParameters
on ij.ImportJobId equals ijp.ImportJobId into ijpGroup
where ij.JobQueuedTimeUtc >= DateTime.Now.AddDays(-30)
orderby ij.JobQueuedTimeUtc descending
select
new
{
ImportDefinition = id,
ImportType = it,
LastImportJob = ij,
LastImportJobParameters = ijpGroup
}).ToList()
My attempt to change this is as follows
(from ij in ImportJobs
join id in ImportDefinitions
on ij.ImportDefinitionId equals id.ImportDefinitionId
join it in ImportTypes
on id.ImportTypeId equals it.ImportTypeId
from ijp in ImportJobParameters.Where(ijp => ij.ImportJobId == ijp.ImportJobId).DefaultIfEmpty()
where ij.JobQueuedTimeUtc >= DateTime.Now.AddDays(-60)
orderby ij.JobQueuedTimeUtc descending
select
new
{
ImportDefinition = id,
ImportType = it,
LastImportJob = ij,
LastImportJobParameter = ijp
}).ToList()
.GroupBy(i => new { i.ImportDefinition, i.ImportType, i.LastImportJob })
.Select(i => new { i.Key.ImportDefinition, i.Key.ImportType, i.Key.LastImportJob, LastImportJobParameters = i.Select(s => s.LastImportJobParameter) })
however this results in a IEnumerable of LastImportJobParameters having 1 item of null where previously there would be 0 items. Just wondering if there is an equivalent EF Core statement otherwise I will filter out once materialised.
** Classes simplified **
public class ImportJob
{
[Key]
public int? ImportJobId { get; set; }
[Required]
public Int16? ImportDefinitionId { get; set; }
[NotMapped]
public ImportDefinition ImportDefinition { get; set; }
public DateTime? JobQueuedTimeUtc { get; set; }
[NotMapped]
public List<ImportJobParameter> ImportJobParameters { get; set; }
}
public class ImportJobParameter
{
[Key]
public int? ImportJobParameterId { get; set; }
[Required]
public int? ImportJobId { get; set; }
[Required]
public short? ImportParameterId { get; set; }
public string ParameterName { get; set; }
public string ParameterValue { get; set; }
}
public class ImportDefinition
{
[Key]
public Int16? ImportDefinitionId
{
get;
set;
}
[Required]
[StringLength(255)]
public string Name
{
get;
set;
}
public ImportType ImportType
{
get;
set;
}
[Required]
public Int16? ImportTypeId
{
get;
set;
}
}
public class ImportType
{
[Key]
public Int16? ImportTypeId
{
get; set;
}
[Required]
[StringLength(100)]
public string Name
{
get;
set;
}
}
Do not use GroupJoin for eager loading, only for LEFT JOIN. EF Core team won't to fix this limitation. Make subquery for retrieveing detail data:
var query =
from ij in ImportJobs
join id in ImportDefinitions
on ij.ImportDefinitionId equals id.ImportDefinitionId
join it in ImportTypes
on id.ImportTypeId equals it.ImportTypeId
where ij.JobQueuedTimeUtc >= DateTime.Now.AddDays(-30)
orderby ij.JobQueuedTimeUtc descending
select new
{
ImportDefinition = id,
ImportType = it,
LastImportJob = ij,
LastImportJobParameters = ImportJobParameters
.Where(ijp => ij.ImportJobId == ijp.ImportJobId)
.ToList()
};
The real and probably faster solution is to fix the entity model and eliminate joins. In fact, it looks like all you have to do is remove [NotMapped] and write :
var flattened=context.Jobs
.Where(job=>job.JobQueuedTimeUtc >= date)
.Select(job=>new {
ImportDefinition = job.ImportDefinition ,
ImportType = job.ImportDefinition.ImportType,
LastImportJob = job,
LastImportJobParameter = job.ImportJobParameters
}).ToList()
What the original query does is a GroupJoin, a client-side operation with no equivalent in SQL. EF executes a LEFT JOIN and then regroups the right-hand rows in memory to reconstruct the Parameters collection. This is an expensive client-side operation that can load far more into memory than programmers realize, especially if they try to filter the right hand objects. EF Core doesn't support this
GroupJoin doesn't translate to the server in many cases. It requires you to get all of the data from the server to do GroupJoin without a special selector (first query below). But if the selector is limiting data being selected then fetching all of the data from the server may cause performance issues (second query below). That's why EF Core doesn't translate GroupJoin.
If the right-hand was an execution log with eg 10K executions per job, executing a GroupJoin to get the last 10 would result in all logs getting loaded and sorted in memory only for 99.9% of them to get rejected.
What the second query does is emulate a GroupJoin, by executing a LEFT JOIN, then grouping the objects in memory. Since this is a LEFT JOIN, nulls are expected on the right hand.
To get the result you want you'll have to filter the parameters, and then convert them to a list or array. Otherwise, every time you try to access LastImportJobParameters the LINQ subquery would run again :
.Select(i => new {
i.Key.ImportDefinition,
i.Key.ImportType,
i.Key.LastImportJob,
LastImportJobParameters = i.Where(s.LastImportJobParameter!=null)
.Select(s => s.LastImportJobParameter)
.ToList() })
Using EF 2.0 Core, code first, I have the following entity which defines a self-referencing table:
class EntityX
{
public int EntityXId { get; set; }
public string Name { get; set; }
public int? ParentId { get; set; }
//navigation properties
public EntityX Parent { get; set; }
public ICollection<EntityX> Children { get; set; }
}
I want to retrieve all EntityX objects and their children in the form of a 'tree'
I can do that using:
var entities = context.EntityX
.Include(p => p.Parent)
.Include(p => p.Children)
.Where(p => p.Parent == null);
When I call entities.ToList() this gets me what I want: a list of parent entities with their children edit only 'first' generation children. When I omit the Where() clause, I get all entities and their children.
I do not understand why the Where() clause works. Objects that are part of the Children collection have a Parent. Why are they not omitted?
Edit: my question was answered but please be aware that I was wrong in my perception of how Include() works.
LINQ applies Where condition only to the objects in the collection being queried. Whatever else you choose to load with the Include method is not subject to the same condition. In fact, EF provides no direct way of restricting the associated entities (see this Q&A).
That is how the top-level query brings you what you expected. EF retrieves the children recursively via a separate RDBMS query using the parent ID to get all its children. The restriction from the Where method does not make it to the child query.
I am creating an Entity Framework 7 project to replace an Entity Framework 6 project.
I have an Item entity which belongs to a country. I then have a linq query that gets the count by country. Here is the query.
var results = allItems
.GroupBy(g => g.Country)
.ToDictionary(s => s.Key, s => s.Count());
This works with EF6 but throws an exception with EF 7 (the exception is at the bottom).
This is my entity:
public class Item
{
[Key]
public int id { get; set; }
[Required]
public int Countryid { get; set; }
[ForeignKey("Countryid")]
public virtual Country Country { get; set; }
}
In EF 7, with the debugger I see that the Country is null (that's the navigation property) but I do have the countryid. In EF 6, I have an object for the navigation property. In addition, I have unit tests using Moq and they work (the show the nav property).
I tried to add an include but I should not need that. I didn't need it in EF 6 or with the Mock.
Using include gives this:
var results = allItems
.Include(i => i.Country)
.GroupBy(g => g.Country)
.ToDictionary(s => s.Key, s => s.Count());
I get the same error.
Here is the error:
Expression of type
'System.Func2[Microsoft.Data.Entity.Query.EntityQueryModelVisitor+TransparentIdentifier2[Microsoft.Data.Entity.Query.EntityQueryModelVisitor+TransparentIdentifier2[FMS.DAL.Entities.ActionItem,Microsoft.Data.Entity.Storage.ValueBuffer],Microsoft.Data.Entity.Storage.ValueBuffer],FMS.DAL.Entities.MemberCountry]'
cannot be used for parameter of type
'System.Func2[FMS.DAL.Entities.ActionItem,FMS.DAL.Entities.MemberCountry]'
of method
'System.Collections.Generic.IEnumerable1[System.Linq.IGrouping2[FMS.DAL.Entities.MemberCountry,FMS.DAL.Entities.ActionItem]]
_GroupBy[ActionItem,MemberCountry,ActionItem](System.Collections.Generic.IEnumerable1[FMS.DAL.Entities.ActionItem],
System.Func2[FMS.DAL.Entities.ActionItem,FMS.DAL.Entities.MemberCountry],
System.Func`2[FMS.DAL.Entities.ActionItem,FMS.DAL.Entities.ActionItem])'
Currently GroupBy is not implemented in EF7 the status of features can be found on the road map page here. https://github.com/aspnet/EntityFramework/wiki/Roadmap
A work around would be:
context.Countries.Select( x => new
{
x.Id,
Items = x.Items.Count
} ).ToDictionary( x => x.Id, x => x.Items );
public class Country
{
public int Id { get; set; }
//Add this property
public virtual ICollection<Item> Items { get; set; }
}
//Generated SQL
SELECT (
SELECT COUNT(*)
FROM [Item] AS [i]
WHERE [x].[Id] = [i].[Countryid]
), [x].[Id]
FROM [Country] AS [x]
This would require adding a Items property to country but would allow you to achieve what you are after all in Linq. You could alse go for writing the query in sql and executing with EF but may not be the best option.
In Entity Framework, three entities have 1 to many relationships as grandparent, children and grandchildren.
How do you obtain an the grandparent object from a grandchild's primary key?
Thank you,
Newby to EF
You could do something like the following assuming the grandchildren have a reference (as a DbSet) to their parents, and those have a reference to their parents in return:
myGrandChildren.SelectMany(x => x.Parents).SelectMany(x => x.Parents);
What SelectMany does here is for each grandchildren select all its parents and return them as a single list (instead of as a list of lists - it concats them).
If you just have one grand child - you only need one SelectMany:
grandChild.Parents.SelectMany(x => x.Parents);
Query to context will be (grandchildId it is grandchild's primary key):
var grandparent = context
.Set<GrandParent>()
.SingleOrDefault(gp => gp.Children.Any(c => c.Children.Any(gc => gc.Id == grandchildId)));
if I understand, your classes looks like this:
class GrandParent
{
...
public List<Child> Children {get; set;}
}
class Child
{
...
public List<GrandChild> Children {get; set;}
}
class GrandChild
{
...
public int Id {get; set;}
}
I have these classes:
public class Parent
{
public int Id { get; set; }
public List<Child> Children { get; set; }
}
public class Child
{
public int Id { get; set; }
public ChildInfo ChildInfo { get; set; }
}
public class ChildInfo
{
public int Age { get; set; }
...other properties...
}
So given a collection of Parents, I need to return a Parent with a specific Id, but only if it does not have a Child with a ChildInfo that has a specific age.
I think I am close. Here is what I have so far:
var childQuery = Query<Child>.NE(c => c.ChildInfo.Age, 5);
var finalresult = collection.Find(Query.And(Query<Parent>.EQ(p => p.Id, 3245),
Query<Parent>.ElemMatch( p => p.Children, builder => childQuery)));
However, I get these results:
When there is no Parent 3245, the query returns nothing (correct).
When Parent 3245 has no children, the query returns nothing (wrong).
When Parent 3245 has one child age 3 the query returns the Parent (correct).
When Parent 3245 has one child age 3 and one child age 5 the query returns the Parent (wrong).
When Parent 3245 has one child age 3 and one child age 7 the query returns the Parent (correct).
It appears as if the first part of the query (Parent.Id) works. But the second half seems to return the Parent all the time except when the list is empty.
To solve the first problem, it seems that you'll have to use a Query.Or to surround the Query.ElemMatch and an additional Query.Exists to test for case when there are no child elements.
Regarding the second problem, it seems that Mongo kind of unwinds the children array and if it finds a document that matches your query, it marks the query as true. So in the case of ages 3 and 5, the fact that it finds 3, which is Not Equal to 5, it satisfies the query and you get a wrong result.
I think, but I'm not 100% sure it's the best way, it'd be better to use Query.Not(Query<Child>.EQ(c => c.ChildInfo.Age, 5)).