EF Core 3 GroupBy multiple columns Count Throws with extensions but linq works - entity-framework

Here is the one that throws full exception:
var duplicateCountOriginal = _db.TableName
.GroupBy(g => new {g.ColumnA, g.ColumnB, g.ColumnC})
.Count(g => g.Count() > 1);
Exception:
System.ArgumentException: Expression of type 'System.Func2[System.Linq.IGrouping2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer],Microsoft.EntityFrameworkCore.Storage.ValueBuffer]' cannot be used for parameter of type 'System.Func2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer]' of method 'System.Collections.Generic.IEnumerable1[Microsoft.EntityFrameworkCore.Storage.ValueBuffer] Select[ValueBuffer,ValueBuffer](System.Collections.Generic.IEnumerable1[Microsoft.EntityFrameworkCore.Storage.ValueBuffer], System.Func2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer])' (Parameter 'arg1')
But the same thing works when it is written as linq (I prefer extensions)
var duplicateCount =
from a in _db.TableName
group a by new {a.ColumnA, a.ColumnB, a.ColumnC}
into g
where g.Count() > 1
select g.Key;
duplicateCount.Count()
I am unable to understand why one works or the other doesn't.
Also if I change the first one a little bit based on EF Core 3 changes like the following
var duplicateCountOriginal = _db.TableName
.GroupBy(g => new {g.ColumnA, g.ColumnB, g.ColumnC})
.AsEnumerable()
.Count(g => g.Count() > 1);
I get the following exception:
System.InvalidOperationException: Client projection contains reference to constant expression of 'Microsoft.EntityFrameworkCore.Metadata.IPropertyBase' which is being passed as argument to method 'TryReadValue'. This could potentially cause memory leak. Consider assigning this constant to local variable and using the variable in the query instead. See https://go.microsoft.com/fwlink/?linkid=2103067 for more information.
According to me, the link given by ms has no meaning to the whatever problem here is.
Please LMK if there is any logical explanation.

There is no logical explanation. Just EF Core query translation is still far from perfect and have many defects/bugs/unhandled cases.
In this particular the problem is not the query syntax or method syntax (what you call extensions), but the lack of Select after GroupBy. If you rewrite the method syntax query similar to the one using query syntax, i.e. add .Where, .Select and then Count:
var duplicateCount = _db.TableName
.GroupBy(g => new {g.ColumnA, g.ColumnB, g.ColumnC})
.Where(g => g.Count() > 1)
.Select(g => g.Key)
.Count();
then it will be translated and executed successfully.

Related

Reusable Functions in Linq To Entites

I have 2 reusable functions that return lists. If the code from these functions is written directly into the linq to entities query all is good. However, separating these out into functions causes an error as it cannot be translated to a stored expression. I'm sure there must be a way of doing this though. Any ideas how to solve this problems. Ideally I want the reusable parts to be used outside of linq to entity queries also.
var activityBands = DbContext.ActivityBand
.OrderBy(x => x.ActivityBandDescription)
.Where(x => x.Active && x.ClientAccountId == clientAccountId)
.Select(x => new ActivityBandDdl
{
Name = x.ActivityBandDescription,
ActivityBandId = x.ActivityBandId,
ApplyAwr = x.ApplyAwr,
AssignmentLineTimeTypeIds = TimeTypesForActivityBand(x.DailyRate) ,
AssignmentTypeIds = AssTypesForActivityBand(x.StagePayment)
}).ToList();
public static Func<bool, List<int>> TimeTypesForActivityBand =
(dailyRate) => (new int[] { 1, 2, 3, 4 }).Where(t =>
((t != 1 && t != 2) || !dailyRate) //No Timed or NTS for daily rates
).ToList();
public static Func<bool, List<int>> AssTypesForActivityBand =
(stagePayment) => (new int[] { 2,3,4,5,6,7,8,9,10 }).Where(t =>
( t!=2 || !stagePayment) //Only stage pay ass have stage pay activity bands
).ToList();
TL;DR;
suggested solution for your problem:
get LinqKit ... have a look at it's Expand() function (in the docs: combining expressions)
https://github.com/scottksmith95/LINQKit#combining-expressions
the details:
the problem boils down to: what is the difference between the queries in both cases...
A LINQ query works with an expression tree ... in other words: just because the code you typed directly into the query and the code you typed into the static Func<...> looks the same, in fact, is the same, the resulting expression trees in both cases are not the same
what is an expression tree?
imagine a simpler query like ... someIQueryable.Where(x => x.a==1 && x.b=="foo")
the lamda that is passed to Where(...) can be seen as a straigt forward c# lambda expression that can be used as a Func
and it can also be seen as a Expression>
the later is a tree of objects that form the expression, in other words a description about the way, in that the passed in parameter can be evaluated to a bool without actually having the executable code, but just the description about what to do ... take the member a from the parameter, equality-compare it to the constant 1 ... take the boolean AND of the result with the result of: take the member b from the parameter, equality-compare it to the constant "foo" ... return the result of the boolean AND
why all of this?
it's the way LINQ works ... LINQ to entiteis takes the expression tree, looks at all the operations, finds the corresponding SQL, and builds an SQL statement which is executed in the end ...
when you have your extracted Func<...> there is a little problem ... at some point in the resulting expression tree there is something like ... take the parameter x and CALL the static Func ... the expression tree does no longer contain a description of whats happening inside the Func, but just a call to that ... as long as you want to compile that to a .net runtime executable function, it's all fun and games ... but when you try to parse it into SQL, LINQ to entiteis does not know a corresponding SQL for "call some c# function" ... therefore it tells you that this part of the expression tree can not be converted into a store expression

How to perform subtraction between DateTime in a Linq to Sql Select() method

I've a linq query :
var NextWeekDate = DateTime.Now.AddDays(7);
var BirthData = db.EmployeeMasterAs
.Where(x => x.Emp_BirthDate >= CurrentDate)
.Select(x => new { x.Emp_BirthDate,
DaysRemaining = (NextWeekDate - x.Emp_BirthDate).Value.TotalDays
})
.OrderBy(x => x.Emp_BirthDate)
.ToList();
Note the element in Select() method, DaysRemaining = (NextWeekDate - x.Emp_BirthDate).Value.TotalDays, I want to get the difference between dates in days and save that to DaysRemaining, this query is not working and the exception that it throws :
DbArithmeticExpression arguments must have a numeric common type.
I don't know exactly how to achieve this, Any Ideas?
I think that LinqToSql does not support DateTime.Subtract. But you can use the preexisting function SqlMethods.DateDiffDay (https://msdn.microsoft.com/en-us/library/bb468730(v=vs.110).aspx)
Edit: SqlMethods is only intended for SQL Server. DbFunctions is the helper class to pick regardless of the database type.
Yep, Got it, So here is how it goes :
Performing arithmetic operations with Dates and Times is not supported in EF. EntityFramework cannot convert your code to SQL. Instead of that you can use DbFunctions.DiffDays( startExpression, endExpression) method:
db.EmployeeMasterAs
.Where(x => x.Emp_BirthDate >= CurrentDate)
.Select(x => new { x.Emp_BirthDate,
DaysRemaining = DbFunctions.DiffDays(x.Emp_BirthDate,NextWeek)
})
.OrderBy(x => x.Emp_BirthDate)
.ToList();
Furthermore : DbFunctions Functions methods are called canonical functions. And these are a set of functions, which are supported by all Entity Framework providers. These canonical functions will be translated to the corresponding data source functionality for the provider. Canonical functions are the preferred way to access functionality outside the core language, because they keep the queries portable.

LINQ to Entities Contains Query

I'm trying to use Contains() in a simple query but it is failing, with the error :
Unable to create a constant value of type 'NewsletterApp.Models.NewsletterUser'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Edit The intellisense actually directs me to use the NewsletterUser in Contains() -'(NewsletterUser item)'
I read that there were issues searching for an object using Contains() using EF with .NET 3.5, but I am using EF4.2 (also tried 4.1) with .NET 4.0 .
Code is below:
var db = new MyContext();
var newsletterUser = db.NewsletterUsers.Find(UserID);
var subscriberList = db.Subscribers
.Where(x => x.NewsletterList.ListOwner.NewsletterUsers.Contains(newsletterUser))
.ToList();
I suspect you want this
var db = new MyContext();
var newsletterUser = db.NewsletterUsers.Find(UserID);
var subscriberList = db.Subscribers
.Where(x => x.NewsletterList.ListOwner.NewsletterUsers
.Any(y => y.UserId == newsletterUser.UserId))
.ToList();
Any() checks for the existence of a item that fulfils the criteria specified in the lambda: "y => y.UserId == newsletterUser.UserId".
The exception you were getting: "Only primitive types ('such as Int32, String, and Guid') are supported in this context" is due to limitations set by LINQ to Entities. LINQ to Entities needs to resolve your query in a way that it can express to the database, and it can't do that with the Contains() method with anything other than a primitive type.
The thing is, the code you posted does run fine if you run it against an in memory collection (LINQ to Objects) - that's why it isn't flagged by the compiler.
Your query is wrong. Instead of comparing a property or field, you are comparing the whole object or entity, which can't be done in this manner.
Try the following code and it will work
var db = new MyContext();
var newsletterUser = db.NewsletterUsers.Find(UserID);
var subscriberList = db.Subscribers
.Where(x => x.NewsletterList.ListOwner.NewsletterUsers.UserId.Contains(newsletterUser.UserID))
.ToList();

Entity Framework - Union causes "Unable to create a constant value of type.."

To select all Schedulings which are active, I have the following code:
var allSchedulesOnALine = CurrentUser.Lines.SelectMany(o => o.Scheduling).Where(o => o.Active);
var allSchedulesUnscheduled = Entities.SchedulingSet
.Where(o => o.Line == null && o.Site.Id == CurrentUser.Site.Id &&
o.Factory == CurrentUser.Factory && o.Active);
IEnumerable<Scheduling> allSchedules = allSchedulesUnscheduled.Union(allSchedulesOnALine);
foreach(Scheduling schedule in allSchedules.OrderBy(o => o.Ordering))
{
//Do Stuff
}
(Factory is an int)
When I run this code, I get this cryptic error on the foreach line:
Unable to create a constant value of type 'System.Collections.Generic.IEnumerable`1'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Strangely enough, I can enumerate both allSchedulesOnALine and allSchedulesUnscheduled separately. Even stranger, if I reorder the union:
IEnumerable<Scheduling> allSchedules = allSchedulesOnALine.Union(allSchedulesUnscheduled);
It works fine!
Does anyone have any idea why this would happen? Am I missing something crucial, or is this a bug?
I should mention I am using Entity Framework 3.5. EF4 is not an option for us currently - it is beyond my control :\
You're calling two different methods with your "reordering".
You don't show the types of allSchedulesOnALine or allSchedulesUnscheduled, but I'm betting allSchedulesOnALine is of type IEnumerable<Schedule> and allSchedulesUnscheduled is of type IQueryable<Schedule>.
So when you call Queryable.Union, you're asking the EF to translate the expression into SQL. But the argument you pass is of type IEnumerable<Schedule>, and it can't translate that into a query.
On the other hand, when you call Enumerable.Union, you're asking LINQ to Objects to do the whole thing in memory, which works fine, albeit perhaps slower.
So the reason the behavior is different is that you're calling two completely different methods, which do different things, but happen to have the same name. No, it's not a bug.

Generated SQL with PredicateBuilder, LINQPad and operator ANY

I previously asked a question about chaining conditions in Linq To Entities.
Now I use LinqKit and everything works fine.
I want to see the generated SQL and after reading this answer, I use LinqPad.
This is my statement:
var predProduct = PredicateBuilder.True<Product>();
var predColorLanguage = PredicateBuilder.True<ColorLanguage>();
predProduct = predProduct.And(p => p.IsComplete);
predColorLanguage = predColorLanguage.And(c => c.IdColorEntity.Products.AsQueryable().Any(expr));
ColorLanguages.Where(predColorLanguage).Dump();
The code works in VS2008, compile and produce the correct result set, but in LinqPad, I've the following error:
NotSupportedException: The overload query operator 'Any' used is not Supported.
How can I see the generated SQL if LINQPad fails?
EDIT
If I write
var predColorLanguage = PredicateBuilder.True<ColorLanguage>();
predColorLanguage = predColorLanguage.And(c => c.IdColorEntity.Products.Any((p => p.IsComplete));
ColorLanguages.Where(predColorLanguage).Dump();
works... WTF?
As you're using LINQKit, you can make this work by calling Compile() on the expression that feeds the EntitySet, and then calling AsExpandable() on the main query:
var predProduct = PredicateBuilder.True<Product>();
var predColorLanguage = PredicateBuilder.True<ColorLanguage>();
predProduct = predProduct.And(p => p.IsComplete);
predColorLanguage = predColorLanguage.And (
c => c.IdColorEntity.Products.Any(predProduct.Compile()));
ColorLanguages.AsExpandable().Where(predColorLanguage).Dump();
As explained in the LINQKit article, the Compile method never actually runs: AsExpandable strips it out and modifies the expression tree so that it works with LINQ to SQL.