Passing an aggregate select expression to Dynamic Linq's GroupBy - entity-framework

I have simplified the following example from my code and hoping there's no obvious compilation errors because of it. Lets say I have the following entities (not what i actually have, please assume I have no EF or schema issues, this is just for example):
public class Company
{
public string GroupProperty {get;set;}
public virtual ICollection<PricingForm> PricingForms {get;set;}
}
public class PricingForm
{
public decimal Cost {get;set;}
}
And I want to query like so:
IQueryable DynamicGrouping<T>(IQueryable<T> query)
{
Expression<Func<Company, decimal?>> exp = c => c.PricingForms.Sum(fr => fr.Cost);
string selector = "new (it.Key as Key, #0(it) as Value)";
IQueryable grouping = query.GroupBy("it.GroupProperty", "it").Select(selector, exp);
return grouping;
}
I get the following error when calling the groupby/select line:
System.Linq.Dynamic.ParseException: 'Argument list incompatible with lambda expression'
What type is "it" when grouped? I have tried using other expressions that assume it is an IGrouping<string, Company>, or a IQueryable<Company>, same error. I've tried just selecting "Cost" and moving the Sum() aggregate into the selector string (i.e. Sum(#0(it)) as Value) and always seem to get the same error.
I eventually tried something along the lines of:
Expression<Func<IEnumerable<Company>, decimal?>> exp = l => l.SelectMany(c => c.PricingForms).Sum(fr => fr.Cost);
However this one, I get farther but when attempting to iterate through the results I got a different error.
The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.
So, with this dynamic grouping and injecting my own select expression, what should I assume the datatype of 'it' is? Will this even work?

The type of it is IGrouping<TKey, TElement>, where TKey is dynamic based on the keySelector result type, and TElement is the element type of the input IQueryable. Luckily IGrouping<TKey, TElement> inherits (is a) IEnumerable<TElement>, so as soon as you know the input element type, you can safely base selector on IEnumerable<TElement>.
In other words, the last attempt based on Expression<Func<IEnumerable<Company>, decimal?>> is correct.
The new error you are getting is because #0(it) generates Expression.Invoke call which is not supported by EF. The easiest way to fix that is to use LINQKit Expand method:
Expression<Func<Company, decimal?>> exp = c => c.PricingForms.Sum(fr => fr.Cost);
string selector = "new (it.Key as Key, #0(it) as Value)";
IQueryable grouping = query.GroupBy("it.GroupProperty", "it").Select(selector, exp);
// This would fix the EF invocation expression error
grouping = grouping.Provider.CreateQuery(grouping.Expression.Expand());
return grouping;

Related

Reusable Functions in Linq To Entites

I have 2 reusable functions that return lists. If the code from these functions is written directly into the linq to entities query all is good. However, separating these out into functions causes an error as it cannot be translated to a stored expression. I'm sure there must be a way of doing this though. Any ideas how to solve this problems. Ideally I want the reusable parts to be used outside of linq to entity queries also.
var activityBands = DbContext.ActivityBand
.OrderBy(x => x.ActivityBandDescription)
.Where(x => x.Active && x.ClientAccountId == clientAccountId)
.Select(x => new ActivityBandDdl
{
Name = x.ActivityBandDescription,
ActivityBandId = x.ActivityBandId,
ApplyAwr = x.ApplyAwr,
AssignmentLineTimeTypeIds = TimeTypesForActivityBand(x.DailyRate) ,
AssignmentTypeIds = AssTypesForActivityBand(x.StagePayment)
}).ToList();
public static Func<bool, List<int>> TimeTypesForActivityBand =
(dailyRate) => (new int[] { 1, 2, 3, 4 }).Where(t =>
((t != 1 && t != 2) || !dailyRate) //No Timed or NTS for daily rates
).ToList();
public static Func<bool, List<int>> AssTypesForActivityBand =
(stagePayment) => (new int[] { 2,3,4,5,6,7,8,9,10 }).Where(t =>
( t!=2 || !stagePayment) //Only stage pay ass have stage pay activity bands
).ToList();
TL;DR;
suggested solution for your problem:
get LinqKit ... have a look at it's Expand() function (in the docs: combining expressions)
https://github.com/scottksmith95/LINQKit#combining-expressions
the details:
the problem boils down to: what is the difference between the queries in both cases...
A LINQ query works with an expression tree ... in other words: just because the code you typed directly into the query and the code you typed into the static Func<...> looks the same, in fact, is the same, the resulting expression trees in both cases are not the same
what is an expression tree?
imagine a simpler query like ... someIQueryable.Where(x => x.a==1 && x.b=="foo")
the lamda that is passed to Where(...) can be seen as a straigt forward c# lambda expression that can be used as a Func
and it can also be seen as a Expression>
the later is a tree of objects that form the expression, in other words a description about the way, in that the passed in parameter can be evaluated to a bool without actually having the executable code, but just the description about what to do ... take the member a from the parameter, equality-compare it to the constant 1 ... take the boolean AND of the result with the result of: take the member b from the parameter, equality-compare it to the constant "foo" ... return the result of the boolean AND
why all of this?
it's the way LINQ works ... LINQ to entiteis takes the expression tree, looks at all the operations, finds the corresponding SQL, and builds an SQL statement which is executed in the end ...
when you have your extracted Func<...> there is a little problem ... at some point in the resulting expression tree there is something like ... take the parameter x and CALL the static Func ... the expression tree does no longer contain a description of whats happening inside the Func, but just a call to that ... as long as you want to compile that to a .net runtime executable function, it's all fun and games ... but when you try to parse it into SQL, LINQ to entiteis does not know a corresponding SQL for "call some c# function" ... therefore it tells you that this part of the expression tree can not be converted into a store expression

Why can't I create a callback for the List Find method in Moq?

I created an extension method that lets me treat a List as DbSet for testing purposes (actually, I found this idea in another question here on stack overflow, and it's been fairly useful). Coded as follows:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var queryable = sourceList.AsQueryable();
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.As<IQueryable<T>>().Setup(m => m.Provider).Returns(queryable.Provider);
mockDbSet.As<IQueryable<T>>().Setup(m => m.Expression).Returns(queryable.Expression);
mockDbSet.As<IQueryable<T>>().Setup(m => m.ElementType).Returns(queryable.ElementType);
mockDbSet.As<IQueryable<T>>().Setup(m => m.GetEnumerator()).Returns(queryable.GetEnumerator());
mockDbSet.Setup(d => d.Add(It.IsAny<T>())).Callback<T>(sourceList.Add);
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback(sourceList.Find);
return mockDbSet.Object;
}
I had been using Add for awhile, and that works perfectly. However, when I try to add the callback for Find, I get a compiler error saying that it can't convert a method group to an action. Why is sourceList.Add an Action, but sourceList.Find is a method group?
I'll admit I'm not particularly familiar with C# delegates, so it's likely I'm missing something very obvious. Thanks in advance.
The reason Add works is because the List<T>.Add method group contains a single method which takes a single argument of type T and returns void. This method has the same signature as an Action<T> which is one of the overloads of the Callback method (the one with a single generic type parameter, Callback<T>), therefore the List<T>.Add method group can be converted to an Action<T>.
With Find, you are trying to call the Callback method (as opposed to Callback<T>) which expects an Action parameter (as opposed to Action<T>). The difference here is that an Action does not take any parameters, but an Action<T> takes a single parameter of type T. The List<T>.Find method group cannot be converted to an Action because all the Find methods (there is only one anyway) take input parameters.
The following will compile:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback<Predicate<T>>(t => sourceList.Find(t));
return mockDbSet.Object;
}
Note that I have called .Callback<Predicate<T>> because the List<T>.Find method expects and argument of type Predicate. Also note I have had to write t => sourceList.Find(t) instead of sourceList.Find because Find returns a value (which means it doesn't match the signature of Action<Predicate<T>>). By writing it as a lambda expression the return value will be thrown away.
Note that although this compiles it will not actually work because the DbSet.Find method actually takes an object[] for it's parameter, not a Predicate<T>, so you will likely have to do something like this:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback<object[]>(keyValues => sourceList.Find(keyValues.Contains));
return mockDbSet.Object;
}
This last point has more to do with how to use the Moq library that how to use method groups, delegates and lambdas - there is all sorts of syntactic sugar going on with this line which is hiding what is actually relevant to the compiler and what isn't.

Conditionally add query operator on properties defined in non-EDM base type, if inheriting

(C# code at end of question)
I have the following inheritance chain:
PreRecord <- Record <- (multiple entity types)
Record declares a property ID As Integer.
PreRecord and Record are not EDM types, and do not correspond to tables in the database.
I have a method that takes a generic parameter constrained to PreRecord and builds an EF query with the generic parameter as the element type. At runtime, in the event that T inherits not just from PreRecord but from Record, I would like add an OrderBy operator on ID:
'Sample 1
Function GetQuery(Of T As PreRecord)(row As T) As IQueryable(Of T)
Dim dcx = New MyDbContext
Dim qry = dcx.Set(Of T).AsQueryable
If TypeOf row Is RecordBase Then
'modify/rewrite the query here
End If
Return qry
End Function
If the parameter constraint were to Record I would have no problem applying query operators that use the ID property. How can I make use of a different (narrowing) generic constraint mid-method and still return an IQueryable(Of T) / IQueryable<T>, where T is still constrained to PreRecord?
I tried this:
'Sample 2
qry = dcx.Set(Of T).Cast(Of Record).OrderBy(Function(x) x.ID).Cast(Of PreRecord)()
which doesn't work:
LINQ to Entities only supports casting EDM primitive or enumeration types.
C# equivalent:
//Sample 1
public IQueryable<T> GetQuery<T>(T row) where T : PreRecord {
var dcx = new MyDbContext();
var qry = dcx.Set<T>.AsQueryable();
if (row is RecordBase) {
//modify/rewrite the query here
}
return qry;
}
and this doesn't work:
//Sample 2
qry = dcx.Set<T>.Cast<Record>.OrderBy(x => x.ID).Cast<PreRecord>()
The problem here is the fact that compiler checks queries already at compile time and PreRecord class does not have ID property. We cannot use simply Cast, because when it is used in definition of the query parser tries to convert it to sql - but there is no such thing that exists in sql. Sql supports only conversion of one column type to another - so on the .NET side it is supported only for primitive and enum types. To overcome compiler query checking we may use Expression class to build dynamic queries:
ParameterExpression e = Expression.Parameter(typeof(Record));
Expression body = Expression.Property(e, "ID");
Expression<Func<PreRecord, int>> orderByExpression = Expression.Lambda<Func<PreRecord, int>>(body, e);
And use your expression in the query:
qry = dcx.Set<T>.OrderBy(orderByExpression);
This way your linq query will not be validated during compile time but execution time. Here I assumed ID is of type int, if the type is different change it accordingly.

Why is EqualityComparer<string>.Default not working?

var dic = context.Treasure.Include("TreasureShare")
.Where(t => t.TreasureShare.IsShared && t.TreasureShare.EvaluationContent.Contains(keyword))
.ToDictionary(t => t.ProductUrl, t => t.ProductId, EqualityComparer<string>.Default);
I got an error:
An item with the same key has already been added.
So why the equalitycomparer not work, and how to use a equalitycomparer to get different records while querying to database.
Updated:
I know IEqualityComparer can only be executed locally, but I didn't get an error like:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[Panli.Service.Share.DataAccess.DbData.Treasure] Distinct[Treasure](System.Linq.IQueryable1[Panli.Service.Share.DataAccess.DbData.Treasure], System.Collections.Generic.IEqualityComparer`1[Panli.Service.Share.DataAccess.DbData.Treasure])' method, and this method cannot be translated into a store expression.
except I change the codes to below:
dic = context.Treasure.Include("TreasureShare")
.Where(t => t.TreasureShare.IsShared && t.TreasureShare.EvaluationContent.Contains(theme))
.Distinct(new TreasureEqualityComparer()).ToDictionary(t => t.ProductUrl, t => t.ProductId);
This is my TreasureEqualityComparer:
public class TreasureEqualityComparer : EqualityComparer<Treasure>
{
public override bool Equals(Treasure x, Treasure y)
{
return x.ProductUrl.ToLower() == y.ProductUrl.ToLower();
}
public override int GetHashCode(Treasure obj)
{
return obj.ProductUrl.ToLower().GetHashCode();
}
}
So why not throw an exception just like the Distinct() when I use ToDictionary(..) which has an IEqualityComparer param ? Anyone can explain this ?
So why not throw an exception?
The ToDictionary part is executed in memory. This is apparent when you investigate the SQL that is executed: nothing that shows any preparation for a conversion to Dictionary.
The query expression with Distinct on the other hand is translated into SQL as a whole (except it isn't because it fails). EF tries to let the database do the hard work of returning distinct values, but of course a comparer can't be translated into SQL, so this overload of Distinct() is not supported.
As for the duplicate key: apparently there are duplicate URL's (ignoring case). Maybe you should use group by.
Dictionary key has to be unique. In this case you are using ProductUrl as dictionary key and ProductId as value, unfortunately as the error indicated there are more then one records in Table having same ProductUrl. So you can't use it as a dictionary key.

EF1: Filtering derived types of entity class using .OfType<> by passing a string value

I have a situation where I'm trying to filter a LINQ select using a derived sub class.
ctx.BaseEntity.OfType<SubClass>() - this works fine.
However I'd like to do this using a string value instead. I've come across a performance barrier when I have lots (>20) Sub Classes and selecting an Entity without using OfType just isn't an option. I have a generic UI that renders from the base class, so I don't know what Class Type will be returned at compile time.
So what I'd like to do is this:
Perform a projected Select where I
return just the SubClassType from
the database
Perform a second select
using this value as the OfType to
only select the relevant related
entity from the database (No mass
unions generated)
int id = 1;
var classType = (from c in ctx.BaseClass.Include("ClassType")
where c.id == id
select new
{
c.ClassType.TypeName
}).First();
BaseClass caseQuery = ctx.BaseClass.OfType<classType.TypeName>()
.Include("ClassType")
.Include("ChildEntity1")
.Include("ChildEntity2")
.Where(x => x.id== id);
But obviously this won't work because OfType requires a Type and not a string.
Any ideas on how I can achieve this?
Update:
As a side note to the original question, it turns out that the moment you project a query that uses a Navigation Property - it builds the monster SQL too, so I've ended up using a stored procedure to populate my ClassType entity from the BaseClass Id.
So I've just got it to work using eSQL, which I'd never used before. I've posted the code here just in case it helps someone. Has anyone else got a more strongly typed solution they can think of?
BaseClass caseQuery = ctx.BaseClass.CreateQuery<BaseClass>("SELECT VALUE c FROM OFTYPE(Entities.[BaseClass],namespace.[" + classType.TypeName + "]) as c")
.Include("ClassType")
.Include("ChildEntity1")
.Include("ChildEntity2")
.Where(x => x.id== id).FirstOrDefault();
To answer the headline question about calling OfType with a string / runtime type, you can do the following:
// Get the type, assuming the derived type is defined in the same assembly
// as the base class and you have the type name as a string
var typeToFilter = typeof(BaseClass)
.Assembly
.GetType("Namespace." + derivedTypeName);
// The use reflection to get the OfType method and call it directly
MethodInfo ofType = typeof(Queryable).GetMethod("OfType");
MethodInfo ofTypeGeneric = method.MakeGenericMethod(new Type[] { typeToFilter });
var result = (IQueryable<Equipment>)generic.Invoke(null, new object[] { equipment });
Combine this with your stored procedure to get the class name and you (should?) avoid the massive join - I don't have table-per-type implementation to play with so I can't test.