EntityFramework and Expressions translation - entity-framework

I have a entity class Foo I've made partial containing the following code
private readonly static Expression<Func<Foo, int>> MyKeyExpression = (x) => x.Key;
public int MyKey
{
get { return MyKeyExpression.Compile()(this); }
}
The above works as in I can use MyKey in EntityFrameworks linq queries.
Why don't the following work?
private readonly static Expression<Func<Foo, int>> MyKeyExpression = (x) => x.Key;
// Set in the constructor with
// _myKeyDelegate = MyKeyExpression.Compile();
private readonly Func<Foo,int> _myKeyDelegate;
public int MyKey
{
get { return _myKeyDelegate(this); }
}
I understand the difference between a delegate and an expression(or maybe i don't?) but is confused how EntityFramework is able to interpret the property differently since MyKeyExpression.Compile() returns just that delegate which is then invoked returning an int. Perhaps its my lack of understanding of how the compiler actually handles C# Properties?
Example of usage where first example works but second examples throw a exception about not being able to translate it to SQL.:
dbContext.Foo.Delete(x => x.MyKey == 5)

I would say you don't fully understand difference between delegates and expressions.
Delegate is a reference to code compiled into IL. Only thing you can with it is execute it within .net CLR.
Expression object is a expression represented as tree, (you can think of AST). You can compile it to IL (Compile method) or you can inspect it and generate code for other execution environment, for example into SQL (that's what EF does).
When C# compiler compiles code, first it builds syntax tree and then compiles it. Basically expression is result of first part without second, so you could use SQL translator to compile it to SQL. Or you can write you own and translate it to anything else.

It's very strange what you are saying...
EF ignores the content of the getter and the setter of a mapped property (MyKey).
The query should be generated with a WHERE clause based on MyKey independent of what getter does.
How did you map the MyKey property? There is the setter missing so EF does not generate a field on the DB table and does not map it automatically.

Related

How to run dynamic query containing subquery with EF Core?

I'm able to run all sorts of dynamically composed queries with EF Core except when containing sub-queries.
Hence, this is not a duplicate of EF Core dynamic lambda subquery not working or any other I was able to find for that matter.
A runnable repro to demonstrate the issue can be found on dotnetfiddle.net/4opEqr that uses a dynamically composed expression representing the following query:
efContext.Products.Where(p => p.Id == efContext.OrderItem.Max(i => i.ProductId)).ToList();
The exception I get is
System.InvalidOperationException : The LINQ expression 'InternalDbSet<OrderItem> { }
.Max(i => i.ProductId)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
I observe the same behaviour with Microsoft.EntityFrameworkCore.SqlServer as with Microsoft.EntityFrameworkCore.InMemory (versions 6.0.8 and 7.0.0-preview.7.22376.2).
The problem seems to be both the way EF Core processes nested queryable expressions inside expression tree, and the way you are trying to test the dynamic expressions you create.
Shortly, here
productQueryable.Where(p => p.Id == orderItemQueryable.Max(i => i.ProductId))
productQueryable is regular variable (and .Where is regular call) and orderItemQueryable is compiler generated closure (as part of the queryable Where predicate expression), while here
() => productQueryable.Where(p => p.Id == orderItemQueryable.Max(i => i.ProductId))
they both are compiler generated closures.
So, the EF Core does not correctly process Constant expression holding nested queryable variable value inside "root" query expression, so for subqueries you should either pass directly IQueryable.Expression property value, or simulate closure (but not constant) expression. While for root queryable you should either wrap it in constant expression, or in simulated closure (but not directly).
Since wrapping the queryable variables work for both cases, the solution is to always wrap such variables in closure emulating expression. And closure emulating expression is any expression which contains constant expression holding some class instance holding the actual variable inside class property or field.
It can be implemented in several ways, for instance using System.Tuple class as holder:
static Expression MakeClosure<T>(T value)
{
var closure = new Tuple<T>(value);
return Expression.Property(Expression.Constant(closure), nameof(closure.Item1));
}
or real compiler generated closure class instance:
static Expression MakeClosure<T>(T value)
{
var closure = new { value };
return Expression.Property(Expression.Constant(closure), nameof(closure.value));
}
or the same using the body of compiler generated lambda expression containing closure:
static Expression MakeClosure<T>(T value)
{
Expression<Func<T>> lambda = () => value;
return lambda.Body;
}
Finally, in all the cases change the sample code as
var productQueryableExp = MakeClosure(productQueryable);
var orderItemQueryableExp = MakeClosure(orderItemQueryable);
and everything will work as expected. At least with EF Core. What about the other library you seem to be using (Remote.Linq), have no idea (that's I guess would be another question).

Passing an aggregate select expression to Dynamic Linq's GroupBy

I have simplified the following example from my code and hoping there's no obvious compilation errors because of it. Lets say I have the following entities (not what i actually have, please assume I have no EF or schema issues, this is just for example):
public class Company
{
public string GroupProperty {get;set;}
public virtual ICollection<PricingForm> PricingForms {get;set;}
}
public class PricingForm
{
public decimal Cost {get;set;}
}
And I want to query like so:
IQueryable DynamicGrouping<T>(IQueryable<T> query)
{
Expression<Func<Company, decimal?>> exp = c => c.PricingForms.Sum(fr => fr.Cost);
string selector = "new (it.Key as Key, #0(it) as Value)";
IQueryable grouping = query.GroupBy("it.GroupProperty", "it").Select(selector, exp);
return grouping;
}
I get the following error when calling the groupby/select line:
System.Linq.Dynamic.ParseException: 'Argument list incompatible with lambda expression'
What type is "it" when grouped? I have tried using other expressions that assume it is an IGrouping<string, Company>, or a IQueryable<Company>, same error. I've tried just selecting "Cost" and moving the Sum() aggregate into the selector string (i.e. Sum(#0(it)) as Value) and always seem to get the same error.
I eventually tried something along the lines of:
Expression<Func<IEnumerable<Company>, decimal?>> exp = l => l.SelectMany(c => c.PricingForms).Sum(fr => fr.Cost);
However this one, I get farther but when attempting to iterate through the results I got a different error.
The LINQ expression node type 'Invoke' is not supported in LINQ to Entities.
So, with this dynamic grouping and injecting my own select expression, what should I assume the datatype of 'it' is? Will this even work?
The type of it is IGrouping<TKey, TElement>, where TKey is dynamic based on the keySelector result type, and TElement is the element type of the input IQueryable. Luckily IGrouping<TKey, TElement> inherits (is a) IEnumerable<TElement>, so as soon as you know the input element type, you can safely base selector on IEnumerable<TElement>.
In other words, the last attempt based on Expression<Func<IEnumerable<Company>, decimal?>> is correct.
The new error you are getting is because #0(it) generates Expression.Invoke call which is not supported by EF. The easiest way to fix that is to use LINQKit Expand method:
Expression<Func<Company, decimal?>> exp = c => c.PricingForms.Sum(fr => fr.Cost);
string selector = "new (it.Key as Key, #0(it) as Value)";
IQueryable grouping = query.GroupBy("it.GroupProperty", "it").Select(selector, exp);
// This would fix the EF invocation expression error
grouping = grouping.Provider.CreateQuery(grouping.Expression.Expand());
return grouping;

Why can't I create a callback for the List Find method in Moq?

I created an extension method that lets me treat a List as DbSet for testing purposes (actually, I found this idea in another question here on stack overflow, and it's been fairly useful). Coded as follows:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var queryable = sourceList.AsQueryable();
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.As<IQueryable<T>>().Setup(m => m.Provider).Returns(queryable.Provider);
mockDbSet.As<IQueryable<T>>().Setup(m => m.Expression).Returns(queryable.Expression);
mockDbSet.As<IQueryable<T>>().Setup(m => m.ElementType).Returns(queryable.ElementType);
mockDbSet.As<IQueryable<T>>().Setup(m => m.GetEnumerator()).Returns(queryable.GetEnumerator());
mockDbSet.Setup(d => d.Add(It.IsAny<T>())).Callback<T>(sourceList.Add);
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback(sourceList.Find);
return mockDbSet.Object;
}
I had been using Add for awhile, and that works perfectly. However, when I try to add the callback for Find, I get a compiler error saying that it can't convert a method group to an action. Why is sourceList.Add an Action, but sourceList.Find is a method group?
I'll admit I'm not particularly familiar with C# delegates, so it's likely I'm missing something very obvious. Thanks in advance.
The reason Add works is because the List<T>.Add method group contains a single method which takes a single argument of type T and returns void. This method has the same signature as an Action<T> which is one of the overloads of the Callback method (the one with a single generic type parameter, Callback<T>), therefore the List<T>.Add method group can be converted to an Action<T>.
With Find, you are trying to call the Callback method (as opposed to Callback<T>) which expects an Action parameter (as opposed to Action<T>). The difference here is that an Action does not take any parameters, but an Action<T> takes a single parameter of type T. The List<T>.Find method group cannot be converted to an Action because all the Find methods (there is only one anyway) take input parameters.
The following will compile:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback<Predicate<T>>(t => sourceList.Find(t));
return mockDbSet.Object;
}
Note that I have called .Callback<Predicate<T>> because the List<T>.Find method expects and argument of type Predicate. Also note I have had to write t => sourceList.Find(t) instead of sourceList.Find because Find returns a value (which means it doesn't match the signature of Action<Predicate<T>>). By writing it as a lambda expression the return value will be thrown away.
Note that although this compiles it will not actually work because the DbSet.Find method actually takes an object[] for it's parameter, not a Predicate<T>, so you will likely have to do something like this:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback<object[]>(keyValues => sourceList.Find(keyValues.Contains));
return mockDbSet.Object;
}
This last point has more to do with how to use the Moq library that how to use method groups, delegates and lambdas - there is all sorts of syntactic sugar going on with this line which is hiding what is actually relevant to the compiler and what isn't.

How to refactor parts of an EF select

Let's say I have the following EF code:
context.Employees.Select(e => e
{
FullName = e.FirstName + " " + e.LastName,
StartDate = e.StartDate,
... // Grab other data
};
Now maybe I notice that I construct the full name in multiple places, but would like in centralized. Is it possible to refactor this?
If I make it a method or a Func, I get EF errors, because it can't translate it into SQL.
NOTE: This is a simple example, assume it can get much more complicated with "Select"s, "Where"s, whatever in the assignment, so adding a ToList and then running additional code would be suboptimal and does not fit the definition of refactoring since I would have to change functionality and not just make it more maintainable.
One solution is to use the AsExpandable method from LinqKit:
Expression<Func<Employee,string>> fullName = e => e.FirstName + " " + e.LastName;
context.Employees.AsExpandable().Select(e => e
{
FullName = fullName.Compile().Invoke(e),
StartDate = e.StartDate,
... // Grab other data
};
From the linked article:
Compile is an inbuilt method in the Expression class. It converts the
Expression into a plain Func which
satisfies the compiler. Of course, if this method actually ran, we'd
end up with compiled IL code instead of an expression tree, and LINQ
to SQL or Entity Framework would throw an exception. But here's the
clever part: Compile never actually runs; nor does LINQ to SQL or
Entity Framework ever get to see it. The call to Compile gets stripped
out entirely by a special wrapper that was created by calling
AsExpandable, and substituted for a correct expression tree.
Alternatively, you could look into creating an Model Defined Function with Entity Framework. There's also the Microsoft.Linq.Translations library if you want to define a FullName property on the Employee class itself.
I think the better centralized way to do it in the entity class itself. You can add ReadOnly property to your entity class which should be NotMapped to the database to return required formatted data.
Public class Employee
{
//...
public string fullName{get { return FirstName + " " + LastName;}}
}

Can LINQ-To-Entities be expanded?

In general, If I create an extension method that acts on an entity:
public static MyEntity Foo(this MyEntity entity)
{
// do something to the entity
}
One cannot directly use this in a projection from Linq-To-Entities such as follows:
var result = myContext.MyEntities.Select(x=> x.Foo());
Doing so yields an error such as:
System.NotSupportedException: LINQ to Entities does not recognize the
method 'Foo(MyEntity)' method, and this method cannot be translated
into a store expression.
I fully understand why this error occurs. My question is this: If I can provide an implementation of Foo that uses an expression tree, is there some way that I can add Foo to the operations that LINQ-to-entities understands? And if so - how?
Note: I can certainly convert it to a list like this:
var result = myContext.MyEntities.ToList().Select(x=> x.Foo());
And it doesn't error. But I no longer have an IQueryable. I have an IEnumerable. If I were to use it like this:
var result = myContext.MyEntities.ToList().Select(x=> x.Foo()).First();
I would end up fetching ALL entities before taking the top one and discarding the rest - which would be horrible for performance.
Use LINQKit project, but instead of your method foo, You will be obliged to write expression tree which is equivalent.