How to bind parameters in replaced expression nodes in Entity Framework on the fly - entity-framework

I'm trying to replace a function call like (simplified) Utility.GetString(MyEntity.SomePropertyWithRelatedEntity)=="abc" with an expression visitor into something like p => p.SubRelatedEntities.FirstOrDefault(sre => sre.SomeFlag==true).SomePropertyWithRelatedEntity.
It means, the datamodel goes like:
MyEntity -> RelatedEntity -> SubRelatedEntity
I'm trying to return a string value from the SubRelatedEntity, based on some rules in the RelatedEntity, so I don't have to re-write / copy/paste the whole filtering rules in every usage; that's why I put inside a "call-signature", so my expression visitor can identify it and replace the fake-call to Utility.GetString to some complicated lambda expressions.
My expression visitor contains something like:
public override Expression Visit(Expression node)
{
if (node == null)
return null;
Expression result = null;
if (node.NodeType == ExpressionType.Call)
{
MethodCallExpression mce = node as MethodCallExpression;
if (mce.Method.DeclaringType == typeof(Utility) && mce.Method.Name == "GetString")
{
Expression<Func<RelatedEntity, string>> exp = re => re.SubRelatedEntities.FirstOrDefault(sre => sre.SomeFlag == true).SomeStringValue;
result = exp.Body;
}
else
result = base.Visit(node);
}
else
result = base.Visit(node);
return result;
}
Now, the problem is, the "sre" parameter is not bound when called the injected lambda expression. After much research, I see the lambda parameters should be replaced with another expression visitor, specifically searching for the new parameters and replacing them with the old ones. In my situation, however, I don't have an "old parameter" - I have the expression MyEntity.SomePropertyWithRelatedEntity (e.g. an property filled with the related entities) which I need to insert somehow in the generated lambda.
I hope my problem is understandable. Thank you for any insights!

After getting no answers for long time and trying hard to find a solution, I've solved it at the end :o)! It goes like this:
The newly injected lambda expression gets an ParameterExpression - well, this is a 'helper', used when directly calling the lambda, what I don't want (hence, 'parameter not bound' exception when ToEnumerable is called). So, the clue is to make a specialized ExpressionVisitor, which replaces this helper with the original expression, which is of course available in the Arguments[] for the method call, which I try to replace.
Works like a charm, like this you can reuse the same LINQ expressions, something like reusable sub-queries, instead of writing all the same LINQ stuff all time. Notice as well, that expression calling a method is not allowed in EF, in Linq2Sql it worked. Also, all the proposed web articles only replace the parameter instances, when constructing/merging more LINQ expressions together - here, I needed to replace a parameter with an faked-method-call argument, e.g. the method should not be called, it only stands for a code-marker, where I need to put my LINQ sub-query.
Hope this helps somebody, at the end it's pretty simple and logical, when one knows how the expression trees are constructed ;-).
Bye,
Andrej

Related

How to run dynamic query containing subquery with EF Core?

I'm able to run all sorts of dynamically composed queries with EF Core except when containing sub-queries.
Hence, this is not a duplicate of EF Core dynamic lambda subquery not working or any other I was able to find for that matter.
A runnable repro to demonstrate the issue can be found on dotnetfiddle.net/4opEqr that uses a dynamically composed expression representing the following query:
efContext.Products.Where(p => p.Id == efContext.OrderItem.Max(i => i.ProductId)).ToList();
The exception I get is
System.InvalidOperationException : The LINQ expression 'InternalDbSet<OrderItem> { }
.Max(i => i.ProductId)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
I observe the same behaviour with Microsoft.EntityFrameworkCore.SqlServer as with Microsoft.EntityFrameworkCore.InMemory (versions 6.0.8 and 7.0.0-preview.7.22376.2).
The problem seems to be both the way EF Core processes nested queryable expressions inside expression tree, and the way you are trying to test the dynamic expressions you create.
Shortly, here
productQueryable.Where(p => p.Id == orderItemQueryable.Max(i => i.ProductId))
productQueryable is regular variable (and .Where is regular call) and orderItemQueryable is compiler generated closure (as part of the queryable Where predicate expression), while here
() => productQueryable.Where(p => p.Id == orderItemQueryable.Max(i => i.ProductId))
they both are compiler generated closures.
So, the EF Core does not correctly process Constant expression holding nested queryable variable value inside "root" query expression, so for subqueries you should either pass directly IQueryable.Expression property value, or simulate closure (but not constant) expression. While for root queryable you should either wrap it in constant expression, or in simulated closure (but not directly).
Since wrapping the queryable variables work for both cases, the solution is to always wrap such variables in closure emulating expression. And closure emulating expression is any expression which contains constant expression holding some class instance holding the actual variable inside class property or field.
It can be implemented in several ways, for instance using System.Tuple class as holder:
static Expression MakeClosure<T>(T value)
{
var closure = new Tuple<T>(value);
return Expression.Property(Expression.Constant(closure), nameof(closure.Item1));
}
or real compiler generated closure class instance:
static Expression MakeClosure<T>(T value)
{
var closure = new { value };
return Expression.Property(Expression.Constant(closure), nameof(closure.value));
}
or the same using the body of compiler generated lambda expression containing closure:
static Expression MakeClosure<T>(T value)
{
Expression<Func<T>> lambda = () => value;
return lambda.Body;
}
Finally, in all the cases change the sample code as
var productQueryableExp = MakeClosure(productQueryable);
var orderItemQueryableExp = MakeClosure(orderItemQueryable);
and everything will work as expected. At least with EF Core. What about the other library you seem to be using (Remote.Linq), have no idea (that's I guess would be another question).

Reusable Functions in Linq To Entites

I have 2 reusable functions that return lists. If the code from these functions is written directly into the linq to entities query all is good. However, separating these out into functions causes an error as it cannot be translated to a stored expression. I'm sure there must be a way of doing this though. Any ideas how to solve this problems. Ideally I want the reusable parts to be used outside of linq to entity queries also.
var activityBands = DbContext.ActivityBand
.OrderBy(x => x.ActivityBandDescription)
.Where(x => x.Active && x.ClientAccountId == clientAccountId)
.Select(x => new ActivityBandDdl
{
Name = x.ActivityBandDescription,
ActivityBandId = x.ActivityBandId,
ApplyAwr = x.ApplyAwr,
AssignmentLineTimeTypeIds = TimeTypesForActivityBand(x.DailyRate) ,
AssignmentTypeIds = AssTypesForActivityBand(x.StagePayment)
}).ToList();
public static Func<bool, List<int>> TimeTypesForActivityBand =
(dailyRate) => (new int[] { 1, 2, 3, 4 }).Where(t =>
((t != 1 && t != 2) || !dailyRate) //No Timed or NTS for daily rates
).ToList();
public static Func<bool, List<int>> AssTypesForActivityBand =
(stagePayment) => (new int[] { 2,3,4,5,6,7,8,9,10 }).Where(t =>
( t!=2 || !stagePayment) //Only stage pay ass have stage pay activity bands
).ToList();
TL;DR;
suggested solution for your problem:
get LinqKit ... have a look at it's Expand() function (in the docs: combining expressions)
https://github.com/scottksmith95/LINQKit#combining-expressions
the details:
the problem boils down to: what is the difference between the queries in both cases...
A LINQ query works with an expression tree ... in other words: just because the code you typed directly into the query and the code you typed into the static Func<...> looks the same, in fact, is the same, the resulting expression trees in both cases are not the same
what is an expression tree?
imagine a simpler query like ... someIQueryable.Where(x => x.a==1 && x.b=="foo")
the lamda that is passed to Where(...) can be seen as a straigt forward c# lambda expression that can be used as a Func
and it can also be seen as a Expression>
the later is a tree of objects that form the expression, in other words a description about the way, in that the passed in parameter can be evaluated to a bool without actually having the executable code, but just the description about what to do ... take the member a from the parameter, equality-compare it to the constant 1 ... take the boolean AND of the result with the result of: take the member b from the parameter, equality-compare it to the constant "foo" ... return the result of the boolean AND
why all of this?
it's the way LINQ works ... LINQ to entiteis takes the expression tree, looks at all the operations, finds the corresponding SQL, and builds an SQL statement which is executed in the end ...
when you have your extracted Func<...> there is a little problem ... at some point in the resulting expression tree there is something like ... take the parameter x and CALL the static Func ... the expression tree does no longer contain a description of whats happening inside the Func, but just a call to that ... as long as you want to compile that to a .net runtime executable function, it's all fun and games ... but when you try to parse it into SQL, LINQ to entiteis does not know a corresponding SQL for "call some c# function" ... therefore it tells you that this part of the expression tree can not be converted into a store expression

Why can't I create a callback for the List Find method in Moq?

I created an extension method that lets me treat a List as DbSet for testing purposes (actually, I found this idea in another question here on stack overflow, and it's been fairly useful). Coded as follows:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var queryable = sourceList.AsQueryable();
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.As<IQueryable<T>>().Setup(m => m.Provider).Returns(queryable.Provider);
mockDbSet.As<IQueryable<T>>().Setup(m => m.Expression).Returns(queryable.Expression);
mockDbSet.As<IQueryable<T>>().Setup(m => m.ElementType).Returns(queryable.ElementType);
mockDbSet.As<IQueryable<T>>().Setup(m => m.GetEnumerator()).Returns(queryable.GetEnumerator());
mockDbSet.Setup(d => d.Add(It.IsAny<T>())).Callback<T>(sourceList.Add);
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback(sourceList.Find);
return mockDbSet.Object;
}
I had been using Add for awhile, and that works perfectly. However, when I try to add the callback for Find, I get a compiler error saying that it can't convert a method group to an action. Why is sourceList.Add an Action, but sourceList.Find is a method group?
I'll admit I'm not particularly familiar with C# delegates, so it's likely I'm missing something very obvious. Thanks in advance.
The reason Add works is because the List<T>.Add method group contains a single method which takes a single argument of type T and returns void. This method has the same signature as an Action<T> which is one of the overloads of the Callback method (the one with a single generic type parameter, Callback<T>), therefore the List<T>.Add method group can be converted to an Action<T>.
With Find, you are trying to call the Callback method (as opposed to Callback<T>) which expects an Action parameter (as opposed to Action<T>). The difference here is that an Action does not take any parameters, but an Action<T> takes a single parameter of type T. The List<T>.Find method group cannot be converted to an Action because all the Find methods (there is only one anyway) take input parameters.
The following will compile:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback<Predicate<T>>(t => sourceList.Find(t));
return mockDbSet.Object;
}
Note that I have called .Callback<Predicate<T>> because the List<T>.Find method expects and argument of type Predicate. Also note I have had to write t => sourceList.Find(t) instead of sourceList.Find because Find returns a value (which means it doesn't match the signature of Action<Predicate<T>>). By writing it as a lambda expression the return value will be thrown away.
Note that although this compiles it will not actually work because the DbSet.Find method actually takes an object[] for it's parameter, not a Predicate<T>, so you will likely have to do something like this:
public static DbSet<T> AsDbSet<T>(this List<T> sourceList) where T : class
{
var mockDbSet = new Mock<DbSet<T>>();
mockDbSet.Setup(d => d.Find(It.IsAny<object[]>())).Callback<object[]>(keyValues => sourceList.Find(keyValues.Contains));
return mockDbSet.Object;
}
This last point has more to do with how to use the Moq library that how to use method groups, delegates and lambdas - there is all sorts of syntactic sugar going on with this line which is hiding what is actually relevant to the compiler and what isn't.

Supporting "recursive objects" in lua

I'm fairly new to lua and have the following problem with an assignment from a class:
We currently extend lua to support objects and inheritance. The Syntax for that is
Class{'MyClass',
attribute1 = String,
attribute2 = Number
}
Class{'MySubClass', MyClass,
attribute3 = Number
}
This works perfectly fine. The real problem lies within the next task: We should support "recursive types", that means a call like
Class{'MyClass', attribute = MyClass}
should result in an class with a field of the same type as the class. When this "class-constructor" is called the variable MyClass is nil, thats why the parameter table doesnt't have an entry attribute. How is it possible to access this attribute?
My first thought was using some kind of nil-table which gets returned every time the global __index is called with an unset key. This nil-table should behave like the normal nil, but can be checked for in the "class-constructor". The problem with this approach are comparisons like nil == unknown. This should return true, but as the __eq meta method of the nil-table is never called we cannot return true.
Is there another approach I'm currently just ignoring? Any hint is greatly appreciated.
Thanks in advance.
Edit:
Here the relevant part of the "testfile". The test by which the code is rated in class is another one and gets published later.
three = 3
print( three == 3 , "Should be true")
print( unknown == nil , "Should be true" )
Class{'AClass', name = String, ref = AClass}
function AClass:write()
print("AClass:write(), name of AClass:", self.name)
end
aclass = AClass:create("A. Class")
aclass:write()
Since MyClass is just a lookup in the global table (_G), you could mess with its metatable's __index to return a newly-defined MyClass object (which you would later need to fill with the details).
However, while feasible, such an implementation is
wildly unsafe, as you could end up with an undefined class (or worse, you may end up inadvertantly creating an infinite lookup loop. Trust me, I've been there)
very hard to debug, as every _G lookup for a non-existing variable will now return a newly created class object instead of nil (this problem could somewhat be reduced by requiring that class names start with an uppercase character)
If you go that route, be sure to also override __newindex.
How about providing the argument in string form?
Class{'MyClass', attribute = 'MyClass'}
Detect strings inside the implementation of Class and process them with _G[string] after creating the class
Or alternatively, use a function to delay the lookup:
Class{'MyClass', attribute = function() return MyClass end}

Entity Framework and Nested Lambda Expressions

I've just started using Lambda expressions, and really like the shortcut. I also like the fact that I have scope within the lambda of the encompassing method. One thing I am having trouble with is nesting lambdas. Here is what I am trying to do:
public void DoSomeWork()
{
MyContext context = new MyDomainContext();
context.GetDocumentTypeCount(ci.CustomerId, io =>
{
if (io.HasError)
{
// Handle error
}
// Do some work here
// ...
// make DB call to get data
EntityQuery<AppliedGlobalFilter> query =
from a in context.GetAppliedGlobalFiltersQuery()
where a.CustomerId == ci.CustomerId && a.FilterId == 1
select a;
context.Load<AppliedGlobalFilter>(query, lo =>
{
if (lo.HasError)
{
}
**// Do more work in this nested lambda.
// Get compile time error here**
}
}, null);
}, null);
}
The second lambda is where I get the following compile time error:
Cannot convert Lambda expression to type 'System.ServiceModel.DomainService.Client.LoadBehavior' because it is not a delegate type
The compiler is choosing the wrong overload for the Load method even though I am using the same override I did in the previous Lambda.
Is this because I am trying to nest? Or do I have something else wrong?
Thanks,
-Scott
Found the problem as described in my comment above. I'll head back to work now - red face and all....
I realize this is not the answer you want, but I suggest caution about lengthy and/or nested lambdas. They work, but they often make code harder to read / maintain by other developers. I try to limit my lambdas in length to three statements, with no nesting.