How to construct Where Expression dynamically in Entity Framework? - entity-framework

I've taken a look at this answer on how to dynamically create an OrderBy expression in Entity Framework. But I'd like to also build a dynamic Where expression. Something along the lines of this:
public IEnumerable<InventoryItem> GetAll(string filterBy, object value)
{
var results = new List<InventoryItem>();
using (var db = new InventoryDb())
{
if (QueryHelper.PropertyExists<InventoryItem>(filterBy))
{
var query = db.rri_InventoryItems.WhereByProperty(filterBy, value);
foreach(var item in query.Where(expr))
{
results.Add(ConvertItem(item));
}
}
}
return results;
}
Passing in the property to filter by and a value as ab object. Queryable has two methods for Where that both take two parameters, so I am not even sure which is the proper one.
And it's at this point I get a little more lost. I'm not sure how to refactor the original OrderByProerty method to provide a WhereByProperty. I know what I have here is completely wrong. I'm not sure what to do with it.
Ideally, I'd want to extend this even more by providing a collection of objects that could be used to build a query with ands and or operators.

Queryable has two methods for Where that both take two parameters, so I am not even sure which is the proper one.
You need the one that receives Expression<Func<T, bool>> predicate.
Here is how you can build dynamically a predicate similar to (T item) => item.Property == value:
public static partial class QueryableExtensions
{
public static IQueryable<T> WhereEquals<T>(this IQueryable<T> source, string member, object value)
{
var item = Expression.Parameter(typeof(T), "item");
var memberValue = member.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
var memberType = memberValue.Type;
if (value != null && value.GetType() != memberType)
value = Convert.ChangeType(value, memberType);
var condition = Expression.Equal(memberValue, Expression.Constant(value, memberType));
var predicate = Expression.Lambda<Func<T, bool>>(condition, item);
return source.Where(predicate);
}
}
I've tried to write it in such a way so you can step over the code in order to understand what it does. The only line that might need some explanation is:
var memberValue = member.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
This is a simple way of handling nested properties like obj.Prop1.Prop2 etc. If you don't need such capability, you can simply use this instead:
var memberValue = Expression.PropertyOrField(item, member);

I didn't need nested properties (yet). I modified your code slightly and have this that is working:
public static IQueryable<T> WhereEquals<T>(
this IQueryable<T> source, string propertyName, object value)
{
if (typeof(T).GetProperty(propertyName, BindingFlags.IgnoreCase |
BindingFlags.Public | BindingFlags.Instance) == null)
{
return null;
}
ParameterExpression parameter = Expression.Parameter(typeof(T), "item");
Expression whereProperty = Expression.Property(parameter, propertyName);
Expression constant = Expression.Constant(value);
Expression condition = Expression.Equal(whereProperty, constant);
Expression<Func<T, bool>> lambda = Expression.Lambda<Func<T, bool>>(condition,parameter);
return source.Where(lambda);
}

Related

Unable to cast the type 'System.Int32' to type 'System.Object during EF Code First Orderby Function

I'm using Specification pattern in EF Code First. When I do order by operation, VS throw a new exception
The specification pattern is copy from eShopOnWeb
I just change a little bit, here is my change code:
public class Specification<T> : ISpecification<T>
{
public Expression<Func<T, object>> OrderBy { get; private set; }
public Specification(Expression<Func<T, bool>> criteria)
{
Criteria = criteria;
}
public Specification<T> OrderByFunc(Expression<Func<T, object>> orderByExpression)
{
OrderBy = orderByExpression;
return this;
}
}
Here is my invoke code, it's very pretty simple:
static void TestSpec()
{
var spec = new Specification<ExcelData>(x => x.RowIndex == 5)
.OrderByFunc(x => x.ColumnIndex);
using (var dbContext = new TechDbContext())
{
var top10Data = dbContext.ExcelData.Take(10).ToList();
var listExcel = dbContext.ApplySpecification(spec).ToList();
Console.WriteLine();
}
}
If I comment OrderByFunc, then everything is fine to me. no error throw from vs.
I had try many times search the error message in google, but none of answer is my case.
So I have to ask a question in here.
When I debug OrderBy property in SpecificationEvaluator.cs, I found there is a Convert method.
So I know the error is about cast error, but how do I fix this cast type error?
Please help me!
The solution is to create new lambda expression with cast (Convert) removed, and then use it to call the Queryable class OrderBy / OrderByDescending method either dynamically (using DLR dispatch or reflection) or by emitting Expression.Call to it.
For the first part, add the following helper method to the SpecificationEvaluator class:
static LambdaExpression RemoveConvert(LambdaExpression source)
{
var body = source.Body;
while (body.NodeType == ExpressionType.Convert)
body = ((UnaryExpression)body).Operand;
return Expression.Lambda(body, source.Parameters);
}
Then replace the code
query = query.OrderBy(specification.OrderBy);
with either
query = Queryable.OrderBy((dynamic)query, (dynamic)RemoveConvert(specification.OrderBy));
or
var keySelector = RemoveConvert(specification.OrderBy);
query = query.Provider.CreateQuery<T>(Expression.Call(
typeof(Queryable), nameof(Queryable.OrderBy),
new[] { typeof(T), keySelector.ReturnType },
query.Expression, keySelector));
Do similar for the specification.OrderByDescending.

entity framework filter logical deleted records

I'm using EF 5.0 and Code First. In my generic repository I have a method for exclude records in a logical way. This method actually perform a update, setting the status of the entity field to false.
I would like to intercept my queries, and filter only where status == true.
Is there a easy way to do that? Ex:
new GenericRepository<Entity>().ToList();
// and internally it will filter where status == true.
create a generic method
public IQueryable<T> All<T>(Expression<Func<T, bool>> predicate) {
return context.Set<T>().Where(predicate);
}
and if you want something more linked to your status property, you have to use reflection and build the Lambda by yourself (as you can't use interfaces with linq to entites queries).
Something like that (untested), calling the generic All method.
public IQueryable<T>AllButDeleted<T>() {
var property = typeof(T).GetProperty("status");
//check if T has a "status" property
if (property == null && || property.PropertyType != typeof(bool)) throw new ArgumentException("This entity doesn't have an status property, or it's not a boolean");
//build the expression
//m =>
var parameter = new ParameterExpression(typeof(T), "m");
// m.status
Expression body = Expression.Property(parameter, property);
//m.status == true (which is just m.status)
body = Expression.IsTrue(body);
//m => m.status
var lambdaPredicate = Expression.Lambda<Func<T, bool>>(body, new[]{parameter});
return All(lambdaPredicate);
}
You can make all your entities implement some IDeletable interface:
public interface IDelitable
{
bool IsDeleted { get; }
}
And add constraint to generic parameter of your repository
public class GenericRepository<T>
where T: class, IDelitable
And add filter when you are returning values:
context.Set<T>().Where(e => !e.IsDeleted)
You can filter it using Where.
.Where(e => e.Status == true).ToList();

Linq Expression building for Entity Framework involving complex object

I am using Entity Framework version 4. I need to compare a large (~1 million record) SQL Server table to a longish (~2000) array of complex objects returned from a web service. Five different properties need to be compared to determine whether an instance of the complex object is already in the database.
I created a function that returns an expression for use in .Where and .Any methods. It looks like this (where A is the complex object, and tblA is the EF class):
function Expression<tblA, bool> GetSearchPredicate(A a)
{
return ta => ta.Field1.Equals(a.Field1)
&& ta.Field2.Equals(a.Field2)
&& ta.Field3.Equals(a.Field3)
&& ta.Field4.Equals(a.Field4)
&& ta.Field5.Equals(a.Field5);
}
This works. And I can compare all 2000 instances of A by doing this:
IEnumerable<A> objects = [web service call];
var result = objects.Select(a => !db.tblA.Any(GetSearchPredicate(a)));
That works, too. But it's slow. So I looked into building a utility method that could build an expression that could be transmitted down to the database directly through EF.
I used the code in this question as a basis for building that utility method. The example in that question shows comparing a single property to a series of constants, while my version would have to compare multiple properties to multiple constants. My best effort is below:
public static IQueryable<TEntity> WhereIn<TEntity>
(
this ObjectQuery<TEntity> query,
IEnumerable<Expression<Func<TEntity, bool>>> predicates
)
{
if (predicates == null) throw new ArgumentNullException("predicates");
IEnumerable<ParameterExpression> p = predicates.Select(pred => pred.Parameters.Single()).ToArray();
IEnumerable<Expression> equals = predicates.Select(value =>
(Expression)value.Body);
Expression bigEqual = equals.Aggregate((accumulate, equal) =>
Expression.Or(accumulate, equal));
var result1 = Expression.Lambda<Func<TEntity, bool>>(bigEqual, p.First());
var result = query.Where(result1);
return result;
}
This would be invoked like this:
IEnumerable<A> objects = [web service call];
var result = db.tblA.WhereIn(objects.Select(a => GetSearchPredicate(a)));
What I get is a message saying that "ta" (the placeholder for the TEntity object) is not bound. I thought this was because I had multiple expressions (the variable predicates) being combined, and maybe this message was being thrown because I was only passing the parameter from the first of the predicates IEnumerable. But this happens even if predicates is one expression long.
I am reasonably sure, based on the method I linked to, that I could build an expression comparing each of the five properties to a constant (the values of A.Field1 through A.Field5), rather than passing in the parameter predicates that already has them assembled into a series of expressions. But I would rather not, since that would require my method to know that it's working with types A and tblA, and that's the opposite of generic and general-purpose. (It'd also be complex and messy.)
I hope the examples I've shown explain what I want to do. Can it be done in a generic way?
You will need to replace the parameter in the predicate bodies with a single parameter. Something like this should work:
public static Expression<Func<T, bool>> BuildOr<T>(
IEnumerable<Expression<Func<T, bool>>> predicates)
{
Expression body = null;
ParameterExpression p = null;
Expression<Func<T, bool>> first = null;
foreach (Expression<Func<T, bool>> item in predicates)
{
if (first == null)
{
first = item;
}
else
{
if (body == null)
{
body = first.Body;
p = first.Parameters[0];
}
var toReplace = item.Parameters[0];
var itemBody = ReplacementVisitor.Transform(item, toReplace, p);
body = Expression.OrElse(body, itemBody);
}
}
if (first == null)
{
throw new ArgumentException("Sequence contains no elements.", "predicates");
}
return (body == null) ? first : Expression.Lambda<Func<T, bool>>(body, p);
}
private sealed class ReplacementVisitor : ExpressionVisitor
{
private IList<ParameterExpression> SourceParameters { get; set; }
private Expression ToFind { get; set; }
private Expression ReplaceWith { get; set; }
public static Expression Transform(
LambdaExpression source,
Expression toFind,
Expression replaceWith)
{
var visitor = new ReplacementVisitor
{
SourceParameters = source.Parameters,
ToFind = toFind,
ReplaceWith = replaceWith,
};
return visitor.Visit(source.Body);
}
private Expression ReplaceNode(Expression node)
{
return (node == ToFind) ? ReplaceWith : node;
}
protected override Expression VisitConstant(ConstantExpression node)
{
return ReplaceNode(node);
}
protected override Expression VisitBinary(BinaryExpression node)
{
var result = ReplaceNode(node);
if (result == node) result = base.VisitBinary(node);
return result;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (SourceParameters.Contains(node)) return ReplaceNode(node);
return SourceParameters.FirstOrDefault(p => p.Name == node.Name) ?? node;
}
}
Your WhereIn method then becomes:
public static IQueryable<TEntity> WhereIn<TEntity>(
this ObjectQuery<TEntity> query,
IEnumerable<Expression<Func<TEntity, bool>>> predicates)
{
if (predicates == null) throw new ArgumentNullException("predicates");
var predicate = BuildOr(predicates);
return query.Where(predicate);
}

Entity Framework eager loading/"IncludeEverything()"?

In EF4, I have a small object map and the volume of data is also small. So for queries I want to eager load all associated data. Is there any single method call that can do the job, like "IQueryable.IncludeEverything()", rather than call Include() repeatedly with hardcoded property names?
There's nothing out of the box, but you could use MetadataWorkspace to implement it:
public static IQueryable<T> IncludeEverything<T>(this IQueryable<T> query, ObjectContext context)
where T : class
{
var ospaceEntityType = context.MetadataWorkspace.GetItem<EntityType>(
typeof(T).FullName, DataSpace.OSpace);
var cspaceEntityType = context.MetadataWorkspace.GetEdmSpaceType(ospaceEntityType);
var includedTypes = new HashSet<EdmType>();
includedTypes.Add(cspaceEntityType);
return IncludeEverything(query, cspaceEntityType as EntityType, "", includedTypes);
}
private static IQueryable<T> IncludeEverything<T>(IQueryable<T> query,
EntityType entity,
string path,
HashSet<EdmType> includedTypes)
where T : class
{
foreach (var navigationProperty in entity.NavigationProperties)
{
var propertyEdmType = navigationProperty.TypeUsage.EdmType;
if (includedTypes.Contains(propertyEdmType))
{
continue;
}
includedTypes.Add(propertyEdmType);
var propertyCollectionType = propertyEdmType as CollectionType;
EntityType propertyEntityType;
if (propertyCollectionType != null)
{
propertyEntityType = propertyCollectionType.TypeUsage.EdmType as EntityType;
} else
{
propertyEntityType = propertyEdmType as EntityType;
}
var propertyPath = string.IsNullOrEmpty(path) ? "" : path + ".";
propertyPath += navigationProperty.Name;
query = query.Include(propertyPath);
query = IncludeEverything(query, propertyEntityType, propertyPath, includedTypes);
}
return query;
}
Note that this code is just for illustration. It doesn't have parameter validation, it may include the same enities several times and it won't include all the related entities if there are cycles in your model.

Generating Cache Keys from IQueryable For Caching Results of EF Code First Queries

I'm trying to implement a caching scheme for my EF Repository similar to the one blogged here. As the author and commenters have reported the limitation is that the key generation method cannot produce cache keys that vary with a given query's parameters. Here is the cache key generation method:
private static string GetKey<T>(IQueryable<T> query)
{
string key = string.Concat(query.ToString(), "\n\r",
typeof(T).AssemblyQualifiedName);
return key;
}
So the following queries will yield the same cache key:
var isActive = true;
var query = context.Products
.OrderBy(one => one.ProductNumber)
.Where(one => one.IsActive == isActive).AsCacheable();
and
var isActive = false;
var query = context.Products
.OrderBy(one => one.ProductNumber)
.Where(one => one.IsActive == isActive).AsCacheable();
Notice that the only difference is that isActive = true in the first query and isActive = false in the second.
Any suggestions/insight to efficiently generating cache keys which vary by IQueryable parameters would be truly appreciated.
Kudos to Sergey Barskiy for sharing the EF CodeFirst caching scheme.
Update
I took the approach of traversing the IQueryable's expression tree myself with the goal of resolving the values of the parameters used in the query. With maxlego's suggestion, I extended the System.Linq.Expressions.ExpressionVisitor class to visit the expression nodes that we're interested in - in this case, the MemberExpression. The updated GetKey method looks something like this:
public static string GetKey<T>(IQueryable<T> query)
{
var keyBuilder = new StringBuilder(query.ToString());
var queryParamVisitor = new QueryParameterVisitor(keyBuilder);
queryParamVisitor.GetQueryParameters(query.Expression);
keyBuilder.Append("\n\r");
keyBuilder.Append(typeof (T).AssemblyQualifiedName);
return keyBuilder.ToString();
}
And the QueryParameterVisitor class, which was inspired by the answers of Bryan Watts and Marc Gravell to this question, looks like this:
/// <summary>
/// <see cref="ExpressionVisitor"/> subclass which encapsulates logic to
/// traverse an expression tree and resolve all the query parameter values
/// </summary>
internal class QueryParameterVisitor : ExpressionVisitor
{
public QueryParameterVisitor(StringBuilder sb)
{
QueryParamBuilder = sb;
Visited = new Dictionary<int, bool>();
}
protected StringBuilder QueryParamBuilder { get; set; }
protected Dictionary<int, bool> Visited { get; set; }
public StringBuilder GetQueryParameters(Expression expression)
{
Visit(expression);
return QueryParamBuilder;
}
private static object GetMemberValue(MemberExpression memberExpression, Dictionary<int, bool> visited)
{
object value;
if (!TryGetMemberValue(memberExpression, out value, visited))
{
UnaryExpression objectMember = Expression.Convert(memberExpression, typeof (object));
Expression<Func<object>> getterLambda = Expression.Lambda<Func<object>>(objectMember);
Func<object> getter = null;
try
{
getter = getterLambda.Compile();
}
catch (InvalidOperationException)
{
}
if (getter != null) value = getter();
}
return value;
}
private static bool TryGetMemberValue(Expression expression, out object value, Dictionary<int, bool> visited)
{
if (expression == null)
{
// used for static fields, etc
value = null;
return true;
}
// Mark this node as visited (processed)
int expressionHash = expression.GetHashCode();
if (!visited.ContainsKey(expressionHash))
{
visited.Add(expressionHash, true);
}
// Get Member Value, recurse if necessary
switch (expression.NodeType)
{
case ExpressionType.Constant:
value = ((ConstantExpression) expression).Value;
return true;
case ExpressionType.MemberAccess:
var me = (MemberExpression) expression;
object target;
if (TryGetMemberValue(me.Expression, out target, visited))
{
// instance target
switch (me.Member.MemberType)
{
case MemberTypes.Field:
value = ((FieldInfo) me.Member).GetValue(target);
return true;
case MemberTypes.Property:
value = ((PropertyInfo) me.Member).GetValue(target, null);
return true;
}
}
break;
}
// Could not retrieve value
value = null;
return false;
}
protected override Expression VisitMember(MemberExpression node)
{
// Only process nodes that haven't been processed before, this could happen because our traversal
// is depth-first and will "visit" the nodes in the subtree before this method (VisitMember) does
if (!Visited.ContainsKey(node.GetHashCode()))
{
object value = GetMemberValue(node, Visited);
if (value != null)
{
QueryParamBuilder.Append("\n\r");
QueryParamBuilder.Append(value.ToString());
}
}
return base.VisitMember(node);
}
}
I'm still doing some performance profiling on the cache key generation and hoping that it isn't too expensive (I'll update the question with the results once I have them). I'll leave the question open, in case anyone has suggestions on how to optimize this process or has a recommendation for a more efficient method for generating cache keys with vary with the query parameters. Although this method produces the desired output, it is by no means optimal.
i suggest to use ExpressionVisitor
http://msdn.microsoft.com/en-us/library/bb882521(v=vs.90).aspx
Just for the record, "Caching the results of LINQ queries" works well with the EF and it's able to work with parameters correctly, so it can be considered as a good second level cache implementation for EF.
While the solution of the OP works quite well, I found that the performance of the solution is a little bit poor.
The duration of the key generation varied between 300ms and 1200ms for my queries.
However, I've found another solution that has quite better performance (<10ms).
public static string ToTraceString<T>(DbQuery<T> query)
{
var internalQueryField = query.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Instance).Where(f => f.Name.Equals("_internalQuery")).FirstOrDefault();
var internalQuery = internalQueryField.GetValue(query);
var objectQueryField = internalQuery.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Instance).Where(f => f.Name.Equals("_objectQuery")).FirstOrDefault();
var objectQuery = objectQueryField.GetValue(internalQuery) as ObjectQuery<T>;
return ToTraceStringWithParameters(objectQuery);
}
private static string ToTraceStringWithParameters<T>(ObjectQuery<T> query)
{
string traceString = query.ToTraceString() + "\n";
foreach (var parameter in query.Parameters)
{
traceString += parameter.Name + " [" + parameter.ParameterType.FullName + "] = " + parameter.Value + "\n";
}
return traceString;
}