Linq Expression building for Entity Framework involving complex object - entity-framework

I am using Entity Framework version 4. I need to compare a large (~1 million record) SQL Server table to a longish (~2000) array of complex objects returned from a web service. Five different properties need to be compared to determine whether an instance of the complex object is already in the database.
I created a function that returns an expression for use in .Where and .Any methods. It looks like this (where A is the complex object, and tblA is the EF class):
function Expression<tblA, bool> GetSearchPredicate(A a)
{
return ta => ta.Field1.Equals(a.Field1)
&& ta.Field2.Equals(a.Field2)
&& ta.Field3.Equals(a.Field3)
&& ta.Field4.Equals(a.Field4)
&& ta.Field5.Equals(a.Field5);
}
This works. And I can compare all 2000 instances of A by doing this:
IEnumerable<A> objects = [web service call];
var result = objects.Select(a => !db.tblA.Any(GetSearchPredicate(a)));
That works, too. But it's slow. So I looked into building a utility method that could build an expression that could be transmitted down to the database directly through EF.
I used the code in this question as a basis for building that utility method. The example in that question shows comparing a single property to a series of constants, while my version would have to compare multiple properties to multiple constants. My best effort is below:
public static IQueryable<TEntity> WhereIn<TEntity>
(
this ObjectQuery<TEntity> query,
IEnumerable<Expression<Func<TEntity, bool>>> predicates
)
{
if (predicates == null) throw new ArgumentNullException("predicates");
IEnumerable<ParameterExpression> p = predicates.Select(pred => pred.Parameters.Single()).ToArray();
IEnumerable<Expression> equals = predicates.Select(value =>
(Expression)value.Body);
Expression bigEqual = equals.Aggregate((accumulate, equal) =>
Expression.Or(accumulate, equal));
var result1 = Expression.Lambda<Func<TEntity, bool>>(bigEqual, p.First());
var result = query.Where(result1);
return result;
}
This would be invoked like this:
IEnumerable<A> objects = [web service call];
var result = db.tblA.WhereIn(objects.Select(a => GetSearchPredicate(a)));
What I get is a message saying that "ta" (the placeholder for the TEntity object) is not bound. I thought this was because I had multiple expressions (the variable predicates) being combined, and maybe this message was being thrown because I was only passing the parameter from the first of the predicates IEnumerable. But this happens even if predicates is one expression long.
I am reasonably sure, based on the method I linked to, that I could build an expression comparing each of the five properties to a constant (the values of A.Field1 through A.Field5), rather than passing in the parameter predicates that already has them assembled into a series of expressions. But I would rather not, since that would require my method to know that it's working with types A and tblA, and that's the opposite of generic and general-purpose. (It'd also be complex and messy.)
I hope the examples I've shown explain what I want to do. Can it be done in a generic way?

You will need to replace the parameter in the predicate bodies with a single parameter. Something like this should work:
public static Expression<Func<T, bool>> BuildOr<T>(
IEnumerable<Expression<Func<T, bool>>> predicates)
{
Expression body = null;
ParameterExpression p = null;
Expression<Func<T, bool>> first = null;
foreach (Expression<Func<T, bool>> item in predicates)
{
if (first == null)
{
first = item;
}
else
{
if (body == null)
{
body = first.Body;
p = first.Parameters[0];
}
var toReplace = item.Parameters[0];
var itemBody = ReplacementVisitor.Transform(item, toReplace, p);
body = Expression.OrElse(body, itemBody);
}
}
if (first == null)
{
throw new ArgumentException("Sequence contains no elements.", "predicates");
}
return (body == null) ? first : Expression.Lambda<Func<T, bool>>(body, p);
}
private sealed class ReplacementVisitor : ExpressionVisitor
{
private IList<ParameterExpression> SourceParameters { get; set; }
private Expression ToFind { get; set; }
private Expression ReplaceWith { get; set; }
public static Expression Transform(
LambdaExpression source,
Expression toFind,
Expression replaceWith)
{
var visitor = new ReplacementVisitor
{
SourceParameters = source.Parameters,
ToFind = toFind,
ReplaceWith = replaceWith,
};
return visitor.Visit(source.Body);
}
private Expression ReplaceNode(Expression node)
{
return (node == ToFind) ? ReplaceWith : node;
}
protected override Expression VisitConstant(ConstantExpression node)
{
return ReplaceNode(node);
}
protected override Expression VisitBinary(BinaryExpression node)
{
var result = ReplaceNode(node);
if (result == node) result = base.VisitBinary(node);
return result;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (SourceParameters.Contains(node)) return ReplaceNode(node);
return SourceParameters.FirstOrDefault(p => p.Name == node.Name) ?? node;
}
}
Your WhereIn method then becomes:
public static IQueryable<TEntity> WhereIn<TEntity>(
this ObjectQuery<TEntity> query,
IEnumerable<Expression<Func<TEntity, bool>>> predicates)
{
if (predicates == null) throw new ArgumentNullException("predicates");
var predicate = BuildOr(predicates);
return query.Where(predicate);
}

Related

Get dependent ids when querying principal

I'm trying to get just the ids for dependents if a principal is queried, every time the principal is queried.
My initial thought is to add it somehow in the OnModelCreating definitions, however that appears to be limited to filtering down larger sets of data, unless I'm missing something.
Something like this:
builder.Entity<ListingModel>()
.AlsoDoThis(
x => x.MenuIds.AddRange(
Menus.Where(y => y.ListingId == x.Id).Select(y => y.Id).ToList()
)
);
There is a need to not do this in code for each individual place I have a Select, since that functionality is normalized in some base classes. The base classes have a <TModel> passed in and don't inherently know what properties need to be handled this way.
I do have a workaround where I'm grabbing everything with an AutoInclude(), then filtering it out in the model definition with customer getter/setter to return a list of ids. But rather than being more performant (grabbing related FK ids at the DB level) it's transferring all of that data to the server and then programmatically selecting a list of ids, as far as I understand it.
private List<int> _topicsIds = new();
[NotMapped]
public List<int> TopicsIds
{
get { return Topics.Count > 0 ? Topics.Select(x => x.Id).ToList() : _topicsIds; }
set { _topicsIds = value; }
}
public List<TopicModel> Topics { get; set; } = new();
"Extra SQL that gets called with every select in a context" is (to my limited knowledge) almost what HasQueryFilter does, with a just slightly broader operation. I think this is the approach I'm looking for, just selecting more stuff instead of filtering stuff out.
You can project everything via Select
var result = ctx.ListingModels
.Select(lm => new // or to DTO
{
Id = lm.Id,
OtherProperty = lm.OtherProperty,
Ids = x.MenuIds.Select(m => m.Id).ToList()
})
.ToList();
To make more general solution we can use annotations and define how to project such entities.
During Model defining:
builder.Entity<TopicModel>()
.WithProjection(
x => x.MenuIds,
x => x.Menus.Where(y => y.ListingId == x.Id).Select(y => y.Id).ToList()
);
Then usage in common code:
public virtual List<TModel> GetList(List<int> ids)
{
var list = _context.Set<TModel>().Where(x => ids.Any(id => id == x.Id))
.ApplyCustomProjection(_context)
.ToList();
return list;
}
ApplyCustomProjection(_context) will find previously defined annotation and will apply custom projection.
And extensions implementation:
public static class ProjectionExtensions
{
public const string CustomProjectionAnnotation = "custom:member_projection";
public class ProjectionInfo
{
public ProjectionInfo(MemberInfo member, LambdaExpression expression)
{
Member = member;
Expression = expression;
}
public MemberInfo Member { get; }
public LambdaExpression Expression { get; }
}
public static bool IsUnderDotnetTool { get; }
= Process.GetCurrentProcess().ProcessName == "dotnet";
public static EntityTypeBuilder<TEntity> WithProjection<TEntity, TValue>(
this EntityTypeBuilder<TEntity> entity,
Expression<Func<TEntity, TValue>> propExpression,
Expression<Func<TEntity, TValue>> assignmentExpression)
where TEntity : class
{
// avoid registering non serializable annotations during migrations update
if (IsUnderDotnetTool)
return entity;
var annotation = entity.Metadata.FindAnnotation(CustomProjectionAnnotation);
var projections = annotation?.Value as List<ProjectionInfo> ?? new List<ProjectionInfo>();
if (propExpression.Body is not MemberExpression memberExpression)
throw new InvalidOperationException($"'{propExpression.Body}' is not member expression");
if (memberExpression.Expression is not ParameterExpression)
throw new InvalidOperationException($"'{memberExpression.Expression}' is not parameter expression. Only single nesting is allowed");
// removing duplicate
projections.RemoveAll(p => p.Member == memberExpression.Member);
projections.Add(new ProjectionInfo(memberExpression.Member, assignmentExpression));
return entity.HasAnnotation(CustomProjectionAnnotation, projections);
}
public static IQueryable<TEntity> ApplyCustomProjection<TEntity>(this IQueryable<TEntity> query, DbContext context)
where TEntity : class
{
var et = context.Model.FindEntityType(typeof(TEntity));
var projections = et?.FindAnnotation(CustomProjectionAnnotation)?.Value as List<ProjectionInfo>;
// nothing to do
if (projections == null || et == null)
return query;
var propertiesForProjection = et.GetProperties().Where(p =>
p.PropertyInfo != null && projections.All(pr => pr.Member != p.PropertyInfo))
.ToList();
var entityParam = Expression.Parameter(typeof(TEntity), "e");
var memberBinding = new MemberBinding[propertiesForProjection.Count + projections.Count];
for (int i = 0; i < propertiesForProjection.Count; i++)
{
var propertyInfo = propertiesForProjection[i].PropertyInfo!;
memberBinding[i] = Expression.Bind(propertyInfo, Expression.MakeMemberAccess(entityParam, propertyInfo));
}
for (int i = 0; i < projections.Count; i++)
{
var projection = projections[i];
var expression = projection.Expression.Body;
var assignExpression = ReplacingExpressionVisitor.Replace(projection.Expression.Parameters[0], entityParam, expression);
memberBinding[propertiesForProjection.Count + i] = Expression.Bind(projection.Member, assignExpression);
}
var memberInit = Expression.MemberInit(Expression.New(typeof(TEntity)), memberBinding);
var selectLambda = Expression.Lambda<Func<TEntity, TEntity>>(memberInit, entityParam);
var newQuery = query.Select(selectLambda);
return newQuery;
}
}

How to construct Where Expression dynamically in Entity Framework?

I've taken a look at this answer on how to dynamically create an OrderBy expression in Entity Framework. But I'd like to also build a dynamic Where expression. Something along the lines of this:
public IEnumerable<InventoryItem> GetAll(string filterBy, object value)
{
var results = new List<InventoryItem>();
using (var db = new InventoryDb())
{
if (QueryHelper.PropertyExists<InventoryItem>(filterBy))
{
var query = db.rri_InventoryItems.WhereByProperty(filterBy, value);
foreach(var item in query.Where(expr))
{
results.Add(ConvertItem(item));
}
}
}
return results;
}
Passing in the property to filter by and a value as ab object. Queryable has two methods for Where that both take two parameters, so I am not even sure which is the proper one.
And it's at this point I get a little more lost. I'm not sure how to refactor the original OrderByProerty method to provide a WhereByProperty. I know what I have here is completely wrong. I'm not sure what to do with it.
Ideally, I'd want to extend this even more by providing a collection of objects that could be used to build a query with ands and or operators.
Queryable has two methods for Where that both take two parameters, so I am not even sure which is the proper one.
You need the one that receives Expression<Func<T, bool>> predicate.
Here is how you can build dynamically a predicate similar to (T item) => item.Property == value:
public static partial class QueryableExtensions
{
public static IQueryable<T> WhereEquals<T>(this IQueryable<T> source, string member, object value)
{
var item = Expression.Parameter(typeof(T), "item");
var memberValue = member.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
var memberType = memberValue.Type;
if (value != null && value.GetType() != memberType)
value = Convert.ChangeType(value, memberType);
var condition = Expression.Equal(memberValue, Expression.Constant(value, memberType));
var predicate = Expression.Lambda<Func<T, bool>>(condition, item);
return source.Where(predicate);
}
}
I've tried to write it in such a way so you can step over the code in order to understand what it does. The only line that might need some explanation is:
var memberValue = member.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
This is a simple way of handling nested properties like obj.Prop1.Prop2 etc. If you don't need such capability, you can simply use this instead:
var memberValue = Expression.PropertyOrField(item, member);
I didn't need nested properties (yet). I modified your code slightly and have this that is working:
public static IQueryable<T> WhereEquals<T>(
this IQueryable<T> source, string propertyName, object value)
{
if (typeof(T).GetProperty(propertyName, BindingFlags.IgnoreCase |
BindingFlags.Public | BindingFlags.Instance) == null)
{
return null;
}
ParameterExpression parameter = Expression.Parameter(typeof(T), "item");
Expression whereProperty = Expression.Property(parameter, propertyName);
Expression constant = Expression.Constant(value);
Expression condition = Expression.Equal(whereProperty, constant);
Expression<Func<T, bool>> lambda = Expression.Lambda<Func<T, bool>>(condition,parameter);
return source.Where(lambda);
}

Generating Cache Keys from IQueryable For Caching Results of EF Code First Queries

I'm trying to implement a caching scheme for my EF Repository similar to the one blogged here. As the author and commenters have reported the limitation is that the key generation method cannot produce cache keys that vary with a given query's parameters. Here is the cache key generation method:
private static string GetKey<T>(IQueryable<T> query)
{
string key = string.Concat(query.ToString(), "\n\r",
typeof(T).AssemblyQualifiedName);
return key;
}
So the following queries will yield the same cache key:
var isActive = true;
var query = context.Products
.OrderBy(one => one.ProductNumber)
.Where(one => one.IsActive == isActive).AsCacheable();
and
var isActive = false;
var query = context.Products
.OrderBy(one => one.ProductNumber)
.Where(one => one.IsActive == isActive).AsCacheable();
Notice that the only difference is that isActive = true in the first query and isActive = false in the second.
Any suggestions/insight to efficiently generating cache keys which vary by IQueryable parameters would be truly appreciated.
Kudos to Sergey Barskiy for sharing the EF CodeFirst caching scheme.
Update
I took the approach of traversing the IQueryable's expression tree myself with the goal of resolving the values of the parameters used in the query. With maxlego's suggestion, I extended the System.Linq.Expressions.ExpressionVisitor class to visit the expression nodes that we're interested in - in this case, the MemberExpression. The updated GetKey method looks something like this:
public static string GetKey<T>(IQueryable<T> query)
{
var keyBuilder = new StringBuilder(query.ToString());
var queryParamVisitor = new QueryParameterVisitor(keyBuilder);
queryParamVisitor.GetQueryParameters(query.Expression);
keyBuilder.Append("\n\r");
keyBuilder.Append(typeof (T).AssemblyQualifiedName);
return keyBuilder.ToString();
}
And the QueryParameterVisitor class, which was inspired by the answers of Bryan Watts and Marc Gravell to this question, looks like this:
/// <summary>
/// <see cref="ExpressionVisitor"/> subclass which encapsulates logic to
/// traverse an expression tree and resolve all the query parameter values
/// </summary>
internal class QueryParameterVisitor : ExpressionVisitor
{
public QueryParameterVisitor(StringBuilder sb)
{
QueryParamBuilder = sb;
Visited = new Dictionary<int, bool>();
}
protected StringBuilder QueryParamBuilder { get; set; }
protected Dictionary<int, bool> Visited { get; set; }
public StringBuilder GetQueryParameters(Expression expression)
{
Visit(expression);
return QueryParamBuilder;
}
private static object GetMemberValue(MemberExpression memberExpression, Dictionary<int, bool> visited)
{
object value;
if (!TryGetMemberValue(memberExpression, out value, visited))
{
UnaryExpression objectMember = Expression.Convert(memberExpression, typeof (object));
Expression<Func<object>> getterLambda = Expression.Lambda<Func<object>>(objectMember);
Func<object> getter = null;
try
{
getter = getterLambda.Compile();
}
catch (InvalidOperationException)
{
}
if (getter != null) value = getter();
}
return value;
}
private static bool TryGetMemberValue(Expression expression, out object value, Dictionary<int, bool> visited)
{
if (expression == null)
{
// used for static fields, etc
value = null;
return true;
}
// Mark this node as visited (processed)
int expressionHash = expression.GetHashCode();
if (!visited.ContainsKey(expressionHash))
{
visited.Add(expressionHash, true);
}
// Get Member Value, recurse if necessary
switch (expression.NodeType)
{
case ExpressionType.Constant:
value = ((ConstantExpression) expression).Value;
return true;
case ExpressionType.MemberAccess:
var me = (MemberExpression) expression;
object target;
if (TryGetMemberValue(me.Expression, out target, visited))
{
// instance target
switch (me.Member.MemberType)
{
case MemberTypes.Field:
value = ((FieldInfo) me.Member).GetValue(target);
return true;
case MemberTypes.Property:
value = ((PropertyInfo) me.Member).GetValue(target, null);
return true;
}
}
break;
}
// Could not retrieve value
value = null;
return false;
}
protected override Expression VisitMember(MemberExpression node)
{
// Only process nodes that haven't been processed before, this could happen because our traversal
// is depth-first and will "visit" the nodes in the subtree before this method (VisitMember) does
if (!Visited.ContainsKey(node.GetHashCode()))
{
object value = GetMemberValue(node, Visited);
if (value != null)
{
QueryParamBuilder.Append("\n\r");
QueryParamBuilder.Append(value.ToString());
}
}
return base.VisitMember(node);
}
}
I'm still doing some performance profiling on the cache key generation and hoping that it isn't too expensive (I'll update the question with the results once I have them). I'll leave the question open, in case anyone has suggestions on how to optimize this process or has a recommendation for a more efficient method for generating cache keys with vary with the query parameters. Although this method produces the desired output, it is by no means optimal.
i suggest to use ExpressionVisitor
http://msdn.microsoft.com/en-us/library/bb882521(v=vs.90).aspx
Just for the record, "Caching the results of LINQ queries" works well with the EF and it's able to work with parameters correctly, so it can be considered as a good second level cache implementation for EF.
While the solution of the OP works quite well, I found that the performance of the solution is a little bit poor.
The duration of the key generation varied between 300ms and 1200ms for my queries.
However, I've found another solution that has quite better performance (<10ms).
public static string ToTraceString<T>(DbQuery<T> query)
{
var internalQueryField = query.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Instance).Where(f => f.Name.Equals("_internalQuery")).FirstOrDefault();
var internalQuery = internalQueryField.GetValue(query);
var objectQueryField = internalQuery.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Instance).Where(f => f.Name.Equals("_objectQuery")).FirstOrDefault();
var objectQuery = objectQueryField.GetValue(internalQuery) as ObjectQuery<T>;
return ToTraceStringWithParameters(objectQuery);
}
private static string ToTraceStringWithParameters<T>(ObjectQuery<T> query)
{
string traceString = query.ToTraceString() + "\n";
foreach (var parameter in query.Parameters)
{
traceString += parameter.Name + " [" + parameter.ParameterType.FullName + "] = " + parameter.Value + "\n";
}
return traceString;
}

LINQ to Entities - cannot cast 'System.DateTime' to type 'System.Object' in orderBy

I am trying to order an IQueryable of entities by date from a passed in Expression< Func< T, object>> and am getting the error: "Unable to cast the type 'System.Nullable`1' to type 'System.Object'. LINQ to Entities only supports casting Entity Data Model primitive types." The entity has a nullable datetime property on it on which I am trying to sort:
Example: (where e.Date is a nullable DateTime)
Expression<Func<T,object>> sorter = (e) => e.Date;
IOrderedQueryable<T> sortedData = data.OrderBy(sorter);
Thanks in advance!
I wrote a simple class for ordering entities based on a lambda expression at runtime.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
namespace DataModeling
{
public class QueryOrderer<TEntity>
where TEntity : class
{
private LambdaExpression defaultSortExpression;
private Dictionary<string, LambdaExpression> orderFieldLookup;
public QueryOrderer()
{
orderFieldLookup = new Dictionary<string, LambdaExpression>();
}
public void AddOrderMapping<TProp>(string fieldName, Expression<Func<TEntity, TProp>> selector)
{
orderFieldLookup[fieldName] = selector;
}
public void SetDefaultSortExpression<TProp>(Expression<Func<TEntity, TProp>> selector)
{
defaultSortExpression = selector;
}
public IQueryable<TEntity> GetOrderedEntries(string field, bool isDescending, IQueryable<TEntity> entries)
{
return orderEntries(entries, field, isDescending);
}
private IQueryable<TEntity> orderEntries(IQueryable<TEntity> entries, string fieldName, bool isDescending)
{
dynamic lambda = getOrderByLambda(entries, fieldName);
if (lambda == null)
{
return entries;
}
if (isDescending)
{
return Queryable.OrderByDescending(entries, lambda);
}
else
{
return Queryable.OrderBy(entries, lambda);
}
}
private dynamic getOrderByLambda(IQueryable<TEntity> entries, string fieldName)
{
if (!String.IsNullOrWhiteSpace(fieldName) && orderFieldLookup.ContainsKey(fieldName))
{
return orderFieldLookup[fieldName];
}
else
{
return defaultSortExpression;
}
}
}
}
You use this class by initially setting up all of the fields:
QueryOrderer<User> orderer = new QueryOrderer<User>();
orderer.SetDefaultSortExpression(u => u.FullName);
orderer.AddOrderMapping("UserId", u => u.UserId);
orderer.AddOrderMapping("Name", u => u.FullName);
orderer.AddOrderMapping("Email", u => u.Email);
orderer.AddOrderMapping("CreatedOn", u => u.CreatedOn);
...
var users = orderer.GetOrderedEntries("CreatedOn", isDescending: false, context.Users);
I nice feature of this code is that it handles look-up values perfectly. For instance, if you're trying to sort using the description rather than a key, you can use the outer context when building up the sort expression:
orderer.AddOrderMapping("UserType",
u => context.UserTypes
.Where(t => t.UserTypeId == u.UserTypeId)
.Select(t => t.Description)
.FirstOrDefault());
Entity Framework is smart enough to just fold the sub-query right into the outer query.
Two problem here: First you use object in your sorter, you should use DateTime. Secondly every element must have a place in the order so you have to define what should happen with elements where Date is null:
Expression<Func<T, DateTime>> sorter = (e) => e.Date ?? DateTime.MaxValue;
IOrderedQueryable<T> sortedData = data.OrderBy(sorter);
Try to reconstruct expression body
private LambdaExpression CreateLambdaPropertyGetter(Expression<Func<TEntity, object>> expression)
{
Expression body;
if (expression.Body is UnaryExpression && ((UnaryExpression)expression.Body).NodeType == ExpressionType.Convert)
body = ((UnaryExpression)expression.Body).Operand;
else
body = expression.Body;
var lambda = Expression.Lambda(body, expression.Parameters);
return lambda;
}
Try using Func delegate instead on Expression<Func>
Func<T,object> sorter = (e) => e.Date;
IOrderedEnumerable<T> sortedData = data.OrderBy(sorter);

Serializing Entity Framework problems

Like several other people, I'm having problems serializing Entity Framework objects, so that I can send the data over AJAX in a JSON format.
I've got the following server-side method, which I'm attempting to call using AJAX through jQuery
[WebMethod]
public static IEnumerable<Message> GetAllMessages(int officerId)
{
SIBSv2Entities db = new SIBSv2Entities();
return (from m in db.MessageRecipients
where m.OfficerId == officerId
select m.Message).AsEnumerable<Message>();
}
Calling this via AJAX results in this error:
A circular reference was detected while serializing an object of type \u0027System.Data.Metadata.Edm.AssociationType
Which is because of the way the Entity Framework creates circular references to keep all the objects related and accessible server side.
I came across the following code from (http://hellowebapps.com/2010-09-26/producing-json-from-entity-framework-4-0-generated-classes/) which claims to get around this problem by capping the maximum depth for references. I've added the code below, because I had to tweak it slightly to get it work (All angled brackets are missing from the code on the website)
using System.Web.Script.Serialization;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System;
public class EFObjectConverter : JavaScriptConverter
{
private int _currentDepth = 1;
private readonly int _maxDepth = 2;
private readonly List<int> _processedObjects = new List<int>();
private readonly Type[] _builtInTypes = new[]{
typeof(bool),
typeof(byte),
typeof(sbyte),
typeof(char),
typeof(decimal),
typeof(double),
typeof(float),
typeof(int),
typeof(uint),
typeof(long),
typeof(ulong),
typeof(short),
typeof(ushort),
typeof(string),
typeof(DateTime),
typeof(Guid)
};
public EFObjectConverter( int maxDepth = 2,
EFObjectConverter parent = null)
{
_maxDepth = maxDepth;
if (parent != null)
{
_currentDepth += parent._currentDepth;
}
}
public override object Deserialize( IDictionary<string,object> dictionary, Type type, JavaScriptSerializer serializer)
{
return null;
}
public override IDictionary<string,object> Serialize(object obj, JavaScriptSerializer serializer)
{
_processedObjects.Add(obj.GetHashCode());
Type type = obj.GetType();
var properties = from p in type.GetProperties()
where p.CanWrite &&
p.CanWrite &&
_builtInTypes.Contains(p.PropertyType)
select p;
var result = properties.ToDictionary(
property => property.Name,
property => (Object)(property.GetValue(obj, null)
== null
? ""
: property.GetValue(obj, null).ToString().Trim())
);
if (_maxDepth >= _currentDepth)
{
var complexProperties = from p in type.GetProperties()
where p.CanWrite &&
p.CanRead &&
!_builtInTypes.Contains(p.PropertyType) &&
!_processedObjects.Contains(p.GetValue(obj, null)
== null
? 0
: p.GetValue(obj, null).GetHashCode())
select p;
foreach (var property in complexProperties)
{
var js = new JavaScriptSerializer();
js.RegisterConverters(new List<JavaScriptConverter> { new EFObjectConverter(_maxDepth - _currentDepth, this) });
result.Add(property.Name, js.Serialize(property.GetValue(obj, null)));
}
}
return result;
}
public override IEnumerable<System.Type> SupportedTypes
{
get
{
return GetType().Assembly.GetTypes();
}
}
}
However even when using that code, in the following way:
var js = new System.Web.Script.Serialization.JavaScriptSerializer();
js.RegisterConverters(new List<System.Web.Script.Serialization.JavaScriptConverter> { new EFObjectConverter(2) });
return js.Serialize(messages);
I'm still seeing the A circular reference was detected... exception being thrown!
I solved these issues with the following classes:
public class EFJavaScriptSerializer : JavaScriptSerializer
{
public EFJavaScriptSerializer()
{
RegisterConverters(new List<JavaScriptConverter>{new EFJavaScriptConverter()});
}
}
and
public class EFJavaScriptConverter : JavaScriptConverter
{
private int _currentDepth = 1;
private readonly int _maxDepth = 1;
private readonly List<object> _processedObjects = new List<object>();
private readonly Type[] _builtInTypes = new[]
{
typeof(int?),
typeof(double?),
typeof(bool?),
typeof(bool),
typeof(byte),
typeof(sbyte),
typeof(char),
typeof(decimal),
typeof(double),
typeof(float),
typeof(int),
typeof(uint),
typeof(long),
typeof(ulong),
typeof(short),
typeof(ushort),
typeof(string),
typeof(DateTime),
typeof(DateTime?),
typeof(Guid)
};
public EFJavaScriptConverter() : this(1, null) { }
public EFJavaScriptConverter(int maxDepth = 1, EFJavaScriptConverter parent = null)
{
_maxDepth = maxDepth;
if (parent != null)
{
_currentDepth += parent._currentDepth;
}
}
public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)
{
return null;
}
public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
{
_processedObjects.Add(obj.GetHashCode());
var type = obj.GetType();
var properties = from p in type.GetProperties()
where p.CanRead && p.GetIndexParameters().Count() == 0 &&
_builtInTypes.Contains(p.PropertyType)
select p;
var result = properties.ToDictionary(
p => p.Name,
p => (Object)TryGetStringValue(p, obj));
if (_maxDepth >= _currentDepth)
{
var complexProperties = from p in type.GetProperties()
where p.CanRead &&
p.GetIndexParameters().Count() == 0 &&
!_builtInTypes.Contains(p.PropertyType) &&
p.Name != "RelationshipManager" &&
!AllreadyAdded(p, obj)
select p;
foreach (var property in complexProperties)
{
var complexValue = TryGetValue(property, obj);
if(complexValue != null)
{
var js = new EFJavaScriptConverter(_maxDepth - _currentDepth, this);
result.Add(property.Name, js.Serialize(complexValue, new EFJavaScriptSerializer()));
}
}
}
return result;
}
private bool AllreadyAdded(PropertyInfo p, object obj)
{
var val = TryGetValue(p, obj);
return _processedObjects.Contains(val == null ? 0 : val.GetHashCode());
}
private static object TryGetValue(PropertyInfo p, object obj)
{
var parameters = p.GetIndexParameters();
if (parameters.Length == 0)
{
return p.GetValue(obj, null);
}
else
{
//cant serialize these
return null;
}
}
private static object TryGetStringValue(PropertyInfo p, object obj)
{
if (p.GetIndexParameters().Length == 0)
{
var val = p.GetValue(obj, null);
return val;
}
else
{
return string.Empty;
}
}
public override IEnumerable<Type> SupportedTypes
{
get
{
var types = new List<Type>();
//ef types
types.AddRange(Assembly.GetAssembly(typeof(DbContext)).GetTypes());
//model types
types.AddRange(Assembly.GetAssembly(typeof(BaseViewModel)).GetTypes());
return types;
}
}
}
You can now safely make a call like new EFJavaScriptSerializer().Serialize(obj)
Update : since version Telerik v1.3+ you can now override the GridActionAttribute.CreateActionResult method and hence you can easily integrate this Serializer into specific controller methods by applying your custom [GridAction] attribute:
[Grid]
public ActionResult _GetOrders(int id)
{
return new GridModel(Service.GetOrders(id));
}
and
public class GridAttribute : GridActionAttribute, IActionFilter
{
/// <summary>
/// Determines the depth that the serializer will traverse
/// </summary>
public int SerializationDepth { get; set; }
/// <summary>
/// Initializes a new instance of the <see cref="GridActionAttribute"/> class.
/// </summary>
public GridAttribute()
: base()
{
ActionParameterName = "command";
SerializationDepth = 1;
}
protected override ActionResult CreateActionResult(object model)
{
return new EFJsonResult
{
Data = model,
JsonRequestBehavior = JsonRequestBehavior.AllowGet,
MaxSerializationDepth = SerializationDepth
};
}
}
and finally..
public class EFJsonResult : JsonResult
{
const string JsonRequest_GetNotAllowed = "This request has been blocked because sensitive information could be disclosed to third party web sites when this is used in a GET request. To allow GET requests, set JsonRequestBehavior to AllowGet.";
public EFJsonResult()
{
MaxJsonLength = 1024000000;
RecursionLimit = 10;
MaxSerializationDepth = 1;
}
public int MaxJsonLength { get; set; }
public int RecursionLimit { get; set; }
public int MaxSerializationDepth { get; set; }
public override void ExecuteResult(ControllerContext context)
{
if (context == null)
{
throw new ArgumentNullException("context");
}
if (JsonRequestBehavior == JsonRequestBehavior.DenyGet &&
String.Equals(context.HttpContext.Request.HttpMethod, "GET", StringComparison.OrdinalIgnoreCase))
{
throw new InvalidOperationException(JsonRequest_GetNotAllowed);
}
var response = context.HttpContext.Response;
if (!String.IsNullOrEmpty(ContentType))
{
response.ContentType = ContentType;
}
else
{
response.ContentType = "application/json";
}
if (ContentEncoding != null)
{
response.ContentEncoding = ContentEncoding;
}
if (Data != null)
{
var serializer = new JavaScriptSerializer
{
MaxJsonLength = MaxJsonLength,
RecursionLimit = RecursionLimit
};
serializer.RegisterConverters(new List<JavaScriptConverter> { new EFJsonConverter(MaxSerializationDepth) });
response.Write(serializer.Serialize(Data));
}
}
You can also detach the object from the context and it will remove the navigation properties so that it can be serialized. For my data repository classes that are used with Json i use something like this.
public DataModel.Page GetPage(Guid idPage, bool detach = false)
{
var results = from p in DataContext.Pages
where p.idPage == idPage
select p;
if (results.Count() == 0)
return null;
else
{
var result = results.First();
if (detach)
DataContext.Detach(result);
return result;
}
}
By default the returned object will have all of the complex/navigation properties, but by setting detach = true it will remove those properties and return the base object only. For a list of objects the implementation looks like this
public List<DataModel.Page> GetPageList(Guid idSite, bool detach = false)
{
var results = from p in DataContext.Pages
where p.idSite == idSite
select p;
if (results.Count() > 0)
{
if (detach)
{
List<DataModel.Page> retValue = new List<DataModel.Page>();
foreach (var result in results)
{
DataContext.Detach(result);
retValue.Add(result);
}
return retValue;
}
else
return results.ToList();
}
else
return new List<DataModel.Page>();
}
I have just successfully tested this code.
It may be that in your case your Message object is in a different assembly? The overriden Property SupportedTypes is returning everything ONLY in its own Assembly so when serialize is called the JavaScriptSerializer defaults to the standard JavaScriptConverter.
You should be able to verify this debugging.
Your error occured due to some "Reference" classes generated by EF for some entities with 1:1 relations and that the JavaScriptSerializer failed to serialize.
I've used a workaround by adding a new condition :
!p.Name.EndsWith("Reference")
The code to get the complex properties looks like this :
var complexProperties = from p in type.GetProperties()
where p.CanWrite &&
p.CanRead &&
!p.Name.EndsWith("Reference") &&
!_builtInTypes.Contains(p.PropertyType) &&
!_processedObjects.Contains(p.GetValue(obj, null)
== null
? 0
: p.GetValue(obj, null).GetHashCode())
select p;
Hope this help you.
I had a similar problem with pushing my view via Ajax to UI components.
I also found and tried to use that code sample you provided. Some problems I had with that code:
SupportedTypes wasn't grabbing the types I needed, so the converter wasn't being called
If the maximum depth is hit, the serialization would be truncated
It threw out any other converters I had on the existing serializer by creating its own new JavaScriptSerializer
Here are the fixes I implemented for those issues:
Reusing the same serializer
I simply reused the existing serializer that is passed into Serialize to solve this problem. This broke the depth hack though.
Truncating on already-visited, rather than on depth
Instead of truncating on depth, I created a HashSet<object> of already seen instances (with a custom IEqualityComparer that checked reference equality). I simply didn't recurse if I found an instance I'd already seen. This is the same detection mechanism built into the JavaScriptSerializer itself, so worked quite well.
The only problem with this solution is that the serialization output isn't very deterministic. The order of truncation is strongly dependent on the order that reflections finds the properties. You could solve this (with a perf hit) by sorting before recursing.
SupportedTypes needed the right types
My JavaScriptConverter couldn't live in the same assembly as my model. If you plan to reuse this converter code, you'll probably run into the same problem.
To solve this I had to pre-traverse the object tree, keeping a HashSet<Type> of already seen types (to avoid my own infinite recursion), and pass that to the JavaScriptConverter before registering it.
Looking back on my solution, I would now use code generation templates to create a list of the entity types. This would be much more foolproof (it uses simple iteration), and have much better perf since it would produce a list at compile time. I'd still pass this to the converter so it could be reused between models.
My final solution
I threw out that code and tried again :)
I simply wrote code to project onto new types ("ViewModel" types - in your case, it would be service contract types) before doing my serialization. The intention of my code was made more explicit, it allowed me to serialize just the data I wanted, and it didn't have the potential of slipping in queries on accident (e.g. serializing my whole DB).
My types were fairly simple, and I didn't need most of them for my view. I might look into AutoMapper to do some of this projection in the future.