GroupBy Clause: To be able to pass in a Column name as a string - group-by

In EF, Order By allows a column name to be used to eg OrderBy("Description")
I need to be able to do some thing similar with GroupBy
Other posts have solutions when the column type is known
var groupByExpressionGN2 = GetGroupByExpressionGuidNull<DebtWaiver>("PersonUID");
...
// in the query
.GroupBy(groupByExpression2)
// the Expression function
private static Expression<Func<TEntity,Guid?>> GetGroupByExpressionGuidNull<TEntity>(string property)
{
var item = Expression.Parameter(typeof(TEntity), "gb");
var itemProperty = Expression.PropertyOrField(item, property);
var lambda = Expression.Lambda<Func<TEntity, Guid?>>(itemProperty, item);
return lambda;
}
But my users may select 1 of any columns by which to group by
So how can I make the function above return an expression for group by
I have tried this:
public static Expression<Func<T, object>> GetMember<T>(string memberName)// where T : EntityObject
{
ParameterExpression pe = Expression.Parameter(typeof(T), "p");
System.Reflection.PropertyInfo pi = typeof(T).GetProperty(memberName);
return (Expression<Func<T, object>>)Expression.Lambda<Func<T, object>>(Expression.Convert(Expression.Property(pe, pi), typeof(object)), pe);
}
but it produces : p =>Convert(p.PersonUID)
instead of:p =>p.PersonUID
Regards
GregJF

After a bit more testing (and a good night's sleep) I got the second method to work (Thanks to JA Rreyes )
My issue was that I was using this:
var groupByExpressionGN2 = GetGroupByExpressionGuidNull<DebtWaiver>("PersonUID");
...
// in the query
.GroupBy(groupByExpression2)
I should have being doing this:
var groupByExpression2 = GetMember<DebtWaiver>("PersonUID");
...
// in the query
.GroupBy(groupByExpression2.Compile())
You can use the second method in my original post( GetMember ), but I use this method('cos I like it!): Thanks: Taher Rahgooy
public static Expression<Func<T, object>> GetPropertySelector<T>(string propertyName)
{
var arg = Expression.Parameter(typeof(T), "gb");
var property = Expression.Property(arg, propertyName);
var conv = Expression.Convert(property, typeof(object));
var exp = Expression.Lambda<Func<T, object>>(conv, new ParameterExpression[] { arg });
return exp;
}
Regards
GregJF

Related

How do check for duplicate before AddRange?

This is my sample model,
Name
Age
I have a list of records for which I will use Entity Framework Core AddRange. This is my sample data, say list 1
John, 21
Mike, 18
Rick, 19
Alex, 20
I have another query which is a list of the existing records, say list 2.
John, 21
Alex, 20
I want to compare these 2 lists to get what exists in list 1 but not in list 2.
Mike, 18
Rick, 19
I tried using LINQ except but it can't be used for complex types. I also tried to use .Any like below but it returns nothing.
var result = list1.Where(x=> !list2.Any(y=>x.Name == y.Name));
I do not want to check and insert one record at one time. I want to insert by list of objects using AddRange.
Is there any Nuget package that can easily get this?
For not so big amount of records, you can use the following extension:
var nonExistent = await ctx.Some.GetNonExistentAsync(list2, x => x.Name);
Or with complex key:
var nonExistent = await ctx.Some.GetNonExistentAsync(list2,
x => new { x.Name, x.Other });
It will generate effective SQL to check which records do not exist.
And implementation:
public static class QueryableExtensions
{
public static async Task<List<T>> GetNonExistentAsync<T, TKey>(this IQueryable<T> query, IEnumerable<T> records,
Expression<Func<T, TKey>> keySelector, CancellationToken cancellationToken = default)
{
var recordsList = records.ToList();
var keySelectorCompiled = keySelector.Compile();
var predicate = BuildPredicate(recordsList, keySelector, keySelectorCompiled);
var existent = (await query
.Where(predicate)
.Select(keySelector)
.ToListAsync(cancellationToken))
.ToHashSet();
var result = recordsList.Where(r => !existent.Contains(keySelectorCompiled(r))).ToList();
return result;
}
private static Expression<Func<T, bool>> BuildPredicate<T, TKey>(IEnumerable<T> records, Expression<Func<T, TKey>> keySelector, Func<T, TKey> keySelectorCompiled)
{
var members = CollectMembers(keySelector.Body).ToList();
if (members.Count == 0)
throw new InvalidOperationException("No key found");
Expression? predicate = null;
if (members.Count == 1 && keySelector.Body.NodeType == ExpressionType.MemberAccess)
{
// we can use Contains
var recordsSequence = Expression.Constant(records.Select(keySelectorCompiled));
predicate = Expression.Call(typeof(Enumerable), nameof(Enumerable.Contains), new[] { typeof(TKey) },
recordsSequence, keySelector.Body);
}
else
{
foreach (var record in records)
{
Expression? subPredicate = null;
var recordExpression = Expression.Constant(record);
foreach (var m in members)
{
var memberExpression =
ReplacingExpressionVisitor.Replace(keySelector.Parameters[0], recordExpression, m);
var equality = Expression.Equal(m, memberExpression);
subPredicate = subPredicate == null ? equality : Expression.AndAlso(subPredicate, equality);
}
if (subPredicate == null)
throw new InvalidOperationException(); // should never happen
predicate = predicate == null ? subPredicate : Expression.OrElse(predicate, subPredicate);
}
}
predicate ??= Expression.Constant(false);
return Expression.Lambda<Func<T, bool>>(predicate, keySelector.Parameters);
}
private static IEnumerable<Expression> CollectMembers(Expression expr)
{
switch (expr.NodeType)
{
case ExpressionType.New:
{
var ne = (NewExpression)expr;
foreach (var a in ne.Arguments)
foreach (var m in CollectMembers(a))
{
yield return m;
}
break;
}
case ExpressionType.MemberAccess:
yield return expr;
break;
default:
throw new InvalidOperationException();
}
}
}

Entity Framework - LINQ - Use Expressions in Select

I am using within my code some EF LINQ expressions to keep complex queries over my model in one place:
public static IQueryable<User> ToCheck(this IQueryable<User> queryable, int age, bool valueToCheck = true)
{
return queryable.Where(ToBeReviewed(age, valueToCheck));
}
public static Expression<Func<User, bool>> ToCheck(int age, bool valueToCheck = true)
{
return au => au.Status == UserStatus.Inactive
|| au.Status == UserStatus.Active &&
au.Age.HasValue && au.Age.Value > age;
}
I am then able to use them in queries:
var globalQuery = db.Users.ToCheck(value);
And also in selects:
var func = EntityExtensions.ToCheck(value);
var q = db.Department.Select(d => new
{
OrdersTotal = d.Orders.Sum(o => o.Price),
ToCheck = d.Users.AsQueryable().Count(func),
})
What I am trying to achieve is to actually use the same expression/function within a select, to evaluate it for each row.
var usersQuery = query.Select(au => new {
Id = au.Id,
Email = au.Email,
Status = au.Status.ToString(),
ToBeChecked = ???, // USE FUNCTION HERE
CreationTime = au.CreationTime,
LastLoginTime = au.LastLoginTime,
});
I am pretty that threre would be a way using plain EF capabilities or LINQKit, but can't find it.
Answering my own question :)
As pointed by #ivan-stoev, the use of Linqkit was the solution:
var globalQueryfilter = db.Users.AsExpandable.Where(au => au.Department == "hq");
var func = EntityExtensions.ToCheck(value);
var usersQuery = globalQueryfilter.Select(au => new
{
Id = au.Id,
Email = au.Email,
Status = au.Status.ToString(),
ToBeChecked = func.Invoke(au),
CreationTime = au.CreationTime,
LastLoginTime = au.LastLoginTime,
});
return appUsersQuery;
It's required to use the AsExpandable extension method from Linqkit along with Invoke with the function in the select method.
I want to add one more example:
Expression<Func<AddressObject, string, string>> selectExpr = (n, a) => n == null ? "[no address]" : n.OFFNAME + a;
var result = context.AddressObjects.AsExpandable().Select(addressObject => selectExpr.Invoke(addressObject, "1"));
Also, expression can be static in a helper.
p.s. please not forget to add "using LinqKit;" and use "AsExpandable".

Sort using Expression builder

I am trying to build generic sort method using Expressions.
I came up with the following method. For some reasons the code is breaking at the statement when a nested property is used in the sort expression.
var sortExpression = Expression.Lambda<Func<T, object>>
(Expression.Convert(Expression.Property(sortParam, sortColumn), typeof(object)), sortParam);
private static IQueryable<T> SortQuery<T>(IQueryable<T> query, string sortColumn)
{
if (!string.IsNullOrEmpty(sortColumn))
{
var sortParam = Expression.Parameter(typeof(T), "x");
Expression expr = sortParam;
foreach (var prop in sortColumn.Split('.'))
{
expr = Expression.PropertyOrField(expr, prop);
}
var sortExpression = Expression.Lambda<Func<T, object>>
(Expression.Convert(Expression.Property(sortParam, sortColumn), typeof(object)), sortParam);
return query.OrderBy(sortExpression);
}
return null;
}
Any idea where I am doing it wrong ?
Try:
private static readonly MethodInfo OrderBy = (from x in typeof(Queryable).GetMethods()
where x.Name == "OrderBy"
let pars = x.GetParameters()
where pars.Length == 2
select x).Single();
private static IQueryable<T> SortQuery<T>(IQueryable<T> query, string sortColumn)
{
if (!string.IsNullOrEmpty(sortColumn))
{
var sortParam = Expression.Parameter(typeof(T), "x");
Expression expr = sortParam;
foreach (var prop in sortColumn.Split('.'))
{
expr = Expression.PropertyOrField(expr, prop);
}
var lambda = Expression.Lambda(expr, sortParam);
var orderBy = OrderBy.MakeGenericMethod(typeof(T), expr.Type);
return (IQueryable<T>)orderBy.Invoke(null, new object[] { query, lambda });
}
return null;
}
You have to use reflection to invoke the Queryable.OrderBy. In general, casting to object doesn't work very well with Entity Framework/LINQ to SQL, so you can try it, but there is only a very little chance it will work.

mvc4 expression tree for contains integer

I want to implement expression tree for integers as in below query:
Select * From TableName Where PKID In (1,2,3,4,5)
The relevant extracted code:
protected Expression<Func<T, bool>> GetExpression<T>(string propertyName, string propertyValue, string propertyType)
{
var parameter = Expression.Parameter(typeof(T), "type");
switch (propertyType)
{
case "string":
case "string?":
var stringProperty = Expression.Property(parameter, propertyName);
var stringValue = Expression.Constant(propertyValue.Trim(), typeof(string));
MethodInfo stringMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var stringContainsMethod = Expression.Call(stringProperty, stringMethod, stringValue);
return Expression.Lambda<Func<T, bool>>(stringContainsMethod, parameter);
case "int?":
var intProperty = Expression.Property(parameter, propertyName);
var intValues = Expression.Constant(propertyValue.Trim(), typeof(string));
MethodInfo intMethod = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var intContainsMethod = Expression.Call(intProperty, intMethod, intValues); // this line gives error
return Expression.Lambda<Func<T, bool>>(intContainsMethod, parameter);
}
}
// Calling above function for string works. Products is a table in Database and Entity framework generated the class based on database first.
var filterLambda = GetExpression<Products>("ProductName", "chicken", "string?"); // No error here, works great
// Calling above function for int gives error
var filterLambda = GetExpression<Products>("ProductId", "1,2,3,4,5", "int?"); // Error here
I get error:
Method 'Boolean Contains(System.String)' declared on type 'System.String' cannot be called with instance of type 'System.Int32'
Of course it is pretty much clear that the column ProductId is integer and the value is string. Integer doesnt have contains so I thought I should use strings contains method. Please can any one tell me how do I solve this?

Get only a specified field in MongoDB with C#

first time i'm using MongoDB.
I have read this example:
SELECT a,b FROM users WHERE age=33
db.users.find({age:33}, {a:1,b:1})
But I can't translate it into C#. Can anyone help me?
I have translated your query below using the new C# driver (2.2)
var mongoClient = new MongoClient(""mongodb://127.0.0.1:27017"");
var database = mongoClient.GetDatabase("databaseName");
IMongoCollection<Users> _collection = database.GetCollection<Users>("Users");
var condition = Builders<Users>.Filter.Eq(p => p.age, 33);
var fields = Builders<Users>.Projection.Include(p => p.a).Include(p => p.b);
var results= _collection.Find(condition).Project<Users>(fields).ToList().AsQueryable();
You can do it using SetFields method of MongoCursor class, below full example:
var server = MongoServer.Create(connectionString);
var db = _server.GetDatabase("dbName");
var users = db.GetCollection("users");
var cursor = users.FindAs<DocType>(Query.EQ("age", 33));
cursor.SetFields(Fields.Include("a", "b"));
var items = cursor.ToList();
you can use anonymous class
public class User
{
public int age;
public string a;
public string b;
}
var collection = db.GetCollection<User>("Users");
var results = collection.Find(Builders<User>.Filter.Eq(user => user.age, 33))
.Project(u => new { u.a, u.b }).ToList();
//create user class
//(not sure how your class looks like)
public class User
{
public int age;
public string a;
public string b;
}
//then you can use LINQ easily
var server = MongoServer.Create(connectionString);
var db = server.GetDatabase("dbName");
var usersCollection = db.GetCollection<User>("users");
var filteredCollection = usersCollection.AsQueryable().Where(x=> x.age < 33).Where(x=> x.a != null).Contains(x=> x.b != null);