LINQ to Entity cannot use System.Object.GetValue - entity-framework

Below find a method that does not work. We fail on the line query.Select(...
Below that find a method with hard coded object property names which does work. But, this method is obviously not dynamic, nor flexible. There may be many properties of a Customer I may wish to search on.
The error string is at bottom. I get it that somehow the LINQ to Entity is unable to deal with conversion of GetValue to some sort of TSQL. Would anyone know of how I might code this up?
public List<Customer> GetForQuery(params Tuple<string, string>[] keyValuePairs) {
using (var db = new DBEntities()) {
var availableProperties = typeof(Customer).GetTypeInfo().DeclaredProperties.ToList();
var query = db.Customers.Select(c => c);
foreach (Tuple<string, string> pair in keyValuePairs) {
PropertyInfo pi = availableProperties.First(p => p.Name.Equals(pair.Item1));
if (pi == null)
continue;
query = query.Where(u => pi.GetValue(u, null).ToString().StartsWith(pair.Item2));
}
var results = query.Select(c => c).ToList();
return results;
}
}
How I might call the above:
CustomerController custController = new CustomerController();
List<Customer> results = custController.GetForQuery(Tuple.Create<string, string>("FName", "Bob" ));
The working fixed method:
public List<Customer> GetForQuery(string firstName = "", string lastName = "", string phoneNumber = "") {
using (var db = new DBEntities()) {
var query = db.Customers.Select(c => c);
if (firstName.HasContent())
query = query.Where(u => u.FName.StartsWith(firstName));
if (lastName.HasContent())
query = query.Where(u => u.LName.StartsWith(lastName));
if (phoneNumber.HasContent())
query = query.Where(u => u.EveningPhone.StartsWith(phoneNumber));
var results = query.Select(c => c).ToList();
return results;
}
}
ERROR:
LINQ to Entities does not recognize the method 'System.Object GetValue(System.Object, System.Object[])' method, and this method cannot be translated into a store expression.

Related

How to loop through dbcontext all dbset in Entity Framework Core to get count?

I have 20 Dbsets in my context which I want to get the row count for each dbset to make sure all dbset row count is 0. To get the count for one dbset, this is my code:
var person = context.Persons.Count();
Is there a way to loop through the context, get the count for each dbset dynamically?
There is solution. Usage is simple:
var tablesinfo = ctx.GetTablesInfo();
if (tablesinfo != null)
{
var withRecords = tablesinfo
.IgnoreQueryFilters()
.Where(ti => ti.RecordCount > 0)
.ToArray();
}
Extension returns IQueryable<TableInfo> and you can reuse this query later. Probably you will need to filter out Views, but I think you can handle that. Note that IgnoreQueryFilters can be important if you have Global Query Filters defined.
What extension do:
It scans Model for entity types registered for particular DbContext and generates big Concat of Count queries. Here we have to do that via grouping by constant value.
Schematically it will generate the following LINQ query:
var tablesinfo =
ctx.Set<Entity1>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "Entity1", RecordCount = g.Count()})
.Concat(ctx.Set<Entity2>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "Entity2", RecordCount = g.Count()}))
.Concat(ctx.Set<Entity3>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "Entity3", RecordCount = g.Count()}))
...
Which wll be converted to the following SQL:
SELECT "Entity1" AS TableName, COUNT(*) AS RecordCount FROM Entity1
UNION ALL
SELECT "Entity2" AS TableName, COUNT(*) AS RecordCount FROM Entity2
UNION ALL
SELECT "Entity3" AS TableName, COUNT(*) AS RecordCount FROM Entity3
...
Implementation:
public static class QueryableExtensions
{
public class TableInfo
{
public string TableName { get; set; } = null!;
public int RecordCount { get; set; }
}
public static IQueryable<TableInfo> GetTablesInfo(this DbContext ctx)
{
Expression query = null;
IQueryProvider provider = null;
var ctxConst = Expression.Constant(ctx);
var groupingKey = Expression.Constant(1);
// gathering information for MemberInit creation
var newExpression = Expression.New(typeof(TableInfo).GetConstructor(Type.EmptyTypes));
var tableNameProperty = typeof(TableInfo).GetProperty(nameof(TableInfo.TableName));
var recordCountProperty = typeof(TableInfo).GetProperty(nameof(TableInfo.RecordCount));
foreach (var entityType in ctx.Model.GetEntityTypes())
{
var entityParam = Expression.Parameter(entityType.ClrType, "e");
var tableName = entityType.GetTableName();
// ctx.Set<entityType>()
var setQuery = Expression.Call(ctxConst, nameof(DbContext.Set), new[] {entityType.ClrType});
// here we initialize IQueryProvider, which is needed for creating final query
provider ??= ((IQueryable) Expression.Lambda(setQuery).Compile().DynamicInvoke()).Provider;
// grouping paraneter has generic type, we have to specify it
var groupingParameter = Expression.Parameter(typeof(IGrouping<,>).MakeGenericType(typeof(int), entityParam.Type), "g");
// g => new TableInfo { TableName = "tableName", RecordCount = g.Count() }
var selector = Expression.MemberInit(newExpression,
Expression.Bind(tableNameProperty, Expression.Constant(tableName)),
Expression.Bind(recordCountProperty,
Expression.Call(typeof(Enumerable), nameof(Enumerable.Count), new[] {entityParam.Type}, groupingParameter)));
// ctx.Set<entityType>.GroupBy(e => 1)
var groupByCall = Expression.Call(typeof(Queryable), nameof(Queryable.GroupBy), new[]
{
entityParam.Type,
typeof(int)
},
setQuery,
Expression.Lambda(groupingKey, entityParam)
);
// ctx.Set<entityType>.GroupBy(e => 1).Select(g => new TableInfo { TableName = "tableName", RecordCount = g.Count()}))
groupByCall = Expression.Call(typeof(Queryable), nameof(Queryable.Select),
new[] {groupingParameter.Type, typeof(TableInfo)},
groupByCall,
Expression.Lambda(selector, groupingParameter));
// generate Concat if needed
if (query != null)
query = Expression.Call(typeof(Queryable), nameof(Queryable.Concat), new[] {typeof(TableInfo)}, query,
groupByCall);
else
query = groupByCall;
}
// unusual situation, but Model can have no registered entities
if (query == null)
return null;
return provider.CreateQuery<TableInfo>(query);
}
}

How to EF.Property<T> method works with included queries

I trying write a generic method for create response to datatables ajax request;
public static Response<T> CreateResponse<T>(IQueryable<T> query, Request request) where T : class
{
query = query.AsNoTracking();
var filtered = query;
if (!string.IsNullOrEmpty(request.Search.Value))
{
var keywords = Regex.Split(request.Search.Value, #"\s+").ToList();
request
.Columns
.Where(p => p.Searchable)
.ToList()
.ForEach(c =>
keywords.ForEach(k =>
{
filtered = filtered.Intersect(query.Where(p => EF.Functions.Like(EF.Property<string>(p, c.Name), $"%{k}%")));
})
);
}
var ordered = filtered;
request.Order.ForEach(p => ordered = p.Dir == "asc" ? ordered.OrderBy(q => EF.Property<T>(q, request.Columns[p.Column].Name)) : ordered.OrderByDescending(q => EF.Property<T>(q, request.Columns[p.Column].Name)));
var paged = ordered.Skip(request.Start).Take(request.Length);
return new Response<T> { draw = request.Draw, recordsTotal = query.Count(), recordsFiltered = filtered.Count(), data = paged.ToList() };
}
My problem is, when query parameter is IIncludableQueryable EF.Property method can't locate sub properties. For example;
DataTables.CreateResponse<Rayon>(context.Rayons.Include(p=>p.User), parameters);
EF.Property<T>.Property<string>(p, "Name") is working but, EF.Property<T>.Property<string>(p, "User.Name") is not working, exception message is "EF.Property called with wrong property name."
Sorry for bad English.

An sql query returns empty when it should not?

I have this sql query which perform an foreing key validation, and validated whether both entries uphold a certain standard.
The query should return something (since the data I pass in should fail), as long there is data in the database.
But what I've noticed is that, when I in the same transaction insert data and then perform the validation in the same transaction the validation query returns nothing, when it in actuality should return numerous rows.
if I do the same step as above, meaning I already have committed data in the database, and insert the same data, my validation script returns the excepted rows?
is there any limitations in regards to using transcations?
in here I insert data
private IEnumerable<long> PersistEntityRegistrations(IEnumerable<EntityRegistration> payload, string entityName, NpgsqlTransaction transaction)
{
IEnumerable<string> customAttributeNames = GetEntitySpecificAttributes(entityName);
IEnumerable<string> allAttributeNames =
EntityRegistration.standardAttributeNames.Union(customAttributeNames);
var insertNewRegistrationSql =
#$"INSERT INTO ""{entityName}_registration"" ({string.Join(",", allAttributeNames)})
SELECT * FROM unnest({string.Join(",", allAttributeNames.Select(x => "#" + x))})
RETURNING row_id;";
NpgsqlCommand insertRegistrations = new NpgsqlCommand(insertNewRegistrationSql, transaction.Connection, transaction);
DateTime now = DateTime.UtcNow;
NpgsqlParameter entity_id = new NpgsqlParameter("entity_id", payload.Where(x => x.EntityId.HasValue).Select(x => x.EntityId ?? 0).ToArray());
insertRegistrations.Parameters.Add(entity_id);
NpgsqlParameter entryName = new NpgsqlParameter("entry_name", payload.Select(x => x.EntryName).ToArray());
insertRegistrations.Parameters.Add(entryName);
NpgsqlParameter registrationBy = new NpgsqlParameter("registration_by", payload.Select(x => x.RegistrationBy).ToArray());
insertRegistrations.Parameters.Add(registrationBy);
NpgsqlRange<DateTime>[] valid = payload.Select(x => x.Valid).ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter("valid", valid));
NpgsqlRange<DateTime>[] registration = payload.Select(x => new NpgsqlRange<DateTime>(now, lowerBoundIsInclusive: true, DateTime.MaxValue, upperBoundIsInclusive: true)).ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter("registration", registration));
Entity entity = schemaService.ReadSchema().Entities.SingleOrDefault(x => x.InternalName == entityName);
Relation relation = schemaService.ReadSchema().Relations.SingleOrDefault(x => x.InternalName == entityName);
ICollection<Attribute> entityAttributeDefinitions = entity != null ? entity.Attributes : relation.Attributes ;
foreach (Attribute attributeDefinition in entityAttributeDefinitions)
{
IEnumerable<object> attributeValues = payload.Where(x => x.Attributes.ContainsKey(attributeDefinition.InternalName))
.Select(x => x.Attributes[attributeDefinition.InternalName]);
switch (attributeDefinition.Type)
{
case DataType.Dropdown:
case DataType.WeekDay:
case DataType.Text:
var castedStringValues = attributeValues.Cast<string?>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedStringValues));
break;
case DataType.Number:
var castedNumberValues = attributeValues.Cast<decimal>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedNumberValues));
break;
case DataType.Lookup:
var castedLookupValues = attributeValues.Cast<decimal>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedLookupValues));
break;
case DataType.DateTime:
var castedDateTimeValues = attributeValues.Cast<DateTime?>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedDateTimeValues));
break;
case DataType.Boolean:
// Npgsql is currently not able to handle bool?[], hence this exact type cannot handle null values.
// This is currently support in v. 5 but since EFCore has a hard dependency on v.4
// which forces us to use bool[].
var castedBooleanValues = attributeValues.Cast<bool>().ToArray();
insertRegistrations.Parameters.Add(new NpgsqlParameter(attributeDefinition.InternalName, castedBooleanValues));
break;
}
}
insertRegistrations.Prepare();
insertRegistrations.CommandTimeout = 0;
var insertedRowIds = new List<long>();
using (NpgsqlDataReader dataReader = insertRegistrations.ExecuteReader())
{
while (dataReader.Read())
{
insertedRowIds.Add((long)dataReader[0]);
}
}
return insertedRowIds;
}
Here I do the validation
private IEnumerable<(string, long, NpgsqlRange<DateTime>, bool, NpgsqlRange<DateTime>, long, string, bool)> DetectLookupLifespanViolations(string entityName, IEnumerable<long> entity_ids,
NpgsqlTransaction transaction)
{
var sql = File.ReadAllText(Path.Combine("SQLQueries", "IntegrityCheck.sql"));
var integrityCheckCommand = new NpgsqlCommand(sql, transaction.Connection, transaction);
integrityCheckCommand.Parameters.Add(new NpgsqlParameter<long[]>("#entity_ids", entity_ids.ToArray()));
integrityCheckCommand.Parameters.Add(new NpgsqlParameter<string>("table_name", entityName));
var integrity_breached =
new List<(string, long, NpgsqlRange<DateTime>, bool, NpgsqlRange<DateTime>, long, string, bool)>();
var has_values = new NpgsqlCommand($#"select count(*) from {entityName}_registration", transaction.Connection, transaction);
var result = has_values.ExecuteScalar();
Console.WriteLine(result);
using NpgsqlDataReader dataReader = integrityCheckCommand.ExecuteReader();
while (dataReader.Read())
{
bool hasDateTimeIntegrity = (bool)(dataReader.GetValue(3) ?? true);
if (!hasDateTimeIntegrity)
{
integrity_breached.Add((dataReader.GetFieldValue<string>(0),
dataReader.GetFieldValue<long>(1),
dataReader.GetFieldValue<NpgsqlRange<DateTime>>(2),
dataReader.GetFieldValue<bool>(3),
dataReader.GetFieldValue<NpgsqlRange<DateTime>>(4),
dataReader.GetFieldValue<long>(5),
dataReader.GetFieldValue<string>(6),
dataReader.GetFieldValue<bool>(7)
));
}
}
return integrity_breached;
}
The both called within a method were
public bool InsertEntityRegistrations(string entityName, IEnumerable<EntityRegistration> payload)
{
using var connection = new NpgsqlConnection(configuration.GetConnectionString("PostgreSQL"));
connection.Open();
using var transaction = connection.BeginTransaction();
....
IEnumerable<long> insertedRowIds = PersistEntityRegistrations(payload, entityName, transaction);
....
var registrationsCausingIntegrityIssues = DetectLookupLifespanViolations(entityName, allEntityIds, transaction);
...
}
}

Entity Framework - LINQ - Use Expressions in Select

I am using within my code some EF LINQ expressions to keep complex queries over my model in one place:
public static IQueryable<User> ToCheck(this IQueryable<User> queryable, int age, bool valueToCheck = true)
{
return queryable.Where(ToBeReviewed(age, valueToCheck));
}
public static Expression<Func<User, bool>> ToCheck(int age, bool valueToCheck = true)
{
return au => au.Status == UserStatus.Inactive
|| au.Status == UserStatus.Active &&
au.Age.HasValue && au.Age.Value > age;
}
I am then able to use them in queries:
var globalQuery = db.Users.ToCheck(value);
And also in selects:
var func = EntityExtensions.ToCheck(value);
var q = db.Department.Select(d => new
{
OrdersTotal = d.Orders.Sum(o => o.Price),
ToCheck = d.Users.AsQueryable().Count(func),
})
What I am trying to achieve is to actually use the same expression/function within a select, to evaluate it for each row.
var usersQuery = query.Select(au => new {
Id = au.Id,
Email = au.Email,
Status = au.Status.ToString(),
ToBeChecked = ???, // USE FUNCTION HERE
CreationTime = au.CreationTime,
LastLoginTime = au.LastLoginTime,
});
I am pretty that threre would be a way using plain EF capabilities or LINQKit, but can't find it.
Answering my own question :)
As pointed by #ivan-stoev, the use of Linqkit was the solution:
var globalQueryfilter = db.Users.AsExpandable.Where(au => au.Department == "hq");
var func = EntityExtensions.ToCheck(value);
var usersQuery = globalQueryfilter.Select(au => new
{
Id = au.Id,
Email = au.Email,
Status = au.Status.ToString(),
ToBeChecked = func.Invoke(au),
CreationTime = au.CreationTime,
LastLoginTime = au.LastLoginTime,
});
return appUsersQuery;
It's required to use the AsExpandable extension method from Linqkit along with Invoke with the function in the select method.
I want to add one more example:
Expression<Func<AddressObject, string, string>> selectExpr = (n, a) => n == null ? "[no address]" : n.OFFNAME + a;
var result = context.AddressObjects.AsExpandable().Select(addressObject => selectExpr.Invoke(addressObject, "1"));
Also, expression can be static in a helper.
p.s. please not forget to add "using LinqKit;" and use "AsExpandable".

Return unique values only in Webapi controller

I have the following line in a WebApi controller;
string Codes = i.Products.FirstOrDefault().Code
As the line states, it gets the code, from the first Product.
But, what I really want it to do is to get all unique codes, and return them as a comma separated string.
So, let's say, there are 6 related products, and they have the following codes:
45
54
45
120
54
45
Right now, the statement just returns "45", given the above data.
But I want the statement above to return "45, 54, 120" (as a string).
How do I do this?
Complete code:
public System.Data.Entity.DbSet<WebAPI.Models.Product> Products { get; set; }
private ApplicationDbContext db = new ApplicationDbContext();
var product = await db.Products.Select(i =>
new ProductDTO()
{
Id = i.Id,
Created = i.Created,
Title = i.Title,
Codes = i.Products.FirstOrDefault().Code
}).SingleOrDefaultAsync(i => i.Id == id);
To convert to a comma separated list:
IEnumerable<string> distinctCodes = i.Products.Select(product => product.Code).Distinct();
return string.Join(",", distinctCodes);
But it's probably better if your controller returns a collection of string instead of a concatenated string.
Edit, after OP code update:
var DBProduct = await db.Products.SingleOrDefaultAsync(i => i.Id == id);
IEnumerable<string> productCodes = DBProduct.Products.Select(p => p.Code).Distinct();
var product = new ProductDTO()
{
Id = DBProduct.Id,
Created = DBProduct.Created,
Title = DBProduct.Title,
Codes = string.Join(",", productCodes)
};
I think Distinctand String.Join works for you.Please try this:
var product = await db.Products.AsEnumerable()//Turn AsEnumarable
.Select(i =>
new ProductDTO()
{
Id = i.Id,
Created = i.Created,
Title = i.Title,
Codes = string.Join(",",
i.Products.Select(l => l.Code).Distinct())
}).SingleOrDefaultAsync(i => i.Id == id);