I have broken out the where clauses in my applications to their own libraries and then pass them to the database at run time. This was done to help with testing.
I attached a log to the db to see what the generated sql was and I noticed that the where clause was not listed. The data is still filtered, so that causes me to believe that the data is filtered in the app instead of in the database. Can anyone confirm this? Is there a better way to do this?
Here is a sample:
Where Clause
private Func<Message, bool> GetSearchWhere(string q, string type)
{
return m => m.Name.Contains(q) && m.Type == type;
}
DB Call
private List<Messages> GetMessages(Func<Message, bool> where)
{
return Messaging.Messages.Where(where).ToList();
}
The data is indeed filtered in memory by LINQ to Objects. When you pass Func<T, bool> to Where method, you actually are calling Enumerable.Where. If you want to call Queryable.Where (thus making the filtering in the database), then you need to pass Expression<Func<T, bool> instead.
To do that, all you need is to change the signature of your methods:
private Expression<Func<Message, bool>> GetSearchWhere(string q, string type)
{
return m => m.Name.Contains(q) && m.Type == type;
}
private List<Messages> GetMessages(Expression<Func<Message, bool>> where)
{
return Messaging.Messages.Where(where).ToList();
}
Related
I created a web api project using .net core and entity framework.
This uses a stored procedure which returns back most of the properties of a database table defined by entity framework.
The entity framwrok does not bring back all the columns of the table. And I get an error when I call the api complaining it cannot find the missing columns when I execute the stored procedure using ,
_context.Set<TableFromSql>().FromSql("execute dbo.spr_GetValue").ToList();
I created another model class which defines the properties brought back from the SP( called NewClass).
_context.Set<NewClass>().FromSql("execute dbo.spr_GetValue").ToList();
This works, but just wanted to check if there is a convention that the SP should only return the model classes from the database.
The SQL query must return data for all properties of the entity or query type
For this limitation, it is caused when mapping the sql query result to Model. It loop through the properties in model and try to retrive the values from query result. If the model properties are not exist in query result, it will throw error.
If you want to return required columns instead of all columns, one options is to define the returned model by Query.
For your demo code, you may define this in OnModelCreating.
builder.Query<TableFromSql>();
Note, for this way, you need to make sure all properties in TableFromSql exist in execute dbo.spr_GetValue.
For another way, you may implement your own FromSql which will add condition to check whether the properties are exist in query result.
public static class DbContextExtensions
{
public static List<T> RawSqlQuery<T>(this DbContext context,string query)
{
using (var command = context.Database.GetDbConnection().CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
context.Database.OpenConnection();
using (var result = command.ExecuteReader())
{
var entities = new List<T>();
return DataReaderMapToList<T>(result);
}
}
}
public static List<T> DataReaderMapToList<T>(IDataReader dr)
{
List<T> list = new List<T>();
T obj = default(T);
while (dr.Read())
{
obj = Activator.CreateInstance<T>();
foreach (PropertyInfo prop in obj.GetType().GetProperties())
{
if (ColumnExists(dr, prop.Name))
{
if (!object.Equals(dr[prop.Name], DBNull.Value))
{
prop.SetValue(obj, dr[prop.Name], null);
}
}
}
list.Add(obj);
}
return list;
}
public static bool ColumnExists(IDataReader reader, string columnName)
{
return reader.GetSchemaTable()
.Rows
.OfType<DataRow>()
.Any(row => row["ColumnName"].ToString() == columnName);
}
}
Use above code like :
var result = _context.RawSqlQuery<ToDoItemVM>("execute [dbo].[get_TodoItem]");
I've taken a look at this answer on how to dynamically create an OrderBy expression in Entity Framework. But I'd like to also build a dynamic Where expression. Something along the lines of this:
public IEnumerable<InventoryItem> GetAll(string filterBy, object value)
{
var results = new List<InventoryItem>();
using (var db = new InventoryDb())
{
if (QueryHelper.PropertyExists<InventoryItem>(filterBy))
{
var query = db.rri_InventoryItems.WhereByProperty(filterBy, value);
foreach(var item in query.Where(expr))
{
results.Add(ConvertItem(item));
}
}
}
return results;
}
Passing in the property to filter by and a value as ab object. Queryable has two methods for Where that both take two parameters, so I am not even sure which is the proper one.
And it's at this point I get a little more lost. I'm not sure how to refactor the original OrderByProerty method to provide a WhereByProperty. I know what I have here is completely wrong. I'm not sure what to do with it.
Ideally, I'd want to extend this even more by providing a collection of objects that could be used to build a query with ands and or operators.
Queryable has two methods for Where that both take two parameters, so I am not even sure which is the proper one.
You need the one that receives Expression<Func<T, bool>> predicate.
Here is how you can build dynamically a predicate similar to (T item) => item.Property == value:
public static partial class QueryableExtensions
{
public static IQueryable<T> WhereEquals<T>(this IQueryable<T> source, string member, object value)
{
var item = Expression.Parameter(typeof(T), "item");
var memberValue = member.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
var memberType = memberValue.Type;
if (value != null && value.GetType() != memberType)
value = Convert.ChangeType(value, memberType);
var condition = Expression.Equal(memberValue, Expression.Constant(value, memberType));
var predicate = Expression.Lambda<Func<T, bool>>(condition, item);
return source.Where(predicate);
}
}
I've tried to write it in such a way so you can step over the code in order to understand what it does. The only line that might need some explanation is:
var memberValue = member.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
This is a simple way of handling nested properties like obj.Prop1.Prop2 etc. If you don't need such capability, you can simply use this instead:
var memberValue = Expression.PropertyOrField(item, member);
I didn't need nested properties (yet). I modified your code slightly and have this that is working:
public static IQueryable<T> WhereEquals<T>(
this IQueryable<T> source, string propertyName, object value)
{
if (typeof(T).GetProperty(propertyName, BindingFlags.IgnoreCase |
BindingFlags.Public | BindingFlags.Instance) == null)
{
return null;
}
ParameterExpression parameter = Expression.Parameter(typeof(T), "item");
Expression whereProperty = Expression.Property(parameter, propertyName);
Expression constant = Expression.Constant(value);
Expression condition = Expression.Equal(whereProperty, constant);
Expression<Func<T, bool>> lambda = Expression.Lambda<Func<T, bool>>(condition,parameter);
return source.Where(lambda);
}
I am working on a JPA/Jersey web app and want to know if there is a better way to update a record. Currently, I am doing:
#PUT
#Path("update/{id}")
#Produces("application/json")
#Consumes("application/x-www-form-urlencoded")
public Response createDevice(
#PathParam("id") int id,
#FormParam("name") String name,
#FormParam("type") int type
/*MultivaluedMap<String, String> formParams*/
) {
try {
Devices newDevice = entityManager.find(Devices.class, id);
if(name==null){name=newDevice.getName();}
if(type != newDevice.getType()){newDevice.setType(type);}
newDevice.setName(name);
//newDevice.setType(type);
entityManager.merge(newDevice);
return Response.status(201).entity(new ResponseObject("success")).build();
} finally {
entityManager.close();
}
}
This works, but if my Devices table had more fields, I would have to check for equality of ALL fields with the values on the original object to see if they've changed, so that my
entityManager.merge(newDevice);
will only change the values passed in.
Is there a better way to do this?
I am using Entity Framework version 4. I need to compare a large (~1 million record) SQL Server table to a longish (~2000) array of complex objects returned from a web service. Five different properties need to be compared to determine whether an instance of the complex object is already in the database.
I created a function that returns an expression for use in .Where and .Any methods. It looks like this (where A is the complex object, and tblA is the EF class):
function Expression<tblA, bool> GetSearchPredicate(A a)
{
return ta => ta.Field1.Equals(a.Field1)
&& ta.Field2.Equals(a.Field2)
&& ta.Field3.Equals(a.Field3)
&& ta.Field4.Equals(a.Field4)
&& ta.Field5.Equals(a.Field5);
}
This works. And I can compare all 2000 instances of A by doing this:
IEnumerable<A> objects = [web service call];
var result = objects.Select(a => !db.tblA.Any(GetSearchPredicate(a)));
That works, too. But it's slow. So I looked into building a utility method that could build an expression that could be transmitted down to the database directly through EF.
I used the code in this question as a basis for building that utility method. The example in that question shows comparing a single property to a series of constants, while my version would have to compare multiple properties to multiple constants. My best effort is below:
public static IQueryable<TEntity> WhereIn<TEntity>
(
this ObjectQuery<TEntity> query,
IEnumerable<Expression<Func<TEntity, bool>>> predicates
)
{
if (predicates == null) throw new ArgumentNullException("predicates");
IEnumerable<ParameterExpression> p = predicates.Select(pred => pred.Parameters.Single()).ToArray();
IEnumerable<Expression> equals = predicates.Select(value =>
(Expression)value.Body);
Expression bigEqual = equals.Aggregate((accumulate, equal) =>
Expression.Or(accumulate, equal));
var result1 = Expression.Lambda<Func<TEntity, bool>>(bigEqual, p.First());
var result = query.Where(result1);
return result;
}
This would be invoked like this:
IEnumerable<A> objects = [web service call];
var result = db.tblA.WhereIn(objects.Select(a => GetSearchPredicate(a)));
What I get is a message saying that "ta" (the placeholder for the TEntity object) is not bound. I thought this was because I had multiple expressions (the variable predicates) being combined, and maybe this message was being thrown because I was only passing the parameter from the first of the predicates IEnumerable. But this happens even if predicates is one expression long.
I am reasonably sure, based on the method I linked to, that I could build an expression comparing each of the five properties to a constant (the values of A.Field1 through A.Field5), rather than passing in the parameter predicates that already has them assembled into a series of expressions. But I would rather not, since that would require my method to know that it's working with types A and tblA, and that's the opposite of generic and general-purpose. (It'd also be complex and messy.)
I hope the examples I've shown explain what I want to do. Can it be done in a generic way?
You will need to replace the parameter in the predicate bodies with a single parameter. Something like this should work:
public static Expression<Func<T, bool>> BuildOr<T>(
IEnumerable<Expression<Func<T, bool>>> predicates)
{
Expression body = null;
ParameterExpression p = null;
Expression<Func<T, bool>> first = null;
foreach (Expression<Func<T, bool>> item in predicates)
{
if (first == null)
{
first = item;
}
else
{
if (body == null)
{
body = first.Body;
p = first.Parameters[0];
}
var toReplace = item.Parameters[0];
var itemBody = ReplacementVisitor.Transform(item, toReplace, p);
body = Expression.OrElse(body, itemBody);
}
}
if (first == null)
{
throw new ArgumentException("Sequence contains no elements.", "predicates");
}
return (body == null) ? first : Expression.Lambda<Func<T, bool>>(body, p);
}
private sealed class ReplacementVisitor : ExpressionVisitor
{
private IList<ParameterExpression> SourceParameters { get; set; }
private Expression ToFind { get; set; }
private Expression ReplaceWith { get; set; }
public static Expression Transform(
LambdaExpression source,
Expression toFind,
Expression replaceWith)
{
var visitor = new ReplacementVisitor
{
SourceParameters = source.Parameters,
ToFind = toFind,
ReplaceWith = replaceWith,
};
return visitor.Visit(source.Body);
}
private Expression ReplaceNode(Expression node)
{
return (node == ToFind) ? ReplaceWith : node;
}
protected override Expression VisitConstant(ConstantExpression node)
{
return ReplaceNode(node);
}
protected override Expression VisitBinary(BinaryExpression node)
{
var result = ReplaceNode(node);
if (result == node) result = base.VisitBinary(node);
return result;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (SourceParameters.Contains(node)) return ReplaceNode(node);
return SourceParameters.FirstOrDefault(p => p.Name == node.Name) ?? node;
}
}
Your WhereIn method then becomes:
public static IQueryable<TEntity> WhereIn<TEntity>(
this ObjectQuery<TEntity> query,
IEnumerable<Expression<Func<TEntity, bool>>> predicates)
{
if (predicates == null) throw new ArgumentNullException("predicates");
var predicate = BuildOr(predicates);
return query.Where(predicate);
}
public static Record FindOrCreate(this IEnumerable<Record> ienumerable, string name)
{
if (ienumerable.Where(element => element.Name == name).FirstOrDefault() != null)
return ienumerable.Where(element => element.Name == name).FirstOrDefault();
else
{
IEnumerable<Record> result = ienumerable.Concat(new[] {new Record(name, "")});
ienumerable = result;
return ienumerable.Where(element => element.Name == name).FirstOrDefault();
}
}
I know there is no Add method to IEnumerable but I want to stay with IEnumerable instead of ICollection and in really rare cases I have to add something. My methodmalways returns null if it doesn't find element.
Code EDITED !
So, now result has all elements that I want, but method still return null object. I don't know if is it possible to switch objects in extension methods? (like I did ienumerable=result) ?
I think you're trying to bend the IEnumerable<T> beyond it's purpose.
An IEnumerable<T> is just a sequence of elements and that's why IEnumerable<T> does not have an Add<T> method in it's interface. It's read only.
I would recommend changing your implementation of your extension method.
So the question is "how can I modify the code to not return null if the element isn't found, but return a new item"?
public static Record FindOrCreate(this IEnumerable<Record> source, string name)
{
Func<Record,bool> pred = element => element.Name == name;
return source.Any(pred) ? source.FirstOrDefault(pred) : new Record(name, "");
}
or
public static Record FindOrCreate(this IEnumerable<Record> source, string name)
{
return source.FirstOrDefault(element => element.Name == name) ?? new Record(name, "");
}
If the question is "how can I add a new item to the IEnumerable<Record> source, the short answer is you can't. The longer answer is cast the source to an ICollection and then add it. But at that point, you may as well just specify that you need an ICollection<T>
Note, I don't mean "you shouldn't" when I say "you can't", really, IEnumberable has no way to add items. Think about what it means to try to add a new number to the following seq
public IEnumerable<int> ShortSeq()
{
yield return 0;
}
var seq = ShortSeq();