we are using EF 6.1 and code first along with the Unit of work and repository pattern.
the confusion we are facing is that at many places its written that if we use Skip & Take on a Dbset, the paging is automatically handled.
but when we see the query being created, it does not contain any clause for paging.
kindly assist what is the possible case , the sample code is as follows:
public virtual IEnumerable<TEntity> GetSorted(Func<TEntity, Object> order,int skip, int take, params Expression<Func<TEntity, object>>[] includes)
{
IQueryable<TEntity> query = dbSet;
foreach (var include in includes)
{
query = dbSet.Include(include);
}
IEnumerable<TEntity> data = dbSet.OrderBy(order).Skip(skip).Take(take).ToList();
//IEnumerable<TEntity> data = query.OrderBy(order).Skip(skip).Take(take).ToList();
return data;
}
any help appreciated..
Your order parameter needs to be of Expression<Func<TEntity, TSortBy>> type in order to match the signature of Queryable.OrderBy(). When you're passing a delegate (Func<TEntity, Object>), the only OrderBy() overload that will match this argument is Enumerable.OrderBy() and not the Queryable.OrderBy() that you're expecting (both are extension-methods of course).
When the sorting is being done by Enumerable.OrderBy() (Linq-to-Objects), the query must be executed first (since Linq-to-Objects is meant for working with objects that are already in memory).
However, when Queryable.OrderBy() is invoked, the order-by expression is being passed to the Linq provider that is then capable of parsing and translating it to a store query (SQL in this case).
So, when you're passing Func<TEntity, Object> to OrderBy, you're getting IEnumerable<TEntity> in return, and further (chained) calls to Skip() and Take() are no longer intercepted by Linq-To-Entities.
This is how your method signature should look like:
public virtual IEnumerable<TEntity> GetSorted<TSortedBy>(Expression<Func<TEntity, TSortedBy>> order, int skip, int take, params Expression<Func<TEntity, object>>[] includes)
See MSDN
Related
I have a service that passes in parameters for how much I want to include for navigation properties. Based upon the boolean args it concatenates an entity list to include each required foreign entity.
At runtime I want to include either no navigation entities or many.
What I can't do is daisy chain with .Include().Include as I don't know which and how many to include based around passed in args.
I want to achieve this, but I don't seem to be able to pass in a comma separated entity list. Any ideas?
var res = db.Entity.Include(entityListCommaSeparated).Where(_=>_.ID == ID).FirstOrDefault();
This looks like a repository pattern, and generally gets messy if you want to try and "hide" EF / the DbContext from calling code.
A couple options you can consider:
Down the complexity rabit hole: use a params Expression<Func<TEntity, object>>[] includes in your applicable repository methods, and then be prepared to also pass OrderBy expressions, as well as pagination values when you want to return multiple entities.
THrough the simplicity mirror: Embrace IQueryable as a return type and let the consumers handle Includes, OrderBy's, Counts/Any/Skip/Take/First/ToList, and .Select() as they need.
Option 1:
public Order GetById(int id, params Expression<Func<Order, object>>[] includes)
{
var query = db.Orders.Where(x => x.ID == id);
// This part can be moved into an extension method or a base repository method.
if(includes.Any)
includes.Aggregate(query, (current, include) =>
{
current.Include(include);
}
// Don't use .FirstOrDefault() If you intend for 1 record to be returned, use .Single(). If it really is optional to find, .SingleOrDefault()
return query.Single();
}
//ToDo
public IEnumerable<Order> GetOrders(/* criteria?, includes?, order by?, (ascending/descending) pagination? */)
{ }
// or
public IEnumerable<Order> GetOrdersByCustomer(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
// plus..
public IEnumerable<Order> GetOrdersByDate(/* includes?, order by?, (ascending/descending) pagination? */)
{ }
public bool CustomerHasOrders(int customerId)
{ }
public bool OrderExists(int id)
{ }
public int OrdersOnDate(DateTime date)
{ }
// etc. etc. etc.
Keep in mind this doesn't handle custom order by clauses, and the same will be needed for methods that are returning lists of entities. Your repository is also going to need to expose methods for .Any() (DoesExist) because everyone loves checking for #null on every return. :) Also .Count().
Option 2:
public IQueryable<Order> GetById(int id)
{
return db.Orders.Where(x => x.ID == id);
}
public IQueryable<Order> GetOrders()
{
return db.Orders.AsQueryable();
}
Callers can grok Linq and .Include() what they want before calling .Single(), or do a .Any().. They may not need the entire entity graph so they can .Select() from the entity and related entities without .Include() to compose and execute a more efficient query to populate a ViewModel / DTO. GetById might be used in a number of places so we can reduce duplication and support it in the repository. We don't need all of the filter scenarios etc, callers can call GetOrders and then filter as they see fit.
Why bother with a repository if it just returns DBSets?
Centralize low-level data filtering. For instance if you use Soft Deletes (IsActive) or are running multi-tenant, or explicit authorization. These common rules can be centralized at the repository level rather than having to remembered everywhere a DbSet is touched.
Testing is simpler. While you can mock a DbContext, or point it at an in-memory database, mocking a repository returning IQueryable is simpler. (Just populate a List<TEntity> and return .AsQueryable().
Repositories handle Create and Delete. Create to serve as a factory to ensure that all required data and relationships are established for a viable entity. Delete to handle soft-delete scenarios, cascades/audits etc. beyond what the DB handles behind the scenes.
I am trying to get Envelope's back from a query. Envelope is defined as follows.
[<CLIMutable>]
type Envelope<'T> = {
Id : Guid
StreamId: Guid
Created : DateTimeOffset
Item : 'T }
MyLibAAS.DataStore.MyLibAASDbContext is a EF DbContext written in c#. When I extend it in f# as follows, I get the error: Only parameterless constructors and initializers are supported in LINQ to Entities.
type MyLibAAS.DataStore.MyLibAASDbContext with
member this.GetEvents streamId =
query {
for event in this.Events do
where (event.StreamId = streamId)
select {
Id = event.Id;
StreamId = streamId;
Created = event.Timestamp;
Item = (JsonConvert.DeserializeObject<QuestionnaireEvent> event.Payload)
}
}
If I return the event and map it to Envelope after the fact, it works fine.
type MyLibAAS.DataStore.MyLibAASDbContext with
member this.GetEvents streamId =
query {
for event in this.Events do
where (event.StreamId = streamId)
select event
} |> Seq.map (fun event ->
{
Id = event.Id
StreamId = streamId
Created = event.Timestamp
Item = (JsonConvert.DeserializeObject<QuestionnaireEvent> event.Payload)
}
)
Why does this make a difference? The Envelope type is not even a EF type.
How F# records work
F# records get compiled into .NET classes with read-only properties and a constructor that takes values of all fields as parameters (plus a few interfaces).
For example, your record would be expressed in C# roughly as follows:
public class Envelope<T> : IComparable<Envelope<T>>, IEquatable<Envelope<T>>, ...
{
public Guid Id { get; private set; }
public Guid StreamId { get; private set; }
public DateTimeOffset Created { get; private set; }
public T Item { get; private set; }
public Envelope( Guid id, Guid streamId, DateTimeOffset created, T item ) {
this.Id = id;
this.StreamId = streamId;
this.Created = created;
this.Item = item;
}
// Plus implementations of IComparable, IEquatable, etc.
}
When you want to create an F# record, the F# compiler emits a call to this constructor, supplying values for all fields.
For example, the select part of your query would look in C# like this:
select new Envelope<QuestionnaireEvent>(
event.Id, streamId, event.Timestamp,
JsonConvert.DeserializeObject<QuestionnaireEvent>(event.Payload) )
Entity Framework limitations
It so happens that Entity Framework does not allow calling non-default constructors in queries. There is a good reason: if it did allow it, you could, in principle, construct a query like this:
from e in ...
let env = new Envelope<E>( e.Id, ... )
where env.Id > 0
select env
Entity Framework wouldn't know how to compile this query, because it doesn't know that the value of e.Id passed to the constructor becomes the value of the property env.Id. This is always true for F# records, but not for other .NET classes.
Entity Framework could, in principle, recognize that Envelope is an F# record and apply the knowledge of the connection between constructor arguments and record properties. But it doesn't. Unfortunately, the designers of Entity Framework did not think of F# as a valid use case.
(fun fact: C# anonymous types work the same way, and EF does make an exception for them)
How to fix this
In order to make this work, you need to declare Envelope as a type with default constructor. The only way to do this is to make it a class, not a record:
type Envelope<'T>() =
member val Id : Guid = Guid.Empty with get, set
member val StreamId : Guid = Guid.Empty with get, set
member val Created : DateTimeOffset = DateTimeOffset.MinValue with get, set
member val Item : 'T = Unchecked.defaultof<'T> with get, set
And then create it using property initialization syntax:
select Envelope<_>( Id = event.Id, StreamId = streamId, ... )
Why does moving the select to a Seq.map work
The Seq.map call is not part of the query expression. It does not end up as part of the IQueryable, so it does not end up compiled to SQL by Entity Framework. Instead, EF compiles just what's inside query and returns you the resulting sequence, after fetching it from SQL Server. And only after that you apply Seq.map to that sequence.
The code inside Seq.map is executed on CLR, not compiled to SQL, so it can call anything it wants, including non-default constructors.
This "fix" comes with a cost though: instead of just the fields you need, the whole Event entity gets fetched from DB and materialized. If this entity is heavy, this may have a performance impact.
Another thing to watch out for
Even if you fix the problem by making Envelope a type with default constructor (as suggested above), you'll still hit the next problem: the method JsonConvert.DeserializeObject can't be compiled to SQL, so Entity Framework will complain about it, too. The way you should do it is fetch all fields to the CLR side, then apply whatever non-SQL-compilable transformations you need.
Using LINQ to Entities, everything that happens in the query computational expression is actually executed within the database engine, which may reside on a remote server. Everything outside of it is executed in the running application on the client.
So, in your first snippet, Entity Framework refuses to execute Envelope<'T>'s constructor because, in order to do so, it would need to translate that into SQL commands for the server. This is plainly not something it can guarantee, because the constructor could potentially contain any sort of side effects and .NET framework code - it could request user input, read files from the client's hard disk, whatever.
What EF can do, in your second snippet, is sending its own POCO event objects back to the client, which is then tasked with Seq.mapping them to your fancy Envelopes, which it can do because it's running on your client machine with access to the full .NET framework.
Addendum: So why are parameterless constructors ok? What if I were to call MsgBox() in a parameterless constructor? I think that parameterless constructors work by having the client construct the objects (without knowing the query results), sending them to the server in serialised form, and having the server just fill the object's properties with the query results.
I haven't actually tested that. But F# record types have no parameterless constructors anyway, so the point is moot in your case.
I've ran into a problem while creating an data access layer for my application. I am working with derived entities in Entity Framework. If I try to create an ObjectSet for a derived Entity, I get this Exception:
Blockquote There are no EntitySets defined for the specified entity type 'Type Name'. If 'Type Name' is a derived type, use the base type instead.
I've tried to resolve this problem via Reflection. (if the Entity Type is derived by an Entity Type => get an ObjectSet for the base Type)
I found this: How can I obtain ObjectSet<T> from Entity-Framework at runtime where T is dynamic? but I haven't found how to use Include and Where on the built up ObjectSet.
protected IEnumerable<IDataObject> GetData(Type entityType, Expression<Func<dynamic, bool>> whereClause, Expression<Func<dynamic, dynamic>>[] includes)
{
if (typeof(IDataObject).IsAssignableFrom(entityType.BaseType))
{
return GetData(entityType.BaseType, whereClause, includes);
}
var contextType = this.Context.GetType();
MethodInfo createObjectSetMethod = contextType.GetMethod("CreateObjectSet", new Type[] {}).MakeGenericMethod(entityType);
// Builds up an ObjectSet<EntityType>
dynamic objectSet = createObjectSetMethod.Invoke(this.Context, new object[] { });
dynamic query = objectSet;
if (includes != null)
{
foreach (var include in includes)
{
query = query.Include(include);
}
}
if (whereClause == null)
{
whereClause = (item) => true;
}
query = query.Where(whereClause);
return query.ToList().OfType<IDataObject>();
}
The code runs as intended, as long I don't use the Includes and WhereClause.
When I call this function I dont know the resolved ObjectSet (T-Parameter) at compile time.
Is there any way to use dynamic Expressions in an generic ObjectSet?
Thanks in advance.
The issue looks to be with the
query = query.Include(include);
the var include would be an Expression<Func<dynamic,dynamic>>
The Include Method you access is expecting a string or
Expression<Func<T, TProperty>>
Have you tried to pass the include as a string path? I would expect that to work.
Otherwise you need to construct the expression tree with code
since you cant pass Dynamic objects to
public static IQueryable<T> Include<T, TProperty>(this IQueryable<T> source, Expression<Func<T, TProperty>> path) where T : class;
There is also dynamic linq library to can check out.
System.Linq.Dynamic can be found at following links
http://msdn.microsoft.com/en-US/vstudio/bb894665.aspx
with an intro here http://www.scottgu.com/blogposts/dynquery/dynamiclinqcsharp.zip
This is an easier alternative using text to build expression trees.
Building expressions trees with code is more powerful and flexible but harder.
I generally use a generic repository to boilerplate my EF queries so I have to write limited code and also use caching. The source code for the repository can be found here.
The backbone query within the code is this one below. FromCache<T>() is an IEnumerable<T> extension method that utilizes the HttpContext.Cache to store the query using a stringified representation of the lambda expression as a key.
public IQueryable<T> Any<T>(Expression<Func<T, bool>> expression = null)
where T : class, new()
{
// Check for a filtering expression and pull all if not.
if (expression == null)
{
return this.context.Set<T>()
.AsNoTracking()
.FromCache<T>(null)
.AsQueryable();
}
return this.context.Set<T>()
.AsNoTracking<T>()
.Where<T>(expression)
.FromCache<T>(expression)
.AsQueryable<T>();
}
Whilst this all works it is subject to the N+1 problem for related tables since If I were to write a query like so:
var posts = this.ReadOnlySession.Any<Post>(p => p.IsDeleted == false)
.Include(p => p.Author);
The Include() will have no effect on my query since it has already been run in order to be cached.
Now I know that I can force Entity Framework to use eager loading within my model by removing the virtual prefix on my navigation properties but that to me feels like the wrong place to do it as you cannot predict the types of queries you will be making. To me it feels like something I would be doing in a controller class. What I am wondering is whether I can pass a list of includes into my Any<T>() method that I could then iterate though when I make the call?
ofDid you mean something like...
IQueryable<T> AnyWithInclude<T,I>(Expression<Func<T,bool>> predicate,
Expression<Func<T,I>> includeInfo)
{
return DbSet<T>.where(predicate).include(includeInfo);
}
the call
Context.YourDbSetReference.AnyWithInclude(t => t.Id==someId, i => i.someNavProp);
In response to extra question on as collection.
I realised late, there was an overload on Property. You can just pass a string
This might work but call is not easy. Well I find it hard.
IQueryable<T> GetListWithInclude<I>(Expression<Func<T, bool>> predicate,
params Expression<Func<T, I>>[] IncludeCollection);
so i tried
public virtual IQueryable<T> GetListWithInclude(Expression<Func<T, bool>> predicate,
List<string> includeCollection)
{ var result = EntityDbSet.Where(predicate);
foreach (var incl in includeCollection)
{
result = result.Include(incl);
}
return result;
}
and called with
var ic = new List<string>();
ic.Add("Membership");
var res = context.DbSte<T>.GetListWithInclude( t=>t.UserName =="fred", ic);
worked as before.
In the interest of clarity I'm adding the solution I came up with based upon #soadyp's answer.
public IQueryable<T> Any<T>(Expression<Func<T, bool>> expression = null,
params Expression<Func<T, object>>[] includeCollection)
where T : class, new()
{
IQueryable<T> query = this.context.Set<T>().AsNoTracking().AsQueryable<T>();
if (includeCollection.Any())
{
query = includeCollection.Aggregate(query,
(current, include) => current.Include(include));
}
// Check for a filtering expression and pull all if not.
if (expression != null)
{
query = query.Where<T>(expression);
}
return query.FromCache<T>(expression, includeCollection)
.AsQueryable<T>();
}
Usage:
// The second, third, fourth etc parameters are the strongly typed includes.
var posts = this.ReadOnlySession.Any<Post>(p => p.IsDeleted == false,
p => p.Author);
For my repository using the entity framework i'm using the following method to return a IList.
public virtual IList<TEntity> ToList(Func<TEntity, bool> expression)
{
return Context.CreateQuery<TEntity>("[" + EntitySetName + "]")
.Where<TEntity>(expression).ToList();
}
The expression parameter allows me to alter the result, but this is not to my expectations. Since createquery is first evaluated and afterwards my where is applied. I'm using this with tables with over 2 mil. records.
Does anybody have a solution how to keep the method generic, but with the abillity to control the outcome without loading all records first?
You need to change the type of your argument to one which can be converted to SQL:
public virtual IList<TEntity> ToList(Expression<Func<TEntity, bool>> expression)
Note the Expression.