Entity framework performance when using methods in the select clause - entity-framework

I'm looking at the SQL generated when performing simple select queries. I'm using code first with the sample blog context from nuget.
If the following is run:
BlogContext _context = new BlogContext();
var comments = _context.Comments.Select(c => new CommentReadOnly {Author = c.Author});
var count = comments.Count();
The following SQL is produced:
SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(1) AS [A1]
FROM [dbo].[Comments] AS [Extent1]
) AS [GroupBy1]
Where the count is performed in the SQL which is expected.
However if I change the code to look like this:
BlogContext _context = new BlogContext();
var comments = _context.Comments.Select(c => new CommentReadOnly {Author = c.Author});
var count = comments.Count();
private CommentReadOnly ToCommentReadOnly(Comment comment)
{
return new CommentReadOnly
{
Author = comment.Author,
};
}
The following SQL is produced:
SELECT
[Extent1].[ID] AS [ID],
[Extent1].[PostID] AS [PostID],
[Extent1].[Text] AS [Text],
[Extent1].[Author] AS [Author]
FROM [dbo].[Comments] AS [Extent1]
With the count done in code.
The reason (I think) is because the first is returned as IQueryable where as the second is IEnumerable.
Is it possible to return the second query as IQueryable without executing the SQL?
The reason I ask is that I'm creating a generic repository layer that can query my entities and convert them to the required type (in the example above comment might have a couple of different 'readonly' objects). I don't want the SQL executing so early as paging may be done or other filtering in different situations.

I don't see any difference in those two queries. However, I guess you want to return a IQueryable object to the client so the client can perform further filtering and get the count from there.
You can simply return the object without doing the select and let the client do the rest.
return _context.Comments
The client can perform additional filtering on this IQueryable object

I think in your second query you execute the function ToCommentReadOnly() so this can't be done entirely in SQL and you end up with a Linq To Objects (IEnumerable).
But you state that you want to return an IQueryable from your Repository. This is not a recommended practice! The code to access the data should be hidden inside your repository otherwhise you will run into problems.
Say for example that your repository (which encapsulates your ObjectContext) goes out of scope after which you try to enumerate the IQueryable result the repository gave you. This will throw an error because the IQueryable can't be executed anymore.
If you expose an IQueryable from your Repository you give the end user of your Repository all the freedom they want in building their own queries, which is the thing you want to avoid by adding a repository!
So returning an IEnumerable from your Repository is a good thing :)

Related

Automapper projectTo generated SQL has no where clause

Using Entity Framework Core I'm trying to map an entity containing a child collection.
Without projectTo the generated SQL includes a where clause and only fetches the entities that belong in the child collection. When I add a projectTo, the result is correct but looking at the SQL I see no where clause. It appears to load all the entities of the child type and then do a where in memory.
I've tried all sorts of mappings on the collection but nothing seems to change this behavior.
Is there a way to improve this query?
The code in question is:
var parent = _context
.Parents
.Where(a => a.FamilyId == familyId && a.Id == id)
.Include(r => r.Children)
//.ProjectTo<ParentDetailViewModel>()
.AsNoTracking() // In case projection contains sub entities
.SingleOrDefault();
Without the project to I see the following SQL in the profiler
exec sp_executesql N'SELECT [r].[Id], [r].[FieldOne], [r].[Cid], [r].[FieldTwo], [r].[ParentId]
FROM [Child] AS [r]
WHERE EXISTS (
SELECT TOP(2) 1
FROM [Parent] AS [a]
WHERE (([a].[FamilyId] = #__familyId_0) AND ([a].[Id] = #__id_1)) AND ([r].[ParentId] = [a].[Id]))
ORDER BY [r].[ParentId]',N'#__familyId_0 int,#__id_1 int',#__familyId_0=1,#__id_1=1
With project I see this. No where clause included.
SELECT [r].[ParentId], [r].[Id]
FROM [Child] AS [r]
As a general rule of thumb, I put the ProjectTo as the last thing before your query materialization. Put ProjectTo after AsNoTracking - HOWEVER, the AsNoTracking is pointless. With ProjectTo, you're skipping entities entirely, going straight from SQL to DTO, no entities involved at all.
Also, the Include is pointless. With ProjectTo/Select, EF knows exactly what entities it needs to join, because they're in the Select projection!
Apparently caused by an Entity Framework Core bug. Fixed in 1.1.0 preview

Attempting to use EF/Linq to Entities for dynamic querying and CRUD operations

(as advised re-posting this question here... originally posted in msdn forum)
I am striving to write a "generic" routine for some simple CRUD operations using EF/Linq to Entities. I'm working in ASP.NET (C# or VB).
I have looked at:
Getting a reference to a dynamically selected table with "GetObjectByKey" (But I don't want anything from cache. I want data from database. Seems like not what this function is intended for).
CRM Dynamic Entities (here you can pass a tablename string to query) looked like the approach I am looking for but I don't get the idea that this CRM effort is necessarily staying current (?) and/or has much assurance for the future??
I looked at various ways of drilling thru Namespaces/Objects to get to where I could pass a TableName parameter into the oft used query syntax var query = (from c in context.C_Contacts select c); (for example) where somehow I could swap out the "C_Contacts" TEntity depending on which table I want to work with. But not finding a way to do this ??
Slightly over-simplyfing, I just want to be able to pass a tablename parameter and in some cases some associated fieldnames and values (perhaps in a generic object?) to my routine and then let that routine dynamically plug into LINQ to Entity data context/model and do some standard "select all" operations for parameter table or do a delete to parameter table based on a generic record id. I'm trying to avoid calling the various different automatically generated L2E methods based on tablename etc...instead just trying to drill into the data context and ultimately the L2E query syntax for dynamically passed table/field names.
Has anyone found any successful/efficient approaches for doing this? Any ideas, links, examples?
The DbContext object has a generic Set() method. This will give you
from c in context.Set<Contact>() select c
Here's method when starting from a string:
public void Test()
{
dynamic entity = null;
Type type = Type.GetType("Contract");
entity = Activator.CreateInstance(type);
ProcessType(entity);
}
public void ProcessType<TEntity>(TEntity instance)
where TEntity : class
{
var result =
from item in this.Set<TEntity>()
select item;
//do stuff with the result
//passing back to the caller can get more complicated
//but passing it on will be fine ...
}

Does converting from DbSet to IEnumerable cause the query to execute?

I have the following 2 methods in my Logs repository.
public IEnumerable<Log> GetAll()
{
var db = new CasLogEntities();
return db.Logs;
}
public DbSet<Log> GetAllSet()
{
var db = new CasLogEntities();
return db.Logs;
}
The only difference is that one returns an IEnumerable of Log and the other a DbSet of Log.
In My Asset Controller I have the following code
var allLogs = _logRepo.GetAllSet();
var Logs = (from log in allLogs
group log by log.DeviceId
into l
select new {DeviceId = l.Key, TimeStamp = l.Max(s => s.TimeStamp)}).ToList();
Now the issues is that I am getting massive performance difference in the group by statement depending on which one of the repo methods I call.
getAllSet which returns the DbSet is lightning fast,
GetAll returns IEnumerable is reallllyyyy slow.
Can anybody explain this. I was thinking that the conversion of the DbSet to the IEnumerable in the GetAll was causing the Query to execute and hence I was doing the group by on a massive in memory set. While as the GetAllSet was deferring the query execution until the "ToList()" and hence was doing the group by work on the server.
Is this correct? Is there another way to explain this?
I would prefer for the GetAll to return the IEnumerable as I am more familiar with it and its a bit easier for testing.
No, converting to IEnumerable<T> does not cause it to execute.
It does, on the other hand, take the query into object space, so the SQL generated will be different when you project onto an anonymous type. Watch SQL Server Profiler to see the difference.
This would explain the performance difference.
If you return IQueryable<T> instead of IEnumerable<T>, the SQL/performance should be identical.

How do you deal with Linq to NHibernate's Fetch exception when selecting aggregates?

I'm using LINQ to NHibernate's IQueryable implementation on a asp.net mvc Grid (telerik specifically), where I know I'll need to fetch something eagerly for this particular grid.
So I have query that looks something like this:
var query = from s in repository.Query<MyClass>()
orderby s.Property.Name, s.Name
select s;
query = query.Fetch(x => x.Property);
Now, if I execute query.ToList(), everything is fine, and I can verify that it works in an integration test.
It's awesome.
However, if I execute query.Count() or something else that aggregates the query, I get an exception:
Query specified join fetching, but the
owner of the fetched association was
not present in the select list
[FromElement{explicit,not a collection
join,fetch join,fetch non-lazy
properties,classAlias=0,role=,tableName=[Property],tableAlias=property1,origin=MyClass
myclass0_,colums={myclass0_.PropertyGuid
,className=Property}}]
[.Count(.Fetch(.ThenBy(.OrderBy(NHibernate.Linq.NhQueryable`1[MyClass],
Quote((s, ) => (s.Property.Name)), ),
Quote((s, ) => (s.Name)), ), Quote((x,
) => (x.Property)), ), )]
I know that it's trying to tell me that I can't eagerly fetch Property because MyClass isn't in the select, but the problem is that Count() is actually being called via the Grid, and handled externally from my code.
All I'm supposed to need to do is give the grid an IQueryable and it should be able to handle paging, sorting, etc. by itself.
Has anyone else had to get around this issue with NHibernate Fetching and how did you resolve it?
var query = from s in repository.Query<MyClass>()
orderby s.Property.Name, s.Name
select s;
query = query.Fetch(x => x.Property).ToList();
and after you can go and do
query.Count()
and it should be in working order.
As to why is that i suspect that is something to do
AsEnumerable()
or
AsQueryable()
but not sure why is this
i had similar problem and this solved it...

MVC 1.0 + EF: Does db.EntitySet.where(something) still return all rows in table?

In a repository, I do this:
public AgenciesDonor FindPrimary(Guid donorId) {
return db.AgenciesDonorSet.Include("DonorPanels").Include("PriceAdjustments").Include("Donors").First(x => x.Donors.DonorId == donorId && x.IsPrimary);
}
then down in another method in the same repository, this:
AgenciesDonor oldPrimary = this.FindPrimary(donorId);
In the debugger, the resultsview shows all records in that table, but:
oldPrimary.Count();
is 1 (which it should be).
Why am I seeing all table entries retrieved, and not just 1? I thought row filtering was done in the DB.
If db.EntitySet really does fetch everything to the client, what's the right way to keep the client data-lite using EF? Fetching all rows won't scale for what I'm doing.
You will see everything if you hover over the AgenciesDonorSet because LINQ to Entities (or SQL) uses delayed execution. When the query is actually executed, it is just retrieving the count.
If you want to view the SQL being generated for any query, you can add this bit of code:
var query = queryObj as ObjectQuery; //assign your query to queryObj rather than returning it immediately
if (query != null)
{
System.Diagnostics.Trace.WriteLine(context);
System.Diagnostics.Trace.WriteLine(query.ToTraceString());
}
Entity Set does not implement IQueryable, so the extension methods that you're using are IEnumerable extension methods. See here:
http://social.msdn.microsoft.com/forums/en-US/linqprojectgeneral/thread/121ec4e8-ce40-49e0-b715-75a5bd0063dc/
I agree that this is stupid, and I'm surprised that more people haven't complained about it. The official reason:
The design reason for not making
EntitySet IQueryable is because
there's not a clean way to reconcile
Add\Remove on EntitySet with
IQueryable's filtering and
transformation ability.