Improve performance in LINQ To Entities query (aggregation) - entity-framework

I have this EF model:
class Reception
{
public string Code { get; set; }
public virtual List<Row> { get; set; }
}
class Row
{
public string Item { get; set; }
public int Quantity { get; set; }
public float Weight { get; set; }
}
Is there a way to improve the following LINQ To Entities query?
dbContext.Receptions.Select(r => new
{
code = r.Code,
quantitySum = r.Rows.Sum(e => e.Quantity),
weightSum = r.Rows.Sum(e => e.Weight),
});
I'm worried about doing twice the "r.Rows" part.
Should I not be worried?

Aggregate queries (and especially these containing multiple aggregates) are better translated to SQL if based on GroupBy, because that's the natural SQL construct for aggregates.
So if you want better translation and suffer the code readability, the query in question can be turned into left outer join + group by like this:
var query = dbContext.Receptions
.SelectMany(r => r.Rows.DefaultIfEmpty(), (r, e) => new
{
r.Code,
Quantity = (int?)e.Quantity ?? 0,
Weight = (float?)e.Weight ?? 0,
})
.GroupBy(e => e.Code, (key, g) => new
{
code = key,
quantitySum = g.Sum(e => e.Quantity),
weightSum = g.Sum(e => e.Weight),
});
which translates to something like this
SELECT
1 AS [C1],
[GroupBy1].[K1] AS [Code],
[GroupBy1].[A1] AS [C2],
CAST( [GroupBy1].[A2] AS real) AS [C3]
FROM ( SELECT
[Join1].[K1] AS [K1],
SUM([Join1].[A1]) AS [A1],
SUM([Join1].[A2]) AS [A2]
FROM ( SELECT
[Extent1].[Code] AS [K1],
CASE WHEN ([Extent2].[Quantity] IS NULL) THEN 0 ELSE [Extent2].[Quantity] END AS [A1],
CASE WHEN ([Extent2].[Weight] IS NULL) THEN cast(0 as real) ELSE [Extent2].[Weight] END AS [A2]
FROM [dbo].[Receptions] AS [Extent1]
LEFT OUTER JOIN [dbo].[Rows] AS [Extent2] ON [Extent1].[Code] = [Extent2].[Reception_Code]
) AS [Join1]
GROUP BY [K1]
) AS [GroupBy1]
which is the best you can get from EF6 for this particular query.

Normally, EF should convert this to a fairly efficient DB query, using SUM functions on the DB side as well. But if you want to be sure, use your SQL Server Profiler to analyze the query.
I ran this code quickly for you and this is what EF does:
SELECT
[Project2].[Id] AS [Id],
[Project2].[Code] AS [Code],
[Project2].[C1] AS [C1],
CAST( [Project2].[C2] AS real) AS [C2]
FROM
(SELECT
[Project1].[Id] AS [Id],
[Project1].[Code] AS [Code],
[Project1].[C1] AS [C1],
(SELECT SUM([Extent3].[Weight]) AS [A1] FROM [dbo].[Rows] AS [Extent3] WHERE [Project1].[Id] = [Extent3].[Reception_Id]) AS [C2]
FROM
(SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Code] AS [Code],
(SELECT SUM([Extent2].[Quantity]) AS [A1] FROM [dbo].[Rows] AS [Extent2] WHERE [Extent1].[Id] = [Extent2].[Reception_Id]) AS [C1]
FROM
[dbo].[Receptions] AS [Extent1]
)AS [Project1]
)AS [Project2]
Surely, we can write a better query, with less subselects, but in reality, this query is not something SQL server will trip over.

Related

Entity Framework generates datetime2 parameters for comparing to a date column -- how to match the data type of the column

Is there anyway to force Entity Framework into sending date parameters to match the column data type? This is causing a very costly key lookup in some of our queries.
An example generated query runs in minutes, I grabbed from SQL Server profiler:
exec sp_executesql N'SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(1) AS [A1]
FROM [table] AS [Extent1]
WHERE ( EXISTS (SELECT
1 AS [C1]
FROM [search] AS [Extent2]
WHERE ([Extent1].[Number] = [Extent2].[Number]) AND ([Extent2].[SearchInstance] = #p__linq__0)
)) AND ([Extent1].[Date] >= #p__linq__1) AND ([Extent1].[Date] <= #p__linq__2)
) AS [GroupBy1]',N'#p__linq__0 uniqueidentifier,#p__linq__1 datetime2(7),#p__linq__2 datetime2(7)',#p__linq__0='1A530478-17F8-442E-B718-32049086717F',#p__linq__1='2012-07-24',#p__linq__2='2015-07-24'
If I manually change the types of the parameters to date, this query executes in 0 seconds.
exec sp_executesql N'SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(1) AS [A1]
FROM [table] AS [Extent1]
WHERE ( EXISTS (SELECT
1 AS [C1]
FROM [search] AS [Extent2]
WHERE ([Extent1].[Number] = [Extent2].[Number]) AND ([Extent2].[SearchInstance] = #p__linq__0)
)) AND ([Extent1].[Date] >= #p__linq__1) AND ([Extent1].[Date] <= #p__linq__2)
) AS [GroupBy1]',N'#p__linq__0 uniqueidentifier,#p__linq__1 date,#p__linq__2 date',#p__linq__0='1A530478-17F8-442E-B718-32049086717F',#p__linq__1='2012-07-24',#p__linq__2='2015-07-24'
Using ef codefirst, I specifed .HasColumType("date") but did not change the parameter type being sent.
** names of tables & columns have been changed to protect the innocent.
Crudely, using a DbInterceptor.
using System.Data;
using System.Data.Common;
using System.Data.Entity.Infrastructure.Interception;
using System.Data.SqlClient;
using System.Linq;
public static class SomewhereDoThis
{
public static void Register()
{
DbInterception.Add(new DateInterceptor());
}
}
public class DateInterceptor : IDbCommandInterceptor
{
/* .. */
public void NonQueryExecuting(DbCommand command, DbCommandInterceptionContext<int> interceptionContext)
{
ShrinkDates(command);
}
public void ReaderExecuting(DbCommand command, DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
ShrinkDates(command);
}
public void ReaderExecuting(DbCommand command, DbCommandInterceptionContext interceptionContext)
{
ShrinkDates(command);
}
public void ScalarExecuting(DbCommand command, DbCommandInterceptionContext<object> interceptionContext)
{
ShrinkDates(command);
}
private static void ShrinkDates(DbCommand command)
{
command.Parameters
.OfType<SqlParameter>()
.Where(p => p.SqlDbType == SqlDbType.DateTime2)
.Where(p => p.Value != DBNull.Value)
.Where(p => p.Value is DateTime)
.Where(p => ((DateTime)p.Value).TimeOfDay == TimeSpan.Zero)
.ToList()
.ForEach(p => p.SqlDbType = SqlDbType.Date);
}
}

Conditional row count in linq to nhibernate doesn't work

I want to translate following simple sql query into Linq to NHibernate:
SELECT NewsId
,sum(n.UserHits) as 'HitsNumber'
,sum(CASE WHEN n.UserHits > 0 THEN 1 ELSE 0 END) as 'VisitorsNumber'
FROM UserNews n
GROUP BY n.NewsId
My simplified UserNews class:
public class AktualnosciUzytkownik
{
public virtual int UserNewsId { get; set; }
public virtual int UserHits { get; set; }
public virtual User User { get; set; } // UserId key in db table
public virtual News News { get; set; } // NewsId key in db table
}
I've written following linq query:
var hitsPerNews = (from n in Session.Query<UserNews>()
group n by n.News.NewsId into g
select new { NewsId = g.Key, HitsNumber = g.Sum(x => x.UserHits),
VisitorsNumber = g.Count(x => x.UserHits > 0) }).ToList();
But generated sql just ignores my x => x.UserHits > 0 statement and makes unnecessary 'left outer join':
SELECT news1_.NewsId AS col_0_0_,
CAST(SUM(news0_.UserHits) AS INT) AS col_1_0_,
CAST(COUNT(*) AS INT) AS col_2_0_
FROM UserNews news0_
LEFT OUTER JOIN News news1_
ON news0_.NewsId=news1_.NewsId
GROUP BY news1_.NewsId
How Can I fix or workaround this issue? Maybe this can be done better with QueryOver syntax?
I finally found answer to my question, my solution is based on an answer to this question:
My QueryOver code (I still don't know how to do it in Linq to NHibernate):
UserHitsDto adDtoAlias = null;
var userHits = Session.QueryOver<UserNews>()
.Select(Projections.Group<UserNews>(c => c.News.NewsId)
.WithAlias(() => adDtoAlias.NewsId),
Projections.Sum<UserNews>(x => x.UserHits)
.WithAlias(() => adDtoAlias.HitsNumber),
Projections.Sum(Projections.Conditional(
Restrictions.Where<UserNews>(f => f.UserHits > 0),
Projections.Constant(1),
Projections.Constant(0)
)).WithAlias(() => adDtoAlias.VisitorsNumber)
)
.TransformUsing(Transformers.AliasToBean<UserHitsDto>())
.List<UserHitsDto>();
It produces following tsql:
SELECT this_.NewsId AS y0_,
SUM(this_.UserHits) AS y1_,
SUM((
CASE
WHEN this_.UserHits > #p0
THEN #p1
ELSE #p2
END)) AS y2_
FROM UserNews this_
GROUP BY this_.NewsId
where #p0 = 0, #p1 = 1, #p2 = 0

Entity Framework code first - many to many filtering

Small EF question.
I have a many to many relationship mapped in EF. X..Y
So when I have one X there is a property X.Ys.
Now what I want to do is use a Linq Query to get several X's but I don't want to have all Y's inside the selected X's.
I want the Y's filtered on Y.RegistrationDate > Date.Today.
So when I have one X and itterate through .Y's I will only get future Y's.
UPDATE
This works, resulting in S having distinct ug's with it's relationship only containing upcoming events.
But don't tell me this cant be simplified??!!
var t = (from ug in uof.Ugs.All().ToList()
from upcomingEvent in ug.Events
where upcomingEvent.Date >= DateTime.Today
select new
{
ug,
upcomingEvent
}).ToList();
var s = (from ug in t.Select(x => x.ug).Distinct()
select new UG
{
Id = ug.Id,
Name = ug.Name,
Description = ug.Description,
WebSite = ug.WebSite,
Events = ug.Events.Where(x => x.Date >= DateTime.Today).ToList()
}).ToList();
UPDATE2
Added image to show that even with basic context manipulation I'm still getting 2 events, event when I take 1!
exampledebugimage
EF does not support this scenario as you want it, what you can do however is this:
var date = DateTime.Date;
var query = from x in Xs
select new
{
X = x
Ys = x.Ys.Where(i = > i.RegistrationDate > date)
}
Which will give you a collection of X's with their corresponding Y's that match your criteria.
Have you tried?:
var query = Xs
.Select(x => new { x, yCol = x.YCol.Where(y => y.Date >= DateTime.Today) })
.AsEnumerable()
.Select(x => x.x)
.ToList();
See: http://blogs.msdn.com/b/alexj/archive/2009/10/13/tip-37-how-to-do-a-conditional-include.aspx
All those .ToList you use will mean you load the whole table from the db before filtering. So watch out for that.
UPDATE: As fixup doesn't work with Many-To-Many
As Slauma mentioned in the comments make sure you don't use this technique if you are going to submit the changes as the changetracking will think you altered the collection. Or even better make sure you use .AsNoTracking() which will improve performance anyway.
We can use the same solution as above but slightly different for many-to-many. See this example:
[TestClass]
public class ContextTest
{
[TestMethod]
public void FixupTest()
{
Database.SetInitializer(new DropCreateDatabaseAlways<Context>());
using (var db = new Context())
{
db.Groups.Add(new Group
{
Name = "G1",
Users = new List<User>{
new User{ Name = "M"},
new User{Name = "S"}
}
});
db.SaveChanges();
}
using (var db = new Context())
{
var group = db.Groups
.Select(g => new { g, Users = g.Users.Where(u => u.Name == "M") })
.AsEnumerable()
.Select(g => {
g.g.Users = g.Users.ToList();
return g.g;
})
.First();
Assert.AreEqual(1, group.Users.Count);
}
}
}
public class User
{
public int ID { get; set; }
public string Name { get; set; }
public ICollection<Group> Groups { get; set; }
}
public class Group
{
public int ID { get; set; }
public string Name { get; set; }
public ICollection<User> Users { get; set; }
}
The test pass and the generated sql is:
SELECT
[Project1].[ID] AS [ID],
[Project1].[Name] AS [Name],
[Project1].[C1] AS [C1],
[Project1].[ID1] AS [ID1],
[Project1].[Name1] AS [Name1]
FROM ( SELECT
[Extent1].[ID] AS [ID],
[Extent1].[Name] AS [Name],
[Join1].[ID] AS [ID1],
[Join1].[Name] AS [Name1],
CASE WHEN ([Join1].[Group_ID] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C1]
FROM [dbo].[Groups] AS [Extent1]
LEFT OUTER JOIN (SELECT [Extent2].[Group_ID] AS [Group_ID], [Extent3].[ID] AS [ID], [Extent3].[Name] AS [Name]
FROM [dbo].[GroupUsers] AS [Extent2]
INNER JOIN [dbo].[Users] AS [Extent3] ON [Extent3].[ID] = [Extent2].[User_ID] ) AS [Join1] ON ([Extent1].[ID] = [Join1].[Group_ID]) AND (N'Mikael' = [Join1].[Name])
) AS [Project1]
ORDER BY [Project1].[ID] ASC, [Project1].[C1] ASC

EF Code First: Retrieving a base type queries all derived type tables

I'm having an odd issue with EF 4.1 Code First where even though I have configured an entity to generate columns for its inherited properties, it still joins to the inherited type's table.
Here are my classes:
public class Human
{
public int Id { get; set; }
public string Name { get; set; }
}
public class SuperHuman : Human
{
public int Id { get; set; }
public string Powers { get; set; }
}
public class MarvelDbContext : DbContext
{
public DbSet<Human> Humans { get; set; }
public DbSet<SuperHuman> SuperHumans { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Entity<SuperHuman>().Map(m => m.MapInheritedProperties());
}
}
Here is the resulting query:
SELECT
[Limit1].[C3] AS [C1],
[Limit1].[C1] AS [C2],
[Limit1].[C2] AS [C3],
[Limit1].[C4] AS [C4]
FROM ( SELECT TOP (1)
[UnionAll1].[Id] AS [C1],
[UnionAll1].[Name] AS [C2],
CASE WHEN ([UnionAll1].[C2] = 1) THEN '0X' ELSE '0X0X' END AS [C3],
CASE WHEN ([UnionAll1].[C2] = 1) THEN CAST(NULL AS varchar(1)) ELSE [UnionAll1].[C1] END AS [C4]
FROM (SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
CAST(NULL AS varchar(1)) AS [C1],
cast(1 as bit) AS [C2]
FROM [dbo].[Humen] AS [Extent1]
UNION ALL
SELECT
[Extent2].[Id] AS [Id],
[Extent2].[Name] AS [Name],
[Extent2].[Powers] AS [Powers],
cast(0 as bit) AS [C1]
FROM [dbo].[SuperHumans] AS [Extent2]) AS [UnionAll1]
) AS [Limit1]
I only want it to query the Humans table.
That is how EF behaves. If you query Human set it always goes over all derived tables as well because SuperHuman is still Human and because of that instances of SuperHuman are valid results of query for humans.

EF Linq to entities query generating UNION for TPC CTP5 code-first entity

I have the following entities:
public class User
{
public int Id { get; set; }
public string Name { get; set; }
}
public class HappyUser : User
{
public bool IsHappy { get; set; }
}
I am configuring the entities using Table Per Concrete Type (TPC) in order to generate a User table and a HappyUser table. I want the HappyUser table to include the properties of the User class, and I don't want any relationship between the two tables.
I configure the entities as follows:
public class UserTest : DbContext
{
public DbSet<User> Users { get; set; }
public DbSet<HappyUser> HappyUsers { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<HappyUser>().Map(m =>
{
m.MapInheritedProperties();
m.ToTable("HappyUser");
});
}
}
The tables are generated correctly, but when I query the tables, EF generates a UNION on the User and HappyUser table. The query is as follows:
UserTest db = new UserTest();
var users = from u in db.Users
select u;
var happyUsers = from u in db.Users.OfType<HappyUser>()
select u;
The SQL for the Users generates a UNION. This is not what I expect or want. I'd like to simply retrieve the rows from the Users table.
SELECT
CASE WHEN ([UnionAll1].[C2] = 1) THEN '0X' ELSE '0X0X' END AS [C1],
[UnionAll1].[Id] AS [C2],
[UnionAll1].[Name] AS [C3],
CASE WHEN ([UnionAll1].[C2] = 1) THEN CAST(NULL AS bit) ELSE [UnionAll1].[C1] END AS [C4]
FROM (SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
CAST(NULL AS bit) AS [C1],
cast(1 as bit) AS [C2]
FROM [dbo].[User] AS [Extent1]
UNION ALL
SELECT
[Extent2].[Id] AS [Id],
[Extent2].[Name] AS [Name],
[Extent2].[IsHappy] AS [IsHappy],
cast(0 as bit) AS [C1]
FROM [dbo].[HappyUser] AS [Extent2]) AS [UnionAll1]
The SQL for the HappyUsers works as expected.
SELECT
'0X0X' AS [C1],
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
[Extent1].[IsHappy] AS [IsHappy]
FROM [dbo].[HappyUser] AS [Extent1]
Any ideas what I am doing wrong? Or is this a defect in EF?
HappyUsers are Users. Hence, db.Users should return both. The EF is correct, here.
However, the EF does have a limitation: There is no way to (in L2E, anyway) return results of only one type. Due to the Liskov Substitution Principal, you don't generally want to do this. But if you do, you can do it in ESQL, and there are (somewhat tedious) workarounds for L2E.
Bottom line: If you find yourself wanting to do this, rethink your design. Inheritance is probably not the right relationship in this case.
#Craig Stuntz is right, since both types are related, EF will keep that relation intact.
If you want to decouple both User types, then you could use an abstract base class.
public abstract class AbstractUser
{
public int Id { get; set; }
public string Name { get; set; }
}
public class User : AbstractUser
{
}
public class HappyUser : AbstractUser
{
public bool IsHappy { get; set; }
}
EF will now treat both entities as seperate and there's no need to call MapInheritedProperties() anymore.
Your select statements would then look like:
var users = db.Users.Where(...);
var happyUsers = db.HappyUsers.Where(...);