DefaultIfEmpty mess ordered items in EF and LINQ to Entities? - entity-framework

Question

Given this entity (and this records as examples)
Discount
Amount Percentage
1000 2
5000 4
10000 8
I want to get the percentage to apply to one P.O. Amount
I.E.: Having a P.O. Amount of 15000
if I use
db.Discount
.Where(d => d.Amount <= PO.Amount)
.OrderByDescending(o => o.Amount)
.Select(s => s.Percentage)
.ToList()
.DefaultIfEmpty(0)
.FirstOrDefault();
I get 8 (correct)
but if I use
db.Discount
.Where(d => d.Amount <= PO.Amount)
.OrderByDescending(o => o.Amount)
.Select(s => s.Percentage)
.DefaultIfEmpty(0)
.FirstOrDefault();
I get 2 (incorrect) and items are not ordered any more.
Am I doing an incorrect use of DefaultIfEmpty?

Answer 1

If you are using Entity Framework 6.*, then it is a known bug:
the workaround is to move the DefaultIfEmpty call to after the ToList, which is arguably better as there is no need for the replacement of an empty result set to be done in the database.
Following examples generated with EF 6.1.2 (and "captured" with Microsoft SQL Profiler on a Microsoft SQL Server 2016).
Now... Your "wrong" query:
var res = db.Discounts
.Where(d => d.Amount <= PO.Amount)
.OrderByDescending(o => o.Amount)
.Select(s => s.Percentage)
.DefaultIfEmpty(0)
.FirstOrDefault();
"removes" the OrderBy:
exec sp_executesql N'SELECT
[Limit1].[C1] AS [C1]
FROM ( SELECT TOP (1)
CASE WHEN ([Project1].[C1] IS NULL) THEN cast(0 as bigint) ELSE [Project1].[Percentage] END AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT
[Extent1].[Percentage] AS [Percentage],
cast(1 as tinyint) AS [C1]
FROM [dbo].[Discount] AS [Extent1]
WHERE [Extent1].[Amount] <= #p__linq__0 ) AS [Project1] ON 1 = 1
) AS [Limit1]',N'#p__linq__0 bigint',#p__linq__0=15000
"best" query would be:
var res = db.Discounts
.Where(d => d.Amount <= PO.Amount)
.OrderByDescending(o => o.Amount)
.Select(s => s.Percentage)
.Take(1)
.ToArray()
.DefaultIfEmpty(0)
.First(); // Or Single(), same result but clearer that there is always *one* element
See the Take(1)? It generates a TOP (1):
exec sp_executesql N'SELECT TOP (1)
[Project1].[Percentage] AS [Percentage]
FROM ( SELECT
[Extent1].[Amount] AS [Amount],
[Extent1].[Percentage] AS [Percentage]
FROM [dbo].[Discount] AS [Extent1]
WHERE [Extent1].[Amount] <= #p__linq__0
) AS [Project1]
ORDER BY [Project1].[Amount] DESC',N'#p__linq__0 bigint',#p__linq__0=15000
Then the ToArray() will move the elaboration to C#. You could use .FirstOrDefault() with ?? instead of using DefaultIfEmpty(), but the result would be different if Amount is already nullable (the null returned by FirstOrDefault() is because there are no rows or because the only row found has Amount == null? Who knows :-) ). To solve this problem it becomes a little more complex (in the most general case):
var res = (db.Discounts
.Where(d => d.Amount <= PO.Amount)
.OrderByDescending(o => o.Amount)
.Select(s => new { s.Percentage })
.FirstOrDefault() ?? new { Percentage = (long)0 }
).Percentage;
Here the (long) in (long)0 should be the data type of Percentage. This query gives:
exec sp_executesql N'SELECT TOP (1)
[Project1].[C1] AS [C1],
[Project1].[Percentage] AS [Percentage]
FROM ( SELECT
[Extent1].[Amount] AS [Amount],
[Extent1].[Percentage] AS [Percentage],
1 AS [C1]
FROM [dbo].[Discount] AS [Extent1]
WHERE [Extent1].[Amount] <= #p__linq__0
) AS [Project1]
ORDER BY [Project1].[Amount] DESC',N'#p__linq__0 bigint',#p__linq__0=15000
Other "worse" variant:
var res = db.Discounts
.Where(d => d.Amount <= PO.Amount)
.OrderByDescending(o => o.Amount)
.Select(s => s.Percentage)
.Take(1)
.DefaultIfEmpty(0)
.First();
that gives an overcomplicated query with two TOP (1):
exec sp_executesql N'SELECT
[Limit2].[C1] AS [C1]
FROM ( SELECT TOP (1)
CASE WHEN ([Project2].[C1] IS NULL) THEN cast(0 as bigint) ELSE [Project2].[Percentage] END AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT TOP (1)
[Project1].[Percentage] AS [Percentage],
cast(1 as tinyint) AS [C1]
FROM ( SELECT
[Extent1].[Amount] AS [Amount],
[Extent1].[Percentage] AS [Percentage]
FROM [dbo].[Discount] AS [Extent1]
WHERE [Extent1].[Amount] <= #p__linq__0
) AS [Project1]
ORDER BY [Project1].[Amount] DESC ) AS [Project2] ON 1 = 1
) AS [Limit2]',N'#p__linq__0 bigint',#p__linq__0=15000

Answer 2

this is normal because
for first statment
db.Discount.Where(d => d.Amount <= PO.Amount).OrderByDescending(o => o.Amount).Select(s => s.Percentage).ToList().DefaultIfEmpty(0).FirstOrDefault();
you call .ToList() before DefaultIfEmpty(0) which means when you call .ToList() statment translated to sql as following
DECLARE #p0 Int = 15000
SELECT [t0].[Percentage]
FROM [AppScreens] AS [t0]
WHERE [t0].[Amount] <= #p0
ORDER BY [t0].[Amount] DESC
then executed and loaded in memory after that these two function run on data in memory .DefaultIfEmpty(0).FirstOrDefault(); so the result is as you expected
but for second statment
db.Discount.Where(d => d.Amount <= PO.Amount).OrderByDescending(o => o.Amount).Select(s => s.Percentage).DefaultIfEmpty(0).FirstOrDefault();
you don't call .ToList() which mean that statement won't be executed until it reach FirstOrDefault() because DefaultIfEmpty(0) function is implemented by using deferred execution and you can read its documentation from this reference of MSDN
When it reach .FirstOrDefault() statment translated to sql as following
DECLARE #p0 Int = 15000
SELECT case when [t2].[test] = 1 then [t2].[Percentage] else [t0].[EMPTY] end AS [value]
FROM (SELECT 0 AS [EMPTY] ) AS [t0]
LEFT OUTER JOIN ( SELECT TOP (1) 1 AS [test], [t1].[Percentage] FROM [Discount] AS [t1] WHERE [t1].[Amount] <= #p0 ) AS [t2] ON 1=1
ORDER BY [t2].[Amount] DESC
then executed and loaded in memory after that so the result isn't as you expected
because it get top 1 first before order so it get first item.