Select *,
(Select DefaultStartDay from Scheduler.ProgramSettings ps where ps.DefaultStartDay = s.Id ) [DefaultStartDay]
from Scheduler.Schedules s
where ScheduleType = 2;
I want to write above SQL query in EF Core, Specially I need subquery in select list to get data from another table with specific condition.
please refer image.Sample Data with SQL Query
I have tried below EF Core but getting wrong result.
var model = _context.Schedules
.Where(s => s.ScheduleType == 2)
.Select(rv => new ProgramSetting
{
Id = rv.Id,
ProgramTemplateId = rv.ProgramTemplateId,
IsActive = rv.IsActive,
DefaultStartDay = rv.Id
}).ToArray();
The SQL query is wrong and this is a misuse of EF Core.
First, that SQL will fail if there's more than 1 result from the subquery. Even in SQL you'd need a different query. An INNER JOIN would return the same results without failing if there are multiple matches.
Select s.*,ps.DefaultStartDay
from Scheduler.Schedules s
inner join Scheduler.ProgramSettings ps on ps.DefaultStartDay = s.Id
where ScheduleType = 2;
Second, using LINQ to emulate SQL is a misuse of both EF Core and LINQ. EF isn't a replacement for SQL, it's an ORM. Its job is to give the impression of working with in-memory objects instead of tables, not allow you to write SQL queries in C#
It's the ORM's job to generate JOINs as needed from the relations between entities (not tables). In this case, if Schedule has a ProgramSettins property, EF would generate the necessary joins automatically. Loading an entire schedule object could be as simple as :
var schedules=_context.Schedules
.Incule(sch=>sch.ProgramSettings)
.Where(s => s.ScheduleType == 2)
.ToArray();
Include is used to eagerly load the settings, not to force a JOIN.
If a Select clause is used that requires a property from ProgramSettings, the JOIN will be generated automatically, eg :
var namesAndDays=_context.Schedules
.Where(s => s.ScheduleType == 2)
.Select(s=>new {
Name = s.Name,
StartDay = s.ProgramSettings.DefaultStartDay
})
.ToArray();
I have a table called Products and I need to find the products with unique title for a particular category. Earlier we used to do with this query in entity framework core 2.2 :
currentContext.Products
.GroupBy(x => x.Title)
.Select(x => x.FirstOrDefault()))
.Select(x => new ProductViewModel
{
Id = x.Id,
Title = x.Title,
CategoryId= x.CategoryId
}).ToList();
But after upgrading to Entity Framework Core 5.0, we get an error for Groupby Shaker exception:
The LINQ expression 'GroupByShaperExpression:KeySelector: t.title, ElementSelector:EntityShaperExpression: EntityType: Project ValueBufferExpression: ProjectionBindingExpression: EmptyProjectionMember IsNullable: False .FirstOrDefault()' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.
I know there are multiple way to client projection but I am searching for most efficient way to search.
Most likely that LINQ query couldn't be translated in EF Core 2.2 either, because of some limitations that the GroupBy operator has.
From the docs:
Since no database structure can represent an IGrouping, GroupBy operators have no translation in most cases. When an aggregate operator is applied to each group, which returns a scalar, it can be translated to SQL GROUP BY in relational databases. The SQL GROUP BY is restrictive too. It requires you to group only by scalar values. The projection can only contain grouping key columns or any aggregate applied over a column.
What happened in EF Core 2.x is that whenever it couldn't translate an expression, it would automatically switch to client evaluation and give just a warning.
This is listed as the breaking change with highest impact when migrating to EF Core >= 3.x :
Old behavior
Before 3.0, when EF Core couldn't convert an expression that was part of a query to either SQL or a parameter, it automatically evaluated the expression on the client. By default, client evaluation of potentially expensive expressions only triggered a warning.
New behavior
Starting with 3.0, EF Core only allows expressions in the top-level projection (the last Select() call in the query) to be evaluated on the client. When expressions in any other part of the query can't be converted to either SQL or a parameter, an exception is thrown.
So if the performance of that expression was good enough when using EF Core 2.x, it will be as good as before if you decide to explicitly switch to client evaluation when using EF Core 5.x. That's because both are client evaluated, before and now, with the only difference being that you have to be explicit about it now. So the easy way out, if the performance was acceptable previously, would be to just client evaluate the last part of the query using .AsEnumerable() or .ToList().
If client evaluation performance is not acceptable (which will imply that it wasn't before the migration either) then you have to rewrite the query. There are a couple of answers by Ivan Stoev that might get you inspired.
I am a little confused by the description of what you want to achieve: I need to find the products with unique title for a particular category and the code you posted, since I believe it's not doing what you explained. In any case, I will provide possible solutions for both interpretations.
This is my attempt of writing a query to find the products with unique title for a particular category.
var uniqueProductTitlesForCategoryQueryable = currentContext.Products
.Where(x => x.CategoryId == categoryId)
.GroupBy(x => x.Title)
.Where(x => x.Count() == 1)
.Select(x => x.Key); // Key being the title
var productsWithUniqueTitleForCategory = currentContext.Products
.Where(x => x.CategoryId == categoryId)
.Where(x => uniqueProductTitlesForCategoryQueryable .Contains(x.Title))
.Select(x => new ProductViewModel
{
Id = x.Id,
Title = x.Title,
CategoryId= x.CategoryId
}).ToList();
And this is my attempt of rewriting the query you posted:
currentContext.Products
.Select(product => product.Title)
.Distinct()
.SelectMany(uniqueTitle => currentContext.Products.Where(product => product.Title == uniqueTitle ).Take(1))
.Select(product => new ProductViewModel
{
Id = product.Id,
Title = product.Title,
CategoryId= product.CategoryId
})
.ToList();
I am getting the distinct titles in the Product table and per each distinct title I get the first Product that matches it (that should be equivalent as GroupBy(x => x.Title)+ FirstOrDefault AFAIK). You could add some sorting before the Take(1) if needed.
You can use Join for this query as below :
currentContext.Products
.GroupBy(x => x.Title)
.Select(x => new ProductViewModel()
{
Title = x.Key,
Id = x.Min(b => b.Id)
})
.Join(currentContext.Products, a => a.Id, b => b.Id,
(a, b) => new ProductViewModel()
{
Id = a.Id,
Title = a.Title,
CategoryId = b.CategoryId
}).ToList();
If you watch or log translated SQL query, it would be as below:
SELECT [t].[Title], [t].[c] AS [Id], [p0].[CategoryId] AS [CategoryId]
FROM (
SELECT [p].[Title], MIN([p].[Id]) AS [c]
FROM [Product].[Products] AS [p]
GROUP BY [p].[Title]
) AS [t]
INNER JOIN [Product].[Products] AS [p0] ON [t].[c] = [p0].[Id]
As you can see, the entire query is translated into one SQL query and it is highly efficient because GroupBy operation is being performed in database and no additional record is fetched by the client.
As mentioned by Ivan Stoev, EFC 2.x just silently loads full table to the client side and then apply needed logic for extracting needed result. It is resource consuming way and thanks that EFC team uncovered such potential harmful queries.
Most effective way is already known - raw SQL and window functions. SO is full of answers like this.
SELECT
s.Id,
s.Title,
s.CategoryId
FROM
(SELECT
ROW_NUMBER() OVER (PARTITION BY p.Title ORDER BY p.Id) AS RN,
p.*
FROM Products p) s
WHERE s.RN = 1
Not sure that EFC team will invent universal algorithm for generating such SQL in nearest future, but for special edge cases it is doable and maybe it is their plan to do that for EFC 6.0
Anyway if performance and LINQ is priority for such question, I suggest to try our adaptation of linq2db ORM for EF Core projects: linq2db.EntityFrameworkCore
And you can get desired result without leaving LINQ:
urrentContext.Products
.Select(x => new
{
Product = x,
RN = Sql.Ext.RowNumber().Over()
.PartitionBy(x.Title)
.OrderBy(x.Id)
.ToValue()
})
.Where(x => x.RN == 1)
.Select(x => x.Product)
.Select(x => new ProductViewModel
{
Id = x.Id,
Title = x.Title,
CategoryId = x.CategoryId
})
.ToLinqToDB()
.ToList();
Short answer is you deal with breaking changes in EF Core versions.
You should consider the total API and behavior changes for migration from 2.2 to 5.0 as I provided bellow:
Breaking changes included in EF Core 3.x
Breaking changes in EF Core 5.0
You may face other problems to write valid expressions using the newer version. In my opinion, upgrading to a newer version is not important itself. This is important to know how to work with a specific version.
You should use .GroupBy() AFTER materialization. Unfortunately, EF core doesn't support GROUP BY. In version 3 they introduced strict queries which means you can not execute IQeuriables that can't be converted to SQL unless you disable this configuration (which is not recommended). Also, I'm not sure what are you trying to get with GroupBy() and how it will influence your final result. Anyway, I suggest you upgrade your query like this:
currentContext.Products
.Select(x=> new {
x.Id,
x.Title,
x.Category
})
.ToList()
.GroupBy(x=> x.Title)
.Select(x => new Wrapper
{
ProductsTitle = x.Key,
Products = x.Select(p=> new ProductViewModel{
Id = p.Id,
Title = p.Title,
CategoryId= p.CategoryId
}).ToList()
}).ToList();
I have an entity which contains several reference navigation properties under it.
The Repository implementation for the entity looks some thing like this:
return await _dbContext.MyEntity
.Include(s => s.Address) //Reference Navigation
.Include(s => s.BuildingDetails) //Reference Navigation
.ThenInclude(s => s.ChildOfBuildingDetails)
.Include(s => s.ContactPersons)
.Include(s => s.Technicians)
.Include(s => s.DeactivationDetails) //Reference Navigation
.FirstOrDefaultAsync(s => s.Id == id, cancellationToken);
When I check the actual DB queries being executed, all the queries contain the reference navigation properties included in them as joins to the parent entity.
SELECT [m92].[Id], .......
FROM [MyDB].[ContactPersons] AS [m92]
INNER JOIN (
SELECT DISTINCT [m93].[Id], [t76].[Id] AS [Id0]
FROM [MyDB].[MyEntity] AS [m93]
LEFT JOIN (
SELECT [m94].*
FROM [MyDB].[DeactivationDetails] AS [m94]
WHERE [m94].[Deleted] = 0
) AS [t75] ON [m93].[Id] = [t75].[MyEntityId]
LEFT JOIN (
SELECT [m95].*
FROM [MyDB].[BuildingDetails] AS [m95]
WHERE [m95].[Deleted] = 0
) AS [t76] ON [m93].[Id] = [t76].[MyEntityId]
LEFT JOIN (
SELECT [m96].*
FROM [MyDB].[Address] AS [m96]
WHERE [m96].[Deleted] = 0
) AS [t77] ON [m93].[Id] = [t77].[MyEntityId]
WHERE [m93].[Deleted] = 0
) AS [t78] ON [m92].[MyEntityId] = [t78].[Id]
WHERE [m92].[Deleted] = 0
ORDER BY [t78].[Id], [t78].[Id0]
Basically, the whole portion inside the INNER JOIN is present in all the queries that are being executed. Ideally we only need to join the child entities with parent entity in the queries.
1) Why does EF core translate to queries such that it includes the reference navigation property in all the split queries?
2) Is there a way to avoid this behavior, to be specific, replace the INNER JOIN block with just the parent entity
1) Why does EF core translate to queries such that it includes the reference navigation property in all the split queries?
It's an implementation defect/missing optimization.
2) Is there a way to avoid this behavior, to be specific, replace the INNER JOIN block with just the parent entity
The only way I found is to materialize the query with collection navigation property includes (which generate the additional queries) removed, then manually execute queries to load the related collections (requires tracking queries and relies on navigation property fix-up).
For instance (assuming navigation properties not marked as reference are collections):
// Query with filters only
var query = _dbContext.MyEntity
.Where(s => s.Id == id);
// Execute and materialize query with only filters and reference includes
var result = await query
.Include(s => s.Address) //Reference Navigation
.Include(s => s.BuildingDetails) //Reference Navigation
//.ThenInclude(s => s.ChildOfBuildingDetails)
//.Include(s => s.ContactPersons)
//.Include(s => s.Technicians)
.Include(s => s.DeactivationDetails) //Reference Navigation
.FirstOrDefaultAsync(cancellationToken);
// Load the related collections
await query.SelectMany(s => s.BuildingDetails.ChildOfBuildingDetails)
.LoadAsync(cancellationToken);
await query.SelectMany(s => s.ContactPersons)
.LoadAsync(cancellationToken);
await query.SelectMany(s => s.Technicians)
.LoadAsync(cancellationToken);
I have the following linq query that is producing the wrong sql (ProductId = PictureId). We just switched over to using Devart Entity Developer to auto-generate our POCO classes and this issue popped up. The mapping looks correct to me and this query returned the correct results previously. Does the mapping look wrong or perhaps the query itself was never correct?
Using Entity Framework 5.0.
Query:
var totalResults = _productRepository.Table.Where(a => a.Pictures.Any()).Count();
SQL:
SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(1) AS [A1]
FROM [dbo].[Product] AS [Extent1]
WHERE EXISTS (SELECT
1 AS [C1]
FROM [dbo].[ProductPicture] AS [Extent2]
WHERE [Extent1].[ProductId] = [Extent2].[PictureId] // this should be ProductId = ProductId
)
) AS [GroupBy1]
Fluent Mapping:
modelBuilder.Entity<Picture>()
.HasMany(p => p.Products)
.WithMany(c => c.Pictures)
.Map(manyToMany => manyToMany
.ToTable("ProductPicture", "dbo")
.MapLeftKey("ProductId")
.MapRightKey("PictureId"));
Diagram:
The bug with association mapping in the DbContext template is fixed in the latest (5.7.276) build of Entity Developer: http://forums.devart.com/viewtopic.php?f=32&t=28723.
Having some experience with Linq to SQL I'm trying out ADO Entity framework now. In Linq to SQL, I would create a Linq to SQL class, drag my tables across to build the data context. Then I'd instantiate the datacontext class and run some lambda against one of the properties in the datacontext class.
Now, with ADO entity framework I add the Entity Data Model class, and add the tables to the data model. My Entity Data Model class now has a bunch of ObjectQuery<> properties, one for each table I added.
Now what do I do with those properties? How do I call them? Anyone have code examples?
Sure. I have a long presentation on this.
As a simple answer to your question, here are some things you can do with the ObjectQuery<T> properties.
Return a list of objects:
IEnumerable<Customer> result = Context.Customers;
return result;
Return one object:
return Context.Customers.Where(c => c.Id == someId).First();
Project onto a presentation model:
return (from c in Customers
where c.Id == someId
select new CustomerPresentation
{
Id = c.Id,
Name = c.Name,
OrderCount = c.Orders.Count(),
PhoneNumbers = from p in c.PhoneNumbers
select new PhoneNumberPresentation
{
AreaCode = p.AreaCode,
// etc.
},
// etc.
}).First();