I have a table called Products and I need to find the products with unique title for a particular category. Earlier we used to do with this query in entity framework core 2.2 :
currentContext.Products
.GroupBy(x => x.Title)
.Select(x => x.FirstOrDefault()))
.Select(x => new ProductViewModel
{
Id = x.Id,
Title = x.Title,
CategoryId= x.CategoryId
}).ToList();
But after upgrading to Entity Framework Core 5.0, we get an error for Groupby Shaker exception:
The LINQ expression 'GroupByShaperExpression:KeySelector: t.title, ElementSelector:EntityShaperExpression: EntityType: Project ValueBufferExpression: ProjectionBindingExpression: EmptyProjectionMember IsNullable: False .FirstOrDefault()' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.
I know there are multiple way to client projection but I am searching for most efficient way to search.
Most likely that LINQ query couldn't be translated in EF Core 2.2 either, because of some limitations that the GroupBy operator has.
From the docs:
Since no database structure can represent an IGrouping, GroupBy operators have no translation in most cases. When an aggregate operator is applied to each group, which returns a scalar, it can be translated to SQL GROUP BY in relational databases. The SQL GROUP BY is restrictive too. It requires you to group only by scalar values. The projection can only contain grouping key columns or any aggregate applied over a column.
What happened in EF Core 2.x is that whenever it couldn't translate an expression, it would automatically switch to client evaluation and give just a warning.
This is listed as the breaking change with highest impact when migrating to EF Core >= 3.x :
Old behavior
Before 3.0, when EF Core couldn't convert an expression that was part of a query to either SQL or a parameter, it automatically evaluated the expression on the client. By default, client evaluation of potentially expensive expressions only triggered a warning.
New behavior
Starting with 3.0, EF Core only allows expressions in the top-level projection (the last Select() call in the query) to be evaluated on the client. When expressions in any other part of the query can't be converted to either SQL or a parameter, an exception is thrown.
So if the performance of that expression was good enough when using EF Core 2.x, it will be as good as before if you decide to explicitly switch to client evaluation when using EF Core 5.x. That's because both are client evaluated, before and now, with the only difference being that you have to be explicit about it now. So the easy way out, if the performance was acceptable previously, would be to just client evaluate the last part of the query using .AsEnumerable() or .ToList().
If client evaluation performance is not acceptable (which will imply that it wasn't before the migration either) then you have to rewrite the query. There are a couple of answers by Ivan Stoev that might get you inspired.
I am a little confused by the description of what you want to achieve: I need to find the products with unique title for a particular category and the code you posted, since I believe it's not doing what you explained. In any case, I will provide possible solutions for both interpretations.
This is my attempt of writing a query to find the products with unique title for a particular category.
var uniqueProductTitlesForCategoryQueryable = currentContext.Products
.Where(x => x.CategoryId == categoryId)
.GroupBy(x => x.Title)
.Where(x => x.Count() == 1)
.Select(x => x.Key); // Key being the title
var productsWithUniqueTitleForCategory = currentContext.Products
.Where(x => x.CategoryId == categoryId)
.Where(x => uniqueProductTitlesForCategoryQueryable .Contains(x.Title))
.Select(x => new ProductViewModel
{
Id = x.Id,
Title = x.Title,
CategoryId= x.CategoryId
}).ToList();
And this is my attempt of rewriting the query you posted:
currentContext.Products
.Select(product => product.Title)
.Distinct()
.SelectMany(uniqueTitle => currentContext.Products.Where(product => product.Title == uniqueTitle ).Take(1))
.Select(product => new ProductViewModel
{
Id = product.Id,
Title = product.Title,
CategoryId= product.CategoryId
})
.ToList();
I am getting the distinct titles in the Product table and per each distinct title I get the first Product that matches it (that should be equivalent as GroupBy(x => x.Title)+ FirstOrDefault AFAIK). You could add some sorting before the Take(1) if needed.
You can use Join for this query as below :
currentContext.Products
.GroupBy(x => x.Title)
.Select(x => new ProductViewModel()
{
Title = x.Key,
Id = x.Min(b => b.Id)
})
.Join(currentContext.Products, a => a.Id, b => b.Id,
(a, b) => new ProductViewModel()
{
Id = a.Id,
Title = a.Title,
CategoryId = b.CategoryId
}).ToList();
If you watch or log translated SQL query, it would be as below:
SELECT [t].[Title], [t].[c] AS [Id], [p0].[CategoryId] AS [CategoryId]
FROM (
SELECT [p].[Title], MIN([p].[Id]) AS [c]
FROM [Product].[Products] AS [p]
GROUP BY [p].[Title]
) AS [t]
INNER JOIN [Product].[Products] AS [p0] ON [t].[c] = [p0].[Id]
As you can see, the entire query is translated into one SQL query and it is highly efficient because GroupBy operation is being performed in database and no additional record is fetched by the client.
As mentioned by Ivan Stoev, EFC 2.x just silently loads full table to the client side and then apply needed logic for extracting needed result. It is resource consuming way and thanks that EFC team uncovered such potential harmful queries.
Most effective way is already known - raw SQL and window functions. SO is full of answers like this.
SELECT
s.Id,
s.Title,
s.CategoryId
FROM
(SELECT
ROW_NUMBER() OVER (PARTITION BY p.Title ORDER BY p.Id) AS RN,
p.*
FROM Products p) s
WHERE s.RN = 1
Not sure that EFC team will invent universal algorithm for generating such SQL in nearest future, but for special edge cases it is doable and maybe it is their plan to do that for EFC 6.0
Anyway if performance and LINQ is priority for such question, I suggest to try our adaptation of linq2db ORM for EF Core projects: linq2db.EntityFrameworkCore
And you can get desired result without leaving LINQ:
urrentContext.Products
.Select(x => new
{
Product = x,
RN = Sql.Ext.RowNumber().Over()
.PartitionBy(x.Title)
.OrderBy(x.Id)
.ToValue()
})
.Where(x => x.RN == 1)
.Select(x => x.Product)
.Select(x => new ProductViewModel
{
Id = x.Id,
Title = x.Title,
CategoryId = x.CategoryId
})
.ToLinqToDB()
.ToList();
Short answer is you deal with breaking changes in EF Core versions.
You should consider the total API and behavior changes for migration from 2.2 to 5.0 as I provided bellow:
Breaking changes included in EF Core 3.x
Breaking changes in EF Core 5.0
You may face other problems to write valid expressions using the newer version. In my opinion, upgrading to a newer version is not important itself. This is important to know how to work with a specific version.
You should use .GroupBy() AFTER materialization. Unfortunately, EF core doesn't support GROUP BY. In version 3 they introduced strict queries which means you can not execute IQeuriables that can't be converted to SQL unless you disable this configuration (which is not recommended). Also, I'm not sure what are you trying to get with GroupBy() and how it will influence your final result. Anyway, I suggest you upgrade your query like this:
currentContext.Products
.Select(x=> new {
x.Id,
x.Title,
x.Category
})
.ToList()
.GroupBy(x=> x.Title)
.Select(x => new Wrapper
{
ProductsTitle = x.Key,
Products = x.Select(p=> new ProductViewModel{
Id = p.Id,
Title = p.Title,
CategoryId= p.CategoryId
}).ToList()
}).ToList();
I am trying to build a query using asp.net core c#
https://www.reflectionit.nl/blog/2017/paging-in-asp-net-core-mvc-and-entityframework-core
I trying to do a filtering however I need the data from another table which have my unique id
var result = _context.UserRoles.Where(y => y.RoleId.Contains(selectedRoles.Id)); // Retrieve the the userid i have from another table with the selected roleid
var query = _context.Users.Where(x => //I have already tried contains, where join );
If there is a site where i can learn this query please recommend. "Join()" does not work as I am doing paging
a least two solutions (please note that I do not check the identity classes members, so the following is the "spirit" of the solution (you miss the select clauses) ):
var result = _context.UserRoles.
Where(y => selectedRoles.Contains(y.RoleId)).
Select(y => y.User);
or
var result = _context.UserRoles.
Where(y => selectedRoles.Contains(y.RoleId)).
Select(y => y.UserId);
query = _context.Users.
Where(x => result.Contains(x.Id));
That said, assuming that there is no UserRoles table exposed in Identity (v2), you probably want:
userManager.Users.
Where(u => u.Roles.Any(r => selectecRoles.Contains(r.RoleId)));
Up to you to instanciate the userManager.
I have 2 entities,
News
FileAttachment
I wanted to configure using code-first fluent API so that Each News can have 0,1 or more than 1 attachments.
here is what i'm using right now
public NewsMap()
{
this.ToTable("News"); // Table Name
this.HasKey(m => m.Id); // Primary Key
// Field Definition
this.Property(m => m.Title).HasMaxLength(255).IsRequired();
this.Property(m => m.Body).HasColumnType("Text").IsRequired();
this.Property(m => m.Summary).HasMaxLength(1000).IsRequired();
this.Property(m => m.AuthorId).IsRequired();
this.Property(m => m.CreatedOn).IsRequired();
this.Property(m => m.UpdatedOn).IsRequired();
this.HasMany(m => m.Attachments).WithMany().Map(m => m.MapLeftKey("NewsId").MapRightKey("AttachmentId"));
}
public class FileAttachmentMap : EntityTypeConfiguration<FileAttachment>
{
public FileAttachmentMap()
{
this.ToTable("FileAttachments"); // Table Name
this.HasKey(m => m.Id); // Primary Key
// Field Definition
this.Property(m => m.DisplayName).HasMaxLength(256).IsRequired();
this.Property(m => m.PhysicalFileName).HasMaxLength(256).IsRequired();
this.Property(m => m.Extension).HasMaxLength(50).IsRequired();
this.Property(m => m.IsImage).IsRequired();
this.Property(m => m.ThumbTiny).HasMaxLength(275).IsOptional();
this.Property(m => m.ThumbSmall).HasMaxLength(275).IsOptional();
this.Property(m => m.ThumbMid).HasMaxLength(275).IsOptional();
this.Property(m => m.ByteSize).IsRequired();
this.Property(m => m.StorageType).IsRequired();
this.Property(m => m.CreatedOn).IsRequired();
this.Property(m => m.UpdatedOn).IsRequired();
}
}
This mapping correctly generates an intermediate table named NewsFileAttachment with two fields :
NewsId
AttachmentId
On News Entity when i call News.Attachments.Add(Attachment); it correctly adds records in both Attachment & NewsAttachment tables.
When i remove some list item from News.Attachments it correctly removes record from NewsAttachment table, but it doesn't delete record in FileAttachment table. I wanted to remove that too.
Can someone please suggest a better Fluent API configuration to achieve this?
Thanks,
Amit
EDIT
In my case FileAttachment stores files for various purpose. i've Blog entity that too have attachments. So, two intermediate tables BlogAttachments & FileAttachments. Now if i use WithOptional as (I can't use WithRequired as i need BlogId & NewsId both in FileAttachment table), i can get rid off intermediate table, but still delete doesn't delete record from FileAttachment table, it just make NewsId/BlogId NULL.
Any suggestion? Main thing is I do not wanted to create separate tables with all the fields i have in FileAttachment table.
That's expected - as it creates many-to-many and extra table - the cascade only applies to that table.
There is no direct 'FK' relationship in between your News and
Attachment, as it goes through a join table. And thus you cannot expect for e.g. attachment to be deleted, if the news does - as attachment could have other news relating to it.
See also this one - it's somewhat relevant.
One to Many Relationship with Join Table using EF Code First
i.e. if your structure permits don't explicitly create many-to-many (don't put collection on both sides, or similar in fluent config).
In your case providing your 'attachments' are not reusable in between News - then just put a collection navigation property in the News - and leave attachment w/o any - or make a 'FK', single instance navigation from Attachment (like a 'Parent') if you need it.
On the other side, if an attach... could be parented by different
news records - then you shouldn't have cascade delete anyways.
note: check your generated migration script - or SQL/Db - to see exactly what it creates - and make sure there is no intermediate table created - and only one 'FK' going from 'attachment' to 'news'.
edit:
modelBuilder.Entity<News>()
.HasMany(c => c.Attachments)
.WithOptional() // or WithRequired (test to see which is better for you)
.WillCascadeOnDelete(true);
...and make one public ICollection<FileAttachment> Attachments {get;set;} in the News.
(actually the collection property is all you need - but configuration is to be safe you get what you want)
That'd make you 1-to-many (or many-to-one), which is the nature of your data (as you said in comments) - and you can have cascade deletes.
I'm using EF4. Having 2 entities:
Person { Name }
Hobbys { Person.Name, IsCoolHobby }
1 Person can have several hobbys.
I now have
IQueryable<Person> p;
p = container.PersonSet.Include("Hobbys").AsQueryable();
p = p.Where(x => x ?????);
List<Person> tmp = p.ToList();
How can i return only those Persons who have cool hobbys (IsCoolHobby == true)? I tried join but i was not able to load them into the list (select can only return Person, Hobby or new Type - but how to map them to entity objects again?)
Thanks
How can i return only those Persons who have cool hobbys (IsCoolHobby
== true)?
List<Person> tmp = container.PersonSet.Include("Hobbys")
.Where(p => p.Hobbys.Any(h => h.IsCoolHobby))
.ToList();
This will load the people who have at least one cool hobby but the Hobbys collection for those people will always contain all hobbys, also the uncool hobbys.
Edit
Unfortunately filtering and sorting children during eager loading (Include) is currently not supported. There is a request on the EF feature suggestion page for this feature. The request has status "Under review", so there is a little hope that it might get implemented in the future. (Probably far future: At least the first docs about EF 5 (beta) on MSDN say explicitly that eager loading with filtering/sorting is still not implemented.)
For now there are only two workarounds. The first is to use a projection:
var projectedData = container.PersonSet
.Where(p => p.Hobbys.Any(h => h.IsCoolHobby))
.Select(p => new
{
Person = p,
CoolHobbys = p.Hobbys.Where(h => h.IsCoolHobby)
})
.ToList();
The result is a collection of anonymous objects which contain a user who has cool hobbys and a collection of those cool hobbys. If you don't disable change tracking (by using the NoTracking option for the query) the person's hobbys collection should be filled with the result automatically.
The second option is to use "explicit" loading with CreateSourceQuery:
List<Person> tmp = container.PersonSet
.Where(p => p.Hobbys.Any(h => h.IsCoolHobby))
.ToList();
foreach (var person in tmp)
{
person.Hobbys.Attach(person.Hobbys.CreateSourceQuery()
.Where(h => h.IsCoolHobby).ToList());
}
Two things to note here:
CreateSourceQuery is only available on EntityCollections, i.e. if you are using EntityObject derived entities. It's not available for POCO entities in EF 4.0. (EF >= 4.1/DbContext has the option for explicit loading also for POCOs -> Query() method.)
The above code represents 1+N roundtrips to the database: The first for the person collection without the hobbys and then one additional query per person to load the cool hobbys.
Good afternoon,
I have three entities (that concern this question)
Company (ID, etc..)
CompanyAddress (AddressID, CompanyID, Rank)
AddressDetails (AddressID, Street, City, State, Zip)
The reason Rank and company id aren't in the AddressDetails is because the address details are shared with contacts via a ContactAddress entity.
Anyway, I need to build an IQueryable given an IQueryable that will check if a string is contained in the City (and eventually or state). I'd like to use Lambda expressions not the from c in companies syntax...I tried
query = query.Select(c => c.Addresses.Where(a => a.AddressDetails.City.Contains(City)).Select(ca => ca.Company));
In this example c.Addresses is the navigation property for CompanyAddress.
Thanks for any help,
Paul
I think that this will work:
query = query.Where(c => c.Addresses.Any(a => a.AddressDetails.City.Contains(City)));
I assume that query is IQueryable<Company>.