I have the following query in the old Entity Framework (.NET Framework):
db.ProductVariations
.Where(pv => pv.Product.Categories
.Any(cat => categorySearchStrings
.Any(categorySearchString => cat.SearchTree.StartsWith(categorySearchString))));
I realize this isn't pretty, but I'm refactoring a legacy app and we have to choose our battles.
So what happens is that you can pass a list of search string (the categorySearchStrings), e.g.:
"38.54.", "45."
This is basically an implementation of a search tree where each category in our database has a SearchTree property. So a category with search tree 38.54.99 would match, but 38. would not.
A product can have multiple categories and we can pass in multiple search tree strings to the query. So we're comparing two collections.
This gets translated to
SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
COUNT(1) AS [A1]
FROM [dbo].[ProductVariation] AS [Extent1]
WHERE EXISTS (SELECT
1 AS [C1]
FROM ( SELECT
[Extent3].[SearchTree] AS [SearchTree]
FROM [dbo].[ProductCategory] AS [Extent2]
INNER JOIN [dbo].[Category] AS [Extent3] ON [Extent2].[CategoryId] = [Extent3].[Id]
WHERE [Extent1].[ProductId] = [Extent2].[ProductId]
) AS [Project1]
WHERE EXISTS (SELECT
1 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
WHERE ( CAST(CHARINDEX(N'38.', [Project1].[SearchTree]) AS int)) = 1
)
)
) AS [GroupBy1]
I'm trying to migrate to Entity Framework Core (6, running on .NET 6) but this now gives me the following error:
System.InvalidOperationException : The LINQ expression 'categorySearchString => categorySearchString == "" || EntityShaperExpression:
Company.Data.Models.Category
ValueBufferExpression:
ProjectionBindingExpression: Inner
IsNullable: False
.SearchTree != null && categorySearchString != null && EntityShaperExpression:
Company.Data.Models.Category
ValueBufferExpression:
ProjectionBindingExpression: Inner
IsNullable: False
.SearchTree.StartsWith(categorySearchString)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
Switching to client evaluation isn't really an option I believe, because there's too many data that will be retrieved. Plus, there's more going on than just this Where clause. I simplified it.
I also tried rewriting it as this:
.Where(pv => pv.Product.Categories.Select(c => c.SearchTree).Any(st => categorySearchStrings.Any(ss => st.StartsWith(ss))));
But I just get the same error.
Is it possible to do this with EF Core?
I'd be inclined to build a dynamic expression tree to represent the filter:
var cat = Expression.Parameter(typeof(Category), "cat");
var parts = new List<Expression>(categorySearchStrings.Count);
var startsWithMethod = typeof(string).GetMethod(nameof(string.StartsWith), new[] { typeof(string) });
foreach (string categorySearchString in categorySearchStrings)
{
var searchTree = Expression.Property(cat, nameof(Category.SearchTree));
var value = Expression.Constant(categorySearchString);
var startsWith = Expression.Call(searchTree, startsWithMethod, value);
parts.Add(startsWith);
}
var body = parts.Aggregate(Expression.OrElse);
var categoryFilter = Expression.Lambda<Func<Category, bool>>(body, cat);
var pv = Expression.Parameter(typeof(ProductVariation), "pv");
var product = Expression.Property(pv, nameof(ProductVariation.Product));
var categories = Expression.Property(product, nameof(Product.Categories));
var any = Expression.Call(typeof(Enumerable), nameof(Enumerable.Any), new[] { typeof(Category) }, categories, categoryFilter);
var finalFilter = Expression.Lambda<Func<ProductVariation, bool>>(any, pv);
db.ProductVariations
.Where(finalFilter)
...
You should also report this as an issue on the efcore repository, to see if it can be fixed in a future version.
Update: the issue was created but was a duplicate of an existing issue.
Related
I have entities like Product(Id, Name) and Keyword(Id, Description), and there is a many-to-many relationship between them.
The essence of my task is the following, I need to do a full-text search on Name and Description columns, using EF CORE 6
I already have some SQL code that works fine.
SELECT a."Id", a."Name" as name, k.txt
FROM "Products" AS a
LEFT JOIN (
SELECT x."ProductsId" as Id, string_agg(y."Description", ' ') as txt
FROM "ProductKeywords" x
JOIN "Keywords" y ON y."Id" = x."KeywordId"
GROUP BY 1
) k ON a."Id" = k.Id
WHERE to_tsvector(concat_ws(' ', a."Name", k.txt))
## to_tsquery('Some text');
And I need to write some LINQ code that will do something similar, but I have a problem with string_agg, and I don't understand how to implement it in LINQ and EF CORE will reflect it correctly
I tried to do the following
var products = _context.Products
.Select(e => new
{
Id = e.Id,
Name = e.Name,
Keywords = string.Join(" ", e.Keywords.Select(q => q.Description))
}).Where(e => EF.Functions.ToTsVector(e.Keywords).Matches("Some text")).ToList();
But I get an error, and it's most likely because of string.Join
could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'
Got the result, using linq2db
var query = _context.Products.ToLinqToDB()
.LeftJoin(_context.ProductsKeywords.ToLinqToDB().GroupBy(r => r.ProductId).Select(e => new {
Key = e.Key,
Txt = e.StringAggregate(",", t => t.Keyword.Description).ToValue()
}),
(product, productKeyword) => product.Id == productKeyword.Key,
(i, j) => new {
Id = i.Id,
Txt = j.Txt
}).Select(e => new {
Id = e.Id,
Txt = EF.Functions.ToTsVector(e.Txt)
}).Where(w => w.Txt.Matches("Some text"));
We are using EFCore.SqlServer.HierarchyId to represent a hierarchy in our data.
My goal is to return the descendants of an object with a particular path of indeterminate length, e.g. given a tree with the hierarchy one->two->three->four, the path one/two/three would return four
Knowing the length of the path, I can make a query like this:
var collections = await context.Collections.Where(c => c.CollectionHierarchyid.IsDescendantOf(
context.Collections.FirstOrDefault(c1 => c1.FriendlyId == "three" &&
context.Collections.Any(c2 => c2.CollectionHierarchyid == c1.CollectionHierarchyid.GetAncestor(1) && c2.FriendlyId == "two" &&
context.Collections.Any(c3 => c3.CollectionHierarchyid == c2.CollectionHierarchyid.GetAncestor(1) && c3.FriendlyId == "one")
)
).CollectionHierarchyid
)).ToListAsync();
But how would you go about this if the length of the path is unknown? I can't call a recursive function from the expression because it won't compile from Linq to Entity Sql.
I know the answer lies somewhere in using System.Linq.Expressions to build the expression, but I am not sure where to start.
The problem can be solved without dynamic expression tree generation, at least not directly, but using standard LINQ query operators.
Let say you have a hierarchical entity like this
public class Entity
{
public HierarchyId Id { get; set; }
// other properties...
}
Given a subquery returning the full set
IQueryable<Entity> fullSet = context.Set<Entity>();
and subquery defining some filtered subset containing the desired ancestors
IQueryable<Entity> ancestors = ...;
Now getting all direct and indirect descendants can easily be achieved with
IQueryable<Entity> descendants = fullSet
.Where(d => ancestors.Any(a => d.Id.IsDescendantOf(a.Id));
So the question is how to build ancestors subquery dynamically.
Applying some filter to the full set and retrieving the direct ancestors filtered by another criteria can be done by using simple join operator
from p in fullSet.Where(condition1)
join c in fullSet.Where(condition2)
on p.Id equals c.Id.GetAncestor(1)
select c
Hence all you need is to apply that recursively, e.g. having
IEnumerable<TArg> args = ...;
representing the filtering criteria arguments ordered by level, then the query can be built as follows
var ancestors = args
.Select(arg => fullSet.Where(e => Predicate(e, arg)))
.Aggregate((prevSet, nextSet) =>
from p in prevSet join c in nextSet on p.Id equals c.Id.GetAncestor(1) select c);
With that being said, applying it to your example:
IEnumerable<string> friendlyIds = new [] { "one", "two", "three" };
var fullSet = context.Collections.AsQueryable();
var ancestors = friendlyIds
.Select(friendlyId => fullSet.Where(e => e.FriendlyId == friendlyId))
.Aggregate((prevSet, nextSet) =>
from p in prevSet join c in nextSet on p.CollectionHierarchyid equals c.CollectionHierarchyid.GetAncestor(1) select c);
var descendants = fullSet
.Where(d => ancestors.Any(a => d.CollectionHierarchyid.IsDescendantOf(a.CollectionHierarchyid));
How can I take each first element in the group with EF 5 ?
var result = await context.SomeDbSet
.Where(...)
.GroupBy(x => new { x.SomeField, ... })
.Select(x => x.First())
.ToListAsync();
I am getting not supported exception.
How to correctly rewrite query? Thanks.
You cannot do that with grouping. SQL has a limitation - with GROUP BY you can select only grouping keys and aggregation result. This limitation for sure extended to LINQ to Entities - after GroupBy you can select only grouping keys and aggregation result.
Such result can be achieved by SQL and Window functions:
SELECT
r.*,
FROM
(
SELECT
s.*,
ROW_NUMBER() OVER(PARTITION BY s.SomeField1, s.SomeField2 ORDER BY s.SomeDate) AS RN
FROM SomeDbSet s
WHERE ...
) r
WHERE r.RN = 1
For those who want to stay with LINQ, I propose extension (disclaimer: I'm extension creator) linq2db.EntityFrameworkCore
And you can write query above via LINQ
var rnQuery =
from s in context.SomeDbSet
where ...
select new
{
Data = s,
RN = Sql.Ext.RowNumber().Over()
.PartitionBy(s.SomeField1, s.SomeField2)
.OrderBy(s.SomeDate)
.ToValue()
}
var resultQuery = await rnQuery
.Where(r => r.RN == 1)
.Select(r => r.Data)
.ToLinqToDB();
var result = resultQuery.ToList();
// async variant may need ToListAsyncLinqToDB() call
// because of collision in async extension methods between EF Core and linq2db
var result = await resultQuery.ToListAsyncLinqToDB();
I have a Site table that has a many-to-many relationship with a UtilityServiceConnection table using a linking table called LinkSiteUtilityServiceConnection. Given a set of ServiceConnectionIds, I need to locate the Site that is exclusively linked to all of them and no more. I think I should be able to write the query using All on the collection but it's not working as expected.
var serviceConnectionIds = new[] { 546892, 546911, 546923 };
var sites1 = db.Sites
.Where(x => x.LinkSiteUtilityServiceConnections.All(y => serviceConnectionIds.Contains(y.UtilityServiceConnectionId)))
.ToList();
Assert.AreEqual(1, sites1.Count); //fails
This produces the query below that returns ~250,000 records when I expect to get one.
SELECT [Extent1].*
FROM [dbo].[Site] AS [Extent1]
WHERE NOT EXISTS (SELECT 1 AS [C1]
FROM [dbo].[LinkSiteUtilityServiceConnection] AS [Extent2]
WHERE ([Extent1].[SiteId] = [Extent2].[SiteId])
AND ((NOT ([Extent2].[UtilityServiceConnectionId] IN (546892, 546911, 546923)))
OR (CASE
WHEN ([Extent2].[UtilityServiceConnectionId] IN (546892, 546911, 546923)) THEN cast(1 as bit)
WHEN (NOT ([Extent2].[UtilityServiceConnectionId] IN (546892, 546911, 546923))) THEN cast(0 as bit)
END IS NULL)))
Why isn't All working as I expect? What's the best way to write this query?
check this code:
query 1:
var sites1 = db.Sites
.Where(x => serviceConnectionIds.All(y =>
x.LinkSiteUtilityServiceConnections
.Select(u => u.UtilityServiceConnectionId).Contains(y)))
.ToList();
query 2:
var query = db.Posts.AsQueryable();
var sites1 = serviceConnectionIds.Aggregate(query,
(current, item) => current.Where(e => e.LinkSiteUtilityServiceConnections
.Any(c => c.UtilityServiceConnectionId== item))).ToList();
I'm trying to write a query for a database that will left join a table to a look up table and the results will be returned based on a case statement.
In normal SQL the query would look like this:
SELECT chis_id, chis_detail, cilt.mhcatID, cilt.mhtID, 'TheFileName' =
CASE
WHEN cilt.mhcatID IS NOT NULL AND cilt.mhtID IS NOT NULL THEN chis_linked_filename
END
FROM chis
LEFT JOIN cilt on cilt.mhcatID = chis.mhcat_id AND cilt.mhtID = chis.mht_id
WHERE cch_id = 50
chis is the table being queried, cilt is a look-up table and does not contain any foreign key relationships to chis as a result (chis has existing FK's to mht and mhcat tables by the mhtID and mhcatID respectively).
The query will be used to return a list of history updates for a record. If the join to the cilt lookup table is successful this means that the caller of the query will have permission to view the filename of any associated files for the history updates.
Whilst during my research I've found various posts on here relating on how to do case statements and left joins in Linq to Entity queries, I've not been able to work out how to join on two different fields. Is this possible?
You need to join on an anonymous type with matching field names like so:
var query = from x in context.Table1
join y in context.Table2
on new { x.Field1, x.Field2 } equals new { y.Field1, y.Field2 }
select {...};
A full working example using the an extra from instead of a join would look something like this:
var query = from chis in context.Chis
from clit in context.Clit
.Where(x => x.mhcatID = chis.mhcat_id)
.Where(x => x.mhtID = chis.mht_id)
.DefaultIfEmpty()
select new
{
chis.id,
chis.detail,
cilt.mhcatID,
cilt.mhtID,
TheFileName = (cilt.mhcatID != null && cilt.mhtID != null) ? chis.linked_filename : null
};
Based on what Aducci suggested, I used a group join and DefaultIsEmpty() to get the results I wanted. For some reason, I couldn't get DefaultIfEmpty() didn't work correctly on its own and the resulting SQL employed an inner join instead of a left.
Here's the final code I used to get the left join working:
var query = (from chis in context.chis
join cilt in context.cilts on new { MHT = chis.mht_id, MHTCAT = chis.mhcat_id } equals new { MHT = cilt.mhtID, MHTCAT = cilt.mhcatID } into tempCilts
from tempCilt in tempCilts.DefaultIfEmpty()
where chis.cch_id == 50
select new {
chisID = chis.chis_id,
detail = chis.chis_detail,
filename = chis.chis_linked_filename,
TheFileName = (tempCilt.mhcatID != null && tempCilt.mhtID != null ? chis.chis_linked_filename : null),
mhtID = chis.mht_id,
mhtcatID = chis.mhcat_id
}).ToList();