I am using EF Core 3.1 (and I am aware that LINQ queries are no longer evaluated on the client).
However, I don't know why this is not working. It seems to work in other examples. This is my code:
var test = Db.TblAppversiondates
.Where(x => x.FkAppversionstatus.ToString() == "1")
.ToList();
FkAppversionstatus is an integer (not nullable).
This is the error message I receive:
The LINQ expression 'DbSet<TblAppversiondates>
.Where(t => t.FkAppversionstatus.ToString() == "1")' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
Related
I have the following exceprt of my query. I am using ASP.NET Core 3.1 project with EF Core.
I read that the server vs client has changed, so how I used to perform the WHERE part in Core 2.1 (using variables from elsewhere in my code) doesn't seem to work anymore.
So as below I have changed (as per something I read) to use ToList() in each part, but now is it not hitting the database more (in my Core 2.1 I would of only had the ToList on the final part as per the code comment below).
So now for Core 3.1 I need to have a dynamic where in the initial "// Load data" part - how do I do a dynamic Where in the initial part, or is there a way, now the server vs client changes are in EF Core to work around that (note it is the final "// Search" part that fails in EF under Core 3.1 (prior to adding the ToList 's)
public List<KBEntryListVM> lstKBEntry;
// Load data
var q = await (from _k in _context.KBEntry
join _kc in _context.KBCategory on _k.CategoryId equals _kc.Id
into _kc2
from _kc3 in _kc2.DefaultIfEmpty()
select new KBEntryListVM()
{
Id = _k.Id,
DateCreated = DateTime.Parse(_k.DateCreated.ToString()),
CategoryId = _k.CategoryId,
CategoryTitle = _kc3.Title.ToString().Trim(),
Text = _k.Text.ToString().Trim(),
Title = _k.Title.ToString().Trim()
}).ToListAsync();
// KBCategory
if (!string.IsNullOrEmpty(c) && Guid.TryParse(c.ToString().Trim(), out var newGuid))
{
q = q.Where(w => w.CategoryId == Guid.Parse($"{c.ToString()}")).ToList();
}
// Search
if (!string.IsNullOrEmpty(s))
{
q = q.Where(w => w.Title.ToLower().Contains($"{s.ToLower()}") || w.CategoryTitle.ToLower().Contains($"{s.ToLower()}") || w.Text.ToLower().Contains($"{s.ToLower()}")).ToList();
}
lstKBEntry = q; //.ToList(); this would of been the only place in Core 2.1 I would of had ToList()
Arthur
So as below I have changed (as per something I read) to use ToList() in each part
EF Core 3.x+ client evaluation exception message suggests to either (1)
rewrite the query in a form that can be translated
or (2)
switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()
So you are taking the option (2) which is easier, but you should try utilizing the option (1) which is harder, but better from performance perspective and the main reason implicit client evaluation have been removed by EFC 3.0. Option (2) should be your last resort only in case there is no way to apply option (1).
The exception message also contains the failed expression. Unfortunately it is not the exact part, but the whole expression (the whole Where predicate for instance), so you need to analyze it, find the failing part(s) and try to replace them with translatable constructs.
One of the general rules for simple data expressions is to avoid explicit conversions (ToString(), Parse). Store dates and numbers in database as such rather than strings, or utilize value conversions when using old existing database and aren't allowed to change it.
In this particular query, the unsupported (non translatable) construct most likely are ToString() calls of string type properties (e.g. Title, Text). EF Core still supports implicit client evaluation of the final Select, so you won't notice it if there is no Where (or other) clause after that referencing such expressions. But as being said at the beginning, you should avoid them regardless - query the raw data and let the usages (UI) do the desired formatting.
Anyway, I can't tell exactly because you haven't show your model, but removing ToString() should make the query translatable, hence no need of intermediate ToList() or similar client materialization:
CategoryTitle = _kc3.Title.Trim(),
Text = _k.Text.Trim(),
Title = _k.Title.Trim()
You should probably also replace
DateCreated = DateTime.Parse(_k.DateCreated.ToString())
with just
DateCreated = _k.DateCreated
because it seems that DateCreated is already DateTime, so the double conversion through string doesn't make sense and would cause similar troubles. And even if the database type is string, still remove Parse / ToString and setup value converter which does that.
I have a entity object of type Complex.
A Complex has a 1:1 to a Forum which has many Topics, each of which have many Posts. I am trying to page the Posts but getting an error which I don't understand.
Message=The ThenInclude property lambda expression 'p => {p.Posts => Skip((__pageIndex_0 - 1)) => Take(__pageSize_1)}' is invalid. The expression should represent a property access: 't => t.MyProperty'. To target navigations declared on derived types, specify an explicitly typed lambda parameter of the target type, E.g. '(Derived d) => d.MyProperty'.
This works ..
public Complex GetComplexWithForumAndPosts(Guid Id, int pageIndex, int pageSize = 10)
{
var complex = CoDBContext.Complexes
.Include(x => x.Forum)
.ThenInclude(x => x.Topics)
.ThenInclude(p => p.Posts)
.Single(x => x.Id == Id);
return complex;
}
but this doesnt
public Complex GetComplexWithForumAndPosts(Guid Id, int pageIndex, int pageSize = 10)
{
var complex = CoDBContext.Complexes
.Include(x => x.Forum)
.ThenInclude(x => x.Topics)
.ThenInclude(p => p.Posts.Skip((pageIndex-1)*pageSize).Take(pageSize))
.Single(x => x.Id == Id);
return complex;
}
Some background: what Include doesn't include
The Include method is a bitch. The most commonly used overload accepting an expression parameter (existing since Entity Framework 4.1 if memory serves) looks suspiciously like those versatile LINQ methods doing all kinds of wonderful stuff with the most wild expressions we feed them.
In reality it --and ThenInclude-- aren't LINQ methods. (Then)Include is nothing but a sturdy old method refusing to do anything outside of its single task: passing a navigation property name to the EF query engine, instructing it to eagerly load a collection or a reference with the root entity. Think of it as the strong-typed version of Include("PropertyName"). Its only purpose is enabling compile-type type checking.
Which means: you can only use expressions that represent a navigation property's name: .ThenInclude(p => p.Posts) is OK. Anything added to it, not.
The expression parameter makes people expect it to do much more than that. And why not? The code compiles alright, why shouldn't it run? But no. Common disappointments are that Include can't be filtered or sorted. Efforts to combine Include and Skip/Take are less common, but equally understandable.
As for me, the EF team might as well consider ditching this overload of (Then)Include altogether now we have the nameof keyword that does a similar thing and could be used likewise. That would put an end to all confusion, to an endless influx of Stack Overflow questions, and to the never-ending requests for change that, so far, have never even been road-mapped. [Also understandable, but that's way beyond the scope of this question].
The issue
In the mean time you still have your issue. The reason that Skip/Take isn't often combined with Include is that it's hard to imagine what it should do. Just suppose you had two nested Includes in there, or ThenInclude(x => x.Topics.Skip().Take().ThenInclude(p => p.Posts)) which, if supported, should all have been equally legal. In fact, only paging on the root entity of a query is well-defined.
So you can only get paged posts by querying posts. For example like so:
CoDBContext.Posts
.Where(p => p.Topic.Forum.ComlexId = Id)
.Skip((pageIndex-1)*pageSize).Take(pageSize))
This is also a much leaner query than the one with Includes.
If you need more context information, for example on the Complex, you could consider querying the Complex in one call, keep it on the page (if this is a SPA) and query the posts in subsequent ajax calls.
I am already have create method for single entity:
override def createEntity(entity: Entity) =
db.run(
( for {
existing <- Entity.filter(e => e.id === entity.id).result //Check, if entity exists
e <- if (existing.isEmpty)
(Entity returning Entity) += Entity(entity.id, entity.name)
else {
throw new DuplicateException(s"Create failed: entity already exists")
}
} yield e ).transactionally
)
How can I reuse this method for transactionally creating list of entities?
//Doesn't work
override def createEntities(entities : List[model.Entity]) = {
db.run(
( for {
e <- entities
}
yield createEntity(e)
).transactionally
)
}
I am newbie in slick :(
P.S. Sorry for my English.
The root of your problem is that you're trying execute multiple inner db.runs (declared in your createEntity) from code called from within outer db.run (declared in createEntities). These outer and inner db.runs are conceptually unrelated and thus are not executed within same transaction.
A bit of perspective
To better understand what is going on and how to deal with situation like this we'll have to talk about monads, specifically DBIOAction (I will try to explain conceptually what is going on with great deal of simplification and cutting some corners since rigorous explanation will take too much time and not that relevant for the solution).
One way of thinking about DBIOAction is as a canned list of steps (code) explaining how to execute queries and get results from db. db.run does actual execution of this list.
Important thing to note is that this list contains steps (code) to be executed in the database (SQL queries) as well as in your local jvm (runtime scala function objects).
Furthermore every element of this list (piece of code) depends on previous either by taking input from previous or by being generated by previous. This all is glued together by series of map/flatMaps (for expressions get desugared to it). This looks something like this:
(1) sql code (generates input for (2))
(2) jvm code (gets input from (1), and generates (3))
(3) sql code (generates input for (4))
(4) jvm code (gets input from (3), and generates (5))
...
Note that plain jvm code gets woven into this list and can be anything as long as it generates instructions for the next step (or delivers end result in case of code passed to map).
This gives monads in general (and thus DBIOAction) huge expressive power since this enables dynamic behavior of the whole list (i.e. every "jvm code" step can influence computations down the road).
The side effect of the fact that you can do anything in "jvm code" step is that you also can spawn and execute new unrelated lists of computations (this is what you are trying to do) which may be fine but also confusing if you are not thinking in terms of monadic composition.
So, what it is that you can actually do to solve the problem?
If you're concerned with code reuse I'd recommend you to get rid of inner db.runs and extract DBIOAction that you can later db.run in createEntity and (with some tweaks) in createEntities.
You should be able to rewrite your code to something similar to this (I do not have your exact version of entities so treat this as a pseudocode):
def createQuery(entity: Entity) = ( for {
existing <- Entity.filter(e => e.id === entity.id).result
e <- if (existing.isEmpty)
(Entity returning Entity) += Entity(entity.id, entity.name)
else {
throw new DuplicateException(s"Create failed: entity already exists")
}
} yield e )
def createEntity(entity: Entity) = db.run(createQuery(entity))
def createEntities(entities : List[model.Entity]) = {
db.run(DBIO.sequence(entities.map(createQuery(_))).transactionally)
}
Note the application of DBIO.sequence combinator to list of DBIOActions and applying transactionally to the result before actual db.run.
Sidenote
If you have control over your schema I'd recommend you to move enforcing "unique id" constraint from entity creation in code to your database.
I guess you are looking for something like that:
http://slick.lightbend.com/doc/3.2.0/dbio.html#sequential-execution
I did not test the code, and I am not used to slick 3.x, but I think it should be something like this:
override def createEntities(entities : List[model.Entity]) = {
db.run(DBIO.seq(entities.map(createEntity):_*).transactionally)
}
I'm constructing and executing my queries in a way that's independent of EF-Core, so I'm relying on IQueryable<T> to obtain the required level of abstraction. I'm replacing awaited SingleAsync() calls with awaited ToAsyncEnumerable().Single() calls. I'm also replacing ToListAsync() calls with ToAsyncEnumerable().ToList() calls. But I just happened upon the ToAsyncEnumerable() method so I'm unsure I'm using it correctly or not.
To clarify which extension methods I'm referring to, they're defined as follows:
SingleAsync and ToListAsync are defined on the EntityFrameworkQueryableExtensions class in the Microsoft.EntityFrameworkCore namespace and assembly.
ToAsyncEnumerable is defined on the AsyncEnumerable class in the System.Linq namespace in the System.Interactive.Async assembly.
When the query runs against EF-Core, are the calls ToAsyncEnumerable().Single()/ToList() versus SingleAsync()/ToListAsync() equivalent in function and performance? If not then how do they differ?
For methods returning sequence (like ToListAsync, ToArrayAsync) I don't expect a difference.
However for single value returning methods (the async versions of First, FirstOrDefault, Single, Min, Max, Sum etc.) definitely there will be a difference. It's the same as the difference by executing those methods on IQueryable<T> vs IEnumerable<T>. In the former case they are processed by database query returning a single value to the client while in the later the whole result set will be returned to the client and processed in memory.
So, while in general the idea of abstracting EF Core is good, it will cause performance issues with IQueryable<T> because the async processing of queryables is not standartized, and converting to IEnumerable<T> changes the execution context, hence the implementation of single value returning LINQ methods.
P.S. By standardization I mean the following. The synchronous processing of IQueryable is provided by IQueryProvider (standard interface from System.Linq namespace in System.Core.dll assembly) Execute methods. Asynchronous processing would require introducing another standard interface similar to EF Core custom IAsyncQueryProvider (inside Microsoft.EntityFrameworkCore.Query.Internal namespace in Microsoft.EntityFrameworkCore.dll assembly). Which I guess requires cooperation/approval from the BCL team and takes time, that's why they decided to take a custom path for now.
When the original source is a DbSet, ToAsyncEnumerable().Single() is not as performant as SingleAsync() in the exceptional case where the database contains more than one matching row. But in in the more likely scenario, where you both expect and receive only one row, it's the same. Compare the generated SQL:
SingleAsync():
SELECT TOP(2) [l].[ID]
FROM [Ls] AS [l]
ToAsyncEnumerable().Single():
SELECT [l].[ID]
FROM [Ls] AS [l]
ToAsyncEnumerable() breaks the IQueryable call chain and enters LINQ-to-Objects land. Any downstream filtering occurs in memory. You can mitigate this problem by doing your filtering upstream. So instead of:
ToAsyncEnumerable().Single( l => l.Something == "foo" ):
SELECT [l].[ID], [l].[Something]
FROM [Ls] AS [l]
you can do:
Where( l => l.Something == "foo" ).ToAsyncEnumerable().Single():
SELECT [l].[ID], [l].[Something]
FROM [Ls] AS [l]
WHERE [l].[Something] = N'foo'
If that approach still leaves you squirmish then, as an alternative, consider defining extension methods like this one:
using System.Linq;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Query.Internal;
static class Extensions
{
public static Task<T> SingleAsync<T>( this IQueryable<T> source ) =>
source.Provider is IAsyncQueryProvider
? EntityFrameworkQueryableExtensions.SingleAsync( source )
: Task.FromResult( source.Single() );
}
According to the official Microsoft documentation for EF Core (all versions, including the current 2.1 one):
This API supports the Entity Framework Core infrastructure and is not intended to be used directly from your code. This API may change or be removed in future releases.
Source: https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.query.internal.asynclinqoperatorprovider.toasyncenumerable?view=efcore-2.1
p.s. I personally found it problematic in combination with the AutoMapper tool (at least, until ver. 6.2.2) - it just doesn't map collection of type IAsyncEnumerable (unlike IEnumerable, with which the AutoMapper works seamlessly).
I took a peek at the source code of Single (Line 90).
It cleary illustrates that the enumerator is only advanced once (for a successful operation).
using (var e = source.GetEnumerator())
{
if (!await e.MoveNext(cancellationToken)
.ConfigureAwait(false))
{
throw new InvalidOperationException(Strings.NO_ELEMENTS);
}
var result = e.Current;
if (await e.MoveNext(cancellationToken)
.ConfigureAwait(false))
{
throw new InvalidOperationException(Strings.MORE_THAN_ONE_ELEMENT);
}
return result;
}
Since this kind of implementation is as good as it gets (nowadays), one can say with certainty that using the Ix Single Operator would not harm performance.
As for SingleAsync, you can be sure that it is implemented in a similar manner, and even if it is not (which is doubtful), it could not outperform the Ix Single operator.
Is it possible to use a function import in a where clause in entity framework? I have tried the following, but I get a rather cryptic exception to which I cannot find any information about:
var q = MyContext.MyEntities.Where("MyContext.MyFunction(it.ID)")
(The function is set to return a Boolean)
System.Data.EntitySqlException: 'MyContext.MyFunction' cannot be resolved into a valid type constructor or function., near WHERE predicate, line 6, column 21..
Regards
Lee
The query you are trying to write is composing a call to a FunctionImport with a query over an EntitySet.
But because FunctionImports are wrappers around StoredProcedures, which are non-composable, this just won't work.
In order for something like this to work, theoretically the function would need to be a wrapper around something composable like a TVF (Table Value Function). But unfortunately TVFs aren't supported in the Entity Framework today.
Alex
I don't think you want to pass this expression as a string. You want a proper lambda expression like:
MyContext.MyEntities.Where( entity => MyContext.MyFunction(entity.ID ) );