How to Parse an int in an EF Core 3 Query? - entity-framework

Upon upgrading to EF Core 3, I am getting the following error at the following code:
System.InvalidOperationException: 'The LINQ expression 'DbSet
.Max(c => Convert.ToInt32(c.ClaimNumber.Substring(c.ClaimNumber.Length - 6)))'
could not be translated. Either rewrite the query in a form that can
be translated, or switch to client evaluation explicitly by inserting
a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or
ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for
more information.'
var maxId = Db.Claims
.Select(c => c.ClaimNumber.Substring(c.ClaimNumber.Length - 6))
.Max(x => Convert.ToInt32(x));
I have also tried using int.Parse instead of Convert.ToInt32, and it produces the same error. I understand the error message. However, it's trivial to get SQL Server to parse a string to an int in T-SQL with CAST or CONVERT, I would hope there's a simple way to write the query so that it translates to a server-side operation right?
UPDATE After Claudio's excellent answer, I thought I should add some info for the next person who comes along. The reason I believed the parsing was the problem with the above code is because the following runs without error and produces the right result:
var maxId = Db.Claims
.Select(c => c.ClaimNumber.Substring(c.ClaimNumber.Length - 6))
.AsEnumerable()
.Max(x => int.Parse(x));
However, I dug deeper and found that this is the SQL query EF is executing from that code:
SELECT [c].[ClaimNumber], CAST(LEN([c].[ClaimNumber]) AS int) - 6
FROM [Claims] AS [c]
WHERE [c].[ClaimNumber] IS NOT NULL
That is clearly not doing anything like what I wanted, and therefore, Claudio is right that the call to Substring is, in fact, the problem.

Disclaimer: although feasable, I strongly recommed you do not use type conversion in your query, because causes heavy query performance degradation.
Fact is that Convert.ToInt(x) part is not the problem here. It is c.ClaimsNumber.Substring(c.ClaimNumber.Length - 6), that the EF Core translator isn't able to translate in T-SQL.
Despite RIGHT function exists in Sql Server, also, you won't able to use it with current versions of EF Core (last version is 3.1.2 at the moment I'm writing).
Only solution to get what you want is to create a Sql Server user function, map it with EF Core and use it in your query.
1) Create function via migration
> dotnet ef migrations add CreateRightFunction
In newly created migration file put this code:
public partial class CreateRightFunctions : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(#"
CREATE FUNCTION fn_Right(#input nvarchar(4000), #howMany int)
RETURNS nvarchar(4000)
BEGIN
RETURN RIGHT(#input, #howMany)
END
");
}
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(#"
DROP FUNCTION fn_Right
");
}
}
Then run db update:
dotnet ef database update
2) Map function to EF Core context
In your context class[DbFunction("fn_Right")]
public static string Right(string input, int howMany)
{
throw new NotImplementedException(); // this code doesn't get executed; the call is passed through to the database function
}
3) Use function in your query
var maxId = Db.Claims.Select(c => MyContext.Right(c.ClaimNumber, 6)).Max(x => Convert.ToInt32(x));
Generated query:
SELECT MAX(CONVERT(int, [dbo].[fn_Right]([c].[ClaimNumber], 6)))
FROM [Claims] AS [c]
Again, this is far from best practice, I think you should consider to add an int column to your table to store this "number", whatever it represents in your domain.
Also, first time last 6 characters of ClaimNumber contain a non-digit character, this won't work anymore. If the ClaimNumber is input by a human, sooner or later this will happen.
You should code and design your database and application for robustness, even if you're super sure that those 6 characters will always represent a number. They could not do it forever :)

Please change your code as below. It's working for me in Dotnet core 3.1 version
var maxId = Db.Claims.Select(c => c.ClaimNumber.Substring(c.ClaimNumber.Length - 6))
.Max(x => (Convert.ToInt32((x == null)? "0" : x.ToString())));

Related

EF Core 3 GroupBy multiple columns Count Throws with extensions but linq works

Here is the one that throws full exception:
var duplicateCountOriginal = _db.TableName
.GroupBy(g => new {g.ColumnA, g.ColumnB, g.ColumnC})
.Count(g => g.Count() > 1);
Exception:
System.ArgumentException: Expression of type 'System.Func2[System.Linq.IGrouping2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer],Microsoft.EntityFrameworkCore.Storage.ValueBuffer]' cannot be used for parameter of type 'System.Func2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer]' of method 'System.Collections.Generic.IEnumerable1[Microsoft.EntityFrameworkCore.Storage.ValueBuffer] Select[ValueBuffer,ValueBuffer](System.Collections.Generic.IEnumerable1[Microsoft.EntityFrameworkCore.Storage.ValueBuffer], System.Func2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer])' (Parameter 'arg1')
But the same thing works when it is written as linq (I prefer extensions)
var duplicateCount =
from a in _db.TableName
group a by new {a.ColumnA, a.ColumnB, a.ColumnC}
into g
where g.Count() > 1
select g.Key;
duplicateCount.Count()
I am unable to understand why one works or the other doesn't.
Also if I change the first one a little bit based on EF Core 3 changes like the following
var duplicateCountOriginal = _db.TableName
.GroupBy(g => new {g.ColumnA, g.ColumnB, g.ColumnC})
.AsEnumerable()
.Count(g => g.Count() > 1);
I get the following exception:
System.InvalidOperationException: Client projection contains reference to constant expression of 'Microsoft.EntityFrameworkCore.Metadata.IPropertyBase' which is being passed as argument to method 'TryReadValue'. This could potentially cause memory leak. Consider assigning this constant to local variable and using the variable in the query instead. See https://go.microsoft.com/fwlink/?linkid=2103067 for more information.
According to me, the link given by ms has no meaning to the whatever problem here is.
Please LMK if there is any logical explanation.
There is no logical explanation. Just EF Core query translation is still far from perfect and have many defects/bugs/unhandled cases.
In this particular the problem is not the query syntax or method syntax (what you call extensions), but the lack of Select after GroupBy. If you rewrite the method syntax query similar to the one using query syntax, i.e. add .Where, .Select and then Count:
var duplicateCount = _db.TableName
.GroupBy(g => new {g.ColumnA, g.ColumnB, g.ColumnC})
.Where(g => g.Count() > 1)
.Select(g => g.Key)
.Count();
then it will be translated and executed successfully.

EntityFramework and Expressions translation

I have a entity class Foo I've made partial containing the following code
private readonly static Expression<Func<Foo, int>> MyKeyExpression = (x) => x.Key;
public int MyKey
{
get { return MyKeyExpression.Compile()(this); }
}
The above works as in I can use MyKey in EntityFrameworks linq queries.
Why don't the following work?
private readonly static Expression<Func<Foo, int>> MyKeyExpression = (x) => x.Key;
// Set in the constructor with
// _myKeyDelegate = MyKeyExpression.Compile();
private readonly Func<Foo,int> _myKeyDelegate;
public int MyKey
{
get { return _myKeyDelegate(this); }
}
I understand the difference between a delegate and an expression(or maybe i don't?) but is confused how EntityFramework is able to interpret the property differently since MyKeyExpression.Compile() returns just that delegate which is then invoked returning an int. Perhaps its my lack of understanding of how the compiler actually handles C# Properties?
Example of usage where first example works but second examples throw a exception about not being able to translate it to SQL.:
dbContext.Foo.Delete(x => x.MyKey == 5)
I would say you don't fully understand difference between delegates and expressions.
Delegate is a reference to code compiled into IL. Only thing you can with it is execute it within .net CLR.
Expression object is a expression represented as tree, (you can think of AST). You can compile it to IL (Compile method) or you can inspect it and generate code for other execution environment, for example into SQL (that's what EF does).
When C# compiler compiles code, first it builds syntax tree and then compiles it. Basically expression is result of first part without second, so you could use SQL translator to compile it to SQL. Or you can write you own and translate it to anything else.
It's very strange what you are saying...
EF ignores the content of the getter and the setter of a mapped property (MyKey).
The query should be generated with a WHERE clause based on MyKey independent of what getter does.
How did you map the MyKey property? There is the setter missing so EF does not generate a field on the DB table and does not map it automatically.

Why is EqualityComparer<string>.Default not working?

var dic = context.Treasure.Include("TreasureShare")
.Where(t => t.TreasureShare.IsShared && t.TreasureShare.EvaluationContent.Contains(keyword))
.ToDictionary(t => t.ProductUrl, t => t.ProductId, EqualityComparer<string>.Default);
I got an error:
An item with the same key has already been added.
So why the equalitycomparer not work, and how to use a equalitycomparer to get different records while querying to database.
Updated:
I know IEqualityComparer can only be executed locally, but I didn't get an error like:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable1[Panli.Service.Share.DataAccess.DbData.Treasure] Distinct[Treasure](System.Linq.IQueryable1[Panli.Service.Share.DataAccess.DbData.Treasure], System.Collections.Generic.IEqualityComparer`1[Panli.Service.Share.DataAccess.DbData.Treasure])' method, and this method cannot be translated into a store expression.
except I change the codes to below:
dic = context.Treasure.Include("TreasureShare")
.Where(t => t.TreasureShare.IsShared && t.TreasureShare.EvaluationContent.Contains(theme))
.Distinct(new TreasureEqualityComparer()).ToDictionary(t => t.ProductUrl, t => t.ProductId);
This is my TreasureEqualityComparer:
public class TreasureEqualityComparer : EqualityComparer<Treasure>
{
public override bool Equals(Treasure x, Treasure y)
{
return x.ProductUrl.ToLower() == y.ProductUrl.ToLower();
}
public override int GetHashCode(Treasure obj)
{
return obj.ProductUrl.ToLower().GetHashCode();
}
}
So why not throw an exception just like the Distinct() when I use ToDictionary(..) which has an IEqualityComparer param ? Anyone can explain this ?
So why not throw an exception?
The ToDictionary part is executed in memory. This is apparent when you investigate the SQL that is executed: nothing that shows any preparation for a conversion to Dictionary.
The query expression with Distinct on the other hand is translated into SQL as a whole (except it isn't because it fails). EF tries to let the database do the hard work of returning distinct values, but of course a comparer can't be translated into SQL, so this overload of Distinct() is not supported.
As for the duplicate key: apparently there are duplicate URL's (ignoring case). Maybe you should use group by.
Dictionary key has to be unique. In this case you are using ProductUrl as dictionary key and ProductId as value, unfortunately as the error indicated there are more then one records in Table having same ProductUrl. So you can't use it as a dictionary key.

NHibernate QueryOver Conversion Error On DB2 Date Type

I'm new to NHibernate and I am hoping I can find some assistance in tracking down the source of a conversion error I'm getting when trying to use a DateTime comparison for a predicate.
return _session.QueryOver<ShipmentSegment>()
.Where(ss => ss.SegmentOrigin == selOrig)
// Whenever I add the predicate for the SegmentDate below
// I receive a conversion error
.And(ss => ss.SegmentDate == selDate)
.List<ShipmentSegment>();
Exception
NHibernate.Exceptions.GenericADOException was unhandled by user code
Message=could not execute query
[ SELECT this_.ajpro# as ajpro1_28_0_, this_.ajleg# as ajleg2_28_0_, this_.ajpu# as ajpu3_28_0_, this_.ajlorig as ajlorig28_0_, this_.ajldest as ajldest28_0_, this_.segdate as segdate28_0_, this_.ajldptwin as ajldptwin28_0_, this_.ajlfrtype as ajlfrtype28_0_, this_.ajlfrdest as ajlfrdest28_0_, this_.ajtpmfst# as ajtpmfst10_28_0_, this_.ajspplan as ajspplan28_0_, this_.ajhload as ajhload28_0_ FROM go52cst.tstshprte this_ WHERE this_.ajlorig = #p0 and this_.segdate = #p1 ]
Name:cp0 - Value:WIC Name:cp1 - Value:3/28/2012 12:00:00 AM
Inner Exception
Message=A conversion error occurred.
Source=IBM.Data.DB2.iSeries
ErrorCode=-2147467259
MessageCode=111
MessageDetails=Parameter: 2.
SqlState=""
StackTrace:
- at IBM.Data.DB2.iSeries.iDB2Exception.throwDcException(MpDcErrorInfo
mpEI, MPConnection conn)
- at IBM.Data.DB2.iSeries.iDB2Command.openCursor()
- at IBM.Data.DB2.iSeries.iDB2Command.ExecuteDbDataReader(CommandBehavior
behavior)
- at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader()
- at NHibernate.AdoNet.AbstractBatcher.ExecuteReader(IDbCommand cmd)
- at NHibernate.Loader.Loader.GetResultSet(IDbCommand st, Boolean autoDiscoverTypes, Boolean callable, RowSelection selection,
ISessionImplementor session)
- at NHibernate.Loader.Loader.DoQuery(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
- at NHibernate.Loader.Loader.DoQueryAndInitializeNonLazyCollections(ISessionImplementor
session, QueryParameters queryParameters, Boolean returnProxies)
- at NHibernate.Loader.Loader.DoList(ISessionImplementor session, QueryParameters queryParameters)
I appreciate anything that can help point me in the right direction.
My experience with this specific iSeries exception came from when I was creating a parameter (of iDB2TypeParameter) for the ADO.NET command parameter list for a store procedure type ADO command. I had to explicitly tell what kind of iDB2DbType to use: iDB2DbType.Date, iDB2DbType.Time or iDB2DbType.TimeStamp. The iSeries ADO provider can't possibly know which of the three types to use when you create a parameter from a .NET System.DateTime type.
new iDB2Parameter(parameterName, iDB2DbType.Date){ Value = myValue };
new iDB2Parameter(parameterName, iDB2DbType.Time){ Value = myValue };
new iDB2Parameter(parameterName, iDB2DbType.TimeStamp){ Value = myValue };
I realize that you are not creating your parameters manually like this example but instead using NHibernate. So, I would make sure that NHibernate's LINQ provider for DB2/iSeries is aware of this. That's nothing against NHibernate, I just find it nearly impossible to find good, solid ORM for DB2/iSeries.

Entity Framework - Union causes "Unable to create a constant value of type.."

To select all Schedulings which are active, I have the following code:
var allSchedulesOnALine = CurrentUser.Lines.SelectMany(o => o.Scheduling).Where(o => o.Active);
var allSchedulesUnscheduled = Entities.SchedulingSet
.Where(o => o.Line == null && o.Site.Id == CurrentUser.Site.Id &&
o.Factory == CurrentUser.Factory && o.Active);
IEnumerable<Scheduling> allSchedules = allSchedulesUnscheduled.Union(allSchedulesOnALine);
foreach(Scheduling schedule in allSchedules.OrderBy(o => o.Ordering))
{
//Do Stuff
}
(Factory is an int)
When I run this code, I get this cryptic error on the foreach line:
Unable to create a constant value of type 'System.Collections.Generic.IEnumerable`1'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.
Strangely enough, I can enumerate both allSchedulesOnALine and allSchedulesUnscheduled separately. Even stranger, if I reorder the union:
IEnumerable<Scheduling> allSchedules = allSchedulesOnALine.Union(allSchedulesUnscheduled);
It works fine!
Does anyone have any idea why this would happen? Am I missing something crucial, or is this a bug?
I should mention I am using Entity Framework 3.5. EF4 is not an option for us currently - it is beyond my control :\
You're calling two different methods with your "reordering".
You don't show the types of allSchedulesOnALine or allSchedulesUnscheduled, but I'm betting allSchedulesOnALine is of type IEnumerable<Schedule> and allSchedulesUnscheduled is of type IQueryable<Schedule>.
So when you call Queryable.Union, you're asking the EF to translate the expression into SQL. But the argument you pass is of type IEnumerable<Schedule>, and it can't translate that into a query.
On the other hand, when you call Enumerable.Union, you're asking LINQ to Objects to do the whole thing in memory, which works fine, albeit perhaps slower.
So the reason the behavior is different is that you're calling two completely different methods, which do different things, but happen to have the same name. No, it's not a bug.