Entity framework exists clause with lambda - entity-framework

I've seen an answer to a similar problem (Entity framework and Exists clause), but not using lambda and I would like to understand what is wrong,
whether it is my mapping or if is the way to use. I'm using Entity Framework 5.
Here is my mapping:
public class Attribute : EntityWithGuid
{
... Other mappings ....
[InverseProperty("Attribute")]
public virtual ICollection<CategoryAttribute> CategoryAttributes { get; set; }
}
public class Category : EntityWithGuid
{
... Other mappings ....
[InverseProperty("Category")]
public virtual ICollection<CategoryAttribute> CategoryAttributes { get; set; }
}
public class CategoryAttribute : EntityWithGuid
{
... Other mappings ....
[ForeignKey("Category_Id")]
public virtual Category Category { get; set; }
public string Category_Id { get; set; }
[ForeignKey("Attribute_Id")]
public virtual Attribute Attribute { get; set; }
public string Attribute_Id { get; set; }
}
If I run the command below, it works fine and the result SQL is
var query = Attribute.Where(x => !x.CategoryAttributes.Any());
SELECT
Extent1.Id,
Extent1.Name,
Extent1.Type,
Extent1.Active
FROM
Attribute AS Extent1
WHERE
NOT EXISTS( SELECT
1 AS C1
FROM
CategoryAttribute AS Extent2
WHERE
Extent1.Id = Extent2.Attribute_Id)
But if I put one more clause appears a Project2 alias for Attribute instead Extent1 and gives error because inside the exists clause within the remains Extent1.Id
var query = Attribute.Where(x => !x.CategoryAttributes.Any(y=>y.Category_Id == idcategory));
SELECT
Project2.Id,
Project2.Name,
Project2.Type,
Project2.Active
FROM
Attribute AS Project2
WHERE
NOT EXISTS( SELECT
1 AS C1
FROM
CategoryAttribute AS Extent2
WHERE
(Extent1.Id = Extent2.Attribute_Id)
AND (Extent2.Category_Id = #p__linq__0))
The exception is
Unknown column 'Extent1.Id' in 'where clause'

Found the problem.
After performing a simple query and receive an error similar thought something was wrong. Below the query, SQL and the undesirable alias Project
User.Where(x => x.Name.Contains(partialName))
SELECT
Project1.Id,
Project1.Name
FROM
User AS Project1
WHERE
(LOCATE('Marcelo', Extent1.Name)) > 0
ORDER BY Project1.Name ASC
Recently I upgraded MySql and with it came the new Net connector, version 6.7.4. I went back to the previous Net connector version (6.5.6) and everything is working again.

Related

EF6 and EFCore context generate different SQL scripts causing performance issue in EFCore

I have recently been upgrading an old solution from EF6 to EFCore and Net48 to Net5.0.
So after migrating to EFCore, I have noticed some of the queries performing alright in EF6 are now SQL timeout in EFCore.
I imported both the old EF6 and EFCore DbContext connections into LINQPad, and investigated the generated SQL.
These are my Entities:
[Table("Asset")]
public class Asset
{
public Asset()
{
MonitoringLogs = new HashSet<MonitoringLog>();
}
[Key]
public int AssetId { get; set; }
public virtual ICollection<MonitoringLog> MonitoringLogs { get; set; }
}
[Table("MonitoringLog")]
public class MonitoringLog
{
[Key]
public int MonitoringLogId { get; set; }
public DateTime LogUTCDateTime { get; set; }
public string OtherProperty { get; set; }
[ForeignKey(nameof(Asset))]
public int AssetId { get; set; }
public virtual Asset Asset { get; set; }
}
This is my LINQ Query:
this.Assets.SelectMany(r => r.MonitoringLogs.OrderByDescending(t => t.LogUTCDateTime).Take(1)).Dump();
In EFCore the generated SQL is:
SELECT [t0].[MonitoringLogId], [t0].[AssetId], [t0].[LogUTCDateTime], [t0].[OtherProperty]
FROM [Asset] AS [a]
INNER JOIN (
SELECT [t].[MonitoringLogId], [t].[AssetId], [t].[LogUTCDateTime], [t].[OtherProperty]
FROM (
SELECT [m].[MonitoringLogId], [m].[AssetId], [m].[LogUTCDateTime], [m].[OtherProperty], ROW_NUMBER() OVER(PARTITION BY [m].[AssetId] ORDER BY [m].[LogUTCDateTime] DESC) AS [row]
FROM [MonitoringLog] AS [m]
) AS [t]
WHERE [t].[row] <= 1
) AS [t0] ON [a].[AssetId] = [t0].[AssetId]
GO
It seems to generate a ROW_NUMBER() with a Partition by AssetId. Then filters out the record with RowNumber == 1. Now I don't see why all this is necessary when TOP (1) ORDER BY LogUTCDateTime can solve this.
Execution Plan: https://www.brentozar.com/pastetheplan/?id=H1a3J6-dY
While the SQL generated in EF6 is:
SELECT
[Limit1].[MonitoringLogId] AS [MonitoringLogId],
[Limit1].[LogUTCDateTime] AS [LogUTCDateTime],
[Limit1].[OtherProperty] AS [OtherProperty],
[Limit1].[AssetId] AS [AssetId]
FROM [dbo].[Asset] AS [Extent1]
CROSS APPLY (SELECT TOP (1) [Project1].[MonitoringLogId] AS [MonitoringLogId], [Project1].[LogUTCDateTime] AS [LogUTCDateTime], [Project1].[OtherProperty] AS [OtherProperty], [Project1].[AssetId] AS [AssetId]
FROM ( SELECT
[Extent2].[MonitoringLogId] AS [MonitoringLogId],
[Extent2].[LogUTCDateTime] AS [LogUTCDateTime],
[Extent2].[OtherProperty] AS [OtherProperty],
[Extent2].[AssetId] AS [AssetId]
FROM [dbo].[MonitoringLog] AS [Extent2]
WHERE [Extent1].[AssetId] = [Extent2].[AssetId]
) AS [Project1]
ORDER BY [Project1].[LogUTCDateTime] DESC ) AS [Limit1]
Execution Plan: https://www.brentozar.com/pastetheplan/?id=SJLQxTbdY
With small datasets, there is not much impact on the Query - but with large data sets this is causing an SQL Timeout.
Here is a Repo with both EF6 and EFCore context, the Repo also includes the SQL script to generate the DB/Tables and create some sample data:
https://github.com/mdawood1991/EFIssue
NOTE: The above example is an extracted example from my large solution.
This appears to be a known issue with EF Core: https://github.com/dotnet/efcore/issues/17936
The generally accepted work-around looks to be adding a ToArray() into the inner condition to force EF to alter the generation without resorting to the row counter:
this.Assets.SelectMany(r => r.MonitoringLogs
.OrderByDescending(t => t.LogUTCDateTime)
.Take(1)
.ToArray()
.FirstOrDefault()).Dump();

EF Core Inner join instead Left

My query with Include generates sql with Inner join instead Left. My FK is nullable, so I can't explain such behavior. With nullable FK I am expect normal Left join.
Have I missed something?
Linq query:
var projectEntity = await _context.Projects
// few more includes were omitted
.Include(p => p.Invoice)
.FirstOrDefaultAsync(c => c.ProjectId == id);
Classes:
[Table("InvoicedHistory")]
public class InvoiceHistory
{
[Key]
[Column("InvoicedHistory_ID")]
public int InvoicedHistoryId { get; set; }
// few properties omitted
[Column("Project_ID")]
public int? ProjectId { get; set; }
}
public class Project
{
public int ProjectId { get; set; }
// few properties were omitted
[ForeignKey(nameof(InvoiceHistory.ProjectId))]
public virtual InvoiceHistory Invoice { get; set; }
}
Project class also use fluent api:
modelBuilder.Entity<Project>(entity =>
{
entity.ToTable("Projects");
entity.HasKey(e => e.ProjectId)
.HasName("PK_Project_Project_ID_Project");
// few statements were omitted
});
Sql which was generated: (Was hard to clean up this query. It contains several joins to include data for properties I have omitted)
SELECT [t].[Project_ID], [t].[Project_Client], [t].[Project_IrsDate], [t].[Project_Name], [t].[Client_ID], [t].[Client_Name], [t].[InvoicedHistory_ID], [t].[DateSubmitted], [t].[Project_ID0], [t0].[Debitor_ID], [t0].[Project_ID], [t0].[Debitor_ID0], [t0].[Address_Number], [t0].[Alias], [t0].[Alpha_Name], [t0].[Co], [t0].[Country_ID], [t0].[Currency_ID], [t0].[Havi_YesOrNo]
FROM (
SELECT TOP(1) [p].[Project_ID], [p].[Project_Client], [p].[Project_IrsDate], [p].[Project_Name], [c].[Client_ID], [c].[Client_Name], [i].[InvoicedHistory_ID], [i].[DateSubmitted], [i].[Project_ID] AS [Project_ID0]
FROM [Projects] AS [p]
INNER JOIN [Clients] AS [c] ON [p].[Project_Client] = [c].[Client_ID]
INNER **<<<<<<<<(expect LEFT)** JOIN [InvoicedHistory] AS [i] ON [p].[Project_ID] = [i].[InvoicedHistory_ID]
WHERE [p].[Project_ID] = 19922
) AS [t]
LEFT JOIN (
SELECT [p0].[Debitor_ID], [p0].[Project_ID], [d].[Debitor_ID] AS [Debitor_ID0], [d].[Address_Number], [d].[Alias], [d].[Alpha_Name], [d].[Co], [d].[Country_ID], [d].[Currency_ID], [d].[Havi_YesOrNo]
FROM [ProjectDebitors] AS [p0]
INNER JOIN [Debitors] AS [d] ON [p0].[Debitor_ID] = [d].[Debitor_ID]
) AS [t0] ON [t].[Project_ID] = [t0].[Project_ID]
ORDER BY [t].[Project_ID], [t].[Client_ID], [t].[InvoicedHistory_ID], [t0].[Debitor_ID], [t0].[Project_ID], [t0].[Debitor_ID0]
Look at this line -
INNER <<<<<<<<(expect LEFT)<<<<<< JOIN [InvoicedHistory] AS [i] ON [p].[Project_ID] = [i].[InvoicedHistory_ID]
Inner join makes my query return nothing, because I have no invoice info. If I manually replace it with Left join, sql query will return me all necessary data.
I think you can use Fluent API to get your desired result:
modelBuilder.Entity<Project>()
.HasOne(p => p.Invoice)
.WithOne()
.HasForeignKey(ih => ih.ProjectId);
This should change it to a left join because we didn't specify .IsRequired()
As mentioned in the following SO Answer - Equivalent for .HasOptional in Entity Framework Core 1 (EF7)
You will not find an equivalent method in EF 7. By convention, a property whose CLR type can contain null will be configured as optional. So what decide if the relationship is optional or not is if the FK property is nullable or not respectively.
and
In case of your FK property is value type like int, you should declare it as nullable (int?).
Now most likely your problem with annotations is that the following is not doing what you think it is:
[ForeignKey(nameof(InvoiceHistory.ProjectId))]
//Does not resolve to:
[ForeignKey("InvoiceHistory.ProjectId")]
//Does resolve to:
[ForeignKey("ProjectId")]
Now even if that is what you are looking for, the order of operations for the ForeignKey detection is to check the parent type then the property type.
public class InvoiceHistory
{
public int? ProjectId { get; set; }
}
public class Project
{
public int ProjectId { get; set; }
// this is pointing to Project.ProjectId
// and Project.ProjectId is not nullable
// so the join becomes an inner join
// and really only works because they both have the same name
[ForeignKey(nameof(InvoiceHistory.ProjectId))]
public virtual InvoiceHistory Invoice { get; set; }
}
If you wanted this to work as pointing to the Property Type, you need to rename the InvoiceHistory name:
public class InvoiceHistory
{
public int? ProjectFk { get; set; }
}
public class Project
{
public int ProjectId { get; set; }
// this is pointing to InvoiceHistory.ProjectFk
// because there is no Project.ProjectFk
[ForeignKey(nameof(InvoiceHistory.ProjectFk))]
public virtual InvoiceHistory Invoice { get; set; }
}
EntityFramework Data Annotations
If you wanted to see it create bad SQL you could do this:
public class InvoiceHistory
{
public int? ProjectId { get; set; }
}
public class Project
{
public int ProjectFk { get; set; }
[ForeignKey("ProjectFk")]
public virtual InvoiceHistory Invoice { get; set; }
}
EF will then create:
INNER JOIN [InvoicedHistory] AS [i] ON [p].[Project_ID] = [i].[ProjectFk]
And will cause a SqlException with the message something like Invalid column name.

Query Postgres Json Field using EF Core 5

I have the following table definition
[Table("MyTable")]
public class MyTable: BaseEntity
{
[Required]
public string A{ get; set; }
[Required]
[Column(TypeName = "json")]
public string B{ get; set; }
}
Column B looks like this:
{"Data": [{"Id":"b8a3cbbc-a4d6-4697-8a0b-cb1d15be179d"}]} (aside from Id there are other properties but for brevity I removed them)
In Entity Framework I want to match all MyTable's where the Id in B is a certain value and A has a certain value. I have tried a lot of things and get numerous errors. How can I add to the following code to achieve what I want?
var results =
_repository.Get<MyTable>(_ => _.A == "Something" && _.B = ???);
You can use "EF.Functions.JsonContains" function, but the B column needs to be "jsonb" type instead "json".
[Required]
[Column(TypeName = "jsonb")]
public string B { get; set; }
Example
var search = "[{\"Id\": \"b8a3cbbc-a4d6-4697-8a0b-cb1d15be179d\"}]";
var results = _context.MyTable2.Where(_ => _.A == "Something" &&
EF.Functions.JsonContains(_.B, search));
Similar answer HERE
Also, you can type your query and use Dapper.
Example
with temp AS(
select t."Id", t."A", json_array_elements(t."B"->'Data') as B1 from "MyTable" t
)
select * from temp t
where
t."A"='Something' and
t.b1->>'Id'='b9a3cbbc-a4d6-4697-8a0b-cb1d15be179a'

Entity Framework Core 2.1 throwing ArgumentException on left join

I recently upgraded my project from .NET Core 2.0 to 2.1. I use Entity Framework Core and have several LINQ expressions that include left joins. The following code is an example:
var tickets = (from ticket in dbContext.Tickets
join member in dbContext.Members on ticket.MemberID equals member.ID into memberLEFT
from member in memberLEFT.DefaultIfEmpty()
join memberType in dbContext.MemberTypes on member.MemberTypeID equals memberType.ID into memberTypeLEFT
from memberType in memberTypeLEFT.DefaultIfEmpty()
select memberType)
.ToList();
If .NET Core 2.0, this line executed with no exceptions. In .NET Core 2.1, it's throwing the following ArgumentException:
System.ArgumentException: 'Field
'Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor+TransparentIdentifier`2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,System.Collections.Generic.IEnumerable`1[Microsoft.EntityFrameworkCore.Storage.ValueBuffer]].Inner'
is not defined for type
'Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor+TransparentIdentifier`2[Microsoft.EntityFrameworkCore.Storage.ValueBuffer,Microsoft.EntityFrameworkCore.Storage.ValueBuffer]''
If I change the LINQ to perform inner joins instead of left, then it executes no problem. The joins need to be left joins though, so I can't just update all of my code to do inner joins.
Does anyone know why EF Core 2.1 is now throwing ArgumentExecptions when performing left joins, and how to get around the issue?
From tools > Options check Enable Just My code
I'm not sure why you creating your joins on your own instead of using the EF releationship model. I've taken your example and changed it a little bit.
Models:
public class Ticket
{
[Key]
public int ID { get; set; }
[Required]
[StringLength(200)]
public string Description { get; set; }
public int? MemberID { get; set; }
[ForeignKey(nameof(MemberID))]
public Member Member { get; set; }
}
public class Member
{
[Key]
public int ID { get; set; }
[Required]
[StringLength(200)]
public string Description { get; set; }
public int? MemberTypeID { get; set; }
public ICollection<Ticket> Tickets { get; set; }
[ForeignKey(nameof(MemberTypeID))]
public MemberType MemberType { get; set; }
}
public class MemberType
{
[Key]
public int ID { get; set; }
[Required]
[StringLength(200)]
public string Description { get; set; }
}
To define an outer join just make the foreign-key nullable.
When you've created your models like this you can just include the properties in your query:
Tickets = (from t in _dbContext.Tickets
.Include(t => t.Member)
.ThenInclude(m => m.MemberType)
select t).ToList();
The generated sql looks like this:
SELECT [t].[ID], [t].[Description], [t].[MemberID], [t.Member].[ID], [t.Member].
[Description], [t.Member].[MemberTypeID], [t.Member.MemberType].[ID],
[t.Member.MemberType].[Description]
FROM [Tickets] AS [t]
LEFT JOIN [Members] AS [t.Member] ON [t].[MemberID] = [t.Member].[ID]
LEFT JOIN [MemberTypes] AS [t.Member.MemberType] ON [t.Member].[MemberTypeID] =
[t.Member.MemberType].[ID]
Hope that helps your issue.
For me the best solution i found here :[https://stackoverflow.com/a/58017017/1983909]
from stackoverflow user: Hakan Çelik MK1
Using Visual Studio 2019 (I assume 2017 too) you can disable this exception just for System.Linq.Expressions.
The way to do it is as follows :
Open Exception Settings window. Debug -> Windows -> Exception Settings (Ctrl+Alt+E)
There is a search box at the top right of the window, type "System.ArgumentException" in it
You will see the exception listed in the window, right click on it and select "Edit Conditions" from the menu
Edit Conditions allows you to set the conditions for the debugger to break when the exception is thrown
From left to right, select Module Name, select Not Equals and type "System.Linq.Expressions.dll" in the edit box at the right
Click OK and close the window

Automapper MapFrom Subquery

UPDATE: Issue fixed in current release https://github.com/AutoMapper/AutoMapper/issues/742
Using AutoMapper 3.3, QueryableExtensions and EF6
I have a user requirement to return a Count of other users created before the current user.
I have the following
public class User
{
public int Id {get;set;}
public string Name { get; set; }
public DateTime? DateActivated {get;set;}
}
public class UserViewModel
{
public int Id {get;set;}
public string Name { get; set; }
public DateTime? DateActivated {get;set;}
public int position {get;set;}
}
public class AutoMapperConfig
{
public static void ConfigAutoMapper() {
var db = new DB();
Mapper.CreateMap<User, UserViewModel>()
.ForMember(a => a.position, opt => opt.MapFrom(src => db.Users.Where(u => u.DateActivated < src.DateActivated).Count()));
Mapper.AssertConfigurationIsValid();
}
}
and finally the actual mapping:
user = db.Users.Project().To<T>(new { db = db }).FirstOrDefault(a => a.id == id);
db is a local DbContext variable and I'm using AutoMapper parameters to insert it into the mapper (https://github.com/AutoMapper/AutoMapper/wiki/Queryable-Extensions#parameterization)
So far so good, this compiles and runs, but the result for user.position is 0
I checked with sql profiler and here is the relevant section of the generated query:
CROSS JOIN (SELECT
COUNT(1) AS [A1]
FROM [dbo].[Users] AS [Extent4]
WHERE ([Extent4].[DateActivated] < [Extent4].[DateActivated]) ) AS [GroupBy1]
Notice how it refers to Extent4.DateActivated in both sides of the comparison, which will obviously yield 0 results.
So is what i'm doing just not possible? or did I do something wrong.
(and if I could do away with the parameterization and have automapper be able to refer to the current underlying db context that would be a bonus).
Thank you
EDIT
Just to make it clear, this count will be dynamic, since there are other criteria to filter prior users that I omitted from simplified the example.