I have been trying to use the GroupJoin statement to do a left outer join on two tables so that i can get a list of rows in table A which does not have any items in table B. However I keep getting an exception
Exception found: Processing of the LINQ expression 'DbSet<People>
.GroupJoin(
outer: DbSet<RiskGroup>,
inner: person => (Nullable<int>)person.Id,
outerKeySelector: risk => risk.AssessorId,
innerKeySelector: (person, risks) => new {
person = person,
risks = risks
.DefaultIfEmpty()
})' by 'NavigationExpandingExpressionVisitor' failed.
which seems to imply that it is not possible in EF Core v3. Does anyone know how to get around this problem or is my LINQ incorrect below:
var Ids = destContext.People.GroupJoin(destContext.RiskGroup,
person => person.Id,
risk => risk.AssessorId,
(person, risks) => new { person, risks = risks.DefaultIfEmpty() }).ToList();
I have trying to find all rows in the person table who doesn't have rows in the RiskGroup table.
I have struggled with this myself and I would like to see if this help you
OK for starters, I am going to keep this simple. Just find things are easier if we do
this is something that every developer comes across Company and address
the first table (Company) is the table that we could have multple records
the second table (Address) we have a single record for
company => company.AddressId,
address => address.AddressId,
So we are querying using the addressId of the address, I always used tableNameId, it makes things easier to read
So for the GroupJoin you now need output to be able to be used later in the query.
(company, address) = new (company, address)
And now you need a SelectMany - this is what screws everyone up
you need a record for when there is a null in Company
s.address.DefaultIfEmpty(),
and the output from the SelectMany
(s, address) => new { company = s.company, address });
Notice, very important - company must be s.company - not just company or the LEFT JOIN will not appear.
var a = context.Companys.GroupJoin(context.Addresses,
company => company.AddressId,
address => address.AddressId,
(company, address) => new { company, address })
.SelectMany(s => s.address.DefaultIfEmpty(),
(s, address) => new { company = s.company, address });
var outA = a.ToList();
for your specific issue
var a = destContext.People.GroupJoin(destContext.RiskGroup,
person => person.Id,
risk => risk.AssessorId,
(person, risk) => new { person, risk })
.SelectMany(s => s.risk.DefaultIfEmpty(),
(s, risk) => new { person = s.person, risk })
.Where(x => x.AssesorId == null);
var outA = a.ToList();
Try that out and see if you get what you need
Related
I updated my .net core 2.2 to 5
I have a error about ef that
System.InvalidOperationException: 'The LINQ expression 'x' could not
be translated. Either rewrite the query in a form that can be
translated, or switch to client evaluation explicitly by inserting a
call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or
'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for
more information.'
public List<CustomerPageModel> GetCustomers(int AccountID)
{
return (from p in context.Customers
join f in context.Patients on p.ID equals f.CustomerID into ps
from t in ps.DefaultIfEmpty()
where p.AccountID == AccountID
select new CustomerPageModel
{
ID = p.ID,
Name = p.Name,
IsActive = p.IsActive,
TC = p.TC,
Surname = p.Surname,
Email = p.Email,
Address = p.Address,
Phone = p.Phone,
Note = p.Note,
AccountID = p.AccountID,
Pats = string.Join(",", ps.Select(x => x.Name)),
PatCount = ps.Count()
})
.GroupBy(p => p.ID)
.Select(g => g.First())
.ToList();
}
How can I convert that code?
Your problem line is:
Pats = string.Join(",", ps.Select(x => x.Name)),
Specifically, the string.Join method doesn't translate to SQL, so in previous versions of EF, it had to retrieve the data from the database then in-memory preform the string.Join function. Now EF explicitly tells you it can't run that on the database server - this is a breaking change but a design decision that tells you (the developer) that it may not have been running as efficiently as you thought it was...
To "fix" this, and based on your particular code example I'd recommend the following:
Add a pet names array property to your CustomerPageModel:
public string[] PetNames {get;set;}
And turn the Pets property into a readonly calculated string:
public string Pets { get => string.Join(",", PetNames); }
And change the problem line in your LINQ expression to:
PetNames = ps.Select(x => x.Name).ToArray()
I changed (lamb to subquery)
string.Join(",", ps.Select(x => x.Name))
to
string.Join(",", (from y in PatientList where y.CustomerID == p.ID select y.Name).ToArray()),
I made the group later (after tolist)
var test = mylist.GroupBy(p => p.ID)
.Select(g => g.First())
.ToList();
problem solved
In this EF query, for the contacts list, I'm trying to query to get records for ContactTypeA and populate results in contacts list. If there are no records for ContactTypeA, then I want it to query for records for ContactTypeB and populate results in contacts list. I tried to use DefaultIfEmpty but that method only accepts single value and not a list. 'contacts' is a List object. Any ideas or even an alternative to DefaultIfEmpty? Thanks.
select(i => new transaction{
....
contacts = contactRepository.All.Where(c => c.AccountId == i.Account.Id && contactTypeRepository.All.Any(ct => ct.ContactId == c.Id && ct.Type == ContactType.ContactTypeA)).ToList().DefaultIfEmpty((contactRepository.All.Where(c => c.AccountId == i.Account.Id && contactTypeRepository.All.Any(ct => ct.ContactId == c.Id && ct.Type == ContactType.ContactTypeB)).ToList()
}
)
Firstly, make absolutely sure your contactRepository.All() method returns IQueryable<Contact> and not IEnumerable<Contact>, IList<Contact> or the like, otherwise you are automatically loading all contacts into memory.
From there, don't be afraid to simplify your query across multiple statements to make it a lot easier to understand. You should also leverage navigation properties rather than relying on completely disconnected entities and generic repositories and manually joining these all up in huge expressions.
Ideally an Account could have a collection of Contacts, but if not, at a minimum a Contact should have ContactType references:
var accountContactsQuery = contactRepository.All
.Where(c => c.AccountId == i.AccountId); //Remains IQueryable
var contacts = accountContactsQuery
.Where(c => c.ContactTypes.Any(ct => c.Type == ContactType.ContactTypeA)
.ToList(); // Gets List of Contacts where contains at least 1 ContactTypeA type.
// If we have none, replace with results for ContactTypeB
if (!contacts.Any())
contacts = accountContactsQuery
.Where(c => c.ContactTypes.Any(ct => c.Type == ContactType.ContactTypeB)
.ToList();
This looked a bit odd in that your ContactType appears to have a ContactId, (As opposed to Contact containing a ContactTypeId?) But the above reflects the relationship in your example.
With Account containing Contacts collection:
var accountContactsQuery = accountRepoitory.All
.Where(a => true /* replace with relevant criteria */);
var contacts = accountContactsQuery
.SelectMany(a => a.Contacts)
.Where(c => c.ContactTypes.Any(ct => c.Type == ContactType.ContactTypeA)
.ToList(); // Gets List of Contacts where contains at least 1 ContactTypeA type.
// If we have none, replace with results for ContactTypeB
if (!contacts.Any())
contacts = accountContactsQuery
.SelectMany(a => a.Contacts)
.Where(c => c.ContactTypes.Any(ct => c.Type == ContactType.ContactTypeB)
.ToList();
When dealing with conditions in expressions, I can suggest returning all relevant details, then building your final "payload" based on the conditions.
For example, if querying accounts to build transactions but wanting to load ContactAs if available and Bs for each account if not available:
var transactionData = accountRepoitory.All
.Where(a => true /* replace with relevant criteria */);
.Select(a => new
{
a.AccountId,
/* populate account and common details.. */
ContactAs = a.Contacts
.Where(c => c.ContactTypes.Any(ct => c.Type == ContactType.ContactTypeA).ToList(), // Consider a Select to get relevant details...
ContactBs = a.Contacts
.Where(c => c.ContactTypes.Any(ct => c.Type == ContactType.ContactTypeB).ToList()
}).ToList(); // Executes query against DB to load relevant data...
var transactions = transactionData
.Select( t => new Transaction
{
AccountId = t.AccountId,
/* Other fields */
Contacts = t.ContactAs.Any() ? t.ContactAs : t.ContactBs
}).ToList();
Essentially use the EF Linq expressions to load the possible data, so include results for both ContactA and ContactB, then afterwards build your final projection using that data and conditionally use the ContactA or B as suited. Generally I don't advise passing Entities back (Actual Contact entities) but project into minimally viable view models in the first EF query using Select.
I want to write the following SQL as a Linq to Entities statement
select min(enddate), Code
from t1
inner join t2
on t1.t2id = t2.id
group by Code
What I have so far is
var data = ctx.t1.Join(ctx.t2,
one => one.t2id,
two => two.Id,
(one, two) => new {one, two})
.GroupBy(onetwo => onetwo.one.t2id)
.Min(onetwo => WHAT GOES HERE?);
I think the only thing I am missing is what goes in WHAT GOES HERE?. I could be wrong and have gone totally astray but as far as I know this is the only thing I am missing. Any idea what I can do?
What you should be using is a Select not a Min, otherwise you will only be able to return a single value. For example, something like this:
var data = ctx.t1
.Join(ctx.t2, one => one.t2id, two => two.Id, (one, two) => new {one, two})
.GroupBy(onetwo => onetwo.one.t2id)
.Select(x => new
{
Code = x.Key,
MinDate = x.Min(g => g.one.EndDate) //This may need to be g.two.EndDate
});
I have a process in which an entity changes its state, i want to get the States that current user is allowed to transfer to, i have the following expression which when evaluated i'm getting an NotSupportedException with the message:
Unable to create a constant value of type
'Microsoft.AspNet.Identity.EntityFramework.IdentityUserRole'. Only
primitive types or enumeration types are supported in this context.
var user = db.Users.Single(...); //Successfully Retrieve the user
var states = db.StateTransitions
.Join( db.RoleStateTransitions
.Join( user.Roles,
rst => rst.RoleID,
ur => ur.RoleId,
( rst, ur ) => rst ),
wst => wst.StateTransitionID,
rst => rst.StateTransitionID,
( wst, rst ) => wst.FinalState ) ;
As you can see joining with user.Roles (collection of IdentityUserRole) is the problem. What is wrong with the previous expression?
The problem is that user.Roles is a local collection of objects. EF has no way of translating local objects into SQL (even though they are mapped objects). But you can make it work by reducing the local collection to primitive types:
var userRoleIds = db.Users.Where(u => u.UserId == id)
.SelectMany(u => u.Roles.Select(r => r.RoleId))
.ToList();
var states = from wst in db.StateTransitions
join rst in db.RoleStateTransitions
.Where(x => userRoleIds.Contains(x.RoleId))
on wst.StateTransitionID equals rst.StateTransitionID
select wst.FinalState;
Or maybe even one query:
var states = from wst in db.StateTransitions
join rst in db.RoleStateTransitions
on wst.StateTransitionID equals rst.StateTransitionID
join role in db.Roles
on rst.ROleId equals role.RoleId
where role.Users.Any(u => u.UserId == id)
select wst.FinalState;
(Assuming there is a many to many between User and Role).
I need to filter a dealer based on id and the uncomplete checkins
Initially, it returned the dealer based only on id:
// TODO: limit checkins to those that are not complete
return this.ObjectContext.Dealers
.Include("Groups")
.Include("Groups.Items")
.Include("Groups.Items.Observations")
.Include("Groups.Items.Recommendations")
.Include("Checkins")
.Include("Checkins.Inspections")
.Include("Checkins.Inspections.InspectionItems")
.Where(d => d.DealerId == id)
.FirstOrDefault();
As you can see the requirement is to limit the checkins.
Here's what I did:
var query = from d in this.ObjectContext.Dealers
.Include("Groups")
.Include("Groups.Items")
.Include("Groups.Items.Observations")
.Include("Groups.Items.Recommendations")
.Include("Checkins.Inspections")
.Include("Checkins.Inspections.InspectionItems")
.Where(d => d.DealerId == id)
select new
{
Dealer = d,
Groups = from g in d.Groups
select new
{
Items = from i in g.Items
select new
{
Group = i.Group,
Observations = i.Observations,
Recommendations = i.Recommendations
}
},
Checkins = from c in d.Checkins
where c.Complete == true
select new
{
Inspections = from i in c.Inspections
select new
{
InspectionItems = i.InspectionItems
}
}
};
var dealer = query.ToArray().Select(o => o.Dealer).First();
return dealer;
It works.
However, I am not convinced I am doing the right thing.
What is the best way to accomplish what I did? A stored procedure maybe?
I am not sure I even have to use Include clause anymore
Thank you.
If you want to load filtered relations with single query you indeed have to execute such projection but you don't need those calls to Include. Once you are building projections includes are not use - you have returned data under your control.
Stored procedure will help you only if you fall back to plain ADO.NET because stored procedures executed through Entity framework are not able to fill related entities (only flattened structures).
Automatic fixupu mentioned by #Andreas requires multiple database queries and as I know it works only if lazy loading is disabled because proxied object somehow doesn't have information about fixup and it still has its internal flags for each relation as not loaded so when you access them for the first time they still execute additional query.
Maybe you can make use of the relation fixup mechanism in the EF ObjectContexts. When you do multiple queries in the same context for entities, that are related by associations, these are resolved.
Assuming your association between Dealers and Checkins is 1:n with navigation properties on each side, you could do like:
var dealer = yourContext.Dealers
.Where(p => p.DealerId == id)
.FirstOrDefault();
if(dealer != null)
{
yourContext.Checkins
.Where(c => c.Complete && c.DealerId == dealer.DealerId)
.ToList();
I have not tested this by now, but since EF recognises that the Checkins, it inserts into the context by the second query belong to the dealer from the first query, corresponding references are created.
#Andreas H:
Awesome, thank you a lot.
I had to adjust your suggestion like this and it worked:
var dealer = this.ObjectContext.Dealers
.Include("Groups")
.Include("Groups.Items")
.Include("Groups.Items.Observations")
.Include("Groups.Items.Recommendations")
.Where(p => p.DealerId == id).
FirstOrDefault();
if (dealer != null)
{
this.ObjectContext.Checkins
.Include("Inspections")
.Include("Inspections.InspectionItems")
.Where(c => !c.Complete && c.Dealer.DealerId == dealer.DealerId)
.ToList();
}
return dealer;
I still have to use the Include otherwise it won't return the referenced entities.
Note also that Dealer.Groups are unrelated to the Dealer.Checkins.
So if there's no checkins satisfying the condition, Groups still need to be returned.
It's interesting to note that at first, I put the two include for checkins to the dealer
var dealer = this.ObjectContext.Dealers
.Include("Groups")
.Include("Groups.Items")
.Include("Groups.Items.Observations")
.Include("Groups.Items.Recommendations")
.Include("Checkins.Inspections")
.Include("Checkins.Inspections.InspectionItems")
.Where(p => p.DealerId == id).
FirstOrDefault();
if (dealer != null)
{
this.ObjectContext.Checkins
.Where(c => c.Complete && c.DealerId == id)
.ToList();
}
return dealer;
but it returned all the Checkins including those which are not complete.
I don't understand exactly why the latter doesn't work but the former does, how are the entities are resolved. I somehow can intuit that the former returns all data.
Your accepted solution will generate multiple database queries. As Ladislav Mrnka said a projection is the only way to pull your result with one query. The maintance of your code indeed hard. Maybe you could use an IQueryable-Extension that builds the projection dynamically and keep your code clean:
var query = this.ObjectContext.Dealers.SelectIncluding( new List<Expression<Func<T,object>>>>(){
x => x.Groups,
x => x.Groups.Select(y => y.Items),
x => x.Groups.Select(y => y.Items.Select(z => z.Observations)),
x => x.Groups.Select(y => y.Items.Select(z => z.Recommendations)),
x => x.Checkins.Where(y => y.Complete==true),
x => x.Checkins.Select(y => y.Inspections),
x => x.Checkins.Select(y => y.Inspections.Select(z => z.InspectionItems))
});
var dealer = query.First();
return dealer;
You can find the extension at thiscode/DynamicSelectExtensions on github