I am working a asp .net core web API with EF core.
I wrote this query. But this take 20-30seconds to execute.
Anyone have idea to improve this query.
var hotels = await _context.Hotels
.Where(i => (i.DestinationCode == request.Destination))
.Select(i => new HotelListHotelVm
{
Item1 = i.Item1,
Item2 = i.Item2,
Item3 = i.Item3,
Item4Code = i.Item4Code,
Item4Description = i.Item4.TypeDescription,
Item5 = i.Item5.Select(x => new HotelListHotelVm.HotelListItem5Vm
{
Code = x.Item5Code,
Description = x.Item5.Description,
}).Where(x =>(incomingItem5s.Length > 0 ) ? (incomingItem5s.Contains(x.Code)) : (x.Code != "")),
Item6 = i.Item6.Select(x => new HotelListHotelVm.HotelListHotelItem6Vm
{
Id = x.Id,
Item6TypeCode = x.Item6TypeCode,
Order = x.Order,
Path = x.Path,
VisualOrder = x.VisualOrder,
}).Take(3),
HotelFacilities = i.Facilities.ToList().Distinct().Take(6).Select(x => new HotelListHotelVm.HotelListFacilityVm {
Id = x.Id,
FacilityGroupCode = x.FacilityGroupCode,
HotelFacilityGroupDescription = x.FacilityGroup.Description,
FacilityCode = x.FacilityCode
}),
})
.Where( i => ((incomingItem4.Length > 0 ) ? (incomingItem4.Contains(i.Item4Code)) : (i.Item4Code != "")) )
.OrderByDescending(i => i.Code)
.PaginatedListAsync(request.PageNumber, request.PageSize);
foreach( var item in hotels.Items){
foreach(var facility in item.HotelFacilities){
foreach( var fac in _context.Facilities){
if(facility.FacilityCode == fac.Code){
facility.HotelFacilityDescription = fac.Description;
}
}
}
}
I f I remove those foreach code, The query takes 8-10s to execute.
But I need those foreach codes. Because I need the HotelFacilityDescription
Any suggestion for optimize the query ?
Edit The i.Facilities - model
public class HotelFacility
{
// removed some
public int FacilityCode { get; set; }
public int FacilityGroupCode { get; set; }
public FacilityGroup FacilityGroup { get; set; }
public int HotelCode { get; set; }
public Hotel Hotel { get; set; }
}
}
_context.Facilities will be enumerated (i.e. database will be called) for every iteration of previous loops. The quick fix is to call it ones and store results in variable:
var facilities = _context.Facilities.ToList();
foreach( var item in hotels.Items){
foreach(var facility in item.HotelFacilities){
foreach(var fac in facilities){
if(facility.FacilityCode == fac.Code){
facility.HotelFacilityDescription = fac.Description;
}
}
}
}
Next improvement can be converting facilities into Dictionary for searching purposes.
Even better approach can be writing query joining with _context.Facilities on database side (but here more info needed).
I've read this a couple times, but it looks like the relationship for Hotel.Facilities is a Facility, so could you not just do:
HotelFacilities = i.Facilities.ToList().Distinct().Take(6).Select(x => new HotelListHotelVm.HotelListFacilityVm {
Id = x.Id,
FacilityGroupCode = x.FacilityGroupCode,
HotelFacilityGroupDescription = x.FacilityGroup.Description,
FacilityCode = x.FacilityCode,
HotelFacilityDescription = x.Description
}),
If for some reason Hotel.Facilities is not pointing at a Facility, but is a Many-to-Many HotelFacilityGroup entity to a FacilityGroup, that also contains a FacilityCode, if the associated FacilityGroup has access to a set of Facilities beneath it you could leverage that:
Edit: It sounds like multiple Facilities share the same Code where some may have a null description. Provided that the facilities matching the code would be within the same facility group and not consider the same Code within different facility groups. If you need to match the code across all facilities then there probably isn't much of an alternative to loading the entire set of facility codes & descriptions.
HotelFacilities = i.Facilities.ToList().Distinct().Take(6).Select(x => new HotelListHotelVm.HotelListFacilityVm {
Id = x.Id,
FacilityGroupCode = x.FacilityGroupCode,
HotelFacilityGroupDescription = x.FacilityGroup.Description,
FacilityCode = x.FacilityCode,
HotelFacilityDescription = x.FacilityGroup.Facilities.Where(f => f.Code == x.FacilityCode && f.Description != null).Select(f => f.Description).FirstOrDefault()
}),
That would avoid the need to go load all of the facilities to resolve that code. Otherwise, if you do need to fetch across all facilities, pre-loading them would be the way to go, but rather than fetching the entire Facility entity I would recommend just the values you need, the Code and the Description. This cuts down on the amount of memory needed and potentially be a faster query:
var facilities = _context.Facilities
.Select(f => new
{
f.Code,
f.Description
}).ToList();
Edit:
From there, finding a match using:
foreach( var facility in hotels.Items.SelectMany(x => x.HotelFacilities)
{
facility.HotelFaciltyDescription = facilities
.Where(x => x.Code == facility.FacilityCode
&& !string.IsNullOrEmpty(x.Description)
.Select(x => x.Description)
.FirstOrDefault();
}
I would recommend an OrderBy clause to ensure the selection of the facility is predictable as it sounds like there could be multiple matches on a code with a non-null description.
The loop can be eliminated by projecting the value in the LINQ to Entities query.
It would have been quite easy if you had relationship and navigation property like other *Code fields. But as clarified in the comments, there is no such relationship, so you have to resort to old good manual left other join to emulate what navigation property provide automatically, e.g.
HotelFacilities = i.Facilities.ToList().Distinct().Take(6)
// left outer join with Facilities
.SelectMany(x => _context.Facilities
.Where(f => x.FacilityCode == f.Code).DefaultIfEmpty(),
(x, x_Facility) => new HotelListHotelVm.HotelListFacilityVm
{
Id = x.Id,
FacilityGroupCode = x.FacilityGroupCode,
HotelFacilityGroupDescription = x.FacilityGroup.Description,
FacilityCode = x.FacilityCode,
HotelFacilityDescription = x_Facility.Description // <--
}),
Here x_Facility emulates optional reference navigation property x.Facility if existed.
In case you need just single property from the related table, instead of a left join you could also use the original query with single value returning correlated subquery inside the projection, e.g.
HotelFacilities = i.Facilities.ToList().Distinct().Take(6)
.Select(x => new HotelListHotelVm.HotelListFacilityVm
{
Id = x.Id,
FacilityGroupCode = x.FacilityGroupCode,
HotelFacilityGroupDescription = x.FacilityGroup.Description,
FacilityCode = x.FacilityCode,
HotelFacilityDescription = _context.Facilities
.Where(f => x.FacilityCode == f.Code)
.Select(f => f.Description)
.FirstOrDefault() // <--
}),
or even
HotelFacilityDescription = _context.Facilities
.FirstOrDefault(f => x.FacilityCode == f.Code).Description
All these will eliminate the need of the post loop executiong additional database queries. You can test them and take the one with best performance (#2 and #3 produce one and the same SQL, so it's a matter of taste - the choice is between #1 and #2/3).
Related
I have a Apps, AppRoles, UserAppRoles and Users. I'm trying to Get All Users but only want the AppRoles where AppId = 1. How do I filter the child collection?
using (var context = new dbContext())
{
var rv = context.Users
.Include(u => u.AppRoles);
}
I tried this but throws and exception:
The Include path expression must refer to a navigation property defined on the type. Use dotted paths for reference navigation properties and the Select operator for collection navigation properties
public static async Task<List<User>> GetAllAsync()
{
var rv = new List<User>();
using (var context = new dbContext())
{
rv = await (context.Users.AsNoTracking()
.Include(a => a.AppRoles.Where(a2 => a2.AppId == 1)).ToListAsync());
}
return rv;
}
The only way I could figure out how to get it to work is like this which I might as well just use a stored procedure at that point:
var rv = new List<User>();
using (var context = new dbContext())
{
rv = context.Users.AsNoTracking()
.Include(a => a.AppRoles).ToList();
}
foreach (var user in rv)
{
if (user.AppRoles.Any())
{
user.AppRoles = user.AppRoles.Where(r2 => r2.AppId == 1).ToList();
}
}
How do I write this in EF?
SELECT
Users.UserId,
Users.UserName
FROM
Users
INNER JOIN UserAppRoles ON Users.UserId = UserAppRoles.UserId
INNER JOIN AppRoles ON UserAppRoles.AppRoleId = AppRoles.AppRoleId
WHERE AppRoles.AppId = 1
Try this:
context.Entry(user)
.Collection(b => b.AppRoles)
.Query()
.Where(r => r.AppId==1)
.Load();
where user is an AppUser entity from the context (like in your foreach example).
More info here:
https://msdn.microsoft.com/en-us/library/jj574232(v=vs.113).aspx ("Applying filters when explicitly loading related entities" section)
You might subquery the AppRoles
var arQuery = from ar in context.AppRoles
where ar.AppId == 1
select ar;
var query = from u in context.Users
join uar in context.UserAppRoles on u.UserId equals uar.UserId
join ar in arQuery on uar.AppRoleId equals ar.AppRoleId
select u;
I have this working like this but still seems inefficient but guess it's better than round trips to database. Isn't this a common pattern? I can't believe a simple inner join isn't included in EF but I'm mapping to DTOs anyways so guess this will work for now until I look into a few open source Include filters that support filtering.
var rv = context.Users.Include(r => r.AppRoles).ToList().Select(u => new User()
{
UserId = u.UserId,
AppRoles = u.AppRoles.Where(x=>x.AppId == 1).ToList()
});
Disclaimer: I'm the owner of the project Entity Framework Plus
The EF+ Query IncludeFilter allow easily filter included entities.
public static async Task<List<User>> GetAllAsync()
{
var rv = new List<User>();
using (var context = new dbContext())
{
rv = await (context.Users.AsNoTracking()
.IncludeFilter(a => a.AppRoles.Where(a2 => a2.AppId == 1)).ToListAsync());
}
return rv;
}
Wiki: EF+ Query Include Filter
In SQL server union, result is sorted based on primary key column. I want to prevent this behavior in entity framework.
In this post, #praveen has explained how to do this in pure sql. But I want to do this in entity framework.
My code:
public virtual ActionResult Search(string keyword)
{
var products = _db.Products
.Where(x => x.IsActive)
.AsQueryable();
var productExactlyTitle = products.Where(x => x.Title == keyword);
var productStartTitle = products.Where(x => x.Title.StartsWith(keyword));
var productContainsTitle = products.Where(x => x.Title.Contains(keyword)
|| x.Title.Contains(keyword)
|| x.SubTitle.Contains(keyword)
|| x.OtherName.Contains(keyword));
var productList = productExactlyTitle.Union(productStartTitle)
.Union(productContainsTitle)
.Take(10)
.AsEnumerable()
.Select(x => new ProductItemViewModel()
{
Id = x.Id,
Title = x.Title,
Price = x.Price.ToPrice(),
Image = x.Images.FirstOrDefault(y => y.IsCoverPhoto)?.ImageUrl
});
// some code ...
}
I want to show records with below order:
First: records of productExactlyTitle
Second: records of productStartTitle
Third: records of productContainsTitle
But result is sorted with Id column! and I don't want this.
Is there a way for do this?
In SQL all queries without an order by explicitly set is considered unordered. (and EF queries a translated into SQL). So if you want a specific order after your union just specify it.
var result = q1.Union(q2).OrderBy(x => x.?);
For your specific case:
var p1 = productExactlyTitle.Select(x => new { Item = x, Order = 1 });
var p2 = productStartTitle.Select(x => new { Item = x, Order = 2 });
var p3 = productContainsTitle.Select(x => new { Item = x, Order = 3 });
var productList = p1.Union(p2)
.Union(p3)
.OrderBy(x => x.Order)
.Select(x => x.Item)
.Take(10);
1I am working with controller code in MVC.
I have the follow code working where the are related entities priority list based on teams:
public JsonResult GetPrioritiesByTeam(int id)
{
List<Priority> priorities = new List<Priority>();
if (id > 0)
{
priorities = db.Priorities.Where(p => p.Team == id ).ToList();
}
else
{
priorities.Insert(0, new Priority { Id = 0, PriorityDescription = "--Select a Team first--" });
}
var result = (from r in priorities
select new
{
id = r.Id,
name = r.PriorityDescription
}).ToList();
return Json(result, JsonRequestBehavior.AllowGet);
}
The statement
priorities = db.Priorities.Where(p => p.Team == id
does get me the list of priorities based on the team passed in by the parameter id.
I am now try to achieve the same successful result with another entity that is related to team- an entity called 'members'. However the relationship is not a one to many. Its a many to many- so I have an intermediate entity call 'teammembers'.
The Team is still the entity driving the choice. When the team is chosen, I need it to give me all the team member entries in teammembers, and then I need similar to the priorities code above, give me a members list which is based on the teammembers identified based on the team selected.
I think based on the code provided the else clause would be similar coding along with the var result coding.
I just cant seem to get the coding right for the clause in the condition for id >0.
How do I write that clause for the many to many so as to get the list of members for the team through teammembers?
You can assume entity team is Id and TeamDescription, entity Member is Id and MemberName, and entity teammember is an ID,and teamId and MemberId -just a straightforward many to many.
Been trying with LINQ method syntax and LINQ query syntax but cant get it working.
Thanks for any help you can provide
New code would look like the following except i need working id>0 clause:
public JsonResult GetMembersByTeam(int id)
{
List<Member> members = new List<Member>();
if (id > 0)
{
**members = Members.Where(needed clause).ToList();**
}
else
{
members.Insert(0, new Member { Id = 0, MemberFullName = "--Select a Team first--" });
}
var result = (from m in members
select new
{
id = m.Id,
name = m.MemberFullName
}).ToList();
return Json(result, JsonRequestBehavior.AllowGet);
}
Entity Diagram
Current code with suggested changes
public JsonResult GetMembersByTeam(int id)
{
List<Member> members = new List<Member>();
if (id > 0)
{
//members = db.Teams.Where(p => p.Id == id)
// .SelectMany(e => e.TeamMembers)
// .Select(e => e.Member)
// .ToList();
members = db.TeamMembers.Where(p => p.Team == id)
.Select(e => e.Member)
.ToList();
}
else
{
members.Insert(0, new Member { Id = 0, MemberFullName = "--Select a Team first--" });
}
var result = (from m in members
select new
{
id = m.Id,
name = m.MemberFullName
}).ToList();
return Json(result, JsonRequestBehavior.AllowGet);
}
results in error :
Cannot implicitly convert type 'System.Collections.Generic.List' to 'System.Collections.Generic.List,ManageHR5.model.Member>'
try this:
members = db.TeamMembers.Where(p => p.Team == id )
.Select(e => e.Member1)
.ToList();
I have the following line in a WebApi controller;
string Codes = i.Products.FirstOrDefault().Code
As the line states, it gets the code, from the first Product.
But, what I really want it to do is to get all unique codes, and return them as a comma separated string.
So, let's say, there are 6 related products, and they have the following codes:
45
54
45
120
54
45
Right now, the statement just returns "45", given the above data.
But I want the statement above to return "45, 54, 120" (as a string).
How do I do this?
Complete code:
public System.Data.Entity.DbSet<WebAPI.Models.Product> Products { get; set; }
private ApplicationDbContext db = new ApplicationDbContext();
var product = await db.Products.Select(i =>
new ProductDTO()
{
Id = i.Id,
Created = i.Created,
Title = i.Title,
Codes = i.Products.FirstOrDefault().Code
}).SingleOrDefaultAsync(i => i.Id == id);
To convert to a comma separated list:
IEnumerable<string> distinctCodes = i.Products.Select(product => product.Code).Distinct();
return string.Join(",", distinctCodes);
But it's probably better if your controller returns a collection of string instead of a concatenated string.
Edit, after OP code update:
var DBProduct = await db.Products.SingleOrDefaultAsync(i => i.Id == id);
IEnumerable<string> productCodes = DBProduct.Products.Select(p => p.Code).Distinct();
var product = new ProductDTO()
{
Id = DBProduct.Id,
Created = DBProduct.Created,
Title = DBProduct.Title,
Codes = string.Join(",", productCodes)
};
I think Distinctand String.Join works for you.Please try this:
var product = await db.Products.AsEnumerable()//Turn AsEnumarable
.Select(i =>
new ProductDTO()
{
Id = i.Id,
Created = i.Created,
Title = i.Title,
Codes = string.Join(",",
i.Products.Select(l => l.Code).Distinct())
}).SingleOrDefaultAsync(i => i.Id == id);
Similar to the simple Membership UserProfiles to Roles in the Table UserInRoles
I have created a Relationship between UserProfiles to Clients in the table UserInClients with this code
modelBuilder.Entity<UserProfiles>()
.HasMany<dbClient>(r => r.Clients)
.WithMany(u => u.UserProfiles)
.Map(m =>
{
m.ToTable("webpages_UsersInClients");
m.MapLeftKey("ClientId");
m.MapRightKey("UserId");
});
My UserProfiles has an public virtual ICollection<dbClient> Clients { get; set; } and my Clients has an public virtual ICollection<UserProfiles> UserProfiles { get; set; }
If you need to see the Models let me know i can post them
In the model and view i would like to
Display all Clients ( Distinct ) and show how many users have access to that client
Create a View that only shows clients a Currently logged in user is allowed to view.
I thought it was as easy as accessing the Properties of my models Clients.ClientID, i have been trying things like Clients.ClientID.select(u=>u.UserId == clientid) but i knew better and know it does not and will not work.
My other thoughts were creating a model with CliendID and UserID in it ( like the table it created ) so i can use a join in my controller to find the right values??
In the end what i'm trying to accomlish is to populate a KendoUI CascadingDropDownList with this line in my GetCascadeClients JsonResult
return Json(db.Clients.Select(c => new { ClientID = c.ClientID, ClientName = c.Client }), JsonRequestBehavior.AllowGet);
My question is, when I'm in my Controller, how do I access this table built by Entity Framework?
EDIT:
SOLUTION QUERY Pieced together by both answers
return Json(db.Clients
.Include(c => c.UserProfiles)
.Where(c => c.UserProfiles.`Any(up => up.UserName == User.Identity.Name))`
.Select(c => new
{
ClientID = c.ClientID,
ClientName = c.Client,
UserCount = c.UserProfiles.Count()
}),
JsonRequestBehavior.AllowGet);
try something like:
return JSON (db.Clients
.Include(c => c.UserProfiles)
.Where(c => c.UserProfiles.UserId == loggedInUserId)
.Select( c => new {
ClientId = c.ClientID,
ClientName = c.Client,
UserCount = c.UserProfiles.Count()}),
JsonRequestBehavior.AllowGet);
the .Include() extension will make sure that you pull all the UserProfiles along with the Clients, allowing you to use that table for filtering, record counts, etc... that .Where clause might need some work actually, but this should be a solid start.
This is more of a LINQ question really:
db.Clients
.Where(c => c.UserProfiles.Any(up => up.UserId == loggedInUserId))
.Select(c => new {
ClientId = c.ClientID,
ClientName = c.Client + " (" + c.UserProfiles.Count() + ")"
})
Due to the fact that it will convert the above into SQL calls, I had to use string concatenation, as if you try to use a nice String.Format("{0} ({1})", c.Client, c.UserProfiles.Count()) it will complain about being unable to translate that to SQL.
The other options is to do a 2-pass query, materializing the data before doing extra formatting:
db.Clients
.Where(c => c.UserProfiles.Any(up => up.UserId == loggedInUserId))
.Select(c => new {
ClientId = c.ClientID,
ClientName = c.Client,
ProfileCount = c.UserProfiles.Count()
})
// this forces SQL to execute
.ToList()
// now we're working on an in-memory list
.Select(anon => new {
anon.ClientId,
ClientName = String.Format("{0} ({1})", anon.ClientName, anon.ProfileCount)
})