Entity Framework, building query based on criteria - entity-framework

I was wondering if anyone had a better idea how to do this. atm returning IQueryable<Member> as ObjectQuery<Member> seems dirty to me.
namespace Falcon.Business.Repositories
{
using System;
using System.Data.Objects;
using System.Linq;
using Falcon.Business.Criteria;
using Falcon.Business.Entities;
using Falcon.Business.Enums;
using Falcon.Business.Extensions;
using Falcon.Business.Repositories.Interfaces;
using Falcon.Business.Services;
using Falcon.Business.Services.Interfaces;
using Falcon.Core.Extensions;
public class MemberRepository : LinqRepository<Member>, IMemberRepository
{
public Member Fetch(MemberCriteria criteria)
{
ObjectQuery<Member> query = base.CreateQuery();
query = this.AddRelations(query);
query = this.AddCriteria(query, criteria);
query = this.AddCriteriaOrder(query, criteria);
return query.FirstOrDefault();
}
public IPagerService<Member> FetchAll(MemberCriteria criteria)
{
int page = (criteria.Page.HasValue) ? criteria.Page.Value : 1;
int limit = criteria.Limit;
int start = (page * limit) - limit;
int total = this.Count(criteria);
ObjectQuery<Member> query = base.CreateQuery();
query = this.AddRelations(query);
query = this.AddCriteria(query, criteria);
query = this.AddCriteriaOrder(query, criteria);
return new PagerService<Member>(query.Skip(start).Take(limit).ToList(), page, limit, total);
}
public int Count(MemberCriteria criteria)
{
ObjectQuery<Member> query = base.CreateQuery();
query = this.AddCriteria(query, criteria);
return query.Count();
}
public ObjectQuery<Member> AddCriteria(IQueryable<Member> query, MemberCriteria criteria)
{
if (criteria.Title.HasValue())
{
query = query.Where(q => q.Title == criteria.Title);
}
if (criteria.TitleUrl.HasValue())
{
query = query.Where(q => q.TitleUrl == criteria.TitleUrl);
}
if (criteria.EmailAddress.HasValue())
{
query = query.Where(q => q.EmailAddress == criteria.EmailAddress);
}
if (criteria.HostAddress.HasValue())
{
query = query.Where(q => q.HostAddress == criteria.HostAddress);
}
query = query.Where(q => q.Status == criteria.Status);
return query as ObjectQuery<Member>;
}
public ObjectQuery<Member> AddCriteriaOrder(IQueryable<Member> query, MemberCriteria criteria)
{
if (criteria.Sort == SortMember.ID)
{
query = criteria.Order == SortOrder.Asc
? query.OrderBy(q => q.ID)
: query.OrderByDescending(q => q.ID);
}
else if (criteria.Sort == SortMember.Posts)
{
query = criteria.Order == SortOrder.Asc
? query.OrderBy(q => q.Posts)
: query.OrderByDescending(q => q.Posts);
}
else if (criteria.Sort == SortMember.Title)
{
query = criteria.Order == SortOrder.Asc
? query.OrderBy(q => q.Title)
: query.OrderByDescending(q => q.Title);
}
else if (criteria.Sort == SortMember.LastLogin)
{
query = criteria.Order == SortOrder.Asc
? query.OrderBy(q => q.LastLogin)
: query.OrderByDescending(q => q.LastLogin);
}
else if (criteria.Sort == SortMember.LastVisit)
{
query = criteria.Order == SortOrder.Asc
? query.OrderBy(q => q.LastVisit)
: query.OrderByDescending(q => q.LastVisit);
}
else
{
query = criteria.Order == SortOrder.Asc
? query.OrderBy(q => q.Created)
: query.OrderByDescending(q => q.Created);
}
return query as ObjectQuery<Member>;
}
private ObjectQuery<Member> AddRelations(ObjectQuery<Member> query)
{
query = query.Include(x => x.Country);
query = query.Include(x => x.TimeZone);
query = query.Include(x => x.Profile);
return query;
}
}
}

I also do not like returning an objectquery, because doing so will make you very dependent on Entity Framwork. Knowing Microsoft they propably make a lot of changes in version 2, so you do not want to do this.
NHibernate uses criteria, a bit like you suggested, but their implementation is a lot more generic. I like the more generic implementation more then you example because then you do not need to build criteria for every object. On the other hand, you implementation is typed, which is also very neat. If you want the best of both, a more generic implementation that is typed, you might want to take a look at the NHibernate implementation but instead of using strings, use lambda functions and .Net generics. I could post an example how to do this, but I'm currently not on my own machine.

Related

EF send Includes list to repository via parametr

I at this moment I have repository filled with multiple gets methods.
E.q. Get1() => cxt.Entites.Include(e => e.obj1);
Get2() => cxt.Entities.Include(e => e.obj1).Include(e => e.obj2)
And so on.
Is there good method, pattern to have one GET method where I can send inclues via parameter?
public virtual IEnumerable<TEntity> Get(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
string includeProperties = "")
{
IQueryable<TEntity> query = dbSet;
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
See repository pattern in msdn
You can use
_sampleRepostiory.Get(h=>h.Id>1,null,"Employees.Departments");
Including same with lambda
public virtual IEnumerable<TEntity> Get(
Expression<Func<TEntity, bool>> filter = null,
Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null,
Expression<Func<TEntity, object>>[] includes)
{
IQueryable<TEntity> query = dbSet;
if (filter != null)
{
query = query.Where(filter);
}
if (includes != null)
{
query = includes.Aggregate(query,
(current, include) => current.Include(include));
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
Consume it like this
var query = context.Customers
.Get(x=>x.Id>1,null,
c => c.Address,
c => c.Orders.Select(o => o.OrderItems));
Similar SO question
I did the following in my projects:
public Entity[] GetAll(bool includeObj1, bool includeAllOthers) {
IQueryable<Entity> entity = ctx.Entities;
if (includeObj1)
entity = entity.Include(e => e.obj1);
if (includeAllOthers) {
entity = entity
.Include(e => e.obj2)
.Include(e => e.obj3)
.Include(e => e.obj4)
.Include(e => e.obj5);
}
return entity.ToArray();
}
Providing arguments like includeObj1 and includeObj2 separates a consumer of repository from implementation and encapsulates any data access logic.
Passing direct "include these properties" orders to a repository means that you know how repository works and assume that it is some sort ORM which blurs abstractions.

Can I convert this extension method to use IDbSet<> instead of DbContext?

Currently I use the below method like this: db.GetProjectsAllowed(profileId, profOrgList, projectList). I would like to convert this to use IDbSet<Project>, but I'm not sure how to get the second LINQ query.
public static IQueryable<Project> GetProjectsAllowed
(
this IMkpContext db,
Guid profileId,
List<Guid> profOrgIds = null,
List<Guid> projectIds = null
)
{
var projects =
(
from p in db.Project
.Include(p => p.Proposals)
.Include(p => p.RoleAssignments)
.Include("RoleAssignments.AssigneeSnapshot")
where p.IsActive
select p);
if (profOrgIds != null && profOrgIds.Any())
{
var profileIds = db.ProfileOrganization
.Where(po => po.IsActive && profOrgIds.Contains(po.OrganizationId))
.Select(po => po.ProfileId);
projects = projects.Where(p => profileIds.Contains(p.CreatedById));
}
if (projectIds != null && projectIds.Any())
projects = projects.Where(proj => projectIds.Contains(proj.ProjectId));
return projects;//.ToList();
}
Can I convert this to use IDbSet<Project> or not?
Here, why not split this into two extension methods? This makes your GetProjectsAllowed extension method more cohesive and single responsible.
First:
public static IEnumerable<Guid> GetProfileIds(
this IDbSet<ProfileOrganization> profileOrganizations,
IEnumerable<Guid> profOrgIds = null)
{
return profOrgIds == null ? null :
from po in profileOrganizations
where po.IsActive
where profOrgIds.Contains(po.OrganizationId)
select po.OrganizationId;
}
And second:
public static IQueryable<Project> GetProjectsAllowed(
this IDbSet<Project> projects,
IEnumerable<Guid> profileIds,
IEnumerable<Guid> projectIds = null)
{
var activeProjects =
from project in projects
//.Include(..
where project.IsActive
select project;
if (profileIds != null && profileIds.Any())
{
activeProjects = activeProjects.Where(p => profileIds.Contains(p.CreatedById));
}
if (projectIds != null && projectIds.Any())
{
activeProjects = activeProjects.Where(proj => projectIds.Contains(proj.ProjectId));
}
return activeProjects;//.ToList();
}
And then the consumer can call it like this:
var profileIds = db.ProfileOrganization.GetProfileIds(profOrgIds);
var projectsAllowed = db.Projects.GetProjectsAllowed(profileIds, projectIds);

Better way to fecth data for EF

We have DB on which CDC is enabled. We had to find the data on basis of date supplied. So we had implemented the data fetch through db functions.
The code block is as follows:
private IQueryable<Letter> GetLetterQuery(DateTime? date = null)
{
IQueryable<Letter> dataSource = null;
dataSource = DBContextHelper.Context.fnGetLetters(date);
return dataSource;
}
public List<Letter> GetLetters(int attemptID, int cycle)
{
var dataSource = GetLetterQuery(DBContextHelper.CurrentDataVersion).Where(u => u.AttemptID == attemptID && u.Cycle == cycle).ToList();
populateLettersChildEntities(dataSource, DBContextHelper.CurrentDataVersion);
return dataSource;
}
private void populateLettersChildEntities(List<Letter> letters, DateTime? date = null)
{
letters.ForEach(u =>
{
u.Documents = DBContextHelper.Context.fnGetDocuments(date).Where(d => d.LetterID == u.ID).ToList();
u.LetterDoctors = DBContextHelper.Context.fnGetLetterDoctor(date).Where(d => d.LetterID == u.ID).ToList();
u.LetterFields = DBContextHelper.Context.fnGetLetterFields(date).Where(d => d.LetterID == u.ID).ToList();
u.LetterDoctors.ForEach(d =>
{
d.Doctor = DBContextHelper.Context.fnGetDoctors(date).FirstOrDefault(q => q.IDdoctor == d.DoctorID);
d.Letter = u;
});
});
}
We had to fill the associated fields. Our problem is that it results in many db calls definitely. Is there any way to reduce the calls? AFAIK we are not able to use the include with such db calls.

Some part of your SQL statement is nested too deeply

I have the following code
[WebGet]
public Bid GetHighestBidInOpenAuctions(int auctionEventId)
{
var auctionEvent = CurrentDataSource.AuctionEvents.Where(x => x.Id == auctionEventId).FirstOrDefault();
var auctionIds = CurrentDataSource.Auctions.Where(x => x.AuctionEventId == auctionEventId && x.Ends > DateTime.UtcNow).Select(x => x.Id).ToList();
var bids = CurrentDataSource.Bids.Where(x => auctionIds.Any(t => t == x.AuctionId));
// If the auction Event has not yet started or there are no bids then show auction with high pre-sale estimate.
if (bids.Count() == 0 || auctionEvent.Starts > DateTime.UtcNow)
{
return null;
}
var highestBid = bids.Where(b => b.IsAutobid == false).OrderByDescending(b => b.Amount).FirstOrDefault();
return highestBid;
}
This line throws the below exception
if (bids.Count() == 0 || auctionEvent.Starts > DateTime.UtcNow)
Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
What's wrong?
EDIT
I have tried doing this
IQueryable<Bid> bids = CurrentDataSource.Bids.Where(b => 0 == 1);
foreach(var auctionId in auctionIds)
{
int id = auctionId;
bids = bids.Union(CurrentDataSource.Bids.Where(b => b.AuctionId == id));
}
But I still get the same error.
Rather than using a subquery, try replacing the bid query with:
var bids = CurrentDataSource.Bids.Where(b => b.AuctionEventId == auctionEventId
&& b.Auction.AuctionEvent.Starts > DateTime.UtcNow
&& b.Auction.Ends > DateTime.UtcNow);
if (bids.Count() == 0
{
return null;
}
It seems when you have too many things in your database then you will get this error (auctionIds in my case) because the generated sql will be too deeply nested. To solve this I came up with this solution. If anyone can do better then do. I'm posting this because someone may have this error in the future and if they do, in the absence of a better solution this might help them.
[WebGet]
public Bid GetHighestBidInOpenAuctions(int auctionEventId)
{
/*
* This method contains a hack that was put in under tight time constraints. The query to
* get bids for all open auctions used to fail when we had a large number of open auctions.
* In this implementation we have fixed this by splitting the open auctions into groups of 20
* and running the query on those 20 auctions and then combining the results.
*/
const int auctionIdSegmentSize = 20;
var auctionEvent = CurrentDataSource.AuctionEvents.Where(x => x.Id == auctionEventId).FirstOrDefault();
var auctionIds = CurrentDataSource.Auctions.Where(x => x.AuctionEventId == auctionEventId && x.Ends > DateTime.UtcNow).Select(x => x.Id).ToList();
int numberOfSegments = auctionIds.Count/auctionIdSegmentSize;
if (auctionIds.Count % auctionIdSegmentSize != 0)
numberOfSegments++;
var bidsList = new List<IQueryable<Bid>>();
for (int i = 0; i < numberOfSegments; i++)
{
int start = i*auctionIdSegmentSize;
int end;
if (i == numberOfSegments - 1)
{
end = auctionIds.Count - 1;
}
else
{
end = ((i + 1)*auctionIdSegmentSize) - 1;
}
var subList = auctionIds.GetRange(start, (end - start) + 1);
bidsList.Add(CurrentDataSource.Bids.Where(b => subList.Any(id => id == b.AuctionId)));
}
// If the auction Event has not yet started or there are no bids then show auction with high pre-sale estimate.
if (IsBidsCountZero(bidsList) || auctionEvent.Starts > DateTime.UtcNow)
{
return null;
}
var highestBid = FindHighestBid(bidsList);
return highestBid;
}
private Bid FindHighestBid(List<IQueryable<Bid>> bidsList)
{
var bids = new List<Bid>();
foreach (var list in bidsList)
{
bids.Add(list.Where(b => b.IsAutobid == false).OrderByDescending(b => b.Amount).FirstOrDefault());
}
bids.RemoveAll(b => b == null);
if (bids.Count == 0)
return null;
bids.Sort(BidComparison);
return bids[0];
}
private int BidComparison(Bid bid1, Bid bid2)
{
if (bid1.Amount < bid2.Amount)
return 1;
if (bid1.Amount > bid2.Amount)
return -1;
return 0;
}
private bool IsBidsCountZero(List<IQueryable<Bid>> bidsList)
{
int count = 0;
foreach (var list in bidsList)
{
count += list.Count();
}
return count == 0;
}
The problem is with auctionIds.Any(t => t == x.AuctionId) where EF cannot create a correct query. You can change it to:
var bids = CurrentDataSource.Bids.Where(x => auctionIds.Contains(x.AuctionId));
Where EF can convert auctionIds to a collection and pass to DB.

conditional where in linq query

I want to return the total sum from a linq query, I pass in 2 parameters that may/may not be included in the query.
OrgId - int
reportType - int
So two questions:
How can I update the query below so that if OrgId = 0 then ignore the organisation field(Return All)?
LocumClaimTF can be True/False/Both, if both then ignore the where query for this field.
Heres what I have done so far, this is working but I'd like something for efficient.
// Using reportType set preferences for LocumClaimTF
bool locumClaimTF1, locumClaimTF2 = false;
if (reportType == 0)
{
locumClaimTF1 = false;
locumClaimTF2 = false;
}
else if (reportType == 1)
{
locumClaimTF1 = true;
locumClaimTF2 = true;
}
else // 2
{
locumClaimTF1 = true;
locumClaimTF2 = false;
}
if (OrgID != 0) // Get by OrgID
{
return _UoW.ShiftDates.Get(x => x.shiftStartDate >= StartDate && x.shiftEndDate <= EndDate)
.Where(x => x.Shift.LocumClaimTF == locumClaimTF1 || x.Shift.LocumClaimTF == locumClaimTF2)
.Where(x => x.Shift.organisationID == OrgID)
.GroupBy(s => s.assignedLocumID)
.Select(g => new dataRowDTO { dataLabel = string.Concat(g.FirstOrDefault().User.FullName), dataCount = g.Count(), dataCurrencyAmount = g.Sum(sd => sd.shiftDateTotal.Value) }
).Sum(g=>g.dataCurrencyAmount);
}
else // Ignore OrgID - Get ALL Orgs
{
return _UoW.ShiftDates.Get(x => x.shiftStartDate >= StartDate && x.shiftEndDate <= EndDate)
.Where(x => x.Shift.LocumClaimTF == locumClaimTF1 || x.Shift.LocumClaimTF == locumClaimTF2)
.GroupBy(s => s.assignedLocumID)
.Select(g => new dataRowDTO { dataLabel = string.Concat(g.FirstOrDefault().User.FullName), dataCount = g.Count(), dataCurrencyAmount = g.Sum(sd => sd.shiftDateTotal.Value) }
).Sum(g => g.dataCurrencyAmount);
}
I'm using EF with unit of work pattern to get data frm
A few things come to mind, from top to bottom:
For handling the Booleans
bool locumClaimTF1 = (reportType == 1 || reportType == 2);
bool locumClaimTF2 = (reportType == 1);
From what I read in the query though, if the report type is 1 or 2, you want the Shift's LocumClaimTF flag to have to be True. If that is the case, then you can forget the Boolean flags and just use the reportType in your condition.
Next, for composing the query, you can conditionally compose where clauses. This is a nice thing about the fluent Linq syntax. However, let's start temporarily with a regular DbContext rather than the UoW because that will introduce some complexities and questions that you'll need to look over. (I will cover that below)
using (var context = new ApplicationDbContext()) // <- insert your DbContext here...
{
var query = context.ShiftDates
.Where(x => x.shiftStartDate >= StartDate
&& x.shiftEndDate <= EndDate);
if (reportType == 1 || reportType == 2)
query = query.Where(x.Shift.LocumClaimTF);
if (OrgId > 0)
query = query.Where(x => x.Shift.organisationID == OrgID);
var total = query.GroupBy(s => s.assignedLocumID)
.Select(g => new dataRowDTO
{
dataLabel = tring.Concat(g.FirstOrDefault().User.FullName),
dataCount = g.Count(),
dataCurrencyAmount = g.Sum(sd => sd.shiftDateTotal.Value)
})
.Sum(g=>g.dataCurrencyAmount);
}
Now this here didn't make any sense. Why are you going through the trouble of grouping, counting, and summing data, just to sum the resulting sums? I suspect you've copied an existing query that was selecting a DTO for the grouped results. If you don't need the grouped results, you just want the total. So in that case, do away with the grouping and just take the sum of all applicable records:
var total = query.Sum(x => x.shiftDateTotal.Value);
So the whole thing would look something like:
using (var context = new ApplicationDbContext()) // <- insert your DbContext here...
{
var query = context.ShiftDates
.Where(x => x.shiftStartDate >= StartDate
&& x.shiftEndDate <= EndDate);
if (reportType == 1 || reportType == 2)
query = query.Where(x.Shift.LocumClaimTF);
if (OrgId > 0)
query = query.Where(x => x.Shift.organisationID == OrgID);
var total = query.Sum(x => x.shiftDateTotal.Value);
return total;
}
Back to the Unit of Work: The main consideration when using this pattern is ensuring that this Get call absolutely must return back an IQueryable<TEntity>. If it returns anything else, such as IEnumerable<TEntity> then you are going to be facing significant performance problems as it will be returning materialized lists of entities loaded to memory rather than something that you can extend to build efficient queries to the database. If the Get method does not return IQueryable, or contains methods such as ToList anywhere within it followed by AsQueryable(), then have a talk with the rest of the dev team because you're literally standing on the code/EF equivalent of a land mine. If it does return IQueryable<TEntity> (IQueryable<ShiftDate> in this case) then you can substitute it back into the above query:
var query = _UoW.ShiftDates.Get(x => x.shiftStartDate >= StartDate && x.shiftEndDate <= EndDate);
if (reportType == 1 || reportType == 2)
query = query.Where(x.Shift.LocumClaimTF);
if (OrgId > 0)
query = query.Where(x => x.Shift.organisationID == OrgID);
var total = query.Sum(x => x.shiftDateTotal.Value);
return total;