conditional where in linq query - entity-framework

I want to return the total sum from a linq query, I pass in 2 parameters that may/may not be included in the query.
OrgId - int
reportType - int
So two questions:
How can I update the query below so that if OrgId = 0 then ignore the organisation field(Return All)?
LocumClaimTF can be True/False/Both, if both then ignore the where query for this field.
Heres what I have done so far, this is working but I'd like something for efficient.
// Using reportType set preferences for LocumClaimTF
bool locumClaimTF1, locumClaimTF2 = false;
if (reportType == 0)
{
locumClaimTF1 = false;
locumClaimTF2 = false;
}
else if (reportType == 1)
{
locumClaimTF1 = true;
locumClaimTF2 = true;
}
else // 2
{
locumClaimTF1 = true;
locumClaimTF2 = false;
}
if (OrgID != 0) // Get by OrgID
{
return _UoW.ShiftDates.Get(x => x.shiftStartDate >= StartDate && x.shiftEndDate <= EndDate)
.Where(x => x.Shift.LocumClaimTF == locumClaimTF1 || x.Shift.LocumClaimTF == locumClaimTF2)
.Where(x => x.Shift.organisationID == OrgID)
.GroupBy(s => s.assignedLocumID)
.Select(g => new dataRowDTO { dataLabel = string.Concat(g.FirstOrDefault().User.FullName), dataCount = g.Count(), dataCurrencyAmount = g.Sum(sd => sd.shiftDateTotal.Value) }
).Sum(g=>g.dataCurrencyAmount);
}
else // Ignore OrgID - Get ALL Orgs
{
return _UoW.ShiftDates.Get(x => x.shiftStartDate >= StartDate && x.shiftEndDate <= EndDate)
.Where(x => x.Shift.LocumClaimTF == locumClaimTF1 || x.Shift.LocumClaimTF == locumClaimTF2)
.GroupBy(s => s.assignedLocumID)
.Select(g => new dataRowDTO { dataLabel = string.Concat(g.FirstOrDefault().User.FullName), dataCount = g.Count(), dataCurrencyAmount = g.Sum(sd => sd.shiftDateTotal.Value) }
).Sum(g => g.dataCurrencyAmount);
}
I'm using EF with unit of work pattern to get data frm

A few things come to mind, from top to bottom:
For handling the Booleans
bool locumClaimTF1 = (reportType == 1 || reportType == 2);
bool locumClaimTF2 = (reportType == 1);
From what I read in the query though, if the report type is 1 or 2, you want the Shift's LocumClaimTF flag to have to be True. If that is the case, then you can forget the Boolean flags and just use the reportType in your condition.
Next, for composing the query, you can conditionally compose where clauses. This is a nice thing about the fluent Linq syntax. However, let's start temporarily with a regular DbContext rather than the UoW because that will introduce some complexities and questions that you'll need to look over. (I will cover that below)
using (var context = new ApplicationDbContext()) // <- insert your DbContext here...
{
var query = context.ShiftDates
.Where(x => x.shiftStartDate >= StartDate
&& x.shiftEndDate <= EndDate);
if (reportType == 1 || reportType == 2)
query = query.Where(x.Shift.LocumClaimTF);
if (OrgId > 0)
query = query.Where(x => x.Shift.organisationID == OrgID);
var total = query.GroupBy(s => s.assignedLocumID)
.Select(g => new dataRowDTO
{
dataLabel = tring.Concat(g.FirstOrDefault().User.FullName),
dataCount = g.Count(),
dataCurrencyAmount = g.Sum(sd => sd.shiftDateTotal.Value)
})
.Sum(g=>g.dataCurrencyAmount);
}
Now this here didn't make any sense. Why are you going through the trouble of grouping, counting, and summing data, just to sum the resulting sums? I suspect you've copied an existing query that was selecting a DTO for the grouped results. If you don't need the grouped results, you just want the total. So in that case, do away with the grouping and just take the sum of all applicable records:
var total = query.Sum(x => x.shiftDateTotal.Value);
So the whole thing would look something like:
using (var context = new ApplicationDbContext()) // <- insert your DbContext here...
{
var query = context.ShiftDates
.Where(x => x.shiftStartDate >= StartDate
&& x.shiftEndDate <= EndDate);
if (reportType == 1 || reportType == 2)
query = query.Where(x.Shift.LocumClaimTF);
if (OrgId > 0)
query = query.Where(x => x.Shift.organisationID == OrgID);
var total = query.Sum(x => x.shiftDateTotal.Value);
return total;
}
Back to the Unit of Work: The main consideration when using this pattern is ensuring that this Get call absolutely must return back an IQueryable<TEntity>. If it returns anything else, such as IEnumerable<TEntity> then you are going to be facing significant performance problems as it will be returning materialized lists of entities loaded to memory rather than something that you can extend to build efficient queries to the database. If the Get method does not return IQueryable, or contains methods such as ToList anywhere within it followed by AsQueryable(), then have a talk with the rest of the dev team because you're literally standing on the code/EF equivalent of a land mine. If it does return IQueryable<TEntity> (IQueryable<ShiftDate> in this case) then you can substitute it back into the above query:
var query = _UoW.ShiftDates.Get(x => x.shiftStartDate >= StartDate && x.shiftEndDate <= EndDate);
if (reportType == 1 || reportType == 2)
query = query.Where(x.Shift.LocumClaimTF);
if (OrgId > 0)
query = query.Where(x => x.Shift.organisationID == OrgID);
var total = query.Sum(x => x.shiftDateTotal.Value);
return total;

Related

Where condition in Entity Framework dose not work correctly

I have Article and ArticleTranslation tables.
When I use this query, it does not retrieve data:
model.CategoryList = await db.ArticleCategoryTranslations
.Where(x => x.LangId == lang.Id &&
x.ArticleCategory.IsActive.Value == true &&
x.ArticleCategory.IsDelete.Value == false)
.Select(x => new ddl { Id = x.RecordId.Value, Name = x.Title }).ToListAsync();
but when I use this it works, Tell me what problem in the first code
model.CategoryList = await db.ArticleCategoryTranslations
.Where(x => x.LangId == lang.Id &&
db.ArticleCategorys.Where(u => u.Id == x.RecordId).FirstOrDefault().IsActive == true &&
db.ArticleCategorys.Where(u => u.Id == x.RecordId).FirstOrDefault().IsDelete == false
)
.Select(x => new ddl { Id = x.RecordId.Value, Name = x.Title }).ToListAsync();
I don't know what the properties on your model objects look like but comparing both code snippets I see in the successful one the IsActive and IsDelete properties don't call the Value property i.e IsActive.Value and IsDelete.Value.
So your code should look like
model.CategoryList = await db.ArticleCategoryTranslations
.Where(x => x.LangId == lang.Id &&
x.ArticleCategory.IsActive == true &&
x.ArticleCategory.IsDelete == false)
.Select(x => new ddl { Id = x.RecordId.Value, Name = x.Title }).ToListAsync();
This assumes you have your navigation properties done right.

Property or indexer 'AnonymousType#1.FilePath' cannot be assigned to

Basically I retrieved the records from the table and wanted to updated one column.
var query = cdrContext.tabless.Where(c => c.FacilityID == facilityID && c.FilePath != null && c.TimeStationOffHook < oldDate)
.OrderBy(c => c.TimeStationOffHook)
.Skip(size)
.Take(pageSize)
.Select(c => new { c.FilePath, c.FileName })
.ToList();
So this query has only two fields: FilePath and FileName, then next I want to assign FilePath = null;
foreach (var y in query)
{
y.FilePath = null;
}
cdrContext.SaveChanges();
Then I got an error:
Property or indexer 'AnonymousType#1.FilePath' cannot be assigned to -- it is read only
You select from query anonymous class. You cannot set properties of anonymous class. To do what you want you should get whole entity:
var query = cdrContext.tabless.Where(c => c.FacilityID == facilityID && c.FilePath != null && c.TimeStationOffHook < oldDate)
.OrderBy(c => c.TimeStationOffHook)
.Skip(size)
.Take(pageSize)
.ToList();
foreach (var y in query)
{
y.FilePath = null;
}
cdrContext.SaveChanges();

best practice on conditional queries with linq to entities

I often find myself writing querys like this:
var voyages = db.VoyageRequests.Include("Carrier")
.Where(u => (fromDate.HasValue ? u.date >= fromDate.Value : true) &&
(toDate.HasValue ? u.date <= toDate.Value : true) &&
u.Carrier != null &&
u.status == (int)VoyageStatus.State.InProgress)
.OrderBy(u => u.date);
return voyages;
With conditionals inside the where statement:
fromDate.HasValue ? u.date >= fromDate.Value : true
I know the other way to do it'll be like:
var voyages = db.VoyageRequests.Include("Carrier").Where(u => u.Carrier != null &&
u.status == (int)VoyageStatus.State.InProgress);
if (fromDate.HasValue)
{
voyages = voyages.Where(u => u.date >= fromDate.Value);
}
if (toDate.HasValue)
{
voyages = voyages.Where(u => u.date <= toDate.Value);
}
return voyages.OrderBy(u => u.date);
Is there any real difference that may affect performance when this 2 approaches get transform to SQL expression?
The second query will create the simpler SQL because the evaluation of fromDate.HasValue and toDate.HasValue happens on the client. In the first query the ternary operators get evaluated on the database server as part of the SQL query. Both fromDate and toDate will be transmitted as constants to the server while in the second query only then if .HasValue is true.
We are talking about a few bytes more in length of the SQL statement and I don't believe that the server-side evaluation of the ternaries has any significant effect on query performance.
I would choose what you find more readable. Personally I would decide for the second query.
If you want the simple SQL and the more readable C# you can create an Extension as suggested by Viktor Mitev http://mentormate.com/blog/improving-linq-to-entities-queries/
public static class WhereExtensions
{
// Where extension for filters of any nullable type
public static IQueryable<TSource> Where<Tsource, TFilter>
(
this IQueryable <TSource> source,
Nullable <TFilter> filter,
Expression<Func<TSource, bool>> predicate
) where TFilter : struct
{
if (filter.HasValue)
{
source = source.Where(predicate);
}
return source;
}
// Where extension for string filters
public static IQueryable<TSource> Where<TSource>
(
this IQueryable<TSource> source,
string filter,
Expression<Func<TSource, bool>> predicate
)
{
if (!string.IsNullOrWhiteSpace(filter))
{
source = source.Where(predicate);
}
return source;
}
// Where extension for collection filters
public static IQueryable<TSource> Where<TSource, TFilter>
(
this IQueryable<TSource> source,
IEnumerable<TFilter> filter,
Expression<Func<TSource, bool>> predicate
)
{
if (filter != null && filter.Any())
{
source = source.Where(predicate);
}
return source;
}
Then your "secound query" will look like this:
var voyages = db.VoyageRequests.Include("Carrier")
.Where(u => u.Carrier != null && u.status == (int)VoyageStatus.State.InProgress)
.Where(u => u.date >= fromDate)
.Where(u => u.date <= toDate)
.OrderBy(u => u.date);
I don't know if it is more readable or if it will be confusing for some developers because it is more difficult to read directly form the code what part of the filtering is in use.
Maybe it will be more readable if you name the extensions function something like WhereIfFilterNotNull (or something meaningful :-)
var voyages = db.VoyageRequests.Include("Carrier")
.Where(u => u.Carrier != null && u.status == (int)VoyageStatus.State.InProgress)
.WhereIfFilterNotNull(u => u.date >= fromDate)
.WhereIfFilterNotNull(u => u.date <= toDate)
.OrderBy(u => u.date);

Some part of your SQL statement is nested too deeply

I have the following code
[WebGet]
public Bid GetHighestBidInOpenAuctions(int auctionEventId)
{
var auctionEvent = CurrentDataSource.AuctionEvents.Where(x => x.Id == auctionEventId).FirstOrDefault();
var auctionIds = CurrentDataSource.Auctions.Where(x => x.AuctionEventId == auctionEventId && x.Ends > DateTime.UtcNow).Select(x => x.Id).ToList();
var bids = CurrentDataSource.Bids.Where(x => auctionIds.Any(t => t == x.AuctionId));
// If the auction Event has not yet started or there are no bids then show auction with high pre-sale estimate.
if (bids.Count() == 0 || auctionEvent.Starts > DateTime.UtcNow)
{
return null;
}
var highestBid = bids.Where(b => b.IsAutobid == false).OrderByDescending(b => b.Amount).FirstOrDefault();
return highestBid;
}
This line throws the below exception
if (bids.Count() == 0 || auctionEvent.Starts > DateTime.UtcNow)
Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
What's wrong?
EDIT
I have tried doing this
IQueryable<Bid> bids = CurrentDataSource.Bids.Where(b => 0 == 1);
foreach(var auctionId in auctionIds)
{
int id = auctionId;
bids = bids.Union(CurrentDataSource.Bids.Where(b => b.AuctionId == id));
}
But I still get the same error.
Rather than using a subquery, try replacing the bid query with:
var bids = CurrentDataSource.Bids.Where(b => b.AuctionEventId == auctionEventId
&& b.Auction.AuctionEvent.Starts > DateTime.UtcNow
&& b.Auction.Ends > DateTime.UtcNow);
if (bids.Count() == 0
{
return null;
}
It seems when you have too many things in your database then you will get this error (auctionIds in my case) because the generated sql will be too deeply nested. To solve this I came up with this solution. If anyone can do better then do. I'm posting this because someone may have this error in the future and if they do, in the absence of a better solution this might help them.
[WebGet]
public Bid GetHighestBidInOpenAuctions(int auctionEventId)
{
/*
* This method contains a hack that was put in under tight time constraints. The query to
* get bids for all open auctions used to fail when we had a large number of open auctions.
* In this implementation we have fixed this by splitting the open auctions into groups of 20
* and running the query on those 20 auctions and then combining the results.
*/
const int auctionIdSegmentSize = 20;
var auctionEvent = CurrentDataSource.AuctionEvents.Where(x => x.Id == auctionEventId).FirstOrDefault();
var auctionIds = CurrentDataSource.Auctions.Where(x => x.AuctionEventId == auctionEventId && x.Ends > DateTime.UtcNow).Select(x => x.Id).ToList();
int numberOfSegments = auctionIds.Count/auctionIdSegmentSize;
if (auctionIds.Count % auctionIdSegmentSize != 0)
numberOfSegments++;
var bidsList = new List<IQueryable<Bid>>();
for (int i = 0; i < numberOfSegments; i++)
{
int start = i*auctionIdSegmentSize;
int end;
if (i == numberOfSegments - 1)
{
end = auctionIds.Count - 1;
}
else
{
end = ((i + 1)*auctionIdSegmentSize) - 1;
}
var subList = auctionIds.GetRange(start, (end - start) + 1);
bidsList.Add(CurrentDataSource.Bids.Where(b => subList.Any(id => id == b.AuctionId)));
}
// If the auction Event has not yet started or there are no bids then show auction with high pre-sale estimate.
if (IsBidsCountZero(bidsList) || auctionEvent.Starts > DateTime.UtcNow)
{
return null;
}
var highestBid = FindHighestBid(bidsList);
return highestBid;
}
private Bid FindHighestBid(List<IQueryable<Bid>> bidsList)
{
var bids = new List<Bid>();
foreach (var list in bidsList)
{
bids.Add(list.Where(b => b.IsAutobid == false).OrderByDescending(b => b.Amount).FirstOrDefault());
}
bids.RemoveAll(b => b == null);
if (bids.Count == 0)
return null;
bids.Sort(BidComparison);
return bids[0];
}
private int BidComparison(Bid bid1, Bid bid2)
{
if (bid1.Amount < bid2.Amount)
return 1;
if (bid1.Amount > bid2.Amount)
return -1;
return 0;
}
private bool IsBidsCountZero(List<IQueryable<Bid>> bidsList)
{
int count = 0;
foreach (var list in bidsList)
{
count += list.Count();
}
return count == 0;
}
The problem is with auctionIds.Any(t => t == x.AuctionId) where EF cannot create a correct query. You can change it to:
var bids = CurrentDataSource.Bids.Where(x => auctionIds.Contains(x.AuctionId));
Where EF can convert auctionIds to a collection and pass to DB.

Entity Framework Using Predicates Issue

I am using predicates to refine search options and am getting two different results when i run what looks to be the same query.
This one is correct and returns the expected results:(hard coded strings)
_predicate = PredicateBuilder.False<TBLDESIGN>();
_predicate = _predicate.Or(a => a.IS_APPROVED == true & a.ACTIVE == true & a.DATE_APPROVED > beginDate & (a.KEYWORDS.Contains("red") || a.NAME.Contains("red")));
_predicate = _predicate.Or(a => a.IS_APPROVED == true & a.ACTIVE == true & a.DATE_APPROVED > beginDate & (a.KEYWORDS.Contains("geos") || a.NAME.Contains("geos")));
This one only returns the results from the second predicate "geos". The first one in the loop("red") is ignored. Values "red" and "geos" are inside the loop below.
string searchTerm = "";
_predicate = PredicateBuilder.False<TBLDESIGN>();
for (int i = 0; i < search.Count(); i++)
{
searchTerm = search[i].SearchTerm.Trim().ToString();
_predicate = _predicate.Or(a => a.IS_APPROVED == true & a.ACTIVE == true & a.DATE_APPROVED > beginDate & (a.KEYWORDS.Contains(searchTerm) || a.NAME.Contains(searchTerm)));
}
Here is the rest of the query if that makes any difference.
var results =
dbContext.TBLDESIGN
.Include(s => s.LKPRICE)
.Include(s => s.TBLDESIGNER)
.AsExpandable().Where(_predicate)
.OrderByDescending(s => s.DATE_APPROVED)
.Select(s => new
{
s.ACTIVE,
s.DATE_CREATED,
s.EXPORTED,
s.IMAGE_PATH,
DesignId = s.ID,
s.IS_APPROVED,
s.TBLDESIGNER.FIRST_NAME,
s.TBLDESIGNER.LAST_NAME,
s.TBLDESIGNER.ALIAS,
s.NAME,
s.LKPRICE.PRICE,
s.COMPLETED,
s.DATE_APPROVED,
DesignerId = s.TBLDESIGNER.ID,
s.VIEW_COUNT
}).ToList();
Question is how to get the second list of predicates to return the same results as the first series. Guessing there is something extra to do when you are passing a string variable in rather than a hard coded value.
Thanks,
Billy
Answer:
_predicate = PredicateBuilder.False<TBLDESIGN>();
for (int i = 0; i < search.Count(); i++)
{
string searchTerm = search[i].SearchTerm.Trim().ToString();
_predicate = _predicate.Or(a => a.IS_APPROVED == true & a.ACTIVE == true & a.DATE_APPROVED > beginDate & (a.KEYWORDS.Contains(searchTerm) || a.NAME.Contains(searchTerm)));
}
string declaration needs to occur inside of the loop.
_predicate = PredicateBuilder.False<TBLDESIGN>();
for (int i = 0; i < search.Count(); i++)
{
string searchTerm = search[i].SearchTerm.Trim().ToString();
_predicate = _predicate.Or(a => a.IS_APPROVED == true & a.ACTIVE == true & a.DATE_APPROVED > beginDate & (a.KEYWORDS.Contains(searchTerm) || a.NAME.Contains(searchTerm)));
}