Why does this query always return ALL records? - entity-framework

I'm using WCF RIA in a Lightswitch project to create some query results. This query brings back all results regardless. I cannot make it filter the records based on the parameter passed (string Town).
public IQueryable<Enquiries> TestQuery(string Town)
{
List<Enquiries> riaenqs = new List<Enquiries>();
var enqs = this.Context.ClientEnquiries
.Include("Client")
.Include("Client.Town")
.OrderBy(enq => enq.Id);
if (Town != null)
{
enqs.Where(enq => enq.Client.Town.TownName == Town);
}
foreach (ClientEnquiry item in enqs.ToList())
{
Enquiries enq = new Enquiries();
enq.Id = item.Id;
enq.ClientName = item.Client.FirstName + " " + item.Client.Surname;
enq.Town = item.Client.Town != null ? item.Client.Town.TownName : null;
riaenqs.Add(enq);
}
return riaenqs.AsQueryable();
}
During debugging I can see that the Town is correctly populated and I can see that the query is built accordingly if Town is not null. However, when I hit the foreach statement where the linq to ef query is executed I always get all the results. I just cannot figure out where I'm slipping up.

The LINQ methods like the Where do not modify the collection/expression but always returning a new one.
So you need to reassign the result of the Where to your original variable enqs:
if (Town != null)
{
enqs = enqs.Where(enq => enq.Client.Town.TownName == Town);
}

Related

Entity Framework is too slow during mapping data up to 100k

I have min 100 000 data into a Job_Details table and I'm using Entity Framework to map the data.
This is the code:
public GetJobsResponse GetImportJobs()
{
GetJobsResponse getJobResponse = new GetJobsResponse();
List<JobBO> lstJobs = new List<JobBO>();
using (NSEXIM_V2Entities dbContext = new NSEXIM_V2Entities())
{
var lstJob = dbContext.Job_Details.ToList();
foreach (var dbJob in lstJob.Where(ie => ie.IMP_EXP == "I" && ie.Job_No != null))
{
JobBO job = MapBEJobforSearchObj(dbJob);
lstJobs.Add(job);
}
}
getJobResponse.Jobs = lstJobs;
return getJobResponse;
}
I found to this line is taking about 2-3 min to execute
var lstJob = dbContext.Job_Details.ToList();
How can i solve this issue?
To outline the performance issues with your example: (see inline comments)
public GetJobsResponse GetImportJobs()
{
GetJobsResponse getJobResponse = new GetJobsResponse();
List<JobBO> lstJobs = new List<JobBO>();
using (NSEXIM_V2Entities dbContext = new NSEXIM_V2Entities())
{
// Loads *ALL* entities into memory. This effectively takes all fields for all rows across from the database to your app server. (Even though you don't want it all)
var lstJob = dbContext.Job_Details.ToList();
// Filters from the data in memory.
foreach (var dbJob in lstJob.Where(ie => ie.IMP_EXP == "I" && ie.Job_No != null))
{
// Maps the entity to a DTO and adds it to the return collection.
JobBO job = MapBEJobforSearchObj(dbJob);
lstJobs.Add(job);
}
}
// Returns the DTOs.
getJobResponse.Jobs = lstJobs;
return getJobResponse;
}
First: pass your WHERE clause to EF to pass to the DB server rather than loading all entities into memory..
public GetJobsResponse GetImportJobs()
{
GetJobsResponse getJobResponse = new GetJobsResponse();
using (NSEXIM_V2Entities dbContext = new NSEXIM_V2Entities())
{
// Will pass the where expression to be DB server to be executed. Note: No .ToList() yet to leave this as IQueryable.
var jobs = dbContext.Job_Details..Where(ie => ie.IMP_EXP == "I" && ie.Job_No != null));
Next, use SELECT to load your DTOs. Typically these won't contain as much data as the main entity, and so long as you're working with IQueryable you can load related data as needed. Again this will be sent to the DB Server so you cannot use functions like "MapBEJobForSearchObj" here because the DB server does not know this function. You can SELECT a simple DTO object, or an anonymous type to pass to a dynamic mapper.
var dtos = jobs.Select(ie => new JobBO
{
JobId = ie.JobId,
// ... populate remaining DTO fields here.
}).ToList();
getJobResponse.Jobs = dtos;
return getJobResponse;
}
Moving the .ToList() to the end will materialize the data into your JobBO DTOs/ViewModels, pulling just enough data from the server to populate the desired rows and with the desired fields.
In cases where you may have a large amount of data, you should also consider supporting server-side pagination where you pass a page # and page size, then utilize a .Skip() + .Take() to load a single page of entries at a time.

declare variable to store linq entity for conditional statements

I am trying to look up record using if I have the key then use Find if not use Where
private ApplicationDbContext db = new ApplicationDbContext();
public bool DeactivatePrice(int priceId = 0, string sponsorUserName = "")
{
var prices = db.BeveragePrices;
// if we have an id then find
if (priceId != 0)
{
prices = prices.Find(priceId);
}
else
{
prices = prices.Where(b => b.UserCreated == sponsorUserName);
}
if (prices != null)
{
// do something
}
return true;
I get the following error for
prices = prices.Find(priceId);
Cannot convert app.Model.BeveragePrices from system.data.entity.dbset
I am copying the pattern from this answer but something must be different.
Seems you forgot to put a predicate inside the Find function call. Also you need to do ToList on the collection. The second option is a lot more efficient. The first one gets the whole collection before selection.
Another note commented by #Alla is that the find returns a single element. So I assume another declaration had been made for 'price' in the first option I state down here.
price = prices.ToList.Find(b => b.PriceId == priceId);
Or
prices = prices.Select(b => b.PriceId == priceId);
I assume the field name is PriceId.

IQueryable in foreach - open DataReader error

When i run this code
public PartialViewResult GetHardware()
{
IQueryable<Hardware> hardware = db.Hardwares;
HardwareState hwState = new HardwareState();
IQueryable<IGrouping<string, Hardware>> groupByCategory = hardware.GroupBy(g => g.Category);
foreach (IGrouping<string, Hardware> group in groupByCategory)
{
hwState.GroupName = group.Key;
hwState.GroupUnitsCount = group.Count();
hwState.StorageReservedCount
= group.Where(m =>
m.Place.IsStorage == true &&
m.PlaceID != (int)Constants.HardwareState.Created &&
m.HardwareState == (int)Constants.HardwareState.Reserved).Count();
}
return PartialView(hwState);
}
I get an error about that the navigation property m.Place = null
when i transfer some of the text with the code of the foreach block
public PartialViewResult GetHardware()
{
IQueryable<Hardware> hardware = db.Hardwares;
HardwareState hwState = new HardwareState();
IQueryable<IGrouping<string, Hardware>> groupByCategory = hardware.GroupBy(g => g.Category);
hwState.StorageReservedCount
= hardware.Where(m =>
m.Place.IsStorage == true &&
m.PlaceID != (int)Constants.HardwareState.Created &&
m.HardwareState == (int)Constants.HardwareState.Reserved).Count();
foreach (IGrouping<string, Hardware> group in groupByCategory)
{
hwState.GroupName = group.Key;
hwState.GroupUnitsCount = group.Count();
}
return PartialView(hwState);
}
,the navigation property is not set to null and the error does not appear
Extension methods such as .AsQueryable or Include(x => x.Place) do not help me
How can i solve this problem?
UPDATE: If i change the type to IEnumerable instead IQueryable it begins to work!
but i would like to work with the IQueryable type
UPDATE2: I'm sorry, i did not put it correctly when i wrote that the error is an empty navigation property. Error that appears in fact
"There is already an open DataReader associated with this Command which must be closed first."
As described in this answer it is because: (quote)
"Another scenario when this always happens is when you iterate through
result of the query (IQueryable) and you will trigger lazy loading for
loaded entity inside the iteration."
But it does not say how to solve the problem without using ToList () or MARS

Generic repository: how to filter eager-loaded navigation properties

this is my generic respoitory that im using, as the title states I want to know how to filter the navigation properties.
public IEnumerable<T> Query(
Expression<Func<T, bool>> filter = null,
Func<IQueryable<T>, IOrderedQueryable<T>> orderBy = null,
string includeProperties = "")
{
IQueryable<T> query = _objectSet.Where(e => !e.IsDeleted);
if (filter != null)
{
query = query.Where(filter);
}
foreach (var includeProperty in includeProperties.Split
(new char[] { ',' }, StringSplitOptions.RemoveEmptyEntries))
{
query = query.Include(includeProperty);
}
if (orderBy != null)
{
return orderBy(query).ToList();
}
else
{
return query.ToList();
}
}
Controller:
var viewModel = new StudentViewModel();
viewModel.Students= _unitOfWork.Students.Query(
includeProperties: "Subjects, Instructors");
Now my problem is I want to add a .Where(e => !e.IsDeleted)
to [Subjects] and [Instructors] using the repository.
Thanks
EDIT:
according to Ladislav, currently it is not possible (also mentioned here in msdn: http://blogs.msdn.com/b/adonet/archive/2011/01/31/using-dbcontext-in-ef-feature-ctp5-part-6-loading-related-entities.aspx)
Can i just use this instead?
viewModel.Subjects = viewModel.Students.Where(i => i.StudentID ==Id.Value)
.Single().Subjects.Where(e => !e.IsDeleted);
My only worry is that the query may return lots of records with isDeleted==true. Sure the code I posted as an alternative works, I just don't want to pull the data that I don't need even though I can filter it using the above code
LINQ to SQL supports this scenario using the LoadWith DataLoadOption. The example at http://msdn.microsoft.com/en-us/library/system.data.linq.dataloadoptions.loadwith.aspx shows the simple case that EF does support with the Include statement.
Northwnd db = new Northwnd(#"c:\northwnd.mdf");
DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Customer>(c => c.Orders);
db.LoadOptions = dlo;
However, unlike EF, LINQ to SQL also supports the following:
dlo.LoadWith<Customer>(c => c.Orders.Where(o => o.ShippedDate is Null);
If you feel this is an important enhancement scenario for EF, consider voting for it at http://data.uservoice.com/forums/72025-entity-framework-feature-suggestions/suggestions/1015345-allow-filtering-for-include-extension-method.
For now, your best option is to project your filter in the Select clause, but that gets tricky with a generic repository.

The 'ArrayIndex' LINQ expression node type is not supported by LINQ to Entities - using interface & ReportViewer

a quick question really.
I'm struggling to implement Linq2Entities statement that could take more than one value for a particular "field". I'm passing a number of strings to the getClientsProjected() I can easily compare single value. But I've got on my page multiple dropdown and out of that I get string separated with coma I then later use to split it to string[] e.g. __ACCOUNT_SITE = "1234,5678" (see the code below) I've tried for/foreach/contains none of which worked...
public IQueryable<ClientViewModel> getClientsProjected(string __ACCOUNT_SITE, string __ACCOUNT)
{
var projectedClients = from c in getClosedSRs()
select new ClientViewModel
{
_ACCOUNT_ID_CSR = c.ACCOUNT_ID_CSR,
_ACCOUNT = c.ACCOUNT,
_ACCOUNT_FAMILY = c.ACCOUNT_FAMILY,
...
...
_ACCOUNT_SITE = c.ACCOUNT_SITE
};
if (String.IsNullOrEmpty(__ACCOUNT) != true && __ACCOUNT != "ALL")
{
//this works fine as an __ACCOUNT is of a single value
projectedClients = projectedClients.Where(c => c._ACCOUNT == __ACCOUNT);
}
if (String.IsNullOrEmpty(__ACCOUNT_SITE) != true && __ACCOUNT_SITE != "ALL")
{
String[] splitSites = __ACCOUNT_SITE.Split(',');
//????????????????????????????????????????????????
}
return projectedClients;
}
Now, to most of you this will make complete sense. I've read many articles but did not find a proper answer. I however can't use Linq2SQL as already built my entire site using L2E, interface and ReportViewer.
Any workaround?
If you are trying to filter projectedClients based on the values in splitSites, then use:
if (String.IsNullOrEmpty(__ACCOUNT_SITE) != true && __ACCOUNT_SITE != "ALL")
{
String[] splitSites = __ACCOUNT_SITE.Split(',');
projectedClients = projectedClients.Where(x => splitSites.Contains(x._ACCOUNT);
}