I am using Entity Framework for querying data and I need to merge data already in memory (including new record) with the ones in db.
For example:
var linqdb = from d in context.Set<DEPARTMENT>()
join i in context.Set<INSTALLATION>()
on d.INSTID equals i.INSTID
select new { d.DEPTNAME, i.INSTNAME };
var linqm = from d in context.Set<DEPARTMENT>().Local
join i in context.Set<INSTALLATION>().Local
on d.INSTID equals i.INSTID
select new { d.DEPTNAME, i.INSTNAME };
var linqunion = linqm.Union(linqdb.AsEnumerable());
The difference is only the source where to execute the query:
the first is database the second is in memory.
How I could achieve a result without having to rewrite the LINQ query 2 times?
Regards
Luigi
Related
Select *,
(Select DefaultStartDay from Scheduler.ProgramSettings ps where ps.DefaultStartDay = s.Id ) [DefaultStartDay]
from Scheduler.Schedules s
where ScheduleType = 2;
I want to write above SQL query in EF Core, Specially I need subquery in select list to get data from another table with specific condition.
please refer image.Sample Data with SQL Query
I have tried below EF Core but getting wrong result.
var model = _context.Schedules
.Where(s => s.ScheduleType == 2)
.Select(rv => new ProgramSetting
{
Id = rv.Id,
ProgramTemplateId = rv.ProgramTemplateId,
IsActive = rv.IsActive,
DefaultStartDay = rv.Id
}).ToArray();
The SQL query is wrong and this is a misuse of EF Core.
First, that SQL will fail if there's more than 1 result from the subquery. Even in SQL you'd need a different query. An INNER JOIN would return the same results without failing if there are multiple matches.
Select s.*,ps.DefaultStartDay
from Scheduler.Schedules s
inner join Scheduler.ProgramSettings ps on ps.DefaultStartDay = s.Id
where ScheduleType = 2;
Second, using LINQ to emulate SQL is a misuse of both EF Core and LINQ. EF isn't a replacement for SQL, it's an ORM. Its job is to give the impression of working with in-memory objects instead of tables, not allow you to write SQL queries in C#
It's the ORM's job to generate JOINs as needed from the relations between entities (not tables). In this case, if Schedule has a ProgramSettins property, EF would generate the necessary joins automatically. Loading an entire schedule object could be as simple as :
var schedules=_context.Schedules
.Incule(sch=>sch.ProgramSettings)
.Where(s => s.ScheduleType == 2)
.ToArray();
Include is used to eagerly load the settings, not to force a JOIN.
If a Select clause is used that requires a property from ProgramSettings, the JOIN will be generated automatically, eg :
var namesAndDays=_context.Schedules
.Where(s => s.ScheduleType == 2)
.Select(s=>new {
Name = s.Name,
StartDay = s.ProgramSettings.DefaultStartDay
})
.ToArray();
In my application I try to execute a Join query using EntityFramework, Repository Pattern, It is throwing bellow error. What is the problem in the link query? Let me explain in details
Error Description
Unable to create a constant value of type 'Anonymous type'. Only
primitive types or enumeration types are supported in this context
Initialization
_repository = new GenericRepository<WeeklyEntry>();
_repositoryGroup = new GenericRepository<Group>();
_repositoryGroupMember = new GenericRepository<GroupMember>();
Fetching Logic
var groups = _repositoryGroup.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.ID, s.Name }).ToList();
var groupMembers = _repositoryGroupMember.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.GroupID, s.ID, s.Name })
.ToList();
Main Query [Not Working]
var results = (from we in _repository.GetAll()
join g in groups on we.GroupID equals g.ID into grpjoin
from g in grpjoin.DefaultIfEmpty()
join gm in groupMembers on we.DepositedByMemberID equals gm.ID into gmjoin
from gm in gmjoin.DefaultIfEmpty()
where gm.GroupID == g.ID
select new
{
GroupID = g.ID,
GroupName = g.Name,
MemberID = grpmresult.ID,
grpmresult.Name,
we.ID
}).ToList();
To try to achieve bellow SQL Query
select w.GroupID, g.Name, gm.Name, w.ID
from [dbo].[WeeklyEntry] as w
left outer join [dbo].[Group] as g on g.ID = w.GroupID
left outer join [dbo].[GroupMember] as gm on gm.GroupID = g.ID
AND gm.ID = w.DepositedByMemberID
order by w.GroupID
Strange Findings
If I include .ToList(); with each query like from we in _repository.GetAll().ToList() the entire query will work & give expected result without Any ERROR!!!
So if I convert each query return type to In-memory Or IEnumerable<> it is working as expected without any error but IQueryable<> query not working as expected.
New Code Snippet [Working]
var results = (from we in _repository.GetAll().ToList()
join g in groups on we.GroupID equals g.ID into grpjoin
from g in grpjoin.DefaultIfEmpty()
join gm in groupMembers on we.DepositedByMemberID equals gm.ID into gmjoin
from gm in gmjoin.DefaultIfEmpty()
where gm.GroupID == g.ID
select new {...}.ToList();
You can't join a database table with an in-memory collection (in your case, List):
Unable to create a constant value of type Only primitive types or enumeration types are supported in this context
You're converting these to in-memory collections by calling ToList:
var groups = _repositoryGroup.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.ID, s.Name }).ToList();
var groupMembers = _repositoryGroupMember.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.GroupID, s.ID, s.Name })
.ToList();
which you then try and join in the next bit of code.
If you simply remove those ToList calls, the join should work (you can keep the one for the final result, if you prefer it).
Remember that IEnumerable is lazy and will only actually run a SQL query if you "execute it" by iterating (usually via a foreach loop or some function like ToList).
Let me explain!
You have a GenericRepository<T> class, that is something like this:
public class GenericRepository<T>
{
MyDbContext dbContext;
public GenericRepository()
{
dbContext = new MyDbContext();
}
public IQueryable<T> GetAll()
{
// whatever
}
}
and then you have:
_repository = new GenericRepository<WeeklyEntry>();
_repositoryGroup = new GenericRepository<Group>();
_repositoryGroupMember = new GenericRepository<GroupMember>();
var groups = _repositoryGroup.GetAll().ToList(); // other query operators are irrelevant and removed
var groupMembers = _repositoryGroupMember.GetAll().ToList();
Calling ToList() runs your queries and bring the data to memory. Now you have two in-memory collection (List<T>).
When you write:
var results = from we in _repository.GetAll() // <-- this is IQueryable<T>
join g in groups // <-- this is List<T> (IEnumerable<T>)
on we.GroupID equals g.ID into grpjoin
...
you are joining an IQueryable, with an in-memory list. When running this query, EF has no way to know that your in-memory lists (groups and groupMembers) are actually queries from the database. It only sees two lists, containing some data. It has no way to translate that into SQL and hence throws an error.
To fix this, you should remove calls to ToList(). That way, you have three IQueryables that are joined together. EF can translate that into SQL, only if they are from a single DbContext. And since they are not, it throws another error, telling you exactly that.
You create an instance DbContext for each of the GenericRepository<T> instances. So, groups, groupMembers and we come from three different DbContexts.
To solve this error, you should somehow manage to use a single DbContext for all your GenericRepository<T>s.
For example:
using (var dbContext = new MyDbContext())
{
var groups = dbContext.Set<Group>();
var groupMembers = dbContext.Set<GroupMember>();
var results = from we in dbContext.Set<WeeklyEntry>()
join g in groups
on we.GroupID equals g.ID into grpjoin
...
}
I am new at Entity framework, and curious what the best way would be to update all tables with records of new data. I have a method which returns a list of objects with updated records. Most of the information stays the same; just two fields will be updated.
Currently I created two ways of doing that update.
The first one is to get data from the database table and iterate from both Lists to find a match and update that match:
var previousDatafromTable= db.Widgets.ToList();
var newDataReturnedFromMethod =.......
foreach (var d in previousDatafromTable)
{
foreach (var l in newDataReturnedFromMethod )
{
if (d.id == l.id)
{
d.PositionColumn = l.PositionColumn;
d.PositionRow = l.PositionRow;
}
}
The second one is:
foreach (var item in newDataReturnedFromMethod )
{
var model = db.Widgets.Find(item.id);
model.PositionColumn = item.PositionColumn;
model.PositionRow = item.PositionRow;
}
I am iterating through the updated data and updating my database table by ID.
So I am interested to know which method is the better way of doing this, and maybe there is an option in Entity Framework to measure the performance of these two tasks? Thanks for your time in answering.
Neither is really efficient.
The first option loops through newDataReturnedFromMethod for each iteration of previousDatafromTable. That's a lot of iterations.
The second options probably executes a database query for each iteration of newDataReturnedFromMethod.
It's far more efficient to join:
var query = from n in newDataReturnedFromMethod
join p in previousDatafromTable on n.id equals p.id
select new { n,p };
foreach (var pair in query)
{
pair.p.PositionColumn = pair.n.PositionColumn;
pair.p.PositionRow = pair.n.PositionRow;
}
EF doesn't have built-in performance measurements. You'd typically use a profiler for that, or the StopWatch class.
I am in the process of converting an application that uses LINQ to SQL over to LINQ to Entities. I use a repository pattern and I have run in a problem that works in LINQ to SQL but not Entities.
In my data layer, I use LINQ statements to fill my object graph so that none of my database entities are exposed anywhere else. In this example, I have a Lookup Respository that returns a list of Categories. It looks like this:
public IQueryable<Entities.DomainModels.Category> getCategories()
{
return (from c in Categories
where !c.inactive
orderby c.categoryName
select new Entities.DomainModels.Category
{
id = c.categoryID,
category = c.categoryName,
inactive = c.inactive
});
}
Later, I want to put the categories into a sub query and it looks like this:
var d = from p in Programs
let categories = (from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select c)
select new
{
id = p.id,
title = p.title
categories = categories.ToList()
};
When I run this, I get the following error:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[Entities.DomainModels.Category] getCategories()' method, and this method cannot be translated into a store expression.
For reference, the following works though it doesn't return the data I need (it's basically a join):
var q = from p in Programs
from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select new
{
id = p.id,
category = c
};
I understand what the error means in concept however LINQ to SQL would make it work. I have this pattern throughout my data layer and I really want to keep it. Should this be working? If not, how can I modify it without mixing my layers.
You cant pass getCategories() to EF.
The query must be destructible to expression tree.
Calculate getCategories() first.
eg
var simpleList = getCategories().Select(id).Tolist;
then use a contains
where(t=> simpleList.Contains(t.CatId) // or the query syntax equivalent
I am using entity framework 5 for a query like this:
var query =
from i in context.Instrument
from p in i.InstrumentPerformance // 1 : n
where p.PortfolioScenarioID == 6013
select i;
I want to store a queryable respresentation of this (filtered) query in memory. Ideally, I would be able to disconnect the context and still request a specific InstrumentPerformance collection like so:
var perf = query.First(i => i.InstrumentID == 407240).InstrumentPerformance;
But this - of course - does not produce the desired result, since the "perf" object will contain an InstrumentPerformance collection that contains every 1:n joined InstrumentPerformance entity (whether its PortfolioScenarioID is 6013 or not) and it will retrieve these entities via lazy loading, with context.ContextOptions.LazyLoadingEnabled = false (or the context runnning out of scope) the query will not yield anyting.
So this is far from where I want to get: an easy to query in-memory representation from the original query. I tried to materialize into dictionaries and similar approaches, but ended up coding custom data objects for the result which I would like to avoid.
So my question is: what is the recommended method to get such in-memory view?
EDIT:
I am currently using two dictionaries to cache the data, e.g:
var instruments = (
from i in context.Instrument
from p in i.InstrumentPerformance
where p.PortfolioScenarioID == 6013
select i)
.ToDictionary (i => p.InstrumentID, i => i);
var performances = (
from i in context.Instrument
from p in i.InstrumentPerformance
where p.PortfolioScenarioID == 6013
select p)
.ToDictionary (p => p.InstrumentID, p => p);
However, this requires two roundtrips to the database where one seems sufficient and more importantly the semantics for querying the performance data (which is now performances[InstrumentID]) is inconsistent with the EF way of querying (which should be instrument.InstrumentPerformance.First() and the like).
It is possible to retrieve the objects in one take first and then create the dictionaries by:
var query =
(from i in context.Instrument
select new {
i,
ps = i.InstrumentPerformance
.Where(p.PortfolioScenarioID == 6013)
}).AsEnumerable()
.Select(x => x.i);
This materializes and selects Instrument entities and, here's the trick, their partly loaded InstrumentPerformance collections. I.e. the instruments only contain InstrumentPerformance entities that meet the condition PortfolioScenarioID == 6013. This is because EF runs a process known as relationship fixup that ties child objects to the right parent object when they are fetched from the database.
So now you can dispose the context and any time after that do
var perf = query.First(i => i.InstrumentID == 407240).InstrumentPerformance;
or build your dictionaries using from i in query in stead of from i in context.Instrument.
IMPORTANT: lazy loading should be disabled, otherwise EF will still try to load the full collections when they are addressed.
Look at EntityCollection<T> CreateSourceQuery and Attach. I think you could do this (not tested):
var instrumentQuery =
from i in context.Instrument
from p in i.InstrumentPerformance // 1 : n
where p.PortfolioScenarioID == 6013
select i;
var instruments = instrumentQuery.ToList();
foreach (var instrument in instruments) {
var performanceQuery =
instrument.InstrumentPerformance.CreateSourceQuery()
.Where(p => p.PortfolioScenarioID == 6013);
instrument.InstrumentPerformance.Attach(performanceQuery);
}
This executes everything at once (no lazy loading) and has a bit of code duplication, but it would result in a list of Instrument where each i.InstrumentPerformance returns the filtered collection, meaning any subsequent code that operates on it can treat it like any other EF collection without needing to know the details of the query.