Left Outer Join in Linq-To-Entities - entity-framework

Is there a way to do a left outer join in linq-to-entities WITHOUT having tables mapped with foreign keys?
Also, if our company decides to stick with using linq-to-entities despite all of its current flaws, do you think it's important to get Visual Studio 2010? In other words, what is in Visual Studio 2010 that would help developing with entities that isn't in Visual Studio 2008?
Thanks.

This may not satisfy you because the objects are not properly flattened, but you can perform an outer join and return a collection of anonymous types.
A left outer join is the same as the union of the inner join and the except set (the part of the first set which does not join onto the second set).
Here I simply
calculate the inner join
calculate the except set (creating an empty B entity to ensure the types are consistent)
combine the two sets.
The LINQ is not beautiful, but it is cute.
class A
{
public int Id { get; set; }
public string PropertyA { get; set; }
}
class B
{
public int Id { get; set; }
public string PropertyB { get; set; }
}
var aThings = new List<A>();
var bThings = new List<B>();
var innerJoin = aThings.SelectMany(a =>
bThings.Where(b => a.Id == b.Id).Select(b => new { a, b })).ToList();
var exceptSet = aThings.Where(a =>
!bThings.Select(b => b.Id).Contains(a.Id)).Select( a =>
{
B b = new B();
return new { a, b };
});
var outerJoin = innerJoin;
outerJoin.AddRange(exceptSet);
The result is a List of anonymous types {a, b}

Microsoft has two data teams that both aim toward the same goal: LINQ-to-SQL and ADO.NET entity. There support lies now with ADO.NET for Entities which means that LINQ-to-SQL won't be as high a priority as the ADO.NET offering.
Visual Studio 2010 has support for .NET 4 which in turn supports ADO.NET for entities and LINQ-to-SQL. Aside from the number of productivity features presents in VS2010, I've not seen much in terms of data support that's different from VS2008.
In terms of left outer join:
var dc = new DataContext();
var query = (
from t1 in dc.MyTable
join a in MyJoinedTable on p.Id equals a.Id into tempTable
from t2 in tempTable.DefaultIfEmpty()
select new { p.Column1, p.Column2, t2.Column1}
);

Related

Entity Framework group by left join query

The Result should look like this
I've started to use Entity Framework for one month so I am not familiar with linq queries. The query I wrote in SQL is:
SELECT
om0001.CUSTOMER, om0001.ITEM_CODE,
SUM(om0001.AMOUNT) AS AMOUNT,
SUM(ep0001.EXPORT_AMOUNT) AS EXPORT_AMOUNT
FROM
om0001
LEFT OUTER JOIN
ep0001 ON om0001.ID = ep0001.om0001_ID
GROUP BY
om0001.CUSTOMER, om0001.ITEM_CODE;
When I run this query in SQL, it runs well so I tried to convert it to linq queries.
What I made so far is
var testjoin = from om0001 in EF.om0001
join ep0001 in EF.ep0001 on om0001.ID equals ep0001.om0001_ID
into jointable
from z in jointable.DefaultIfEmpty()
group z by new {om0001.CUSTOMER, om0001.ITEM_CODE } into g
select new
{
CUSTOMER = g.Key.CUSTOMER,
ITEM_CODE = g.Key.ITEM_CODE,
om0001SUMamount = g.Sum(x => x.AMOUNT),
ep0001EXPORTsumAmount = g.Sum(y => y.EXPORT_AMOUNT)
};
The problem over this linq query is I can not get om0001SUMamount. I get only ep0001 column data. Please help
Obviously, I cant peek into your EF database, so I created some sample data (the 'item' class structures are implied):
var EF = new efClass {
om0001 = new List<om0001item> {
new om0001item { ID = 0, CUSTOMER = 0, ITEM_CODE = 0, AMOUNT = 10 },
new om0001item { ID = 1, CUSTOMER = 0, ITEM_CODE = 0, AMOUNT = 20 },
new om0001item { ID = 2, CUSTOMER = 1, ITEM_CODE = 1, AMOUNT = 30 },
new om0001item { ID = 3, CUSTOMER = 1, ITEM_CODE = 1, AMOUNT = 40 }
},
ep0001 = new List<ep0001item> {
new ep0001item { om0001_ID = 0, EXPORT_AMOUNT = -20 },
new ep0001item { om0001_ID = 1, EXPORT_AMOUNT = -20 }
}
};
With this data, I created a query that frankly feels inelegant and left me disappointed, but that's the nature of left joins in LINQ:
var testjoin = from om0001 in EF.om0001
join ep0001 in EF.ep0001 on om0001.ID equals ep0001.om0001_ID into jointable
select new { om0001, ep0001 = jointable.DefaultIfEmpty() } into combined
group combined by new {
combined.om0001.CUSTOMER,
combined.om0001.ITEM_CODE
} into g
select new {
CUSTOMER = g.Key.CUSTOMER,
ITEM_CODE = g.Key.ITEM_CODE,
om0001SUMamount = g.Sum(x => x.om0001.AMOUNT),
ep0001EXPORTsumAmount = g.Sum(x => x?.ep0001.Sum(y => y?.EXPORT_AMOUNT ?? 0))
};
Bottom line is that when you group by 'jointable', you've lost ep0001 references. So select both ep0001 and om0001 into a new 'combined' object, and then group based off of it.
When I created a javascript library (fluent-data) that had some LINQ-like functionality, I developed a lot of respect and compassion for the LINQ developers. Nevertheless, I don't know why they don't just create a left join operator to add so much more value to all the C# developers who use LINQ.
So you have a table with Oms (actually Om00001, but I'm not going to write all those 0001 over and over again), and a table with Eps, and you have a one-to-many relation between Oms and Eps: Every Om has zero or more Eps, and every Ep belongs to exactly one Om, namely the Om that foreign key EpId refers to.
If you have followed Entity Framework code first conventions you will have classes similar to the following:
class Om
{
public int Id {get; set;}
public string Customer {get; set;}
public string ItemCode {get; set;}
...
// Every Om has zero or more Eps (one-to-many)
public virtual ICollection<Ep> Eps {get; set;}
}
class Ep
{
public int Id {get; set;}
public int Amount {get; set;}
public int ExportAmount {get; set;}
...
// every Ep belongs to exactly one Om, using foreign key
public int OmId {get; set;}
public virtual Om Om {get; set;}
}
This is enough for entity framework to detect your one-to-many relationship. Because I followed the conventions, there is no need for Attributes, nor fluent API. If you want different table names, or columns, you need fluent API / attributes.
In entity framework the non-virtual properties represent the columns of your tables, the virtual properties represent the relations between the tables (one-to-many, many-to-many, ...)
Solution using GroupJoin
var result = dbContext.Oms.GroupJoin(dbContext.Eps,
om => om.Id, // from every Om take the primary key
ep => ep.OmId, // from every ep take the foreign key to the Om
(om, epsOfThisOm) => new // from every om and all eps having the correct foreign key
{ // make one new object
// Select only the properties that you plan to use:
Customer = om.Customer,
ItemCode = om.ItemCode,
Amount = epsOfThisOm.Sum(ep => ep.Amount);
ExportAmount = epsOfThisOm.Sum(ep => ep.ExportAmount);
});
Solution using the virtual ICollection
Instead of executing a GroupJoin, you could also use the virtual ICollection.
Requirement: from every Om, give me the Customer and the ItemCode, and the sum of all Amounts of its Eps, and the sum of all ExportAmounts of its Eps.
var result = dbContext.Oms.Select(om => new
{
Customer = om.Customer,
ItemCode = om.ItemCode,
Amount = om.Eps.Sum(ep => ep.Amount);
ExportAmount = om.Eps.Sum(ep => ep.ExportAmount);
});
This looks much neater, and it matches more directly your requirement. Entity framework knows the relations, and will do the correct GroupJoin for you.

Why only primitive types or enumeration types are supported in this context using EF?

In my application I try to execute a Join query using EntityFramework, Repository Pattern, It is throwing bellow error. What is the problem in the link query? Let me explain in details
Error Description
Unable to create a constant value of type 'Anonymous type'. Only
primitive types or enumeration types are supported in this context
Initialization
_repository = new GenericRepository<WeeklyEntry>();
_repositoryGroup = new GenericRepository<Group>();
_repositoryGroupMember = new GenericRepository<GroupMember>();
Fetching Logic
var groups = _repositoryGroup.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.ID, s.Name }).ToList();
var groupMembers = _repositoryGroupMember.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.GroupID, s.ID, s.Name })
.ToList();
Main Query [Not Working]
var results = (from we in _repository.GetAll()
join g in groups on we.GroupID equals g.ID into grpjoin
from g in grpjoin.DefaultIfEmpty()
join gm in groupMembers on we.DepositedByMemberID equals gm.ID into gmjoin
from gm in gmjoin.DefaultIfEmpty()
where gm.GroupID == g.ID
select new
{
GroupID = g.ID,
GroupName = g.Name,
MemberID = grpmresult.ID,
grpmresult.Name,
we.ID
}).ToList();
To try to achieve bellow SQL Query
select w.GroupID, g.Name, gm.Name, w.ID
from [dbo].[WeeklyEntry] as w
left outer join [dbo].[Group] as g on g.ID = w.GroupID
left outer join [dbo].[GroupMember] as gm on gm.GroupID = g.ID
AND gm.ID = w.DepositedByMemberID
order by w.GroupID
Strange Findings
If I include .ToList(); with each query like from we in _repository.GetAll().ToList() the entire query will work & give expected result without Any ERROR!!!
So if I convert each query return type to In-memory Or IEnumerable<> it is working as expected without any error but IQueryable<> query not working as expected.
New Code Snippet [Working]
var results = (from we in _repository.GetAll().ToList()
join g in groups on we.GroupID equals g.ID into grpjoin
from g in grpjoin.DefaultIfEmpty()
join gm in groupMembers on we.DepositedByMemberID equals gm.ID into gmjoin
from gm in gmjoin.DefaultIfEmpty()
where gm.GroupID == g.ID
select new {...}.ToList();
You can't join a database table with an in-memory collection (in your case, List):
Unable to create a constant value of type Only primitive types or enumeration types are supported in this context
You're converting these to in-memory collections by calling ToList:
var groups = _repositoryGroup.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.ID, s.Name }).ToList();
var groupMembers = _repositoryGroupMember.GetAll().OrderBy(o => o.ID)
.Select(s => new { s.GroupID, s.ID, s.Name })
.ToList();
which you then try and join in the next bit of code.
If you simply remove those ToList calls, the join should work (you can keep the one for the final result, if you prefer it).
Remember that IEnumerable is lazy and will only actually run a SQL query if you "execute it" by iterating (usually via a foreach loop or some function like ToList).
Let me explain!
You have a GenericRepository<T> class, that is something like this:
public class GenericRepository<T>
{
MyDbContext dbContext;
public GenericRepository()
{
dbContext = new MyDbContext();
}
public IQueryable<T> GetAll()
{
// whatever
}
}
and then you have:
_repository = new GenericRepository<WeeklyEntry>();
_repositoryGroup = new GenericRepository<Group>();
_repositoryGroupMember = new GenericRepository<GroupMember>();
var groups = _repositoryGroup.GetAll().ToList(); // other query operators are irrelevant and removed
var groupMembers = _repositoryGroupMember.GetAll().ToList();
Calling ToList() runs your queries and bring the data to memory. Now you have two in-memory collection (List<T>).
When you write:
var results = from we in _repository.GetAll() // <-- this is IQueryable<T>
join g in groups // <-- this is List<T> (IEnumerable<T>)
on we.GroupID equals g.ID into grpjoin
...
you are joining an IQueryable, with an in-memory list. When running this query, EF has no way to know that your in-memory lists (groups and groupMembers) are actually queries from the database. It only sees two lists, containing some data. It has no way to translate that into SQL and hence throws an error.
To fix this, you should remove calls to ToList(). That way, you have three IQueryables that are joined together. EF can translate that into SQL, only if they are from a single DbContext. And since they are not, it throws another error, telling you exactly that.
You create an instance DbContext for each of the GenericRepository<T> instances. So, groups, groupMembers and we come from three different DbContexts.
To solve this error, you should somehow manage to use a single DbContext for all your GenericRepository<T>s.
For example:
using (var dbContext = new MyDbContext())
{
var groups = dbContext.Set<Group>();
var groupMembers = dbContext.Set<GroupMember>();
var results = from we in dbContext.Set<WeeklyEntry>()
join g in groups
on we.GroupID equals g.ID into grpjoin
...
}

Entity Framework: Left Join with List Result

I'm trying to optimize my EF queries. I have an entity called Employee. Each employee has a list of tools. Ultimately, I'm trying to get a list of employees with their tools that are NOT broken. When running my query, I can see that TWO calls are made to the server: one for the employee entities and one for the tool list. Again, I'm trying to optimize the query, so the server is hit for a query only once. How can I do this?
I've been exploring with LINQ's join and how to create a LEFT JOIN, but the query is still not optimized.
In my first code block here, the result is what I want, but -- again -- there are two hits to the server.
public class Employee
{
public int EmployeeId { get; set; }
public List<Tool> Tools { get; set; } = new List<Tool>();
...
}
public class Tool
{
public int ToolId { get; set; }
public bool IsBroken { get; set; } = false;
public Employee Employee { get; set; }
public int EmployeeId { get; set; }
...
}
var x = (from e in db.Employees.Include(e => e.Tools)
select new Employee()
{
EmployeeId = e.EmployeeId,
Tools = e.Tools.Where(t => !t.IsBroken).ToList()
}).ToList();
This second code block pseudoly mimics what I'm trying to accomplish. However, the GroupBy(...) is being evaluated locally on the client machine.
(from e in db.Employees
join t in db.Tools.GroupBy(tool => tool.EmployeeId) on e.EmployeeId equals t.Key into empTool
from et in empTool.DefaultIfEmpty()
select new Employee()
{
EmployeeId = e.EmployeeId,
Tools = et != null ? et.Where(t => !t.IsBroken).ToList() : null
}).ToList();
Is there anyway that I can make ONE call to the server as well as not having my GroupBy() evaluate locally and have it return a list of employees with a filtered tool list with tools that are not broken? Thank you.
Shortly, it's not possible (and I don't think it ever will be).
If you really want to control the exact server calls, EF Core is simply not for you. While EF Core still has issues with some LINQ query translation which leads to N+1 query or client evaluation, one thing is by design: unlike EF6 which uses single huge union SQL query for producing the result, EF Core uses one SQL query for the main result set plus one SQL query per each correlated result set.
This is sort of explained in the How Queries Work EF Core documentation section:
The LINQ query is processed by Entity Framework Core to build a representation that is ready to be processed by the database provider
The result is cached so that this processing does not need to be done every time the query is executed
The result is passed to the database provider
The database provider identifies which parts of the query can be evaluated in the database
These parts of the query are translated to database specific query language (for example, SQL for a relational database)
One or more queries are sent to the database and the result set returned (results are values from the database, not entity instances)
Note the word more in the last bullet.
In your case, you have 1 main result set (Employee) + 1 correlated result set (Tool), hence the expected server queries are TWO (except if the first query returns empty set).
You can use this:
var x = from e in _context.Employees
select new
{
e,
Tools = from tool in e.Tools where !tool.IsBroken select tool
};
var result = x.AsEnumerable().Select(y => y.e);
Which will be finally translated to a SQL query like below depending on your provider:
SELECT
`Project1`.`EmployeeId`,
`Project1`.`Name`,
`Project1`.`C1`,
`Project1`.`ToolId`,
`Project1`.`IsBroken`,
`Project1`.`EmployeeId1`
FROM (SELECT
`Extent1`.`EmployeeId`,
`Extent1`.`Name`,
`Extent2`.`ToolId`,
`Extent2`.`IsBroken`,
`Extent2`.`EmployeeId` AS `EmployeeId1`,
CASE WHEN (`Extent2`.`ToolId` IS NOT NULL) THEN (1) ELSE (NULL) END AS `C1`
FROM `Employees` AS `Extent1` LEFT OUTER JOIN `Tools` AS `Extent2` ON (`Extent1`.`EmployeeId` = `Extent2`.`EmployeeId`) AND (`Extent2`.`IsBroken` != 1)) AS `Project1`
ORDER BY
`Project1`.`EmployeeId` ASC,
`Project1`.`C1` ASC
I change my previous answer which was wrong, thanks to comments.

LINQ to Entities does not recognize the method with Let Statement

I am in the process of converting an application that uses LINQ to SQL over to LINQ to Entities. I use a repository pattern and I have run in a problem that works in LINQ to SQL but not Entities.
In my data layer, I use LINQ statements to fill my object graph so that none of my database entities are exposed anywhere else. In this example, I have a Lookup Respository that returns a list of Categories. It looks like this:
public IQueryable<Entities.DomainModels.Category> getCategories()
{
return (from c in Categories
where !c.inactive
orderby c.categoryName
select new Entities.DomainModels.Category
{
id = c.categoryID,
category = c.categoryName,
inactive = c.inactive
});
}
Later, I want to put the categories into a sub query and it looks like this:
var d = from p in Programs
let categories = (from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select c)
select new
{
id = p.id,
title = p.title
categories = categories.ToList()
};
When I run this, I get the following error:
LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[Entities.DomainModels.Category] getCategories()' method, and this method cannot be translated into a store expression.
For reference, the following works though it doesn't return the data I need (it's basically a join):
var q = from p in Programs
from pc in p.Categories
join c in getCategories() on pc.categoryID equals c.id
select new
{
id = p.id,
category = c
};
I understand what the error means in concept however LINQ to SQL would make it work. I have this pattern throughout my data layer and I really want to keep it. Should this be working? If not, how can I modify it without mixing my layers.
You cant pass getCategories() to EF.
The query must be destructible to expression tree.
Calculate getCategories() first.
eg
var simpleList = getCategories().Select(id).Tolist;
then use a contains
where(t=> simpleList.Contains(t.CatId) // or the query syntax equivalent

Entity Framework 4.1 : How to do sub query in Select

I am trying to do following using Entity Framework 4.1 Code First:
Select c1,
c2,
(select v1 from table2 where id = c3) as col3
from table1
Any help is really appreciated.
EDIT:
For example, let's say I have two tables in database,
LookupValues [ ID, Code, DescResourceID ]
Resources [ResourceID, Culture, Value ]
DescResourceID and ResourceID are Foreign Keys.
So I want to create a Configuration or mapping for...
public partial class LookupValues
{
public virtual string ID;
public virtual string Code;
public virtual string Description;
}
Description is mapped to subquery as mentioned above.
You can try something like this:
var query = from x in context.Table1
let z = context.Table2
.Where(y => y.Id == c3)
.Select(y => y.V1).FirstOrDefault()
select new
{
C1 = x.C1,
C2 = x.C2,
C3 = z
};
I would create an Entity Model and use LINQ.
I assume you know what LINQ is as you're working with Entity Framework.
Let's assume you have a data structure like the following;
Products and products has (Column1, Column2, Column3)
Categories and Categories key has a Column3 which is equivilant to the Products.Column3
Do the following;
var categories_Column3 = List<Categories>().Select(c => c.Column3 == some value);
var products_Column3_Query = List<Products>().Select(c => c.Column3 == categories_Column3).ToList<value_type_of_column3>();
Of course, this is just some psuedo code, and because I want to make sure you get the hang of LINQ and realize you can do what you want in LINQ using Entity Framework as it's OR/M I will now point you to the following site, which should give you a thorough tutorial on LINQ.
http://weblogs.asp.net/scottgu/archive/2007/05/19/using-linq-to-sql-part-1.aspx
Hope this helps!
Happy Coding! ;)