I am trying to make a left join with Linq to Entities in EF and eager load a collection in one of the entities.
The left join works with this code:
var joinedResult = context.Set<Client>()
.GroupJoin(
context.Set<Project>(),
client => client.Id,
project => project.ClientId,
(client, projects) => new
{
Client = client,
Project = projects.FirstOrDefault()
})
.ToList();
Now the Client entity has a Categories collection I want to load. Since this is a projection, I cannot include the collection before performing the group join, neither after the join since I am creating an anonymous type.
I tried to add the collection in the anonymous type as suggested by this anwser to a similar question:
ClientCategories = client.Categories.Select(s => s.Name),
but i keep getting an ObjectDisposedException when accessing the collection from the Client object outside the business layer.
Accessing the field directly after the query before returning from the business layer like this:
joinedResult.ForEach(result => result.Client.Categories.Count() /* NOP */);
works, but of course it results in a new database query per result.
How should I proceed to load this collection ?
Related
The property is defined as virtual. But before I access the order property, the order entity's data has been loaded, why?
Full source code:
A couple of things:
When you state: doesn't work, you are getting an Order back, but the $ figure is 0.0 when you expect a different value? You appear have 2x order records, but based on what is coming back you're expecting a non-zero figure, are both records non-zero? In your debug view, expand the "Orders" in the pop-up context menu, this will reveal what order details EF has loaded.
Firstly you should be careful around the use of "OrDefault" renditions of the methods. Your code is assuming that a value is returned. In these cases you're better off using Single() or First() as applicable.
Additionally, when using First you should be specifying an OrderBy clause to ensure that there is a reliable ordering.
SaveChanges Should only be called if you modify data.
Lastly, lazy loading is an enabler for loading infrequently used data on demand. You should largely work to avoid relying on lazy-load calls. If you need the entire entities and you know you are going to use orders, then eager-load them.
I.e.
using (var context = new EfContext())
{
var customer = context.Customers
.Include(c => c.Orders)
.Single(c => c.CustomerId = customerId);
// Do stuff...
}
If you want just 1 applicable order for a given employee then consider using Select to retrieve it:
I.e.
using (var context = new EfContext())
{
var data = context.Customers.Where(c => c.CustomerId = customerId)
.Select(c => new { Customer = c, FirstOrder = c.Orders.OrderBy(o => o.OrderDate).FirstOrDefault()})
.Single();
// Do Stuff...
}
This will give you an anonymous type containing the Customer (without eager loading the orders) and the matching first order.
Better is just to use Select to retrieve the specific fields you need from customer and order(s). This reduces the amount of data (rows and columns) pulled back from the database.
sorry,It looks like the problem of visual studio，When the call statement is commented,the order table's sql querie no longer being executed,but when the call statement is included,although the breakpoint is set,the order table's sql querie still be executed.
visual studio debug execute the Expression automatically
comment the staement
sql profiler
include the statement
sql profiler
I have an app using EF 6 and MVC 5 that works fine for inputting data, but now when I try to display some of it I'm having troubles. The basic layout of my entities can be seen in the following diagram:
The first part where I'm having trouble is in querying and filtering the data. I would like to return a list of premises and related data where a survey and signoff exist, but an approval does not. In straight SQL, the query that works now is:
SELECT *
FROM Premises p LEFT OUTER JOIN Approvals a ON a.Id = p.Id
JOIN Surveys s ON s.PremiseId = p.Id
JOIN SignOffs so ON so.Id = s.Id
WHERE a.ApprovedBy IS NULL
The code that I started with is like this:
var premises = Premises.Include(p => p.Approval)
.Include(p => p.Surveys)
.Include(p => p.Surveys.Select(s => s.SignOff));
This appears* to return all records including the child data, but when I try to filter it so I get only records that have a signoff record but do not have an approval, it doesn't work.
var premises = Premises.Include(p => p.Approval).Where(p => p.Approval.ApprovedBy == null)
.Include(p => p.Surveys)
.Include(p => p.Surveys.Select(s => s.SignOff).Where(s => s.Signature != null));
If I use this code, I get this error:
The Include path expression must refer to a navigation property defined on the type. Use dotted paths for reference navigation properties and the Select operator for collection navigation properties.
Parameter name: path
I've changed this query around a lot to try different things, so I'm not sure what all I have done, but I think the first Where statement might work by itself, but the second one definetly causes the error.
How do I need to structure my query to get it to return the requested data properly filtered?
Also, I put an asterisk above on stating that the query appears to return all the data and child data because I can't actually test it. When I'm trying to write my Razor CSHTML page for this, it's not giving me intellisense for the child and grandchild data, and if I enter what I think it should be I get errors. How do I need to reference this data on the page?
You cannot use Include() like this, it is only good for specifying to load a navigation property, not to specify to load an entity when a navigation property is something (not null, in your case).
To do the filtering, I suggest something like this:
var premises = Premises.Include(p => p.Approval).Include(p => p.Surveys).Include(p => p.Surveys.Select(s => s.SignOff))
.Where(p=>p.Approval.ApprovedBy!=null && p.Surveys.Any(s=>s.SignOff.Signature!=null));
So basically, the includes and the filtering have nothing to do with each other. With the includes, you only specify what to load, you can still use the filter on the original entity set.
You're confusing what the Include LINQ method does. It only tells EF to eagerly load that relationship, which is actually unnecessary if your query itself utilizes that relationship; EF will include the relationship by default in that case.
What it doesn't do is allow you filter those relationships. For example, in this portion of your code:
.Include(p => p.Surveys.Select(s => s.SignOff).Where(s => s.Signature != null));
The where clause is applied to Premises, not SignOff as you seem to think. In other words, Where filters the main table being queried, not the table you're including.
There's two paths forward here. You can simply filter Premises by the important parts, i.e.:
var premises = Premises.Where(p => p.Approval.ApprovedBy == null && p => p.Surveys.Any(s => s.SignOff.Signature != null));
That will return only premises where these conditions are true, but the included Surveys collection will contain all surveys related to each premise, not just the ones with null signoff signatures.
If you need to filter the related items as well, then you must explicitly load them:
foreach (premise in premises)
{
context.Entry(premise)
.Collection(p => p.Surveys)
.Query()
.Where(s => s.SignOff.Signature != null)
.Load();
}
Two things of note:
Because of the nature of how this query must be applied, there's no way to do it once for all premises. You'll have to iterate over the premises and explicitly load the Surveys collection for each.
Since this will issue a new query, you want to avoid loading the Surveys collection either lazily or eagerly before this explict load. Otherwise, you're querying the same information twice, which is very inefficient. The easiest way to ensure that is to remove the virtual keyword from the collection property. However, if you do that, then you will have to eager or explicitly load the collection or it will be null. For more information, see: https://msdn.microsoft.com/en-us/library/jj574232(v=vs.113).aspx
Quick question regarding this issue.
I have an error unable to get the data from database,
Is there something issue with EF Core or my linq query?
var query = _postRepository.Table
.Where(x => x.Approval == true)
.Include(p => p.Industry)
.GroupBy(grp => new { grp.Industry.Id, grp.Industry.Name })
.Select(s => new JobPostStatistics
{
IndustryId = s.Key.Id,
IndustryName = s.Key.Name,
TotalJobPost = s.Count()
});
return query.Take(6).ToList();
InvalidOperationException: Sequence contains more than one element
Seems in EF RC1 GroupBy cause error, here is reported bug:
GroupBy query cause error Sequence contains more than one element
EF Core 2.1 will not only work, but it will also be capable of translating GroupBy to SQL GROUP BY. The expected release date is Q4 2017. Previous versions also support GroupBy but execute it on the client side, which can be a huge problem if you're using it to retrieve aggregates, like SUM, from the database. Imagine retrieving 2 million rows from the server to calculate the sum on the client side, instead of getting it directly from the database server.
In the meantime, you can always write your own SQL query and map it to an entity or anonymous type, using DbSet.FromSql, i.e. invoking FromSql as a member of an entity collection of your context.
Here you have enabled eager loading by using include keyword. You try with disable lazy loading
Server-side GroupBy-support of EF Core has been postponed to after the release: https://github.com/aspnet/EntityFramework/issues/2341
When you log the queries EF creates you can see that it does not create GROUP BY in the SQL.
Here somebody else posted the same error with another GroupBy query.
https://github.com/aspnet/EntityFramework/issues/4775
I had another error with GroupBy and Include(). As EF does not support GroupBy in the SQL and the grouping is done on the client side, I split my query into a server side part and a client side part. The first part queries from the database with the Include() into an anonymous type. Then I do the grouping on the client side.
var event2Country = DbContext.Event2Countries
.Include(j => j.Location)
.Select(j => new { j.Location.Id, j.Location.Name })
.AsNoTracking();
viewModel.MostUsedCountries = event2Country
.GroupBy(j => new { j.Id, j.Name })
.OrderByDescending(j => j.Count())
.ThenBy(j => j.Key.Id)
.Select(j => new SimpleListCountViewModel()
{
Id = j.Key.Id,
Name = j.Key.Name,
Count = j.Count()
})
.Take(15)
.ToList();
Currently you cannot use GroupBy in EF Core if you have a lot of data. A workaround would be to use FromSql. See issue 4959 on Github.
I need to retrieve data using EF 4.1 Code first that are contained in several related tables. Currently if i use this
return Context.Set<Entity>()
.Include(x => x.Children.Select(y => y.GrandChildren.Select(z => z.Child)))
.Include(x => x.SomeEntity)
.Include(x => x.AnotherEntity)
.OrderByDescending(x => x.Id)
.FirstOrDefault();
The data is fetched correctly, but I am worried about two things:
1) it seems that there is no way to sort Chilren / GrandChildren
2) the data is flattened in one table, which means that Entity (and all others) data is duplicated for each GrandChild record
Questions:
What Do I need to do to be able to sort Children / GrandChildren?
The 2nd point itself may not be a problem in this particular case, since the amount of data that is transferred is not big - max 30 records with 30 columns. Still, I would like to know if there is a way to load Enttity, Children and GrandChildren separately (3 queries), and join them on client site?
Returned resultset needs to be updatable.
What Do I need to do to be able to sort Children / GrandChildren?
Sort them in your application. That is the only reliable way. Include doesn't allow sorting at all and the approach showed below is not reliable because you don't have control over EF's internal mechanism and you cannot have your navigation property as SortedList or other collection maintaining the sort order (which is requirement to have reliably sorted relation).
the data is flattened in one table, which means that Entity (and all others) data is duplicated for each GrandChild record
This is valid objection. You can avoid it by turning off lazy loading and use separate queries to load data:
context.Configuration.LazyLoadingEnabled = false;
var parent = context.Set<Entity>().First(e => e.Name = "ABC");
// Load relations in separate query
context.Set<Child>()
.Where(c => c.Parent.Name == "ABC")
.OrderBy(c => c.Name) // You can at least try it but as mentioned above it may not work in all scenarios
.Load();
// Now parent.Children collection should be filled
You can follow same approach for other relation and for nested relations as well. The key is to correctly construct Where condition for children and grandchildren loading.
It doesn't mean that this approach will be faster that flattened table. This approach makes separate database roundtrip for every executed query so the flattened table can be faster in smaller data sets.
Folks,
I am mapping EF objects to simple Pocos using automapper in my service layer. I have certain entities that have many relationships in EF,but I want to restrict how much of this gets pulled back from the db. One of these entities would map to a database table but would have many relationships to other tables that would appear as entity collections in the generated model (EDMX).
So I have created a "shallow" poco for the entity which only has poco properties for the first level of properties in the entity, i.e. some integer Ids instead of associated collections/entity collections. I map in the following manner....
var simplepocoforentity= Mapper.Map(_readOnlyDb.Single<EfEntity>(x => x.Id== Id),
new SimplPocoForEntity());
My question is.. because I am only mapping the entity to a simple poco here, can I be confident that EF will not try and query other non referenced data from the underlying db tables relationships when I do the mapping?
I know that I can investigate this via SQL Profiling, but I would appreciate any input befor going down that route.
Thanks
K.
It depends on the internal implementation of AutoMapper. I would assume that AutoMapper does not try to access the navigation properties of the entity (in which case lazy loading and an additional database query would kick in). I don't see a reason why AutoMapper should do this if you only map scalar properties. But who knows...
If you want to be on the safe side you could disable lazy loading temporarily:
try
{
_readOnlyDb.ContextOptions.LazyLoadingEnabled = false;
var simplepocoforentity = Mapper.Map(_readOnlyDb.Entities
.Single(x => x.Id == Id), new SimplPocoForEntity());
// ...
}
finally
{
_readOnlyDb.ContextOptions.LazyLoadingEnabled = true;
}
As a side note: Be aware that you load the full entity into the context with this approach. After that the mapping happens. Potentially you load more columns from the database than you really need (perhaps not in your special case here). In general I would do the mapping with a projection which ensures that the database only queries the columns which are needed:
var simplepocoforentity = _readOnlyDb.Entities
.Where(e => e.Id == Id)
.Select(e => new SimplPocoForEntity
{
PocoProperty1 = e.EntityProperty1,
PocoProperty2 = e.EntityProperty2,
PocoProperty3 = e.EntityProperty3
// etc.
})
.Single();
With this approach no lazy loading would happen at all because you don't load the entity but directly the PocoForEntity from the database.