I am trying to use Entity Framework with an existing database. Using the code-first approach, I have gotten the following auto-created model (among others - I've tried to shorten the code to get to the essence of the question):
namespace Fidd.Models
{
using System;
using System.Collections.Generic;
public partial class Movies
{
public Movies()
{
...
this.MoviesPictures = new HashSet<MoviesPictures>();
...
}
public int MovieID { get; set; }
public string MovieName { get; set; }
...
public virtual ICollection<MoviesPictures> MoviesPictures { get; set; }
}
}
So basically this is a 1-n relationship between Movies and MoviesPictures. I am still in the process of learning EF.
If I want to load a single Movie with
var movie = from m in dbContext.Movies
where m.MovieID == 5
select m;
How do I get the MoviesPictures collection to be loaded automatically? Either eager or lazy.
UPDATE: There is actually an extra association:
Movies 1..n MoviesPictures n..1 Pictures
The MoviesPictures model is defined like this:
public partial class MoviesPictures
{
public int MoviePictureID { get; set; }
public int MoviePictureMovieID { get; set; }
public int MoviePicturePictureID { get; set; }
public System.DateTime MoviePictureAddDatetime { get; set; }
public bool MoviePictureRemoved { get; set; }
public Nullable<System.DateTime> MoviePictureRemovedDatetime { get; set; }
public virtual Movies Movies { get; set; }
public virtual Pictures Pictures { get; set; }
}
Is there any way to eager load this 2. layer of association within the same query? I've tried to do this:
var model = from m in db.Movies.Include("MoviesPictures").Include("Pictures")
where m.MovieID == id
select m
which does not work - I get a runtime Exception that Pictures is not defined with an navigation-attribute in Movies. Which of course makes sense. I just don't know how to specify the query otherwise.
Another thing that worries me... The above Include() statement does not catch any errors during compile-time. Is there a way to specify this in a type-safe fashion?
/Carsten
You would want to do:
var movie = from m in dbContext.Movies.Include("MoviesPictures")
where m.MovieId == 5
select m;
This is eagerly fetching the records from the "MoviesPictures" table. You can read a bit more about it here: http://msdn.microsoft.com/en-us/magazine/cc507640.aspx. Also if you google on "entity framework include" you can probably find a lot more information.
UPDATE
You might be able to do .Include("MoviesPictures.Pictures") it depends on how you have things set up. If not, then you want to do some joins; there's a good blog post here: http://weblogs.asp.net/salimfayad/archive/2008/07/09/linq-to-entities-join-queries.aspx on joins.
In regards to do it in "type safe"ly; this is the only method of "including" related records. You could, as I mentioned, use joining which might be a little closer to being "type safe".
Related
I have the ReportingActivity entity class.
public class ReportingActivity
{
public int ReportingActivityID { get; set; }
public DateTime ReportingActivitySend { get; set; }
public string Remark { get; set; }
public string SendersCSV { get; set; }
public string MailSenderStatus { get; set; }
public long RptGenerationCostMiliseconds { get; set; }
public DateTime RptGeneratedDateTime { get; set; }
public string RptGeneratedByWinUser { get; set; }
public string RptGeneratedOnMachine { get; set; }
public Int64 Run { get; set; }
public byte[] AttachmentFile { get; set; }
public virtual Report Report { get; set; }
public virtual Employee Employee { get; set; }
public virtual ReportingTask ReportingTask { get; set; }
}
I use this code to load data:
ctxDetail = new ReportingContext();
ctxDetail.ReportingActivity
.Where(x => x.Employee.EmployeeID == currentEmployee.EmployeeID)
.Load();
My code gets all the columns in (like SELECT * FROM... )
My question is how to skip the byte[] column, ideally recommend me a way how to improve my lines of code to be able specify exact list of columns.
Normally when dealing with a schema where records have large, seldom accessed details, it is beneficial to split those details off into a separate table as David mentions /w a one-to-one relationship back to the main record. These can be eager or lazy loaded as desired when they are needed.
If changing the schema is not practical then another option when retrieving data from the table is to utilize Projection via Select to populate a view model containing just the fields you need, excluding the larger fields. This will help speed up things like reads for views, however for things like performing updates you will still need to load the entire entity including the large fields to ensure you don't accidentally overwrite/erase data. It is possible to perform updates without loading this data, but it will add a bit of complexity and risk of introducing bugs if mismanaged later.
You can use Table Splitting, and optionally Lazy Loading to have only commonly needed columns loaded.
See
https://learn.microsoft.com/en-us/ef/core/modeling/table-splitting
This is for EF Core but it works the same on EF6 code-first.
We are using Entity Framework + Repository Pattern in a web based application to fetch database . Because of our complex business, our models are getting complex sometimes and this cause strange behaviour at Entity Framework eager loading system.
Please imagine our real model like this. We have tables, boxes which are on table, pencil cases which can be on table or in the box and pencils that can be on the table or in the box or in the pencil case.
We had modelled this in our application like this.
public class Table
{
public int TableID{ get; set; }
public virtual ICollection<Box> Boxes{ get; set; }
public virtual ICollection<PencilCases> PencilCases{ get; set; }
public virtual ICollection<Pencils> Pencils{ get; set; }
}
public class Box
{
public int BoxID{ get; set; }
public int TableID{ get; set; }
[ForeignKey("TableID")]
public virtual Table Table{ get; set; }
public virtual ICollection<PencilCases> PencilCases{ get; set; }
public virtual ICollection<Pencils> Pencils{ get; set; }
}
public class PencilCases
{
public int PencilCaseID{ get; set; }
public int? BoxID{ get; set; }
public int TableID{ get; set; }
[ForeignKey("TableID")]
public virtual Table Table{ get; set; }
[ForeignKey("BoxID")]
public virtual Box Box{ get; set; }
public virtual ICollection<Pencils> Pencils{ get; set; }
}
public class Pencils
{
public int PencilID{ get; set; }
public int? PencilCaseID{ get; set; }
public int? BoxID{ get; set; }
public int TableID{ get; set; }
[ForeignKey("TableID")]
public virtual Table Table{ get; set; }
[ForeignKey("BoxID")]
public virtual Box Box{ get; set; }
[ForeignKey("PencilCaseID")]
public virtual PencilCase PancelCase{ get; set; }
}
Our repository pattern implementation similar with this tutorial, http://www.asp.net/mvc/tutorials/getting-started-with-ef-5-using-mvc-4/implementing-the-repository-and-unit-of-work-patterns-in-an-asp-net-mvc-application
So we call get method like this.
var tables = unitOfWork.TableRepository.Get(includeProperties: "Boxes, PencilCases, Boxes.Pencils");
So the problem is the result is very different from my expectations;i expect only Boxes,PencilCases and Boxes.Pencils collections will be fetched, but all the Pencil entities fetched from database including Pencils, PencilCases.Pencils and Boxes.PencilCases.Pencils. This recursive fetch causes OutOfMemoryException because amount of data.
I couldn't understand why Entity Framework fetches all Pencils except Boxes.Pencils. I also tried to specify including list with Expression instead of Query Path but result didn't change.
first off - I'm fairly new to EF myself so please excuse if the following is not 100% accurate. However, I've dealt with this exact same problem just a couple of days ago, so hopefully this will help.
The problem is that when EF loads a specific entity, it will add that entity to every part of the Data Model that it appears in - not just the parts that were explicitly loaded.
This means that every Pencil in Boxes.Pencils that is also in the ICollection of Table.Pencils will be automatically resolved even though you did not specifically ask for it.
By itself that fact does not present a problem, and can even be helpful in a user-driven MVC application.
Where it all goes wrong is when you try to do anything that recurses trough the Data Entity, such as trying to map the self-recursing Data Entity to a Business Model or trying to turn the self-recursing data entity into JSON/XML.
Now, there are several solutions to this problem:
Implement a mapper / encoder that hashes / remembers each object and only adds it once:
The problem with this one is that it can lead to some hard-to-predict results, especially when you want / need the object in multiple places. Additionally, hashing and comparing every object could be costly.
Implement a mapper / encoder that can be configured to ignore some properties
Relatively simple - if you can specify that you don't want to map or encode Pencil at all, you won't have any issues. Downsides are of course that you could still encounter a stackoverflow if you are not vigilant about specifying the ignored properties.
Implement a mapper / encoder with specifyable recursion depth
This is a very simple and pretty decent solution - simply set a hard limit on recursion depth, either on a global or on a per-type basis, and you won't have any more stackoverflows. Downside is that you would still end up with elements that you don't want, and thus get a unnecessarily bloated return object.
Implement custom business entities
This is probably the best solution - simply create a new business entity with the offending navigational properties removed. The primary downside is that it would require you to create different business entities for different purposes.
Here is a example:
// Removed Pencils
public class BusinessTable
{
public int TableID{ get; set; }
public IEnumerable<Box> Boxes{ get; set; }
public IEnumerable<PencilCases> PencilCases{ get; set; }
}
// Removed Table & PencilCases
public class BusinessBox
{
public int BoxID{ get; set; }
public int TableID{ get; set; }
public IEnumerable<Pencils> Pencils{ get; set; }
}
// Removed Table & Box & Pencils
public class BusinessPencilCases
{
public int PencilCaseID{ get; set; }
public int? BoxID{ get; set; }
public int TableID{ get; set; }
}
// Removed Table, Box, PencilCase
public class BusinessPencils
{
public int PencilID{ get; set; }
public int? PencilCaseID{ get; set; }
public int? BoxID{ get; set; }
public int TableID{ get; set; }
}
Now when you map your Data Entity to this set of Business Entities, you won't get any more errors.
For the mapping aspect of this, theres 2 solutions: Manually doing things / using a mapping factory Example of Model Factory, ValueInjecter and AutoMapper - the latter two being available NuGet packages.
For AutoMapper:
I don't use AutoMapper, but you'd have to create a config file that looks something like this:
Mapper.CreateMap<Table, BusinessTable>();
Mapper.CreateMap<Box, BusinessBox>();
Mapper.CreateMap<PencilCases, BusinessPencilCases>();
Mapper.CreateMap<Pencils, BusinessPencils>();
And then in your query:
var tables = unitOfWork.TableRepository.Get(includeProperties: "Boxes, PencilCases, Boxes.Pencils");
var result = Mapper.Map<IEnumerable<Table>, IEnumerable<BusinessTable>>(tables);
Or
var tables = unitOfWork.TableRepository.Get(includeProperties: "Boxes, PencilCases, Boxes.Pencils").Project().To<IEnumerable<BusinessTable>;
For more info pertaining AutoMapper ( like how to set up a config file ): https://github.com/AutoMapper/AutoMapper/wiki/Getting-started
For ValueInjecter:
var tables = unitOfWork.TableRepository.Get(includeProperties: "Boxes, PencilCases, Boxes.Pencils");
var result = new List<BusinessTable>().InjectFrom(tables);
Or:
var tables = unitOfWork.TableRepository.Get(includeProperties: "Boxes, PencilCases, Boxes.Pencils");
var result = tables.Select(x => new BusinessTable.InjectFrom(x).Cast<BusinessTable>());
It might also be worthwhile to look at additional ValueInjecter Injections, like SmartConventionInjection, Deep Cloning, Useful Injections and a ORM with ValueInjecter guide.
I also made a few injections for my own project that may be of use to you, which you can find On my Github
With MaxDepthCloneInjector for example, you can supply a dictionary of (property names, max recursion depth) and it will only map values included in the dictionary, and only until the specified level.
Two more pieces of advice:
If you want a bit more freedom with your queries, you should consider using the Query Expression Syntax for some of your more complex needs. Theres also some good information in this answer on SO: How to limit number of related data with Include
If you are planning to run queries including navigational properties like the one in your example: STICK WITH EAGER LOADING. A query like that in Lazy Loading would lead to the N + 1 problem. As a rule of thumb:
Use Lazy Loading if you don't need the entire result set right away, for example if you are developing a application where data requirements naturally expand based on the User's interaction with the application.
Use Eager Loading if you need the entire result-set right away, for example in a Web Api, or a application that needs to work with the complete entity.
Best of luck,
Felix
My proc returns all results but MVC is showing the lookups are null.
I have a VIEW where the "grid" for loop looks like this:
#foreach (var item in Model) {
<tr>
<td>#Html.ActionLink(item.Headline, "ViewDetails", "Advisory", new { id=item.AdvisoryId }, "")</td>
<td>#item.AdvisoryStartDate.ToShortDateString()</td>
<td>#item.AdvisoryType.AdvisoryType</td>
<td>#item.AdvisoryProvider.AdvisoryProvider</td>
<td>#item.AdvisoryCategory.AdvisoryCategory</td>
</tr>
}
The last three fields are lookups. I initially populated this with this controller call:
var advisories = db.Advisories.Include(i => i.AdvisoryType)
.Include(i => i.AdvisoryProvider)
.Include(i => i.AdvisoryCategory)
.OrderByDescending(i => i.AdvisoryStartDate);
ViewData["Advisories"] = advisories;
But I've now replaced that with this proc call (because I need to get "advisories" the user hasn't seen yet, so it's more complicated than the above, which is basically "get all"; I also need to query the DB by UserID whereas MVC only has User.Identity.Name):
var results = db.Database.SqlQuery<Advisory>("EXEC [ponzi].[uspUserAdvisories] {0}", User.Identity.Name).ToList();
ViewData["Advisories"] = results;
I know the proc returns all results (when I run it in SQL Server), but in MVC, when I mouse over "results", the AdvisoryCategoryID is filled in but AdvisoryCategory is null. Same with the other two lookups. I assume it's because AdvisoryCategoryID is in my Advisory model, but AdvisoryCategory is from the AdvisoryCategory model.
Here is the relevant part of the Advisory model:
[Key]
[DatabaseGeneratedAttribute(DatabaseGeneratedOption.Identity)]
public int AdvisoryId { get; set; }
public virtual int AdvisoryTypeID { get; set; }
public virtual AdvisoryTypes AdvisoryType { get; set; }
public virtual AdvisoryProviders AdvisoryProvider { get; set; }
public virtual int AdvisoryProviderID { get; set; }
public virtual AdvisoryCategories AdvisoryCategory { get; set; }
public virtual int AdvisoryCategoryID { get; set; }
How can I get these three lookup fields to show the data returned by the proc? Thanks.
I think it's about the eager loading of navigation properties (AdvisoryCategory etc), if I understand properly what you're trying to do...
If you're trying to load navigation properties - from your SQL query, SP - I don't think that'd work. That's the Include part of the normal linq statement above - and that generates
an more complex SQL to join in other table's properties etc. It gets
more complicated with more of it.
That doesn't work for code-first and even if it would requires special 'techniques' to get it to match.
See this link here for the confirmation Eager Load from Entity Framework SP
You can try setting it to lazy loading by default (virtual etc.) - and hope it'd load on first access. Also try different loading methods.
Or just explicitely load on demand sort of - see this link and Explicitly Loading.
Eager Load from Entity Framework SP
http://msdn.microsoft.com/en-US/data/jj574232
Eager loading in EntityFramework with DbContext.Database.SqlQuery
I found a solution: ViewModels. Relevant portion of the new AdvisoriesViewModel here (to replace my Advisories model):
public int AdvisoryTypeID { get; set; }
public string AdvisoryType { get; set; }
public string AdvisoryProvider { get; set; }
public int AdvisoryProviderID { get; set; }
public string AdvisoryCategory { get; set; }
public int AdvisoryCategoryID { get; set; }
Now I call my proc with this line, having replaced "Advisory" with "AdvisoryViewModel":
var advisories = db.Database.SqlQuery<AdvisoryViewModel>("EXEC [ponzi].[uspUserAdvisories] {0}", User.Identity.Name).ToList();
Since the returned data set has a model with the same fields, and which aren't relying on another model, it works.
I want to know what is the difference between creating classes with or without using "hashset" in constructor.
Using code first approach (4.3) one can creat models like this:
public class Blog
{
public int Id { get; set; }
public string Title { get; set; }
public string BloggerName { get; set;}
public virtual ICollection<Post> Posts { get; set; }
}
public class Post
{
public int Id { get; set; }
public string Title { get; set; }
public DateTime DateCreated { get; set; }
public string Content { get; set; }
public int BlogId { get; set; }
public ICollection<Comment> Comments { get; set; }
}
or can create models like this :
public class Customer
{
public Customer()
{
BrokerageAccounts = new HashSet<BrokerageAccount>();
}
public int Id { get; set; }
public string FirstName { get; set; }
public ICollection<BrokerageAccount> BrokerageAccounts { get; set; }
}
public class BrokerageAccount
{
public int Id { get; set; }
public string AccountNumber { get; set; }
public int CustomerId { get; set; }
}
What is hashset doing here?
should i use hashset in the first two models also?
is there any article which shows the application of hashset?
Generally speaking, it is best to use the collection that best expresses your intentions. If you do not specifically intend to use the HashSet's unique characteristics, I would not use it.
It is unordered and does not support lookups by index. Furthermore, it is not as well suited for sequential reads as other collections, and the fact that it allows you to add the same item multiple times without creating duplicates is only useful if you have a reason to use it for that. If that is not your intention, it can hide misbehaving code and make problems difficult to isolate.
The HashSet is mostly useful in situations where insertion and removal times are very important, such as when processing data. It is also extremely useful for comparing sets of data (again when processing) using operations like intersect, except, and union. In any other situation, the cons generally outweigh the pros.
Consider that when working with blog posts, inserts and removes are quite rare compared to reads, and you generally want to read the data in a specific order, anyway. That is more or less the exact opposite of what the HashSet is good at. It is highly doubtful that you would ever intend to add the same post twice, for any reason, and I see no reason why you would use set-based operations on posts in a class like that.
The HashSet does not define the type of collection that will be generated when you actually fetch data. This will always be of type ICollection as declared.
The HashSet created in the constructor is to help you avoid NullReferenceExceptions when no records are fetched or exist in the many side of the relationship. It is in no way required.
For example, based on your question, when you try to use a relationship like...
var myCollection = Blog.Posts();
If no Posts exist then myCollection will be null. Which is OK, until you fluent chain things and do something like
var myCollectionCount = Blog.Posts.Count();
which will error with a NullReferenceException.
Where as
var myCollection = Customer.BrokerageAccounts();
var myCollectionCount = Customer.BrokerageAccounts.Count();
will result in and empty ICollection and a zero count. No exceptions :-)
I'm fairly new to Entity Framework but this is my understanding. The collection types can be any type that implements ICollection<T>. In my opinion a HashSet is usually the semantically correct collection type. Most collections should only have one instance of a member (no duplicates) and HashSet best expresses this. I have been writing my classes as shown below and this has worked well so far. Note that the collection is typed as ISet<T> and the setter is private.
public class Customer
{
public Customer()
{
BrokerageAccounts = new HashSet<BrokerageAccount>();
}
public int Id { get; set; }
public string FirstName { get; set; }
public ISet<BrokerageAccount> BrokerageAccounts { get; private set; }
}
Imagine I have an entity called Product and a repository for it:
public class Product
{
public int Id { get; set; }
public bool IsHidden { get; set; }
}
public class ProductRepository
{
public ObservableCollection<Product> AllProducts { get; set; }
public ObservableCollection<Product> HiddenProducts { get; set; }
}
All products contains every single Product in the database, while HiddenProducts must only contain those, whose IsHidden == true. I wrote the type as ObservableCollection<Product>, but it does not have to be that.
The goal is to have HiddenProducts collection be like a proxy to AllProducts with filtering capabilities and for it to refresh every time when IsHidden attribute of a Product is changed.
Is there a normal way to do this? Or maybe my logic is wrong and this could be done is a better way?
Ended up on CollectionView/CollectionViewSource stuff.