I am using Entity Framework 6.1 with breeze 1.4.11. I am trying to save some data in local storage. My Breeze controller:
[HttpGet]
public object Lookups()
{
return new
{
lookupItems = _contextProvider.Context.LookupItem
};
}
My EF settings:
this.Configuration.ProxyCreationEnabled = false;
this.Configuration.LazyLoadingEnabled = false;
The class I am trying to load:
public class LookupItem : BaseEntityInt
{
public int LookupTypeId { get; set; }
public string Description { get; set; }
public int Ordinal { get; set; }
public virtual ICollection<Party> PartyCountries { get; set; }
}
My breeze client code:
_lookupsQuery = new breeze.EntityQuery.from("Lookups"),
em.executeQuery(_lookupsQuery).then(function (data) {
_lookups = data.results;
// Save data to local storage
store.setItem("Lookups", _lookups); // Fails here
});
The problem is when I save it in local storage, store uses amplifyJs to persist to local storage. AmplifyJs calls JSON.stringify on the persisted object. In the console of the browser I get the following error:
Error: cyclic object value
This is because breeze is automatically loading the navigation property of lookupitem 'partycountries' which itself links back to lookupitem. Anyone any idea why it is doing this? My understanding was you had to explicityly use eager loading or using the expand property (which I am not using) so I can't understand why it is having this behaviour.
The why it is happening can be explained here -
Calling ko.toJS on Breeze entity causes slow long running script
To get around it I believe you can over-ride the toString method on the entities that you are trying to serialize so that it ignores related navigation properties.
The reason I didn't mark this as a duplicate is you are using Amplify.js and some other users in the future might as well.
I can assure you Breeze is not loading the related PartyCountries. Either your server api is including them along with the "Lookups" query payload or you queried them into cache earlier and Breeze wired them up to you "Lookups" as they arrived. Based on the code you're showing us, I'd guess that you loaded PartyCountries earlier in the em (EntityManager) lifetime.
You might let Breeze serialize the entities for you. It takes care of circular references, won't include related entities, and also stores the changed-state with the serialized entities.
For example, with Amplify.store:
var exported = em.exportEntities(data.results, false /* exclude metadata */);
// Save data to local storage
store("Lookups", exported ); // Works!
Later you can retrieve and import them:
var imported = store("Lookups");
var importedEntities = em.importEntities(imported);
You can store an entire graph of entities (e.g., an order and its line items) with the help of the Breeze Labs "getEntityGraph" extension. Here it is applied to your model.
// graph of queried `LookupItems` and their in-cache, related 'PartyCountries'
var graph = em.getEntityGraph(data.results, 'PartyCountries');
var exported = em.exportEntities(graph , false /* exclude metadata */);
store("LookupsAndCountries", exported );
Related
I'm trying to reply an http request with a serialized object from my database with EntityFramework.
To achieve that I do the following:
message m = dbContext.messages.FirstOrDefault(e => e.idmessage == id);
return Request.CreateResponse(HttpStatusCode.OK, m);
But the message model has an attribute like this:
public virtual fichero fichero { get; set; }
If I set lazy load to false the fichero property returned is null and everything works fine. If lazy load is set to true I get an exception like this Lazy load true exception:
I guess it may be because I try to return the serialized object before it is fully loaded from the database, but I'm noobie with this so I'm not sure.
How can I wait till it is fully loaded before I return it? Is there any kind of subscribe or similar? Thanks.
wait till it is fully loaded
Lazy loading doesn't mean that navigation properties (such as your fichero fichero) are loaded on the background. It means that they are loaded when the property is requested, for example when you write code like this:
var message = dbContext.Messages.FirstOrDefault(some query); // one database hit
var fichero = message.Fichero; // another database hit
As soon as you access the message.Fichero getter, the dynamically generated proxy class will execute another database query to retrieve the Fichero.
it may be because I try to return the serialized object
Exactly. MVC (or Web API, or whatever) will serialize the object using reflection, and in doing so, it accesses the properties' getters, issuing another database query - but the dbContext is already disposed, so unusable for any further queries.
Possible solutions:
Explicitly load everything you want to return from your API (.Include(m => m.Fichero)
Disable lazy loading, so only your "root" entity will be returned
Map your entities to a viewmodel/DTO, which is the recommended approach anyway
The latter would look like this:
public class MessageViewModel
{
public string Message { get; set; }
public string FicheroFoo { get; set; }
// ...
}
var message = dbContext.Messages.FirstOrDefault(some query);
var model = new MessageViewModel
{
Message = message.Message,
FicheroFoo = message.Fichero.Foo,
// ...
};
return Request.CreateResponse(HttpStatusCode.OK, model);
I'm having an issue trying to convert an object to json. The error is a Newtonsoft.Json.JsonSerializationException:
Self referencing loop detected for property 'Project' with type 'System.Data.Entity.DynamicProxies.Project_F29F70EF89942F6344C5B0A3A7910EF55268857CD0ECC4A484776B2F4394EF79'. Path '[0].Categories[0]'.
The problem is that the object (it's actually a list of objects) has a property which is another object that refers back to the first object:
public partial class Project
{
...
public virtual ICollection<Category> Categories { get; set; }
...
}
public partial class Category
{
...
public virtual Project Project { get; set; }
...
}
This is all fine and dandy as far as Entity Framework is concerned, but to convert this to json would result in an infinite regress, hence the exception.
Here is my code:
public async Task<HttpResponseMessage> GetProjects()
{
var projects = _projectService.GetProjects().ToList();
string jsonString = JsonConvert.SerializeObject(projects); // <-- Offending line
return Request.CreateResponse(HttpStatusCode.OK, jsonString);
}
I've looked online for solutions to this and I found this stackoverflow post:
JSON.NET Error Self referencing loop detected for type
They suggest three solutions, none of which work:
1) Ignore the circular reference:
public async Task<HttpResponseMessage> GetProjects()
{
var projects = _projectService.GetProjects().ToList();
JsonSerializerSettings settings = new JsonSerializerSettings()
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
};
string jsonString = JsonConvert.SerializeObject(projects, settings);
return Request.CreateResponse(HttpStatusCode.OK, jsonString);
}
This resulted in the call to SerializeObject(...) hanging for a bit then throwing a System.OutOfMemoryException (which tells me the circular references were NOT being ignored).
Mind you, the author of this proposed solution at stackoverflow says to set the ignore setting in WebApiConfig.cs but I tried that and it has no effect.
He also says:
"If you want to use this fix in a non-api ASP.NET project, you can add the above line to Global.asax.cs, but first add: var config = GlobalConfiguration.Configuration;"
Mine's a web API with no global file so I shouldn't have to do this.
I also don't want to ignore circular references because I don't want to lose data.
2) Preserve the circular reference:
public async Task<HttpResponseMessage> GetProjects()
{
var projects = _projectService.GetProjects().ToList();
JsonSerializerSettings settings = new JsonSerializerSettings()
{
ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
PreserveReferencesHandling = PreserveReferencesHandling.Objects
};
string jsonString = JsonConvert.SerializeObject(projects, settings);
return Request.CreateResponse(HttpStatusCode.OK, jsonString);
}
This just resulted in the request timing out because it would just hang.
Again, the author says to put this in WebApiConfig.cs, but again this had no effect.
3) Add ignore/preserve reference attributes to the objects and properties:
Ignoring Categories:
public partial class Project
{
...
[JsonIgnore]
public virtual ICollection<Category> Categories { get; set; }
...
}
This has no effect. I hover over the project list and see that it still has categories, and each category still has an instance of the project. I still get the same exception.
Again, even if this worked, I don't want to ignore the categories.
Preserve Categories:
[JsonObject(IsReference = true)]
public partial class Project
{
...
public virtual ICollection<Category> Categories { get; set; }
...
}
Again, same results.
Even if this method worked, the attributes wouldn't be preserved. I'd be doing it on Entity Framework classes which are re-generated automatically every time I recompile. (Is there a way to tell it to set these attributes in the model? Can I set them on the other half of the partial class?)
Alternatively, I'm open to suggestions other than converting to json and sending back in the response. Is there another way to get the data back to the client?
What would be the fix to this problem? Thanks.
Briefly
The best way to fix this problem is to create completely brand-new Models (xxxModel, xxxViewModel, xxxResponse, etc..) on Presentation layer which will be returned to end-users. Than just cast one object to another using AutoMapper or your own custom methods.
Keep your database entities separate from real world!
In detail
There are so many problems that you will encounter:
Disclosure of sensitive data. Your database entity could/will contain sensitive data which end-users shouldn't receive;
Performance issues and waste of RAM and CPU. It would be better to load only those properties that end-users is required, instead all;
Serialization problems. EF entities almost always contain Navigation properties which will be serialized together in case lazy-loading enabled. Imagine dozens related entities, which will be lazy-loaded when your composite root is being serialized. It will cause dozens unexpected requests to database;
Fragility. Any changes related your EF entities will affect on Presentation Layer and on end-users. For instance, in case with API, new added property just extend response, but deleted or renamed will break logic in your customers' application.
There are a lot of other problems, just be careful.
I would recommend not Serializing Entity Framework classes and creating a specific class that only inherits from Object and has only the data you need
I have the following in Entity Framework Core:
public class Book {
public Int32 Id { get; set; }
public String Title { get; set; }
public virtual Theme Theme { get; set; }
}
public class Theme {
public Int32 Id { get; set; }
public String Name { get; set; }
public Byte[] Illustration { get; set; }
public virtual ICollection<Ebook> Ebooks { get; set; }
}
And I have the following linq query:
List<BookModel> books = await context.Books.Select(x =>
new BookModel {
Id = x.Id,
Name = x.Name,
Theme = new ThemeModel {
Id = x.Theme.Id,
Name = x.Theme.Name
}
}).ToListAsync();
I didn't need to include the Theme to make this work, e.g:
List<BookModel> books = await context.Books.Include(x => x.Theme).Select(x => ...
When will I need to use Include in Entity Framework?
UPDATE
I added a column of type Byte[] Illustration in Theme. In my projection I am not including that column so will it be loaded if I use Include? Or is never loaded unless I have it in the projection?
In search for an official answer to your question from Microsoft's side, I found this quote from Diego Vega (part of the Entity Framework and .NET team) made at the aspnet/Announcements github
repository:
A very common issue we see when looking at user LINQ queries is the use of Include() where it is unnecessary and cannot be honored. The typical pattern usually looks something like this:
var pids = context.Orders
.Include(o => o.Product)
.Where(o => o.Product.Name == "Baked Beans")
.Select(o =>o.ProductId)
.ToList();
One might assume that the Include operation here is required because of the reference to the Product navigation property in the Where and Select operations. However, in EF Core, these two things are orthogonal: Include controls which navigation properties are loaded in entities returned in the final results, and our LINQ translator can directly translate expressions involving navigation properties.
You didn't need Include because you were working inside EF context. When you reference Theme inside the anonymous object you are creating, that's not using lazy loading, that's telling EF to do a join.
If you return a list of books and you don't include the themes, then when you try to get the theme you'll notice that it's null. If the EF connection is open and you have lazy loading, it will go to the DB and grab it for you. But, if the connection is not opened, then you have to get it explicitely.
On the other hand, if you use Include, you get the data right away. Under the hood it's gonna do a JOIN to the necessary table and get the data right there.
You can check the SQL query that EF is generating for you and that's gonna make things clearer for you. You'll see only one SQL query.
If you Include a child, it is loaded as part of the original query, which makes it larger.
If you don't Include or reference the child in some other way in the query, the initial resultset is smaller, but each child you later reference will lazy load through a new request to the database.
If you loop through 1000 users in one request and then ask for their 10 photos each, you will make 1001 database requests if you don't Include the child...
Also, lazy loading requires the context hasn't been disposed. Always an unpleasant surprise when you pass an Entity to a view for UI rendering for example.
update
Try this for example and see it fail:
var book = await context.Books.First();
var theme = book.Theme;
Then try this:
var book = await context.Books.Include(b => b.Theme).First();
var theme = book.Theme;
I am working on an N-tier application consisting of a UI layer (MVC), a Business Layer, a Domain layer (for the models) and a DAL for repositories and the EF DbContext.
I'm a bit confused about the inner workings of Entity Framework when updating the properties of an existing object and I'm looking for a good way to validate an object before updating its values in the database.
I have the following model:
public class BlogPost
{
public int BlogPostId { get; set; }
[Required]
public String Title { get; set; }
[Required]
public String Description { get; set; }
[Required]
public DateTime DateTime { get; set; }
public byte[] Image { get; set; }
}
I have the following methods in my manager in BL:
public BlogPost AddBlogPost(string title, string description, byte[] image = null)
{
BlogPost blogPost = new BlogPost()
{
Title = title,
Description = description,
DateTime = DateTime.Now
};
Validate(blogPost);
moduleRepository.CreateBlogPost(blogPost);
return blogPost;
}
public BlogPost ChangeBlogPost(BlogPost blogPost)
{
moduleRepository.UpdateBlogPost(blogPost);
return blogPost;
}
And I have the following methods in my DAL:
public BlogPost CreateBlogPost(BlogPost b)
{
b = context.BlogPosts.Add(b);
context.SaveChanges();
return b;
}
public BlogPost UpdateBlogPost(BlogPost b)
{
context.Entry(b).State = EntityState.Modified;
context.SaveChanges();
return b;
}
My question now is: what's a good way to check that the model is valid before actually trying to change its values in the database?
I was thinking something like this:
public BlogPost ChangeBlogPost(BlogPost blogPost)
{
// STEP 1: put the updated data in a new object
BlogPost updatedBlogPost = new BlogPost()
{
Title = blogPost.Title,
Description = blogPost.Description,
Image = blogPost.Image,
DateTime = blogPost.DateTime
};
// STEP 2: check if the model is valid
this.Validate(updatedBlogPost);
// STEP 3: read the existing blog post with that ID and change the properties
BlogPost b = moduleRepository.ReadBlogPost(blogPost.BlogPostId);
b.Title = blogPost.Title;
b.Description = blogPost.Description;
b.Image = blogPost.Image;
b.DateTime = blogPost.DateTime;
moduleRepository.UpdateBlogPost(blogPost);
return blogPost;
}
EDIT: I figured it's maybe better to just accept primitive types as parameter in the above method instead of the object.
I have a feeling that's too much work for a simple update, but I couldn't find anything else on the internet.
It's probably also worth noting that I'm using a singleton for the DbContext so I have to make sure Entity Framework doesn't change the values in the database before checking that those values are valid (since another call to the context by another class can cause SaveChanges()).
I know singleton on a DbContext is bad practice, but I saw no other option to avoid countless exceptions when working with multiple repositories and entities being tracked by multiple context instances.
PS: I also read about change tracking in Entity Framework but I'm not 100% sure how this will affect what I'm trying to do.
All suggestions and explanations are welcome.
Thanks in advance.
You would check ModelState.IsValid. There are a lot of validation mechanisms built into MVC that you can take advantage of. Built in attributes such as [Required] that you reference above, custom validators, making your business class implement IValidatableObject, overriding EF SaveChanges() to name a few. This article is a good start: https://msdn.microsoft.com/en-us/data/gg193959.aspx
Ok so I kinda answered my own question while doing some research and testing with some dummy data. I thought that when a property changed in MVC as a result of an Edit view, EF also tracked it and changed it in the database.
I figured out that's not how model binding works and realized after some fooling around that model binding actually creates a new object (instead of editing the properties of a dynamic proxy).
I guess I can now just validate the model and then just update the one with the same primary key in the database.
I am using DBContext and have two classes whose properties are all virtual. I can see in the debugger that I am getting a proxy object when I query the context. However, a collection property is still null when I try to add to it. I thought that the proxy would ensure that collection is initialized.
Because my Poco object can be used outside of its data context, I added a check for the collection being null in the constructor and create it if necessary:
public class DanceStyle
{
public DanceStyle()
{
if (DanceEvents == null)
{
DanceEvents = new Collection<DanceEvent>();
}
}
...
public virtual ICollection<DanceEvent> DanceEvents { get; set; }
}
That works outside the data context but if I retrieve an object using a query, although the test is true, when I try to set it, I get following exception: 'The property 'DanceEvents' on type 'DanceStyle_B6089AE40D178593955F1328A70EAA3D8F0F01DDE9F9FBD615F60A34F9178B94' cannot be set because the collection is already set to an EntityCollection.'
I can see it is null and I cannot add to it, but neither can I set it to a collection because the proxy says it is already set. Therefore I cannot use it. I'm confused.
Here is the DanceEvent class:
public class DanceEvent
{
public DanceEvent()
{
if (DanceStyles == null)
{
DanceStyles = new Collection<DanceStyle>();
}
}
...
public virtual ICollection<DanceStyle> DanceStyles { get; set; }
}
I have omitted the other value-type properties from the code above. I have no other mappings for those classes in the context class.
As you correctly observed in the answer to your own question, removing the "virtual" keyword from the collection properties works around the problem, by preventing the Entity Framework from creating a change tracking proxy. However, this is not a solution for many people, because change tracking proxies can be really convenient and can help prevent issues when you forget to detect changes at the right places in your code.
A better approach would be to modify your POCO classes, so that they instantiate the collection properties in their get accessor, rather than in the constructor. Here's your POCO class, modified to allow change tracking proxy creation:
public class DanceEvent
{
private ICollection<DanceStyle> _danceStyles;
public virtual ICollection<DanceStyle> DanceStyles
{
get { return _danceStyles ?? (_danceStyles = new Collection<DanceStyle>()); }
protected set { _danceStyles = value; }
}
}
In the above code the collection property is no longer automatic, but rather has a backing field. It's better if you leave the setter protected, preventing any code (other than the proxy) from subsequently modifying these properties. You will notice that the constructor was no longer necessary and was removed.
I found the solution to this problem here: Code First adding to collections? How to use Code First with repositories?
I removed 'virtual' from all properties except collections and lazy loaded objects, that is, all native types.
But I still don't understand how you can end up with the situation where you have a null collection that you cannot use and have no way to set it to a valid collection.
I also found this answer from Rowan Miller on an MSDN forum
Hi,
If you make all your properties virtual then EF will generate proxy classes at runtime that derives from your POCO classed, these proxies allow EF to find out about changes in real time rather than having to capture the original values of your object and then scan for changes when you save (this is obviously has performance and memory usage benefits but the difference will be negligible unless you have a large number of entities loaded into memory). These are known as 'change tracking proxies', if you make your navigation properties virtual then a proxy is still generated but it is much simpler and just includes some logic to perform lazy loading when you access a navigation property.
Because your original code was generating change tracking proxies, EF was replacing your collection property with a special collection type to help it find out about changes. Because you try and set the collection back to a simple list in the constructor you are getting the exception.
Unless you are seeing performance issues I would follow Terrence's suggestion and just remove 'virtual' from your non-navigation properties.
~Rowan
So it appears that I only have the problem with a full 'change tracking proxy' if all my properties are virtual. But given that, why can I still not use the virtual property on the change tracking proxy? This code blows up on line three because ds2.DanceEvents is null and cannot be set in the constructor:
DanceStyle ds2 = ctx.DanceStyles.Where(ds => ds.DanceStyleId == 1).Single();
DanceEvent evt = CreateDanceEvent();
ds2.DanceEvents.Add(evt);
I'm still confused, even though my code is now working because of the fix above.
Old question...
Poco class:
public partial class MyPOCO
{
public MyPOCO()
{
this.MyPocoSub = new HashSet<MyPocoSub>();
}
//VIRTUAL
public virtual ICollection<MyPocoSub> MyPocoSub { get; set; }
}
and proxy code:
public override ICollection<MyPocoSubSet> MyPocoSubSets
{
get
{
ICollection<MyPocoSubSet> myPocoSubSets = base.MyPocoSubSets;
if (!this.ef_proxy_interceptorForMyPocoSubSets(this, myPocoSubSets))
{
return base.MyPocoSubSets;
}
return myPocoSubSets;
}
set
{
if (value != this.RelationshipManager.GetRelatedEnd("WindowsFormsApplication.Models.MyPocoSubSet_MyPOCO", "MyPocoSubSet_MyPOCO_Source"))
{
// EXCEPTION
throw new InvalidOperationException("The property 'MyPocoSubSets' on type 'MyPOCO_A78FCE6C6A890855C68B368B750864E3136B589F9023C7B1D90BF7C83FD291AC' cannot be set because the collection is already set to an EntityCollection.");
}
base.MyPocoSubSets = value;
}
}
As you can see that exception raised in proxy class in ExtityFramework 5. This means that behavior still exist.