I'm trying to reply an http request with a serialized object from my database with EntityFramework.
To achieve that I do the following:
message m = dbContext.messages.FirstOrDefault(e => e.idmessage == id);
return Request.CreateResponse(HttpStatusCode.OK, m);
But the message model has an attribute like this:
public virtual fichero fichero { get; set; }
If I set lazy load to false the fichero property returned is null and everything works fine. If lazy load is set to true I get an exception like this Lazy load true exception:
I guess it may be because I try to return the serialized object before it is fully loaded from the database, but I'm noobie with this so I'm not sure.
How can I wait till it is fully loaded before I return it? Is there any kind of subscribe or similar? Thanks.
wait till it is fully loaded
Lazy loading doesn't mean that navigation properties (such as your fichero fichero) are loaded on the background. It means that they are loaded when the property is requested, for example when you write code like this:
var message = dbContext.Messages.FirstOrDefault(some query); // one database hit
var fichero = message.Fichero; // another database hit
As soon as you access the message.Fichero getter, the dynamically generated proxy class will execute another database query to retrieve the Fichero.
it may be because I try to return the serialized object
Exactly. MVC (or Web API, or whatever) will serialize the object using reflection, and in doing so, it accesses the properties' getters, issuing another database query - but the dbContext is already disposed, so unusable for any further queries.
Possible solutions:
Explicitly load everything you want to return from your API (.Include(m => m.Fichero)
Disable lazy loading, so only your "root" entity will be returned
Map your entities to a viewmodel/DTO, which is the recommended approach anyway
The latter would look like this:
public class MessageViewModel
{
public string Message { get; set; }
public string FicheroFoo { get; set; }
// ...
}
var message = dbContext.Messages.FirstOrDefault(some query);
var model = new MessageViewModel
{
Message = message.Message,
FicheroFoo = message.Fichero.Foo,
// ...
};
return Request.CreateResponse(HttpStatusCode.OK, model);
Related
I'm having an issue trying to convert an object to json. The error is a Newtonsoft.Json.JsonSerializationException:
Self referencing loop detected for property 'Project' with type 'System.Data.Entity.DynamicProxies.Project_F29F70EF89942F6344C5B0A3A7910EF55268857CD0ECC4A484776B2F4394EF79'. Path '[0].Categories[0]'.
The problem is that the object (it's actually a list of objects) has a property which is another object that refers back to the first object:
public partial class Project
{
...
public virtual ICollection<Category> Categories { get; set; }
...
}
public partial class Category
{
...
public virtual Project Project { get; set; }
...
}
This is all fine and dandy as far as Entity Framework is concerned, but to convert this to json would result in an infinite regress, hence the exception.
Here is my code:
public async Task<HttpResponseMessage> GetProjects()
{
var projects = _projectService.GetProjects().ToList();
string jsonString = JsonConvert.SerializeObject(projects); // <-- Offending line
return Request.CreateResponse(HttpStatusCode.OK, jsonString);
}
I've looked online for solutions to this and I found this stackoverflow post:
JSON.NET Error Self referencing loop detected for type
They suggest three solutions, none of which work:
1) Ignore the circular reference:
public async Task<HttpResponseMessage> GetProjects()
{
var projects = _projectService.GetProjects().ToList();
JsonSerializerSettings settings = new JsonSerializerSettings()
{
ReferenceLoopHandling = ReferenceLoopHandling.Ignore
};
string jsonString = JsonConvert.SerializeObject(projects, settings);
return Request.CreateResponse(HttpStatusCode.OK, jsonString);
}
This resulted in the call to SerializeObject(...) hanging for a bit then throwing a System.OutOfMemoryException (which tells me the circular references were NOT being ignored).
Mind you, the author of this proposed solution at stackoverflow says to set the ignore setting in WebApiConfig.cs but I tried that and it has no effect.
He also says:
"If you want to use this fix in a non-api ASP.NET project, you can add the above line to Global.asax.cs, but first add: var config = GlobalConfiguration.Configuration;"
Mine's a web API with no global file so I shouldn't have to do this.
I also don't want to ignore circular references because I don't want to lose data.
2) Preserve the circular reference:
public async Task<HttpResponseMessage> GetProjects()
{
var projects = _projectService.GetProjects().ToList();
JsonSerializerSettings settings = new JsonSerializerSettings()
{
ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
PreserveReferencesHandling = PreserveReferencesHandling.Objects
};
string jsonString = JsonConvert.SerializeObject(projects, settings);
return Request.CreateResponse(HttpStatusCode.OK, jsonString);
}
This just resulted in the request timing out because it would just hang.
Again, the author says to put this in WebApiConfig.cs, but again this had no effect.
3) Add ignore/preserve reference attributes to the objects and properties:
Ignoring Categories:
public partial class Project
{
...
[JsonIgnore]
public virtual ICollection<Category> Categories { get; set; }
...
}
This has no effect. I hover over the project list and see that it still has categories, and each category still has an instance of the project. I still get the same exception.
Again, even if this worked, I don't want to ignore the categories.
Preserve Categories:
[JsonObject(IsReference = true)]
public partial class Project
{
...
public virtual ICollection<Category> Categories { get; set; }
...
}
Again, same results.
Even if this method worked, the attributes wouldn't be preserved. I'd be doing it on Entity Framework classes which are re-generated automatically every time I recompile. (Is there a way to tell it to set these attributes in the model? Can I set them on the other half of the partial class?)
Alternatively, I'm open to suggestions other than converting to json and sending back in the response. Is there another way to get the data back to the client?
What would be the fix to this problem? Thanks.
Briefly
The best way to fix this problem is to create completely brand-new Models (xxxModel, xxxViewModel, xxxResponse, etc..) on Presentation layer which will be returned to end-users. Than just cast one object to another using AutoMapper or your own custom methods.
Keep your database entities separate from real world!
In detail
There are so many problems that you will encounter:
Disclosure of sensitive data. Your database entity could/will contain sensitive data which end-users shouldn't receive;
Performance issues and waste of RAM and CPU. It would be better to load only those properties that end-users is required, instead all;
Serialization problems. EF entities almost always contain Navigation properties which will be serialized together in case lazy-loading enabled. Imagine dozens related entities, which will be lazy-loaded when your composite root is being serialized. It will cause dozens unexpected requests to database;
Fragility. Any changes related your EF entities will affect on Presentation Layer and on end-users. For instance, in case with API, new added property just extend response, but deleted or renamed will break logic in your customers' application.
There are a lot of other problems, just be careful.
I would recommend not Serializing Entity Framework classes and creating a specific class that only inherits from Object and has only the data you need
I have a method that receives an IEnumerable<Guid> of IDs to objects I want to delete. One suggested method is as follows
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
DataContext.Attach(tempInstance); // Exception here
DataContext.Remove(tempInstance);
}
This works fine if the objects aren't already loaded into memory. But my problem is that when they are already loaded then the Attach method throws an InvalidOperationException - The instance of entity type 'MyEntity' cannot be tracked because another instance with the key value 'Id:...' is already being tracked. The same happens if I use DataContext.Remove without calling Attach.
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
DataContext.Remove(tempInstance); // Exception here
}
I don't want to use DataContext.Find to grab the instance of an already loaded object because that will load the object into memory if it isn't already loaded.
I cannot use DataContext.ChangeTracker to find already loaded objects because only objects with modified state appear in there and my objects might be loaded and unmodified.
The following approach throws the same InvalidOperationException when setting EntityEntry.State, even when I override GetHashCode and Equals on MyEntity to ensure dictionary lookups see them as the same object.
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
EntityEntry entry = DataContext.Entry(tempInstance);
entry.State == EntityState.Deleted; // Exception here
}
The only way so far I have found that I can achieve deleting objects by ID without knowing if the object is the following:
foreach(Guid id in ids)
{
var tempInstance = new MyEntity { Id = id };
try
{
DataContext.Attach(tempInstance); // Exception here
}
catch (InvalidOperationException)
{
}
DataContext.Remove(tempInstance);
}
It's odd that I am able to call DataContext.Remove(tempInstance) without error after experiencing an exception trying to Attach it, but at this point it does work without an exception and also deletes the correct rows from the database when DataContext.SaveChanges is executed.
I don't like catching the exception. Is there a "good" way of achieving what I want?
Note: If the class has a self-reference then you need to load the objects into memory so EntityFrameworkCore can determine in which order to delete the objects.
Strangely, although this is a quite common exception in EF6 and EF Core, neither of them expose publicly a method for programmatically detecting the already tracked entity instance with the same key. Note that overriding GetHashCode and Equals doesn't help since EF is using reference equality for tracking entity instances.
Of course it can be obtained from the DbSet<T>.Local property, but it would not be as efficient as the internal EF mechanism used by Find and the methods throwing the aforementioned exception. All we need is the first part of the Find method and returning null when not found instead of loading from the database.
Luckily, for EF Core the method that we need can be implemented relatively easily by using some of the EF Core internals (under the standard This API supports the Entity Framework Core infrastructure and is not intended to be used directly from your code. This API may change or be removed in future releases. policy). Here is the sample implementation, tested on EF Core 2.0.1:
using Microsoft.EntityFrameworkCore.Internal;
namespace Microsoft.EntityFrameworkCore
{
public static partial class CustomExtensions
{
public static TEntity FindTracked<TEntity>(this DbContext context, params object[] keyValues)
where TEntity : class
{
var entityType = context.Model.FindEntityType(typeof(TEntity));
var key = entityType.FindPrimaryKey();
var stateManager = context.GetDependencies().StateManager;
var entry = stateManager.TryGetEntry(key, keyValues);
return entry?.Entity as TEntity;
}
}
}
Now you can use simply:
foreach (var id in ids)
DataContext.Remove(DataContext.FindTracked<MyEntity>(id) ?? new MyEntity { Id = id }));
or
DataContext.RemoveRange(ids.Select(id =>
DataContext.FindTracked<MyEntity>(id) ?? new MyEntity { Id = id }));
I have the following in Entity Framework Core:
public class Book {
public Int32 Id { get; set; }
public String Title { get; set; }
public virtual Theme Theme { get; set; }
}
public class Theme {
public Int32 Id { get; set; }
public String Name { get; set; }
public Byte[] Illustration { get; set; }
public virtual ICollection<Ebook> Ebooks { get; set; }
}
And I have the following linq query:
List<BookModel> books = await context.Books.Select(x =>
new BookModel {
Id = x.Id,
Name = x.Name,
Theme = new ThemeModel {
Id = x.Theme.Id,
Name = x.Theme.Name
}
}).ToListAsync();
I didn't need to include the Theme to make this work, e.g:
List<BookModel> books = await context.Books.Include(x => x.Theme).Select(x => ...
When will I need to use Include in Entity Framework?
UPDATE
I added a column of type Byte[] Illustration in Theme. In my projection I am not including that column so will it be loaded if I use Include? Or is never loaded unless I have it in the projection?
In search for an official answer to your question from Microsoft's side, I found this quote from Diego Vega (part of the Entity Framework and .NET team) made at the aspnet/Announcements github
repository:
A very common issue we see when looking at user LINQ queries is the use of Include() where it is unnecessary and cannot be honored. The typical pattern usually looks something like this:
var pids = context.Orders
.Include(o => o.Product)
.Where(o => o.Product.Name == "Baked Beans")
.Select(o =>o.ProductId)
.ToList();
One might assume that the Include operation here is required because of the reference to the Product navigation property in the Where and Select operations. However, in EF Core, these two things are orthogonal: Include controls which navigation properties are loaded in entities returned in the final results, and our LINQ translator can directly translate expressions involving navigation properties.
You didn't need Include because you were working inside EF context. When you reference Theme inside the anonymous object you are creating, that's not using lazy loading, that's telling EF to do a join.
If you return a list of books and you don't include the themes, then when you try to get the theme you'll notice that it's null. If the EF connection is open and you have lazy loading, it will go to the DB and grab it for you. But, if the connection is not opened, then you have to get it explicitely.
On the other hand, if you use Include, you get the data right away. Under the hood it's gonna do a JOIN to the necessary table and get the data right there.
You can check the SQL query that EF is generating for you and that's gonna make things clearer for you. You'll see only one SQL query.
If you Include a child, it is loaded as part of the original query, which makes it larger.
If you don't Include or reference the child in some other way in the query, the initial resultset is smaller, but each child you later reference will lazy load through a new request to the database.
If you loop through 1000 users in one request and then ask for their 10 photos each, you will make 1001 database requests if you don't Include the child...
Also, lazy loading requires the context hasn't been disposed. Always an unpleasant surprise when you pass an Entity to a view for UI rendering for example.
update
Try this for example and see it fail:
var book = await context.Books.First();
var theme = book.Theme;
Then try this:
var book = await context.Books.Include(b => b.Theme).First();
var theme = book.Theme;
I am using Entity Framework 6.1 with breeze 1.4.11. I am trying to save some data in local storage. My Breeze controller:
[HttpGet]
public object Lookups()
{
return new
{
lookupItems = _contextProvider.Context.LookupItem
};
}
My EF settings:
this.Configuration.ProxyCreationEnabled = false;
this.Configuration.LazyLoadingEnabled = false;
The class I am trying to load:
public class LookupItem : BaseEntityInt
{
public int LookupTypeId { get; set; }
public string Description { get; set; }
public int Ordinal { get; set; }
public virtual ICollection<Party> PartyCountries { get; set; }
}
My breeze client code:
_lookupsQuery = new breeze.EntityQuery.from("Lookups"),
em.executeQuery(_lookupsQuery).then(function (data) {
_lookups = data.results;
// Save data to local storage
store.setItem("Lookups", _lookups); // Fails here
});
The problem is when I save it in local storage, store uses amplifyJs to persist to local storage. AmplifyJs calls JSON.stringify on the persisted object. In the console of the browser I get the following error:
Error: cyclic object value
This is because breeze is automatically loading the navigation property of lookupitem 'partycountries' which itself links back to lookupitem. Anyone any idea why it is doing this? My understanding was you had to explicityly use eager loading or using the expand property (which I am not using) so I can't understand why it is having this behaviour.
The why it is happening can be explained here -
Calling ko.toJS on Breeze entity causes slow long running script
To get around it I believe you can over-ride the toString method on the entities that you are trying to serialize so that it ignores related navigation properties.
The reason I didn't mark this as a duplicate is you are using Amplify.js and some other users in the future might as well.
I can assure you Breeze is not loading the related PartyCountries. Either your server api is including them along with the "Lookups" query payload or you queried them into cache earlier and Breeze wired them up to you "Lookups" as they arrived. Based on the code you're showing us, I'd guess that you loaded PartyCountries earlier in the em (EntityManager) lifetime.
You might let Breeze serialize the entities for you. It takes care of circular references, won't include related entities, and also stores the changed-state with the serialized entities.
For example, with Amplify.store:
var exported = em.exportEntities(data.results, false /* exclude metadata */);
// Save data to local storage
store("Lookups", exported ); // Works!
Later you can retrieve and import them:
var imported = store("Lookups");
var importedEntities = em.importEntities(imported);
You can store an entire graph of entities (e.g., an order and its line items) with the help of the Breeze Labs "getEntityGraph" extension. Here it is applied to your model.
// graph of queried `LookupItems` and their in-cache, related 'PartyCountries'
var graph = em.getEntityGraph(data.results, 'PartyCountries');
var exported = em.exportEntities(graph , false /* exclude metadata */);
store("LookupsAndCountries", exported );
Simplest example of this, I get a collection and try to output it via Web API:
// GET api/items
public IEnumerable<Item> Get()
{
return MyContext.Items.ToList();
}
And I get the error:
Object of type
'System.Data.Objects.ObjectQuery`1[Dcip.Ams.BO.EquipmentWarranty]'
cannot be converted to type
'System.Data.Entity.DbSet`1[Dcip.Ams.BO.EquipmentWarranty]'
This is a pretty common error to do with the new proxies, and I know that I can fix it by setting:
MyContext.Configuration.ProxyCreationEnabled = false;
But that defeats the purpose of a lot of what I am trying to do. Is there a better way?
I would suggest Disable Proxy Creation only in the place where you don't need or is causing you trouble. You don't have to disable it globally you can just disable the current DB context via code...
[HttpGet]
[WithDbContextApi]
public HttpResponseMessage Get(int take = 10, int skip = 0)
{
CurrentDbContext.Configuration.ProxyCreationEnabled = false;
var lista = CurrentDbContext.PaymentTypes
.OrderByDescending(x => x.Id)
.Skip(skip)
.Take(take)
.ToList();
var count = CurrentDbContext.PaymentTypes.Count();
return Request.CreateResponse(HttpStatusCode.OK, new { PaymentTypes = lista, TotalCount = count });
}
Here I only disabled the ProxyCreation in this method, because for every request there is a new DBContext created and therefore I only disabled the ProxyCreation for this case .
Hope it helps
if you have navigation properties and you do not want make them non virtual, you should using JSON.NET and change configuration in App_Start to using JSON not XML!
after install JSON.NET From NuGet, insert this code in WebApiConfig.cs in Register method
var json = config.Formatters.JsonFormatter;
json.SerializerSettings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects;
config.Formatters.Remove(config.Formatters.XmlFormatter);
If you have navigation properties make them non virtual. Mapping will still work but it prevents the creation of Dynamic Proxy entities which cannot be serialized.]
Not having lazy loading is fine in a WebApi as you don't have a persistent connection and you ran a .ToList() anyway.
I just disabled proxy classes on a per needed basis:
// GET: ALL Employee
public IEnumerable<DimEmployee> Get()
{
using (AdventureWorks_MBDEV_DW2008Entities entities = new AdventureWorks_MBDEV_DW2008Entities())
{
entities.Configuration.ProxyCreationEnabled = false;
return entities.DimEmployees.ToList();
}
}
Add the following code in Application_Start function of Global.asax.cs:
GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings
.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore;
GlobalConfiguration.Configuration.Formatters
.Remove(GlobalConfiguration.Configuration.Formatters.XmlFormatter);
This instruct the API to serialize every response into JSON and remove XML responses.
In my case the object being returned had a property within it with a type that did not have an argumentless/default constructor. By adding a zero-argument constructor to that type the object could be serialized successfully.
I had the same problem and my DTO was missing an parameter less constructor.
public UserVM() { }
public UserVM(User U)
{
LoginId = U.LoginId;
GroupName = U.GroupName;
}
First constructor was missing.
I got this error message and it turns out the problem was that I had accidentally set my class to use the same serialized property name for two properties:
public class ResultDto
{
//...
[JsonProperty(PropertyName="DataCheckedBy")]
public string ActualAssociations { get; set; }
[JsonProperty(PropertyName="DataCheckedBy")]
public string ExpectedAssociations { get; set; }
//...
}
If you're getting this error and you aren't sending entities directly through your API, copy the class that's failing to serialize to LINQPad and just call JsonConvert.SerializeObject() on it and it should give you a better error message than this crap. As soon as I tried this it gave me the following error message: A member with the name 'DataCheckedBy' already exists on 'UserQuery+ResultDto'. Use the JsonPropertyAttribute to specify another name.
After disable Proxy Creation, use eager loading (Include()) to load the proxy object.
In my Project EntityCollection returned from the WebApi action method.
Configuration.ProxyCreationEnabled = false not applicable. I have tried the below approach it is working fine for me.
Control Panel.
2.Turn on Windows Features on or off
Choose Internet Information Service
Check all the World Wide Web Components it would be better to check all the components in IIS.
Install the components.
Go to (IIS) type inetmgr in command prompt.
select the published code in the Virtual directory.
Convert into application
Browse it the application.
The answer by #Mahdi perfectly fixes the issue for me, however what I noticed is that if my Newtonsoft.JSON is 11.0 version then it doesn't fix the issue, but the moment I update Newtonsoft.JSON to latest 13.0 it starts working.