Here is my situation. I use Entity Framework 4 with the Web API
The structure of my code is quite simple, I have the Service layer where all my rest API is organized, I have my Business logic layer where I have business controllers to manage Transactions between the rest calls and the data layer. Finally, I have a data layer with generic repositories and a DAO to access the whole thing.
In my Business controllers, I use using to inject a non transactionnal (read only methods) OR a transactional (CRUD methods) DbContext.
When returning values to my REST API, I parse it into JSON.
The problem is that I keep having this exception: Newtonsoft.Json.JsonSerializationException
I return my entities / collections / lists outside of my using{} statement, which I think EF does not like by default.
In debug mode, sometimes, I will manage to retrieve all data, but not all the time. Since my entities come from a query within a DbContext, I think that the behavior is to remove loaded sub-properties after the context has been disposed of.
Fact is, I want to keep my structure as is, and I was wondering the following:
Is there a way of returning complete (not lazy-loaded) entities after leaving the using{} statement?
Thanks a lot
I actually read more about Entity Frameworks behavior. What I get is actually standard for EF. I have to force the context to Load() my refered entities in order to get them after leaving the context.
Related
I have a working project with the following layers:
DataAccess - The project hits multiple DBs and web services. This layer defines the interfaces to each of them. It exposes the native types for each source (EF types for DBs, SOAP-defined types for web services, etc.). Let's say it exposes an EFProject and SoapProject.
Repository - This layer stiches results from the various sources to form a single entity, and exposes it. Let's call this ModelProject
Service - Adds REST attributes to to the entity (action links, etc.). This exposes a ProjectDTO.
WebApi - The controller spits out the ProjectDTO directly.
I'm trying to implement OData, specifically to page the results of very large queries. I've read a lot of examples, but they all seem to expose the source objects directly, and then map them to the final DTOs in the controller.
I would like to somehow push the ODataQueryOptions down to the Repository. This would allow me to keep the existing structure, and pass the query logic down to SQL. I understand that, because the ODataQueryOptions reference the ProjectDTO type, they can't be applied until an object of that type is available. Is there a way to "translate" the ODataQueryOptions from one type to another? Is there another way of doing this that I'm not aware of?
It is able to change the ODataQueryOptions, and such actions in a controller are for you to handle the options yourself:
public IQueryable Get(ODataQueryOptions queryOptions)
public IQueryable Get(int key, ODataQueryOptions queryOptions)
Here is a sample about this:
https://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v4/ODataQueryableSample/Controllers/OrdersController.cs .
For your reference, the source code of ODataQueryOptions is: https://aspnetwebstack.codeplex.com/SourceControl/latest#src/System.Web.OData/OData/Query/ODataQueryOptions.cs
Is there a way to "translate" the ODataQueryOptions from one type to another? Is there another way of doing this that I'm not aware of?
There is a way to actually translate the IQueryable<DomainModel> to IQueryable<DtoModel>.
I've done something similar in the past by leveraging AutoMapper's projection functionality. By calling the Project<TSource>/To<TTarget> methods, you can change an IQueryable that points to your domain models to another IQueryable that targets the Dto models, without actually executing it.
This means that you can now perform any OData operations on the DTO level and they will transfer through projection to the DAL layer into EntityFramework and SQL. In a scenario like this, there shouldn't be any need to manually handle the query logic so you can just use [EnableQuery] on the API route and let OData do its thing on the resulting IQueryable<DtoModel>.
I used this very successfully in one of the projects I worked on: as long as you rely just on AutoMapper projection to convert the types, it should work fine.
Granted, you can't do a lot of fancy mapping that way. The project methods will not be able to apply all kinds of mappings that you create, so I recommend checking the documentation on that front.
You also have to keep in mind that the original IQueryable needs to be exposed outside of the repository layer for this to work properly, otherwise the query will be executed too early. Some people will find that a boundary violation and will advocate for materializing the query inside the repository layer, but I don't have an issue with that particular aspect.
I'm refactoring my ASP.NET MVC 4 WebAPI project for performance optimization reasons.
Within my controller code, I'm searching for entities in a context (DbContext, EF6). There are a few thousands of such entities, new ones are added on an hourly basis (i.e. "slowly"), they are rarely deleted (and I don't care if deleted entities are still found on the context's cache!) and are never modified.
After reading the answers to this question, to this one and a few more discussions, I'm still not sure it's a bad idea to use a single static DbContext for the purpose described above - a DbContext which never updates the database.
Performance-wise, I'm not worried about the instantiation cost, but rather about the uselessness of caching requested entities if the DbContext is created for each request. I'm also using a 2nd level caching, which makes the persistence of the context even more acute.
My questions are:
1. Regardless of the specific implementation, is a "static" DbContext a valid solution in my case?
2. If so, what would be the most appropriate way of implementing such a DbContext?
3. Should I periodically "flush" the context to clear the cache in order to prevent if from growing too big?
DbContext caches entity instances when you get/query the data. It ensures different queries that return the same data map to the same entity (based on type and id). Otherwise, if you modify the same entity in different object instances, the context would not know which one has the correct data. Therefore a static DbContext would blow up over time until the process crashes.
DbContexts should be short lived. Request.Properties is a good place to save it in Web API (maps to HttpContext.Items in IIS).
I am using the Service Layer --> Repository --> Entity Framework (Code-First) w/POCO objects approach, and I am having a hard time with updating entities.
I am using AutoMapper to map my Domain Objects to my View Models and that works good for getting the data, no how do I get that changes back into the database?
Using pure POCO objects, I would assume that there is no sort of change tracking, so I see my only option is to handle it myself. Do you just make sure that your View Models have the EXACT same properties as your Domain Objects? What if I just change a field or two on the View Model? Won't the rest of the fields on the Domain Object get overwritten in the database with default values?
With that said, what is the best approach?
Thanks!
Edit
So what I am stumbling on is this, lets take for example a simple Customer:
1) The Controller has a service, CustomerService, that calls the services GetCustmoerByID method.
2) The Service calls into the CustomerRepository and retrieves the Customer object.
3) Controller uses AutoMapper to map the Customer to the ViewModel.
4) Controller hands the model to the View. Everything is great!
Now in the view you do some modifications of the customer and post it back to the controller to persist the changes to the database.
I would assume at this point the object is detached. So should the model have the EXACT same properties as the Customer object? And do you have to make hidden fields for each item that you do not want to show, so they can persist back?
How do you handle saving the object back to the database? What happens if your view/model only deals with a couple of the fields on the object?
If you're using EF Code First, i.e: the DbContext API, then you still do have change tracking which is taken care of by your context class.
after making changes to your objects, all you have to do is call SaveChanges() on your context and that will persist the changes to your database.
EDIT:
Since you are creating a "copy" of the entity using AutoMapper, then it's no longer attached to your context.
I guess what you could do is something similar to what you would in ASP.NET MVC (with UpdateModel). You can get the original entity from your context, take your ViewModel (which may contain changed properties) and update the old entity, either manually (just modified properties), or using AutoMapper. And then persist the changes using context.SaveChanges().
Another solution would be to send the model entity as [part of] the ViewModel. This way, you'll have your entity attached to the container and change tracking will still work.
Hope this helps :)
You are absolutely right that with a detached object you are responsible for informing the context about changes in your detached entity.
The basic approach is just set the entity as modified. This works for scalar and complex properties but it doesn't work for navigation properties (except FK relations) - for further reading about problems with navigation properties check this answer (it is related to EFv4 and ObjectContext API but same problems are with DbContext API). The disadvantage of this approach is that all fields in DB will be modified. If you just want to modify single field you still have to correctly fill others or your database record will be corrupted.
There is a way to explicitly define which fields have changed. You will set the modified state per property instead of whole entity. It is little bit harder to solve this on generic approach but I tried to show some way for EFv4 and for EFv4.1.
I agree with #AbdouMoumen that it's much simpler to use the model entities at the view level. The service layer should provide an API to persist those entities in the data store (db). The service layer shouldn't dumbly duplicate the repository lawyer (ie: Save(entity) for every entity) but rather provide a high level save for an aggregate of entities. For instance, you could have a Save(order) in the service layer which results in updating more basic entities like inventory, customer, account.
I'm using DataNucleus as a JPA implementation to store my classes in my web application. I use a set of converters which all have toDTO() and fromDTO().
My issue is, that I want to avoid the whole DB being sent over the wire:
If I lazy load, the converter will try to access ALL the fields, and load then (resulting in very eager loading).
If I don't lazy load, I'll get a huge part of the DB, since user contains groups, and groups contains users, and so on.
Is there a way to explicitly load some fields and leave the others as NULL in my loaded class?
I've tried the DataNucleus docs with no luck.
Your DTOs are probably too fine-grained. i.e. dont plan to have a DTO per JPA entity. If you have to use DTOs then make them more coarse grained and construct them manually.
Recently we have had the whole "to DTO or not to DTO, that is the question" discussion AGAIN. The requirement for them (especially in the context of a JPA app) is often no longer there, but one of the arguments FOR DTOs tends to be that the view has coarser data requirements.
To only load the data you really require, you would need to use a custom select clause containing only these elements that you are about to use for your DTOs. I know how painful this is, especially when it involves joins, which is why I created Blaze-Persistence Entity Views which will take care of making the query efficient.
You define your DTO as an interface with mappings to the entity, using the attribute name as default mapping, this looks very simple and a lot like a subset of an entity, though it doesn't have to. You can use any JPQL expression as mapping for your DTO attributes.
I'm starting a new project and have decided to try to incorporate DDD patterns and also include Linq to Entities. When I look at the EF's ObjectContext it seems to be performing the functions of both Repository and Unit of Work patterns:
Repository in the sense that the underlying data level interface is abstracted from the entity representation and I can request and save data through the ObjectContext.
Unit Of Work in the sense that I can write all my inserts/updates to the objectContext and execute them all in one shot when I do a SaveChanges().
It seems redundant to put another layer of these patterns on top of the EF ObjectContext? It also seems that the Model classes can be incorporated directly on top of the EF generated entities using 'partial class'.
I'm new at DDD so please let me know if I'm missing something here.
I don't think that the Entity Framework is a good implementation of Repository, because:
The object context is insufficiently abstract to do good unit testing of things which reference it, since it is bound to the DB access. Having an IRepository reference instead works much better for creating unit tests.
When a client has access to the ObjectContext, the client can do pretty much anything it cares to. The only real control you have over this at all is to make certain types or properties private. It is hard to implement good data security this way.
On a non-trivial model, the ObjectContext is insufficiently abstract. You may, for example, have both tables and stored procedures mapped to the same entity type. You don't really want the client to have to distinguish between the two mappings.
On a related note, it is difficult to write comprehensive and well-enforce business rules and entity code. Indeed, whether or not it this is even a good idea is debatable.
On the other hand, once you have an ObjectContext, implementing the Repository pattern is trivial. Indeed, for cases that are not particularly complex, the Repository is something of a wrapper around the ObjectContext and the Entity types.
I would say that you should look at the ObjectContext as your UnitOfWork, and not as a repository.
An ObjectContext cannot be a repository -imho- since it is 'to generic'.
You should create your own Repositories, which have specialized methods (like GetCustomersWithGoldStatus for instance) next to the regular CRUD methods.
So, what I would do, is create repositories (one for each aggregate-root), and let those repositories use the ObjectContext.
I like to have a repository layer for the following reasons:
EF gotcha's
When you look at some of the current tutorials on EF (Code First version), it is apparent there's a number of gotcha's to be handled, particularly around object graphs (entities containing entities) and disconnected scenarios. I think a repository layer is great for wrapping these up in one place.
A clear picture of data access mechanisms
A repository gives a specific picture as to how the BL is accessing and updating the data store. It exposes methods that have a clear single purpose, and can be tested independently of the BL. Standard example from the textbooks, Find() to find a single entity. A more application specific example, Clear() to clear down a db table.
A place for optimizations
Inevitably you come up against performance hits when using vanilla EF. I use the repository to hide the optimization mechanisms from the BL.
Examples,
GetKeys() to project cached keys from the tables (for Insert/Update decisions). The reading of key only is faster and uses less memory than reading the full entity.
Bulk load via SqlBulkCopy. EF will insert by individual SQL statements. If you want a single statement to insert multiple rows, SqlBulkCopy is a good mechanism. The repository encapsulates this and provides metadata for SqlBulkCopy. As well as the Insert method, you need a StartBatch() and EndBatch() method, which is also an argument for a UnitOfWork layer.