I have a working project with the following layers:
DataAccess - The project hits multiple DBs and web services. This layer defines the interfaces to each of them. It exposes the native types for each source (EF types for DBs, SOAP-defined types for web services, etc.). Let's say it exposes an EFProject and SoapProject.
Repository - This layer stiches results from the various sources to form a single entity, and exposes it. Let's call this ModelProject
Service - Adds REST attributes to to the entity (action links, etc.). This exposes a ProjectDTO.
WebApi - The controller spits out the ProjectDTO directly.
I'm trying to implement OData, specifically to page the results of very large queries. I've read a lot of examples, but they all seem to expose the source objects directly, and then map them to the final DTOs in the controller.
I would like to somehow push the ODataQueryOptions down to the Repository. This would allow me to keep the existing structure, and pass the query logic down to SQL. I understand that, because the ODataQueryOptions reference the ProjectDTO type, they can't be applied until an object of that type is available. Is there a way to "translate" the ODataQueryOptions from one type to another? Is there another way of doing this that I'm not aware of?
It is able to change the ODataQueryOptions, and such actions in a controller are for you to handle the options yourself:
public IQueryable Get(ODataQueryOptions queryOptions)
public IQueryable Get(int key, ODataQueryOptions queryOptions)
Here is a sample about this:
https://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v4/ODataQueryableSample/Controllers/OrdersController.cs .
For your reference, the source code of ODataQueryOptions is: https://aspnetwebstack.codeplex.com/SourceControl/latest#src/System.Web.OData/OData/Query/ODataQueryOptions.cs
Is there a way to "translate" the ODataQueryOptions from one type to another? Is there another way of doing this that I'm not aware of?
There is a way to actually translate the IQueryable<DomainModel> to IQueryable<DtoModel>.
I've done something similar in the past by leveraging AutoMapper's projection functionality. By calling the Project<TSource>/To<TTarget> methods, you can change an IQueryable that points to your domain models to another IQueryable that targets the Dto models, without actually executing it.
This means that you can now perform any OData operations on the DTO level and they will transfer through projection to the DAL layer into EntityFramework and SQL. In a scenario like this, there shouldn't be any need to manually handle the query logic so you can just use [EnableQuery] on the API route and let OData do its thing on the resulting IQueryable<DtoModel>.
I used this very successfully in one of the projects I worked on: as long as you rely just on AutoMapper projection to convert the types, it should work fine.
Granted, you can't do a lot of fancy mapping that way. The project methods will not be able to apply all kinds of mappings that you create, so I recommend checking the documentation on that front.
You also have to keep in mind that the original IQueryable needs to be exposed outside of the repository layer for this to work properly, otherwise the query will be executed too early. Some people will find that a boundary violation and will advocate for materializing the query inside the repository layer, but I don't have an issue with that particular aspect.
Related
I have this situation:
Spring Data JPA: Work with Pageable but with a specific set of fields of the entity
It about to work with Spring Data and working with a specific set of fields of an #Entity
The two suggestions are totally valid for me:
DTO projections
Projection interfaces
Even more, in spring-data-examples appears both together (I know for sample purposes):
CustomerRepository.java
Thus:
When is mandatory use one over the other and why?
Exists a cost of performance one over the other?
Note in the Class-based Projections (DTOs) section says the following:
Another way of defining projections is by using value type DTOs (Data
Transfer Objects) that hold properties for the fields that are
supposed to be retrieved. These DTO types can be used in exactly the
same way projection interfaces are used, except that no proxying
happens and no nested projections can be applied.
Seems the advantages are: except that no proxying happens and no nested projections can be applied
DTO Approach
Pro
Simple and straigt forward
Con
It will result in more code as you have to create DTO class with constructor and getters/setters (unless you utilize Project Lombok to avoid boilerplate
code for DTOs).
No nested projections can be applied.
Projections
Pro
Less code as it uses only interfaces.
Nested projections can be applied
Dynamic projection allows you write one generic repository method to return
different subset of the attributes in entity object based on client's needs.
Con
Spring generates proxy at runtime
Query could return the entire entity object from database to Spring layer though a trimmed version (via Projection) is returned from Spring layer to client. I wasn't sure about this specific disadvantage, hoping someone to edit this answer if necessary.
If you need nested or dynamic projection, you probably want Projection approach rather than DTO approach.
Refer to official Spring doc for details.
I think that DTO was the first possible solution to work with a small set of data from the Entities. Today, many operations can also be made with projections, but you need to be careful with performance. If you see this Janssen's post Entities or DTOs – When should you use which projection? you will note that DTOs have better performance than projections for reading operations.
If you don't have the problem with performance, projections will be more graceful.
Reading about using Java Generics in DAO layer, I have a doubt applying this in spring data repositories. I mean, with spring data repositories, you have something like this:
public interface OrderRepository extends CrudRepository<Order,OrderPK>{
}
But if I have other 10 entities, I have to create 10 interfaces like the one above to execute CRUD operations and so on and I think this is not very scalable. Java Generics and DAO is about creating one interface and one implementation and reuse this for entities but with Spring Data repositories I have to create one interface for each entity so ...
You didn't really state a question, so I just add
Is this really true? And if so, why?
and answer it:
Yes, this is (almost) correct. Almost, because you should not create one repository per entity, but one repository per Aggregate Root. See http://static.olivergierke.de/lectures/ddd-and-spring/
Spring Data Repositories offer various features for which Spring Data needs to know, what entity it is dealing with. For example query methods need to know the properties of the entity, in order to convert the method name to JPA based query. So you have to pass in the information to Spring Data at some point and you also have to pass in the information, which entities should be considered Aggregate Roots. The way you do that, is by specifying the interface.
Do you really need that? Well if all you want is generic Crud functionality, you can get that straight out of the box with JPA. But if you want query methods, Pagination, simple native queries and much more Spring Data is a nice way to avoid lots of boiler-plate code.
(Please keep in mind that I'm biased)
Here is my situation. I use Entity Framework 4 with the Web API
The structure of my code is quite simple, I have the Service layer where all my rest API is organized, I have my Business logic layer where I have business controllers to manage Transactions between the rest calls and the data layer. Finally, I have a data layer with generic repositories and a DAO to access the whole thing.
In my Business controllers, I use using to inject a non transactionnal (read only methods) OR a transactional (CRUD methods) DbContext.
When returning values to my REST API, I parse it into JSON.
The problem is that I keep having this exception: Newtonsoft.Json.JsonSerializationException
I return my entities / collections / lists outside of my using{} statement, which I think EF does not like by default.
In debug mode, sometimes, I will manage to retrieve all data, but not all the time. Since my entities come from a query within a DbContext, I think that the behavior is to remove loaded sub-properties after the context has been disposed of.
Fact is, I want to keep my structure as is, and I was wondering the following:
Is there a way of returning complete (not lazy-loaded) entities after leaving the using{} statement?
Thanks a lot
I actually read more about Entity Frameworks behavior. What I get is actually standard for EF. I have to force the context to Load() my refered entities in order to get them after leaving the context.
I have two EntityFramework models that I want to combine into a single DTO. Is there a way to do this? There are a couple ideas in the following question, but you would either have to create a composite model, or lose the ability to call Mapper.AssertConfigurationIsValid to verify all of the properties will be set.
Is it possible to map multiple DTO objects to a single ViewModel using Automapper?
single-viewmodel-using-automappe
From my point of view, It is highly recommanded to create a composite type for merging entities. Entities are part of your Business logic or your Domain logic (depending on your architecture), whereas DTO are part of Presentation logic or Transport layer. You can create an explicit mapping that can be easily tested ; automatic mapping (create maps without options) is good for testing only. If you are using a DTO, then you will probably use it somewhere : in WCF ? as a ViewModel ?
Visual Studio and .Net Framework can manage many files and you have not to sacrifice testability or simplicity (do you know "Technical Debt"?)
Note : The role of Mapper.AssertConfigurationIsValid is to validate all mapping, generated by automatic or explicit mapping. I suggest you call this every time.
I'm starting a new project and have decided to try to incorporate DDD patterns and also include Linq to Entities. When I look at the EF's ObjectContext it seems to be performing the functions of both Repository and Unit of Work patterns:
Repository in the sense that the underlying data level interface is abstracted from the entity representation and I can request and save data through the ObjectContext.
Unit Of Work in the sense that I can write all my inserts/updates to the objectContext and execute them all in one shot when I do a SaveChanges().
It seems redundant to put another layer of these patterns on top of the EF ObjectContext? It also seems that the Model classes can be incorporated directly on top of the EF generated entities using 'partial class'.
I'm new at DDD so please let me know if I'm missing something here.
I don't think that the Entity Framework is a good implementation of Repository, because:
The object context is insufficiently abstract to do good unit testing of things which reference it, since it is bound to the DB access. Having an IRepository reference instead works much better for creating unit tests.
When a client has access to the ObjectContext, the client can do pretty much anything it cares to. The only real control you have over this at all is to make certain types or properties private. It is hard to implement good data security this way.
On a non-trivial model, the ObjectContext is insufficiently abstract. You may, for example, have both tables and stored procedures mapped to the same entity type. You don't really want the client to have to distinguish between the two mappings.
On a related note, it is difficult to write comprehensive and well-enforce business rules and entity code. Indeed, whether or not it this is even a good idea is debatable.
On the other hand, once you have an ObjectContext, implementing the Repository pattern is trivial. Indeed, for cases that are not particularly complex, the Repository is something of a wrapper around the ObjectContext and the Entity types.
I would say that you should look at the ObjectContext as your UnitOfWork, and not as a repository.
An ObjectContext cannot be a repository -imho- since it is 'to generic'.
You should create your own Repositories, which have specialized methods (like GetCustomersWithGoldStatus for instance) next to the regular CRUD methods.
So, what I would do, is create repositories (one for each aggregate-root), and let those repositories use the ObjectContext.
I like to have a repository layer for the following reasons:
EF gotcha's
When you look at some of the current tutorials on EF (Code First version), it is apparent there's a number of gotcha's to be handled, particularly around object graphs (entities containing entities) and disconnected scenarios. I think a repository layer is great for wrapping these up in one place.
A clear picture of data access mechanisms
A repository gives a specific picture as to how the BL is accessing and updating the data store. It exposes methods that have a clear single purpose, and can be tested independently of the BL. Standard example from the textbooks, Find() to find a single entity. A more application specific example, Clear() to clear down a db table.
A place for optimizations
Inevitably you come up against performance hits when using vanilla EF. I use the repository to hide the optimization mechanisms from the BL.
Examples,
GetKeys() to project cached keys from the tables (for Insert/Update decisions). The reading of key only is faster and uses less memory than reading the full entity.
Bulk load via SqlBulkCopy. EF will insert by individual SQL statements. If you want a single statement to insert multiple rows, SqlBulkCopy is a good mechanism. The repository encapsulates this and provides metadata for SqlBulkCopy. As well as the Insert method, you need a StartBatch() and EndBatch() method, which is also an argument for a UnitOfWork layer.