I am having a hard time understanding the concept of Repositories in Loopback 4 the documentation says:
A Repository represents a specialized Service interface that provides strong-typed data access (for example, CRUD) operations of a domain model against the underlying database or service.
but this description doesn't help me fully grasp the idea behind them, Can someone explain in simple terms what it is exactly and provide some examples of similar concepts in other frameworks?
When using an ORM (Object Relational Mapper) framework, we represent data as model classes:
#model()
class Person {
#property()
name: string;
}
To persist and query the data, we need to add behavior to our models. Repositories are classes providing such behavior.
For example, LoopBack's EntityCrudRepository interface describes methods for creating, updating, deleting and querying data in SQL tables/NoSQL document collections.
// simplified version
interface EntityCrudRepository<T, ID> {
create(data: DataObject<T>): Promise<T>;
find(filter?: Filter<T>): Promise<T[]>;
updateById(id: ID, data: DataObject<T>): Promise<void>;
deleteById(id: ID): Promise<void>;
// etc.
}
The KeyValueRepository describes API for working with key-value storage like Redis:
interface KeyValueRepository<T> {
get(key: string): Promise<T>;
set(key: string, value: DataObject<T>): Promise<void>;
delete(key: string): Promise<void>;
}
See also Patterns of Enterprise Application Architecture:
Conceptually, a Repository encapsulates the set of objects persisted in a data store and the operations performed over them, providing a more object-oriented view of the persistence layer. Repository also supports the objective of achieving a clean separation and one-way dependency between the domain and data mapping layers.
Related
Let's say you have a domain object:
class ArgumentEntity
{
public int Id { get; set; }
public List<AnotherEntity> AnotherEntities { get; set; }
}
And you have ASP.NET Web API controller to deal with it:
[HttpPost("{id}")]
public IActionResult DoSomethingWithArgumentEntity(int id)
{
ArgumentEntity entity = this.Repository.GetById(id);
this.DomainService.DoDomething(entity);
...
}
It receives entity identifier, load entity by id and execute some business logic on it with domain service.
The problem:
The problem here is with related data. ArgumentEntity has AnotherEntities collection that will be loaded by EF only if you explicitly ask to do so via Include/Load methods.
DomainService is a part of business layer and should know nothing about persistence, related data and other EF concepts.
DoDomething service method expects to receive ArgumentEntity instance with loaded AnotherEntities collection.
You would say - it's easy, just Include required data in Repository.GetById and load whole object with related collection.
Now lets come back from simplified example to reality of the large application:
ArgumentEntity is much more complex. It contains multiple related collections and that related entities have their related data too.
You have multiple methods of DomainService. Each method requires different combinations of related data to be loaded.
I could imagine possible solutions, but all of them are far from ideal:
Always load the whole entity -> but it is inefficient and often impossible.
Add several repository methods: GetByIdOnlyHeader, GetByIdWithAnotherEntities, GetByIdFullData to load specific data subsets in controller -> but controller become aware of which data to load and pass to each service method.
Add several repository methods: GetByIdOnlyHeader, GetByIdWithAnotherEntities, GetByIdFullData to load specific data subsets in each service method -> it is inefficient, sql query for each service method call. What if you call 10 service methods for one controller action?
Each domain method call repository method to load additional required data ( e.g: EnsureAnotherEntitiesLoaded) -> it is ugly because my business logic become aware of EF concept of related data.
The question:
How would you solve the problem of loading required related data for the entity before passing it to business layer?
In your example I can see method DoSomethingWithArgumentEntity which obviously belongs to Application Layer. This method has call to Repository which belongs to Data Access Layer. I think this situation does not conform to classic Layered Architecture - you should not call DAL directly from Application Layer.
So your code can be rewritten in another manner:
[HttpPost("{id}")]
public IActionResult DoSomethingWithArgumentEntity(int id)
{
this.DomainService.DoDomething(id);
...
}
In DomainService implementation you can read from repo whatever it needs for this specific operation. This avoids your troubles in Application Layer. In Business Layer you will have more freedom to implement reading: with serveral repository methods reads half-full entity, or with EnsureXXX methods, or something else. Knowledge about what you need to read for operation will be placed into operation's code and you don't need this knowledge in app-layer any more.
Every time situation like this emerged it is a strong signal about your entity is not preperly designed. As krzys said the entity has not cohesive parts. In other words if you often need parts of an entity separately you should split this entity.
Nice question :)
I would argue that "related data" in itself is not a strict EF concept. Related data is a valid concept with NHibernate, with Dapper, or even if you use files for storage.
I agree with the other points mostly, though. So here's what I usually do: I have one repository method, in your case GetById, which has two parameters: the id and a params Expression<Func<T,object>>[]. And then, inside the repository I do the includes. This way you don't have any dependency on EF in your business logic (the expressions can be parsed manually for another type of data storage framework if necessary), and each BLL method can decide for themselves what related data they actually need.
public async Task<ArgumentEntity> GetByIdAsync(int id, params Expression<Func<ArgumentEntity,object>>[] includes)
{
var baseQuery = ctx.ArgumentEntities; // ctx is a reference to your context
foreach (var inlcude in inlcudes)
{
baseQuery = baseQuery.Include(include);
}
return await baseQuery.SingleAsync(a=>a.Id==id);
}
Speaking in context of DDD, It seems that you had missed some modeling aspects in your project that led you to this issue. The Entity you wrote about looked not to be highly cohesive. If different related data is needed for different processes (service methods) it seems like you didn't find proper Aggregates yet. Consider splitting your Entity into several Aggregates with high cohesion. Then all processes correlated with particular Aggregate will need all or most of all data that this Aggregate contains.
So I don't know the answer for your question, but if you can afford to make few steps back and refactor your model, I believe you will not encounter such problems.
I am writing RESTful services using spring and hibernate. I read many resource in internet, but they did not clarify my doubts. Please explain me in details what are DAO, DTO and Service layers in spring framework? And why usage of these layers is required in spring to develop RESTfull API services.
First off, these concepts are Platform Agnostic and are not exclusive to Spring Framework or any other framework, for that matter.
Data Transfer Object
DTO is an object that carries data between processes. When you're working with a remote interface, each call it is expensive. As a result you need to reduce the number of calls. The solution is to create a Data Transfer Object that can hold all the data for the call. It needs to be serializable to go across the connection. Usually an assembler is used on the server side to transfer data between the DTO and any domain objects. It's often little
more than a bunch of fields and the getters and setters for them.
Data Access Object
A Data Access Object abstracts and encapsulates all access to
the data source. The DAO manages the connection with the data source to
obtain and store data.
The DAO implements the access mechanism required to work with the data source.
The data source could be a persistent store like an RDBMS, or a business service accessed via REST or SOAP.
The DAO abstracts the underlying data access implementation for the Service objects to
enable transparent access to the data source. The Service also delegates
data load and store operations to the DAO.
Service
Service objects are doing the work that the
application needs to do for the domain you're working with. It involves calculations based on inputs and
stored data, validation of any data that comes in from the presentation, and figuring out exactly what data
source logic to dispatch, depending on commands received from the presentation.
A Service Layer defines an application's boundary and its set of available operations from
the perspective of interfacing client layers. It encapsulates the application's business logic, controlling
transactions and coordinating responses in the implementation of its operations.
Recommended References
Martin Fowler has a great book on common Application Architecture Patterns named Patterns of Enterprise Application Architecture. There is also, Core J2EE Patterns that worth looking at.
DAO - Data Access Object:
An object that provides a common interface to perform all database operations like persistence mechanism.
public interface GenericDao<T> {
public T find(Class<T> entityClass, Object id);
public void save(T entity);
public T update(T entity);
public void delete(T entity);
public List<T> findAll(Class<T> entityClass);
}
See this example : Spring – DAO and Service layer
DTO - Data Transfer Object:
An object that carries data between processes in order to reduce the number of method calls means you combine more than one POJO entities in service layer.
For example a GET request /rest/customer/101/orders is to retrieve all the orders for customer id 101 along with customer details hence you need combine entity Customer and entity Orders with details.
Enterprise application is divided into tiers for easy maintenance and development. Tiers are dedicated to particular type of tasks like
presentation layer (UI)
Business layer
Data access layer (DAO, DTO)
Why this design:
Let's pick an example you have an application which reads data from db and performs some business logic on it then present it to user. Now if you want to change your DB let say earlier application was running on Oracle now you want to use mysql so if you don't develop it in tiers you will doing changes everywhere in application. But if you implement DAO in application then this can be done easily
DAO: Data Access Object is design pattern
just provide an interface for accessing data to service layer and provide different implementations for different data sources (Databases, File systems)
Example code:
public interface DaoService {
public boolean create(Object record);
public CustomerTemp findTmp(String id);
public Customer find(String id);
public List getAllTmp();
public List getAll();
public boolean update(Object record);
public boolean delete(Object record);
public User getUser(String email);
public boolean addUser(User user);
}
Service layer using Dao
#Service("checkerService")
public class CheckerServiceImpl implements CheckerService{
#Autowired
#Qualifier("customerService")
private DaoService daoService;
Now i can provide any implementation of DaoService interface.
Service and DTO are also used for separation of concerns.
DTO: The data object that we pass between different processes or within the same process. It can be a wrapper around the actual entity object. Using entity object as is for DTO is not safe and not recommended. Design of this object is based on various factors like simplicity of representation, security of exposing Ids, consumer requirement and so on.
In Spring, DTO can be formed with a simple model/pojo object.
DAO: The object responsible for CRUD operations.
In Spring, this can be an object that implements JPARepository interface, or any bean that connects with database and does a CRUD for us. Please remember your journey from JDBC to Hibernate to Spring data JPA. :)
Service: Core bean for business logic implementation. This object may has DAO object as its dependency. Core business logic for the particular use case will go here.
In Spring, Service object/bean can be created either by annotating the bean with #Service or #Component annotation or simply representing the object as a bean using java configuration. Make sure you inject all the required dependencies into the service bean for its heavy lifting job.
SERVICE LAYER:
It receives the request from controller layer and process the request to Persistence layer
#Controller:It is the annotation which initialize whole controller layer.
#Service:It is the annotation which initialize whole service layer.
#Repository: It is the annotation which initialize whole persistence layer.
DTO:
It is an Data Transfer object which used to pass the properties from service layer to persistence layer.
DAO:
It is an Data Access object. it is also known as persistence layer. In this DAO we receive property values from service layer in DTO object. Here we write an persistence logic to db.
I was starting a design involving Scala case classes as models, and I was wondering about a design decision.
Let's imagine we have two models, a User model and an Order model. The Order model references the User model as a foreign key.
case class User(id: UserId, [Other fields...], password: String)
case class Order(id: OrderId, [Other fields...], userId: UserId)
Then, given this design, we would have an Orders DAO with a method findByUser method.
My question is: would it be good design to have an orders method in User that calls this DAO method, and thus making the system more OO, or is it better to keep layers isolated and not include this method?
Thanks!
If you understand you correctly, you're asking about the Active Record Pattern. As any pattern, it has its pros and cons, you can find more about it online. Here are some of them:
http://www.mehdi-khalili.com/orm-anti-patterns-part-1-active-record
https://softwareengineering.stackexchange.com/questions/70291/what-are-the-drawbacks-to-the-activerecord-pattern
Active Record Design Pattern?
In a Play2 project, I firstly used the pattern, mainly because of Ebean support. However, since I needed a bit more logic for persisting some models, extension of it turned cumbersome. In the end, I separated everything: separate service, separate model, separate DAO. That also helped me easily switch between repository layer implementations (in the end I could experiment with Spring Data JPA, Hibernate etc. more freely)
I'm studying MVC and EF at the moment and I see quite often in the articles on those subjects that controller manipulates data via a Repository object instead of directly using LINQ to Entities.
I created a small web application to test some ideas. It declares the following interface as an abstraction of the data storage
public interface ITinyShopDataService
{
IQueryable<Category> Categories { get; }
IQueryable<Product> Products { get; }
}
Then I have a class that inherits from the generated class TinyShopDataContext (inherited from ObjectContext) and implements ITinyShopDataService.
public class TinyShopDataService : TinyShopDataContext, ITinyShopDataService
{
public new IQueryable<Product> Products
{
get { return base.Products; }
}
public new IQueryable<Category> Categories
{
get { return base.Categories; }
}
}
Finally, a controller uses an implementation of ITinyShopDataService to get the data, e.g. for displaying products of a specific category.
public class HomeController : Controller
{
private ITinyShopDataService _dataService;
public HomeController(ITinyShopDataService dataService)
{
_dataService = dataService;
}
public ViewResult ProductList(int categoryId)
{
var category = _dataService.Categories.First(c => c.Id == categoryId);
var products = category.Products.ToList();
return View(products);
}
}
I think the controller above possesses some positive properties.
It is easily testable because of data service being injected.
It uses LINQ statements that query an abstract data storage in an implementation-neutral way.
So, I don't see any benefit of adding a Repository between the controller and the ObjectContext-derived class. What do I miss?
(Sorry, it was a bit too long for the first post)
You can test your controllers just fine with what you propose.
But how do you test your queries inside the data service? Imagine you have very complicated query behavior, with multiple, complex queries to return a result (check security, then query a web service, then query the EF, etc.).
You could put that in the controller, but that's clearly wrong.
It should go in your service layer, of course. You might mock/fake out the service layer to test the controller, but now you need to test the service layer.
It would be nice if you could do this without connecting to a DB (or web service). And that's where Repository comes in.
The data service can use a "dumb" Repository as a source of data on which it can perform complex queries. You can then test the data service by replacing the repository with a fake implementation which uses a List<T> as its data store.
The only thing which makes this confusing is the large amount of examples that show controllers talking directly to repositories. (S#arp Architecture, I'm looking at you.) If you're looking at those examples, and you already have a service layer, then I agree that it makes it difficult to see what the benefit of the repository is. But if you just consider the testing of the service layer itself, and don't plan on having controllers talking directly to the repository, then it starts to make more sense.
Ayende of the NHibernate project and other fame agrees with you.
Some Domain Driven Design people would suggest that you should have a service (not same as web service) layer that is called from your controller, but DDD is usually applied to complex domains. In simple applications, what you're doing is fine and testable.
What you're missing is the ability to update, create and delete objects. If that's not an issue, than the interface is probably good enough as it is (although I would let it derive from IDisposable to make sure you can always dispose the context (and test for this))
The repository pattern has nothing to do with Entity Framework, although Entity Framework implements a "repository" pattern.
The repository interface (lets say IRepositoryProducts) lives in the domain layer. It understands domain objects, or even Entities if you don't want to use domain driven design.
But its implementation (lets say RepositoryProducts) is the actual repository pattern implementation and does not live in the domain layer but in the persistance layer.
This implementation can use Entity or any ORM ..or not to persist the information in the database.
So the answer is: It's not really necessary, but it's recommended to use Repository pattern despite using Entity ORM as a good practice to keep separation between domain layer and persistance layer. Because that's the purpose of the Repository Pattern: separation of concerns between the domain logic and the way the information is persisted. When using the repository pattern from the domain level you just abstract yourself and think "I want to store this information, I don't care how it's done", or "I want a list of these things, I don't care where do you get them from or how do you retrieve them. Just give 'em to me".
Entity Framework has nothing to do with domain layer, it's only a nice way to persist objects, but should live in the repository implementation (persistance layer)
I am considering to use DTOs instead of passing around my domain objects. I have read several posts here as well as elsewhere, and i understand there are several approaches to getting this done.
If i only have about 10 domain classes in all, and considering that i want to use DTOs rather than domain objects for consumption in my Views (WPF front ends), what is the recommended approach.
I think using tools like automapper etc maybe an overkill for my situation. So i am thinking of writing my custom mapper class that will have methods for converting a domain type to a DTO type.
What is the best way to do this, are there any sample to get me started to do this?
Second question: When writing those methods that will create DTOs, how do i deal with setting up all the data, especially when the domain type has references to other domain objects? Do i write equivalent properties in the DTO for mapping to those refernece types in the domain class?
Please ask if i have not put my second question in proper words. But i think you understand what i am trying to ask.
Thrid question: When writing DTOs, should i write multiple DTOs, each containing partial data for a given domain model, so that each of it can be used to cater to a specific View's requirement, or should the DTO have all the data that are there in the corresponding model class.
I've been reading a few posts here regarding DTO's and it seems to me that a lot of people equate them to what I would consider a ViewModel. A DTO is just that, Data Transfer Object - it's what gets passed down the wire. So I've got a website and services, only the services will have access to real domain/entity objects, and return DTO's. These may map 1:1, but consider that the DTO's may be populated from another service call, a database query, reading a config - whatever.
After that, the website then can take those DTO and either add them to a ViewModel, or convert into one. That ViewModel may contain many different types of DTO's. A simple example would be a task manager - the ViewModel contains both the task object you are editing, as well as a group of Dto.User objects that the task can be assigned to.
Keep in mind that the services returning DTO's maybe used by both a website, and maybe a tablet or phone application. These applications would have different views to take advantage of their displays and so the ViewModels would differ, but the DTO's would remain the same.
At any rate, I love these types of discussions, so anyone please let me know what you think.
Matt
I'm kind of using DTOs in a project. I tend to make the DTOs only to show the data I need for an specified view. I fetch all the data shown in the view in my data access class. For example, I may have an Order object which references a Client object:
public class Client{
public int Id{get;set;}
public string Name{get;set;}
}
public class Order{
public int OrderID{get;set;}
public Client client{get;set;}
public double Total{get;set;}
public IEnumerable<OrderLine> lines {get;set;}
}
Then in my OrderListDTO I may have something like:
public class OrderListDTO{
public int OrderId{get;set;}
public string ClientName{get;set;}
...
}
Which are the fields I want to show in my view. I fetch all these fields in my Database access code so I don't have to bother with entity asociations in my view or controller code.
Best Way to develop DTOs
The way to start developing DTOs is to understand that their sole purpose is to transfer subset of data of your business entities to different clients(could be UI, or an external service). Given this understanding you could create seperate packages for each client...and write your DTO classes. For mapping you could write your own mapper defining interfaces to be passed to a factory creating DTO objects based on which data from the entity for which the DTO is being created would be extracted. You could also define annotations to be placed on your entity fields but personally given the number of annotations used I would prefer the interface way. The main thing to note about DTOs is that they are also classes and data among the DTOs should be reused, in other words while it may seem tempting to create DTOs for each use case try to reuse existing DTOs to minimize this.
Getting started
Regarding getting started as stated above the sole purpose of the DTO is to give the client the data it needs....so you keeping in mind you could just set data into the dto using setters...or define a factory which creates a DTO from an Entity based on an interface.....
Regarding your third question, do as is required by your client :)
I come to project with spring-jdbc and there are used DAO layer. Some times existing entities doesn't cover all possible data from DB. So I start using DTO.
By applying '70 structure programming rule I put all DTOs into separate package:
package com.evil.dao; // DAO interfaces for IOC.
package com.evil.dao.impl; // DAO implementation classes.
package com.evil.dao.dto; // DTOs
Now I rethink and decide to put all DTO as inner classes on DAO interfaces for result-sets which have no reuse. So DAO interface look like:
interface StatisticDao {
class StatisticDto {
int count;
double amount;
String type;
public static void extract(ResultSet rs, StatisticDto dto) { ... }
}
List<StatisticDto> getStatistic(Criteria criteria);
}
class StatisticDaoImpl implements StatisticDao {
List<StatisticDto> getStatistic(Criteria criteria) {
...
RowCallbackHandler callback = new RowCallbackHandler() {
#Override
public void processRow(ResultSet rs) throws SQLException {
StatisticDao.StatisticDto.extract(rs, dto);
// make action on dto
}
}
namedTemplate.query(query, queryParams, callback);
}
}
I think that holding related data together (custom DTO with DAO interface) make code better for PageUp/PageDown.
Question 1: If the DTO's you need to transfer are just a simple subset of your domain object, you can use a modelmapper to avoid filling your codebase with logic-less mapping. But if you need to apply some logic/conversion to your mapping then do it yourself.
Question 2: You can and probably should create a DTO for each domain object you have on your main DTO. A DTO can have multiple DTO's inside of it, one for each domain object you need to map. And to map those you could do it yourself or even use some modelmapper.
Question 3: Don't expose all your domain if your view does not require it to. Also you don't need to create a DTO for each view, try to create DTO's that expose what need to be exposed and may be reused to avoid having multiples DTO's that share a lot of information. But it mainly depend's on your application needs.
If you need clarification, just ask.
I'm going to assume that your domain model objects have a primary key ID that may correspond to the ID's from the database or store they came from.
If the above is true, then your DTO will overcome type referecning to other DTO's like your domain objects do, in the form of a foreign key ID. So an OrderLine.OrderHeader relationship on the domain object, will be OrderLine.OrderHeaderId cin the DTO.
Hope that helps.
Can I ask why you have chosen to use DTO's instead of your rich domain objects in the view?
We all know what Dtos are (probably).
But the important thing is to overuse DTOs or not.
Transfering data using Dtos between "local" services is a good practice but have a huge overhead on your developer team.
There is some facts:
Clients should not see or interact with Entities (Daos). So you
always need Dtos for transferig data to/from remote (out of the process).
Using Dtos to pass data between services is optional. If you don't plan to split up your project to microservices there is no need to do that. It will be just an overhead for you.
And this is my comment: If you plan to distribute your project to
microservices in long future. or don't plan to do that, then
DON'T OVERUSE DTOs
You need to read this article https://martinfowler.com/bliki/LocalDTO.html