I am creating the high level design for a new service. The complexity of the service warrants using DDD (I think). So I did the conventional thing and created domain services, aggregates, repositories, etc. My repositories encapsulate the data source. So a query can look for an object in the cache, failing that look in the db, failing that make a REST call to an external service to fetch the required information. This is fairly standard. Now the argument put forward by my colleagues is that abstracting the data source this way is dangerous because the developer using the repository will not be aware of the time required to execute the api and consequently not be able to calculate the execution time for any apis he writes above it. May be he would want to set up his component's behaviour differently if he knew that his call would result in a REST call. They are suggesting I move the REST call outside of the repository and maybe even the caching strategy along with it. I can see their point but the whole idea behind the repository pattern is precisely to hide this kind of information and not have each component deal with caching strategies and data access. My question is, is there a pattern or model which addresses this concern?
They are suggesting I move the REST call outside of the repository
Then you won't have a repository. The repository means we don't know persistence details, not that we don't know there is persistence. Every time we're using a repository, regardless of its implementation (from a in memory list to a REST call) we expect 'slowness' because it's common knowledge that persistence usually is the bottleneck.
Someone who will use a certain repository implementation (like REST based) will know it will deal with latency and transient errors. A service having just a IRepository dependency still knows it deals with persistence.
About caching strategies, you can have some service level (more generic) caching and repository level (persistence specific) caching. These probably should be implementation details.
Now the argument put forward by my colleagues is that abstracting the data source this way is dangerous because the developer using the repository will not be aware of the time required to execute the api and consequently not be able to calculate the execution time for any apis he writes above it. May be he would want to set up his component's behaviour differently if he knew that his call would result in a REST call.
This is wasting time trying to complicate your life. The whole point of an abstraction is to hide the dirty details. What they suggest is basically: let's make the user aware of some implementation detail, so that the user can couple its code to that.
The point is, a developer should be aware of the api they're using. If a component is using an external service (db, web service), this should be known. Once you know there's data to be fetched, you know you'll have to wait for it.
If you go the DDD route then you have bounded contexts (BC). Making a model dependent on another BC is a very bad idea . Each BC should publish domain events and each interested BC should subscribe and keep their very own model based on those events. This means the queries will be 'local' but you'll still be hitting a db.
Repository pattern aim to reduce the coupling with persistence layer. In my opinion I wouldn't risk to make a repository so full of responsibility.
You could use an Anti Corruption Layer against changes in external service and a Proxy to hide the caching related issues.
Then in the application layer I will code the fallback strategy.
I think it all depends where you think the fetching/fallback strategy belongs, in the Service layer or in the Infrastructure layer (latter sounds more legit to me).
It could also be a mix of the two -- the Service is passed an ordered series of Repositories to use one after the other in case of failure. Construction of the series of Repos could be placed in the Infrastructure layer or somewhere else. Fallback logic in one place, fallback configuration in another.
As a side note, asynchrony seems like a good way to signal the users that something is potentially slow and would be blocking if you waited for it. Better than hiding everything behind a vanilla, inconspicuous Repository name and better than adding some big threatening "this could be slow" prefix to your type, IMO.
Related
I want to make an API using REST which interacts (stores) data in a database.
While I was reading some design patterns and I came across remote facade, and the book I was reading mentions that the role of this facade is to translate the course grained methods from the remote calls into fine grained local calls, and that it should not have any extra logic. As an explaination, it says that the program should still work without this facade.
Here's an example
Yet I have two questions:
Considering I also have a database, does it make sense to split the general call into specific calls for each attribute? Doesn't it make more sense to just have a general "get data" method that runs one query against the database and converts it into an usable object, to reduce the number of database calls? So instead of splitting the get address to get street, get city, get zip, make on db call for all that info.
With all this in mind, and, in my case using golang, how should the project be structured in terms of files and functions?
I will have the main file with all the endpoints from the REST API, calling the controllers that handle these requests.
I will have a set of files that define those controllers. Are these controllers the remote facade? Should those methods not have logic in that case, and just call the equivalent local methods?
Should the local methods call the database directly, or should they use some sort of helper class that accesses the database?
Assuming all questions are positive, does the following structure make sense?
Main
Controllers
Domain
Database helper
First and foremost, as Mike Amundsen has stated
Your data model is not your object model is not your resource model is not your affordance model
Jim Webber did say something very similar, that by implementing a REST architecture you have an integration model, in the form of the Web, which is governed by HTTP and the other being the domain model. Resources adept and project your domain model to the world, though there is no 1:1 mapping between the data in your database and the representations you send out. A typical REST system does have many more resources than you have DB entries in your domain model.
With that being said, it is hard to give concrete advice on how you should structure your project, especially in terms of a certain framework you want to use. In regards to Robert "Uncle Bob" C. Martin on looking at the code structure, it should tell you something about the intent of the application and not about the framework¹ you use. According to him Architecture is about intent. Though what you usually see is the default-structure imposed by a framework such as Maven, Ruby on Rails, ... For golang you should probably read through certain documentation or blogs which might or might not give you some ideas.
In terms of accessing the database you might either try to follow a micro-service architecture where each service maintains their own database or you attempt something like a distributed monolith that acts as one cohesive system and shares the database among all its parts. In case you scale to the broad and a couple of parallel services consume data, i.e. in case of a message broker, you might need a distributed lock and/or queue to guarantee that the data is not consumed by multiple instances at the same time.
What you should do, however, is design your data layer in a way that it does scale well. What many developers often forget or underestimate is the benefit they can gain from caching. Links are basically used on the Web to reference from one resource to an other and giving the relation some semantic context by the utilization of well-defined link-relation names. Link relations also allow a server to control its own namespace and change URIs as needed. But URIs are not only pointers to a resource a client can invoke but also keys for a cache. Caching can take place on multiple locations. On the server side to avoid costly calculations or look ups on the client side to avoid sending requests out in general or on intermediary hops which allow to take away pressure from heavily requested servers. Fielding made caching even a constraint that needs to be respected.
In regards to what attributes you should create queries for is totally dependent on the use case you attempt to depict. In case of the address example given it does make sense to return the address information all at once as the street or zip code is rarely queried on its own. If the address is part of some user or employee data it is more vague whether to return that information as part of the user or employee data or just as a link that should be queried on its own as part of a further request. What you return may also depend on the capabilities of the media-type client and your service agree upon (content-type negotiation).
If you implement something like a grouping for i.e. some football players and certain categories they belong to, such as their teams and whether they are offense or defense players, you might have a Team A resource that includes all of the players as embedded data. Within the DB you could have either an own table for teams and references to the respective player or the team could just be a column in the player table. We don't know and a client usually doesn't bother as well. From a design perspective you should however be aware of the benefits and consequences of including all the players at the same time in regards to providing links to the respective player or using a mixed approach of presenting some base data and a link to learn further details.
The latter approach is probably the most sensible way as this gives a client enough information to determine whether more detailed data is needed or not. If needed a simple GET request to the provided URI is enough, which might be served by a cache and thus never reach the actual server at all. The first approach has for sure the disadvantage that it doesn't reuse caching optimally and may return way more data then actually needed. The approach to include links only may not provide enough information forcing the client to perform a follow-up request to learn data about the team member. But as mentioned before, you as the service designer decide which URIs or queries are returned to the client and thus can design your system and data model accordingly.
In general what you do in a REST architecture is providing a client with choices. It is good practice to design the overall interaction flow as a state machine which is traversed through receiving requests and returning responses. As REST uses the same interaction model as the Web, it probably feels more natural to design the whole system as if you'd implement it for the Web and then apply the design to your REST system.
Whether controllers should contain business logic or not is primarily an opinionated question. As Jim Webber correctly stated, HTTP, which is the de-facto transport layer of REST, is an
application protocol whose application domain is the transfer of documents over a network. That is what HTTP does. It moves documents around. ... HTTP is an application protocol, but it is NOT YOUR application protocol.
He further points out that you have to narrow HTTP into a domain application protocol and trigger business activities as a side-effect of moving documents around the network. So, it's the side-effect of moving documents over the network that triggers your business logic. There is no straight rule whether to include business logic in your controller or not, but usually you try to keep the business logic in yet their own layer, i.e. as a service that you just invoke from within the controller. That allows to test the business logic without the need of the controller and thus without the need of a real HTTP request.
While this answer can't provide more detailed information, partly due to the broad nature of the question itself, I hope I could shed some light in what areas you should put in some thoughts and that your data model is not necessarily your resource or affordance model.
I am building simple REST deployable using Spring Boot. Decided to create it by using failing acceptance test first followed with TDD until its green.
My module is pretty simple, I have 3 API's:
Retrieving list of data from datastore.
Adds item to datastore.
Deletes item from datastore.
I feel like it is good idea to abstract datastore and have maybe backed by Map data structure for testing purposes and use it with either NoSQL or SQL db if I want to for deployments/releases and end to end testing.
On the service layer side I am unsure since it would just delegate call to repository with no logic.
So standard approach would be controller->service->repository. In my case service does not do much(possible some exception handling but not more) and I will end up with interface and implementation as an extra as well as few more lines of code. I fell like going for controller->repository solution in my situation but it is not a practice I have seen and not sure how others would see it.
What's the best way to implement this sort of system?
I feel like it is good idea to abstract datastore
You are right. The abstraction is called 'Repository' in DDD (Domain Driven Design) for example.
On the service layer side I am unsure since it would just delegate call to repository with no logic.
I'm pretty sure there are data that you want to validate. So you should have a layer in the middle (e.g. the domain layer) which will be in charge of this validation.
Even so, if you feel like your application is simple and doesn't require such layers, go without. You will have less supple design, but more simplicity at first. Be careful: while evolving your app, you could run into trouble.
Hope this will help.
This is rather an opinion based question, but if you are asking whether a 3 layer architecture is a must, to that I say no. Be pragmatic, if you don' see a reason for a class/layer/module to exist, it does not need to exist.
A repository has a purpose (to store/retrieve), and the api layer has a purpose, to offer those things through HTTP.
Here is an article for building small services with the sparkframework: https://dzone.com/articles/building-simple-restful-api
I started with reading about CQRS and I'm little confused.
Is it allowed to call the read side within the write side for getting additional informations?
http://cqrs.nu/Faq/command-handlers here they say it is not allowed, but in the cqrs journey code I found that they call a service 'IPricingService' which internally uses a DAO service class.
So what I must do to get additional informations inside my aggregation root?
CQRS Journey should not be seen as a manual. This is just a story of some team fighting their way to CQRS and having all limitations of using only Microsoft stack. Per se you should not use your read model in the command handlers or domain logic. But you can query your read model from the client to fetch the data you need in for your command and to validate the command.
Since I got some downvotes on this answer, I need to point, that what I wrote is the established practice within the pattern. Neither read side accesses the write side, not write side gets data from the read side.
However, the definition of "client" could be a subject of discussion. For example, I would not trust a public facing JS browser application to be a proper "client". Instead, I would use my REST API layer to be the "client" in CQRS and the web application would be just a UI layer for this client. In this case, the REST API service call processing will be a legitimate read side reader since it needs to validate all what UI layer send to prevent forgery and validate some business rules. When this work is done, the command is formed and sent over to the write side. The validations and everything else is synchronous and command handling is then asynchronous.
UPDATE: In the light of some disagreements below, I would like to point to Udi's article from 2009 talking about CQRS in general, commands and validation in particular.
The CQRS FAQ (http://cqrs.nu/Faq) suggests:
"How can I communicate between bounded contexts?
Exclusively in terms of their public API. This could involve subscribing to events coming from another bounded context. Or one bounded context could act like a regular client of another, sending commands and queries."
So although within one BC its not possible to use read-side from write-side and vice-versa, another bounded context or service could. In essence, this would be acting like a human using the user interface.
Yes assuming you have accepted the eventual consistency of the read side. Now the question is where. Although there is no hard rule on this, it is preferable to pass the data to command handler as opposed to retrieving it inside. From my observation there are two ways:
Do it on domain service
Basically create a layer where you execute necessary queries to build the data. This is as straightforward as doing API calls. However if you have your microservices running on Lambda/Serverless it's probably not a great fit as we tend to avoid a situation where lambda is calling another lambda.
Do it on the client side
Have the client query the data then pass it to you. To prevent tampering, encrypt it. You can implement the decryption in the same place you validate and transform the DTO to a command. To me this is a better alternative as it requires fewer moving parts.
I think it depends.
If in your architecture the "command side" updates the projections on real-time (synchronously) you could do that calling the query api. (although that seems strange)
But, if your projections (query side) is updated asyncronously would a bad idea to do it. Would be a posibility to get a "unreal" data.
Maybe this situation suggests a design problem that you should solve.
For instance: If from one context/domain you think you need information from another, could a domain definition problem.
I assume this, because read data from itself (same domain) during a command operation doesn't make much sense. (in this case could be a API design problem)
(Note: My question has very similar concerns as the person who asked this question three months ago, but it was never answered.)
I recently started working with MVC3 + Entity Framework and I keep reading that the best practice is to use the repository pattern to centralize access to the DAL. This is also accompanied with explanations that you want to keep the DAL separate from the domain and especially the view layer. But in the examples I've seen the repository is (or appears to be) simply returning DAL entities, i.e. in my case the repository would return EF entities.
So my question is, what good is the repository if it only returns DAL entities? Doesn't this add a layer of complexity that doesn't eliminate the problem of passing DAL entities around between layers? If the repository pattern creates a "single point of entry into the DAL", how is that different from the context object? If the repository provides a mechanism to retrieve and persist DAL objects, how is that different from the context object?
Also, I read in at least one place that the Unit of Work pattern centralizes repository access in order to manage the data context object(s), but I don't grok why this is important either.
I'm 98.8% sure I'm missing something here, but from my readings I didn't see it. Of course I may just not be reading the right sources... :\
I think the term "repository" is commonly thought of in the way the "repository pattern" is described by the book Patterns of Enterprise Application Architecture by Martin Fowler.
A Repository mediates between the domain and data mapping layers,
acting like an in-memory domain object collection. Client objects
construct query specifications declaratively and submit them to
Repository for satisfaction. Objects can be added to and removed from
the Repository, as they can from a simple collection of objects, and
the mapping code encapsulated by the Repository will carry out the
appropriate operations behind the scenes.
On the surface, Entity Framework accomplishes all of this, and can be used as a simple form of a repository. However, there can be more to a repository than simply a data layer abstraction.
According to the book Domain Driven Design by Eric Evans, a repository has these advantages:
They present clients with a simple model for obtaining persistence objects and managing their life cycle
They decouple application and domain design from persistence technology, multiple database strategies, or even multiple data sources
They communicate design decisions about object access
They allow easy substitution of a dummy implementation, for unit testing (typically using an in-memory collection).
The first point roughly equates to the paragraph above, and it's easy to see that Entity Framework itself easily accomplishes it.
Some would argue that EF accomplishes the second point as well. But commonly EF is used simply to turn each database table into an EF entity, and pass it through to UI. It may be abstracting the mechanism of data access, but it's hardly abstracting away the relational data structure behind the scenes.
In simpler applications that mostly data oriented, this might not seem to be an important point. But as the applications' domain rules / business logic become more complex, you may want to be more object oriented. It's not uncommon that the relational structure of the data contains idiosyncrasies that aren't important to the business domain, but are side-effects of the data storage. In such cases, it's not enough to abstract the persistence mechanism but also the nature of the data structure itself. EF alone generally won't help you do that, but a repository layer will.
As for the third advantage, EF will do nothing (from a DDD perspective) to help. Typically DDD uses the repository not just to abstract the mechanism of data persistence, but also to provide constraints around how certain data can be accessed:
We also need no query access for persistent objects that are more
convenient to find by traversal. For example, the address of a person
could be requested from the Person object. And most important, any
object internal to an AGGREGATE is prohibited from access except by
traversal from the root.
In other words, you would not have an 'AddressRepository' just because you have an Address table in your database. If your design chooses to manage how the Address objects are accessed in this way, the PersonRepository is where you would define and enforce the design choice.
Also, a DDD repository would typically be where certain business concepts relating to sets of domain data are encapsulated. An OrderRepository may have a method called OutstandingOrdersForAccount which returns a specific subset of Orders. Or a Customer repository may contain a PreferredCustomerByPostalCode method.
Entity Framework's DataContext classes don't lend themselves well to such functionality without the added repository abstraction layer. They do work well for what DDD calls Specifications, which can be simple boolean expressions sent in to a simple method that will evaluate the data against the expression and return a match.
As for the fourth advantage, while I'm sure there are certain strategies that might let one substitute for the datacontext, wrapping it in a repository makes it dead simple.
Regarding 'Unit of Work', here's what the DDD book has to say:
Leave transaction control to the client. Although the REPOSITORY will insert into and delete from the database, it will ordinarily not
commit anything. It is tempting to commit after saving, for example,
but the client presumably has the context to correctly initiate and
commit units of work. Transaction management will be simpler if the
REPOSITORY keeps its hands off.
Entity Framework's DbContext basically resembles a Repository (and a Unit of Work as well). You don't necessarily have to abstract it away in simple scenarios.
The main advantage of the repository is that your domain can be ignorant and independent of the persistence mechanism. In a layer based architecture, the dependencies point from the UI layer down through the domain (or usually called business logic layer) to the data access layer. This means the UI depends on the BLL, which itself depends on the DAL.
In a more modern architecture (as propagated by domain-driven design and other object-oriented approaches) the domain should have no outward-pointing dependencies. This means the UI, the persistence mechanism and everything else should depend on the domain, and not the other way around.
A repository will then be represented through its interface inside the domain but have its concrete implementation outside the domain, in the persistence module. This way the domain depends only on the abstract interface, not the concrete implementation.
That basically is object-orientation versus procedural programming on an architectural level.
See also the Ports and Adapters a.k.a. Hexagonal Architecture.
Another advantage of the repository is that you can create similar access mechanisms to various data sources. Not only to databases but to cloud-based stores, external APIs, third-party applications, etc.
You're right,in those simple cases the repository is just another name for a DAO and it brings only one value: the fact that you can switch EF to another data access technique. Today you're using MSSQL, tomorrow you'll want a cloud storage. OR using a micro orm instead of EF or switching from MSSQL to MySql.
In all those cases it's good that you use a repository, as the rest of the app won't care about what storage you're using now.
There's also the limited case where you get information from multiple sources (db + file system), a repo will act as the facade, but it's still a another name for a DAO.
A 'real' repository is valid only when you're dealing with domain/business objects, for data centric apps which won't change storage, the ORM alone is enough.
It would be useful in situations where you have multiple data sources, and want to access them using a consistent coding strategy.
For example, you may have multiple EF data models, and some data accessed using traditional ADO.NET with stored procs, and some data accessed using a 3rd party API, and some accessed from an Access database living on a Windows NT4 server sitting under a blanket of dust in your broom closet.
You may not want your business or front-end layers to care about where the data is coming from, so you build a generic repository pattern to access "data", rather than to access "Entity Framework data".
In this scenario, your actual repository implementations will be different from each other, but the code that calls them wouldn't know the difference.
Given your scenario, I would simply opt for a set of interfaces that represent what data structures (your Domain Models) need to be returned from your data layer. Your implementation can then be a mixture of EF, Raw ADO.Net or any other type of Data Store/Provider. The key strategy here is that the implementation is abstracted away from the immediate consumer - your Domain layer. This is useful when you want to unit test your domain objects and, in less common situations - change your data provider / database platform altogether.
You should, if you havent already, consider using an IOC container as they make loose coupling of your solution very easy by way of Dependency Injection. There are many available, personally i prefer Ninject.
The domain layer should encapsulate all of your business logic - the rules and requirements of the problem domain, and can be consumed directly by your MVC3 web application. In certain situations it makes sense to introduce a services layer that sits above the domain layer, but this is not always necessary, and can be overkill for straightforward web applications.
Another thing to consider is that even when you know that you will be working with a single data store it still might make sense to create a repository abstraction. The reason is that there might be a function that your application needs that your ORM du jour either does badly (performance), not at all, or you just don't know how to make the ORM bend to your needs.
If you are wrapping your ORM behind a well thought out repository interface, you can easily switch between different technologies as you see fit. It's not uncommon in my repositories to see some methods use EF for their work and others to use something like PetaPoco, or (gasp) ADO.net code. The repository abstraction enables you to use exactly the right tool for the job at hand without leaking these complexities into the client code.
I think there is a big misunderstanding of what many articles call "repository." And that's why there are doubts about what real value those abstractions bring.
In my opinion the repository in it's pure form is IEnumerable, while you and many articles are talking about "data access service."
I've blogged about it here.
This is a discussion that seems to reappear regularly in the SOA world. I heard it as far back as '95, but it's probably been a topic of conversation long before that. I definitely have my own opinions about it, but I'd like to hear some good, solid arguments for having a Data Services Layer, and likewise for arguments against having one.
What value does it add to a systems architecture?
What are the inherent pitfalls?
What are common anti-patterns?
Links to articles are definitely acceptable.
To avoid confusion, this article describes the type of Data Service Layer I'm talking about. Essentially, a thin layer above the database that provides SOAP access to data and includes no business logic.
Data services are quite data oriented, for projects without logic always doing crud. For instance, it can suit if you have a log service or a properties service, you will just do the crud to it.
If the domain that involves that DDBB is complex, with complex logic, you will need to manage that logic up to that service (maybe in an orchestration), so you will divide the logic into several services. In that case I think is better to use a thicker unique service (DAL, BLL and SIL) that manage that domain and expose just one interface.
At the end it is another tool, depend of the problem.