My team is torn with how we should architect our microservices with using a message bus.
We currently have an API Gateway with many microservices behind it all communicating over http.
After looking into implementing Message Buses (Kafka) the team is torn on whether the consumer and API should live in the same service or if they should be two separate services.
Some think they should be separate as they have different scaling concerns, while others think they should be in the same service since they are communicating with the same database and have the same domain concerns. IE) Not duplicating code between two services.
What are your thoughts?
We prefer them to be in the same service as they logically are doing work on same objects.
This also highly depends upon how you write your business logic. Like we prefer here to write our business logic in Aggregates(Domain Driven Design) and that's why writing consumer in the same service makes sense.
In some case where they are just updating data for searching kind of things , you may write them in separate service.
You can also look at Lagom (microservices framework for java)
I am learning about microservices and I don't understand what the real difference
between creating a REST API and creating microservices?
I’m working in Go, but my question applies over all languages.
The Microservices approach is about breaking your system ("pile of code") into many small services, each typically has its own:
Clear business-related responsibility
Running process
Database
Code version control (e.g. git) repository
API (the protocol how other services / clients will contact the Microservice)
UI
The services themselves are kept small so as your system grow, there are more services - rather than larger services.
Microservices can use REST, RPC, or any other method to communicate with one another, so REST or an API is really orthogonal to the topic of microservices...
Reference: What is an API? In English, please.
API = Application Programming Interface
Microservices = an architecture
In short
Microservice should expose a well-defined API.
Microservice is the way you may want to architect your solution
API is what your consumers see.
You can expose API without microservices in the backend (in fact, most non-training scenarios don't require microservices).
You may want to read http://samnewman.io/books/building_microservices/ before you decide on using microservices (unless this is for training purposes).
Microservice is well defined when you are following SOC - seperation of Concern on the entity/domain level ,where each entity / domain are independent of any other service.
for example user service will only be responsible for storing, updating and deleting user related informations.
Microservice backend and frontend microservice can further be splitted in 2 parts
frontend microservice which exposes rest endpoint just like Web API
backend microservice which actually perform all the operations.
Rest API is more of endpoints exposed to outer world and can be used with microservices as well, as explained above.
The majority of the answers is based on the old-school understanding of API as a programmatic interface. Nowadays, this meaning is melted and start confusing people becuase some developers started (for simplicit or by mistake) interpred the API of an application as the application per se. In such case, it is impossible to distinguish between the modern API and Microservices. Nonetheless, we can say that an API-application can comprise many Microservices, the most of which interact within the application via Microservice's APIs while others may expose their APIs as Applications's APIs. Also, a Microservice (as a service) may not include other Microservices (services), but may orchestrate a composition of Microservices via API-bases invocations. Applications may contain Microservices but, in the best practices, may not contain other Applications.
Microservices
A microservice architecture is about slicing an application logic into small pieces or "components" that can act between them and/or be exposed through an API.
API
A (web) application need to design the business logic with all set of object entities (model) and possible operations on them.
An (Application Programming Interface][https://en.wikipedia.org/wiki/Application_programming_interface) is a way of making the requests to an application by exposing specific entry-points that are in charge of invoking the appropriate application operations.
ReST(ful) APIs
("ReST" as in Representational State Transfer) are APIs complying with at least these 5 constraints:
User-interface is distinct from data storage and manipulation (Client-Server architecture)
No client context is stored on the server ("stateless")
Server responses must, implicitly or explicitly, define themselves as cacheable or not
Client does not have to be aware of the layers between him and the server
Response/request messages must be: be self-descriptive; allow to identify a resource; use representations allowing to manipulate the resources; announce available actions and resources ("Uniform interface").
"The real difference"
So, while these notions are obviously related, they are clearly distinct concepts:
Being ReSTful or not, an API exposes operations provided by a server that might (but not necessarily) be shelled into smaller components (microservices).
Also, while a typical web (ReST)API uses the HTTP protocol between the client and the server, components within a microservice architecture might communicate using other protocol(s) (e.g. WAMP, AMQP, JSON-RPC, XML-RPC, SOAP, ...)
In layman's term, if you have a web API server and you split them into several independent mini servers, use a proxy-server and load-balancer to clusterize them, and (optionally, give each a separate database entity), that is a microservice architecture.
In designing a platform infrastructure, I am creating several services that are
used internally. These services might use binary protocol messages to exchange
information. Down the line, I may want to expose these services externally, as
a RESTful service over http.
Are there design patterns or techniques that can ease this process of
converting an internal service to an external one?
The usual way is to organize the software into logical layers according to Multilayered Architecture. More specifically, Service Layer design pattern described by Martin Fowler captures the essence what you are trying to achieve.
This way you will have an inventory of internal services that are then used from the upper layers e.g. user interfaces, REST gateways, web services etc.
In the book The Java EE Architect's Handbook Mr. Ashmore describes the same concept using the term deployment layer that sits on top of the service layer.
What is the difference between an SOA service and other kinds of services like an application or domain service ?
Have a look here. http://www.bennadel.com/blog/2385-application-services-vs-infrastructure-services-vs-domain-services.htm
Short answer
DDD Domain Services operate on Domain Entities. Usually where the work that needs to be done spans multiple Aggregate roots.
DDD Application Services drives workflow. For example if you want to do some work on a domain entity, the Application Service would be responsible to fetch the entity from the data store, call the domain service to do the work, do some work via an integration service if needed, and then lastly persist the change.
This is an interesting question since SOA is such as broad and overloaded term.
If we take SOA to mean any implementation that results in a mechanism to reach 'services' then even application and domain services will form part of SOA services. Application and domain services will even fall within the realm of micro-services although application services are usually surfaced through some integration mechanism.
I like to think of these things in terms of 'reachability'. WikiPedia:
In graph theory, reachability refers to the ability to get from one vertex to another within a graph
So, it depends on how reachable your code is. A bunch of domain services could, theoretically, form a service-oriented architecture.
The only differences is in how you surface your services.
I am in the process of rewriting the architecture for an existing application that will be hosted on Azure and have at least two API's - one public, and the other private for internal RESTful-related CRUD calls.
The main technology stack is ODATA, WebAPI2, C#, MVC5, EF, AngularJS. My thoughts were that the RESTful endpoints would all be exposed through the Azure Web Role (just like any other URL).
In reading more about OWIN and using it with the WebAPI and the Azure Worker Role, it seems like exactly the same thing, but as a service.
Is there any advantage to going this OWIN route (where come to think of it, I could even write a WCF service to do the same thing) rather than the web-based RESTful API calls?
I'm looking for some reasons why I should go through the exercise of moving to a service-based API.
Technology speaking
For your public API, I would suggest to do it with Owin in a RESTful way. This will create a facade to your internal systems.
Internally, WCF will offer better performances.
PROS
Maintenance is way easier with a good plan. Because of its decentralized design, the risks due to refactoring are contained in a specific context. Maintenance is all about minimizing and controlling risks related to changes.
Those links are more talking about SOA but I'm sure you can extract the information needed from them:
Services by Contracts
Amazon SOA
Amazon Shareholders Letter
CONS
You should be aware though since it also has drawbacks. You will need solid Unit Test, Logging and Versioning strategies.
Hope this helps a little. I tried to keep only the essentials since a complete answer to this question could be published as a book.