Is SOA solution by making a centralized service which works as a central backbone service for all other services is a good approach?
Its doable, but you need to consider service granularity.
Incorrect granularity could mean that a service covers too much functionality or too little functionality.
Incorrect granularity of services in your SOA can lead to bad performance, low reuse possibilities, leaky abstractions and services without added business value.
Other problems:
You're not able to explain to business people (e.g. sales/marketing) what the service does. It simply does not fit in their understanding of the business the company is in. The services are at a level of detail that is irrelevant from a business perspective.
Many different services are needed to achieve something of value to the business.
There is no clear ownership of the service, multiple department feel they should own the service (or parts of it).
Related
I'm designing a microservice-based architecture for an existing monolithic app. The database has a User table, and then a separate Subscription database storing subscription info for premium, subscribed users.
My initial idea was to do something like User Service <-> Subscription Service <-> Payment Service. But a colleague believes that the subscription service is unnecessary and can just be merged with the User Service. What is the best practice for this?
The problem is analogue with the choosing bounded context boundaries problem in DDD. Choosing bounded context can be done with context mapping: https://www.infoq.com/articles/ddd-contextmapping/ As far as I remember another approach is imagining the organization which does the same job manually what you automate with your services and check what departments it should have. The usual example is a webshop for this where for example security, shopping, storage, invoicing, delivery, support can go in different services. All of them have user and some of them has product, but they watch these things from different viewpoints, e.g. in the shopping context the product is something that you have in the catalogue and what the customers choose to add to their basket, for invoicing it is an item in the order with a certain price at the moment of the ordering, for storage it is something that must be found in the physical world, probably has a container number, for delivery it is part of a package and must be delivered to an address. Many times a lot of properties are just copy-paste with different property names in these bounded contexts and the DRY principle is not met. So if the reason why your colleague wants to merge them is meeting the DRY principle, then it does not matter. If they belong together logically and in a real company a single department would do them, then they can be merged. There is not much detail in your question, but I think I would separate the subscription service too and I guess I would rename the user service to security if all it does is registration and handling logins and authorization. If it contains something else too, then it is better to check the boundaries, maybe some of the properties belong to a different service, maybe there is another service which does not exist yet.
Another practical aspect is scalability. You can wait until certain parts of a single service is used a lot more than other parts and think about splitting because of scalability reasons.
Another analogy is choosing class size in OOP. Some people claim the smaller the better, but everybody is comfortable with different class sizes. Of course there are too small classes and overengineering and too big classes, god objects, but there is a range where the class size is acceptable and it depends on the personal taste or convenience which size you choose. I think for microservices this can happen too, so maybe your colleague is just comfortable with bigger services. If you check the name of the technology the service size should be micro. Those this is relative too, micro compared to what...
In my view a single deployable service should be no bigger than a
bounded context, but no smaller than an aggregate. I would also
suggest that it’s better to start with relatively broad service
boundaries to begin with, refactoring to smaller ones as time goes on.
Perhaps “micro” is a misleading prefix here. These are not necessarily
“small” as in “little”. These are services in the classic SOA mould,
except created with more of an agile mind-set including lightweight
infrastructure, decentralized governance and greater emphasis on
automation. Perhaps it’s these more agile aspects of microservice
design that distinguishes them as opposed to size.
https://www.ben-morris.com/how-big-is-a-microservice/
A Microservice shall be no larger than that which allows a two-pizza
team to release a single complete, appropriately sized user story to
production within a single day.
https://kylegenebrown.medium.com/whats-the-right-size-for-a-microservice-bf1740370d47
Microservice developers can start wherever it “feels right”, and over
time split or merge microservices to improve the architecture.
Although I agree that microservice architectures shouldn’t be carved
in stone and that there should always be the option of changing
microservice decomposition, it’d be nice to have more clarity as to
where a good starting point would be.
Identify critical business transactions that require high performance
and/or solid data integrity and trace those transactions through the
microservices in your architecture. If you see problems, consider
changing microservice boundaries and merging a few microservices
together, as an option.
https://www.pipservices.org/blog/whats-the-right-size-for-a-microservice
Let us imagine we have an orders processing system for a pizza shop to design and build.
The requirements are:
R1. The system should be client- and use-case-agnostic, which means that the system can be accessed by a client which was not taken into account during the initial design. For example, if the pizza shop decides that many of its customers use the Samsung Bada smartphones later, writing a client for Bada OS will not require rewriting the system's API and the system itself; or for instance, if it turns out that using iPads instead of Android devices is somehow better for delivery drivers, then it would be easy to create an iPad client and will not affect the system's API in any way;
R2. Reusability, which means that the system can be easily reconfigured without rewriting much code if the business process changes. For example, if later the pizza shop will start accepting payments online along with accepting cash by delivery drivers (accepting a payment before taking an order VS accepting a payment on delivery), then it would be easy to adapt the system to the new business process;
R3. High-availability and fault-tolerance, which means that the system should be online and should accept orders 24/7.
So, in order to meet R3 we could use Erlang/OTP and have the following architecture:
The problem here is that this kind of architecture has a lot of "hard-coded" functionality in it. If, for example, the pizza shop will move from accepting cash payments on delivery to accepting online payments before the order is placed, then it will take a lot of time and effort to rewrite the whole system and modify the system's API.
Moreover, if the pizza shop will need some enhancements to its CRM client, then again we would have to rewrite the API, the clients and the system itself.
So, the following architecture is aimed to solve those problems and thus to help meeting R1, R2 and R3:
Each 'service' in the system is a Webmachine webserver with a RESTful API. Such an approach has the following benefits:
all goodness of Erlang/OTP, since each Webmachine is an Erlang application, which can be supervised and can be put into an Erlang release;
service oriented architecture with all the benefits of SOA;
easy adaptable to changes in the business process;
easy to add new clients and new functions to clients (e.g. to the CRM client), because a client can use RESTful APIs of all the services in the system instead of one 'central' API (Service composability in terms of SOA).
So, essentially, the system architecture proposed in the second picture is a Service Oriented Architecture where each service has a RESTful API instead of a WSDL contract and where each service is an Erlang/OTP application.
And here are my questions:
Picture 2: Am I trying to reinvent the wheel here? Should I just stick with the pure Erlang/OTP architecture instead? ("Pure Erlang" means Erlang applications packed into a release, talking to each other via
gen_server:call and gen_server:cast function calls);
Can you name any disadvantages in suggested approach? (Picture 2)
Do you think it would be easier to maintain and grow (R1 and R2) a system like this (Picture 2) than a truly Erlang/OTP one?
The security of such a system (Picture 2) could be an issue, since there are many entry points open to the web (RESTful APIs of all services) instead of just one entry point (Picture 1), isn't it so?
Is it ok to have several 'orchestrating modules' in such a system or maybe some better practice exists? ("Accept orders", "CRM" and "Dispatch orders" services on Picture 2);
Does pure Erlang/OTP (Picture 1) have any advantages over this approach (Picture 2) in terms of message passing and the limitations of the protocol? (partly discussed in my previous similar question, gen_server:call VS HTTP RESTful calls)
The thing to keep in mind regarding SOA is that the architecture is not about the technology (REST, WS* ). So you can get a good SOA in place with endpoints of several types if/when needed (what I call an Edge component - separating business logic from other concerns like communications and protocols)
Also it is important to note that service boundary is a trust boundary so when you cross it you may need to authenticate and authorize, cross network etc. Additionally, separation into layers (like data and logic) shouldn't drive the way you partition your services.
So from what I am reading in your questions, I'd probably partition the services into more coarse grained services (see below). communications within the boundary of the service can be whatever, where-as communications across services uses a public API (REST or Erlang native is up to you, but the point is that it is managed, versioned, secured etc.) - again, a service may have endpoints in multiple technologies to facilitate different users (sometimes you'd use an ESB to mediate between services and protocols but the need for that depends on size and complexity of your system)
Regarding your specific questions
1 As noted above, I think theres a place to expose more public APIs
than just a single entry point, I am not sure that exposing each and
every capability as a service with public-api is the right way to go
see.
2&3 The disadvantages of exposing every little thing is
management overhead, decreased performance (e.g. you'd have to
authenticate on these calls). You get nano-services services
whose overhead is more than their utility.
One thing to add about the security is that the fact that some service has a REST API does not have to translate to having that API available to the general public. Deployment-wise you can keep it behind a firewall and restrict the access to it for known addresses
etc.
5 It is ok to have several orchestrating modules, though if you get beyond a few you should probably consider some orchestration module (and ESB or an Orchestration engine) alternatively you can use event based integration and get choreography based integration which is more flexible (but somewhat less manageable )
6 The first option has the advantage of easy development and probably better performance (if that's an issue). The hard-coded integration layer can prove harder to maintain over time. The erlang services, if you wrote them write should be able to evolve independently if you keep API integration and message passing between them (luckily Erland makes it relatively easy to get this right by its inherent features (e.g. immutability))
I'd introduce the third way that is rather more cost effective and change-reactive. The architecture definitely should be service oriented because you have services explicitly. But there's no requirement to expose each service as Restful or WSDL-defined one. I'm not an Erlang developer but I believe there's a way to invoke local and remote processes by messaging and thus avoid unnecessary serialisation/serialisation activities for internal calls. But one day you will be faced with new integration issue. For example you will be to integrate accounting or logistic system. Then if you designed architecture well regarding SOA principles the most efforts will be related to exposing existing service with RESTful front-end wrapper with no effort to refactor existing connections to other services. But the issue is to keep domain of responsibilities clean. I mean each service should be responsible to the activity it was originally designed.
The security issue you mentioned is known one. You should have authentication/authorization in all exposed services using tokens for example.
Does Restful Webservices have any service registries like the UDDI? Or can UDDI hold Restful Webservices as well?
UDDI can be used for REST services. WSDLs can be used to described HTTP web services, but frankly I feel that it's not a real match for a REST resource architecture.
At a most basic level, UDDI is simply mapping of attributes to service endpoints. So, if you're simply looking for a system that can do that, then UDDI will fit the bill.
UDDI is not popular in the wild, wide open internet, but it is used "behind the scenes" as an orchestration component.
As Darrel mentioned, DNS is another valid discovery mechanism.
My personal complaint with DNS is simply that even though DNS has all of the advantages that's mentioned in the article he cites, the downside is that DNS is such a critical part of the network fabric, it tends to not be available to developers. Typically, the network operations folks (who tend to be more notorious than even DBAs) hold infrastructure like DNS quite close. Finally, while DNS is quite capable of these tasks, in many cases the standard default configuration and deployment of DNS may need to be changed. For example, We've started serving certificates from DNS, for example, and we had to enable TCP for DNS. Again, this meant more involvement of network ops.
On top of that, while there is a lot of expertise and knowledge of DNS out in the world, knowledge and expertise of HTTP and "doing stuff" on a web server is far greater. That consequences of that simply means that when developers think about and look to some kind of solution to this problem, the first place they're going to look is likely an HTTP based solution.
So, in that sense UDDI is possibly a better solution, just in terms of being able to get it rolled out quickly with little hassle.
Of course, UDDI is a SOAP based service. That's not that big a deal, really. Not a great fit for a RESTful system, but it's not awful. Functional, if a little "impure".
As for a standard HTTP based service registry, there's nothing that I know of. It's reasonably simply to adhoc one simply with HTML, for example. The fact that UDDI hasn't taken off in the World at large isn't so much a limitation or slight against UDDI. Rather it's simply that the vision of discovering arbitrary services hasn't really come to fruition, the need simply isn't quite there. There's a lot more involved out of band with service discovery beyond location and semantics, like business relationships and such.
Internally, within the enterprise, those logistics are solved, so service discovery has value. Out in the wild, not so much.
It's not dead ;)
signed a jUDDI developer
juddi.apache.org
Edit: There's also WS-Discovery which is supported by both CXF and WCF. Worth checking out.
FWIW, UDDIv3 does specify a REST interface, however I don't think anyone other than jUDDI has implemented it. It will be included with v3.2 and up using CXF, Jettison and WADL. Source: http://svn.apache.org/repos/asf/juddi/trunk/juddi-rest-cxf/src/main/java/org/apache/juddi/api/impl/rest/UDDIInquiryJAXRS.java
UDDI was designed for SOAP services, however, it is not even used for those any more. UDDI was pretty much dead as of 2006.
This article shows how to use DNS to do discovery.
It seems that SaaS and Cloud computing are old concepts with new names, and I am curious if I am wrong.
For cloud computing you can look at: Difference between cloud computing and distributed computing?
Basically, it seems that when we have been hosting that that is cloud computing, it is just that now some companies have put in much great resources to ensure better uptime than my local ISP. But, it seems that there is nothing really new here.
For REST, it seems that it is what we have been doing with cgis for 15 years.
Here is a question on REST: What am I not understanding about REST?
It appears that REST is an old concept, and I am curious how it is different than has been done since the early days of the web, and, to a large extent, the early days of using telnet (which http is on top of).
Am I mistaken in my simplification of these? I try to see how what is new is like what I know so I can see what more has to be learned in that topic, but for cloud computing and REST it seems that very little needs to be learned.
You are both right and wrong. You are right in the sense that new ideas are normally similar to old ideas, and indeed cloud computing is based significantly on distributed computing.
What is new in cloud computing is
virtualization
self-service
With virtualization, you can run multiple operating systems on a single hardware. While that, in itself, isn't new, either, it was never considered in distributed systems as a relevant piece of the architecture. Using virtualization allows self-service: users can create their own clusters of nodes without the administrator of the hardware taking any action. This allows a significant acceleration of deployment, and a significant reduction of cost.
For ReST, what you are missing is the client API. It is true that on the server side, a ReST service can be implemented with CGI. What is new here is that it is not an end user which retrieves the URL, but a program.
Saying that HTTP is on top of telnet ignores realities; this is like saying that we made no progress since the introduction of copper wires for communication. Strictly speaking, HTTP is not in top of telnet, but on top of TCP (which telnet is also on top of, these days).
Considering Roy's dissertation coined the term REST back in 2000, you can definitely argue that there is nothing new about REST. Additionally, the REST architectural style was synthesized from successful existing practices, so REST implementations pre-date the definition. Having said that, there is nothing simple about designing REST interfaces. Ever since Netscape first abused cookies to allow servers to maintain session state people have been swimming upstream against the web.
REST's recent resurrection has come mainly from people becoming disillusioned with SOAP based Web Services. SOAP tried to hide HTTP instead of embracing it and I think people are starting to realize how effective HTTP can be as an distributed application protocol that can do more than just deliver HTML to web browsers.
RESTful web applications don't use session state, so one could argue that by that virtue alone it is different than most web applications in existence at the moment.
As for Cloud Computing, I find myself agreeing with Larry Ellison for once in my life.
I'm in agreement on what you've posted. You might consider making this community wiki since it's likely to garner many answers based on opinion. Cloud computing seems to have taken off as a buzzword, and this is largely due to a decrease in cost for mass quantities of hardware. And then there is REST which is really just a formal name and definition for something that has been in place for a long time. Some people like to encapsulate ideas with buzzwords and acronyms. Sometimes it's useful to put a name to an idea though.
Not only this, the concept of things being old concepts with new names is old. It's hard to be original these days :P
You are right about REST -- its mostly old concepts with a lot of added pedantry and not much added substance.
Cloud computing has a small but fundamental difference from distributed computing. In distributed computing you had servers dedicated to particular functions, and usually some sort of directory service to locate the correct server. In cloud computing any server is capable of any task and usually the servers queue up for work which is distributed from a central point.
I was involved in couple of cloud computing platform recently.
First of all please note that I am not trying to criticize any platform.
Cloud computing is large area but to make my point simple and understandable. Let me come up with very simple scenario and that is data storage services hosted on the cloud.
If you take any storage service like Amazon EC2, SQL Data Service(SDS), Salesforce.com services.
If you want to consume any of such data storage service platform goal of all such service are same and that is to serve requested data on demand. Without warring about how it store and where it stored and who is maintaining it etc... (all cloud goodies)
Now my area of concern is the way ANSI-SQL regulated platform venders to make sure they follow similar language across all the product can’t they regulate similar concept across
service providers?
Why no such initiatives??
Any thoughts appreciated
It seems to me like you're worried about vendor lock-in with cloud computing. I may be naive but I would normally choose technologies and then go look for cloud vendors that'd be able to deliver these technologies. And if I was aiming for a "write once run anywhere approach" I'd have to select technology that'd make this as realistic as possible.
With the fairly rapid speed of development I really think standardization committees would struggle to keep up. ANSI-SQL has had 20 + years of history. It seems to me like you're requesting for standardization long before we even know what the cloud is up to....
I think that this emerging cloud computing initiative is just too young in order to have standards.
Service providers right now just worry about rushing into the market, rather than interoperability and standards.
Later on, when the situation is more established, some common guidelines may emerge. But there is still a long way to go.
You seem to be asking specifically about cloud storage services, rather than cloud computing in general. So your Amazon example would be S3, not EC2.
I think the field is a little young to be standardising on an API just yet. The services differentiate themselves in ways which rule this out. For example, S3 trades sophistication for scalability/reliability/performance: you can't do a complex SQL LIKE query. You can store and retrieve blobs of data based on a key, and that's about it.
I think as such services become more and more the mainstream way to do things, standards will emerge. Users will want the freedom to switch providers on a whim, move their data around, test against free local storage, etc.
The APIs used are all based on Web Standards already. Making an abstraction layer to make them look the same is fairly trivial.