For a long time, when it comes to the microservice architecture, NATS and Kafka are the first options that come to my mind. But recently I found this gRPC template in dotnet core and that grasped my attention. I read a lot about it and watched a lot of videos but I don't think any of those could address gRPC correctly as they usually contrast between gRPC and message brokers or protocols such as REST which I guess is pretty inappropriate although SOAP would be relevant here.
My assumption is that gRPC is a modern version of SOAP with better performance and less implementation hassle due to it protocol buffer. And I think that gRPC can by no means be compared against Kafka or NATS. And also that it cannot replace RESTful service as neither could SOAP.
Now, the question, to what extent are my assumptions true? For example, when it comes to selecting a communication bridge between nodes on a cluster, do I have to put gPRC among my options now (NATS, Kafkam Rabbit, etc) or should I consider that when creating a web proxy to bridge external request to my microservices?
Finally, how about real-time communication, can gRPC replace websocket/socket.io/signalR completely? What does it replace?
I often see people misplacing these technologies by one crucial aspect: public authentication.
For instance, check this graph:
This is a benchmark of Inverted Json (https://github.com/lega911/ijson), comparing some tools, such as iJson, RabbitMQ, Nats, 0MQ, etc.
Notice that Nats, ZeroMQ and iJson are not meant to be used as public end-points (for instance, Nats have user/password, token and keys, but it is useless in an open environment, such as web browsers, because there is no way to make the key non public).
On the other hand, GRPC works just fine with JWT and Oauth2, making it completely safe to public end-points (as safer as any other HTTP endpoint), 'cos those tokens are server-signed (so, even tough they are public, they can't be forged or tempered with)
So, what I'm trying to say is: there are techs meant to face public and techs meant to glue together servers and process within servers (which are private connections).
GRPC is public, ZeroMQ and iJson are totally private (iJson, for instance, don't have any kind of authentication). Nats works with keys or passwords, so, although is "safer" than iJson and ZeroMQ, it is not meant to be public.
When you say REST (I'm assuming HTTP here, because REST is just an architecture), websocket/socket.io/signalR, you are depicting all public interfaces. GRPC will cover you here (it's comparable to REST as request/response and websocket/socket.io/signalR because it supports half and full duplex streaming (similar to sockets)).
Nats, iJson, ZeroMQ, on the other hand, are not meant to do that. They are meant to communicate between services.
So, basically, REST/websocket/socket.io/signalR = gRPC.
Internal communication between services (in the same or in different servers) = NATs, iJSON, ZeroMQ.
(notice that I'm not even considering the other technologies in the graph, because they are products, IMO, not simple libraries you can use to achieve an end, such as RabbitMQ, nginx, etc. The other ones I'm not familiar enough to be able to make an opinion (but I'm surprised by the uvloop in that graph)).
Your intuition is correct that gRPC is not comparable to an asynchronous queueing system like kafka, Rabbit, etc.
It is however a replacement for synchronous server to server communication technologies often implemented over SOAP, RPC, REST, etc. where you are expecting to get a response from another server rather than firing a message into a queue and then effectively forgetting about the message.
gRPC is definitely an option for real-time communication. It can replace socket communication if you are not streaming to the browser(No gRPC support), have a look at the Bidirectional streaming support.
About replacing Kafka/Rabbit, gRPC can be used as a PubSub system as it supports Bidirectional streaming but I would not recommend it.
Related
If I have a static site on AWS S3 (and maybe using CloudFront) that's pretty cool, because it scales easily, and has zero-downtime deployments, because you're just updating static assets, and gets distributed to edge locations, woohoo!
But, if I wanted to have a little live chat support feature using Socket.io on the contact page, how would I tell Amazon to deal with Websockets? Could I use Route53 to do something different with Websocket requests to a particular domain, like redirect them to Lambda? (Lambda can't run socket.io can it?)
Similar to your other question, the answer here involves the fact that DNS is not involved in path resolution, so Route 53 is not a factor in this question.
Socket.io is almost certainly going to require a server and connecting through CloudFront seems unlikely.
Although I am not versed in socket.io's underlying transport protocol(s?), I don't see a way around this. CloudFront is a reverse proxy that only supports proper, standard HTTP request/response behavior, which is not well-suited to real-time event-oriented operations. CloudFront does not support websockets, though Socket.io may not need them and may have the flexibility to fall back to a compatible behavior but it will -- at best -- be suboptimal if possible at all, because even with long-polling (inefficient), you're limited to under 30 seconds for a single response, because CloudFront has a fixed timeout timer of 30 seconds, that cannot be modified.
Similarly, Lambda functions accessed through API Gateway are suited only to handling a single HTTP request/response cycle, not anything persistent and no intrinsic mechanism for handling "state" across requests.
My assumption going in would be that you'd need one or more servers behind an ELB Classic Load Balancer with SSL and operating in TCP mode, using a subdomain of your site's domain, connecting browsers to the back-end for persistent connections.
Even if this answer is helpful, I'm honestly not certain that it is sufficiently helpful... so you may wish to hold off on accepting it, since someone may come along and offer an answer that delves more deeply into the internals of socket.io and how that is going to interoperate with CloudFront, if such interoperation is possible.
A little background.
Very big monolithic Django application. All components use the same database. We need to separate services so we can independently upgrade some parts of the system without affecting the rest.
We use RabbitMQ as a broker to Celery.
Right now we have two options:
HTTP Services using a REST interface.
JSONRPC over AMQP to a event loop service
My team is leaning towards HTTP because that's what they are familiar with but I think the advantages of using RPC over AMQP far outweigh it.
AMQP provides us with the capabilities to easily add in load balancing, and high availability, with guaranteed message deliveries.
Whereas with HTTP we have to create client HTTP wrappers to work with the REST interfaces, we have to put in a load balancer and set up that infrastructure in order to have HA etc.
With AMQP I can just spawn another instance of the service, it will connect to the same queue as the other instances and bam, HA and load balancing.
Am I missing something with my thoughts on AMQP?
At first,
REST, RPC - architecture patterns, AMQP - wire-level and HTTP - application protocol which run on top of TCP/IP
AMQP is a specific protocol when HTTP - general-purpose protocol, thus, HTTP has damn high overhead comparing to AMQP
AMQP nature is asynchronous where HTTP nature is synchronous
both REST and RPC use data serialization, which format is up to you and it depends of infrastructure. If you are using python everywhere I think you can use python native serialization - pickle which should be faster than JSON or any other formats.
both HTTP+REST and AMQP+RPC can run in heterogeneous and/or distributed environment
So if you are choosing what to use: HTTP+REST or AMQP+RPC, the answer is really subject of infrastructure complexity and resource usage. Without any specific requirements both solution will work fine, but i would rather make some abstraction to be able switch between them transparently.
You told that your team familiar with HTTP but not with AMQP. If development time is an important time you got an answer.
If you want to build HA infrastructure with minimal complexity I guess AMQP protocol is what you want.
I had an experience with both of them and advantages of RESTful services are:
they well-mapped on web interface
people are familiar with them
easy to debug (due to general purpose of HTTP)
easy provide API to third-party services.
Advantages of AMQP-based solution:
damn fast
flexible
cost-effective (in resources usage meaning)
Note, that you can provide RESTful API to third-party services on top of your AMQP-based API while REST is not a protocol but rather paradigm, but you should think about it building your AQMP RPC api. I have done it in this way to provide API to external third-party services and provide access to API on those part of infrastructure which run on old codebase or where it is not possible to add AMQP support.
If I am right your question is about how to better organize communication between different parts of your software, not how to provide an API to end-users.
If you have a high-load project RabbitMQ is damn good piece of software and you can easily add any number of workers which run on different machines. Also it has mirroring and clustering out of the box. And one more thing, RabbitMQ is build on top of Erlang OTP, which is high-reliable,stable platform ... (bla-bla-bla), it is good not only for marketing but for engineers too. I had an issue with RabbitMQ only once when nginx logs took all disc space on the same partition where RabbitMQ run.
UPD (May 2018):
Saurabh Bhoomkar posted a link to the MQ vs. HTTP article written by Arnold Shoon on June 7th, 2012, here's a copy of it:
I was going through my old files and came across my notes on MQ and thought I’d share some reasons to use MQ vs. HTTP:
If your consumer processes at a fixed rate (i.e. can’t handle floods to the HTTP server [bursts]) then using MQ provides the flexibility for the service to buffer the other requests vs. bogging it down.
Time independent processing and messaging exchange patterns — if the thread is performing a fire-and-forget, then MQ is better suited for that pattern vs. HTTP.
Long-lived processes are better suited for MQ as you can send a request and have a seperate thread listening for responses (note WS-Addressing allows HTTP to process in this manner but requires both endpoints to support that capability).
Loose coupling where one process can continue to do work even if the other process is not available vs. HTTP having to retry.
Request prioritization where more important messages can jump to the front of the queue.
XA transactions – MQ is fully XA compliant – HTTP is not.
Fault tolerance – MQ messages survive server or network failures – HTTP does not.
MQ provides for ‘assured’ delivery of messages once and only once, http does not.
MQ provides the ability to do message segmentation and message grouping for large messages – HTTP does not have that ability as it treats each transaction seperately.
MQ provides a pub/sub interface where-as HTTP is point-to-point.
UPD (Dec 2018):
As noticed by #Kevin in comments below, it's questionable that RabbitMQ scales better then RESTful servies. My original answer was based on simply adding more workers, which is just a part of scaling and as long as single AMQP broker capacity not exceeded, it is true, though after that it requires more advanced techniques like Highly Available (Mirrored) Queues which makes both HTTP and AMQP-based services have some non-trivial complexity to scale at infrastructure level.
After careful thinking I also removed that maintaining AMQP broker (RabbitMQ) is simpler than any HTTP server: original answer was written in Jun 2013 and a lot of changed since that time, but the main change was that I get more insight in both of approaches, so the best I can say now that "your mileage may vary".
Also note, that comparing both HTTP and AMQP is apple to oranges to some extent, so please, do not interpret this answer as the ultimate guidance to base your decision on but rather take it as one of sources or as a reference for your further researches to find out what exact solution will match your particular case.
The irony of the solution OP had to accept is, AMQP or other MQ solutions are often used to insulate callers from the inherent unreliability of HTTP-only services -- to provide some level of timeout & retry logic and message persistence so the caller doesn't have to implement its own HTTP insulation code. A very thin HTTP gateway or adapter layer over a reliable AMQP core, with option to go straight to AMQP using a more reliable client protocol like JSONRPC would often be the best solution for this scenario.
Your thoughts on AMQP are spot on!
Furthermore, since you are transitioning from a monolithic to a more distributed architecture, then adopting AMQP for communication between the services is more ideal for your use case. Here is why…
Communication via a REST interface and by extension HTTP is synchronous in nature — this synchronous nature of HTTP makes it a not-so-great option as the pattern of communication in a distributed architecture like the one you talk about. Why?
Imagine you have two services, service A and service B in that your Django application that communicate via REST API calls. This API calls usually play out this way: service A makes an http request to service B, waits idly for the response, and only proceeds to the next task after getting a response from service B. In essence, service A is blocked until it receives a response from service B.
This is problematic because one of the goals with microservices is to build small autonomous services that would always be available even if one or more services are down– No single point of failure. The fact that service A connects directly to service B and in fact, waits for some response, introduces a level of coupling that detracts from the intended autonomy of each service.
AMQP on the other hand is asynchronous in nature — this asynchronous nature of AMQP makes it great for use in your scenario and other like it.
If you go down the AMQP route, instead of service A making requests to service B directly, you can introduce an AMQP based MQ between these two services. Service A will add requests to the Message Queue. Service B then picks up the request and processes it at its own pace.
This approach decouples the two services and, by extension, makes them autonomous. This is true because:
If service B fails unexpectedly, service A will keep accepting requests and adding them to the queue as though nothing happened. The requests would always be in the queue for service B to process them when it’s back online.
If service A experiences a spike in traffic, service B won’t even notice because it only picks up requests from the Message Queues at its own pace
This approach also has the added benefit of being easy to scale— you can add more queues or create copies of service B to process more requests.
Lastly, service A does not have to wait for a response from service B, the end users don’t also have to wait for long— this leads to improved performance and, by extension, a better user experience.
Just in case you are considering moving from HTTP to AMQP in your distributed architecture and you are just not sure how to go about it, you can checkout this 7 parts beginner guide on message queues and microservices. It shows you how to use a message queue in a distributed architecture by walking you through a demo project.
I am planning to develop a web based chat application which takes in ReSTful requests, translate them to XMPP and deliver them to an XMPP server.
Using websockets for this kind of chat based application looked promising as the events (or responses) can be delivered asynchronously. But if I use websockets as the underlying protocol for transferring the requests from the browser, can this still be considered as a ReSTful design? If yes, how are the URIs, verbs (GET, POST...), parameters represented in the websocket message? Wrap them in an xml/json and send it?
Also, ReSTful architecture states that no session state will be stored on the server. But here in this case when an XMPP client session is created, the state of this session will be stored on the server (violating the stateless constraint)
REST is an architectural style that does not impose a protocol. So yes, you can do REST with Web Sockets, REST with HTTP and REST with FTP if you like.
The main reason to use HTTP is that it is easy and fairly simple to communicate with any component or programming language via HTTP and also because HTTP supports distributed environments with multiple intermediaries: proxies, firewalls...; So you can deploy your service on any topology and anyone will be able to access it.
My rant:
If you are a RESTliban and Roy Fielding’s dissertation is the source of truth, verbs are never acknowledged as part of the semantic. URIs are the semantic. The usage of different verbs for different actions has been an elegant evolution of REST over HTTP, but not part of the "truth". You can check the scenario of rest over HTTP evaluated by Roy in chapter six of his dissertation. No mention to verbs. And notice it is an evaluation scenario, not the specification.
TLDR;
If you need realtime two way communications via the internet and the client is a web browser, the best choice is Web Sockets. You could then implement an application level protocol on top of web sockets to implement a RESTful Web Service.
Yes. You can use REST over WebSocket with library like
SwaggerSocket.
Why would you want to build a REST API on top of socket? IMHO the benefit of a REST API is to leverage standard HTTP protocol possibilities like stateless requests, semantic verbs like GET, DELETE to build an API that can be easily understood by (client) developers. Since sockets do not offer HTTP verbs and so on, you would build some kind of HTTP layer for sockets which is IMHO not reasonable.
In case you would really build such a thing, I'd recommend to use the HTTP protocol as a blueprint and implement the socket protocol like HTTP.
REST architectural style mostly presumes 2 entities viz. client and server.
As we move more towards real time web and development of reactive systems WebSocket would prominently start replacing usage of REST API's.
WS allows data push and pull which dismisses the concept of server and client.
STOMP,AMQP ,XMPP can be used as messaging protocols.
The data itself maybe JSON or Google protocol buffers or maybe Apache Avro.
WebSockets is not tied to web servers but can be developed in stand alone apps like mobile apps or desktop apps too.
I don't understand why you would convert XMPP into REST and then run REST over WS. The point of WebSocket is to take the XMPP protocol directly to the browser, thereby avoiding all of the translation issues.
There are JavaScript libraries that can talk XMPP from the browser to the server. All you need is to proxy the XMPP traffic from WS over into TCP and then straight into your XMPP server. Kaazing has a gateway that allows you to do this.
If you want to use open source, you will need to write a JavaScript XMPP library. There are examples that show how to write a JS library for simple protocols. You just have to find one and extend the concept to the XMPP protocol.
So to recap, here are the way the architecture would look:
Your XMPP Client code <-> XMPP JavaScript Library <-> WebSocket over http <-> WebSocket to TCP Proxy <-> XMPP Server
where the XMPP Client code and the XMPP JavaScript Library runs in the browser, and the WS to TCP proxy along with the XMPP server are all server-side.
I understand this post is really old, but wanted to interject a bit further on the notion that "So if I choose a REST architecture I forfeit the ability for real-time communications?".
In a word, no. A number of REST style implementations I have had experience with leverage REST for compatibility, discoverability, and as a means to scale across different devices in the shadow of IoT.
However, in addition to using WS in addition to REST to facilitate near real-time transmission. There are also a number of abstractions which really help with this and allow you to focus on building your API and deciding how the RT components of the consuming applications should operate.
I would suggest taking a look at things like Tibco Smart-Sockets, or SignalR if you're looking to build a REST API and would like to avoid re-creating the wheel for your RT needs.
I created a project that adds callbacks to the web socket send function: https://github.com/ModernEdgeSoftware/WebSocketR2
Message IDs are established so the client can implement callbacks. It handles message retries after timeouts as well as reconnects to the server if the connection gets dropped. You can then structure you payload to be as RESTful as you want by adding verbs and paths.
This is similar to when a video game studio uses UDP to achieve the speeds they need, but their net code implements a lot of TCP like features for reliability.
The OP's original question is: "Is ReST over websockets possible?"
What this question implies is the following: Is REST API possible over Websockets as a transport.
Of course, OP did not mean the following: Is REST architectural style possible over Websockets. His question was more an operational one i.e. can REST API requests, such as GET, PUT, POST, DELETE etc. be exchanged over a Websockets pipe.
To answer this question, we have to understand that both sockets and Websockets are the same type of interface (full duplex, 3-way handshake protocol), but the difference is that sockets interface originated in ARPANET reference model. In that network model, sockets were an interface between Session layer and Transport layer. The word "interface" means that it resides "in between" network layers, i.e. within their boundary. In other words, sockets are not part of any specific network layer.
Websockets are the same type of socket interface, but in OSI 7-layer network model they no longer reside in between Session and Transport layers. Instead, they reside in between Session layer and Presentation layer. Why there? Why this "move"? A motivation for this was to be able to leverage HTTP protocol as a transport for sockets. And what is so special about HTTP protocol? In enterprise establishments, there are a lot of network zones and segments and these security domains are protected by firewalls. And firewalls, as we know, have associated rules for inbound/outbound traffic. If we want two components in two different network zones to talk to each other, we have to ensure that ports on corresponding firewalls are open. That would involve collaboration of infrastructure, operations teams, business approvals etc. and would introduce significant delays in achieving a simple thing: two components communicating with each other.
Which brings us to our use case: Websockets interface placed between Session OSI layer (where HTTP resides) and Presentation OSI layer (where things like TLS reside). By default, port 80 is open on all firewalls thus no involvement of operations and infrastructure is needed. And our two components can now converse over Websockets communication pipe.
Back to the OP's question. Any type of a string list can be transferred over sockets. Sockets/Websockets are an ideal mechanism for transporting all sorts of custom protocols, whether they are STOMP, HL7, FHIR, or many others. GET, PUT, POST, DELETE requests are different operations on a REST API endpoint. These operations are in the form of a specific string list, and as we saw, sockets/Websockets are very convenient for passing string lists back and forth. In the case of REST over HTTP, though, you are leveraging the whole HTTP "infrastructure" available in all modern Browsers, such as Chrome, Firefox, Edge etc., as well as web servers such as Apache, nginx, IIS, OHS, IHS etc. In other words, REST API piggybacks on an established, string list-based protocol called HTTP that is built-in (part of) both clients and servers' sides. This cannot be said about Websockets. You would have to ensure every type of client and server complies with your (custom) transport solution based on Websockets!
I just spot new topic on the blog of one company who providing cloud solution and "Server-end/Service as a Platform" (SaaS) for games.
I'm not advertising this company, nor I used them, so I don't even know how good or bad they are.
However, they very clearly explain reasons and what are the benefits of using WebSockets in REST
Have a read on their blog
REST requires a stateless protocol according to the statelessness constraint, websockets is a stateful protocol, so it is not possible.
To cut a long story short, I am working on a project where we are rewriting a large web application for all the usual reasons. The main aim of the rewrite is to separate this large single application running on single server into many smaller decoupled applications, which can be run on many servers.
Ok here's what I would like:
I would like HTTP to be the main transport mechanism. When one application for example the CMS has been updated it will contact the broker via http and say "I've changed", then the broker will send back a 200 OK to say "thanks I got the message".
The broker will then look on its list of other applications who wanted to hear about CMS changes and pass the message to the url that the application left when it told the broker it wanted to hear about the message.
The other applications will return 200 OK when they receive the message, if not the broker keeps the message and queues it up for the next time someone tries to contact that application.
The problem is I don't even know where to start or what I need to make it happen. I've been looking at XMPP, ActiveMQ, RabbitMQ, Mule ESB etc. and can see I could spend the next year going around in circles with this stuff.
Could anyone offer any advice from personal experience as I would quite like to avoid learning lessons the hard way.
I've worked with JMS messaging in various software systems since around 2003. I've got a web app where the clients are effectively JMS topic subscribers. By the mere act of publishing a message into a topic, the message gets server-pushed dissemenated to all the subscribing web clients.
The web client is Flex-based. Our middle-tier stack consist of:
Java 6
Tomcat 6
BlazeDS
Spring-Framework
ActiveMQ (JMS message broker)
BlazeDS has ability to be configured as a bridge to JMS. It's a Tomcat servlet that responds to Flex client remoting calls but can also do message push to the clients when new messages appear in the JMS topic that it is configured to.
BlazeDS implements the Comet Pattern for doing server-side message push:
Asynchronous HTTP and Comet architectures
An introduction to asynchronous, non-blocking HTTP programming
Farata Systems has announced that they have modified BlazeDS to work with the Jetty continuations approach to implementing the Comet Pattern. This enables scaling to thousands of Comet connections against a single physical server.
Farata Systems Achieves Performance Breakthrough with Adobe BlazeDS
We are waiting for Adobe to implement support of Servlet 3.0 in BlazeDS themselves as basically we're fairly wedded to using Tomcat and Spring in combo.
The key to the technique of doing massively scalable Comet pattern is to utilize Java NIO HTTP listeners in conjunction to a thread pool (such as the Executor class in Java 5 Concurrency library). The Servlet 3.0 is an async event-driven model for servlets that can be tied together with such a HTTP listener. Thousands (numbers like 10,000 to 20,000) concurrent Comet connections can then be sustained against a single physical server.
Though in our case we are using Adobe Flex technology to turn web clients into event-driven messaging subscribers, the same could be done for any generic AJAX web app. In AJAX circles the technique of doing server-side message push is often referred to as Reverse AJAX. You may have caught that Comet is a play on words, as in the counterpart to Ajax (both household cleaners). The nice thing for us, though, is we just wire together our pieces and away we go. Generic AJAX web coders will have a lot more programming work to do. (Even a generic web app could play with BlazeDS, though - it just wouldn't have any use for the AMF marshaling that BlazeDS is capable of.)
Finally, Adobe and SpringSource are cooperating on establishing a smoother, out-of-the-box integration of BlazeDS in conjunction to the Spring-Framework:
Adobe Collaborates with SpringSource for Enhanced Integration Between Flash and SpringSource Platforms
First of all, don't worry about ESBs. The situation you've described lies well within the bounds of straightforward message-oriented middleware. You only "need" an ESB if you're doing things like mediations, content-based routing, protocol transformations; things where the middleware does stuff to the message, on top of routing it to the right place.
If you have a diverse set of destination applications that need to speak to each other - and it sounds like you do - you're right that messaging over a language agnostic protocol (like XMPP, STOMP or HTTP) is a neat solution. It basically means you don't have to write and run loads of Java daemons to translate messages into your favourite flavour of JMS.
STOMP is increasingly supported by message brokers, especially by the open-source ones, and there's a number of different client libraries. It is a lightweight protocol, specifically designed for messaging so you get a much richer feature set out of the box than you would with HTTP.
For me, XMPP is a bit of a weak option as it's not so well supported on the server side, although it is fun to be able to IM your broker :)
If you are set on HTTP, OpenMQ is very good, and I've personally used its Universal Message Service - basically a webapp wrapper around JMS destinations. It provides a REST-ful interface, with a similar set of verbs as STOMP provides.
As someone has already said, what your describing is basically the Publisher/Subscribe Model. This is very easily achieved using either an ESB or a message queue. I have had some experience with RabbitMQ. Its very good. Nothing gets lost and it deals with the publish subscribe model very well. I have in the past gone down the route on small scale systems of developing my own Message broker with a bespoke protocol over http. I wouldn't advise this, reason being is that as you start to develop it you keep thinking of ways of how to extend it.
RabbitMQ is developed in Erlang but it has java,net,python etc clients that can hook into it very easily. I have used the .net and python clients, it works well. I chose it for Erlangs reputation for creating solid systems that can cope with multiple things going on at the same time, very well. I would call them threads but I think that its smarter than just threads, I think I remember mutterings of the Actor Model and mailboxes, which I recall were pretty neat.
I was in a similar position as yourself but with very bad experiences of other messaging systems (Biztalk et al.) that were too propriety that tied you into a solution. If you can keep the messages separate from the transport and delivery mechanisms, then you can develop your system to your hearts content. I used JSON in the end as the packet sizes are small. You could use anything you like, some opt for SOAP messages, but I feel that these are way too heavy for most stuff, although it does allow you to nicely give XSD schemas to outsiders so that they can/could develop systems that interop with your system in the future.
http://www.rabbitmq.com/tutorials/tutorial-three-java.html, this is a link to the tutorial on the Publish/Subscribe model and how you would achieve it using a message queue system. Its for rabbitMQ, but to be honest it will work with ESB and any other Messaging queue system out there.
ESB (Enterprise Serial Bus) - Consider this when your application have much interaction with two or more external/separate applications where each of these won't communicate in a similar data format. Ex: Some systems may accept objects, XML, JSON, SMTP, TCP/IP, HTTP, HTTPS etc.
ESB has many features like:
Routing,Addressing,Messaging styles,Transport protocols,Service messaging model.
Consider queue system if the producer - consumer applications follows the same type of data format.
Web services (SOAP / REST) is best if one application need the other application to complete the work flow.
Use Queues if the application need asynchronous data transfer.
You're really talking about publish and subscribe with assured delivery. Most MOM software should easily support your use case.
As it was already said earlier, having an ESB for you current case seems to me like to smash a fly with a hammer.
The ESB software itself will be time consuming and will require maintenance. If you go to open source solution, it might be more time consuming than using a licensed solution (IBM, ORACLE, ...).
Of course an ESB would do the job, and it would be really easy to develop a solution, but setting up an ESB would be way more difficult than doing the solution itself.
If your problem is limited to the case described, I would highly suggest you to build a simple architecture over OpenMQ (or similar), and using it through JMS
why dont you see more people using REST architecture for client server system. You see people using sockets, or TIBCO RV or EMS or MQ but i haven't seen much basic REST architecture
does anyone know any reason why you would avoid using this architecture for client / server communication for high through put / low latency
REST is not a good fit for every problem.
REST is best for Resource management. If you are writing web services (as with a client-server system) then you find you want things like language-agnostic data representation, argument validation, client/server code generation, error handling, access controls. REST basically requires you to code those things yourself.
On the other hand, it adds the HTTP layer. You get seamless integration of proxies, caching etc, but you do lose some speed due to HTTP headers, the webserver frontend, etc.
I don't know that I would necessarily avoid it but I can think of a couple of reasons why I might not choose it for a high through-put, low latency service. First, you have to deal with the entire web stack to get your message to your service. This could introduce a number of unnecessary layers and services that would delay messages. A custom service need only support the protocol layers required by the service itself.
Second, unless your service is the only service hosted on the web server, you'll be competing with other requests for your messages to be serviced. While having a custom endpoint for your service may not solve all resource contention problems, at least you don't have to compete for access from other services to your endpoint.
Third, a custom protocol need only support the actual service-related protocol information and may result in smaller packet sizes because you don't need to support the additional HTTP protocol overhead. This would particularly effect protocols that exchange small messages as the header information would be a larger fraction of the message size.