We have a legacy application that allows our developers to "add" messages via a ThreadLocal in Java.
The current SOAP endpoints will scoop these messages off the thread and then package them up in the response.
The endpoints also catch all exceptions and then marshal those exceptions via this same mechanism to normalize the passing of messages (be they informational, warning, or error).
These messages are rich objects (they have a code, severity, classification, and then the actual message text.)
This is nice in many ways because now we have a standard way to communicate meaningful messages to the user (or calling service) but it also makes using the API more challenging because now the client must pick out the messages from the response AND also pick out the real payload.
Any web service can communicate messages this way...but only a handful do.
I would like to start moving our application towards a REST API but I am struggling on how best to handle the messaging. I am not super keen on adding an envelope to each of our REST responses because this really pollutes the API.
The alternative appears to be adding these messages to custom HTTP headers. Is this the "preferred" approach? Remember I will have a list of one or more of these messages and I will likely have to serialize them as json as well.
Thanks.
Related
Creating a user, adding some record to collection in the DB, updating some stuff, etc..
All of these we regularly do with HTTP requests against REST api.
Think about making Event bus as server instead of REST api.
In that method, create user will be an event name: "CreateUser" instead of REST api endpoint: POST /users.
In reflect to any action done in the event bus, it will re-emit a following event telling to any body needed to know about, that the event was done.
If for example someone viewing the vehicles collection and another user just edit one of the columns or add a new vehicle instance, it will be reflected immediately to who views it online.
My question is if there attitudes like I mentioned above, if there some formally names for it, if it a good practice, if you know someone who regularly uses it, a framework or something etc. Does the socket.io server can handle and behave like http server in high workloads?
You can use websockets for this; they provide a bidirectional channel between client and server to send messages across. You will have to catch and parse the messages on each end yourself, as there is no additional protocol on top of them.
They don't hold state, so there is no knowledge of who is looking at what, or who got what. You could send the same update message to all connected clients and leave it to the client to use it or not.
You would have to reprogram your client code and the API endpoints, because it's a different way of doing things, and it can also do server push.
I have no idea about frameworks though, as I always use them without one. Websockets are fast, but server behaviour at high workloads depends on implementation, and I only have experience with the websocket server I wrote myself. I suppose the performance of the socket.io can easily be googled.
I have a system that exposes a REST API with a rich set of CRUD endpoints to manage different resources.
The REST API is used also by a front-end application that executes calls by using Ajax.
I would like to make some of these calls asynchronous and add reliability.
The obvious choice seems a message broker (ActiveMQ, RabbitMQ, etc...).
Never used message brokers before and I am wondering if they can be "put in front of" the REST API without having to rewrite them.
I do not want to access the REST API only through the messaging system: for some endpoints, a call must always be synchronous and the reliability is less important (mainly because in case of error the user receives an immediate feedback).
Would a full ESB be a better option for this use case?
If I understand your question, you would like to "register" an API endpoint as a subscriber so that it could receive the messages sent to a given queue.
I do not think that a message broker can be configured to do this.
For example, if you want to use a message broker, both your producers and subscribers need to use the JMS API.
I do not know if a solution can be to implement a subscriber that will execute the corresponding API call. In this case, the reliability is compromised because the message will be dequeued before the API call is executed. It can make sense if the subscriber is running in the same process of the API, but in this case it is not clear why you should use a REST API instead of a library.
IMO #EligioEleuterioFontana you have a misunderstanding of the roles of:
an RESTful Api
a message broker
These are two different subsystems which provide different services.
Now, let's explain their roles with respect to your requirements:
You have clients (desktop browsers, mobile phone browsers or apps) which need to get/push data to your system. (Assumption from the REST API mention).
Requests from the clients are using HTTP/HTTPS (that's the REST API part of your requirement).
Any data that is pushed, you wish to make this more responsive, quicker, reliable.
If I've gotten that right, then I would answer it as:
All clients need to push requests to a REST API because this does just more than simple CRUD. The Api also handles things like security (authentication and authorization), caching, possibly even request throttling, etc.
REST API should always been the front end to clients as this also 'hides' the subsystems that the API uses. Users should never see/know about any of your subsystem choices (eg. what DB you are using. Are you caching? if so, with what? etc).
Message Brokers are great for offloading the work that was requested now and handling the work later. There's heaps of ways this can be done (queues or pub/sub, etc) but the point here is this is a decision the clients should never see or know about.
MB's are also great for resilience (as you noted). If something fails, the message on a queue would be re-attempted after 'x' time ... etc. (no, I'm not going to mention poison queues, dead letter queue, etc).
You can have some endpoints of the Api that are synchronous. Sure! Then have others that leverage some eventual consistency (i.e. for that request, I'll deal with it later (even if later in 5 secs later) and just return the response to the client saying "thanks! got it! I'll do it soon"). This is the asynchronous workflow you are after.
The API endpoints needs to be simple, concise and hopefully pretty stable. What you do behind the scenes as you change things hopefully will be hidden away from the clients. This includes the use of message brokers.
Anyway, that my take on how I see REST APIs and Message Brokers and how they related to each other.
It might be worth looking into the Google Cloud sub/pub? -
https://cloud.google.com/pubsub/docs/overview
Here are the details of my use case:
What's my data..
There would be user experiences, error report, state info and so on. The data is fragmented and may change in the future. So I plan to use NoSQL, maybe mongodb, to save data in the server.
What are the clients..
They are clients written in different languages, like C#, C++, LabVIEW and so on. Some don't even have an access to a mongodb driver, so of course it's not an option to communicate with database directly. And framework like below is needed.
Clients -> (Some protocol) -> Broker -> Database.
As those clients are not web client, so common web server using http may not suit for my case, right? Is there any suggestion for the protocol, broker and database, Or even a new framework.
My goal is to make the clients can send data as convenient as possible.
Thank you!
This is not really new, but a message driven application, which is a well understood pattern.
I did this mostly in Java, so I will stick to this language here.
A broker alone would be not enough here. Let us say you use Apache ActiveMQ as you message broker, you would still need to get your data into the database, since MQ is... ...a message queue. So you need a part which gets the messages out of MQ, processes them according to your business rules and stores them in the (correct) database instance, and the correct collection/bucket/table. Of course you could write this part by hand, but that would be pretty much reinventing the wheel. There is a notion of a "message routing and mediation engine", and the most commonly suggested here is Apache Camel, which has quite some components to communicate with databases and other so called consumers and producers. And that is the key point. In general, if possible, your clients should send their data to the message broker directly. But, if they can't, they can simply send text files or make REST calls – there are actually too many options to list here. This incoming data can be preprocessed and normalized to your standard format by a "route" in Apache Camel (a set of a consumer, conversion rules and a producer, in it's simplest form) and send as an AMQP message to MQ. From there, another Camel route can process the AMQP messages, apply your business rules and store the data in the database... ...or whatever else may come to your mind (for example sending an email).
So this solution supports a multitude of protocols for incoming and outgoing messages (as long as they are supported by Camel) and you have your business rules in a centralized and well defined location.
To implement this, I'd strongly suggest using Apache ServiceMix, which is a distribution of ActiveMQ, Camel and a system to manage the components and business rules.
Finally, web server with http protocal could suit for the use case, I think.
Mostly I want is a universal API for different kinds of clients to save data to cloud. Http has method GET, POST, PUT, DELETE, so with a RESTful API it is naturlly suitable for operate data, I think.
My solution at last is Node.js(Express) + Mongodb (a quite common group), and a RESTful API is provided via Express web server, clients can use http to operate data conviencely. Also, it is quite light weight and easy to get started.
Here is some tutorial: http://cwbuecheler.com/web/tutorials/2013/node-express-mongo/
We are looking to develop a number of services, but are not sure which "response" mechanism is the best route to go. The two contenders are:
HTTP callbacks, where the service would update the client application via "pinging" it with update messages sent via HTTP requests
Message Passing, where the service would update the client via publishing messages into a pub-sub queue on a message server
In both cases, both the caller and the services are within our network, we have full control over them, and things we develop are the only users of the services.
What are the pros / cons of each way of providing status updates to the calling application, and what, if any, pros / cons would there be for making the initial request via one method or the other?
Note: The first service we have in mind for this is an email service similar to SendGrid, which we can't use for various reasons, but still need the same functionality.
The main difference would be the quality of service that you get "out of the box" with a messaging server.
If you go with HTTP then your application has to take care of what happens when a message doesn't arrive as expected. To get an idea of the issues you need to consider and the complexities involved in solving them, take a look at WS-ReliableMessaging or HTTPLR.
With messaging, you get a configurable level of reliability out of the box. And there's a lot of good choice these days such as ActiveMQ, RabbitMQ, 0MQ.
My personal preference is for reliability to be handled at the transport layer (by messaging), but then for a good discussion and dissenting view, check out "Nobody Needs Reliable Messaging."
I have a project that is currently in production delivering some web-services using the REST approach. Right now, I need to delivery some of this web-services in SOAP too (it means that I will need to deliver some of the same web-services in SOAP and others a bit different), so, I ask you:
Should I incorporate to the existent project the SOAP stack (libraries, configuration files, ...), building another layer that deliver the data in envelopes way (some people call it "anti-corruption layer") ?
Should I build another project using just the canonical model in common (become it in a shared-library) ?
... Or how do you proceed in similar situations ?
Please, consider our ideal target a SOA architecture.
Thanks.
In our projects we have a facade layer which exposes the services and maps to business entities, and a business layer where the business logic is run.
So to add a SOAP end point for an existing service, we just create a new facade and call in to the same business logic.
In many cases it is even simpler, since we use WCF we can have a http SOAP endpoint for external clients, and a binary tcpip endpoint for internal clients. The new endpoint can be added by changing the configuration without any need to change the code.
The way I think about an SOA system, you have messages and pub/sub. The message is the interface. Getting those messages into and out of the system is an implementation detail. I create an endpoint that accepts a raw message document (more REST-like, but not really REST) as well as an endpoint that accepts the message as a single parameter to a SOAP call. The code that processes the incoming message is a separate concern from the HTTP endpoint enablement.
You can use an ESB for this. Where ESB receive the soap messages and send the rest request to the back end. WSO2 ESB provides this functionality. Please look at this sample[1].
[1] http://wso2.org/project/esb/java/4.0.0/docs/samples/proxy_samples.html#Sample152