How is gRPC client streaming/bidirectional streaming implemented with HTTP/2?
Server streaming makes sense, in that it could utilize server push to send multiple responses to a request, but it's not clear to me how it does bidirectional message passing over HTTP/2 the way one would over a websocket.
gRPC encodes streams as HTTP bodies. There is a five byte header before each message, consisting of the message length and a flag byte. It does not use SERVER_PUSH or other HTTP/2-specific features for streaming.
At its core, gRPC is streaming. Unary (single request, single response) and server streaming (single request) are simply special cases for producing cleaner APIs or more optimized I/O behavior. But on-the-wire, everything looks the same as streaming.
The specification of HTTP/1 allows but does not require streaming and bidirectional connections, but some implementations don't support them. But with the nature of HTTP/2, it is generally more work to not support them. Also, there aren't decade-old HTTP/2 proxies to cause compatibility problems; gRPC is able to work with the HTTP/2 ecosystem to encourage streaming to be supported.
For more information of the gRPC encoding, see gRPC's PROTOCOL-HTTP2.md, especially Length-Prefixed-Message.
Related
Multiplexing is a pretty cool feature of http/2. It allows using one connection to serve multiple requests from a single client simultaneously.
My question is: does this multiplexing feature violate REST API rules?
I understand that REST API enforces request-response architecture, but multiplexing without server-push (streaming) feature enabled is essentially one request -> one response paradigm, so that's not a violation, is that right?
REST API also enforces stateless, and I'm lost there: is multiplexing through a single connection considered as stateful or stateless?
If I want to upgrade a REST API which is currently implemented with HTTP/1.1 to use HTTP/2, do I have the privilege to use the multiplexing feature, or I have to do stream after stream (req1, res1, req2, res2...)?
Network multiplexing and REST API are two absolutely different matters/layers of responsibility.
Multiplexing is about how do communication signals flow, and not about what is the architectural pattern of HTTP messages' communication (which is what REST is all about).
From the REST perspective, it does not matter:
how electrical signals flow in the cable or wirelessly;
what type of cable or other physical mean you use for transferring data;
even if you maintain a single physical (TCP) connection throughout several request-response cycles or you open and close TCP connection per each HTTP request-response;
even if you use something else than TCP (yes, that's not a good idea, but theoretically, as long as communication is ensured to have integrity, consistency and stability (which is all TCP brings), it doesn't much matter how the physical connection is established).
Because,
REST is an architectural (design of the web application) pattern for implementing web applications.
Multiplexing is about how the physical signals/connection is being implemented.
As long as HTTP messages flow seamlessly between client and server, physical or transport layer have nothing to do with REST endpoints; hence, there is nothing in multiplexing, that can violate anything in REST, as - again: these two serve absolutely different purposes.
For a long time, when it comes to the microservice architecture, NATS and Kafka are the first options that come to my mind. But recently I found this gRPC template in dotnet core and that grasped my attention. I read a lot about it and watched a lot of videos but I don't think any of those could address gRPC correctly as they usually contrast between gRPC and message brokers or protocols such as REST which I guess is pretty inappropriate although SOAP would be relevant here.
My assumption is that gRPC is a modern version of SOAP with better performance and less implementation hassle due to it protocol buffer. And I think that gRPC can by no means be compared against Kafka or NATS. And also that it cannot replace RESTful service as neither could SOAP.
Now, the question, to what extent are my assumptions true? For example, when it comes to selecting a communication bridge between nodes on a cluster, do I have to put gPRC among my options now (NATS, Kafkam Rabbit, etc) or should I consider that when creating a web proxy to bridge external request to my microservices?
Finally, how about real-time communication, can gRPC replace websocket/socket.io/signalR completely? What does it replace?
I often see people misplacing these technologies by one crucial aspect: public authentication.
For instance, check this graph:
This is a benchmark of Inverted Json (https://github.com/lega911/ijson), comparing some tools, such as iJson, RabbitMQ, Nats, 0MQ, etc.
Notice that Nats, ZeroMQ and iJson are not meant to be used as public end-points (for instance, Nats have user/password, token and keys, but it is useless in an open environment, such as web browsers, because there is no way to make the key non public).
On the other hand, GRPC works just fine with JWT and Oauth2, making it completely safe to public end-points (as safer as any other HTTP endpoint), 'cos those tokens are server-signed (so, even tough they are public, they can't be forged or tempered with)
So, what I'm trying to say is: there are techs meant to face public and techs meant to glue together servers and process within servers (which are private connections).
GRPC is public, ZeroMQ and iJson are totally private (iJson, for instance, don't have any kind of authentication). Nats works with keys or passwords, so, although is "safer" than iJson and ZeroMQ, it is not meant to be public.
When you say REST (I'm assuming HTTP here, because REST is just an architecture), websocket/socket.io/signalR, you are depicting all public interfaces. GRPC will cover you here (it's comparable to REST as request/response and websocket/socket.io/signalR because it supports half and full duplex streaming (similar to sockets)).
Nats, iJson, ZeroMQ, on the other hand, are not meant to do that. They are meant to communicate between services.
So, basically, REST/websocket/socket.io/signalR = gRPC.
Internal communication between services (in the same or in different servers) = NATs, iJSON, ZeroMQ.
(notice that I'm not even considering the other technologies in the graph, because they are products, IMO, not simple libraries you can use to achieve an end, such as RabbitMQ, nginx, etc. The other ones I'm not familiar enough to be able to make an opinion (but I'm surprised by the uvloop in that graph)).
Your intuition is correct that gRPC is not comparable to an asynchronous queueing system like kafka, Rabbit, etc.
It is however a replacement for synchronous server to server communication technologies often implemented over SOAP, RPC, REST, etc. where you are expecting to get a response from another server rather than firing a message into a queue and then effectively forgetting about the message.
gRPC is definitely an option for real-time communication. It can replace socket communication if you are not streaming to the browser(No gRPC support), have a look at the Bidirectional streaming support.
About replacing Kafka/Rabbit, gRPC can be used as a PubSub system as it supports Bidirectional streaming but I would not recommend it.
The goal is to introduce a transport and application layer protocol that is better in its latency and network throughput. Currently, the application uses REST with HTTP/1.1 and we experience a high latency. I need to resolve this latency problem and I am open to use either gRPC(HTTP/2) or REST/HTTP2.
HTTP/2:
Multiplexed
Single TCP Connection
Binary instead of textual
Header compression
Server Push
I am aware of all the above advantages. Question No. 1: If I use REST with HTTP/2, I am sure, I will get a significant performance improvement when compared to REST with HTTP/1.1, but how does this compare with gRPC(HTTP/2)?
I am also aware that gRPC uses proto buffer, which is the best binary serialization technique for transmission of structured data on the wire. Proto buffer also helps in developing an language agnostic approach. I agree with that and I can implement the same feature in REST using graphQL. But my concern is over serialization: Question No. 2: When HTTP/2 implements this binary feature, does using proto buffer give an added advantage on top of HTTP/2?
Question No. 3: In terms of streaming, bi-directional use-cases, how does gRPC(HTTP/2) compare with (REST and HTTP/2)?
There are so many blogs/videos out in the internet that compares gRPC(HTTP/2) with (REST and HTTP/1.1) like this. As stated earlier, I would like to know the differences, benefits on comparing GRPC(HTTP/2) and (REST with HTTP/2).
gRPC is not faster than REST over HTTP/2 by default, but it gives you the tools to make it faster. There are some things that would be difficult or impossible to do with REST.
Selective message compression. In gRPC a streaming RPC can decide to compress or not compress messages. For example, if you are streaming mixed text and images over a single stream (or really any mixed compressible content), you can turn off compression for the images. This saves you from compressing already compressed data which won't get any smaller, but will burn up your CPU.
First class load balancing. While not an improvement in point to point connections, gRPC can intelligently pick which backend to send traffic to. (this is a library feature, not a wire protocol feature). This means you can send your requests to the least loaded backend server without resorting to using a proxy. This is a latency win.
Heavily optimized. gRPC (the library) is under continuous benchmarks to ensure that there are no speed regressions. Those benchmarks are improving constantly. Again, this doesn't have anything to do with gRPC the protocol, but your program will be faster for having used gRPC.
As nfirvine said, you will see most of your performance improvement just from using Protobuf. While you could use proto with REST, it is very nicely integrated with gRPC. Technically, you could use JSON with gRPC, but most people don't want to pay the performance cost after getting used to protos.
I am not an expert on this by any means and I have no data to back any of this up.
The "binary feature" you're talking about is the binary representation of HTTP/2 frames. The content itself (a JSON payload) will still be UTF-8. You can compress that JSON and set Content-Encoding: gzip, just like HTTP/1.
But gRPC does gzip compression as well. So really, we're talking about the difference between gzip-compressed JSON vs gzip-compressed protobufs.
As you can imagine, compressed protobufs should beat compressed JSON in every way, or else protobufs have failed at their goal.
Besides the ubiquity of JSON vs protobufs, the only downside I can see to using protobufs is that you need the .proto to decode them, say in a tcpdump situation.
What I found out is,
If you want to send files such as txt, img or video files then REST/HTTP is much faster than gRPC.
If you want to send objects over the wire, then gRPC is effecient.
Sources:
For sending files,, sending large files,, another blog
I'm writing a client using AsyncHttpClient (AHC) v2.0beta (using Netty 4 as a provider) that streams audio in real-time and it needs to receive server data in real-time too (while streaming). Imagine a HTTP client streaming the microphone's output as the user speaks and receiving the audio transcription has it happens in real time. In short, it's a bidirectional real-time communication over HTTP (chunked multipart request/response).
In order to do that, I had to hack AHC a bit. For instance, there is a blocking call to wait for input data in org.asynchttpclient.multipart.MultipartBody#read(ByteBuffer buffer) which is implemented on top of Netty's io.netty.handler.stream.ChunkedInput.
This somewhat works. The problem is that my custom AsyncHandler will not get onBodyPartReceived() callbacks until the request has finished streaming. They receiving events get pilled up, probably because Netty isn't reading while there is still content to write. Playing with the network stack, I noticed I was only able to receive server responses while streaming if the client was having network contention while writing.
Can someone tell me if this behavior is the result of my particular implementation (blocking in MultipartBody#read()) or an architectural design constrain imposed by Netty's internal implementation?
As a side note, reading and writing happens inside a single IO thread nioEventLoopGroup-X.
I am currently working on a project that requires the client requesting a big job and sending it to the server. Then the server divides up the job and responds with an array of urls for the client to make a GET call on and stream back the data. I am the greenhorn on the project and I am currently using Spring websockets to improve efficiency. Instead of the clients constantly pinging the server to see if it has results ready to stream back, the websocket will now just directly contact the client hooray!
Would it be a bad idea to have websockets manage the whole process from end to end? I am using STOMP with Spring websockets, will there still be major issues with ditching REST?
With RESTful HTTP you have a stateless request/response system where the client sends request and server returns the response.
With webSockets you have a stateful (or potentially stateful) message passing system where messages can be sent either way and sending a message has a lower overhead than with a RESTful HTTP request/response.
The two are fairly different structures with different strengths.
The primary advantages of a connected webSocket are:
Two way communication. So, the server can notify the client of anything at any time. So, instead of polling a server on some regular interval to see if there is something new, a client can establish a webSocket and just listen for any messages coming from the server. From the server's point of view, when an event of interest for a client occurs, the server simply sends a message to the client. The server cannot do this with plain HTTP.
Lower overhead per message. If you anticipate a lot of traffic flowing between client and server, then there's a lower overhead per message with a webSocket. This is because the TCP connection is already established and you just have to send a message on an already open socket. With an HTTP REST request, you have to first establish a TCP connection which is several back and forths between client and server. Then, you send HTTP request, receive the response and close the TCP connection. The HTTP request will necessarily include some overhead such as all cookies that are aligned with that server even if those are not relevant to the particular request. HTTP/2 (newest HTTP spec) allows for some additional efficiency in this regard if it is being used by both client and server because a single TCP connection can be used for more than just a single request/response. If you charted all the requests/responses going on at the TCP level just to make an https REST request/response, you'd be surpised how much is going on compared to just sending a message over an already established webSocket.
Higher Scale in some circumstances. With lower overhead per message and no client polling to find out if something is new, this can lead to added scalability (higher number of clients a given server can serve). There are downsides to the webSocket scalability too (see below).
Stateful connections. Without resorting to cookies and session IDs, you can directly store state in your program for a given connection. While a lot of development has been done with stateless connections to solve most problems, sometimes it's just simpler with stateful connections.
The primary advantages of a RESTful HTTP request/response are:
Universal support. It's hard to get more universally supported than HTTP. While webSockets enjoy relatively good support now, there are still some circumstances where webSocket support isn't regularly available.
Compatible with more server environments. There are server environments that don't allow long running server processes (some shared hosting situations). These environments can support HTTP request, but can't support long running webSocket connections.
Higher Scale in some circumstances. The webSocket requirement for a continuously connected TCP socket adds some new scale requirements to the server infrastructure that HTTP requests don't demand. So, this ends up being a tradeoff space. If the advantages of webSockets aren't really needed or being used in a significant way, then HTTP requests might actually scale better. It definitely depends upon the specific usage profile.
For a one-off request/response, a single HTTP request is more efficient than establishing a webSocket, using it and then closing it. This is because opening a webSocket starts with an HTTP request/response and then after both sides have agreed to upgrade to a webSocket connection, the actual webSocket message can be sent.
Stateless. If your job is not made more complicated by having a stateless infrastruture, then a stateless world can make scaling or fail-over much easier (just add or remove server processes behind a load balancer).
Automatically Cacheable. With the right server settings, http responses can be cached by browser or by proxies. There is no such built-in mechanism for requests sent via webSockets.
So, to address the way you asked the question:
What are the pitfalls of using websockets in place of RESTful HTTP?
At large scale (hundreds of thousands of clients), you may have to do some special server work in order to support large numbers of simultaneously connected webSockets.
All possible clients or toolsets don't support webSockets or requests made over them to the same level they support HTTP requests.
Some of the less expensive server environments don't support the long running server processes required to support webSockets.
If it's important to your application to get progress notifications back to the client, you could either use a long running http connection with continuing progress being sent down or you can use a webSocket. The webSocket is likely easier. If you really only need the webSocket for the relatively short duration of this particular activity, then you may find the best overall set of tradeoffs comes by using a webSocket only for the duration of time when you need the ability to push data to the client and then using http requests for the normal request/response activities.
It really depends on your requirements. REST services can be much more transparent and easier to pick up by developer compared to Websockets.
Using Websockets, you remove most of the advantages that RESTful webservices offer, such as the ability to reference a resource via a URI. Really what you should be doing is to figure out what the advantages are of REST and hypermedia, and based on that decide whether those advantages are important to you.
It's of course entirely possible to create a RESTful webservice, and augment it with a a websocket-based API for real-time responses.
But if you are creating a service that only you are going to consume in a controlled environment, the only disadvantage might be that not every client supports websockets, while pretty much any type of environment can do a simple http call.