Related
I have two machines talking to each other over TCP, with Machine A using Machine B's service. Ideally I'd love to use an RPC framework for this. gRPC comes to mind since we already use protobuf.
However, Machine A already uses an external library that wraps TCP sockets for communication, so that the client app code can only send raw bytes (encoded in protobuf) to Machine B. So I wonder if I could adapt the code to working with the gRPC server on Machine B.
gRPC is built on top of http/2, and you can theoretically use any kind of connection (e.g. domain sockets, named pipes) to communicate using gRPC, so it is certainly possible. In fact, at least in go, gRPC uses TCP by default.
If the requirement is only that machine A communicates using TCP sockets, but you can use external libraries, then it should be fairly easy to use the gRPC library of the language of your choice to implement the client/server interaction on top of the raw TCP sockets.
On the other hand, if you can not use the gRPC library, while still possible you would have to implement the gRPC protocol yourself, and possibly the http/2 one as well if you can't use any external library; this would likely be a lot of work, and you may be better off creating a simpler, ad-hoc RPC protocol for your use case.
EDIT
The updated question makes the requirements clearer. If you need to use a specific TCP library, but you can use gRPC on top of that, then I think the difficulty of the task really depends on the programming language.
For instance, in go it would be as easy as creating a wrapper struct, implement the net.conn interface for it, and use it with the withDialer option when creating the client. In c++ it looks like it would be more difficult and entail implementing a custom transport (altough I'm not that familiar with the c++ gRPC API so there may be an easier way)
I always use REST API when I get or post some data.
But WebSocket can also do that.
So, I am confused about the difference between WebSocket and REST API
when I try to get or post some data.
A REST API uses HTTP as the underlying protocol for communication, which in turn follows the request and response paradigm. However, with WebSockets, although the communication still starts off over HTTP, it is further elevated to follow the WebSockets protocol if both the server and the client are compliant with the protocol (not all entities support the WebSockets protocol).
Now with WebSockets, it is possible to establish a full duplex and persistent connection between the client and a server. This means that unlike a request and a response, the connection stays open for as long as the application is running, and since it is full duplex, two way simultaneous communication is possible i.e now the server is capable of initiating a communication and 'push' some data to the client.
This is the primary concept use in the realtime technology where you are able to get new updates in the form of server push without the client having to request (refresh the page) repeatedly. Examples of such applications are Uber car's location tracking, Push Notifications, Stock market prices updating in realtime etc.
Here's a video from a presentation I gave earlier this month about websockets and how they are different than using the regular REST APIs: https://www.youtube.com/watch?v=PJZ06MLXGwA&list=PLZWI9MjJG-V_Y52VWLPZE1KtUTykyGTpJ&index=2
You can provide a REST API along with a WebSocket API for different purposes. It's up to your requirements and it depends on what you want to achieve.
For example, a WebSocket API can be used to provide real-time notifications while the REST API can be used to manage your resources.
There are a few details you should be aware of:
REST is a protocol independent architectural style frequently implemented over the HTTP protocol and it's supposed to be stateless.
WebSocket is a bi-directional, full-duplex and persistent connection protocol, hence it's stateful.
Just to mention one example of application that provides different APIs: Stack Exchange provides a REST API along with a WebSocket API.
I haven't yet fully understood what a REST API is, but I guess you refer to it in a broarder way so as to web systems that provide structured data referd to a specific resource as can be a customer, or a product, upon a POST or GET call over http.
The main difference from a practical and simplistic approach is that HTTP GET / POST are request - response protocols. The server will send a response upon a request sent by the client.
In the case of Websockets, communication is bidirectional. Either the server or the client can send information to the counterpart at any time.
To visualize the difference a page that provides stock market data if using HTTP GET will issue a new request to the server each X seconds to get the updated price. Using websockets, the SERVER could directly send the new price to the web browser as soon as it has changed.
You may also be interested in looking into long polling which is a techinc used with HTTP GET / POST to provide similar functionality as Websockets (though it is a totally different thing)
WebSockets are, just like sockets, an interface between two adjacent layers.
Specifically, in ARPANET reference model, sockets are an interface between Transport layer and Application layer; in OSI reference model, they represent an interface between Session layer and Transport layer. Interface means that they reside "in between" layers (at their boundary).
WebSockets are the sockets interface that was "migrated" from the Session/Transport layer boundary to the Session/Presentation boundary of the OSI model. This was done in order to overcome limitations of sockets in the world of web where all communications are "free" by default only on the port 80, the port of HTTP traffic. HTTP protocol, which sits on top of (guaranteed delivery) TCP Transport layer, is part of the Session layer in OSI model thus it can be considered as a "transport" as well for the layer above, the Presentation layer.
Since "I" in "API" stands for "Interface", both sockets and WebSockets are a form of API, although the term API belongs to a modern jargon. REST API is also an interface between Session and Presentation layers of the OSI model.
The difference between the REST API interface and WebSockets interface is that WebSockets is a full duplex persistent TCP connection established via 3-way handshake protocol over HTTP. REST API over HTTP is, just like HTTP, a Request/Response (non-standard) protocol where a TCP connection is created on each new request i.e. it is not persistent.
Help me out to understand the difference between Socket, Websockets, and XMPP protocols.
"Socket" is a term usually used to refer to some abstraction in software of what goes on in a plain TCP/IP (or equivalent) conversation. A socket is something provides reliable, point-to-point conversation in data packets between two points identified by IP numbers. Most programming languages or libraries provide something that models a socket in this sense.
Websockets is a protocol that allows socket-like communication to be initiated between a web browser and its cients, as an extension to the basic HTTP protocol. The conversation need not be strictly point-to-point, as it can pass through proxies, as HTTP can. Websocket conversation is initiated by an exchange of headers similar to HTTP.
XMPP is an XML-based messaging protocol, used by "instant" messaging-type applications.
I am planning to develop a web based chat application which takes in ReSTful requests, translate them to XMPP and deliver them to an XMPP server.
Using websockets for this kind of chat based application looked promising as the events (or responses) can be delivered asynchronously. But if I use websockets as the underlying protocol for transferring the requests from the browser, can this still be considered as a ReSTful design? If yes, how are the URIs, verbs (GET, POST...), parameters represented in the websocket message? Wrap them in an xml/json and send it?
Also, ReSTful architecture states that no session state will be stored on the server. But here in this case when an XMPP client session is created, the state of this session will be stored on the server (violating the stateless constraint)
REST is an architectural style that does not impose a protocol. So yes, you can do REST with Web Sockets, REST with HTTP and REST with FTP if you like.
The main reason to use HTTP is that it is easy and fairly simple to communicate with any component or programming language via HTTP and also because HTTP supports distributed environments with multiple intermediaries: proxies, firewalls...; So you can deploy your service on any topology and anyone will be able to access it.
My rant:
If you are a RESTliban and Roy Fielding’s dissertation is the source of truth, verbs are never acknowledged as part of the semantic. URIs are the semantic. The usage of different verbs for different actions has been an elegant evolution of REST over HTTP, but not part of the "truth". You can check the scenario of rest over HTTP evaluated by Roy in chapter six of his dissertation. No mention to verbs. And notice it is an evaluation scenario, not the specification.
TLDR;
If you need realtime two way communications via the internet and the client is a web browser, the best choice is Web Sockets. You could then implement an application level protocol on top of web sockets to implement a RESTful Web Service.
Yes. You can use REST over WebSocket with library like
SwaggerSocket.
Why would you want to build a REST API on top of socket? IMHO the benefit of a REST API is to leverage standard HTTP protocol possibilities like stateless requests, semantic verbs like GET, DELETE to build an API that can be easily understood by (client) developers. Since sockets do not offer HTTP verbs and so on, you would build some kind of HTTP layer for sockets which is IMHO not reasonable.
In case you would really build such a thing, I'd recommend to use the HTTP protocol as a blueprint and implement the socket protocol like HTTP.
REST architectural style mostly presumes 2 entities viz. client and server.
As we move more towards real time web and development of reactive systems WebSocket would prominently start replacing usage of REST API's.
WS allows data push and pull which dismisses the concept of server and client.
STOMP,AMQP ,XMPP can be used as messaging protocols.
The data itself maybe JSON or Google protocol buffers or maybe Apache Avro.
WebSockets is not tied to web servers but can be developed in stand alone apps like mobile apps or desktop apps too.
I don't understand why you would convert XMPP into REST and then run REST over WS. The point of WebSocket is to take the XMPP protocol directly to the browser, thereby avoiding all of the translation issues.
There are JavaScript libraries that can talk XMPP from the browser to the server. All you need is to proxy the XMPP traffic from WS over into TCP and then straight into your XMPP server. Kaazing has a gateway that allows you to do this.
If you want to use open source, you will need to write a JavaScript XMPP library. There are examples that show how to write a JS library for simple protocols. You just have to find one and extend the concept to the XMPP protocol.
So to recap, here are the way the architecture would look:
Your XMPP Client code <-> XMPP JavaScript Library <-> WebSocket over http <-> WebSocket to TCP Proxy <-> XMPP Server
where the XMPP Client code and the XMPP JavaScript Library runs in the browser, and the WS to TCP proxy along with the XMPP server are all server-side.
I understand this post is really old, but wanted to interject a bit further on the notion that "So if I choose a REST architecture I forfeit the ability for real-time communications?".
In a word, no. A number of REST style implementations I have had experience with leverage REST for compatibility, discoverability, and as a means to scale across different devices in the shadow of IoT.
However, in addition to using WS in addition to REST to facilitate near real-time transmission. There are also a number of abstractions which really help with this and allow you to focus on building your API and deciding how the RT components of the consuming applications should operate.
I would suggest taking a look at things like Tibco Smart-Sockets, or SignalR if you're looking to build a REST API and would like to avoid re-creating the wheel for your RT needs.
I created a project that adds callbacks to the web socket send function: https://github.com/ModernEdgeSoftware/WebSocketR2
Message IDs are established so the client can implement callbacks. It handles message retries after timeouts as well as reconnects to the server if the connection gets dropped. You can then structure you payload to be as RESTful as you want by adding verbs and paths.
This is similar to when a video game studio uses UDP to achieve the speeds they need, but their net code implements a lot of TCP like features for reliability.
The OP's original question is: "Is ReST over websockets possible?"
What this question implies is the following: Is REST API possible over Websockets as a transport.
Of course, OP did not mean the following: Is REST architectural style possible over Websockets. His question was more an operational one i.e. can REST API requests, such as GET, PUT, POST, DELETE etc. be exchanged over a Websockets pipe.
To answer this question, we have to understand that both sockets and Websockets are the same type of interface (full duplex, 3-way handshake protocol), but the difference is that sockets interface originated in ARPANET reference model. In that network model, sockets were an interface between Session layer and Transport layer. The word "interface" means that it resides "in between" network layers, i.e. within their boundary. In other words, sockets are not part of any specific network layer.
Websockets are the same type of socket interface, but in OSI 7-layer network model they no longer reside in between Session and Transport layers. Instead, they reside in between Session layer and Presentation layer. Why there? Why this "move"? A motivation for this was to be able to leverage HTTP protocol as a transport for sockets. And what is so special about HTTP protocol? In enterprise establishments, there are a lot of network zones and segments and these security domains are protected by firewalls. And firewalls, as we know, have associated rules for inbound/outbound traffic. If we want two components in two different network zones to talk to each other, we have to ensure that ports on corresponding firewalls are open. That would involve collaboration of infrastructure, operations teams, business approvals etc. and would introduce significant delays in achieving a simple thing: two components communicating with each other.
Which brings us to our use case: Websockets interface placed between Session OSI layer (where HTTP resides) and Presentation OSI layer (where things like TLS reside). By default, port 80 is open on all firewalls thus no involvement of operations and infrastructure is needed. And our two components can now converse over Websockets communication pipe.
Back to the OP's question. Any type of a string list can be transferred over sockets. Sockets/Websockets are an ideal mechanism for transporting all sorts of custom protocols, whether they are STOMP, HL7, FHIR, or many others. GET, PUT, POST, DELETE requests are different operations on a REST API endpoint. These operations are in the form of a specific string list, and as we saw, sockets/Websockets are very convenient for passing string lists back and forth. In the case of REST over HTTP, though, you are leveraging the whole HTTP "infrastructure" available in all modern Browsers, such as Chrome, Firefox, Edge etc., as well as web servers such as Apache, nginx, IIS, OHS, IHS etc. In other words, REST API piggybacks on an established, string list-based protocol called HTTP that is built-in (part of) both clients and servers' sides. This cannot be said about Websockets. You would have to ensure every type of client and server complies with your (custom) transport solution based on Websockets!
I just spot new topic on the blog of one company who providing cloud solution and "Server-end/Service as a Platform" (SaaS) for games.
I'm not advertising this company, nor I used them, so I don't even know how good or bad they are.
However, they very clearly explain reasons and what are the benefits of using WebSockets in REST
Have a read on their blog
REST requires a stateless protocol according to the statelessness constraint, websockets is a stateful protocol, so it is not possible.
I'm building web app that needs to communicate with another application using socket connections. This is new territory for me, so want to be sure that sockets are different than websockets. It seems like they're only conceptually similar.
Asking because initially I'd planned on using Django as the foundation for my project, but in the SO post I linked to above it's made very clear that websockets aren't possible (or at least not reliable, even with something like django-websockets) using the preferred Django setup (Apache with mod_wsgi). Yet I've found other posts that casually import Python's socket module for something as simple as grabbing the server's hostname.
So:
Are they really different?
Is there any reason not to use Django for a project that relies on establishing socket connections with an outside server?
To answer your questions.
Even though they achieve (in general) similar things, yes, they are really different. WebSockets typically run from browsers connecting to Application Server over a protocol similar to HTTP that runs over TCP/IP. So they are primarily for Web Applications that require a permanent connection to its server. On the other hand, plain sockets are more powerful and generic. They run over TCP/IP but they are not restricted to browsers or HTTP protocol. They could be used to implement any kind of communication.
No. There is no reason.
Websockets use sockets in their implementation. Websockets are based on a standard protocol (now in final call, but not yet final) that defines a connection "handshake" and message "frame." The two sides go through the handshake procedure to mutually accept a connection and then use the standard message format ("frame") to pass messages back and forth.
I'm developing a framework that will allow you to communicate directly machine to machine with installed software. It might suit your purpose. You can follow my blog if you wish: http://highlevellogic.blogspot.com/2011/09/websocket-server-demonstration_26.html
WebSocket is just another application level protocol over TCP protocol, just like HTTP.
Some snippets < Spring in Action 4> quoted below, hope it can help you understand WebSocket better.
In its simplest form, a WebSocket is just a communication channel
between two applications (not necessarily a browser is
involved)...WebSocket communication can be used between any kinds of
applications, but the most common use of WebSocket is to facilitate
communication between a server application and a browser-based application.
You'd have to use WebSockets (or some similar protocol module e.g. as supported by the Flash plugin) because a normal browser application simply can't open a pure TCP socket.
The Socket.IO module available for node.js can help a lot, but note that it is not a pure WebSocket module in its own right.
It's actually a more generic communications module that can run on top of various other network protocols, including WebSockets, and Flash sockets.
Hence if you want to use Socket.IO on the server end you must also use their client code and objects. You can't easily make raw WebSocket connections to a socket.io server as you'd have to emulate their message protocol.
WebSocket is a computer communications transport protocol (like TCP, HTTP 1.0, HTTP 1.1, HTTP 2.0, QUIC, WebRTC, etc.)
Socket is an endpoint for sending and receiving data across the network (like Port number)
Example of Socket:
(TCP, 8.8.8.4, 8080, 8.8.8.8, 8070)
where:
(protocol, local address, local port, remote address, remote port)
Regarding your question (b), be aware that the Websocket specification hasn't been finalised. According to the W3C:
Implementors should be aware that this specification is not stable.
Personally I regard Websockets to be waaay too bleeding edge to use at present. Though I'll probably find them useful in a year or so.