I have a question about the reason the web applications continues setting cookies, because the persistent HTTP conections use sockets, i.e.: websocket.
HTTP 1.1 and 2 uses persistent http conections, with sockets in the client and server. These sockets are active a necessary time for loading a complete web page (HTML, CSS, images, etc), then the sockets are killed by the server. It is logic due to the server does not know what is doing the client. So, in this scenario, the use of the cookies is justified.
But, with websocket i think the scenary is different, because it uses only one socket, so it means that after the conection is done, the server and the client uses the sockets for sending data.
So, the question is... why are the cookies necessary if the server know who is the client?
This question is impossibly broad, since many different web applications work in many different ways.
In general, cookies are used to store data that needs to persist beyond the momentary connection between the client and the server.
More specifically, the connection between the client and the server can be very transient. The server receives a request, sends a page, and moves on to the next request. It doesn't maintain a constant connection to every browser that contacts it.
Related
I have a website being served from a custom webserver, and it loads and works fine when loaded from a laptop/desktop browser, but loads inconsistently on mobile browsers. (In my case I tested specifically Samsung Internet and Chrome on Android)
(The exact behaviour is: load the web page, refresh, and then after a couple of refreshes it will sometimes not be able to load a background image, or any resource on the page at all - but only on mobile browsers)
In case this was just some cached data issue, I've cleared all browser data, restarted my phone, asked friends to try on their devices etc, but I've only been able to reproduce this on mobile devices.
My web server is written using liburing, nginx as a reverse proxy, though I doubt that would be the issue
I read Can Anyone Explain These Long Network Stalled Times? and it ocurred to me that an issue could be me using multiple different HTTP requests to get resources (I've not implemented Connection: Keep-Alive), but I also get this issue on WiFi, and I get the issue even when loading a single asset (such as a background image)
Additional possibly relevant info:
I was initially having a similar issue on desktop as well, and I fixed it by using shutdown() before calling close() on the HTTP requests
I'm using the following response headers:
Keep-Alive: timeout=0, max=0
Connection: close
Cache-Control: no-cache
I'm using the following socket options:
SO_REUSEADDR (mainly for debug convenience)
SO_REUSEPORT (sockets in multiple threads bind to and listen on the same port)
SO_KEEPALIVE, TCP_KEEPIDLE, TCP_KEEPINTVL and TCP_KEEPCNT (to kill off inactive clients)
Oddly enough though I think this disappears for a while after restarting my phone
I have tried not using nginx, instead using WolfSSL for TLS, and I get the same issue
I am inclined to think that this could be an issue with what headers I'm setting in responses (or possibly some HTTPS specific detail I'm missing?), but I'm not sure
And here's the actual site if anyone wants to verify the issue https://servertest.erewhon.xyz/
It looks to me like your server does not do a proper TLS shutdown, but is simply shutting down the underlying TCP connection. This causes your server to send a RST (packet 28) when the client is doing the proper TLS shutdown by sending the appropriate close notify TLS alert (packet 27).
This RST will result in a connection close on the client side. Depending on how fast the client has processed the incoming data this can result in abandoning still unread data in the TCP socket buffer, thus causing the problems you see.
The difference in behavior between mobile and desktop might just be caused by the performance of the systems and maybe by the underlying TCP stack. But no matter if the desktop works fine - your web server behaves wrong.
For details on how the connection close should happen at the HTTP level see RFC 7230 section 6.6. Note especially the following parts of this section:
If a server performs an immediate close of a TCP connection, there is
a significant risk that the client will not be able to read the last
HTTP response. If the server receives additional data from the
client on a fully closed connection, such as another request that was
sent by the client before receiving the server's response, the
server's TCP stack will send a reset packet to the client;
unfortunately, the reset packet might erase the client's
unacknowledged input buffers before they can be read and interpreted
by the client's HTTP parser.
To avoid the TCP reset problem, servers typically close a connection
in stages. First, the server performs a half-close by closing only
the write side of the read/write connection. The server then
continues to read from the connection until it receives a
corresponding close by the client, or until the server is reasonably
certain that its own TCP stack has received the client's
acknowledgement of the packet(s) containing the server's last
response. Finally, the server fully closes the connection.
Background
We are writing a Messenger-like app. We have setup Websockets to Inbox and Chat.
Question
My question is simple. What are the advantages and disadvantages when sending data from Client to Server using REST instead of Websockets? (I am not interested in updates now.)
We know that REST has higher overhead in terms of message sizes and that WS is duplex (thus open all time). What about the other things we didn't keep in mind?
Here's a summary of the tradeoffs I'm aware of.
Reasons to use webSocket:
You need/want server-push of data.
You are sending lots of small pieces of data from client to server and doing it very regularly. Using webSocket has significantly less overhead per transmission.
Reasons to use REST:
You want to use server-side frameworks or modules that are built for REST, not for webSocket (such as auth, rate limiting, security, streaming, etc...).
You aren't sending data very often from client to server and thus the server-side burden of keeping a webSocket connection open all the time may lessen your server scalability.
You want your client to run in places where a long-connected webSocket during inactive periods of time may not be practical (perhaps mobile).
You want your client to run in old browsers that don't support webSocket.
You want the browser to enforce same-origin restrictions (those are enforced for REST Ajax calls, but not for webSocket connections).
You don't want to have to write code that detects when the webSocket connection has died and then auto-reconnects and handles back-offs and handles mobile issues with battery usage issues, etc...
You need to run in situations where there are proxies or other network infrastructure that may not support long running webSocket connections.
If you want request/response built in. REST is request/response. WebSocket is not - it's message based. Responses from a webSocket are done by sending a messge back. That message back is not, by itself, a response to any specific request, it's just data being sent back. If you want request/response with webSocket, then you have to build some infrastructure yourself where you tag an id into a request and the response for that particular request contains that specific id. Otherwise, if there are every multiple requests in flight at the same time, then you don't know which response belongs with which request because all the data is being sent over the same connection and you would have no way of matching response with request.
If you want other clients to be able to carry out this operation via an Ajax call.
So, if you already have a webSocket implementation, don't have any problem with it that are lessened with REST and aren't interested in any of the reasons that REST might be better, then stick with your webSocket implementation.
Related references:
websocket vs rest API for real time data?
Ajax vs Socket.io
Adding comments per your request:
It sounds like you're expecting someone to tell you the "right" way to do it. There are reasons to pick one way over the other. If none of those reason compel you one way vs. the other, then it's just an architectural choice and you must take in the whole context of what you are doing and decide which architectural choice makes more sense to you. If you already have the reliably established webSocket connection and none of the advantages of REST apply to your situation then you can optimize for "efficiency" and send your data to the server over the webSocket connection.
On the other hand, if you wanted there to be a simple API on your server that could be reached with an Ajax call from other clients, then you'd want your server to support this operation via REST so it would simplest for these other clients to carry out this one operation. So, it all depends upon which direction your requirements drive you and, if there is no particular driving reason to go one way or the other, you just make an architectural choice yourself.
I am currently working on a project that requires the client requesting a big job and sending it to the server. Then the server divides up the job and responds with an array of urls for the client to make a GET call on and stream back the data. I am the greenhorn on the project and I am currently using Spring websockets to improve efficiency. Instead of the clients constantly pinging the server to see if it has results ready to stream back, the websocket will now just directly contact the client hooray!
Would it be a bad idea to have websockets manage the whole process from end to end? I am using STOMP with Spring websockets, will there still be major issues with ditching REST?
With RESTful HTTP you have a stateless request/response system where the client sends request and server returns the response.
With webSockets you have a stateful (or potentially stateful) message passing system where messages can be sent either way and sending a message has a lower overhead than with a RESTful HTTP request/response.
The two are fairly different structures with different strengths.
The primary advantages of a connected webSocket are:
Two way communication. So, the server can notify the client of anything at any time. So, instead of polling a server on some regular interval to see if there is something new, a client can establish a webSocket and just listen for any messages coming from the server. From the server's point of view, when an event of interest for a client occurs, the server simply sends a message to the client. The server cannot do this with plain HTTP.
Lower overhead per message. If you anticipate a lot of traffic flowing between client and server, then there's a lower overhead per message with a webSocket. This is because the TCP connection is already established and you just have to send a message on an already open socket. With an HTTP REST request, you have to first establish a TCP connection which is several back and forths between client and server. Then, you send HTTP request, receive the response and close the TCP connection. The HTTP request will necessarily include some overhead such as all cookies that are aligned with that server even if those are not relevant to the particular request. HTTP/2 (newest HTTP spec) allows for some additional efficiency in this regard if it is being used by both client and server because a single TCP connection can be used for more than just a single request/response. If you charted all the requests/responses going on at the TCP level just to make an https REST request/response, you'd be surpised how much is going on compared to just sending a message over an already established webSocket.
Higher Scale in some circumstances. With lower overhead per message and no client polling to find out if something is new, this can lead to added scalability (higher number of clients a given server can serve). There are downsides to the webSocket scalability too (see below).
Stateful connections. Without resorting to cookies and session IDs, you can directly store state in your program for a given connection. While a lot of development has been done with stateless connections to solve most problems, sometimes it's just simpler with stateful connections.
The primary advantages of a RESTful HTTP request/response are:
Universal support. It's hard to get more universally supported than HTTP. While webSockets enjoy relatively good support now, there are still some circumstances where webSocket support isn't regularly available.
Compatible with more server environments. There are server environments that don't allow long running server processes (some shared hosting situations). These environments can support HTTP request, but can't support long running webSocket connections.
Higher Scale in some circumstances. The webSocket requirement for a continuously connected TCP socket adds some new scale requirements to the server infrastructure that HTTP requests don't demand. So, this ends up being a tradeoff space. If the advantages of webSockets aren't really needed or being used in a significant way, then HTTP requests might actually scale better. It definitely depends upon the specific usage profile.
For a one-off request/response, a single HTTP request is more efficient than establishing a webSocket, using it and then closing it. This is because opening a webSocket starts with an HTTP request/response and then after both sides have agreed to upgrade to a webSocket connection, the actual webSocket message can be sent.
Stateless. If your job is not made more complicated by having a stateless infrastruture, then a stateless world can make scaling or fail-over much easier (just add or remove server processes behind a load balancer).
Automatically Cacheable. With the right server settings, http responses can be cached by browser or by proxies. There is no such built-in mechanism for requests sent via webSockets.
So, to address the way you asked the question:
What are the pitfalls of using websockets in place of RESTful HTTP?
At large scale (hundreds of thousands of clients), you may have to do some special server work in order to support large numbers of simultaneously connected webSockets.
All possible clients or toolsets don't support webSockets or requests made over them to the same level they support HTTP requests.
Some of the less expensive server environments don't support the long running server processes required to support webSockets.
If it's important to your application to get progress notifications back to the client, you could either use a long running http connection with continuing progress being sent down or you can use a webSocket. The webSocket is likely easier. If you really only need the webSocket for the relatively short duration of this particular activity, then you may find the best overall set of tradeoffs comes by using a webSocket only for the duration of time when you need the ability to push data to the client and then using http requests for the normal request/response activities.
It really depends on your requirements. REST services can be much more transparent and easier to pick up by developer compared to Websockets.
Using Websockets, you remove most of the advantages that RESTful webservices offer, such as the ability to reference a resource via a URI. Really what you should be doing is to figure out what the advantages are of REST and hypermedia, and based on that decide whether those advantages are important to you.
It's of course entirely possible to create a RESTful webservice, and augment it with a a websocket-based API for real-time responses.
But if you are creating a service that only you are going to consume in a controlled environment, the only disadvantage might be that not every client supports websockets, while pretty much any type of environment can do a simple http call.
I have a perl web application (CGI::Application with ModPerl::Registry) which connects to a authenticated custom server over a socket and exchanges data (command/response) with it. Currently the web application connects to the server, authenticates and disconnects on every page request - even for the same user.
Is there some way I can use the same socket over multiple page requests which share a common session id? Creating a separate daemon that proxies connections and makes them persistent is an option I am exploring, but would like to know if there are any simpler solutions.
I have no control over the design of the custom server unfortunately.
Looks like the same question was asked on PerlMonks. The responses there point in the right direction, but the issue seems to be that you want one cached connection per session, not one cached connection per session per httpd thread/process. You might have to resort to a separate proxy process to get the behaviour you want.
I'm writing a simple HTTP server and learning about TIME_WAIT. How do real web servers in heavy environments handle requests from thousands of users without all the sockets getting stuck in TIME_WAIT after a request is handled? (Not asking about keep-alive -- that would help for a single client, but not for thousands of different clients coming through).
I've read that you try and get the client to close first, so that all the TIME_WAITs get spread out among all the clients, instead of concentrated on the server.
How is this done? At some point the server has to call close/closesocket.
The peer that initiates the active close is the one that goes into TIME_WAIT. So as long as the client closes the connection the client gets the TIME_WAIT and not the server. I go into this all in a little more detail in this blog posting. If you are unable to reach that link then the wayback machine has it.