In sockets, keepalive will not close the socket. The client sends a keepalive flag to the server and if the server agrees the connection will not be closed. If I understand the concept well, the client will send a keepalive packet (which contains null data) to the server whenever the client has no data to send.
However, when can a socket be closed?
The following are the scenarios I can think of
the server/client specifically close the connection
timeout exceptions
Can the router close a connection?
You're confusing two different things called keepalive.
In HTTP keepalive, it is on by default from HTTP 1.1, and the client has to specifically request it to be turned off. In HTTP 1.0 there was no such thing but there was an informal conventions to request it: a Connection: keep-alive header. When a keep-alive is used, neither party closes the connection after the HTTP response has been sent/received. When a keep-alive is not used, both parties close the connection.
In TCP keepalive, the TCP stack sends a byte with a sequence number lower than what has already been ACKed, which should provoke the receiver into sending an ACK with the current sequence number.
In HTTP or TCP the connection can really be closed any time, and the peer just has to detect and cope with that.
Related
I need to know how HTTP/1.1, webSocket and HTTP/2.0 works in terms of Socket (I am not interested in a list of different features between these three technologies).
So, when I start an HTTP/1.1 request I know that after server response, my connection will be closed.
But, let me say, When I start an HTTP/1.1. request, at transport layer level, a socket will be inizialized to send my HTTP request (header and data) to the webserver.
So I have three questions:
If HTTP/1.1 implements a socket (open from my pc and webserver) to send its request, why it can not use that socket to implement request-response cycle more and more times ?
Is The principal different between HTTP/1.1 and webSocket the fact that HTTP/1.1 close the socket after the first request-response cycle and webSocket don't close the socket after first cycle ?
How HTTP/2.0 manages socket between client and server ?
Thanks in advance.
To answer your question:
Actually, HTTP/1.1 allows the connection to be used for more than a single request by using the "keep-alive" feature.
This means that multiple HTTP/1.1 requests might be sent over a single TCP/IP connections.
However, since HTTP/1.1 doesn't allow for multiplexing, the requests (and responses) are serialized, which might cause a longer request/response to delay short request/response due to the strict queue.
FYI: closing the connection is an HTTP/1 approach, where the end of a response would be marked by the socket closing. On HTTP/1.1, the end of the response is usually known by the "Content-Length" header (or the chunked encoding marker).
No, the difference is much bigger. The WebSocket protocol isn't a request-response protocol, it's a message based protocol, which is a totally different beast.
In effect, you can think about WebSockets as more similar to TCP/IP than to HTTP, except that TCP/IP is a streaming protocol and WebSockets is a message based protocol...
The WebSockets protocol promises that messages don't arrive fragmented while TCP/IP read calls might return a fragment of a message (or more than a single message).
HTTP/2.0 uses a single connection, but it has a binary message wrapping layer that allows the server and the client to multi-plex (manage more than a single information stream using a single connection).
This means that the request-response queue is parallel (instead of the HTTP/1.1 serial queue). For example, a response to request #2 might arrive before a response to request #1.
This solves the HTTP/1.1 issue of "pipelining" and message ordering, where a long request/response cycle might cause all the other requests to "wait".
There are other properties and differences, but in some ways this is probably the main one (in addition to other performance factors such as header compression, binary data formats, etc').
I'm investigating resetting a TCP connection as a solution to the TIME_WAIT issue.
Let's use the following request-reply protocol as an example:
The client opens a connection to the server.
The client sends a request.
The server replies.
The server closes.
The client closes as well.
This causes a TIME_WAIT state at the server. As a variation, the client could close first. Then, the TIME_WAIT lands on the client.
Can we not replace steps 4 and 5 by the following?
The client resets.
The server resets in response to the incoming reset.
This seems to be a way to avoid the TIME_WAIT issue. The server has proven that it received and processed the request by sending its reply. Once the client has the reply the connection is expendable and can just go away.
Is this a good idea?
I would say: No it's not a good idea. Every possible solution ends up with the same "problem" that TIME_WAIT ultimately addresses: how does party A, acknowledging the ending of the connection (or acknowledging the other side's final acknowledgment of the ending of the connection), know that party B got the acknowledgment? And the answer is always: it can't ever know that for sure.
You say:
the server has proven that it received and processed the request by sending its reply
... but what if that reply gets lost? The server has now cleaned up its side of the session, but the client will be waiting forever for that reply.
The TCP state machine may seem overly complicated at first glance but it's all done that way for good reason.
The only problem is that the server doesn't know whether the client received everything. The situation is ambiguous: did the client connection reset because the client received the whole reply, or was it reset for some other reason?
Adding an application level acknowledgement doesn't reliably fix the problem. If the client acknowledges, and then immediately closes abortively, the client can't be sure that the server received that acknowledgement, because the abortive close discards untransmitted data. Moreover, even if the data are transmitted, it can be lost since the connection is unreliable; and once the connection is aborted, the TCP stack will no longer provide re-transmissions of that data.
The regular, non-abortive situation addresses the problem by having the client and server TCP stacks take care of the final rites independently of application execution.
So, in summary, the aborts are okay if all we care about is that the client receives its reply, and the server doesn't care whether or not that succeeded: not an unreasonable assumption in many circumstances.
I suspect you are wrong about the TIME_WAIT being on the server.
If you follow the following sequence for a single TCP-based client-server transaction, then the TIME_WAIT is on the client side:
client initiates active connection to server
client sends request to server.
client half-closes the connection (i.e. sends FIN)
server reads client request until EOF (FIN segment)
server sends reply and closes (generating FIN)
clients reads response to EOF
client closes.
Since client was the first to send the FIN, it goes into TIME_WAIT.
The trick is that the client must close the sending direction first, and the server synchronizes on it by reading the entire request. In other words, you use the stream boundaries as your message boundaries.
What you're trying to do is do the request framing purely inside the application protocol and not use the TCP framing at all. That is to say, the server recognizes the end of the client message without the client having closed, and likewise the client parses the server response without caring about reading until the end.
Even if your protocol is like this, you can still go through the motions of the half-close dance routine. The server, after having retrieve the client request, can nevertheless keep reading from its socket and discarding bytes until it reads everything, even though no bytes are expected.
Hi guys I'm new to understanding protocols used over the web and need some help in understanding the basics of websockets,TCP/IP and HTTP.
My understanding of the relation between TCP/IP and HTTP is that IP is required to connect all networks. TCP is a mechanism that allows us to transfer data safely and HTTP, which utilizes TCP to transfer its data, is a specific protocol used by Web servers and clients.
Does this mean you can't send a HTTP request without TCP?
Websockets communicate using TCP layer and a connection between client and server is established through HTTP which is known as the handshake process.
Does websockets have its own protocol? How can you send a http request(hand shake process) to establish TCP/IP when you need TCP to carry out an HTTP request. I know I am missing something really important here and would be great to get my understanding of these protocols sharpened!
Firstly, IP is not necessarily required to connect all networks. However, it is the most widely used and adopted today (For now that is). Old network protocols such as Appletalk, IPX, and DECNet are all legacy network protocols that are not much used anymore, however they are still around to an extent. Don't forget IPv6 is out there as well, in some places, and can ride over IPv4 networks if your configuration is done properly.
When you say TCP being "safe", I would give it another word, and that would be intelligent. TCP is a transport protocol, and is the header that comes directly after the IPv4 header. TCP is primarily used for flow control and has become very efficient at error recovery in case a part of a packet or packets has been last when transferring/receiving. While this is great for some transactions, the error control requires an additional amount of overhead in the packet. Some applications, let's say VoIP for example, is very sensitive to delay, jitter (Variation in delay) and congestion. This is why it uses UDP.
Like TCP, UDP is a transport protocol, however there is no flow control. Think of it this way: When sending packets over TCP, it's like asking the other end if they received your message. If they did, they will acknowledge it. If not, you now have to determine how you will resend this information. UDP has none of this. You send your message to the other side, and hope it gets there.
Now if you want to talk about "safe" protocols, this is usually done at either the network layer (IPSec) or the application layer (SSL). Safe typically means secured.
A usual TCP three-way handshake looks like this:
Whoever sends the SYN is the client. Whoever receives that initial SYN is the server.
Client sends SYN --> Server
Now, if the server is listening, and/or there's not a firewall blocking the service (Which in that case you'd receive a TCP frame from the server with the RST,ACK bits set most likely), the server will respond with a SYN-ACK:
Server sends SYN/ACK --> Client
If the client received this packet, he will acknowledge he received it. This completes the three-way handshake and these two may begin exchanging information.
Client sends ACK --> Server
Here's a good site for some info:
http://www.tcpipguide.com/free/index.htm
Keep-alives were added to HTTP to basically reduce the significant
overhead of rapidly creating and closing socket connections for each
new request. The following is a summary of how it works within HTTP
1.0 and 1.1:
HTTP 1.0 The HTTP 1.0 specification does not really delve into how
Keep-Alive should work. Basically, browsers that support Keep-Alive
appended an additional header to the request as [edited for clarity] explained below:
When the server processes the request and
generates a response, it also adds a header to the response:
Connection: Keep-Alive
When this is done, the socket connection is
not closed as before, but kept open after sending the response. When
the client sends another request, it reuses the same connection. The
connection will continue to be reused until either the client or
the server decides that the conversation is over, and one of them drops the connection.
The above explanation comes from here. But I don't understand one thing
When this is done, the socket connection is not closed as before, but
kept open after sending the response.
As I understand we just send tcp packets to make requests and responses, how this socket connection helps and how does it work? We still have to send packets, but how can it somehow establish the persistent connection? It seems so unreal.
There is overhead in establishing a new TCP connection (DNS lookups, TCP handshake, SSL/TLS handshake, etc). Without a keep-alive, every HTTP request has to establish a new TCP connection, and then close the connection once the response has been sent/received. A keep-alive allows an existing TCP connection to be re-used for multiple requests/responses, thus avoiding all of that overhead. That is what makes the connection "persistent".
In HTTP 0.9 and 1.0, by default the server closes its end of a TCP connection after sending a response to a client. The client must close its end of the TCP connection after receiving the response. In HTTP 1.0 (but not in 0.9), a client can explicitly ask the server not to close its end of the connection by including a Connection: keep-alive header in the request. If the server agrees, it includes a Connection: keep-alive header in the response, and does not close its end of the connection. The client may then re-use the same TCP connection to send its next request.
In HTTP 1.1, keep-alive is the default behavior, unless the client explicitly asks the server to close the connection by including a Connection: close header in its request, or the server decides to includes a Connection: close header in its response.
Let's make an analogy. HTTP consists in sending a request and getting the response. This is similar to asking someone a question, and receiving a response.
The problem is that the question and the answer need to go through the network. To communicate through the network, TCP (sockets) is used. That's similar to using the phone to ask a question to someone and having this person answer.
HTTP 1.0 consists, when you load a page containing 2 images for example, in
make a phone call
ask for the page
get the page
end the phone call
make a phone call
ask for the first image
get the first image
end the phone call
make a phone call
ask for the second image
get the second image
end the phone call
Making a phone call and ending it takes time and resources. Control data (like the phone number) must transit over the network. It would be more efficient to make a single phone call to get the page and the two images. That's what keep-alive allows doing. With keep-alive, the above becomes
make a phone call
ask for the page
get the page
ask for the first image
get the first image
ask for the second image
get the second image
end the phone call
This is is indeed networking question, but it may be appropriate here after all.
The confusion arises from distinction between packet-oriented and stream-oriented connections.
Internet is often called "TCP/IP" network. At the low level (IP, Internet Protocol) the Internet is packet-oriented. Hosts send packets to other hosts.
However, on top of IP we have TCP (Transmission Control Protocol). The entire purpose of this layer of the internet is to hide the packet-oriented nature of the underlying medium and to present the connection between two hosts (hosts and ports, to be more correct) as a stream of data, similar to a file or a pipe. We can then open a socket in the OS API to represent that connection, and we can treat that socket as a file descriptor (literally an FD in Unix, very similar to file HANDLE in Windows).
Most of the rest of Internet client-server protocols (HTTP, Telnet, SSH, SMTP) are layered on top of TCP. Thus a client opens a connection (a socket), writes its request (which is transmitted as one or more pockets in the underlying IP) to the socket, reads the response from a socket (and the response can contain data from multiple IP packets as well) and then... Then the choice is to keep the connection open for the next request or to close it. Pre-KeepAlive HTTP always closed the connection. New clients and servers can keep it open.
The advantage of KeepAlive is that establishing a connection is expensive. For short requests and responses it may take more packets than the actual data exchange.
The slight disadvantage may be that the server now has to tell the client where the response ends. The server cannot simply send the response and close the connection. It has to tell the client: "read 20KB and that will be the end of my response". Thus the size of the response has to be known in advance by the server and communicated to the client as part of higher-level protocol (e.g. Content-Length: in HTTP). Alternatively, the server may send a delimiter to specify the end of the response - it all depends on the protocol above TCP.
You can understand it this way:
HTTP uses TCP as transport. Before sending and receiving packets via TCP,
Client need to send the connect request
The server responds
Data transfer transfer is done
Connection is closed.
However if we are using keep-alive feature, the connection is not closed after receiving the data. The connection stays active.
This helps improving performance as for the next calls, the Connect establishment will not take place as the connection to the server is already there. This means less time taken. Although time takes in connecting is small but it do make a lot of difference in systems where every ms counts.
This question already has answers here:
What is the fundamental difference between WebSockets and pure TCP?
(4 answers)
Closed 4 years ago.
Trying to understand as best as I can the differences between TCP socket and websocket, I've already found a lot of useful information within these questions:
fundamental difference between websockets and pure TCP
How to establish a TCP Socket connection from a web browser (client side)?
and so on...
In my investigations, I went through this sentence on wikipedia:
Websocket differs from TCP in that it enables a stream of messages instead of a stream of bytes
I'm not totally sure about what it means exactly. What are your interpretations?
When you send bytes from a buffer with a normal TCP socket, the send function returns the number of bytes of the buffer that were sent. If it is a non-blocking socket or a non-blocking send then the number of bytes sent may be less than the size of the buffer. If it is a blocking socket or blocking send, then the number returned will match the size of the buffer but the call may block. With WebSockets, the data that is passed to the send method is always either sent as a whole "message" or not at all. Also, browser WebSocket implementations do not block on the send call.
But there are more important differences on the receiving side of things. When the receiver does a recv (or read) on a TCP socket, there is no guarantee that the number of bytes returned corresponds to a single send (or write) on the sender side. It might be the same, it may be less (or zero) and it might even be more (in which case bytes from multiple send/writes are received). With WebSockets, the recipient of a message is event-driven (you generally register a message handler routine), and the data in the event is always the entire message that the other side sent.
Note that you can do message based communication using TCP sockets, but you need some extra layer/encapsulation that is adding framing/message boundary data to the messages so that the original messages can be re-assembled from the pieces. In fact, WebSockets is built on normal TCP sockets and uses frame headers that contains the size of each frame and indicate which frames are part of a message. The WebSocket API re-assembles the TCP chunks of data into frames which are assembled into messages before invoking the message event handler once per message.
WebSocket is basically an application protocol (with reference to the ISO/OSI network stack), message-oriented, which makes use of TCP as transport layer.
The idea behind the WebSocket protocol consists of reusing the established TCP connection between a Client and Server. After the HTTP handshake the Client and Server start speaking WebSocket protocol by exchanging WebSocket envelopes. HTTP handshaking is used to overcome any barrier (e.g. firewalls) between a Client and a Server offering some services (usually port 80 is accessible from anywhere, by anyone). Client and Server can switch over speaking HTTP in any moment, making use of the same TCP connection (which is never released).
Behind the scenes WebSocket rebuilds the TCP frames in consistent envelopes/messages. The full-duplex channel is used by the Server to push updates towards the Client in an asynchronous way: the channel is open and the Client can call any futures/callbacks/promises to manage any asynchronous WebSocket received message.
To put it simply, WebSocket is a high level protocol (like HTTP itself) built on TCP (reliable transport layer, on per frame basis) that makes possible to build effective real-time application with JS Clients (previously Comet and long-polling techniques were used to pull updates from the Server before WebSockets were implemented. See Stackoverflow post: Differences between websockets and long polling for turn based game server ).