We have a TCP connection.
Nothing is sent over; how many traffic(bytes) are needed for each second to keep that connection open?
What is the duration of opening a connection from a client in South America to a server in North Europe?
If I have to send small amount of data (max 256bytes) at x seconds interval, what would be x for which is better to close the connection and reopen again instead of keeping the connection always open?
I do not expect exact data - estimates will suffice.
1) none.
2) some time. Try it and see. For a rough estimate, ping one end from the other and double it.
3) try it. It depends on bandwidth and, more importantly, latency. These vary over wide ranges. Usually, it's better, speed-wise, to keep connections open. 256 bytes at intervals of seconds? I would keep the connection open, especially over paths with possibly high latency, (eg. intercontinental).
1. According to the TCP/IP standard, nothing. However, depending on the network conditions and any middleboxes (NAT devices, firewalls, etc.), a connection with no data going over it may be dropped. That could be a staic timeout (say two minutes, or ten minutes, or an hour), or it could be based on a least-recently-used table in some device.
2. It depends on a lot of factors, and the biggest delay may be from the client's local network rather than the intercontinental connection. However, the surface of the earth between the points is about 40 light-millisenconds, so (without TCP Fast Open) that would be 120 ms for the first data packet to get from the client to the server and 40 ms for the response, 80 ms more than in an active connection.
3. Assuming no broken middleboxes, always better to keep the connection open. However, the delay to recover from a "silently dropped" connection may be a lot longer than the time to open a new one; it might be appropriate for the client to manage its own timeout (on hte order of a second or so), and open a new connection and retry the last message if it hasn't gotten a response by then. Depends on what you're sending; transactional messages might merit such explicit fast retry more than a remote copy of syslog.
Related
I keep hearing people say, to get a better throughput you create multiple socket connection.
But my understanding is that however many tcp sockets you open between two end points. the ip layer is still one. So not sure where this additional throughput comes from
The additional throughput comes from increasing the amount of data sent in the first couple of round trip times (RTTs). TCP can send only IW packets the first round trip time (RTT). The amount is then doubled each RTT (slow start). If you open 4 connections you can send 4 * IW packets the first RTT. The throughput is quadrupled.
Lets say that a client requests a file that requires IW+1 packets. Opening two connections can complete the sending in one RTT, rather than two RTTS.
HOWEVER, this comes at a price. The initial packets are sent as a burst, which can cause severe congestion and packet loss.
What is the time consumed in order to open a TCP socket with remote server? For example, in C, How long does it take from the time of calling connect() function until the time it returns with 0?
What are the factors affecting this time?
The major factor affecting this is the time needed for the initial TCP handshake which mostly depends on how far away (in terms on network, i.e hops and duration) the peer is. Other factors like performance of the system are mostly irrelevant as long as neither client nor server starving on resources (i.e. DOS attack).
I have a LPD server running on vxworks 6.3. The client application (over which I have no control) is sending me a LPQ query every tenth of a second. After 235 requests, the client receives a RST when trying to connect. After a time device will again accept some queries (about 300), until it again starts sending out RST.
I have confirmed that it is the TCP stack that is causing the RST. There are some things that I have noticed.
1) I can somewhat change the number of sockets that will accepted if I change the number of other applications that are running. For example, I freed up 4 sockets thereby changing the number accepted from 235 to 239.
2) If I send requests to lpr (port 515) and another port (say, port 80), the total number of connections that are accepted before the RST start happening stays constant at 235.
3) There are lots of sockets sitting TIME_WAIT.
4) I have a mock version of the client. If I slow the client down to one request every quarter second, the server doesn't reject the connections.
5) If I slow down the server's responses, I don't have any connections rejected.
So my theory is that there is some share resource (my top guess is total number of socket handles) that VxWorks can have consumed at a given time. I'm also guessing that this number tops out at 255.
Does anyone know how I can get VxWorks to accept more connections, and leave them in TIME_WAIT when closed? I have looked through the kernel configuration and changed all the values that looked remotely likely, but I have not been able change the number.
We know that we could set SO_LINGER but this not an acceptable solution. However, this does prevent the client connections from getting rejected. We have also tried changed the timeout value for SO_LINGER. This does not appear to be supported in VxWorks. It's either on or off.
Thanks!
Gail
To me it sounds like you are making a new connection for every LPQ query, and after the query is done you aren't closing the connection. In my opinion the correct thing to do is to accept one TCP connection and then use that to get all of the LPQ queries, however that may require mods to the client application. To avoid mods to the client, you should just close the TCP connection after each LPQ query.
Furthermore you can set the max number of FDs open in vxworks by adjusting the #define NUM_FILES config.h (or configall.h or one of those files), but that will just postpone an error if you have a FD leak, which you probably do.
I have a socket server which accepts multiple connections from various clients. I'm testing it on localhost with a client application which connects - sends data and closes connection 10 times every 10 ms. Some times the test client raises an error: Connection refused by the remote server or something similar.
Is this a normal behavior of the server application ?
10 connects every 10mS is one connection per millisecond, which seems a rather fast rate. Are these connection attempts being made in parallel? If so, perhaps you are filling up the server's listen() backlog-queue; IIRC clients who try to connect while the backlog-queue is full will get a connection-refused error.
To test that hypothesis, try passing in larger or smaller numbers as the second argument to listen() on your server, and see if that makes the connection-refused error occur more or less often.
I'm with Jeremy. You didn't mention the protocol, but I assume it's SOCK_STREAM. It will take longer than 10ms to do the tcp handshake on anything but the most local connection, eventually causing a backlog (and subsequent connection refused error) no matter how high you set your listen backlog to.
You'd be way ahead if you could keep the connection open, and not close it down during each of your computation cycles.
I'd like to know the general cost of creating a new connection, compared to UDP. I know TCP requires an initial exchange of packets (the 3 way handshake). What would be other costs? For instance is there some sort of magic in the kernel needed for setting up buffers etc?
The reason I'm asking is I can keep an existing connection open and reuse it as needed. However if there is little overhead reconnecting it would reduce complexity.
Once a UDP packet's been dumped onto the wire, the UDP protocol stack is free to completely forget about it. With TCP, there's at bare minimum the connection details (source/dest port and source/dest IP), the sequence number, the window size for the connection etc... It's not a huge amount of data, but adds up quickly on a busy server with many connections.
And then there's the 3-way handshake as well. Some braindead (and/or malicious systems) can abuse the process (look up 'syn flood'), or just drop the connection on their end, leaving your system waiting for a response or close notice that'll never come. The plus side is that with TCP the system will do its best to make sure the packet gets where it has to. With UDP, there's no guarantees at all.
Compared to the latency of the packet exchange, all other costs such as kernel setup times are insignificant.
OPTION 1: The general cost of creating a TCP connection are:
Create socket connection
Send data
Tear down socket connection
Step 1: Requires an exchange of packets, so it's delayed by to & from network latency plus the destination server's service time. No significant CPU usage on either box is involved.
Step 2: Depends on the size of the message.
Step 3: IIRC, just sends a 'closing now' packet, w/ no wait for destination ack, so no latency involved.
OPTION 2: Costs of UDP:*
Create UDP object
Send data
Close UDP object
Step 1: Requires minimal setup, no latency worries, very fast.
Step 2: BE CAREFUL OF SIZE, there is no retransmit in UDP since it doesn't care if the packet was received by anyone or not. I've heard that the larger the message, the greater probability of data being received corrupted, and that a rule of thumb is that you'll lose a certain percentage of messages over 20 MB.
Step 3: Minimal work, minimal time.
OPTION 3: Use ZeroMQ Instead
You're comparing TCP to UDP with a goal of reducing reconnection time. THERE IS A NICE COMPROMISE: ZeroMQ sockets.
ZMQ allows you to set up a publishing socket where you don't care if anyone is listening (like UDP), and have multiple listeners on that socket. This is NOT a UDP socket - it's an alternative to both of these protocols.
See: ZeroMQ.org for details.
It's very high speed and fault tolerant, and is in increasing use in the financial industry for those reasons.