I've got two small programs communicating nicely over a socket where the receiving side is in Go. Everything works peachy when my messages are tiny enough to fit in the 1024 byte buffer and can be received in a single Read from the connection but now I want to transfer data from an image that is 100k+ or more. I'm assuming the correct solution is not to increase the buffer until any image can fit inside.
Pseudo-go:
var buf = make([]byte,1024)
conn, err := net.Dial("tcp", ":1234")
for {
r, err := conn.Read(buf[0:])
go readHandler(string(buf[0:r]),conn)
}
How can I improve my socket read routine to accept both simple messages of a few bytes and also larger data? Bonus points if you can turn the total image data into an io.Reader for use in image.Decode.
I have no direct experience with TCP in Go but to me it seems that you fell victim of a quite typical misunderstanding of what guarntees TCP offers.
The thing is, in contrast with, say, UDP and SCTP, TCP does not have the concept of message boundaries because it's stream-oriented. It means, TCP transports opaque streams of bytes and you have very little control of "chunking" that stream with regard to the receiving side.
I suspect what you observe as "sending a 100k+ message" is the runtime/network library on the sender side typically "deceiving" you by consuming your "message" into its internal buffers and then streaming it in whatever chunks OS's TCP stack allows it to (on ubiquitous hardware/software it's usually about 8k). The size of pieces the receiver gets that stream is completely undefined; the only thing defined is ordering of the bytes in the stream, which is preserved.
Hence it might turn out you have to resonsider your approach to receiving data. The exact approach varies depending on the nature of the data being streamed:
The easiest way (if you have the control over the application-level protocol) is to pass the length of the following "message payload" in a special length field of fixed format. Then destreaming the whole message is a two-step process: 1) receive that many bytes to get the length field, read it, check the value for sanity, then 2) read that many following bytes and be done with it.
If you have no control over the app-level protocol, parsing messages becomes more involved and usually requires some sort of complicated state machine.
For more info, look at this and this.
You can use io.ReadFull to read a []byte of a specific length. This assumes that you know beforehand how many bytes you need to read.
As for image.Decode, it should be possible to pass the conn directly to the image.Decode function. This assumes that you do not perform any reads from the connection until the image is decoded.
Your code
for {
r, err := conn.Read(buf[0:])
go readHandler(string(buf[0:r]),conn)
}
seems to be suggesting that the goroutine you are starting is reading from conn This doesn't seem like a good idea, because you will end up having multiple concurrent reads from the connection (without having control over the order in which the reads will happen): one in the for-loop, another one in readHandler.
Related
Problem
I want to run a load test with a high number of requests per second. I have written a socket sender and a receiver in Go. The sender sends a lot of packets to port 7357, each one containing the current time in nanoseconds. The receiver listens in port 7357 and parses each message, computing the latency.
The problem is that when reading I get multiple packets in one conn.Read(). I understand that this means that I am in fact sending multiple messages per packet: each conn.Write() does not send a socket packet, but it waits for some time and then gets coalesced with the next (or the next few) before sending.
Question
How can I make sure that each conn.Write() is sent individually through the socket as a separate packet? Note: I don't want to reinvent TCP, I just want to simulate the load from a number of external entities that send a message each.
Steps Taken
I have searched the documentation but there seems to be no conn.Flush() or similar. I have tried using a buffered writer:
writer := bufio.NewWriter(conn)
...
bytes, err := writer.Write(message)
err = writer.Flush()
No errors, but still I get mixed packets at the receiving end. I have also tried doing a fake conn.Read() of 0 bytes after every conn.Write(), but it didn't work either. Sending a message terminator such as \r\n does not seem to make any difference. Finally, Nagle algorithm is disabled by default, but I have called tcp.SetNoDelay(true) for good measure.
In Node.js I managed to do the trick with a setImmediate() after each socket.write(): setImmediate() waits for all I/O to finish before continuing. How can I do the same in Go so I get separate packets?
Code Snippets
Send:
func main() {
conn, _ := net.Dial("tcp", ":7357")
defer conn.Close()
for {
timestamp := strconv.FormatInt(time.Now().UnixNano(), 10)
conn.Write([]byte(timestamp))
conn.Read(buff)
}
}
Receive:
func main() {
listen, _ := net.Listen("tcp4", ":7357")
defer listen.Close()
for {
conn, _ := listen.Accept()
go handler(conn)
}
}
func handler(conn net.Conn) {
defer conn.Close()
var buf = make([]byte, 1024)
for {
conn.Read(buf)
data := string(buf[:n])
timestamp, _ := strconv.ParseInt(data, 10, 64)
elapsed := timestamp - time.Now().UnixNano()
log.Printf("Elapsed %v", elapsed)
}
}
Error handling has been removed for legibility, but it is thoroughly checked in the actual code. It crashes when running the strconv.ParseInt() the first time, with a value out of range error since it receives a lot of timestamps coalesced.
There used to a be a rule that before anyone was permitted to write any code that uses TCP, they were required to repeat the following sentence from memory and explain what it means: "TCP is not a message protocol, it is a reliable byte-stream protocol that does not preserve application message boundaries."
Aside from the fact that your suggested solution is simply not possible reliably with TCP, it is not the solution to reducing latency. If the network is overwhelmed, using more packets to send the same data will just make the latency worse.
TCP is a byte stream protocol. The service it provides is a stream of bytes. Period.
It seems that you want a low-latency message protocol that works over TCP. Great. Design one and implement it.
The main trick to getting low latency is to use application-level acknowledgements. The TCP ACK flag will piggy-back onto the acknowledgements, providing low latency.
Do not disable Nagling. That's a hack that's only needed when you can't design a proper protocol that's intended to work with TCP in the first place. It will make latency worse under non-ideal conditions for same reason the solution you suggested, even if it were possible, would be a poor idea.
But you MUST design and implement a message protocol or use an existing one. Your code is expecting TCP, which is not a message protocol, to somehow deliver messages to it. That is just not going to happen, period.
How can I make sure that each conn.Write() is sent individually through the socket as a separate packet? Note: I don't want to reinvent TCP, I just want to simulate the load from a number of external entities that send a message each.
Even if you could, that wouldn't do what you want anyway. Even if they were sent in separate packets, that would not guarantee that read on the other side wouldn't coalesce. If you want to send and receive messages, you need a message protocol which TCP is not.
In Node.js I managed to do the trick with a setImmediate() after each socket.write(): setImmediate() waits for all I/O to finish before continuing. How can I do the same in Go so I get separate packets?
You may have changed it from "happens not to work" to "happened to work when I tried it". But for the reasons I've explained, you can never make this work reliably and you are on a fool's errand.
If you want to send and receive messages, you need to precisely defined what a "message" will be and write code to send and receive them. There are no shortcuts that are reliable. TCP is a byte stream protocol, period.
If you care about latency and throughput, design an optimized message protocol to layer over TCP that optimizes these. Do not disable Nagle as Nagle is required to prevent pathological behavior. It should only be disabled when you cannot change the protocol and are stuck with a protocol that was not designed to layer on top of TCP. Disabling Nagle ties one hand behind your back and causes dramatically worse latency and throughput under poor network conditions by increasing the number of packets required to send the data even when that doesn't make any sense.
You probably want/need application-level acknowledgements. This works nicely with TCP because TCP ACKs will piggyback on the application-level acknowledgements.
You can read predefined number of bytes from the socket on each iteration, that might help, but you need to create your own protocol, that will be handled by your application. Without proto impossible to guarantee that everything will work stable, because on the receiver you cannot understand where is the begin and where is the end of message.
Although it's possible to read from a Gio.Socket by wrapping it's file-descriptor in Gio.DataInputStream, using Gio.Socket.receive_from() in GJS to receive is not possible because as commented here:
GJS will clone array arguments before passing them to the C-code which will make the call to Socket.receive_from work and return the number of bytes received as well as the source of the packet. The buffer content will be unchanged as buffer actually read into is a freed clone.
Thus, input arguments are cloned and data will be written to the cloned buffer, not the instance of buffer actually passed in.
Although reading from a data stream is not a problem, Gio.Socket.receive_from() is the only way I can find to get the remote address from a UDP listener, since Gio.Socket.remote_address will be undefined. Unfortunately as the docs say for Gio.Socket.receive():
For G_SOCKET_TYPE_DATAGRAM [...] If the received message is too large to fit in buffer, then the data beyond size bytes will be discarded, without any explicit indication that this has occurred.
So if I try something like Gio.Socket.receive_from(new Uint8Array(0), null); just to get the address, the packet is swallowed, but if I read via the file-descriptor I can't tell where the message came from. Is there another non-destructive way to get the incoming address for a packet?
Since you’re using a datagram socket, it should be possible to use Gio.Socket.receive_message() and pass the Gio.SocketMsgFlags.PEEK flag to it. This isn’t possible for a stream-based socket, but you are not going to want the sender address for each read you do in that case.
If you want improved performance, you may be able to use Gio.Socket.receive_messages(), although I am not sure whether that’s completely introspectable at the moment.
I am a newbie in socket programming(in C), maybe this question is a litter bit stupid. In C socket programming, how should I determine the size of buffer of the function recv()/read()? As in many cases, we don't know the size of data sent using send()/write(). Thanks a lot!
how should I determine the size of buffer of the function
recv()/read()
Ideally one shouldn't look at these buffers and keep to the olden TCP model: keep reading bytes while bytes are available.
If you are asking this question for things like: "how big should be the buffer into which I receive?", the simple answer is to pick a size and just pass that. If there's more data you can read again.
Back to your original question, different stacks give you different APIs. For example on some Unixes you have things like SIOCINQ and FIONREAD. These give you the amount of data the kernel has in its receive buffer, waiting for you to copy it out.
If you don't really know how many bytes are expected, use a large buffer and pass a large buffer size to recv/read. These functions will return how many bytes were put into the buffer. Then you can deal with this data printing it, for example.
But keep in mind that data is often either sent in chunks of known size or sent with a message-size in the first bytes, so the receiver side is able to identify how many bytes should be read.
I user gen_tcp:recv(Socket, 0). for data receiveng, but i can receive only 1418 bytes for 1 time. How can I receive how much data was sent?
in gen_tcp:recv(Socket, 0) you are asking the kernel: "Give me all data there is available right now in the receive buffer". The kernel is also free to give you less however. Even for a rather fast link, you will probably hit slow start on the TCP connection so in the beginning you will not get much data.
The solution is to do your own buffering. You will have to eat data from the underlying socket until you have enough to construct a message. It is quite common for binary protocols to implement their own kind of messaging on top of the stream due to this.
For the longer term record: A common message format is to encode a message as:
decode(Bin) when is_binary(Bin) ->
<<Len:32/integer, R/binary>> = Bin,
<<Payload:Len/binary, Remain/binary>>,
{msg, {Len, Payload}, Remaining}.
That is, messages are 4 bytes representing a 32-bit bigendian integer followed by the payload, where the length is given by the integer. This format, and others like it, are so common Erlang includes optimized parsers for it directly in the C-layer. To get access to these, you set options on the socket through inet/setops/2, in our case we set {packet, 4}. Then we can get messages by setting {active, once} on the socket and wait for the next message. When it arrives, we can {active, once} again on the socket to get the next message, and so on. There is an example in the documentation of gen_tcp (erl -man gen_tcp if you have the Erlang man-pages installed appropriately).
Other common formats are asn.1 or even http headers(!).
Tricks
It is often beneficial to create a process which is separate that can encode and decode your message format and then send on data to the rest of the system. Usually a good solution in Erlang is to demux incoming data as fast as possible and get the data to a process which can then handle the rest of the problem.
Recently, while reading a Socket Programming HOWTO the following section jumped out at me:
But if you plan to reuse your socket for further transfers, you need to realize that there is no "EOT" (End of Transfer) on a socket. I repeat: if a socket send or recv returns after handling 0 bytes, the connection has been broken. If the connection has not been broken, you may wait on a recv forever, because the socket will not tell you that there's nothing more to read (for now). Now if you think about that a bit, you'll come to realize a fundamental truth of sockets: messages must either be fixed length (yuck), or be delimited (shrug), or indicate how long they are (much better), or end by shutting down the connection. The choice is entirely yours, (but some ways are righter than others).
This section highlights 4 possibilities for how a socket "protocol" may be written to pass messages. My question is, what is the preferred method to use for real applications?
Is it generally best to include message size with each message (presumably in a header), as the article more or less asserts? Are there any situations where another method would be preferable?
The common protocols either specify length in the header, or are delimited (like HTTP, for instance).
Keep in mind that this also depends on whether you use TCP or UDP sockets. Since TCP sockets are reliable you can be sure that you get everything you shoved into them. With UDP the story is different and more complex.
These are indeed our choices with TCP. HTTP, for example, uses a mix of second, third, and forth option (double new-line ends request/response headers, which might contain the Content-Length header or indicate chunked encoding, or it might say Connection: close and not give you the content length but expect you to rely on reading EOF.)
I prefer the third option, i.e. self-describing messages, though fixed-length is plain easy when suitable.
If you're designing your own protocol then look at other people's work first; there might already be something similar out there that you could either use 'as is' or repurpose and adjust. For example; ISO-8583 for financial txns, HTTP or POP3 all do things differently but in ways that are proven to work... In fact it's worth looking at these things anyway as you'll learn a lot about how real world protocols are put together.
If you need to write your own protocol then, IMHO, prefer length prefixed messages where possible. They're easy and efficient to parse for the receiver but possibly harder to generate if it is costly to determine the length of the data before you begin sending it.
The decision should depend on the data you want to send (what it is, how is it gathered). If the data is fixed length, then fixed length packets will probably be the best. If data can be easily (no escaping needed) split into delimited entities then delimiting may be good. If you know the data size when you start sending the data piece, then len-prefixing may be even better. If the data sent is always single characters, or even single bits (e.g. "on"/"off") then anything different than fixed size one character messages will be too much.
Also think how the protocol may evolve. EOL-delimited strings are good as long as they do not contain EOL characters themselves. Fixed length may be good until the data may be extended with some optional parts, etc.
I do not know if there is a preferred option. In our real-world situation (client-server application), we use the option of sending the total message length as one of the first pieces of data. It is simple and works for both our TCP and UDP implementations. It makes the logic reasonably "simple" when reading data in both situations. With TCP, the amount of code is fairly small (by comparison). The UDP version is a bit (understatement) more complex but still relies on the size that is passed in the initial packet to know when all data has been sent.