I've been debugging AoE in one of my setups.
In it I noticed that the AoE packets do not seem to have checksum fields on the frames.
I looked up the AoE specification and it doesn't seem to have CRC checksums on the ends of the packets defined.
Is the checksum field optional?
Found this AoE pdf with frame diagrams showing the crc.
It looks like originated from Coraid.
Related
I am attempting to use libMPSSE to perform I2C communications. The example code listed in the attached document connects to a 24LC024H EEPROM device.
The address for the device used in the example as defined in it's documentation is 1010XXX_ where the X's are configurable. In the examples associated diagram you can see the values are configured to be 1. It also states that the R/W bit (_) should not be included meaning the address passed to the library should be 10101110. The address actually used in the example code is 0x57 which is 01010111.
I do not see how we got from A to B here. I cannot figure out how to format the address of the device I am trying to communicate with nor can I find any documentation spelling it out. The only documenation on the address parameter says:
Address of the I2C slave. This is a 7bit value and it
should not contain the data direction bit, i.e. the
decimal value passed should be always less than 128
This confusing since the data direction bit is usually the LSB.
I was updating my question to clarify what the address should be and a coincidence in the editor cause the answer to smack me in the face.
By "should not be included" they do not mean that the bit should be zero but rather by completely nonexistent. To them this means shifting the address bits down to remove it as the LSB. It also implies that the MSB should always be zero even though it's not explicitly defined anywhere.
I want to decode a MP3 file. I manage to find the 32 bits in the header (sync word, ID, Layer, Bitrate, etc). The problem is I have no idea on how to find the starting (the position) of main_data_begin (side information). I am using MATLAB in this case.
I know it may be a simple question, but I really need your help. Please.
Thank you.
MPEG1/2 Layer III uses main_data_begin as a kind of pseudo-VBR over the granule headers & data. The simplest way to do it is to implement a circular buffer that receives all the physical frame data after the side info and throws-away the unused bytes at the beginning of the buffer (as indicated by main_data_begin) before starting frame decode.
Your best bet is to read an existing decoder's source. The spec is also really good for this, but main_data_begin is mis-documented in publicly-available versions (as best as I can find).
I user gen_tcp:recv(Socket, 0). for data receiveng, but i can receive only 1418 bytes for 1 time. How can I receive how much data was sent?
in gen_tcp:recv(Socket, 0) you are asking the kernel: "Give me all data there is available right now in the receive buffer". The kernel is also free to give you less however. Even for a rather fast link, you will probably hit slow start on the TCP connection so in the beginning you will not get much data.
The solution is to do your own buffering. You will have to eat data from the underlying socket until you have enough to construct a message. It is quite common for binary protocols to implement their own kind of messaging on top of the stream due to this.
For the longer term record: A common message format is to encode a message as:
decode(Bin) when is_binary(Bin) ->
<<Len:32/integer, R/binary>> = Bin,
<<Payload:Len/binary, Remain/binary>>,
{msg, {Len, Payload}, Remaining}.
That is, messages are 4 bytes representing a 32-bit bigendian integer followed by the payload, where the length is given by the integer. This format, and others like it, are so common Erlang includes optimized parsers for it directly in the C-layer. To get access to these, you set options on the socket through inet/setops/2, in our case we set {packet, 4}. Then we can get messages by setting {active, once} on the socket and wait for the next message. When it arrives, we can {active, once} again on the socket to get the next message, and so on. There is an example in the documentation of gen_tcp (erl -man gen_tcp if you have the Erlang man-pages installed appropriately).
Other common formats are asn.1 or even http headers(!).
Tricks
It is often beneficial to create a process which is separate that can encode and decode your message format and then send on data to the rest of the system. Usually a good solution in Erlang is to demux incoming data as fast as possible and get the data to a process which can then handle the rest of the problem.
I am studying a simple web server using c, and came up with some of these questions. How does IPv6 used in TCP? To use IPv6, do we have to use some form of modified version of TCP?? If we have to used the modified version of TCP, what do we have to change?? I think I read about Little Endian, as well as Big Endian, but I am not sure if there should be some special cases for IPv6.
As you'll probably be wanting the more gory details of the API changes, it's here: http://www.faqs.org/rfcs/rfc2553.html
Mostly it's a couple of longer address structures to pass in that can take a longer number and a new Family and Protocol name specified so the API can destiguish which struct you are using. Byte ordering is the same.
The actual TCP SYN, SYN/ACK, ACK stuff and all that is identical, it is literally a different IP layer frame with a longet number and other changes.
Recently, while reading a Socket Programming HOWTO the following section jumped out at me:
But if you plan to reuse your socket for further transfers, you need to realize that there is no "EOT" (End of Transfer) on a socket. I repeat: if a socket send or recv returns after handling 0 bytes, the connection has been broken. If the connection has not been broken, you may wait on a recv forever, because the socket will not tell you that there's nothing more to read (for now). Now if you think about that a bit, you'll come to realize a fundamental truth of sockets: messages must either be fixed length (yuck), or be delimited (shrug), or indicate how long they are (much better), or end by shutting down the connection. The choice is entirely yours, (but some ways are righter than others).
This section highlights 4 possibilities for how a socket "protocol" may be written to pass messages. My question is, what is the preferred method to use for real applications?
Is it generally best to include message size with each message (presumably in a header), as the article more or less asserts? Are there any situations where another method would be preferable?
The common protocols either specify length in the header, or are delimited (like HTTP, for instance).
Keep in mind that this also depends on whether you use TCP or UDP sockets. Since TCP sockets are reliable you can be sure that you get everything you shoved into them. With UDP the story is different and more complex.
These are indeed our choices with TCP. HTTP, for example, uses a mix of second, third, and forth option (double new-line ends request/response headers, which might contain the Content-Length header or indicate chunked encoding, or it might say Connection: close and not give you the content length but expect you to rely on reading EOF.)
I prefer the third option, i.e. self-describing messages, though fixed-length is plain easy when suitable.
If you're designing your own protocol then look at other people's work first; there might already be something similar out there that you could either use 'as is' or repurpose and adjust. For example; ISO-8583 for financial txns, HTTP or POP3 all do things differently but in ways that are proven to work... In fact it's worth looking at these things anyway as you'll learn a lot about how real world protocols are put together.
If you need to write your own protocol then, IMHO, prefer length prefixed messages where possible. They're easy and efficient to parse for the receiver but possibly harder to generate if it is costly to determine the length of the data before you begin sending it.
The decision should depend on the data you want to send (what it is, how is it gathered). If the data is fixed length, then fixed length packets will probably be the best. If data can be easily (no escaping needed) split into delimited entities then delimiting may be good. If you know the data size when you start sending the data piece, then len-prefixing may be even better. If the data sent is always single characters, or even single bits (e.g. "on"/"off") then anything different than fixed size one character messages will be too much.
Also think how the protocol may evolve. EOL-delimited strings are good as long as they do not contain EOL characters themselves. Fixed length may be good until the data may be extended with some optional parts, etc.
I do not know if there is a preferred option. In our real-world situation (client-server application), we use the option of sending the total message length as one of the first pieces of data. It is simple and works for both our TCP and UDP implementations. It makes the logic reasonably "simple" when reading data in both situations. With TCP, the amount of code is fairly small (by comparison). The UDP version is a bit (understatement) more complex but still relies on the size that is passed in the initial packet to know when all data has been sent.