I've read many stack overflow questions similar to this, but I don't think any of the answers really satisfied my curiosity. I have an example below which I would like to get some clarification.
Suppose the client is blocking on socket.recv(1024):
socket.recv(1024)
print("Received")
Also, suppose I have a server sending 600 bytes to the client. Let us assume that these 600 bytes are broken into 4 small packets (of 150 bytes each) and sent over the network. Now suppose the packets reach the client at different timings with a difference of 0.0001 seconds (eg. one packet arrives at 12.00.0001pm and another packet arrives at 12.00.0002pm, and so on..).
How does socket.recv(1024) decide when to return execution to the program and allow the print() function to execute? Does it return execution immediately after receiving the 1st packet of 150 bytes? Or does it wait for some arbitrary amount of time (eg. 1 second, for which by then all packets would have arrived)? If so, how long is this "arbitrary amount of time"? Who determines it?
Well, that will depend on many things, including the OS and the speed of the network interface. For a 100 gigabit interface, the 100us is "forever," but for a 10 mbit interface, you can't even transmit the packets that fast. So I won't pay too much attention to the exact timing you specified.
Back in the day when TCP was being designed, networks were slow and CPUs were weak. Among the flags in the TCP header is the "Push" flag to signal that the payload should be immediately delivered to the application. So if we hop into the Waybak
machine the answer would have been something like it depends on whether or not the PSH flag is set in the packets. However, there is generally no user space API to control whether or not the flag is set. Generally what would happen is that for a single write that gets broken into several packets, the final packet would have the PSH flag set. So the answer for a slow network and weakling CPU might be that if it was a single write, the application would likely receive the 600 bytes. You might then think that using four separate writes would result in four separate reads of 150 bytes, but after the introduction of Nagle's algorithm the data from the second to fourth writes might well be sent in a single packet unless Nagle's algorithm was disabled with the TCP_NODELAY socket option, since Nagle's algorithm will wait for the ACK of the first packet before sending anything less than a full frame.
If we return from our trip in the Waybak machine to the modern age where 100 Gigabit interfaces and 24 core machines are common, our problems are very different and you will have a hard time finding an explicit check for the PSH flag being set in the Linux kernel. What is driving the design of the receive side is that networks are getting way faster while the packet size/MTU has been largely fixed and CPU speed is flatlining but cores are abundant. Reducing per packet overhead (including hardware interrupts) and distributing the packets efficiently across multiple cores is imperative. At the same time it is imperative to get the data from that 100+ Gigabit firehose up to the application ASAP. One hundred microseconds of data on such a nic is a considerable amount of data to be holding onto for no reason.
I think one of the reasons that there are so many questions of the form "What the heck does receive do?" is that it can be difficult to wrap your head around what is a thoroughly asynchronous process, wheres the send side has a more familiar control flow where it is much easier to trace the flow of packets to the NIC and where we are in full control of when a packet will be sent. On the receive side packets just arrive when they want to.
Let's assume that a TCP connection has been set up and is idle, there is no missing or unacknowledged data, the reader is blocked on recv, and the reader is running a fresh version of the Linux kernel. And then a writer writes 150 bytes to the socket and the 150 bytes gets transmitted in a single packet. On arrival at the NIC, the packet will be copied by DMA into a ring buffer, and, if interrupts are enabled, it will raise a hardware interrupt to let the driver know there is fresh data in the ring buffer. The driver, which desires to return from the hardware interrupt in as few cycles as possible, disables hardware interrupts, starts a soft IRQ poll loop if necessary, and returns from the interrupt. Incoming data from the NIC will now be processed in the poll loop until there is no more data to be read from the NIC, at which point it will re-enable the hardware interrupt. The general purpose of this design is to reduce the hardware interrupt rate from a high speed NIC.
Now here is where things get a little weird, especially if you have been looking at nice clean diagrams of the OSI model where higher levels of the stack fit cleanly on top of each other. Oh no, my friend, the real world is far more complicated than that. That NIC that you might have been thinking of as a straightforward layer 2 device, for example, knows how to direct packets from the same TCP flow to the same CPU/ring buffer. It also knows how to coalesce adjacent TCP packets into larger packets (although this capability is not used by Linux and is instead done in software). If you have ever looked at a network capture and seen a jumbo frame and scratched your head because you sure thought the MTU was 1500, this is because this processing is at such a low level it occurs before netfilter can get its hands on the packet. This packet coalescing is part of a capability known as receive offloading, and in particular lets assume that your NIC/driver has generic receive offload (GRO) enabled (which is not the only possible flavor of receive offloading), the purpose of which is to reduce the per packet overhead from your firehose NIC by reducing the number of packets that flow through the system.
So what happens next is that the poll loop keeps pulling packets off of the ring buffer (as long as more data is coming in) and handing it off to GRO to consolidate if it can, and then it gets handed off to the protocol layer. As best I know, the Linux TCP/IP stack is just trying to get the data up to the application as quickly as it can, so I think your question boils down to "Will GRO do any consolidation on my 4 packets, and are there any knobs I can turn that affect this?"
Well, the first thing you can do is disable any form of receive offloading (e.g. via ethtool), which I think should get you 4 reads of 150 bytes for 4 packets arriving like this in order, but I'm prepared to be told I have overlooked another reason why the Linux TCP/IP stack won't send such data straight to the application if the application is blocked on a read as in your example.
The other knob you have if GRO is enabled is GRO_FLUSH_TIMEOUT which is a per NIC timeout in nanoseconds which can be (and I think defaults to) 0. If it is 0, I think your packets may get consolidated (there are many details here including the value of MAX_GRO_SKBS) if they arrive while the soft IRQ poll loop for the NIC is still active, which in turn depends on many things unrelated to your four packets in your TCP flow. If non-zero, they may get consolidated if they arrive within GRO_FLUSH_TIMEOUT nanoseconds, though to be honest I don't know if this interval could span more than one instantiation of a poll loop for the NIC.
There is a nice writeup on the Linux kernel receive side here which can help guide you through the implementation.
A normal blocking receive on a TCP connection returns as soon as there is at least one byte to return to the caller. If the caller would like to receive more bytes, they can simply call the receive function again.
I've been working for 2 months in a MODBUS project and now I found a problem.
My client is asking me to write in an input register (Address 30001 to 40000).
I thought that was not a thing for me because every modbus documentation says that 30001 to 40000 registers are read-only.
Is it even possible to write in those registers? Thanks in advance
Both holding and input register related functions contain a 2-byte address value. This means that you can have 65536 input registers and 65536 holding registers in a device at the same time.
If your client is developing the firmware of the slave, they can place holding registers into the 3xxxx - 4xxxx area. They don't need to follow the memory layout of the original Modicon devices.
If one can afford diverging from the Modbus standard, it's even possible to increase the number of registers. In one of my projects, I was considering to use Preset Single Register (06) function as a bank select command. Of course, you can't call it Modbus anymore. But, the master can still access the slave using a standard library or diagnostics tools.
You can't write to Input Contacts or Input Registers, there is no Modbus function to write to them, they are read only by definition
Modbus is a protocol and in no case specifies where the values are stored, only how they are transmitted
Currently there are devices that support 6-digit addresses and therefore can address up to 65536 registers per group
In Ethernet networks, the MAC layer is the first layer to detect the destination address of the received message.
my questions: is that means that the transceiver shall take a copy of each message on the bus and forward it to the MAC layer who will decide to accept that message or discard it? If so, this means that the MAC layer must have a very large buffers to save all that intended and non intended message. am I correct ?
The MAC layer does not typically have much buffering. It may not even be able to store a full packet. Packets instead stream through the MAC.
Packets enter and exit the MAC one flit at a time. It may take hundreds of cycles for a full packet to pass into a MAC depending on the size of the packet and the width of the interface. For example, a MAC with an 8-byte interface (8-byte flit size) will take 1000 cycles to receive an 8kB packet.
The MAC may only have 800 bytes of buffering. In that case, the packet will start coming out the other end after 100 cycles when only 10% of the packet has entered. In fact, many MACs have a latency well below 100 cycles.
Packets which are rejected on the basis of destination address stream in one side but nothing comes out the other side. The frames are simply forgotten/dropped as they arrive.
I am using MAX144 ADC and in the Datasheet there is no information given about the control register to read the ADC values. I am using STM32L452RE micro-controller and using SPI to get data from ADC. Datasheet of the ADC is:
https://datasheets.maximintegrated.com/en/ds/MAX144-MAX145.pdf
anyone who encountered the same problem please guide.
my idea is to create a buffer of 2 bytes for SPI RX and store values in it. but i don't know what control register address should be assigned to it.
The conversion data is not stored internally in a register set. When you pull CS low the state of SCLK will determine wither it holds the conversion product(after a high to low transition to start it) or start streaming it on the falling edge of the second clock pulse.
This is all noted on page 9 of the data sheet. Pages 10 & 11 detail how to interface them to standard SPI.
As you can see in the below picture of the AT24C512C datasheets from ATMEL,after reading desired data from EEPROM,there is a NOT ACK bit following the data which I don't understand is produced by the EEPROM or MCU(master)?
As you know past ACK bits in the writing data to EEPROM was produced by EEPROM to acknowledging a correct data receiving.Reading section
NOT ACK bit is produced by MCU (master) to generate stop condition here and if there is any error while slave receiving the data then it would be hardware generated by the slave or you can say EEPROM. It is a two-way communication.
If you are using this module I will suggest you to use random read by providing the address on ehich data is written and you read the same address. Then , it would work. It would be similar to codes in the following link. You can choose the language as per the mcu you are using.
https://github.com/ControlEverythingCommunity/AT24HC02C