Related
I have a setup with 2 SDM120 kWh energy meters daisy chained on the same serial port (in the future I want to add a SDM630). I found "Using multiple instruments" in the MinimalModbus communication. I succeed in reading registers on the SDM120 on address 1, but I get an error on reading address 2. The error: minimalmodbus.NoResponseError: No communication with the instrument (no answer).
I can work around it by adding time.sleep(0.1), but I would think that RS485 allows to immediately read the registers of a second address after the first one is completed. I also tried lower values, but eg. time.sleep(0.01) also gave a NoResponseError.
I personally thought the setting instrument.serial.timeout = 1 already would have had the desired effect, but apparently I really need the time.sleep. Is the time.sleep(0.1) the correct way of doing? If so: how can I know the lowest value, so that I don't have a NoResponseError? Trial and error? Could my script be optimized? Especially when timing is important, eg. to avoid injection in the grid (pv diverter, ...). Thanks in advance!
The script:
#!/usr/bin/env python3
import minimalmodbus
import time
instrumentA = minimalmodbus.Instrument('/dev/ttyUSB0', 1, debug = True) # port name, slave address (in decimal)
instrumentA.serial.baudrate = 9600
instrumentA.serial.timeout = 1 # seconds
instrumentA.serial.bytesize = 8
instrumentA.serial.parity = minimalmodbus.serial.PARITY_NONE
instrumentA.serial.stopbits = 1
instrumentA.mode = minimalmodbus.MODE_RTU
instrumentB = minimalmodbus.Instrument('/dev/ttyUSB0', 2, debug = True)
instrumentB.mode = minimalmodbus.MODE_RTU
print ("====== SDM120 instrumentA on addres 1 ======")
print (instrumentA)
P = instrumentA.read_float(12, 4, 2)
print ("Active Power in Watts:", P)
#time.sleep(0.1) #workaround to avoid NoResponseError
print ("====== SDM120 instrumentB on addres 2 ======")
print (instrumentB)
P = instrumentB.read_float(12, 4, 2)
print ("Active Power in Watts:", P)
Output without the time.sleep(0.1):
MinimalModbus debug mode. Create serial port /dev/ttyUSB0
MinimalModbus debug mode. Serial port /dev/ttyUSB0 already exists
====== SDM120 instrumentA on addres 1 ======
minimalmodbus.Instrument<id=0x7f36e3dc0df0, address=1, mode=rtu, close_port_after_each_call=False, precalculate_read_size=True, clear_buffers_before_each_transaction=True, handle_local_echo=False, debug=True, serial=Serial<id=0x7f36e3dd90d0, open=True>(port='/dev/ttyUSB0', baudrate=9600, bytesize=8, parity='N', stopbits=1, timeout=1, xonxoff=False, rtscts=False, dsrdtr=False)>
MinimalModbus debug mode. Will write to instrument (expecting 9 bytes back): '\x01\x04\x00\x0c\x00\x02±È' (01 04 00 0C 00 02 B1 C8)
MinimalModbus debug mode. Clearing serial buffers for port /dev/ttyUSB0
MinimalModbus debug mode. No sleep required before write. Time since previous read: 190954.73 ms, minimum silent period: 4.01 ms.
MinimalModbus debug mode. Response from instrument: '\x01\x04\x04\x00\x00\x00\x00û\x84' (01 04 04 00 00 00 00 FB 84) (9 bytes), roundtrip time: 53.3 ms. Timeout for reading: 1000.0 ms.
Active Power in Watts: 0.0
====== SDM120 instrumentB on addres 2 ======
minimalmodbus.Instrument<id=0x7f36e3c55940, address=2, mode=rtu, close_port_after_each_call=False, precalculate_read_size=True, clear_buffers_before_each_transaction=True, handle_local_echo=False, debug=True, serial=Serial<id=0x7f36e3dd90d0, open=True>(port='/dev/ttyUSB0', baudrate=9600, bytesize=8, parity='N', stopbits=1, timeout=1, xonxoff=False, rtscts=False, dsrdtr=False)>
MinimalModbus debug mode. Will write to instrument (expecting 9 bytes back): '\x02\x04\x00\x0c\x00\x02±û' (02 04 00 0C 00 02 B1 FB)
MinimalModbus debug mode. Clearing serial buffers for port /dev/ttyUSB0
MinimalModbus debug mode. Sleeping 2.31 ms before sending. Minimum silent period: 4.01 ms, time since read: 1.70 ms.
MinimalModbus debug mode. Response from instrument: '' () (0 bytes), roundtrip time: 1001.3 ms. Timeout for reading: 1000.0 ms.
Traceback (most recent call last):
File "./sdm120-daisychain_v3.py", line 25, in <module>
P = instrumentB.read_float(12, 4, 2)
File "/home/mattias/.local/lib/python3.8/site-packages/minimalmodbus.py", line 662, in read_float
return self._generic_command(
File "/home/mattias/.local/lib/python3.8/site-packages/minimalmodbus.py", line 1170, in _generic_command
payload_from_slave = self._perform_command(functioncode, payload_to_slave)
File "/home/mattias/.local/lib/python3.8/site-packages/minimalmodbus.py", line 1240, in _perform_command
response = self._communicate(request, number_of_bytes_to_read)
File "/home/mattias/.local/lib/python3.8/site-packages/minimalmodbus.py", line 1406, in _communicate
raise NoResponseError("No communication with the instrument (no answer)")
minimalmodbus.NoResponseError: No communication with the instrument (no answer)
Output with the time.sleep(0.1):
MinimalModbus debug mode. Create serial port /dev/ttyUSB0
MinimalModbus debug mode. Serial port /dev/ttyUSB0 already exists
====== SDM120 instrumentA on addres 1 ======
minimalmodbus.Instrument<id=0x7f91feddcdf0, address=1, mode=rtu, close_port_after_each_call=False, precalculate_read_size=True, clear_buffers_before_each_transaction=True, handle_local_echo=False, debug=True, serial=Serial<id=0x7f91fedf50d0, open=True>(port='/dev/ttyUSB0', baudrate=9600, bytesize=8, parity='N', stopbits=1, timeout=1, xonxoff=False, rtscts=False, dsrdtr=False)>
MinimalModbus debug mode. Will write to instrument (expecting 9 bytes back): '\x01\x04\x00\x0c\x00\x02±È' (01 04 00 0C 00 02 B1 C8)
MinimalModbus debug mode. Clearing serial buffers for port /dev/ttyUSB0
MinimalModbus debug mode. No sleep required before write. Time since previous read: 176619.62 ms, minimum silent period: 4.01 ms.
MinimalModbus debug mode. Response from instrument: '\x01\x04\x04\x00\x00\x00\x00û\x84' (01 04 04 00 00 00 00 FB 84) (9 bytes), roundtrip time: 53.3 ms. Timeout for reading: 1000.0 ms.
Active Power in Watts: 0.0
====== SDM120 instrumentB on addres 2 ======
minimalmodbus.Instrument<id=0x7f91fec70940, address=2, mode=rtu, close_port_after_each_call=False, precalculate_read_size=True, clear_buffers_before_each_transaction=True, handle_local_echo=False, debug=True, serial=Serial<id=0x7f91fedf50d0, open=True>(port='/dev/ttyUSB0', baudrate=9600, bytesize=8, parity='N', stopbits=1, timeout=1, xonxoff=False, rtscts=False, dsrdtr=False)>
MinimalModbus debug mode. Will write to instrument (expecting 9 bytes back): '\x02\x04\x00\x0c\x00\x02±û' (02 04 00 0C 00 02 B1 FB)
MinimalModbus debug mode. Clearing serial buffers for port /dev/ttyUSB0
MinimalModbus debug mode. No sleep required before write. Time since previous read: 102.09 ms, minimum silent period: 4.01 ms.
MinimalModbus debug mode. Response from instrument: '\x02\x04\x04\x00\x00\x00\x00È\x84' (02 04 04 00 00 00 00 C8 84) (9 bytes), roundtrip time: 52.8 ms. Timeout for reading: 1000.0 ms.
Active Power in Watts: 0.0
There seems to be nothing wrong with your code or the library you are using (minimalmodbus).
As you probably know, Modbus works in a query-response mode over a half-duplex link. In plain English: you first send a query and the device that query is addressed to answers with the data you asked for.
Both parts of the transaction (queries and responses) travel over the same bus. But the bus is a shared medium and only one device is allowed to take control of the bus (to talk) at any time.
When you have a single master and one or multiple slaves this process works with no issues as long as you guarantee a short silent period after any device writes to the bus. The Modbus specification established this value at 3.5 characters (the time it takes to send 3 and a half characters serially on the bus at the baud rate you are using).
Unfortunately, some manufacturers do not stick to this rule. So some of those devices just take longer than 3.5 characters time to release control of the bus.
This seems to be the case at least with one of your devices. This manual can give you some clues:
My bet is out of your two devices one of them takes significantly less than the other to release the bus, but that's something you will have to confirm with the debug details. It might even be that the device takes longer to release the bus if you query 20 or 40 registers instead of 4 or 8...
What can you do about it? Well, from the device side, not much, it is what it is. On your software you can do many different things. As I said in the comments above you should not feel bad about using time.sleep() considering that's the way minimalmodbus tries to cope with the bus contention problem.
To make your code more robust you can add try: ... except:. This approach is explained in the documentation. You can keep retrying to read within a loop for a number of attempts or add a small delay to the except chunk. Maybe something like this.
The answer of Marcos G. (answered Apr 13 at 9:24) includes some background details. In short:
With some trial and error, one can have a value for time.sleep so that minimalmodbus can cope with the bus contention problem.
You can have more robust code with try: ... except:. It might be a good idea to only try a number of times, to avoid a infinite loop.
I include two scripts which use those approaches to my posted problem. Compared to my original question a for loop and an array for the addresses is used.
The first one is with time.sleep
#!/usr/bin/env python3
import minimalmodbus
import time
addr = 1
instrument = minimalmodbus.Instrument('/dev/ttyUSB0', addr) # port name, slave address (in decimal)
instrument.serial.baudrate = 9600 # Baud
instrument.serial.bytesize = 8
instrument.serial.parity = minimalmodbus.serial.PARITY_NONE
instrument.serial.stopbits = 1
instrument.serial.timeout = 1 # seconds
instrument.mode = minimalmodbus.MODE_RTU # rtu or ascii mode
addresses = [1,2]
registers = [ 0, 6, 12, 18, 24, 30, 70, 72, 74, 76, 78, 84, 86, 88, 90, 92, 94, 258, 264, 342, 344]
names = ["V","I","P","S", "Q","PF","f","IAE","EAE", "IRE", "ERE","TSP","MSP","ISP","MIP","ESP","MEP","ID","MID","TAE", "TRE"]
units = ["V","A","W","VA","var", "","Hz","kWh","kWh","kvarh","kvarh", "W", "W", "W", "W", "W", "W", "A", "A","kWh","kvarh"]
info = [
"(V for Voltage in volt)",
"(I for Current in ampere)",
"(P for Active Power in watt)",
"(S for Apparent power in volt-ampere)",
"(Q for Reactive power in volt-ampere reactive)",
"(PF for Power Factor)",
"(f for Frequency in hertz)",
"(IAE for Import active energy in kilowatt-hour)",
"(EAE for Export active energy in kilowatt-hour)",
"(IRE for Import reactive energy in kilovolt-ampere reactive hours)",
"(ERE for Export reactive energy in kilovolt-ampere reactive hours)",
"(TSP for Total system power demand in watt)",
"(MSP for Maximum total system power demand in watt)",
"(ISP for Import system power demand in watt)",
"(MIP for Maximum import system power demand in watt)",
"(ESP for Export system power demand in watt)",
"(MEP for MaximumExport system power demand in watt)",
"(ID for current demand in ampere)",
"(MID for Maximum current demand in ampere)",
"(TAE for Total active energy in kilowatt-hour)",
"(TRE for Total reactive energy in kilovolt-ampere reactive hours)",
]
for addr in addresses:
instrument.address = addr
print ("=== General info about address", addr, "===")
print (instrument)
print ("=== The registers for address", addr, "===")
for i in range(len(registers)):
value = instrument.read_float(registers[i], 4, 2)
print (str(registers[i]).rjust(3), str(value).rjust(20), units[i].ljust(5), info[i])
time.sleep(0.1) # To avoid minimalmodbus.NoResponseError
print ("")
The second one with try: ... except:
#!/usr/bin/env python3
import minimalmodbus
# This alternative script `sdm120-daisy-alt.py` will try to reread a device an extra 9 times before giving up and continuing with the other addresses in the array.
addr = 1
instrument = minimalmodbus.Instrument('/dev/ttyUSB0', addr) # port name, slave address (in decimal)
instrument.serial.baudrate = 9600 # Baud
instrument.serial.bytesize = 8
instrument.serial.parity = minimalmodbus.serial.PARITY_NONE
instrument.serial.stopbits = 1
instrument.serial.timeout = 1 # seconds
instrument.mode = minimalmodbus.MODE_RTU # rtu or ascii mode
addresses = [1,2]
registers = [ 0, 6, 12, 18, 24, 30, 70, 72, 74, 76, 78, 84, 86, 88, 90, 92, 94, 258, 264, 342, 344]
names = ["V","I","P","S", "Q","PF","f","IAE","EAE", "IRE", "ERE","TSP","MSP","ISP","MIP","ESP","MEP","ID","MID","TAE", "TRE"]
units = ["V","A","W","VA","var", "","Hz","kWh","kWh","kvarh","kvarh", "W", "W", "W", "W", "W", "W", "A", "A","kWh","kvarh"]
info = [
"(V for Voltage in volt)",
"(I for Current in ampere)",
"(P for Active Power in watt)",
"(S for Apparent power in volt-ampere)",
"(Q for Reactive power in volt-ampere reactive)",
"(PF for Power Factor)",
"(f for Frequency in hertz)",
"(IAE for Import active energy in kilowatt-hour)",
"(EAE for Export active energy in kilowatt-hour)",
"(IRE for Import reactive energy in kilovolt-ampere reactive hours)",
"(ERE for Export reactive energy in kilovolt-ampere reactive hours)",
"(TSP for Total system power demand in watt)",
"(MSP for Maximum total system power demand in watt)",
"(ISP for Import system power demand in watt)",
"(MIP for Maximum import system power demand in watt)",
"(ESP for Export system power demand in watt)",
"(MEP for MaximumExport system power demand in watt)",
"(ID for current demand in ampere)",
"(MID for Maximum current demand in ampere)",
"(TAE for Total active energy in kilowatt-hour)",
"(TRE for Total reactive energy in kilovolt-ampere reactive hours)",
]
for addr in addresses:
instrument.address = addr
print ("=== General info about address", addr, "===")
print (instrument)
print ("=== The registers for address", addr, "===")
for attempt in range(10):
try:
for i in range(len(registers)):
value = instrument.read_float(registers[i], 4, 2)
print (str(registers[i]).rjust(3), str(value).rjust(20), units[i].ljust(5), info[i])
except minimalmodbus.NoResponseError:
print("NoResponseError: did't work on attempt ", attempt+1, "I will retry")
else:
print ("Succeeded on attempt", attempt+1)
break
print ("")
I have mrd6 installed on my raspberry pi. It registers with a local interface (tun0) and periodically transmits MLDv2 queries over it.
According to [RFC3810], MLDv2 message types are a subset of ICMPv6 messages, and are identified in IPv6 packets by a preceding Next Header value of 58 (0x3a). They are sent with a link-local IPv6 Source Address, an IPv6 Hop Limit of 1, and an IPv6 Router Alert option [RFC2711] in a Hop-by-Hop Options header.
I can confirm that I'm seeing these packets periodically over tun0:
pi#machine:~ $ sudo tcpdump -i tun0 ip6 -vv -XX
01:22:52.125915 IP6 (flowlabel 0x71df6, hlim 1, next-header Options (0)
payload length: 36)
fe80::69bf:be2d:e087:9921 > ip6-allnodes: HBH (rtalert: 0x0000) (padn)
[icmp6 sum ok] ICMP6, multicast listener query v2 [max resp delay=10000]
[gaddr :: robustness=2 qqi=125]
0x0000: 6007 1df6 0024 0001 fe80 0000 0000 0000 `....$..........
0x0010: 69bf be2d e087 9921 ff02 0000 0000 0000 i..-...!........
0x0020: 0000 0000 0000 0001 3a00 0502 0000 0100 ........:.......
0x0030: 8200 b500 2710 0000 0000 0000 0000 0000 ....'...........
0x0040: 0000 0000 0000 0000 027d 0000 .........}..
I have a socket set up in my application on tun0 as follows, since I expect these to be ICMP packets:
int fd = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); // ICMP
// ... bind this socket to tun0
int interfaceIndex = // tun0 interface Index
int mcastTTL = 10;
int loopBack = 1;
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_MULTICAST_IF,
&interfaceIndex,
sizeof(interfaceIndex))
< 0) {
perror("setsockopt:: IPV6_MULTICAST_IF:: ");
}
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_MULTICAST_LOOP,
&loopBack,
sizeof(loopBack))
< 0) {
perror("setsockopt:: IPV6_MULTICAST_LOOP:: ");
}
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_MULTICAST_HOPS,
&mcastTTL,
sizeof(mcastTTL))
< 0) {
perror("setsockopt:: IPV6_MULTICAST_HOPS:: ");
}
struct ipv6_mreq mreq6 = {{{{0}}}};
MEMCOPY(&mreq6.ipv6mr_multiaddr.s6_addr, sourceAddress, 16);
mreq6.ipv6mr_interface = interfaceIndex;
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_JOIN_GROUP,
&mreq6,
sizeof(mreq6))
< 0) {
perror("setsockopt:: IPV6_JOIN_GROUP:: ");
}
Setting up the socket this way, I can receive ICMP echo requests, replies to my own address, and multicasts sent using the link-local multicast address. However, I don't see any MLDv2 queries.
Here's my receive loop:
uint8_t received[1000] = { 0 };
struct sockaddr_storage peerAddress = { 0 };
socklen_t addressLength = sizeof(peerAddress);
socklen_t addressLength = sizeof(peerAddress);
int receivedLength = recvfrom(sockfd,
received,
sizeof(received),
0,
(struct sockaddr *)&peerAddress,
&addressLength);
if (receivedLength > 0) {
// Never get here for MLDv2 queries.
}
Researching this a bit further, I discovered the IPV6_ROUTER_ALERT socket option, which the man page describes as follows:
IPV6_ROUTER_ALERT
Pass forwarded packets containing a router alert hop-by-hop option to this socket.
Only allowed for SOCK_RAW sockets. The tapped packets are not forwarded by the
kernel, it is the user's responsibility to send them out again. Argument is a
pointer to an integer. A positive integer indicates a router alert option value
to intercept. Packets carrying a router alert option with a value field
containing this integer will be delivered to the socket. A negative integer
disables delivery of packets with router alert options to this socket.
So I figured I was missing this option, and tried setting it as follows. [RFC2710] 0 means Multicast Listener Discovery message.
int routerAlertOption = 0;
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_ROUTER_ALERT,
&routerAlertOption,
sizeof(routerAlertOption))
< 0) {
perror("setsockopt:: IPV6_ROUTER_ALERT:: ");
}
However, this gives me the ENOPROTOOPT error (errno 92). Some more Googling (http://www.atm.tut.fi/list-archive/usagi-users-2005/msg00317.html) led me to the fact that you can't set the IPV6_ROUTER_ALERT option with the IPPROTO_ICMPV6 protocol. It needs a socket defined using the IPPROTO_RAW protocol.
However, defining my socket as:
int fd = socket(AF_INET6, SOCK_RAW, IPPROTO_RAW);
means I'm not able to receive any ICMP packets in my recvfrom anymore.
TL;DR: How do I read MLDv2 queries using an IPv6 socket?
edit (answer):
It appears conventional implementations of Linux will drop MLDv2 packets when passing them to an ICMPV6 socket. Why this is, I'm not sure. (Could be because of the next-header option.)
I followed the accepted answer below and went with an approach of reading raw packets on the tun0 interface. I followed the ping6_ll.c example here: http://www.pdbuchan.com/rawsock/rawsock.html.
It uses a socket with (SOCK_RAW, ETH_P_ALL). You can also set some SOL_PACKET options to filter on specific multicast rules on your interface.
From a quick look at RFCs things aren't looking good. Per RFC4443 (ICMPv6) 2.4:
2.4. Message Processing Rules
Implementations MUST observe the following rules when processing
ICMPv6 messages (from [RFC-1122]):
(b) If an ICMPv6 informational message of unknown type is received,
it MUST be silently discarded.
According to MLDv2 spec it makes use of types 130, 143, perhaps something else (not seeing more diagrams in the RFC), while valid ICMPv6 types are 1, 2, 3, 4, 101, 107, 127, 128, 129, 200, 201, 255.
It looks like the implementation (kernel) must drop MLDv2 packets if they are to be passed to an ICMPv6 socket. Personally I don't see much sense in making MLDv2 look like ICMPv6 if conventional implementations will drop the packet anyways, but I didn't see anything that contradicts this claim.
You can surely go deeper and use a raw socket, especially given that your stack doesn't recognize MLDv2 (perhaps there's a kernel patch to fix that?). But you'll have to parse IP and ICMP headers on your own then.
I am using Log::Syslog::Fast to forward logs to a syslog server. I was testing the script to see how it would react if the syslog server suddenly crashed.
To test I created a file with test messages, started the script & then shutdown the syslog server after 2 messages were received at the syslog server.
The script sent the third message & then dies. The termination is not being caught by eval & 'use warnings 'FATAL' => 'all';' does not help.
Could someone please help me catch the exception & close down the script more gracefully?
What needs to happen here is - After the Command2 is sent, the script should catch the exception & display:
Fail: Command3
Code extract:
$logger = Log::Syslog::Fast->new(LOG_TCP,$server, 514, 13, 6, "test_machine", "Syslog");
$logger->set_pid(0);
foreach $line(<SPOOL>)
{
($machine,$time,$message)=(split '\|',$line);
eval{
$logger->set_sender($machine);
$logger->send($message,$time);
};
if($#)
{
print "\nFail: $message\n";
exit;
}
else
{
print "\nSuccess: $message\n";
}
sleep 5;
}
Input File:
test_machine1|1461201306|Command1
test_machine1|1461201311|Command2
test_machine1|1461203214|Command3
test_machine1|1461203219|Command4
test_machine2|1461204005|Command5
test_machine2|1461204006|Command6
test_machine2|1461204149|Command7
test_machine3|1461204154|Command8
test_machine3|1461206936|Command9
test_machine3|1461206942|Command10
Output:
Success: Command1
Success: Command2
Success: Command3
Strace Output:
read(4, "test_machine1|1461201306|Command"..., 4096) = 341
read(4, "", 4096) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
sendto(3, "<110>Apr 20 21:15:06 test_machin"..., 59, 0, NULL, 0) = 59
write(1, "Success Command1\n\n\n", 19Success Command1
) = 19
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({5, 0}, 0x7ffc707478f0) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
sendto(3, "<110>Apr 20 21:15:11 test_machin"..., 59, 0, NULL, 0) = 59
write(1, "Success Command2\n\n\n", 19Success Command2
) = 19
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({5, 0}, 0x7ffc707478f0) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
sendto(3, "<110>Apr 20 21:46:54 test_machin"..., 59, 0, NULL, 0) = 59
I want the script to fail here when it tries to send the third message but it does not.
write(1, "Success Command3\n\n\n", 19Success Command3
) = 19
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
nanosleep({5, 0}, 0x7ffc707478f0) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
sendto(3, "<110>Apr 20 21:46:59 test_machin"..., 59, 0, NULL, 0) = -1 EPIPE (Broken pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=26037, si_uid=3179} ---
+++ killed by SIGPIPE +++
The script finally dies while trying to send the fourth message. Unfortunately the eval is not catching the exception.
Try adding a line
$SIG{PIPE} = sub {
die "SIGPIPE";
};
before anything is being sent.
You might also want to try "print" instead of die.
You might want to trap SIGPIPE like this:
$SIG{PIPE} = "IGNORE";
From the perlipc:
If you're writing to a pipe, you should also trap SIGPIPE. Otherwise, think of what happens when you start up a pipe to a command that doesn't exist: the open() will in all likelihood succeed (it only reflects the fork()'s success), but then your output will fail--spectacularly. Perl can't know whether the command worked, because your command is actually running in a separate process whose exec() might have failed. Therefore, while readers of bogus commands return just a quick EOF, writers to bogus commands will get hit with a signal, which they'd best be prepared to handle.
Also, have a look at this behavior in a C program when writing to a broken socket.
I want the script to fail here when it tries to send the third message but it does not.
TL;DR You can't do this because of the way the TCP protocol works.
The client and server communicate via a socket. When the client writes to the socket, it's actually writing to a buffer; there's no indication whether the message is actually delivered.
The client can only know that the connection is closed after some data is actually sent to the server, so the first write to the buffer will succeed.
Here's what's happening:
When you shut down the server, it sends a TCP FIN packet. A FIN indicates that one side of the connection is done sending data, but can still receive; it doesn't indicate that the connection is closed.
The client writes your third log message to the socket buffer successfully, so no exception is thrown.
The server sends a TCP RST packet to indicate that it's no longer listening.
Because of the RST, the OS now knows that the server's end of the TCP connection is closed. When the client tries to write to the socket buffer, the process is signaled with SIGPIPE and the write returns EPIPE.
The script finally dies while trying to send the fourth message. Unfortunately the eval is not catching the exception.
You're not handling SIGPIPE so your program dies when it's signaled. Add the following near the top of your script to ignore SIGPIPE:
$SIG{PIPE} = 'IGNORE';
Now you can handle the exception raised by the send method however you like.
Further reading:
Writing on a TCP socket closed by the peer
Writing to a closed, local TCP socket not failing
write on closed socket doesn't generate sigpipe immediatly
Programming UNIX Sockets in C - Frequently Asked Questions section 2.22, "When will my application receive SIGPIPE?"
Are there a constants in scapy for TCP and UDP?
I mean
TCP=6, UDP=17
etc...
a)
looking up the implementation for IP we see that IP.proto is a ByteEnumField("proto", 0, IP_PROTOS),. This means, it takes values from the IP_PROTOS list which just loads your os /etc/protocols/. So you could either parse /etc/protocols yourself or, of scapy is already loaded, access the IP_PROTOS object directly:
>>> IP_PROTOS
</etc/protocols/ pim ip ax_25 esp tcp ah mpls_in_ip rohc ipv6_opts xtp st mobility_header dccp igmp ipv6_route igp ddp etherip wesp xns_idp ipv6_frag vrrp gre ipcomp encap ipv6 iso_tp4 sctp ipencap rsvp hip udp ggp hmp idpr_cmtp hopopt fc skip icmp pup manet isis rdp l2tp ipv6_icmp udplite egp ipip ipv6_nonxt eigrp idrp shim6 rspf ospf vmtp>
>>> IP_PROTOS.tcp
6
>>> IP_PROTOS.udp
17
>>> IP_PROTOS.ip
0
b) An alternative approach would be to read scapys layer binding information directly. This is the information that is added to a layer when you (or scapy core itself) calls bind_layers(lower,upper[,overload_fields]). You can easily read that information as follows:
>>> TCP.overload_fields
{<class 'scapy.layers.inet6.IPv6'>: {'nh': 6}, <class 'scapy.layers.inet.IP'>: {'frag': 0, 'proto': 6}}
Means, in case TCP is a payload to IPv4 (scapy.layers.inet.IP) it will override IP.proto=6.
Here's that same information for UDP
>>> UDP.overload_fields
{<class 'scapy.layers.inet6.IPv6'>: {'nh': 17}, <class 'scapy.layers.inet.IP'>: {'frag': 0, 'proto': 17}}
For reference, here is the bind_layers call for TCP/UDP
TCP and UDP are the initiators of TCP/UDP packets.
For example:
pack = IP(dst="www.google.com") / UDP(dport=80)
pack.show()
Result:
>>> pack = IP(dst="www.google.com") / UDP(dport=80)
>>> pack.show()
###[ IP ]###
version= 4
ihl= None
tos= 0x0
len= None
id= 1
flags=
frag= 0
ttl= 64
proto= udp
chksum= None
src= 'Your local address'
dst= Net('www.google.com')
\options\
###[ UDP ]###
sport= domain
dport= http
len= None
chksum= None
>>>
While going through an open source code-base, I was thinking of an interesting scenario.
Lets say after the successful TCP connection establishment , A TCP client has to send a packet with sequence number = 101. Instead it sends a FIN with sequence number 201. Now that the TCP server thinks FIN as out of order and queues it and waits for a data packet to arrive.
My question is what should be the behavior of a TCP end point according to RFC, if a server receives a data packet with sequence number = 101 and length = 150. Does it overwrite the FIN sent earlier? Or server trims the data packet till the FIN sequence number? Or it dependent on the TCP implementations?
According to some paragraphs in RFC 793
"3. If the connection is in a synchronized state (ESTABLISHED,
FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT),
any unacceptable segment (out of window sequence number or
unacceptable acknowledgment number) must elicit only an empty
acknowledgment segment containing the current send-sequence number
and an acknowledgment indicating the next sequence number expected
to be received, and the connection remains in the same state."
....
"A natural way to think about processing incoming segments is to
imagine that they are first tested for proper sequence number (i.e.,
that their contents lie in the range of the expected "receive window"
in the sequence number space) and then that they are generally queued
and processed in sequence number order.
When a segment overlaps other already received segments we reconstruct
the segment to contain just the new data, and adjust the header fields
to be consistent."
...
My Resposne:
Remember that if this happen, it is because of a bad-behaving TCP at the client. Not for the out-of-order but for the wrong sequence in the segment with the FIN flag. Or maybe an attack.
When TCP, in the server side, receives the segment with SEQ=201, it will store this segment for a limited time and will send back an ACK for 101 because it is waiting for that SEQ number.
Then when the segment with SEQ=101 arrives, TCP in the receiving side, after receiving the segment with SEQ=101 will have a new receive window. From the first arrived segment with SEQ=201, it should get only the data beyond Byte 251 (in my test instead of doing this it removed the overlapping Bytes from the segment with SEQ=101 - This might be implementation dependent), if any, and accept the FIN. The receiving TCP will send back an ACK. When the socket is closed in the server side, the receiving TCP will send back a [FIN, ACK] segment.
To test it I have a client that does exactly what you described (this is done with raw sockets in user space, it's not possible to simulate it with TCP sockets. The server is a simple nodejs server), a FIN segment is sent and 15 seconds later the previous segment is sent. The server reads the data received and after 10 seconds it closes the socket.
Here is the tcpdump, you can see TCP server side responses:
[rodolk#localhost ~]$ sudo tcpdump -i p2p1 -vv tcp
tcpdump: listening on p2p1, link-type EN10MB (Ethernet), capture size 65535 bytes
19:33:03.648216 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
192.168.56.101.16345 > 192.168.56.1.webcache: Flags [S], cksum 0x5f49 (correct), seq 523645, win 500, length 0
19:33:03.649826 IP (tos 0x0, ttl 128, id 26590, offset 0, flags [DF], proto TCP (6), length 44)
192.168.56.1.webcache > 192.168.56.101.16345: Flags [S.], cksum 0x1ac8 (correct), seq 1576251572, ack 523646, win 8192, options [mss 1460], length 0
19:33:03.651208 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40)
192.168.56.101.16345 > 192.168.56.1.webcache: Flags [.], cksum 0x5091 (correct), seq 1, ack 1, win 500, length 0
19:33:03.651567 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 74)
192.168.56.101.16345 > 192.168.56.1.webcache: Flags [F.], cksum 0x8121 (correct), seq 122:156, ack 1, win 500, length 34
19:33:03.651891 IP (tos 0x0, ttl 128, id 26591, offset 0, flags [DF], proto TCP (6), length 40)
192.168.56.1.webcache > 192.168.56.101.16345: Flags [.], cksum 0x5314 (correct), seq 1, ack 1, win 65392, length 0
19:33:18.652083 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 171)
192.168.56.101.16345 > 192.168.56.1.webcache: Flags [P.], cksum 0xf973 (correct), seq 1:132, ack 1, win 500, length 131
19:33:18.652834 IP (tos 0x0, ttl 128, id 26593, offset 0, flags [DF], proto TCP (6), length 40)
192.168.56.1.webcache > 192.168.56.101.16345: Flags [.], cksum 0x5313 (correct), seq 1, ack 157, win 65237, length 0
19:33:28.661041 IP (tos 0x0, ttl 128, id 26594, offset 0, flags [DF], proto TCP (6), length 40)
192.168.56.1.webcache > 192.168.56.101.16345: Flags [F.], cksum 0x5312 (correct), seq 1, ack 157, win 65237, length 0
19:33:28.961756 IP (tos 0x0, ttl 128, id 26595, offset 0, flags [DF], proto TCP (6), length 40)
192.168.56.1.webcache > 192.168.56.101.16345: Flags [F.], cksum 0x5312 (correct), seq 1, ack 157, win 65237, length 0