Python simple UDP server does not receive message with IP options - sockets

I have a simple udp server/client setup where I send a message from the client and print it on the server. This works well for a regular IP packet but the message is not received when I add an IP options header to the packet, even though I can sniff the packet using scapy.
Here's the packet without IP options
###[ Ethernet ]###
dst = 00:04:00:00:04:01
src = 00:aa:00:02:00:04
type = 0x800
###[ IP ]###
version = 4L
ihl = 5L
tos = 0x0
len = 47
id = 1
flags =
frag = 0L
ttl = 61
proto = udp
chksum = 0x62f4
src = 10.0.2.101
dst = 10.0.4.101
\options \
###[ UDP ]###
sport = 10001
dport = 3478
len = 27
chksum = 0x2bd1
###[ Raw ]###
load = 'message from a game'
And here's the packet with IP options header:
###[ Ethernet ]###
dst = 00:04:00:00:04:01
src = 00:aa:00:02:00:04
type = 0x800
###[ IP ]###
version = 4L
ihl = 8L
tos = 0x0
len = 59
id = 1
flags =
frag = 0L
ttl = 61
proto = udp
chksum = 0x5fe8
src = 10.0.2.101
dst = 10.0.4.101
\options \
|###[ IPOption ]###
| copy_flag = 1L
| optclass = control
| option = 31L
| length = 12
| value = '\x00\x01\x00\x00RTGAME'
###[ UDP ]###
sport = 10001
dport = 3478
len = 27
chksum = 0x2bd1
###[ Raw ]###
load = 'message from a game'
And here's the UDP server:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(('', args.port))
while True:
try:
data, addr = sock.recvfrom(1024)
print("received: %s" % data)
except KeyboardInterrupt:
sock.close()
break
I've been stuck on this for a few days and would love if someone could figure it out.
Thanks

have just been playing and the following works as a self-contained/minimal working example for me with Python 3.7.1 under both OSX and Linux
generating a valid set of IP Options:
from scapy.all import IPOption, raw
ipopts = raw(IPOption(
copy_flag=1, optclass='control', option=31,
value='\x00\x01\x00\x00RTGAME'))
(if you don't have Scapy, the above should generate: b'\x9f\x0c\x00\x01\x00\x00RTGAME')
client code:
import socket
from time import sleep
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.connect(('127.0.0.1', 3478))
s.setsockopt(socket.IPPROTO_IP, socket.IP_OPTIONS, ipopts)
while True:
s.send(b'message from a game')
sleep(1)
server code:
import socket
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:
s.bind(('', 3478))
s.setsockopt(socket.IPPROTO_IP, socket.IP_RECVOPTS, 1)
while True:
print(*s.recvmsg(4096, 1024))
this should result in the "server" displaying lines like:
b'message from a game\n' [(0, 6, b'\x9f\x0c\x00\x01\x00\x00RTGAME')] 0 ('127.0.0.1', 46047)
furthermore, I can watch packets move over the network by running:
sudo tcpdump -i lo0 -vvv -n 'udp and port 3478'
at the command line, or this in Scapy:
sniff(iface='lo0', filter='udp and port 3478', prn=lambda x: x.show())
for some reason I don't actually receive the ancillary data containing the IP Options under OSX, but the data shows up in the packet sniffers.

The problem was due to an incorrect IPv4 checksum. I failed to mention in the question that I'm running this in a mininet environment with custom switches. The IP options get added in transit by a switch, but the checksum wasn't updated. Once I fixed that, the packet made it to the server.
Thanks for the help and pointers everyone!

Related

Socket recvfrom function hanging, not recognizing ICMP packet

So I'm trying to do a mock Traceroute function where a UDP packet is sent to an IP address. I'm trying to design the program in such a way where a packet is sent each time the packet makes it to a router. I am trying to do this by making a very short TTL. However, the recvfrom function is stalling.
Here's the code:
host_addr = gethostbyname(host)
port = 33434
max_hops = 30
ttl = 1
while True:
recv_socket = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP)
send_socket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
send_socket.setsockopt(0, 4, ttl)
recv_socket.bind(("", port))
send_socket.sendto("waddup".encode(), (host_addr, port))
cur_addr = None
cur_name = None
path_end = False
cur_bytes = None
attempts_left = 3
timeReceived = 0
pingStartTime = 0
while not path_end and attempts_left > 0:
try:
pingStartTime = time.time()
cur_bytes, cur_addr = recv_socket.recvfrom(1024)
timeReceived = time.time()
path_end = True
cur_addr = cur_addr[0]
try:
cur_name = gethostbyaddr(cur_addr)[0]
except error:
cur_name = cur_addr
except error:
attempts_left -= 1
send_socket.close()
recv_socket.close()
if not path_end:
pass
if cur_addr is not None:
cur_host = "%s (%s) " % (cur_name, cur_addr)
else:
cur_host = ""
print("%d: %.0f ms " %
(
ttl,
(timeReceived - pingStartTime) * 1000,
) + cur_host
)
ttl += 1
if cur_addr == host_addr or ttl > max_hops:
break
I have set up the receiving socket for an ICMP packet as far as I can tell but it just hangs on recvfrom. I've allowed all ICMP connections on Windows Defender and when I run Wireshark, an appropriate ICMP packet is sent to my router.
The packet with the TTL expired message is the one I want to receive
I'm new to networking but all the code I've seen online has this exact setup. Would anyone be able to tell me why it's stalling and what I can do to fix it? I want the program to read the ICMP packet with the TTL expiration message.

read data from PCAP and print details when conditions met

I have assignment to Read packets from a file and output the details of those packets having.
Do not fragment(DF) flag set for IP header and SYN and ACK flags
set (together) for TCP header (all the three flags should be set). For
packets qualifying the above condition print the following:
Packet number
Source IP address and Source port number
Destination IP address and Destination port number
I have done packet capture but not able to print values matching condition of all 3 flag set from that PCAP file
print("Starting ")
for packet in PcapReader(filename):
if packet[IP].flags == 'DF' and packet[TCP].flags == 'S' and packet[TCP].flags == 'A':
print("Source IP address = {} , source port number = {} , destination IP addr = {} , destination port number = {} ".format(packet[IP].src,packet[TCP].sport,packet[IP].dst,packet[TCP].dport))
else:
print("Finishing. ")

How do I access the data from rp_remote_acquire?

I currently have a python program that (very slowly) recieves data from a Red Pitaya board by recursively calling:
redpitaya_scpi.scpi(192.169.1.100).rx_txt()
I would like to use rp_remote_acquire to achieve a higher throughput with a ring buffer.
I am able to execute ./rp_remote_acquire on both the Red Pitaya (server) and a linux machine (client) thanks to stackoverflow.
I get some unique content in /tmp/out every time I execute the following commands on the Red Pitaya (which suggests that the program on the server has access to the data from its hardware).
rm /tmp/out
./rp_remote_acquire -m 3
cat /tmp/out
In order to transfer data from the Red Pitaya (client) to the linux machine (server), I launch ./rp_remote_acquire with the following parameters:
Server (192.169.1.100):
./rp_remote_acquire -m 2 -a 192.169.1.102 -p 14000
Client (192.169.1.102):
./rp_remote_acquire -m 1 -a 192.169.1.100 -p 14000
Where:
-m --mode <(1|client)|(2|server)|(3|file)>
operating mode (default client)
-a --address <ip_address>
target address in client mode (default empty)
-p --port <port_num>
port number in client and server mode (default 14000)
Both machines are able ping eachother and the machines are able to establish a connection (ie. int connection_start(option_fields_t *options, struct handles *handles) at transfer.c:251 returns zero).
The client ends up executing the following code snippet from transfer.c
533 while (!size || transferred < size) {
(gdb) n
534 if (pos == buf_size)
(gdb) n
539 if (pos + CHUNK <= curr) {
(gdb) n
552 memcpy(buf, mapped_base + pos, len);
(gdb) n
554 if (handles->sock >= 0) {
(gdb) n
552 memcpy(buf, mapped_base + pos, len);
(gdb) n
554 if (handles->sock >= 0) {
(gdb) n
555 if (send_buffer(handles->sock, options, buf, len) < 0) {
(gdb) n
569 pos += len;
(gdb) n
533 while (!size || transferred < size) {
It seems like the client is effectively just doing the following (note size = 0 by default):
533 while (!size || transferred < size) {
552 memcpy(buf, mapped_base + pos, len);
552 memcpy(buf, mapped_base + pos, len);
569 pos += len;
}
This behaviour seems to be the intention of the programmer because the client stops as soon as the server is halted:
554 if (handles->sock >= 0) {
(gdb)
556 if (!interrupted)
the program doesn't get stuck in this loop when I change size such that it is not equal to zero (=> smaller packets?).
I would like to be able to access the data that is (hopefully) being sent from the Red Pitaya (server) to the linux machine (client) and somehow make this data available to a python program on the client machine.
My question(s):
What is going on here and how can I access the data?
Do I need to synchronously run a second program on the client that somehow reads the data that rp_remote_acquire is copying into the clients memory?
The solution is surprisingly simple.
When it is running properly in server mode, rp_remote_acquire writes the data to a socket:
/*
* transfers samples to socket via read() call on rpad_scope
*/
static u_int64_t transfer_readwrite(struct scope_parameter *param,
option_fields_t *options, struct handles *handles)
In client mode it reads the data from the socket and does something with it.
Since we are working with sockets here, we don't need to care what rp_remote_acquire does in client mode. We can simply create our own socket with a python script and recieve the data in the script (which is where I want to have the data).
This is an example from #otobrzo:
import socket
import numpy as np
import matplotlib.pyplot as plt
client = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ip=socket.gethostbyname("XX.XX.XX.XX") # IP of redpitaya in server mode:
# run cat ddrdump.bit > /dev/xdevcfg
#compiled and run on redpitay ./rp_remote_acquire -m 2 -k 0 -c 0 -d 0
port=14000 # default port for TCP
address=(ip,port)
client.connect(address)
Nl = 10000
#while True:
for x in range(0, Nl):
# print("test1")
bytes_data = client.recv(1024) # set the amount data transferred
if x == 0:
data = np.frombuffer(bytes_data, dtype=np.int16) # from 16bit data to int16
data = np.array(data, dtype=float)
data_all = data
else:
data = np.frombuffer(bytes_data, dtype=np.int16) # from 16bit data to int16
data = np.array(data, dtype=float)
data_all= np.hstack((data_all,data))
#%%
FPS = 125e6
time = np.arange(0,np.size(data_all))/FPS
plt.plot(time,data_all)

Are there a constants in scapy for TCP and UDP?

Are there a constants in scapy for TCP and UDP?
I mean
TCP=6, UDP=17
etc...
a)
looking up the implementation for IP we see that IP.proto is a ByteEnumField("proto", 0, IP_PROTOS),. This means, it takes values from the IP_PROTOS list which just loads your os /etc/protocols/. So you could either parse /etc/protocols yourself or, of scapy is already loaded, access the IP_PROTOS object directly:
>>> IP_PROTOS
</etc/protocols/ pim ip ax_25 esp tcp ah mpls_in_ip rohc ipv6_opts xtp st mobility_header dccp igmp ipv6_route igp ddp etherip wesp xns_idp ipv6_frag vrrp gre ipcomp encap ipv6 iso_tp4 sctp ipencap rsvp hip udp ggp hmp idpr_cmtp hopopt fc skip icmp pup manet isis rdp l2tp ipv6_icmp udplite egp ipip ipv6_nonxt eigrp idrp shim6 rspf ospf vmtp>
>>> IP_PROTOS.tcp
6
>>> IP_PROTOS.udp
17
>>> IP_PROTOS.ip
0
b) An alternative approach would be to read scapys layer binding information directly. This is the information that is added to a layer when you (or scapy core itself) calls bind_layers(lower,upper[,overload_fields]). You can easily read that information as follows:
>>> TCP.overload_fields
{<class 'scapy.layers.inet6.IPv6'>: {'nh': 6}, <class 'scapy.layers.inet.IP'>: {'frag': 0, 'proto': 6}}
Means, in case TCP is a payload to IPv4 (scapy.layers.inet.IP) it will override IP.proto=6.
Here's that same information for UDP
>>> UDP.overload_fields
{<class 'scapy.layers.inet6.IPv6'>: {'nh': 17}, <class 'scapy.layers.inet.IP'>: {'frag': 0, 'proto': 17}}
For reference, here is the bind_layers call for TCP/UDP
TCP and UDP are the initiators of TCP/UDP packets.
For example:
pack = IP(dst="www.google.com") / UDP(dport=80)
pack.show()
Result:
>>> pack = IP(dst="www.google.com") / UDP(dport=80)
>>> pack.show()
###[ IP ]###
version= 4
ihl= None
tos= 0x0
len= None
id= 1
flags=
frag= 0
ttl= 64
proto= udp
chksum= None
src= 'Your local address'
dst= Net('www.google.com')
\options\
###[ UDP ]###
sport= domain
dport= http
len= None
chksum= None
>>>

packetsocket opened on loopback device receives all the packets twice. How to filter these duplicate entries?

when i open a packetsocket on a loopback interface (lo) and listen all the packets are seen twice. why is it so?
But a capture on the interface using tcpdump correctly ignores the duplicate entries. see the 'packets received by filter' (which contains the duplicate packets) and 'packets captured'. How is this filtering done
tcpdump -i lo -s 0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
11:00:08.439542 IP 12.0.0.3 > localhost.localdomain: icmp 64: echo request seq 1
11:00:08.439559 IP localhost.localdomain > 12.0.0.3: icmp 64: echo reply seq 1
11:00:09.439866 IP 12.0.0.3 > localhost.localdomain: icmp 64: echo request seq 2
11:00:09.439884 IP localhost.localdomain > 12.0.0.3: icmp 64: echo reply seq 2
11:00:10.439389 IP 12.0.0.3 > localhost.localdomain: icmp 64: echo request seq 3
11:00:10.439410 IP localhost.localdomain > 12.0.0.3: icmp 64: echo reply seq 3
6 packets captured
12 packets received by filter
0 packets dropped by kernel
My code:
int main()
{
int sockFd;
if ( (sockFd=socket(PF_PACKET, SOCK_DGRAM, 0))<0 ) {
perror("socket()");
return -1;
}
/* bind the packet socket */
struct sockaddr_ll addr;
struct ifreq ifr;
strncpy (ifr.ifr_name, "lo", sizeof(ifr.ifr_name));
if(ioctl(sockFd, SIOCGIFINDEX, &ifr) == -1)
{
perror("iotcl");
return -1;
}
memset(&addr, 0, sizeof(addr));
addr.sll_family=AF_PACKET;
addr.sll_protocol=htons(ETH_P_ALL);
addr.sll_ifindex=ifr.ifr_ifindex;
if ( bind(sockFd, (struct sockaddr *)&addr, sizeof(addr)) ) {
perror("bind()");
return -1;
}
char buffer[MAX_BUFFER+1];
int tmpVal = 1;
while(tmpVal > 0)
{
tmpVal = recv (sockFd, buffer, MAX_BUFFER, 0);
cout<<"Received Pkt with Bytes "<<tmpVal <<endl;
}
}
Figured out the problem.
from libcaps code:
* - The loopback device gives every packet twice; on 2.2[.x] kernels,
* if we use PF_PACKET, we can filter out the transmitted version
* of the packet by using data in the "sockaddr_ll" returned by
* "recvfrom()", but, on 2.0[.x] kernels, we have to use
* PF_INET/SOCK_PACKET, which means "recvfrom()" supplies a
* "sockaddr_pkt" which doesn't give us enough information to let
* us do that.
the listening entity needs to filter the duplicate packet using the if_index got from recvfrom api.