the BPF filter dit not work with vlan packets - bpf

I captured some packets with pcapplusplus on our Ubuntu server, and wrote to .pcap files, then I read the .pcap files, it just worked fine; but when I set the filter with BPF Syntax,it could not read from the .pcap files, the filter is just a tcp string, and it worked well with the example input.pcap, but not work with my pcap files,
pcpp::IFileReaderDevice* reader = pcpp::IFileReaderDevice::getReader("input.pcap");
// verify that a reader interface was indeed created
if (reader == NULL)
{
printf("Cannot determine reader for file type\n");
exit(1);
}
// open the reader for reading
if (!reader->open())
{
printf("Cannot open input.pcap for reading\n");
exit(1);
}
// create a pcap file writer. Specify file name and link type of all packets that
// will be written to it
pcpp::PcapFileWriterDevice pcapWriter("output.pcap", pcpp::LINKTYPE_ETHERNET);
// try to open the file for writing
if (!pcapWriter.open())
{
printf("Cannot open output.pcap for writing\n");
exit(1);
}
// create a pcap-ng file writer. Specify file name. Link type is not necessary because
// pcap-ng files can store multiple link types in the same file
pcpp::PcapNgFileWriterDevice pcapNgWriter("output.pcapng");
// try to open the file for writing
if (!pcapNgWriter.open())
{
printf("Cannot open output.pcapng for writing\n");
exit(1);
}
// set a BPF filter for the reader - only packets that match the filter will be read
if (!reader->setFilter("tcp"))
{
printf("Cannot set filter for file reader\n");
exit(1);
}
// the packet container
pcpp::RawPacket rawPacket;
// a while loop that will continue as long as there are packets in the input file
// matching the BPF filter
while (reader->getNextPacket(rawPacket))
{
// write each packet to both writers
printf("matched ...\n");
pcapWriter.writePacket(rawPacket);
pcapNgWriter.writePacket(rawPacket);
}
Here are some packets:[enter image description here][1]
[1]: https://i.stack.imgur.com/phYA0.png , Anyone can help ?

TL;DR. You need to use the filter vlan and tcp to catch TCP packets with a VLAN tag.
Explanations
We can look at what BPF filter is generated when using only tcp:
$ tcpdump -d -i eth0 tcp
(000) ldh [12]
(001) jeq #0x86dd jt 2 jf 7
(002) ldb [20]
(003) jeq #0x6 jt 10 jf 4
(004) jeq #0x2c jt 5 jf 11
(005) ldb [54]
(006) jeq #0x6 jt 10 jf 11
(007) jeq #0x800 jt 8 jf 11
(008) ldb [23]
(009) jeq #0x6 jt 10 jf 11
(010) ret #262144
(011) ret #0
We can see 2 bytes are first loaded from offset 12 in the packet. That corresponds to the Ethertype in the Ethernet header. It is then used to check if we are parsing an IPv6 (jeq #0x86dd) or IPv4 (jeq #0x800) packet.
However, when there is a VLAN tag, the Ethertype field is shifted by 4 bytes (the length of the VLAN tag field). For packets with a VLAN tag, we should therefore be reading the Ethertype at offset 16.
Using filter vlan and tcp implements this change, by first checking that there is a VLAN tag and then taking it into account when reading the Ethertype. Therefore, to filter TCP packets regardless of whether they have a VLAN tag, you'll need tcp or (vlan and tcp).

#pchaigno is correct; you need to do vlan and tcp or, to catch both VLAN-encapsulated and non-VLAN-encapsulated TCP packets, tcp or (vlan and tcp).

Related

What is the purpose of the `withFdSocket sock setCloseOnExecIfNeeded` call in Network.Socket example?

In the Network.Socket package, there is an echo server example. In the echo server, a call to withFdSocket sock setCloseOnExecIfNeeded is made just before actually binding to the socket. The relevant function is below:
open :: AddrInfo -> IO Socket
open addr = E.bracketOnError (openSocket addr) close $ \sock -> do
setSocketOption sock ReuseAddr 1
withFdSocket sock setCloseOnExecIfNeeded
bind sock $ addrAddress addr
listen sock 1024
return sock
The full example can be found on the Network.Socket package documentation.
What is the purpose of this call? It's my understanding that withFdSocket creates a file descriptor associated with the socket, and setCloseOnExecIfNeeded sets the CLOEXEC flag on the descriptor. However this descriptor is immediately 'discarded' and nothing is done with it.
Am I confused that the file descriptor always exists and withFdSocket just provides this existing descriptor, which we have to update with the flag in order for the socket to close when the program exits?
withFdSocket does not create an Fd; it just reads the one already stored inside the socket data structure.

how to capture packets sent by XDP user space program

Say I have XDP kernel and user programs. Kernel code examines ingress packets and "forwards" them to XDP user space program. XDP user space program then does some processing and sends packets out (could be same NIC or other one).
For controlling reasons we want to examine the outgoing packets sent by XDP user space code. Our first idea was using tc and insert BPF code to egress part, clearly didn't work.
Any idea?
EDIT1: egress program:
SEC("tx")
int track_tx(struct __sk_buff *skb)
{
__u64 t2 = bpf_ktime_get_ns();
void *data_end = (void *)(long)skb->data_end;
void *data = (void *)(long)skb->data;
bpf_custom_printk("\n__________send length %d\n", data_end-data);
return TC_ACT_OK;
}
run:
sudo tc filter add dev ens13 egress bpf da obj layercoop.o sec tx
EDIT2: Code
USer: https://github.com/torvalds/linux/blob/master/samples/bpf/xdpsock_user.c
Kernel (not interesting): https://github.com/torvalds/linux/blob/master/samples/bpf/xdpsock_kern.c
I want to forward incoming packets, so run it with make -j 4; sudo ./xdpsock -i ens13 -l

Receiving multicast traffic using GNAT.Sockets

I am experimenting with IP multicasting in Ada, but doesn't seem to receive any traffic sent to the multicast group. Somehow, it seems like I cannot get the application to fetch the incoming packets.
I can verify (using Wireshark) that a multicast join is sent from my computer, and also I can verify that there is data being sent to the multicast group.
I can verify that OS has the multicast join registered by the netsh command:
netsh interfaces ip show joins
My group is listed with a reference of 1 if I run my program and 0 if it is not.
The following procedure shows my listener, and I invoke it using Mcast_IP => "239.255.128.128" and Mcast_Port => "8807":
with GNAT.Sockets;
with Ada.Streams;
with Ada.Text_IO;
procedure Receive_Multicast (Mcast_IP : in String;
Mcast_Port : in String)
is
package GS renames GNAT.Sockets;
package AS renames Ada.Streams;
package Tio renames Ada.Text_IO;
use GS;
use type Ada.Streams.Stream_Element_Offset;
Socket : GS.Socket_Type;
Address : GS.Sock_Addr_Type;
Data : AS.Stream_Element_Array (1 .. 2**16);
Offset : AS.Stream_Element_Offset;
Sender : GS.Sock_Addr_Type;
begin
Address.Addr := Any_Inet_Addr;
Address.Port := Port_Type'Value (Mcast_Port);
Create_Socket (Socket => Socket,
Family => Family_Inet,
Mode => Socket_Datagram);
Bind_Socket (Socket, Address);
-- Set socket options
Set_Socket_Option (Socket,
Socket_Level,
(Reuse_Address, True));
Set_Socket_Option
(Socket,
IP_Protocol_For_IP_Level,
(Multicast_TTL, 1));
Set_Socket_Option
(Socket,
IP_Protocol_For_IP_Level,
(Multicast_Loop, True));
Set_Socket_Option
(Socket,
IP_Protocol_For_IP_Level,
(Add_Membership, Inet_Addr (Mcast_IP), Any_Inet_Addr));
Tio.Put_Line ("Listening for MULTICASTS on port " & Address.Port'Img);
-- Receive the packet from the socket.
loop
Tio.Put_Line ("Waiting for incoming packets...");
Receive_Socket (Socket => Socket,
Item => Data,
Last => Offset,
From => Sender);
Tio.Put_Line ("Received " & Offset'Img & " bytes.");
end loop;
end Receive_Multicast;
The procedure works its way down to the Receive_Socket call (which is a procedure in GNAT.Sockets package). However, even if I can confirm multicast traffic using Wireshark, the call to Receive_Socket keeps blocking.
UPDATE/SOLUTION:
The code above does actually work, although I had to completely uninstall Kaspersky which apparently did prevent multicasts sent from my own machine to be received (i.e. loopback). The accepted answer does also work flawlessly.
Based on the example in GNAT.Sockets, the code below should work. I've removed some options as they are not relevant for receiving.
receive_multicast.ads
procedure Receive_Multicast
(IP_Address : String;
Port : String);
receive_multicast.adb
with Ada.Text_IO;
with Ada.Streams;
with GNAT.Sockets;
procedure Receive_Multicast
(IP_Address : String;
Port : String)
is
use GNAT.Sockets;
Address : Sock_Addr_Type;
Socket : Socket_Type;
begin
Create_Socket (Socket, Family_Inet, Socket_Datagram);
Set_Socket_Option
(Socket => Socket,
Level => Socket_Level,
Option => (Reuse_Address, True));
Address.Addr := Any_Inet_Addr;
Address.Port := Port_Type'Value (Port);
Bind_Socket (Socket, Address);
-- Join a multicast group
-- Portability note: On Windows, this option may be set only
-- on a bound socket.
Set_Socket_Option
(Socket => Socket,
Level => IP_Protocol_For_IP_Level,
Option => (Add_Membership, Inet_Addr (IP_Address), Any_Inet_Addr));
-- Receive the packet from the socket.
declare
use Ada.Text_IO;
use Ada.Streams;
Data : Stream_Element_Array (1 .. 2**16);
Offset : Stream_Element_Offset;
Sender : Sock_Addr_Type;
begin
Put_Line ("Waiting for incoming packets...");
Receive_Socket
(Socket => Socket,
Item => Data,
Last => Offset,
From => Sender);
Put_Line ("Received " & Offset'Image & " bytes.");
end;
end Receive_Multicast;
main.adb
with Receive_Multicast;
procedure Main is
begin
Receive_Multicast
(IP_Address => "239.255.128.128",
Port => "8807");
end Main;
I couldn't test the code extensively, but when I open Windows PowerShell ISE, load and run the script Send-UdpDatagram.ps1 (see this GitHub Gist) and then execute:
PS C:\> Send-UdpDatagram -EndPoint "239.255.128.128" -Port 8807 -Message "testing"
Then the Ada program responds with:
Waiting for incoming packets...
Received 7 bytes.
[2019-09-29 10:55:58] process terminated successfully, elapsed time: 07.60s
Update
I also tested the example code with a Raspberry Pi running Raspbian GNU/Linux 10 (buster):
Installed APT packages gnat and gprbuild on the Raspberry Pi.
Copied the code to the Raspberry Pi.
Compiled it with GNAT FSF (gprbuild -p <proj_name>.gpr).
Started four instances of the program, each in a separate terminal.
Emitted a packet from a Windows 10 host using the PowerShell function as before.
The result was the same: the packet was received by all four program instances on the Raspberry Pi. While the programs were waiting for the packet, I could see the memberships (see also this post on SO):
pi#raspberrypi:~ $ netstat -g
IPv6/IPv4 Group Memberships
Interface RefCnt Group
--------------- ------ ---------------------
[...]
eth0 4 239.255.128.128
[...]
pi#raspberrypi:~ $ netstat -anu | sort -nk4
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
[...]
udp 0 0 0.0.0.0:8807 0.0.0.0:*
udp 0 0 0.0.0.0:8807 0.0.0.0:*
udp 0 0 0.0.0.0:8807 0.0.0.0:*
udp 0 0 0.0.0.0:8807 0.0.0.0:*
[...]

Can't use CAIDA Dataset with tcpreplay to simulate network traffic in network

I have a server and I need to simulate real network traffic. I've been asked to do this using a CAIDA Dataset. I have downloaded the public Dataset which can be found here:
CAIDA Public Dataset
I also need to rewrite the source ip address in the .pcap file to be the one of the server. I tried doing it the way it's described at the end of this page: tcprewrite wiki
I run:
tcprewrite --infile=oc48-mfn.dirA.20020814-160000.UTC.anon.pcap --outfile=oc48-mfn.dirA.20020814-160000.UTC.anon_rewrite.pcap --dstipmap=0.0.0.0/0:10.101.30.60 --enet-dmac=00:0c:29:00:b1:bd
And I get:
Warning: oc48-mfn.dirA.20020814-160000.UTC.anon.pcap was captured using a snaplen of 48 bytes. This may mean you have truncated packets.
Fatal Error: From ./plugins/dlt_hdlc/hdlc.c:dlt_hdlc_encode() line 255:
Non-HDLC packet requires --hdlc-address
So after some tries like this I finally run these to get an error free tcprewrite:
tcpprep --auto=bridge --pcap=oc48-mfn.dirA.20020814-160000.UTC.anon.pcap --cachefile=cache1.cache
Which gives:
Warning: oc48-mfn.dirA.20020814-160000.UTC.anon.pcap was captured using a snaplen of 48 bytes. This may mean you have truncated packets.
Warning: oc48-mfn.dirA.20020814-160000.UTC.anon.pcap was captured using a snaplen of 48 bytes. This may mean you have truncated packets.
And then I run:
tcprewrite --infile=oc48-mfn.dirA.20020814-160000.UTC.anon.pcap --outfile=oc48-mfn.dirA.20020814-160000.UTC.anon_rewrite.pcap --dstipmap=0.0.0.0/0:10.101.30.60 --enet-dmac=00:0c:29:00:b1:bd --cachefile=cache1.cache --hdlc-control=0 --hdlc-address=0xBF
And I get:
Warning: oc48-mfn.dirA.20020814-160000.UTC.anon.pcap was captured using a snaplen of 48 bytes. This may mean you have truncated packets.
So it seems like a success, except the warning that shows up in every command.
I open the new .pcap file with tcpdump to check that the destination IP addresses have changed to the one of the server and it has been done.
So then I run tcpreplay:
tcpreplay -i ens160 --loop 5 --unique-ip oc48-mfn.dirA.20020814-160000.UTC.anon.pcap
And I run tcpdump on the server to see the traffic from the .pcap file, but the traffic looks like this:
13:30:50.194780 05:8c:55:6f:40:00 (oui Unknown) > 0f:00:08:00:45:00 (oui
Unknown), ethertype Unknown (0x3406), length 60:
0x0000: ed11 f484 7785 f477 0d79 0050 0487 007c ....w..w.y.P...|
0x0010: e7d5 d203 c32b 5010 27f7 aa51 0000 4854 .....+P.'..Q..HT
0x0020: 5450 0000 0000 0000 0000 0000 0000 TP............
I have tried the smallFlow.pcap from the sample captures of tcpreplay: Sample Captures
and it worked just fine.
So any suggestions on how to properly use the CAIDA .pcap files?
Your stated goal is "I need to simulate real network traffic", but you're using pcaps where "the payload has been removed from all packets" (per the CAIDA web page you linked to).
These two statements are in conflict with each other. All your packets are literally no larger then 48bytes which is merely enough for the TCP/IP header (and then even so, may not be sufficient in all cases). This is what the warning is telling you. You can't put the data back.
You'll need to find a new source of pcaps.

Can't read data from OBD2 serial port connection on Raspberry Pi

I'm currently working on a raspberry pi project for school where I read data from an OBD2 to usb censor in my car.
When I'm using Screen to connect to the serial port everything works fine, but when I try to do it in python the serial.readline() returns an empty string.
Does anybody know how I can retrieve data from the serial port in python?
I've tried about every option available.
import serial
ser = 0
#Function to Initialize the Serial Port
def init_serial():
global ser
ser = serial.Serial()
ser.baudrate = 38400
ser.port = '/dev/ttyUSB0'
ser.timeout = 1
ser.open() #Opens SerialPort
# print port open or closed
if ser.isOpen():
print 'Open: ' + ser.portstr
#Function Ends Here
init_serial()
temp = raw_input('Type what you want to send, hit enter:\r\n')
ser.write(temp) #Writes to the SerialPort
while 1:
bytes = ser.readline() #Read from Serial Port
print bytes #Print What is Read from Port
You may not be sending valid data to get a response. I believe the ODB2 interface uses the AT command set. Sending AT\n may be a good starting point.
I'm using a Bluetooth ODB2 interface and found that the serial baudrate was fixed. Using any other baudrate failed to correctly get data.
I recommend testing from putty or other terminal that supports serial ports, until you get the device to respond correctly. Then use valid settings to troubleshoot your code.
You're not sending a \r\n with your command, ELM327 requires a new line character at the end of a command.