I want to stop packet capture while sniffing continuously once a condition is met - packet

Problem
I have written a script that sniffs packet from a host, however, I am sniffing the packets in continuous mode and would like to stop sniffing on a timeout. I have written the following code to stop packet sniffing, but it doesn't seem to stop when the time has clearly exceeded the timeout. What could I be doing wrong in here?
import time
import pyshark
prog_start = time.time()
capture = pyshark.LiveCapture(interface='en0')
capture.sniff(timeout=10)
start_time = capture[0].frame_info.time_epoch
end_time = capture[-1].frame_info.time_epoch
print("Capture lasted:", float(end_time) - float(start_time))
pkt_num = 0
for pkt in capture:
pkt_num += 1
print("Time", time.time() - prog_start, "Pkt#", pkt_num)
We then get this output, with thousands of additional packets a second, past when the capture should have stopped:
Capture lasted: 9.148329019546509
Time 10.346031188964844 Pkt# 1
Time 10.348641157150269 Pkt# 2
Time 10.351708889007568 Pkt# 3
Time 10.353564977645874 Pkt# 4
Time 10.35555100440979 Pkt# 5
...
Question
Why does PyShark continue to capture packets after the timeout?

I was having this same issue, I managed to find a bit of a solution for it. It isn't perfect but it works by telling the capture loop to stop on the next packet and sends an empty packet it will see to make it end. I made it a udp packet on a high port for my case because I use a filter that filters out most traffic so this solution worked for me
class PacketCapture(threading.Thread):
capture = 1
def __init__(self, interface_name):
threading.Thread.__init__(self)
self.interface_name = interface_name
def stop(self):
self.capture = 0
def run(self):
capture = pyshark.LiveCapture(interface=self.interface_name)
try:
for packet in capture.sniff_continuously():
if not self.capture:
capture.close()
except pyshark.capture.capture.TSharkCrashException:
self.exited = 1
print("Capture has crashed")
#start capture
pcap = PacketCapture(interface_name)
pcap.start()
#stop capture
pcap.stop()
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
msg = bytes("", "UTF-8")
sock.sendto(msg, ("external IP", 12345))
sock.close
I'm relatively new to python myself but I think this should be a somewhat acceptable solution given the scenario

Problems with PyShark
It looks like you're running into a known issue with PyShark that hasn't been fixed in years. Per the thread, the author wrote
You can subclass LiveCapture and override the get_parameters() function, adding your own parameters.
You could modify the parameters sent to tshark, but at this point, why not just use a tshark command directly?
Using Tshark Instead
PyShark is just a wrapper for tshark on your system. If you want to use subprocess with Python, the equivalent tshark command is tshark -a duration:5. The other advantage of using tshark directly is that subprocess gives you a pid that you can kill on an arbitrary condition.
See the manpage for more details.

You can use the following line in bash or even in python within os.system, os.popen or even subprocess:
while IFS= read -r line; do if [[ $line =~ 'some protocol' ]]; then <SOME_ACTION>; break; fi; done < <(sudo tshark)

Related

ffmpeg: What is the best practice to keep a live connection/socket with a camera, and save time on ffprobe

Today... I used the following command: with subprocess.PIPE and subprocess.Popen in python 3:
ffmpeg -i udp://{address_of_camera} \
-vf select='if(eq(pict_type,I),st(1,t),gt(t,ld(1)))' setpts=N/FRAME_RATE/TB \
-f rawvideo -an -vframes {NUM_WANTED_FRAMES} pipe:`
This command helps me to capture NUM_WANTED_FRAMES frames from a live camera at a given moment.
However... it takes me about 4 seconds to read the frames, and about 2.5 seconds to open a socket between my computer and the camera's computer.
Is there a way, to have a socket/connection always open between my computer and the camera's computer, to save the 2.5 seconds?
I read something about fifo_size and overrun_fatal. I thought that maybe I can set fifo_size to be equal to NUM_WANTED_FRAMES, and overrun_fatal to True? Will this solve my problem? Or is there a different and simpler/better solution?
Should I try to record always (no -vframes flag) store the frames in a queue(With max size), and upon a wish to slice the video, read from my queue buffer? Will it work well with the keyframe?
Also... What to do when ffmpeg fails? restart the ffmpeg command?
FFmpeg itself is an one-n-done type of app. So, to keep the camera running, the best option is to "record always (no -vframes flag)" and handle whether to drop/record frames in Python.
So, a rough sketch of the idea:
import subprocess as sp
from threading import Thread, Event
from queue import Queue
NUM_WANTED_FRAMES = 4 # whatever it is
width = 1920
height = 1080
ncomp = 3 # rgb
framesize = width*height*ncomp # in bytes
nbytes = framesize * NUM_WANTED_FRAMES
proc = Popen(<ffmpeg command>, stdout=sp.PIPE)
stdout = proc.stdout
buffer = Queue(NUM_WANTED_FRAMES)
req_frame = Event() # set to record, default to drop
def reader():
while True:
if req_frame.is_set():
queue.put(stdout.read(nbytes))
record_frame.clear()
else:
# frames not requested, drop
stdout.read(framesize)
rd_thread = threading.Thread(target=reader)
rd_thread.start()
...
# elsewhere in your program, do this when you need to get the camera data
req_frame.set()
framedata = queue.get()
....
Will it work well with the keyframe?
Yes, if your FFmpeg command has -discard nokey it'll read just keyframes.
What to do when ffmpeg fails? restart the ffmpeg command?
Have another thread to monitor the health of proc (Popen object) and if it is dead, you need to restart subprocess with the same command and overwrite with the new stdout. You probably want to protect your code with try-except blocks as well. Also, adding timeouts to queue ops would be a good idea, too.

Sending spoofed syslog messages in Perl

TL;DR I am trying to send syslog messages I have already created to a syslog server using spoofed source IPs but I'm making it really hard for myself and could do with a succinct approach
Most questions about syslog do not include the spoofed IP problem hence I am asking afresh.
I am writing (well updating) a script I wrote a long time ago that generates spoofed syslog messages (using UDP). It currently uses Net::RawIP which is terrible for portability and also the code for the transmission I wrote ages ago has decided to stop working in Perl 5 (I haven't used this for ages and I am refreshing it). I have been meaning to get rid of Net::RawIP for ages but its the only one I know how to use!
Given I have to fix it and I have a little time at the moment I probably want to move to use the Socket capability, which is what I have been playing with - using code from SO or gists or other places I can find - rather than something like IO::Socket as I need the spoofed IPs permission given a low level ability to write to sockets.
However, I have tied myself in knots with this, what I have right now forms the packets from scratch and then creates a socket and sends it, but in the processes wraps a superflous IPv4 header (I can see using wireshark) and without starting afresh I think its stuck like that as it has a fundamental flaw, hence I'm not sharing old code.
Basically, I can keep playing with the overly complicated code I have or ask for help simplify it, as I am beyond my knowledge of sockets and many hours of googling haven't helped much. What I keep finding is code that will work but it not compliant in some way - probably not an issue for what they are usually for which is for DDOS or syn attacks or whatever.
Key requirements of this are (which every attempt I have done has failed in some form!):
must come from a spoofed source IP and go to a known destination IP
(hence I'm using UDP) (both of which I have in config variables) so that the syslog server things many different devices generated the logs
must come from a set port and go to a set port (both of which I have in my
existing config variables)
must contain a message I have already formed which includes all the syslog content (the PRI and the syslog message content)
must be fully complaint with checksums and packet lengths etc when received
be as portable as possible (I'll probably embed it in my main script to keep it all in one file I can share with others).
I just feel there must be a simple and easy way to do this as everything I have is overly flexible and complicated and a nightmare to unwind.
NB This is shared in Sourceforge as "must syslog" and so you can see what I used to do but be aware its stopped working so it wont work if you run it currently! Once I fix this I'll upload a new version.
Cheers, --Chris
Actually I just had a breakthrough, I have linked the sources in the code but I found a test harness that is pretty basic and some UDP checksum code that with a bit of playing worked together and wireshark confirms its all correct
Posted in case someone else needs it as its taken me a good couple of days to get to this
#!/usr/bin/perl
use Socket;
use List::Util qw(sum);
sub udp_checksum {
# thanks to ikegami in post https://stackoverflow.com/questions/46181281/udp-checksum-function-in-perl
my $packed_src_addr = shift; # As 4-char string, e.g. as returned by inet_aton.
my $packed_dst_addr = shift; # As 4-char string, e.g. as returned by inet_aton.
my $udp_packet = shift; # UDP header and data as a string.
my $sum = sum(
IPPROTO_UDP,
length($udp_packet),
map({ unpack('n*', $_) }
$packed_src_addr,
$packed_dst_addr,
$udp_packet."\0", # Extra byte ignored if length($udp_packet) is even.
),
);
while (my $hi = $sum >> 16) {
$sum = ($sum & 0xFFFF) + $hi;
}
return ~$sum & 0xFFFF;
}
#this was found and adapted from http://rhosted.blogspot.com/2009/08/creating-udp-packetip-spoofing-through.html
$src_host = $ARGV[0];
$dst_host = $ARGV[1];
$src_port = 33333;
$dest_port = 514;
$cksum = 0; #initialise, we will sort this later
#$udp_len is the udp packet length in the udp header. Its just 8 plus the length of the data
#for this test harness we will set the data here too
$data = "<132> %FWSM-3-106010: Deny inbound tcp src outside:215.251.218.222/11839 dst inside:192.168.1.1/369";
$udp_len = 8+length($data);
$udp_proto = 17; #17 is the code for udp
#Prepare the udp packet, needed for the checksum to happen, then get the checksum (horrifically complicated, just google it)
$udp_packet = pack("nnnna*", $src_port,$dest_port,$udp_len, $cksum, $data);
$cksum = udp_checksum(inet_aton($src_host),inet_aton($dst_host),$udp_packet);
$zero_cksum = 0;
#test harness checks about host IPs
my $dst_host = (gethostbyname($dst_host))[4]; my $src_host = (gethostbyname($src_host))[4];
# Now lets construct the IP packet
my $ip_ver = 4;
my $ip_len = 5;
my $ip_ver_len = $ip_ver . $ip_len;
my $ip_tos = 00;
my ($ip_tot_len) = $udp_len + 20;
my $ip_frag_id = 19245;
my $ip_frag_flag = "010";
my $ip_frag_oset = "0000000000000";
my $ip_fl_fr = $ip_frag_flag . $ip_frag_oset;
my $ip_ttl = 30;
#H2H2nnB16C2na4a4 for the IP Header part#nnnna* for the UDP Header part.
#To understand these, see the manual of pack function and IP and UDP Header formats
#IP checksum ($zero_cksum is calculated by the kernel. Dont worry about it.)
my ($pkt) = pack('H2H2nnB16C2na4a4nnnna*',
$ip_ver_len,$ip_tos,$ip_tot_len,$ip_frag_id,
$ip_fl_fr,$ip_ttl,$udp_proto,$zero_cksum,$src_host,
$dst_host,$src_port,$dest_port,$udp_len, $cksum, $data);
#bit that makes the socket and sends the packet
socket(RAW, AF_INET, SOCK_RAW, 255) || die $!; setsockopt(RAW, 0, 1, 1);
my ($destination) = pack('Sna4x8', AF_INET, $dest_port, $dst_host);
send(RAW,$pkt,0,$destination);

On demand inter-process communication with perl

Imagine inter-processes communication
+--------------+ +------------+
| main_process | ==produces data somewhat to=> | monitoring |
+--------------+ +------------+
where:
the main_process is running non-stop and produces some data for monitoring
the monitoring is running only sometimes, and when it is running it should read the data produced by main_process
and when the monitoring is not running the data produced by main_process should be not saved.
The question is: How to write like "on demand" IPC?
The code for main_process is basically the next (the real one is more complicated):
use 5.014;
use warnings;
my $box = new BlackBox( callback => sub {
my ($self, $jref) = #_;
#
# processing of $jref
#
});
$box->run();
The callback is called every 2-5 seconds and as i told above, this process should run non-stop. I can't change the BlackBox.
I need:
write the processing of $jref part - what should send $jref to somewhere
and the monitoring process itself, what should read the data, when it is running...
Don't need any code, need only some pointers to the right direction, or idea how to do this, without filling up my memory or HDD, so the simplest way:
write the $jref to the file is not suitable because it will fill my HDD when the monitoring is not running.
If someone care, the $jref is a reference to json string, so i can do:
use JSON::XS qw(decode_json):
my $perlref = decode_json($$jref);
My first thought was "UDP to localhost?"
Variations on that idea include AF_UNIX or a named pipe. With a stream socket you'd do a non-blocking connect, and with the pipe you'd do O_WRONLY|O_NONBLOCK, and if you get EAGAIN just return without writing.
You can save your file handle and reuse it across multiple calls, just close it and reopen if you get EPIPE. You'll want $SIG{PIPE}='IGNORE'; hopefully the black box doesn't object to that.
The reading side is as simple as cat $path_to_fifo or nc -l -u -p $udpport, slightly harder if you do an AF_UNIX socket.
It sounds like using a socket will be enough. Choose a port number P and try to connect to the monitor from the main application (localhost:P). If the port is open, send the actual data, not the reference. If the port is closed then just ignore the data. Your monitor will listen on P and process any data received through this port.

Newbie perl serial programming

I have a RFXCOM transceiver for 433 mhz signals. I managed to put together a program that can transmit signals without a problem (and for example turn on a lamp). However I also want to be able to receive signals from my remote control. A bit of googling gave me this working code;
use Device::SerialPort;
my $PortObj=Device::SerialPort->new("/dev/ttyUSB1");
$PortObj->user_msg(ON);
$PortObj->databits(8);
$PortObj->baudrate(38400);
$PortObj->parity("none");
$PortObj->stopbits(1);
$PortObj->handshake("rts");
my $STALL_DEFAULT=10; # how many seconds to wait for new input
my $timeout=$STALL_DEFAULT;
$PortObj->read_char_time(0); # don't wait for each character
$PortObj->read_const_time(1000); # 1 second per unfulfilled "read" call
my $chars=0;
my $buffer="";
while ($timeout>0) {
my ($count,$saw)=$PortObj->read(1); # will read _up to_ 255 chars
if ($count > 0) {
$chars+=$count;
$buffer.=$saw;
print $saw;
# Check here to see if what we want is in the $buffer
# say "last" if we find it
}
else {
$timeout--;
}
}
if ($timeout==0) {
die "Waited $STALL_DEFAULT seconds and never saw what I wanted\n";
}
One thing I can't figure out - this script gives me the output after about 10 seconds, but I want to see the received data instantly. Any idea what I need to change? I don't think it has to do with the timeout part since that just seems to measure the time since the last received signal. Any ideas?
Suffering from buffering? Set
$| = 1;
at the top of your script.

pySerial buffer won't flush

I'm having a problem with serial IO under both Windows and Linux using pySerial. With this code the device never receives the command and the read times out:
import serial
ser = serial.Serial('/dev/ttyUSB0',9600,timeout=5)
ser.write("get")
ser.flush()
print ser.read()
This code times out the first time through, but subsequent iterations succeed:
import serial
ser = serial.Serial('/dev/ttyUSB0',9600,timeout=5)
while True:
ser.write("get")
ser.flush()
print ser.read()
Can anyone tell what's going on? I tried to add a call to sync() but it wouldn't take a serial object as it's argument.
Thanks,
Robert
Put some delay in between write and read
e.g.
import serial
ser = serial.Serial('/dev/ttyUSB0',9600,timeout=5)
ser.flushInput()
ser.flushOutput()
ser.write("get")
# sleep(1) for 100 millisecond delay
# 100ms dely
sleep(.1)
print ser.read()
Question is really old, but I feel this might be relevant addition.
Some devices (such as Agilent E3631, for example) rely on DTR. Some ultra-cheap adapters do not have DTR line (or do not have it broken out), and using those, such devices may never behave in expected manner (delays between reads and writes get ridiculously long).
If you find yourself wrestling with such a device, my recommendation is to get an adapter with DTR.
This is because pyserial returns from opening the port before it is actually ready. I've noticed that things like flushInput() don't actually clear the input buffer, for example, if called immediately after the open(). Following is code to demonstrate:
import unittest
import serial
import time
"""
1) create a virtual or real connection between COM12 and COM13
2) in a terminal connected to COM12 (at 9600, N81), enter some junk text (e.g.'sdgfdsgasdg')
3) then execute this unit test
"""
class Test_test1(unittest.TestCase):
def test_A(self):
with serial.Serial(port='COM13', baudrate=9600) as s: # open serial port
print("Read ASAP: {}".format(s.read(s.in_waiting)))
time.sleep(0.1) # wait for 100 ms for pyserial port to actually be ready
print("Read after delay: {}".format(s.read(s.in_waiting)))
if __name__ == '__main__':
unittest.main()
"""
output will be:
Read ASAP: b''
Read after delay: b'sdgfdsgasdg'
.
----------------------------------------------------------------------
Ran 1 test in 0.101s
"""
My workaround has been to implement a 100ms delay after opening before doing anything.
Sorry that this is old and obvious to some, but I didn't see this option mentioned here. I ended up calling a read_all() when flush wasn't doing anything with my hardware.
# Stopped reading for a while on the connection so things build up
# Neither of these were working
conn.flush()
conn.flushInput()
# This did the trick, return value is ignored
conn.read_all()
# Waits for next line
conn.read_line()