How to convert pcap file to nfcapd file - pcap

I have a pcap file of size 1.4 GB. I am trying to convert this file to nfcapd. But I cannot convert it.
First of all, I typed
nfcapd -p 12345 -l ./
Then the computer creates some nfcapd file continuously. I tried it 2 days ago and the computer keeps creating nfcapd file until today. Then I stopped the process. Then I tried
softflowd -n localhost:12345 -r myFile.pcap
Then the terminal gives an error
Unsupported data link type 104
Do you have any idea to convert this to nfcapd through terminal?
I am using Debian Linux 64 bit OS.

You could have nfcapd read the pcap file directly and save it as you specify in your config (Netflow v5 or v9. IPFIX support is currently only experimental).
According to the man page of nfcapd:
-f <pcap_file>
Read netflow packets from a give pcap_file instead of the network. This requires nfcapd to be compiled with the pcap option and is intended for debugging only.
In case you have installed nfdump through a packet repository, the chance is high that it is not compiled with the required --enable-readpcap flag (which is default off).
Try and download the source from here and compile it yourself.

FYI, on my x86_64 Fedora 26, I need to use nfpcapd (note the extra p) to convert a pcap file to a netflow file.
$ mkdir sipp
$ nfpcapd -l sipp -r /usr/share/sipp/pcap/g711a.pcap
Add extension: 2 byte input/output interface index
Add extension: 4 byte input/output interface index
Add extension: 2 byte src/dst AS number
Add extension: 4 byte src/dst AS number
Add extension: 4 byte output bytes
Add extension: 8 byte output bytes
Add extension: NSEL Common block
Add extension: NSEL xlate ports
Add extension: NSEL xlate IPv4 addr
Add extension: NSEL xlate IPv6 addr
Add extension: NSEL ACL ingress/egress acl ID
Add extension: NSEL username
Add extension: NSEL max username
Add extension: NEL Common block
Startup.
[140499169166528] WaitDone() waiting
pcap_next_ex() end of file
Packet processing stats: Total: 236, Skipped: 0, Unknown: 0, Short snaplen: 0
Packet processing stats: Total: 236, Skipped: 0, Unknown: 0, Short snaplen: 0
Terminating packet dumping: exit: 0
[140499169166528] WaitDone() signal 10
Exit status thread[140498942019328]: 0
[140498950412032] Signal handler: 12
Nodes in use: 1, Flows: 1 CacheOverflow: 0
Ident: 'none' Flows: 1, Packets: 236, Bytes: 61360, Max Flows: 1
Terminating flow processng: exit: 0
Exit status thread[140498950412032]: 0
Terminating nfpcapd.
$ nfdump -v sipp/nfcapd.200207260815
File : sipp/nfcapd.200207260815
Version : 1 - not compressed
Blocks : 1
Type 1 : 0
Type 2 : 1
Type 3 : 0
Records : 2

Related

rsync tries to copy although file is copied

I'm trying to understand the following, maybe one of you guys can help me out by explaining on what exactly happens here:
My goal is to write a script, which copies files from the find-command, parallels rsync-commands and backup those, by re-creating the folder-structure from the source on the destination as well.
Somehow my initial script does not work like that, and i don't really have an idea on how to fix it to behave that way.
Initially, i've copied the files with "ls -1 /foldername" as sourceFolder, which is not best-practice. So i've tried to change it to using "find" as described beyond.
find "$sourceFolder" -maxdepth 2 -mindepth 2 -print0 | xargs --verbose -0 -I {} -P $maxThreads -n 1 rsync -valhP {}/ "$destFolder"/ --checksum --human-readable --stats --dry-run >> "$logDir"/result.log
--
If i run this script, it literaly would recopy the file(s), although those exist in the destination folder i would expect it to be.
sending incremental file list
.d..tp..... ./
>f+++++++++ macs.txt
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 242 bytes
Total transferred file size: 242 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.484 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 83
Total bytes received: 22
So as it turned out, sourceFolder and destFolder have to match, and i can't run anything like sourceFolder/folder1/folder1a destFolder/.
Can anyone help me out on what's wrong and why it behaves like that?
Thanks a Lot,
M.

"image is too large" keeps on happening to openbmc image for Raspberrypi platform

Could someone please give me advice to make an openbmc image for Raspberrypi platform ?
Before I tried, I looked through related documents and believed an openbmc image can be worked on Raspberrypi.
Like OpenBMC with Raspberry Pi (2 or 3) and build bmcweb?
and https://kevinleeblog.github.io/project1/2019/11/25/openbmc-for-raspberry-pi-zero/.
So, I followed these instructions and tried the following steps.
#1: Git clone openbmc.git to my local PC.
tm#tm-VB1:~/Rpi4-64$ git clone https://github.com/openbmc/openbmc.git
Snip the logs but it looks no problem.
Receiving objects: 100% (182121/182121), 84.10 MiB | 5.55 MiB/s, done.
Resolving deltas: 100% (96860/96860), done.
#2: set TEMPLATECONF for raspberrypi
tm#tm-VB1:~/Rpi4-64$ export TEMPLATECONF=meta-evb/meta-evb-raspberrypi/conf
tm#tm-VB1:~/Rpi4-64$ echo $TEMPLATECONF
meta-evb/meta-evb-raspberrypi/conf
#3: set up the environment by "openbmc-env"
tm#tm-VB1:~/Rpi4-64/openbmc$ . openbmc-env
### Initializing OE build env ###
Snip the logs but it looks no problem. As you know, the script automatically creates a subdirectory,build, under openbmc.
Common targets are:
obmc-phosphor-image
tm#tm-VB1:~/Rpi4-64/openbmc/build$
#4: Change the directory and edit local.conf for my Raspberrypi platform.
tm#tm-VB1:~/Rpi4-64/openbmc/build$ cat ./conf/local.conf
Snip the log for unchanged part.
MACHINE ??= "raspberrypi4-64" <<< Change here for my platform.
DL_DIR ?= "/home/tm/Yocto/downloads" <<< Add here for build-time reduction at retry.
SSTATE_DIR ?= "/home/tm/Yocto/sstate-cache" <<< Add here for build-time reduction at retry.
#5: Change FLASH_SIZE variable based on the following sugestion. https://github.com/openbmc/openbmc/issues/3590
tm#tm-VB1:~/Rpi4-64/openbmc/meta-phosphor/classes$ cat image_types_phosphor.bbclass
Snip the log.
# Flash characteristics in KB unless otherwise noted
FLASH_SIZE ?= "131072" <<< I changed only this variable from 32768 to 131072.
#6: bitbake starts.
tm#tm-VB1:~/Rpi4-64/openbmc/bitbake obmc-phosphor-image
Then, ERROR happened.
ERROR: Logfile of failure stored in: /home/tm/Rpi/openbmc/build/tmp/work/raspberrypi-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/temp/log.do_generate_static.2055074
DEBUG: Executing python function do_generate_static
DEBUG: Executing shell function do_mk_static_nor_image
32768+0 records in
32768+0 records out
33554432 bytes (34 MB, 32 MiB) copied, 0.09147 s, 367 MB/s
DEBUG: Shell function do_mk_static_nor_image finished
DEBUG: Considering file size=495980 name=/home/tm/Rpi/openbmc/build/tmp/deploy/images/raspberrypi/u-boot.bin
DEBUG: Spanning start=0K end=512K
DEBUG: Compare needed=495980 available=524288 margin=28308
484+1 records in
484+1 records out
495980 bytes (496 kB, 484 KiB) copied, 0.00120141 s, 413 MB/s
DEBUG: Considering file size=8266960 name=/home/tm/Rpi/openbmc/build/tmp/deploy/images/raspberrypi/fitImage-obmc-phosphor-initramfs-raspberrypi-raspberrypi
DEBUG: Spanning start=512K end=4864K
>>>DEBUG: Compare needed=8266960 available=4456448 margin=-3810512
ERROR: Image '/home/tm/Rpi/openbmc/build/tmp/deploy/images/raspberrypi/fitImage-obmc-phosphor-initramfs-raspberrypi-raspberrypi' is too large!
DEBUG: Python function do_generate_static finished
It said margin=-3810512.
Now, my 2nd try.
I removed the whole openbmc directory and did the same steps above.
But this time, I change FLASH_SIZE from 32768 to 262144.
It is the same result like below.
ERROR: obmc-phosphor-image-1.0-r0 do_generate_static: Image '/home/tm/Rpi4/openbmc/build/tmp/deploy/images/raspberrypi4/u-boot.bin' is too large!
ERROR: Logfile of failure stored in: /home/tm/Rpi4/openbmc/build/tmp/work/raspberrypi4-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/temp/log.do_generate_static.2061792
ERROR: Task (/openbmc/meta-phosphor/recipes-phosphor/images/obmc-phosphor-image.bb:do_generate_static) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3915 tasks of which 2633 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/openbmc/meta-phosphor/recipes-phosphor/images/obmc-phosphor-image.bb:do_generate_static
Summary: There were 2 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
tm#tm-VB1:~/Rpi4/openbmc/build$ cat /home/tm/Rpi4/openbmc/build/tmp/work/raspberrypi4-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/temp/log.do_generate_static.2061792
DEBUG: Executing python function do_generate_static
DEBUG: Executing shell function do_mk_static_nor_image
32768+0 records in
32768+0 records out
33554432 bytes (34 MB, 32 MiB) copied, 0.177223 s, 189 MB/s
DEBUG: Shell function do_mk_static_nor_image finished
DEBUG: Considering file size=548224 name=/home/tm/Rpi4/openbmc/build/tmp/deploy/images/raspberrypi4/u-boot.bin
DEBUG: Spanning start=0K end=512K
>>>DEBUG: Compare needed=548224 available=524288 margin=-23936
ERROR: Image '/home/tm/Rpi4/openbmc/build/tmp/deploy/images/raspberrypi4/u-boot.bin' is too large!
DEBUG: Python function do_generate_static finished
tm#tm-VB1:~/Rpi4/openbmc/build$
It said margin=-23936.
OK. Image is too large. So,my 3rd try.
I removed the whole openbmc directory and did the same steps above.
But this time, I change FLASH_SIZE from 32768 to 9437184.
It is the same result like below.
ERROR: obmc-phosphor-image-1.0-r0 do_generate_static: Image '/home/tm/Rpi4/openbmc/build/tmp/deploy/images/raspberrypi4/u-boot.bin' is too large!
ERROR: Logfile of failure stored in: /home/tm/Rpi4/openbmc/build/tmp/work/raspberrypi4-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/temp/log.do_generate_static.2058361
ERROR: Task (/openbmc/meta-phosphor/recipes-phosphor/images/obmc-phosphor-image.bb:do_generate_static) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3935 tasks of which 0 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/openbmc/meta-phosphor/recipes-phosphor/images/obmc-phosphor-image.bb:do_generate_static
Summary: There were 4 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
tm#tm-VB1:~/Rpi4/openbmc$
tm#tm-VB1:~/Rpi4/openbmc$ cat /home/tm/Rpi4/openbmc/build/tmp/work/raspberrypi4-openbmc-linux-gnueabi/obmc-phosphor-image/1.0-r0/temp/log.do_generate_static.2058361
DEBUG: Executing python function do_generate_static
DEBUG: Executing shell function do_mk_static_nor_image
32768+0 records in
32768+0 records out
33554432 bytes (34 MB, 32 MiB) copied, 0.173685 s, 193 MB/s
DEBUG: Shell function do_mk_static_nor_image finished
DEBUG: Considering file size=548224 name=/home/tm/Rpi4/openbmc/build/tmp/deploy/images/raspberrypi4/u-boot.bin
DEBUG: Spanning start=0K end=512K
>>>DEBUG: Compare needed=548224 available=524288 margin=-23936
ERROR: Image '/home/tm/Rpi4/openbmc/build/tmp/deploy/images/raspberrypi4/u-boot.bin' is too large!
DEBUG: Python function do_generate_static finished
tm#tm-VB1:~/Rpi4/openbmc$
It said the same margin as 256MB case.
My 4th try.
I removed the whole openbmc directory and did the same steps above.
I changed MACHINE ??= "raspberrypi4-64" to "raspberrypi2"
But this time, I change FLASH_SIZE from 32768 to 33554432.
It is the same result before.
My 5th try.
I removed the whole openbmc directory and did the same steps above.
I used MACHINE ??= "raspberrypi2"
But this time, I change FLASH_SIZE from 32768 to 67108864.
It is the same result before.
After I tried several variations, it always said "image is too large" although I changed FLASH_SIZE to much much larger one.
So, I am wondering if I have missed some important configuration or it needs another parameter to fix this except FLASH_SIZE.
By the way, I tried romulus and made it.
My environment is ubuntu-20.04.2.0-desktop-amd64.
I really appreciate someone could kindly give me advice to make this work.
Interesting, I don't have a quick fix for you but I did notice the partition that is over sized is the uboot partition. The uboot is a smaller separate binary installed on the machine. It looks as if your uboot build is over 512k and the partition is set for 512k. Your flash size is massize
FLASH_SIZE = 9437184" that is more then a gig, (because FLASH_SIZE is in K)
If I were you I would first try to build an older version of openbmc for raspberry pi. (It used to work so you just need to find the commit before uboot grew to big). Use git to move back a month until you find it works.
If that does not work I would try to modify the partition table.
here is where you failing
this looks fine building the uboot image looks fine
increasing the kernel offset make if build, but the other targets in openbmc will not be happy with this solution. So maybe meta-raspberry-pi will have to override the partition table (if uboot can not be shrunk)
What ever you do, open an issue on the github and share you changes. Also use the discord, and gerrit.
I just replicated this issue. We should fix it

Raspberry Pi 4 saving corrupt h264 files?

I've been able to record .h264 files without a problem, both using raspivid and through Python, but whatever I try, I can't convert them to a .mp4, .mkv or .avi file.
I've tried converting using MP4box (suggested in several places) and mkvmerge, but to no avail.
When I use MP4box (MP4box -add <source.h264> <dest.mp4>), I get :
AVC-H264 import - frame size 1920 x 1080 at 25.000 FPS
WARNING: NAL Unit type 0 not handled - adding5/100)
AVC Import results: 44 samples - Slices: 3 I 41 P 0 B - 0 SEI - 3 IDR
Saving video.mp4: 0.500 secs Interleaving
Then I try to play this file using VLC, it doesn't give an error, but shows garbled data.
To be clear, the camera works fine (shows perfect video using Python code and raspistill outputs a perfect jpg file). I tried a different camera as well.
An attempt with mkvmerge :
pi#raspberrypi:~/cam $ raspivid -o test.h264 -fps 30 -t 15000 -w 1920 -h 1080
pi#raspberrypi:~/cam $ mkvmerge --default-duration 0:30p -o video.mkv test.h264
mkvmerge v31.0.0 ('Dolores In A Shoestand') 32-bit
'test.h264': Using the demultiplexer for the format 'AVC/h.264'.
'test.h264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
The file 'video.mkv' has been opened for writing.
Error: 'test.h264' track 0: mkvmerge encountered broken or unparsable data in this AVC/h.264 video track. Either your file is damaged (which mkvmerge cannot cope with yet) or this is a bug in mkvmerge itself. The error message was:
Success
pi#raspberrypi:~/cam $
This is on a Raspberry Pi 4.
A sample file can be found here : https://filebin.net/c40usz0crhgggadf
Created with : raspivid -t 30000 -w 640 -h 480 -fps 25 -b 1200000 -p 0,0,640,480 -o pivideo.h264
I'm going to respond to this myself. After doing an 'apt-get upgrade', the problem is gone.
If anyone wants to figure this out to find the root cause, this is what apt-get did :
The following packages will be upgraded:
bind9-host bluez-firmware firmware-atheros firmware-brcm80211 firmware-libertas firmware-misc-nonfree firmware-realtek libbind9-161 libdns-export1104 libdns1104 libgs9 libgs9-common libisc-export1100 libisc1100 libisccc161
libisccfg163 liblwres161 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc libraspberrypi0 pi-bluetooth raspberrypi-bootloader raspberrypi-kernel raspberrypi-kernel-headers raspi-config rpd-plym-splash
The following packages will be DOWNGRADED:
python3-pgzero
27 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.

Does signing a DLL invalidate a PDB file?

Analyzing a crash dump, WinDbg says my symbols (PDB file) does not match the module. The symbols are the ones generated when the DLL was compiled. The only thing that I can imagine would cause a mismatch is that the DLL was signed.
I'm using !chksym to validate symbols:
!chksym libcef.dll D:\sym\libcef.dll.pdb
libcef.dll
Timestamp: 5BB3D477
SizeOfImage: 626D000
pdb: F:\src\out\libcef.dll.pdb
pdb sig: B0065D83-113F-63BE-53BC-AEF07EC816B4
age: 1
libcef.dll.pdb
pdb sig: 9BA88A40-D168-44F2-44C1-DD2D73A38B38
age: 1
sig MISMATCH: libcef.dll.pdb and libcef.dll
Code signing an executable or DLL does not affect the debug header of the executable. Thus, it will still match the PDB.
...\SigningPdb\bin\Release>symchk signed.exe /s .
SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1
...\SigningPdb\bin\Release>symchk unsigned.exe /s .
SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1
And
...\SigningPdb\bin\Release>ChkMatch.exe -c signed.exe SigningPdb.pdb
ChkMatch - version 1.0
Copyright (C) 2004 Oleg Starodumov
http://www.debuginfo.com/
Executable: signed.exe
Debug info file: SigningPdb.pdb
Executable:
TimeDateStamp: bc78c18e
Debug info: 2 ( CodeView )
TimeStamp: a7b373e5 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 97 RVA: 000026a0 FileOffset: 000008a0
CodeView format: RSDS
Signature: {b8ed520c-cdfc-486b-8e1a-7c0752a2a41f} Age: 1
PdbFile: ...\Release\SigningPdb.pdb
Debug info: 16 ( Unknown )
TimeStamp: 00000000 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 0 RVA: 00000000 FileOffset: 00000000
Debug information file:
Format: PDB 7.00
Signature: {b8ed520c-cdfc-486b-8e1a-7c0752a2a41f} Age: 1
Result: Matched
The timestamp is in the COFF header. That header is only 24 bytes in size and will not change during code signing.
Most changes will happen in a new section for certificates. This section, however, will also be ignored during code signing. Otherwise, a second signature would destroy the first signature. (BTW: this section has been used to transport malicious code inside a signed executable)
Of course, the "usual" checksums, which do not consider the EXE/DLL file structure, will report a different checksum.
What may have happened to your DLL or EXE?
you have accidentally rebuilt it, so the time stamp of your DLL does not match the PDB any more
you are using aspect oriented programming (AOP) in .NET and there is some code weaving happening after a rebuild. These tools might not be able to rebuild the PDB after weaving, so the PDB mismatch is already there before the DLL is code signed.

How can I remove COM ports by command line (w/o installing)

The setup: An automated test station, built around a Windows 7 PC. The UUT (Unit Under Test) are connected and disconnected often, creating many COM ports.
The Problem: The test is searching for the device at a specific COM port, requiring the user to manually remove the "ghost" com ports.
The question: Since the software is used by several test stations in parallel, at a production floor, I cannot install additional software (e.g. Devcon, part of Windows SDK). Is there a command line option to remove the COM ports?
Based on This blog entry
The only thing that actually worked for me is not the intended solution... But it fit the case at hand:
Created a batch file at C:\windows\system32
The content:
REG ADD "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\COM Name Arbiter" /v ComDB /t REG_BINARY /d 0206 /f
The actual value of the registry key (0206 in my example) can either be read from that key (use regedit) or calculated from binary:
com8 com7 com6 com5 com4 com3 com2 com1 com16 com15 com14 com13 com12 com11 com10 com9
0 0 0 0 0 0 1 1 0 0 0 0 0 1 1 0
since I wished to keep com1, com2, com10 and com11 - 0000 0011 0000 0110 - which stand for 0206
executing this batch file will remove the unnecessary comports while leaving the one's I intended
*The batch should be at system32 to be executed as elevated (administrator)
*for more details refer to this PDF