Perl Net::Pcap hex dump packet session - perl

I have the following information below being produced by the Net::Pcap module to print the payload of the packets of interest within a capture.
The data below is the excerpt of a Windows executable file being captured within Perl.
I would like to be able to capture all of the hex data output into one file or variable to assess the session data of the file download while retaining the integrity of the hex dump.
The problem I am having is for each packet being produced for the download of the file it obviously produces a hex dump output. This is easily seen by the output below by the string "Payload" I print per packet/hex dump output.
I want to tie all relevant data together for a given file download session. How can I do this in Perl?
Payload:HTTP/1.1 200 OK
Date: Fri, 15 Jun 2012 02:31:32 GMT
Server: Apache
Last-Modified: Sat, 10 Dec 2011 13:38:37 GMT
ETag: "dc44da-4d000-4b3bd04c7a2f1"
Accept-Ranges: bytes
Content-Length: 315392
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Content-Type: application/x-msdos-program
MZ<90>^#^C^#^#^#^D^#^#^#<FF><FF>^#^#<B8>^#^#^#^#^#^#^##^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#<F8>^#^#^#^N^_<BA>^N^#<B4> <CD>!<B8>^AL<CD>!This program cannot be run in DOS mode.^M
$^#^#^#^#^#^#^#4^TGmpu)>pu)>pu)>c}#>ru)>uyI>ru)>uy&>ku)>c}t>ru)><F3>}t>uu)>pu(>
u)>uyv><DA>u)><9C>~w>qu)>uys>qu)>Richpu)>^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#PE^#^#L^A^D^#^_R<E3>N^#^#^#^#^#^#^#^#<E0>^#^O^A^K^A^G
^#<80>^C^#^#<90>^A^#^#^#^#^#g^F^C^#^#^P^#^#^#<90>^C^#^#^##^#^#^P^#^#^#^P^#^#^D^#^#^#^#^#^#^#^D^#^#^#^#^#^#^#^# ^E^#^#^P^#^#^#^#^#^#^C^#^#^#^#^#^P^#^#^P^#^#^#^#^P^#^#^P^#^#^#^#^#^#^P^#^#^#^#^#^#^#^#^#^#^#<A8><91>^D^#P^#^#^#^#^#^E^#<C8>^T^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^##<91>^D^#H^#^#^#^#^#^#^#^#^#^#^#^#<90>^C^#<D0>^A^#^#^#^#^#^#^#^#
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#.text^#^#^#As^C^#^#^P^#^#^#<80>^C^#^#^P^#^#^#^#^#^#^#^#^#^#^#^#^#^# ^#^#`.rdata^#^#<C2>^K^A^#^#<90>^C^#^#^P^A^#^#<90>^C^#^#^#^#^#^#^#^#^#^#^#^#^##^#^##.data^#^#^#D]^#^#^#<A0>^D^#^#^P^#^#^#<A0>^D^#^#^#^#^#^#^#^#^#^#^#^#^##^#^#<C0>.rsrc^#^#^#<C8>^T^#^#^#^#^E^#^# ^#^#^#<B0>^D^#^#^#^#^#^#^#^#^#^#^#^#^##^#^##^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#Payload:^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^
Thank you Borodin for the suggestion. However, I don't know how to use this module correctly and it is apparent as I am still getting the same output. Here is the snippet of code I am using with this module and the printed hex. As you can see the printed hex is not tied together as one hex output but two seperate for the given TCP stream for which I want to tie together. Any help is appreciated.
my $user_data;
my $header;
my $packet;
my $err ='';
my $pcap = Net::Pcap::open_offline("./pcap", \$err) or die "can't open ./pcap...$err\n";
Net::Pcap::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::Reassemble::loop($pcap, -1, \&rend_callback, '');
Net::Pcap::close($pcap);
my $ip;
my $tcp;
my $payload;
sub process_pkt
{
my ($user_data,$header, $packet) = #_;
$ip = NetPacket::IP->decode(eth_strip($packet));
$tcp = NetPacket::TCP->decode($ip->{data});
$payload = $tcp->{data};
my $hexPayload = hexdump(data => $payload, start_position => 0) if length $payload;
print "Hex Payload:". $hexPayload;
}
print output:
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:./..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
Hex Payload: 0x0000 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0010 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0020 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0030 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0040 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0050 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0060 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0070 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0080 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0090 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep

Take a look at Net::Pcap::Reassemble
This module performs reassembly of fragmented datagrams in libpcap
packet capture data returned by the Net::Pcap loop() function

Related

Snort logs in OSSIM show hex in payload but I want only the text to be there. Is there a config I can change in Snort?

I am new to snort and I am testing things out with OSSIM. I've installed snort and using rsyslog I am getting snort alerts.log to OSSIM. But the thing is payloads of events in OSSIM show as
length = 219
000 : 31 31 2F 32 35 2F 31 39 2D 31 30 3A 30 34 3A 32 11/25/19-10:04:2
010 : 39 2E 37 38 30 31 32 34 20 20 5B 2A 2A 5D 20 5B 9.780124 [**] [
020 : 31 32 30 3A 31 38 3A 33 5D 20 28 68 74 74 70 5F 120:18:3] (http_
030 : 69 6E 73 70 65 63 74 29 20 50 52 4F 54 4F 43 4F inspect) PROTOCO
040 : 4C 2D 4F 54 48 45 52 20 48 54 54 50 20 73 65 72 L-OTHER HTTP ser
050 : 76 65 72 20 72 65 73 70 6F 6E 73 65 20 62 65 66 ver response bef
060 : 6F 72 65 20 63 6C 69 65 6E 74 20 72 65 71 75 65 ore client reque
070 : 73 74 20 20 5B 2A 2A 5D 20 5B 43 6C 61 73 73 69 st [**] [Classi
080 : 66 69 63 61 74 69 6F 6E 3A 20 55 6E 6B 6E 6F 77 fication: Unknow
090 : 6E 20 54 72 61 66 66 69 63 5D 20 5B 50 72 69 6F n Traffic] [Prio
0a0 : 72 69 74 79 3A 20 33 5D 20 7B 54 43 50 7D 20 31 rity: 3] {TCP} 1
0b0 : 39 32 2E 31 36 38 2E 30 2E 31 36 38 3A 38 30 38 92.168.0.168:808
0c0 : 30 20 2D 3E 20 31 39 32 2E 31 36 38 2E 30 2E 31 0 -> 192.168.0.1
0d0 : 32 32 3A 33 39 31 37 30 22 20 0A 22:39170" .
But I want it to be like
11/25/19-10:04:29.780124 [**] [120:18:3] (http_inspect) PROTOCOL-OTHER HTTP server response before client request [**] [Classification: Unknown Traffic] [Priority: 3] {TCP} 192.168.0.168:8080 -> 192.168.0.122:39170" .
Is there any config in snort I can change to make this happen?
Thanks in advance for any help!

Can I tell GitHub (or eq.) to use ASCII to make my binary files readable?

I want to host a binary file on a web-based hosting service for git (i.e. GitHub) so I can easily see any changes made to it.
The binary file in question uses the common ASCII character encoding so that this binary
73 63 6F 70 65 20 68 75 72 72 72 20 69 6E 69 74 69 61 6C 69 7A 65 72 20 64 65 72 70 0D 0A 20 20 20 20 66 75 6E 63 74 69 6F 6E 20 64 65 72 70 20 74 61 6B 65 73 20 6E 6F 74 68 69 6E 67 20 72 65 74 75 72 6E 73 20 6E 6F 74 68 69 6E 67 0D 0A 20 20 20 20 20 20 20 20 63 61 6C 6C 20 53 65 74 53 74 61 72 74 4C 6F 63 50 72 69 6F 28 24 42 2C 24 41 2C 24 41 2C 4D 41 50 5F 4C 4F 43 5F 50 52 49 4F 5F 48 49 47 48 29 0D 0A 20 20 20 20 65 6E 64 66 75 6E 63 74 69 6F 6E 0D 0A 65 6E 64 73 63 6F 70 65
becomes this readable text (†)
scope hurrr initializer derp
function derp takes nothing returns nothing
call SetStartLocPrio($B,$A,$A,MAP_LOC_PRIO_HIGH)
endfunction
endscope
The problem is that services like GitHub will only show me the raw binary when I want to view the file in-browser (or have me download and open it in a text editor):
Right now, to have any changes made, I have to download the changed binary file, convert it to readable text, then use diff to see what changes have been made. This is tedious and loses the beautiful web interface that GitHub has.
So my question is this: Can I tell GitHub (or any equivalent service) to translate a binary file to readable text?
--
(†) For anyone interested in trivia, this is indeed vJass syntax for WarCraft III.

Encoding problems importing

I am having trouble importing a csv to postgres. I import using UTF8 encoding. I am returned the error:
ERROR: invalid byte sequence for encoding "UTF8": 0xcc 0xc1
CONTEXT: COPY a_household , line 2
When I open the file in Notepad++ it says at the bottom it is encoded in UTF-8 w/o BOM. All the text is displayed - all characters appear to display correctly.
The hexdump (suggested below) returns
00000060 2D 32 33 2E 39 32 35 38-37 33 39 32 2C 33 32 2E -23.92587392,32.
00000070 37 33 32 31 38 33 30 33-2C 37 35 2E 31 32 32 36 73218303,75.1226
00000080 32 39 33 31 2C 34 2C 31-2C 61 62 6F 76 65 31 79 2931,4,1,above1y
00000090 72 2C 31 37 2C 2C 58 61-6E 67 61 6E 61 2C 56 65 r,17,,Xangana,Ve
000000A0 6C 68 61 20 61 70 6F 73-74 6F 6C 6F 2C 53 69 6D lha apostolo,Sim
000000B0 2C 47 72 69 70 65 20 4D-61 6C CC C1 72 69 61 2C ,Gripe Mal..ria,
000000C0 50 72 61 74 69 63 61 6E-74 65 5F 64 65 5F 6D 65 Praticante_de_me
000000D0 64 69 63 69 6E 61 20 50-72 CC 5F 70 72 69 6F 2C dicina Pr._prio,
000000E0 4D 65 64 69 63 61 6D 65-6E 74 6F 73 5F 64 61 5F Medicamentos_da_
000000F0 66 61 72 6D CC C1 63 69-61 2C 36 2C 4E CC A3 6F farm..cia,6,N..o
00000100 2C 2C 31 34 2C 4E CC A3-6F 2C 22 44 65 5F 62 6F ,,14,N..o,"De_bo
00000110 72 6C 61 2C 5F 63 6F 6D-5F 61 75 74 6F 72 69 7A rla,_com_autoriz
00000120 61 CC A4 CC A3 6F 22 2C-2C 2C 35 2C 41 72 65 69 a....o",,,5,Arei
00000130 61 20 65 20 65 73 74 61-63 61 73 2C 43 68 61 70 a e estacas,Chap
00000140 61 73 5F 64 65 5F 5A 69-6E 63 6F 2C 41 72 65 69 as_de_Zinco,Arei
00000150 61 2C 4C 65 6E 68 61 2C-4E 61 64 61 2C 4C 61 74 a,Lenha,Nada,Lat
00000160 72 69 6E 61 5F 6D 65 6C-68 6F 72 61 64 61 2C 50 rina_melhorada,P
00000170 61 72 61 5F 6D 65 6D 62-72 6F 73 5F 61 67 72 65 ara_membros_agre
00000180 67 61 64 6F 5F 66 61 6D-69 6C 69 61 72 2C 4C 61 gado_familiar,La
00000190 67 6F 61 2F 6C 61 67 6F-2C 50 6F CC A4 6F 5F 70 goa/lago,Po..o_p
000001A0 CC BC 62 6C 69 63 6F 5F-61 62 65 72 74 6F 2C 37 ..blico_aberto,7
000001B0 2C 50 CC A9 2C 30 2C 2C-30 2C 2C 4E 75 6E 63 61 ,P..,0,,0,,Nunca
000001C0 2C 32 2C 50 6F 72 5F 61-6E 6F 2C 52 61 72 61 6D ,2,Por_ano,Raram
000001D0 65 6E 74 65 2C 52 61 72-61 6D 65 6E 74 65 2C 4D ente,Raramente,M
000001E0 CC C1 2C 32 35 2C 4E CC-A3 6F 2C 54 65 72 72 61 ..,25,N..o,Terra
000001F0 5F 70 65 72 74 65 6E 63-65 6E 74 65 5F 61 5F 65 _pertencente_a_e
00000200 73 74 61 5F 61 6C 64 65-69 61 2C 4E 75 6E 63 61 sta_aldeia,Nunca
00000210 2C 2C 2C 4D 61 69 73 5F-6F 75 5F 6D 65 6E 6F 73 ,,,Mais_ou_menos
00000220 5F 61 5F 6D 65 73 6D 61-2C 4E CC A3 6F 2C 2C 4D _a_mesma,N..o,,M
00000230 61 69 73 5F 66 CC C1 63-69 6C 5F 64 65 5F 6F 62 ais_f..cil_de_ob
00000240 74 65 72 2C 2C 53 69 6D-2C 4E CC A3 6F 2C 2C 2C ter,,Sim,N..o,,,
00000250 2C 4E CC A3 6F 2C 53 69-6D 2C 56 61 63 61 73 2C ,N..o,Sim,Vacas,
00000260 32 20 34 2C 2C 53 69 6D-2C 31 2C 53 69 6D 2C 6A 2 4,,Sim,1,Sim,j
00000270 61 6E 65 69 72 6F 20 66-65 76 65 72 65 69 72 6F aneiro fevereiro
00000280 20 6D 61 72 CC A4 6F 20-61 62 72 69 6C 20 6D 61 mar..o abril ma
00000290 69 6F 20 6A 75 6E 68 6F-20 6A 75 6C 68 6F 20 61 io junho julho a
000002A0 67 6F 73 74 6F 2C 31 30-30 2C 53 61 63 6F 73 2C gosto,100,Sacos,
000002B0 30 2C 4D 61 69 73 5F 64-6F 5F 71 75 65 5F 75 6D 0,Mais_do_que_um
000002C0 61 5F 76 65 7A 2C 53 61-75 64 65 2C 4E CC A3 6F a_vez,Saude,N..o
000002D0 2C 4E CC A3 6F 2C 2C 2D-39 39 38 2C 53 69 6D 2C ,N..o,,-998,Sim,
000002E0 4D 75 69 74 6F 5F 73 61-74 69 73 66 65 69 74 6F Muito_satisfeito
000002F0 2C 4D 75 69 74 6F 5F 73-61 74 69 73 66 65 69 74 ,Muito_satisfeit
00000300 6F 2C 4D 75 69 74 6F 5F-73 61 74 69 73 66 65 69 o,Muito_satisfei
00000310 74 6F 2C 49 6E 73 61 74-69 73 66 65 69 74 6F 2C to,Insatisfeito,
00000320 22 4E 65 6D 5F 69 6E 73-61 74 69 73 66 65 69 74 "Nem_insatisfeit
00000330 6F 2C 5F 6E 65 6D 5F 73-61 74 69 73 66 65 69 74 o,_nem_satisfeit
00000340 6F 22 2C 49 6E 73 61 74-69 73 66 65 69 74 6F 2C o",Insatisfeito,
00000350 4D 75 69 74 6F 5F 73 61-74 69 73 66 65 69 74 6F Muito_satisfeito
00000360 2C 49 6E 73 61 74 69 73-66 65 69 74 6F 2C 49 6E ,Insatisfeito,In
00000370 73 61 74 69 73 66 65 69-74 6F 2C 53 61 74 69 73 satisfeito,Satis
00000380 66 65 69 74 6F 2C 53 61-74 69 73 66 65 69 74 6F feito,Satisfeito
00000390 2C 4E 61 5F 6D CC A9 64-69 61 2C 4D 65 6C 68 6F ,Na_m..dia,Melho
000003A0 72 5F 61 67 6F 72 61 2C-22 44 69 66 69 63 69 6C r_agora,"Dificil
000003B0 20 64 65 20 64 69 7A 65-72 2C 20 70 6F 69 73 20 de dizer, pois
000003C0 6E 61 6F 20 73 65 69 20-6F 20 71 75 65 20 6F 20 nao sei o que o
000003D0 61 6E 6F 20 6D 65 20 72-65 73 65 72 76 61 22 2C ano me reserva",
000003E0 2C 75 75 69 64 3A 66 63-32 34 66 62 38 39 2D 35 ,uuid:fc24fb89-5
000003F0 34 63 30 2D 34 39 66 34-2D 61 31 36 31 2D 35 64 4c0-49f4-a161-5d
00000400 64 34 63 63 39 62 37 39-65 39 0D 0A d4cc9b79e9..
...
Any help would be very much appreciated.
Becky

How do I reassemble IP fragments with Perl's Net::Pcap::Reassemble?

I am new to Perl and trying to use the Net::Pcap::Reassemble - IP fragment reassembly for Net::Pcap. I am trying to reassemble TCP packets and "tie" the packets streams of interest and print the "tied" hex output for that data of interest. Here is the code below and and the printed output I get. The printed output is two separate Hex dumps (denoted by "Hex Payload:" string). It is apparent I am not calling Net::Pcap::Reassemble module correctly. The desire end output that I am trying to achieve is below, taken from the printed output. Can someone please point me into the right direction in using this module to achieve my desired output? Thank you.
my $user_data;
my $header;
my $packet;
my $err ='';
my $pcap = Net::Pcap::open_offline("./pcap", \$err) or die "can't open ./pcap...$err\n";
Net::Pcap::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::Reassemble::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::close($pcap);
my $ip;
my $tcp;
my $payload;
sub process_pkt
{
my ($user_data,$header, $packet) = #_;
$ip = NetPacket::IP->decode(eth_strip($packet));
$tcp = NetPacket::TCP->decode($ip->{data});
$payload = $tcp->{data};
my $hexPayload = hexdump(data => $payload, start_position => 0) if length $payload;
print "Hex Payload:". $hexPayload;
}
Output (this is what I currently get, but want to concatenate it together):
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:.*/*..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
Hex Payload: 0x0000 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0010 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0020 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0030 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0040 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0050 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0060 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0070 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0080 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0090 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep
Desired output (the above tied/concatenated together):
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:.*/*..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
0x0160 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0170 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0180 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0190 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0200 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0210 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0220 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0230 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0240 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0250 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep
You are trying to reassemble a network session, not a fragmented network packet. You should be using the module 'Net::Analysis'. It can, with some effort on your part, reassemble a complete network session. You will soon learn to hate pipe-lining.

MSMQ How best to handle classes when using binary encoding

I'm new here, so please be gentle.
This question revolves around VB.net / VS2010 / MSMQ 4.0
I'm developing an application that has MSMQ at its heart. There are (currently) 3 separate VB solutions each of which send and receive message to a queue.
I tried using the XMLMessageFormatter and ran into problems with that, plus this is a high performance, time critical app and I understand that XMLMessaegFormatter has a high overhead, so I've switched over to using BinaryMessageFormatter for the messages.
I've established a class (clsTMessage) which provides the structure for the message data and resides in its own .vb file attached to the solution. I realize that the downside of using Binaryformatter is that the exact same class (down to version and all) has to encode and decode the messages and indeed I'm seeing that problem.
So I figured, no problem, I'd just copy clsTmessage.vb to each solution, but that doesn't quite do the trick as the messages encodes with the namespace of the host assemby and therefore the next solution to pick up the message is technically looking for a different class to decode it.
In this example, for instance, you can see that TelemanusWorkbench Version 1.0.0.0 encoded the message using TelemanusWorkbench.clsTMessage.
00 01 00 00 00 FF FF FF .....ÿÿÿ
FF 01 00 00 00 00 00 00 ÿ.......
00 0C 02 00 00 00 49 54 ......IT
65 6C 65 6D 61 6E 75 73 elemanus
57 6F 72 6B 62 65 6E 63 Workbenc
68 2C 20 56 65 72 73 69 h, Versi
6F 6E 3D 31 2E 30 2E 30 on=1.0.0
2E 30 2C 20 43 75 6C 74 .0, Cult
75 72 65 3D 6E 65 75 74 ure=neut
72 61 6C 2C 20 50 75 62 ral, Pub
6C 69 63 4B 65 79 54 6F licKeyTo
6B 65 6E 3D 6E 75 6C 6C ken=null
05 01 00 00 00 1E 54 65 ......Te
6C 65 6D 61 6E 75 73 57 lemanusW
6F 72 6B 62 65 6E 63 68 orkbench
2E 63 6C 73 54 4D 65 73 .clsTMes
73 61 67 65 09 00 00 00 sage....
0E 6E 65 77 4D 65 73 73 .newMess
61 67 65 54 79 70 65 12 ageType.
6E 65 77 50 72 6F 74 6F newProto
63 6F 6C 56 65 72 73 69 colVersi
6F 6E 0D 6E 65 77 49 64 on.newId
65 6E 74 69 66 69 65 72 entifier
0B 6E 65 77 53 6F 75 72 .newSour
63 65 49 50 0D 6E 65 77 ceIP.new
53 6F 75 72 63 65 50 6F SourcePo
72 74 10 6E 65 77 44 65 rt.newDe
73 74 69 6E 61 74 69 6F stinatio
6E 49 50 12 6E 65 77 44 nIP.newD
65 73 74 69 6E 61 74 69 estinati
6F 6E 50 6F 72 74 0C 6E onPort.n
65 77 54 69 6D 65 73 74 ewTimest
61 6D 70 0E 6E 65 77 4D amp.newM
65 73 73 61 67 65 42 6F essageBo
64 79 01 01 01 01 01 01 dy......
01 00 01 0D 02 00 00 00 ........
06 03 00 00 00 03 44 46 ......DF
58 06 04 00 00 00 01 30 X......0
06 05 00 00 00 0C 30 30 ......00
30 30 30 30 30 30 30 30 00000000
30 30 06 06 00 00 00 07 00......
30 2E 30 2E 30 2E 30 06 0.0.0.0.
07 00 00 00 01 30 06 08 .....0..
00 00 00 0B 31 39 32 2E ....192.
31 36 38 2E 31 2E 31 06 168.1.1.
09 00 00 00 04 35 30 30 .....500
30 20 46 FE 12 F9 32 CF 0 Fþ.ù2Ï
88 06 0A 00 00 00 49 70 .....Ip
2C 31 2C 31 32 33 34 35 ,1,12345
36 37 38 39 30 31 32 33 67890123
34 35 36 37 38 39 2C 31 456789,1
32 33 34 35 36 37 38 39 23456789
30 31 32 33 34 35 2C 31 012345,1
2C 69 6E 74 65 72 6E 65 ,interne
74 2C 75 73 65 72 6E 61 t,userna
6D 65 2C 70 61 73 73 77 me,passw
6F 72 64 2C 30 2C 33 30 ord,0,30
0B .
When I pick up the message from another solution/project within the app, it fails to parse the message even though it has an identical copy of clsTMessage it's in namespace TelemanusListener.clsTMessage.
Given that it's generically a bad idea to have multiple copies of the class in different parts of the app anyway, what's the reccomended way to do this ? I've read what MSDN has to say bout this, but it's very thin on how to actually implement it.
Hope I've explained that well enought, if not please ask for more info.
Duncan
Yes. One class library with a public message type needs to be referenced from the two projects.
Bit of warning about automatic properties - don't use them within classes that need to be serialised/deserialised. Each time a class type is compiled into an assembly, the compiler creates a randomly named backing field for each automatic property. This can cause serialisation problems when you deploy the one/same class library compiled at different times with different projects.