Snort logs in OSSIM show hex in payload but I want only the text to be there. Is there a config I can change in Snort? - snort

I am new to snort and I am testing things out with OSSIM. I've installed snort and using rsyslog I am getting snort alerts.log to OSSIM. But the thing is payloads of events in OSSIM show as
length = 219
000 : 31 31 2F 32 35 2F 31 39 2D 31 30 3A 30 34 3A 32 11/25/19-10:04:2
010 : 39 2E 37 38 30 31 32 34 20 20 5B 2A 2A 5D 20 5B 9.780124 [**] [
020 : 31 32 30 3A 31 38 3A 33 5D 20 28 68 74 74 70 5F 120:18:3] (http_
030 : 69 6E 73 70 65 63 74 29 20 50 52 4F 54 4F 43 4F inspect) PROTOCO
040 : 4C 2D 4F 54 48 45 52 20 48 54 54 50 20 73 65 72 L-OTHER HTTP ser
050 : 76 65 72 20 72 65 73 70 6F 6E 73 65 20 62 65 66 ver response bef
060 : 6F 72 65 20 63 6C 69 65 6E 74 20 72 65 71 75 65 ore client reque
070 : 73 74 20 20 5B 2A 2A 5D 20 5B 43 6C 61 73 73 69 st [**] [Classi
080 : 66 69 63 61 74 69 6F 6E 3A 20 55 6E 6B 6E 6F 77 fication: Unknow
090 : 6E 20 54 72 61 66 66 69 63 5D 20 5B 50 72 69 6F n Traffic] [Prio
0a0 : 72 69 74 79 3A 20 33 5D 20 7B 54 43 50 7D 20 31 rity: 3] {TCP} 1
0b0 : 39 32 2E 31 36 38 2E 30 2E 31 36 38 3A 38 30 38 92.168.0.168:808
0c0 : 30 20 2D 3E 20 31 39 32 2E 31 36 38 2E 30 2E 31 0 -> 192.168.0.1
0d0 : 32 32 3A 33 39 31 37 30 22 20 0A 22:39170" .
But I want it to be like
11/25/19-10:04:29.780124 [**] [120:18:3] (http_inspect) PROTOCOL-OTHER HTTP server response before client request [**] [Classification: Unknown Traffic] [Priority: 3] {TCP} 192.168.0.168:8080 -> 192.168.0.122:39170" .
Is there any config in snort I can change to make this happen?
Thanks in advance for any help!

Related

Failed to run BookInfo example behind proxy server, failed calling webhook "pilot.validation.istio.io"

Issue also posted at:
https://github.com/istio/istio/issues/21195
https://www.reddit.com/r/istio/comments/f57v2a/help_failed_to_run_bookinfo_example_behind_proxy/
Problem:
Following the bookinfo example, when trying to apply bookinfo-gateway, I get the following error:
Error from server (InternalError): error when creating "samples/bookinfo/networking/bookinfo-gateway.yaml": Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
Output:
$ kubectl --v=9 apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
I0216 18:01:08.548290 4904 loader.go:375] Config loaded from file: /home/user/.kube/config
I0216 18:01:08.550426 4904 round_trippers.go:423] curl -k -v -XGET -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" -H "Accept: application/com.github.proto-openapi.spec.v2#v1.0+protobuf" 'https://ha-lb-ip:6443/openapi/v2?timeout=32s'
I0216 18:01:08.600310 4904 round_trippers.go:443] GET https://ha-lb-ip:6443/openapi/v2?timeout=32s 200 OK in 49 milliseconds
I0216 18:01:08.600348 4904 round_trippers.go:449] Response Headers:
I0216 18:01:08.600355 4904 round_trippers.go:452] Accept-Ranges: bytes
I0216 18:01:08.600361 4904 round_trippers.go:452] X-Varied-Accept: application/com.github.proto-openapi.spec.v2#v1.0+protobuf
I0216 18:01:08.600366 4904 round_trippers.go:452] Content-Type: application/octet-stream
I0216 18:01:08.600371 4904 round_trippers.go:452] Etag: "DCA49D599C62F0A8DDF840BBF0F4DB11A2B0C9805F7F6CEB19F163F61CA7D40F9E7A3607B007A74CCD6DBA6565BE6E6E3085528F7FD18EDAE99BABE9702D8700"
I0216 18:01:08.600378 4904 round_trippers.go:452] Last-Modified: Sun, 16 Feb 2020 11:54:46 GMT
I0216 18:01:08.600430 4904 round_trippers.go:452] Vary: Accept-Encoding
I0216 18:01:08.600435 4904 round_trippers.go:452] Vary: Accept
I0216 18:01:08.600439 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:08 GMT
I0216 18:01:08.817775 4904 request.go:1015] Response Body:
00000000 0a 03 32 2e 30 12 15 0a 0a 4b 75 62 65 72 6e 65 |..2.0....Kuberne|
00000010 74 65 73 12 07 76 31 2e 31 37 2e 33 42 93 f7 a9 |tes..v1.17.3B...|
00000020 01 12 ae 27 0a 29 2f 61 70 69 2f 76 31 2f 77 61 |...'.)/api/v1/wa|
00000030 74 63 68 2f 6e 61 6d 65 73 70 61 63 65 73 2f 7b |tch/namespaces/{|
00000040 6e 61 6d 65 73 70 61 63 65 7d 2f 70 6f 64 73 12 |namespace}/pods.|
00000050 80 27 12 97 04 0a 07 63 6f 72 65 5f 76 31 1a 6f |.'.....core_v1.o|
00000060 77 61 74 63 68 20 69 6e 64 69 76 69 64 75 61 6c |watch individual|
00000070 20 63 68 61 6e 67 65 73 20 74 6f 20 61 20 6c 69 | changes to a li|
00000080 73 74 20 6f 66 20 50 6f 64 2e 20 64 65 70 72 65 |st of Pod. depre|
00000090 63 61 74 65 64 3a 20 75 73 65 20 74 68 65 20 27 |cated: use the '|
000000a0 77 61 74 63 68 27 20 70 61 72 61 6d 65 74 65 72 |watch' parameter|
000000b0 20 77 69 74 68 20 61 20 6c 69 73 74 20 6f 70 65 | with a list ope|
000000c0 72 61 74 69 6f 6e 20 69 6e 73 74 65 61 64 2e 2a |ration instead.*|
000000d0 1c 77 61 74 63 68 43 6f 72 65 56 31 4e 61 6d 65 |.watchCoreV1Name|
000000e0 73 70 61 63 65 64 50 6f 64 4c 69 73 74 32 10 61 |spacedPodList2.a|
000000f0 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 32 |pplication/json2|
00000100 10 61 70 70 6c 69 63 61 74 69 6f 6e 2f 79 61 6d |.application/yam|
00000110 6c 32 23 61 70 70 6c 69 63 61 74 69 6f 6e 2f 76 |l2#application/v|
00000120 6e 64 2e 6b 75 62 65 72 6e 65 74 65 73 2e 70 72 |nd.kubernetes.pr|
00000130 6f 74 6f 62 75 66 32 1d 61 70 70 6c 69 63 61 74 |otobuf2.applicat|
00000140 69 6f 6e 2f 6a 73 6f 6e 3b 73 74 72 65 61 6d 3d |ion/json;stream=|
00000150 77 61 74 63 68 32 30 61 70 70 6c 69 63 61 74 69 |watch20applicati|
00000160 6f 6e 2f 76 6e 64 2e 6b 75 62 65 72 6e 65 74 65 |on/vnd.kubernete|
00000170 73 2e 70 72 6f 74 6f 62 75 66 3b 73 74 72 65 61 |s.protobuf;strea|
00000180 6d 3d 77 61 74 63 68 3a 03 2a 2f 2a 4a 6b 0a 50 |m=watch:.*/*Jk.P|
00000190 0a 03 32 30 30 12 49 0a 47 0a 02 4f 4b 12 41 0a |..200.I.G..OK.A.|
000001a0 3f 0a 3d 23 2f 64 65 66 69 6e 69 74 69 6f 6e 73 |?.=#/definitions|
000001b0 2f 69 6f 2e 6b 38 73 2e 61 70 69 6d 61 63 68 69 |/io.k8s.apimachi|
000001c0 6e 65 72 79 2e 70 6b 67 2e 61 70 69 73 2e 6d 65 |nery.pkg.apis.me|
000001d0 74 61 2e 76 31 2e 57 61 74 63 68 45 76 65 6e 74 |ta.v1.WatchEvent|
000001e0 0a 17 0a 03 34 30 31 12 10 0a 0e 0a 0c 55 6e 61 |....401......Una|
000001f0 75 74 68 6f 72 69 7a 65 64 52 05 68 74 74 70 73 |uthorizedR.https|
00000200 6a 23 0a 13 78 2d 6b 75 62 65 72 6e 65 74 65 73 |j#..x-kubernetes|
00000210 2d 61 63 74 69 6f 6e 12 0c 12 0a 77 61 74 63 68 |-action....watch|
00000220 6c 69 73 74 0a 6a 45 0a 1f 78 2d 6b 75 62 65 72 |list.jE..x-kuber|
00000230 6e 65 74 65 73 2d 67 72 6f 75 70 2d 76 65 72 73 |netes-group-vers|
00000240 69 6f 6e 2d 6b 69 6e 64 12 22 12 20 6b 69 6e 64 |ion-kind.". kind|
00000250 3a 20 50 6f 64 0a 76 65 72 73 69 6f 6e 3a 20 76 |: Pod.version: v|
00000260 31 0a 67 72 6f 75 70 3a 20 22 22 0a 4a 82 04 0a |1.group: "".J...|
00000270 ff 03 12 fc 03 1a f9 03 12 05 71 75 65 72 79 1a |..........query.|
00000280 ce 03 61 6c 6c 6f 77 57 61 74 63 68 42 6f 6f 6b |..allowWatchBook|
00000290 6d 61 72 6b 73 20 72 65 71 75 65 73 74 73 20 77 |marks requests w|
000002a0 61 74 63 68 20 65 76 65 6e 74 73 20 77 69 74 68 |atch events with|
000002b0 20 74 79 70 65 20 22 42 4f 4f 4b 4d 41 52 4b 22 | type "BOOKMARK"|
000002c0 2e 20 53 65 72 76 65 72 73 20 74 68 61 74 20 64 |. Servers that d|
000002d0 6f 20 6e 6f 74 20 69 6d 70 6c 65 6d 65 6e 74 20 |o not implement |
000002e0 62 6f 6f 6b 6d 61 72 6b 73 20 6d 61 79 20 69 67 |bookmarks may ig|
000002f0 6e 6f 72 65 20 74 68 69 73 20 66 6c 61 67 20 61 |nore this flag a|
00000300 6e 64 20 62 6f 6f 6b 6d 61 72 6b 73 20 61 72 65 |nd bookmarks are|
00000310 20 73 65 6e 74 20 61 74 20 74 68 65 20 73 65 72 | sent at the ser|
00000320 76 65 72 27 73 20 64 69 73 63 72 65 74 69 6f 6e |ver's discretion|
00000330 2e 20 43 6c 69 65 6e 74 73 20 73 68 6f 75 6c 64 |. Clients should|
00000340 20 6e 6f 74 20 61 73 73 75 6d 65 20 62 6f 6f 6b | not assume book|
00000350 6d 61 72 6b 73 20 61 72 65 20 72 65 74 75 72 6e |marks are return|
00000360 65 64 20 61 74 20 61 6e 79 20 73 70 65 63 69 66 |ed at any specif|
00000370 69 63 20 69 6e 74 65 72 76 61 6c 2c 20 6e 6f 72 |ic interval, nor|
00000380 20 6d 61 79 20 74 68 65 79 20 61 73 73 75 6d 65 | may they assume|
00000390 20 74 68 65 20 73 65 72 76 65 72 20 77 69 6c 6c | the server will|
000003a0 20 73 65 6e 64 20 61 6e 79 20 42 4f 4f 4b 4d 41 | send any BOOKMA|
000003b0 52 4b 20 65 76 65 6e 74 20 64 75 72 69 6e 67 20 |RK event during |
000003c0 61 20 73 65 73 73 69 6f 6e 2e 20 49 66 20 74 68 |a session. If th|
000003d0 69 73 20 69 73 20 6e 6f 74 20 61 20 77 61 74 63 |is is not a watc|
000003e0 68 2c 20 74 68 69 73 20 66 69 65 6c 64 20 69 73 |h, this field is|
000003f0 20 69 67 6e 6f 72 65 64 2e 20 49 66 20 74 68 65 | ignored. If the|
00000400 20 66 65 61 74 75 72 65 20 67 61 74 65 20 57 61 | feature gate Wa|
00000410 74 63 68 42 6f 6f 6b 6d 61 72 6b 73 20 69 73 20 |tchBookmarks is |
00000420 6e 6f 74 20 65 6e 61 62 6c 65 64 20 69 6e 20 61 |not enabled in a|
00000430 70 69 73 65 72 76 65 72 2c 20 74 68 69 73 20 66 |piserver, this f|
00000440 69 65 6c 64 20 69 73 20 69 67 6e 6f 72 65 64 2e |ield is ignored.|
00000450 22 13 61 6c 6c 6f 77 57 61 74 63 68 42 6f 6f 6b |".allowWatchBook|
00000460 6d 61 72 6b 73 32 07 62 6f 6f 6c 65 61 6e a0 01 |marks2.boolean..|
00000470 01 4a ef 09 0a ec 09 12 e9 09 1a e6 09 12 05 71 |.J.............q|
00000480 75 65 72 79 1a c7 09 54 68 65 20 63 6f 6e 74 69 |uery...The conti|
00000490 6e 75 65 20 6f 70 74 69 6f 6e 20 73 68 6f 75 6c |nue option shoul|
000004a0 64 20 62 65 20 73 65 74 20 77 68 65 6e 20 72 65 |d be set when re|
000004b0 74 72 69 65 76 69 6e 67 20 6d 6f 72 65 20 72 65 |trieving more re|
000004c0 73 75 6c 74 73 20 66 72 6f 6d 20 74 68 65 20 73 |sults from the s|
000004d0 65 72 76 65 72 2e 20 53 69 6e 63 65 20 74 68 69 |erver. Since thi|
000004e0 73 20 76 61 6c 75 65 20 69 73 20 73 65 72 76 65 |s value is serve|
000004f0 72 20 64 65 66 69 6e 65 64 2c 20 63 6c 69 65 6e |r defined, clien|
00000500 74 73 20 6d 61 79 20 6f 6e 6c 79 20 75 73 65 20 |ts may only use |
00000510 74 68 65 20 63 6f 6e 74 69 6e 75 65 20 76 61 6c |the continue val|
00000520 75 65 20 66 72 6f 6d 20 61 20 70 72 65 76 69 6f |ue from a previo|
00000530 75 73 20 71 75 65 72 79 20 72 65 73 75 6c 74 20 |us query result |
00000540 77 69 74 68 20 69 64 65 6e 74 69 63 61 6c 20 71 |with identical q|
00000550 75 65 72 79 20 70 61 72 61 6d 65 74 65 72 73 20 |uery parameters |
00000560 28 65 78 63 65 70 74 20 66 6f 72 20 74 68 65 20 |(except for the |
00000570 76 61 6c 75 65 20 6f 66 20 63 6f 6e 74 69 6e 75 |value of continu|
00000580 65 29 20 61 6e 64 20 74 68 65 20 73 65 72 76 65 |e) and the serve|
00000590 72 20 6d 61 79 20 72 65 6a 65 63 74 20 61 20 63 |r may reject a c|
000005a0 6f 6e 74 69 6e 75 65 20 76 61 6c 75 65 20 69 74 |ontinue value it|
000005b0 20 64 6f 65 73 20 6e 6f 74 20 72 65 63 6f 67 6e | does not recogn|
000005c0 69 7a 65 2e 20 49 66 20 74 68 65 20 73 70 65 63 |ize. If the spec|
000005d0 69 66 69 65 64 20 63 6f 6e 74 69 6e 75 65 20 76 |ified continue v|
000005e0 61 6c 75 65 20 69 73 20 6e 6f 20 6c 6f 6e 67 65 |alue is no longe|
000005f0 72 20 76 61 6c 69 64 20 77 68 65 74 68 65 72 20 |r valid whether |
00000600 64 75 65 20 74 6f 20 65 78 70 69 72 61 74 69 6f |due to expiratio|
00000610 6e 20 28 67 65 6e 65 72 61 6c 6c 79 20 66 69 76 |n (generally fiv|
00000620 65 20 74 6f 20 66 69 66 74 65 65 6e 20 6d 69 6e |e to fifteen min|
00000630 75 74 65 73 29 20 6f 72 20 61 20 63 6f 6e 66 69 |utes) or a confi|
00000640 67 75 72 61 74 69 6f 6e 20 63 68 61 6e 67 65 20 |guration change |
00000650 6f 6e 20 74 68 65 20 73 65 72 76 65 72 2c 20 74 |on the server, t|
00000660 68 65 20 73 65 72 76 65 72 20 77 69 6c 6c 20 72 |he server will r|
00000670 65 73 70 6f 6e 64 20 77 69 74 68 20 61 20 34 31 |espond with a 41|
00000680 30 20 52 65 73 6f 75 72 63 65 45 78 70 69 72 65 |0 ResourceExpire|
00000690 64 20 65 72 72 6f 72 20 74 6f 67 65 74 68 65 72 |d error together|
000006a0 20 77 69 74 68 20 61 20 63 6f 6e 74 69 6e 75 65 | with a continue|
000006b0 20 74 6f 6b 65 6e 2e 20 49 66 20 74 68 65 20 63 | token. If the c|
000006c0 6c 69 65 6e 74 20 6e 65 65 64 73 20 61 20 63 6f |lient needs a co|
000006d0 6e 73 69 73 74 65 6e 74 20 6c 69 73 74 2c 20 69 |nsistent list, i|
000006e0 74 20 6d 75 73 74 20 72 65 73 74 61 72 74 20 74 |t must restart t|
000006f0 68 65 69 72 20 6c 69 73 74 20 77 69 74 68 6f 75 |heir list withou|
00000700 74 20 74 68 65 20 63 6f 6e 74 69 6e 75 65 20 66 |t the continue f|
00000710 69 65 6c 64 2e 20 4f 74 68 65 72 77 69 73 65 2c |ield. Otherwise,|
00000720 20 74 68 65 20 63 6c 69 65 6e 74 20 6d 61 79 20 | the client may |
00000730 73 65 6e 64 20 61 6e 6f 74 68 65 72 20 6c 69 73 |send another lis|
00000740 74 20 72 65 71 75 65 73 74 20 77 69 74 68 20 74 |t request with t|
00000750 68 65 20 74 6f 6b 65 6e 20 72 65 63 65 69 76 65 |he token receive|
00000760 64 20 77 69 74 68 20 74 68 65 20 34 31 30 20 65 |d with the 410 e|
00000770 72 72 6f 72 2c 20 74 68 65 20 73 65 72 76 65 72 |rror, the server|
00000780 20 77 69 6c 6c 20 72 65 73 70 6f 6e 64 20 77 69 | will respond wi|
00000790 74 68 20 61 20 6c 69 73 74 20 73 74 61 72 74 69 |th a list starti|
000007a0 6e 67 20 66 72 6f 6d 20 74 68 65 20 6e 65 78 74 |ng from the next|
000007b0 20 6b 65 79 2c 20 62 75 74 20 66 72 6f 6d 20 74 | key, but from t|
000007c0 68 65 20 6c 61 74 65 73 74 20 73 6e 61 70 73 68 |he latest snapsh|
000007d0 6f 74 2c 20 77 68 69 63 68 20 69 73 20 69 6e 63 |ot, which is inc|
000007e0 6f 6e 73 69 73 74 65 6e 74 20 66 72 6f 6d 20 74 |onsistent from t|
000007f0 68 65 20 70 72 65 76 69 6f 75 73 20 6c 69 73 74 |he previous list|
00000800 20 72 65 73 75 6c 74 73 20 2d 20 6f 62 6a 65 63 | results - objec|
00000810 74 73 20 74 68 61 74 20 61 72 65 20 63 [truncated 17345571 chars]
I0216 18:01:08.944681 4904 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways/bookinfo-gateway'
I0216 18:01:08.988784 4904 round_trippers.go:443] GET https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways/bookinfo-gateway 404 Not Found in 44 milliseconds
I0216 18:01:08.988834 4904 round_trippers.go:449] Response Headers:
I0216 18:01:08.988840 4904 round_trippers.go:452] Content-Length: 258
I0216 18:01:08.988844 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:08 GMT
I0216 18:01:08.988848 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:01:08.988896 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"gateways.networking.istio.io \"bookinfo-gateway\" not found","reason":"NotFound","details":{"name":"bookinfo-gateway","group":"networking.istio.io","kind":"gateways"},"code":404}
I0216 18:01:08.989774 4904 request.go:1017] Request Body: {"apiVersion":"networking.istio.io/v1alpha3","kind":"Gateway","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"networking.istio.io/v1alpha3\",\"kind\":\"Gateway\",\"metadata\":{\"annotations\":{},\"name\":\"bookinfo-gateway\",\"namespace\":\"default\"},\"spec\":{\"selector\":{\"istio\":\"ingressgateway\"},\"servers\":[{\"hosts\":[\"*\"],\"port\":{\"name\":\"http\",\"number\":80,\"protocol\":\"HTTP\"}}]}}\n"},"name":"bookinfo-gateway","namespace":"default"},"spec":{"selector":{"istio":"ingressgateway"},"servers":[{"hosts":["*"],"port":{"name":"http","number":80,"protocol":"HTTP"}}]}}
I0216 18:01:08.989839 4904 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways'
I0216 18:01:38.996165 4904 round_trippers.go:443] POST https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways 500 Internal Server Error in 30006 milliseconds
I0216 18:01:38.996302 4904 round_trippers.go:449] Response Headers:
I0216 18:01:38.996315 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:01:38.996320 4904 round_trippers.go:452] Content-Length: 481
I0216 18:01:38.996349 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:38 GMT
I0216 18:01:38.996563 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded","reason":"InternalError","details":{"causes":[{"message":"failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"}]},"code":500}
I0216 18:01:38.999383 4904 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/bookinfo'
I0216 18:01:39.042269 4904 round_trippers.go:443] GET https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/bookinfo 404 Not Found in 42 milliseconds
I0216 18:01:39.042304 4904 round_trippers.go:449] Response Headers:
I0216 18:01:39.042310 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:01:39.042315 4904 round_trippers.go:452] Content-Length: 256
I0216 18:01:39.042319 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:39 GMT
I0216 18:01:39.042352 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"virtualservices.networking.istio.io \"bookinfo\" not found","reason":"NotFound","details":{"name":"bookinfo","group":"networking.istio.io","kind":"virtualservices"},"code":404}
I0216 18:01:39.043083 4904 request.go:1017] Request Body: {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"networking.istio.io/v1alpha3\",\"kind\":\"VirtualService\",\"metadata\":{\"annotations\":{},\"name\":\"bookinfo\",\"namespace\":\"default\"},\"spec\":{\"gateways\":[\"bookinfo-gateway\"],\"hosts\":[\"*\"],\"http\":[{\"match\":[{\"uri\":{\"exact\":\"/productpage\"}},{\"uri\":{\"prefix\":\"/static\"}},{\"uri\":{\"exact\":\"/login\"}},{\"uri\":{\"exact\":\"/logout\"}},{\"uri\":{\"prefix\":\"/api/v1/products\"}}],\"route\":[{\"destination\":{\"host\":\"productpage\",\"port\":{\"number\":9080}}}]}]}}\n"},"name":"bookinfo","namespace":"default"},"spec":{"gateways":["bookinfo-gateway"],"hosts":["*"],"http":[{"match":[{"uri":{"exact":"/productpage"}},{"uri":{"prefix":"/static"}},{"uri":{"exact":"/login"}},{"uri":{"exact":"/logout"}},{"uri":{"prefix":"/api/v1/products"}}],"route":[{"destination":{"host":"productpage","port":{"number":9080}}}]}]}}
I0216 18:01:39.043172 4904 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices'
I0216 18:02:09.049842 4904 round_trippers.go:443] POST https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices 500 Internal Server Error in 30006 milliseconds
I0216 18:02:09.050031 4904 round_trippers.go:449] Response Headers:
I0216 18:02:09.050043 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:02:09.050052 4904 round_trippers.go:452] Content-Length: 481
I0216 18:02:09.050059 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:02:09 GMT
I0216 18:02:09.050249 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded","reason":"InternalError","details":{"causes":[{"message":"failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"}]},"code":500}
I0216 18:02:09.051955 4904 helpers.go:203] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "error when creating \"samples/bookinfo/networking/bookinfo-gateway.yaml\": Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded",
"reason": "InternalError",
"details": {
"causes": [
{
"message": "failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"
}
]
},
"code": 500
}]
I0216 18:02:09.052104 4904 helpers.go:203] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "error when creating \"samples/bookinfo/networking/bookinfo-gateway.yaml\": Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded",
"reason": "InternalError",
"details": {
"causes": [
{
"message": "failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"
}
]
},
"code": 500
}]
F0216 18:02:09.052210 4904 helpers.go:114] Error from server (InternalError): error when creating "samples/bookinfo/networking/bookinfo-gateway.yaml": Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
Error from server (InternalError): error when creating "samples/bookinfo/networking/bookinfo-gateway.yaml": Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
Related Information:
Docker version 19.03.5
Kubernetes version 1.17.3
Istio version 1.4.4
Kubernetes clusters are run behind company's proxy server. CNI using kube-flannel
The docker.service.d config has proxy configured at /etc/systemd/system/docker.service.d/proxy.conf.
NO_PROXY/no_proxy is set to localhost,127.0.0.1,::1,.grameenphone.com,10.10.18.188,10.10.23.57,10.10.23.58,10.10.23.59,10.10.23.60,10.10.23.61,kubernetes.default,.validation.istio.io,.istio.io,.istio-system.svc,.svc,.istio-system,.svc.cluster.local,.cluster.local,10.244.0.0/16.
Installed ISTIO using istioctl, yaml given below:
apiVersion: install.istio.io/v1alpha2
kind: IstioControlPlane
spec:
defaultNamespace: istio-system
cni:
enabled: true
gateways:
components:
egressGateway:
enabled: false
ingressGateway:
enabled: true
enabled: true
values:
cni:
excludeNamespaces:
- istio-system
- kube-system
gateways:
istio-ingressgateway:
type: NodePort
global:
configValidation: true
grafana:
enabled: true
kiali:
enabled: true
tracing:
enabled: true
Istio install was successful, verified using gisitoctl verify-install.
Tried modifying /etc/kubernetes/manifests/kube-apiserver.yaml by adding env: section to the container, but still fails to create bookinfo-gateway.
env:
- name: http_proxy
value: http://10.10.20.107:3828
- name: https_proxy
value: http://10.10.20.107:3828
- name: no_proxy
value: localhost,127.0.0.1,::1,.grameenphone.com,10.10.18.188,10.10.23.57,10.10.23.58,10.10.23.59,10.10.23.60,10.10.23.61,kubernetes.default,.validation.istio.io,.istio.io,.istio-system.svc,.svc,.istio-system,.svc.cluster.local,.cluster.local,10.244.0.0/16,10.96.0.0/12
For sidecar injection, I'm following the manual procedure.
All Isito pods are up & running.
Log from kube-apiserver:
I0217 06:36:08.719672 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0217 06:37:19.151894 1 trace.go:116] Trace[2116455659]: "Call validating webhook" configuration:istio-galley,webhook:pilot.validation.istio.io,resource:networking.istio.io/v1alpha3, Resource=gateways,subresource:,operation:CREATE,UID:de57f49e-fd19-44ea-99d7-414dfec0981f (started: 2020-02-17 06:36:49.150893916 +0000 UTC m=+6172.698922922) (total time: 30.000884409s):
Trace[2116455659]: [30.000884409s] [30.000884409s] END
W0217 06:37:19.151963 1 dispatcher.go:133] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
I0217 06:37:19.152376 1 trace.go:116] Trace[2129026394]: "Create" url:/apis/networking.istio.io/v1alpha3/namespaces/default/gateways,user-agent:kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960,client:10.10.18.188 (started: 2020-02-17 06:36:49.149255324 +0000 UTC m=+6172.697284349) (total time: 30.003096239s):
Trace[2129026394]: [30.003096239s] [30.002534955s] END
I0217 06:37:49.167492 1 trace.go:116] Trace[1063136940]: "Call validating webhook" configuration:istio-galley,webhook:pilot.validation.istio.io,resource:networking.istio.io/v1alpha3, Resource=virtualservices,subresource:,operation:CREATE,UID:2dd4b3e7-8333-4c1d-8222-dd53f8ce2db4 (started: 2020-02-17 06:37:19.166772885 +0000 UTC m=+6202.714801862) (total time: 30.000661809s):
Trace[1063136940]: [30.000661809s] [30.000661809s] END
W0217 06:37:49.167530 1 dispatcher.go:133] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
I0217 06:37:49.167996 1 trace.go:116] Trace[639287810]: "Create" url:/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices,user-agent:kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960,client:10.10.18.188 (started: 2020-02-17 06:37:19.165649222 +0000 UTC m=+6202.713678242) (total time: 30.002325938s):
Trace[639287810]: [30.002325938s] [30.001822425s] END
I googled/looked into other related issues & tried their solutions, but did not worked.
If there is any additional information required, please do let me know.

Seems that galley stopped or doesn't answer.
Galley provides configuration management services for Istio.
So, pls check these two steps:
1. Is galley running?
galley probe
2. If it runs, try the WA
Due to this known issue istio/istio#17162, seems that your problem is related to api-server proxy settings.
And there is a workaround
Try to use --set global.configValidation=false when installing istio
See also manual on istio validation

How can I decrypt this so I can view it

Hex (148 bytes): 3c 68 74 6d 6c 3e 3c 68 65 61 64 3e 3c 2f 68 65 61 64 3e 3c 62 6f 64 79 3e 3c 69 6d 67 20 73 72 63 3d 22 64 61 74 61 3a 69 6d 61 67 65 2f 6a 70 65 67 3b 62 61 73 65 36 34 2c 2f 39 6a 2f 34 41 41 51 53 6b 5a 4a 52 67 41 42 41 51 45 41 57 67 42 61 41 41 44 2f 34 67 78 59 53 55 4e 44 58 31 42 53 54 30 5a 4a 54 45 55 41 41 51 45 41 41 41 78 49 54 47 6c 75 62 77 49 51 41 41 42 2e 2e 2e ... �����ExifII*bj(1r2�i����
'��
'Adobe Photoshop CS4 Macintosh2012-02-06T20:39:30�0220�������Ducky<��-�http://ns.adobe.com/xap/1.0/ ��Adobed����
��^��� !1AQq"a����2BR#�br3�႒�$%��CS��D5!1AQq�a��2B�����"R�br�3��?��ݣW���DB�{6�xֵ�,�=����A���NY>C%�nE^j?��
��;\��So��k���9̇�X���I%4�|�1����m�vC���:G�Y��;}�R�D"���0p1���3�,�^}NVn���)��G�W7y��^�u�Գ��ywҰM�y�y3^�m�,Z�B {�x������d1E;�s���6q�e�~��{ۺ"=��7-�r>�e��t.I�u$���v�SG�qXwҒ�u��M������j��" �<�tW��ą�|�c��ꮖ�����:.��0DO:����RFX��E��4g�e�>Xv״����o�L��sw���c�'���i}I��IYaFz�(�������C�Zu(8�Xky�?����b�\j���#�c��.#s����G��g_i4�K��n�+)^a�..�F]VXQ��B�%�O8��e g�T����HѸ
�T4P���K��j|T�)���Lv��wO�$uq���J��L�l���c��h��2AsO�Ϲs�晝#��u��{�q�t�xN���ɪ�pT############A&z�6�ޠ�����C�#��2\&�xK�����{]ރ��<��"��Ǥ�uO����9�6�x��-�={޶7�ft���hg��sC�Ai��VlĪ������?W��C�m�����63�IN���J7��3q��:G(x>��?�ɥ~
���Ϲ-w ########A��L��u-��qlR;��S�d�_�-������C���˖i?4}��!����kS�t��L�~l&���Q)|��5�����#ݭ�=�U����z3/�6�厫fc\xV��˭�]
��
�
� �F�8~���a��K���2����TI��2��E����]t��Mg�������m�N�*Z�D�������rA0k�70�TD�;K.�����̍�z��&����id�9�Yg��ִ����]c��mc<�j�MSz�Йg�<�44<
G*tre�RQ� 8�^jH#��.�s0b�!� h�7���>�g�-o,M��Z�ֆ�
��|�Ӭ�*��Lc��.{��*I<������FY��_��Z>��t�{"��[8�|��C͗��e}|��F#����kM�&��>��ycI�6=�2V�v��|]�=o_�[O�s���.���V�{LMq�,b�v7��v�ʕ�<]�y�WM"l=�������4��oҶ��J��P�N.qD��Y��d�Qe;��=����N��Ԟ�8���4���_Ok/ַ,ȳ����x�o(5����������j�㥸K.ߨ�/�Y�ɓ������VgX##A�Ϩ�7>�/m�m��:{7}����ߪW'5<��κ��p�{5�<'�G0�4Ddisi���ݎ��*km'V}�iŒ���O]��c�l���+�+y��Q��5�a��-1�] ��H��Ԡgb��oX��J��E��I��,�#:������k'l׎�J�,�m���������(�����#����AA4c�% ��U�Z:sfw~A�K1����[����M�и�Š���}��)�%����q�io�{�c�i��ۿ
�V2�e��:�[�9�S�w?���o�����[D�Lr�bc��84�ujI��+%���"vH{sa�J�O��q��%y�md�ZO�*��Y���1�ʣ!hPP��%��#�$ Ɨ-x(�k.z�6W|���h��
�\�a$y���Y��V�N���v�9$<��)�zf?�rdW�n��4���4�vB��m�
�K�i�ٲ���~�#j�.6�Z֍hA�=���na�F�7�_rn�҂�WAV�PP\�5�<���1�o�$#B�x�{�-�df!��,�$��qAv<~fWn�尷�F�������˧wk�#�6+X"��h�
�ATA�08iM�|ET�;���:�J4�8(��/RX�ַQH̾k=��U�zkI������#Og&�EF��+��ЈU�-pp�
G��]7��-���bv���ǟc�Gs�f��"_J�f���x���+6D�_u�5�t��L��b�[�\�N������,�<��sz��6���+�������dw�vN��� ��$�$��4�a��h�[9KK|v�.�et�7���.�8�\qѽ��I��i�~���Gs�}C���G�|�B+Z�> �=ȶ,�K,��pm�-�I�����6o>ں�=��c�Fn>Pw�'�{˨�����(A�(����^zצ��tI:i��_��1:����p�UksmŬ�H�k���e4tV����N�:�M�h#MP��z��ܑ-����q��2��PI;�
(CV$����Y��1�q/�)n�ᴶm-�c)�#D�q��hЃ�{d;Z<�PlV�vҠ �A�;���Pcϔ��Ѻ�6�m�M{�w��'$X��;���<�k�s�]��ȣ��͝���: ʏ� s�˩��}(C�C�4A���Y[�C��#AT!k�w��p>*�k�W���v�5Ӵp�� �}e�������i]<M&��Lt������iq7����ԁ����s2�KL<S��Y�^�Z�########A)z=�Dg������L�ַ.���-q��|�y[{l�^���N0���d���Jzc��1���
�=�N�aTH�����P0�5 d���M�[�5��?Wu+�w�,���.n���Ɵ�+�sG��%��fWu�G�\f4�6ٜZKOC��u�9��[�Oc�ɓ'����w��<R�w���~�Xt�5nl�i�E��WO1�G�a͚)�+�u:�q�<��N/��no�滺��q;̒��R\�RJ���:��6���fV* ������V�CS�����;U���~׵�z4���_�>���G����ʭ�uzX�s[�����Jb:��K�� ~�OhMd�I��;����'�4����[V��ꤴ�����0�)�F�͗#�$���E�Av�Eh�7�^���!�\H5;�R7v��~?�imj)P*Pd�Fm �-�C�AguI�y���Ν��K��EG'��AV�ݹ�R��⍏�v���t�y�l�>�=���Mnn�ioA��GrL��^��&����y��C�Q׳O��X���R��6�J����5�P
#�P*��T
� �-q���t�J��*P�ӡ~�� |��cG
9� p��C�2C�x��ɏZ}*�R�������B}%u�9-��G��?�h��%澥��H��.�j�������������oS3�?umj�6$���o/�5���2un��,�ھ�ǩ��1:Ӷ'��}0������Ka�t�7A�법����&X�6������׏sI���D��d�a��'��E���<aS��~�h��^�X��=�Ve��o� �i-�b��;�w�s��9���9�&2[�aKS���a����{���㧺~_͵���;W�.��*x�ro�_j�<5���ϥb�>o������[������\H�a��m���Q3���V5��o[zᏳk��p��Z���g�4�s��Z�wQ�{��+��]�g��r9+ܕ䗗�:{�N�$y�%h��β򗽭3kN�,e
�-�\�o�W�|Or�����p_-���e��җ9K����m������ ����J��ycGж[XJ)���2�Zh��Fi��fmtm]R+��F��i�>h=�A�s_��W�$
t.B[C�:Ӛ
1�Iv�M���H��Y�Or'pv�N���M�8;Zº��<3hv�֤�b#Ѯ�֨.7q���~H(���[� ��_��ۏ�l&,��Ĭ%��wD���z��<��:����x��XL�5d#AP�ֱ����{u��)C�?��
%hs���yK)i�y��X�Mu�Ej���������]sa'�E3���oȭ\�֞T�9�����7L�2�뜖R��ٻ����F=�4��5�gFLx���9����
��}:#D�#��3x�ln���z}�LE}�Y_z����!�����ZG����Rs}(�4�ƽۤ������;��QbŚi��lz�]���5�N �
�gEߵ��I1����\����0�wU�S��oo�ZO�s���:.f�fnΟ�3~ �������-��}��y�p�s�vT�٘��i%'-{�6�[x���i��c�;W���}Ӝ#��#�����i
<�{m^S6�~�ծhp5TaYݘ�I�A��7�k�id��챡���]�Z{�p�y��rG���̠��b
��H��ݍ�
kY�Q�c�.O��-��Oen�ضW���U��Kϩ���3��ҭ�~�uE�Z-�s\��ɩc���iX�:����׍�m�CӞ��a���Iv�®m�ul���wrخ8�N�,4�R"��J-�!"lv��1���pWёG4��;t;��#D�h�hC�S���7p�=�.m�k����
8��a�j�̯��PWy4�%�b�&��}ćO.^��\�:'q�:�zvѻ�j3�߹�}����M��m;��A��i6�*(��ڂ��og�
�Q����H|/itN�m����$��<���9Fk4g�4!̽zɱ�N�HI8瘪���7��c~2�a��O�Vx"]�5
ꪀ��+T|F�ô����;Oz�.�#퍐�L��$q#�k�ٚci;CO�H�ޡ/��eśN������\�)�ӗ�Se\u�j�)���͜�{X��h'���ںN����,���J�\#######A#�g�ޟ���wSf�s|�-�8�t�~Ԏ.=��8/J�n�K��O��i�٤xqt=K��f���nl�w4��[I���[���u��i��z�܇�on��廻��\���ey���ĕ�3��/3kM�fgY��
�21�s����[��{�lK�#
b5��&��s����?����������A�Ӵ�#.��V8��|�Ï[Ε�|���]7T��-�<�8���i_i$��Yry��o��7�\��ͬm!76���O�)���ˏ����&u���v��?.?%��k�f�����8��W�E�p,�ᇘn��g�b�]�v˭���G�i���Æ��1�jZ+!�Y낕�����X������<��0����0�
~�hX��C}Vv����_�)J�t�� �>:����?h��<�l�c�k��ʠC��Q�T
q�AV� �N-51���[J4�p�>���4T��e���lev�0���u�D3-�y� 73Enʅ��O���+��5��q�]��yx�/�� ��kov�X&�>�P*��?ז�o�ɯ��!�?��o��?��P��&��>��"<��l�5� g�����
����H��ntDo<�tU���z+�_���v����/�����Y�i�,��� �q}il��2&�opoҚ
�^�!y� w0�4���
tF�~����rc���lm.s�4p"�[NuQ$:�H�������v�,W�����
�����㺣�w
n��.;����v�H'��8 �=um��-s��N.A�f���{A�۪�np��;��=4G�0�0��>Y�Q�U����xϨb��k�H��wX!g������FS���lsK0��%���}8
���<#[:]3$�'쯟/ˬ�Q��[���p�N5����ݧ��7=���T�����S��G����?f�?7�9�
k#=��byΞ�����r��k�~4f�)�o��ö���}������:A��0�~�+�܏g�lR0����kӣᘙ�m7k����r_�p����_�ܮ��AO.��M=N�
z\�ql]�����%+��t�
�P[Jk�C����t�{*����uk�j�a�����̊Z��fA]���?q yu|ָ�ͣe=� �}<��Hm���J��6~��A�G��5�X,c�b��R9R�Z>h/Gi���__\\�^%�ckNb6��A��~���<���J��1NJA
ٱ�e<��}xm��� �}?",L��>+ɿF�w�Kj�3��n�K|#V�h��2��)�+_b���!�;rۉZȋ]��M;%D�/��3-�.4ҳ�C������-/�p�cn��#ExU�R�U[�hu��.siͩpӈ�uH�l���r7SW��.>lhR�g��[��� ��+mR7�n�,�GfI��:��]
E[�)$;� U��

I'm not sure what you mean by decrypting this so you can view it. It looks like the contents of an image file. Try opening this file with an image viewer such as MS Paint, Adobe Photoshop or try uploading it to https://pixlr.com/editor. You might have to change the file extension for this to work (try PSD).
In case you're not interested in the image but actually the contents of the file, encoding is not really your problem here since this is not a text file but a binary file. Its contents don't represent characters but are contents of data structures used by Photoshop to represent an image.

How do I reassemble IP fragments with Perl's Net::Pcap::Reassemble?

I am new to Perl and trying to use the Net::Pcap::Reassemble - IP fragment reassembly for Net::Pcap. I am trying to reassemble TCP packets and "tie" the packets streams of interest and print the "tied" hex output for that data of interest. Here is the code below and and the printed output I get. The printed output is two separate Hex dumps (denoted by "Hex Payload:" string). It is apparent I am not calling Net::Pcap::Reassemble module correctly. The desire end output that I am trying to achieve is below, taken from the printed output. Can someone please point me into the right direction in using this module to achieve my desired output? Thank you.
my $user_data;
my $header;
my $packet;
my $err ='';
my $pcap = Net::Pcap::open_offline("./pcap", \$err) or die "can't open ./pcap...$err\n";
Net::Pcap::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::Reassemble::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::close($pcap);
my $ip;
my $tcp;
my $payload;
sub process_pkt
{
my ($user_data,$header, $packet) = #_;
$ip = NetPacket::IP->decode(eth_strip($packet));
$tcp = NetPacket::TCP->decode($ip->{data});
$payload = $tcp->{data};
my $hexPayload = hexdump(data => $payload, start_position => 0) if length $payload;
print "Hex Payload:". $hexPayload;
}
Output (this is what I currently get, but want to concatenate it together):
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:.*/*..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
Hex Payload: 0x0000 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0010 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0020 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0030 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0040 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0050 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0060 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0070 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0080 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0090 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep
Desired output (the above tied/concatenated together):
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:.*/*..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
0x0160 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0170 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0180 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0190 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0200 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0210 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0220 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0230 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0240 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0250 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep

You are trying to reassemble a network session, not a fragmented network packet. You should be using the module 'Net::Analysis'. It can, with some effort on your part, reassemble a complete network session. You will soon learn to hate pipe-lining.

Perl Net::Pcap hex dump packet session

I have the following information below being produced by the Net::Pcap module to print the payload of the packets of interest within a capture.
The data below is the excerpt of a Windows executable file being captured within Perl.
I would like to be able to capture all of the hex data output into one file or variable to assess the session data of the file download while retaining the integrity of the hex dump.
The problem I am having is for each packet being produced for the download of the file it obviously produces a hex dump output. This is easily seen by the output below by the string "Payload" I print per packet/hex dump output.
I want to tie all relevant data together for a given file download session. How can I do this in Perl?
Payload:HTTP/1.1 200 OK
Date: Fri, 15 Jun 2012 02:31:32 GMT
Server: Apache
Last-Modified: Sat, 10 Dec 2011 13:38:37 GMT
ETag: "dc44da-4d000-4b3bd04c7a2f1"
Accept-Ranges: bytes
Content-Length: 315392
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Content-Type: application/x-msdos-program
MZ<90>^#^C^#^#^#^D^#^#^#<FF><FF>^#^#<B8>^#^#^#^#^#^#^##^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#<F8>^#^#^#^N^_<BA>^N^#<B4> <CD>!<B8>^AL<CD>!This program cannot be run in DOS mode.^M
$^#^#^#^#^#^#^#4^TGmpu)>pu)>pu)>c}#>ru)>uyI>ru)>uy&>ku)>c}t>ru)><F3>}t>uu)>pu(>
u)>uyv><DA>u)><9C>~w>qu)>uys>qu)>Richpu)>^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#PE^#^#L^A^D^#^_R<E3>N^#^#^#^#^#^#^#^#<E0>^#^O^A^K^A^G
^#<80>^C^#^#<90>^A^#^#^#^#^#g^F^C^#^#^P^#^#^#<90>^C^#^#^##^#^#^P^#^#^#^P^#^#^D^#^#^#^#^#^#^#^D^#^#^#^#^#^#^#^# ^E^#^#^P^#^#^#^#^#^#^C^#^#^#^#^#^P^#^#^P^#^#^#^#^P^#^#^P^#^#^#^#^#^#^P^#^#^#^#^#^#^#^#^#^#^#<A8><91>^D^#P^#^#^#^#^#^E^#<C8>^T^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^##<91>^D^#H^#^#^#^#^#^#^#^#^#^#^#^#<90>^C^#<D0>^A^#^#^#^#^#^#^#^#
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#.text^#^#^#As^C^#^#^P^#^#^#<80>^C^#^#^P^#^#^#^#^#^#^#^#^#^#^#^#^#^# ^#^#`.rdata^#^#<C2>^K^A^#^#<90>^C^#^#^P^A^#^#<90>^C^#^#^#^#^#^#^#^#^#^#^#^#^##^#^##.data^#^#^#D]^#^#^#<A0>^D^#^#^P^#^#^#<A0>^D^#^#^#^#^#^#^#^#^#^#^#^#^##^#^#<C0>.rsrc^#^#^#<C8>^T^#^#^#^#^E^#^# ^#^#^#<B0>^D^#^#^#^#^#^#^#^#^#^#^#^#^##^#^##^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#Payload:^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^
Thank you Borodin for the suggestion. However, I don't know how to use this module correctly and it is apparent as I am still getting the same output. Here is the snippet of code I am using with this module and the printed hex. As you can see the printed hex is not tied together as one hex output but two seperate for the given TCP stream for which I want to tie together. Any help is appreciated.
my $user_data;
my $header;
my $packet;
my $err ='';
my $pcap = Net::Pcap::open_offline("./pcap", \$err) or die "can't open ./pcap...$err\n";
Net::Pcap::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::Reassemble::loop($pcap, -1, \&rend_callback, '');
Net::Pcap::close($pcap);
my $ip;
my $tcp;
my $payload;
sub process_pkt
{
my ($user_data,$header, $packet) = #_;
$ip = NetPacket::IP->decode(eth_strip($packet));
$tcp = NetPacket::TCP->decode($ip->{data});
$payload = $tcp->{data};
my $hexPayload = hexdump(data => $payload, start_position => 0) if length $payload;
print "Hex Payload:". $hexPayload;
}
print output:
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:./..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
Hex Payload: 0x0000 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0010 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0020 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0030 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0040 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0050 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0060 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0070 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0080 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0090 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep

Take a look at Net::Pcap::Reassemble
This module performs reassembly of fragmented datagrams in libpcap
packet capture data returned by the Net::Pcap loop() function

using sed, how does one match square brackets in a character class?

Here's a chunk of the raw data:
00000000 54 6f 70 69 63 20 46 6f 72 75 6d 20 52 65 70 6c |Topic Forum Repl|
00000010 69 65 73 20 4c 61 73 74 20 70 6f 73 74 20 31 20 |ies Last post 1 |
00000020 4c 69 6e 75 78 20 54 6f 64 61 79 20 31 34 3a 34 |Linux Today 14:4|
00000030 36 3a 35 37 20 62 79 20 4c 69 6e 75 78 20 4f 75 |6:57 by Linux Ou|
00000040 74 6c 61 77 73 20 32 36 39 20 e2 80 93 20 53 6f |tlaws 269 ... So|
00000050 6d 65 6f 6e 65 20 4b 6c 6f 73 65 20 54 68 61 74 |meone Klose That|
00000060 20 4f 75 74 6c 61 77 73 20 32 38 20 73 79 73 79 | Outlaws 28 sysy|
00000070 70 68 75 73 2e 6a 6f 6e 65 73 20 48 6f 6c 65 20 |phus.jones Hole |
00000080 62 79 20 59 4f 42 41 20 5b 20 31 20 32 20 5d 20 |by YOBA [ 1 2 ] |
00000090 32 20 4c 69 6e 75 78 20 26 20 54 6f 64 61 79 20 |2 Linux & Today |
000000a0 31 31 3a 34 34 3a 35 31 20 62 79 20 4c 6f 6f 6b |11:44:51 by Look|
000000b0 73 20 6c 69 6b 65 20 43 61 6e 6f 6e 69 63 61 6c |s like Canonical|
000000c0 20 69 73 20 61 6e 6e 6f 75 63 69 6e 67 20 70 6c | is annoucing pl|
000000d0 61 6e 73 20 46 72 65 65 64 6f 6d 20 31 20 6b 72 |ans Freedom 1 kr|
It's a hex dump and I'm interested in isolating the text part.
Here's a sed expression that almost works:
$ sed 's/.* |\([a-zA-Z0-9:& \.]*\)|$/\1/g' hex.dat
Topic Forum Repl
ies Last post 1
Linux Today 14:4
6:57 by Linux Ou
tlaws 269 ... So
meone Klose That
Outlaws 28 sysy
phus.jones Hole
00000080 62 79 20 59 4f 42 41 20 5b 20 31 20 32 20 5d 20 |by YOBA [ 1 2 ] |
2 Linux & Today
11:44:51 by Look
s like Canonical
is annoucing pl
ans Freedom 1 kr
Almost. But how to filter that last line though?
$ sed 's/.* |\([a-zA-Z0-9:&\[\] \.]*\)|$/\1/g' hex.dat
And:
$ sed 's/.* |\([a-zA-Z0-9:&\\[\\] \.]*\)|$/\1/g' hex.dat
Don't work at all (they fail to translate anything).
And:
$ sed 's/.* |\([a-zA-Z0-9:&[] \.]*\)|$/\1/g' hex.dat
obviously can't work.
Thanks for any help.

You almost had it.
Look at this section of a Unix regular expressions tutorial.
The way that yours could be done is by placing ][ immediately after you begin your character class.
So, try sed 's/.* |\([][a-zA-Z0-9:& \.]*\)|$/\1/g' hex.dat
For clarification, it does not matter where in the character class the [ is, so long as the closing bracket you intend to include in your character class (]) immediately follows the opening of your character class.
Also, as a further edit, try typing man cut and using what Tomasz said in a comment.
cut -d='|' -f2 hex.dat will cut your file, delimiting on a pipe, and take the second field.