Encoding problems importing - postgresql

I am having trouble importing a csv to postgres. I import using UTF8 encoding. I am returned the error:
ERROR: invalid byte sequence for encoding "UTF8": 0xcc 0xc1
CONTEXT: COPY a_household , line 2
When I open the file in Notepad++ it says at the bottom it is encoded in UTF-8 w/o BOM. All the text is displayed - all characters appear to display correctly.
The hexdump (suggested below) returns
00000060 2D 32 33 2E 39 32 35 38-37 33 39 32 2C 33 32 2E -23.92587392,32.
00000070 37 33 32 31 38 33 30 33-2C 37 35 2E 31 32 32 36 73218303,75.1226
00000080 32 39 33 31 2C 34 2C 31-2C 61 62 6F 76 65 31 79 2931,4,1,above1y
00000090 72 2C 31 37 2C 2C 58 61-6E 67 61 6E 61 2C 56 65 r,17,,Xangana,Ve
000000A0 6C 68 61 20 61 70 6F 73-74 6F 6C 6F 2C 53 69 6D lha apostolo,Sim
000000B0 2C 47 72 69 70 65 20 4D-61 6C CC C1 72 69 61 2C ,Gripe Mal..ria,
000000C0 50 72 61 74 69 63 61 6E-74 65 5F 64 65 5F 6D 65 Praticante_de_me
000000D0 64 69 63 69 6E 61 20 50-72 CC 5F 70 72 69 6F 2C dicina Pr._prio,
000000E0 4D 65 64 69 63 61 6D 65-6E 74 6F 73 5F 64 61 5F Medicamentos_da_
000000F0 66 61 72 6D CC C1 63 69-61 2C 36 2C 4E CC A3 6F farm..cia,6,N..o
00000100 2C 2C 31 34 2C 4E CC A3-6F 2C 22 44 65 5F 62 6F ,,14,N..o,"De_bo
00000110 72 6C 61 2C 5F 63 6F 6D-5F 61 75 74 6F 72 69 7A rla,_com_autoriz
00000120 61 CC A4 CC A3 6F 22 2C-2C 2C 35 2C 41 72 65 69 a....o",,,5,Arei
00000130 61 20 65 20 65 73 74 61-63 61 73 2C 43 68 61 70 a e estacas,Chap
00000140 61 73 5F 64 65 5F 5A 69-6E 63 6F 2C 41 72 65 69 as_de_Zinco,Arei
00000150 61 2C 4C 65 6E 68 61 2C-4E 61 64 61 2C 4C 61 74 a,Lenha,Nada,Lat
00000160 72 69 6E 61 5F 6D 65 6C-68 6F 72 61 64 61 2C 50 rina_melhorada,P
00000170 61 72 61 5F 6D 65 6D 62-72 6F 73 5F 61 67 72 65 ara_membros_agre
00000180 67 61 64 6F 5F 66 61 6D-69 6C 69 61 72 2C 4C 61 gado_familiar,La
00000190 67 6F 61 2F 6C 61 67 6F-2C 50 6F CC A4 6F 5F 70 goa/lago,Po..o_p
000001A0 CC BC 62 6C 69 63 6F 5F-61 62 65 72 74 6F 2C 37 ..blico_aberto,7
000001B0 2C 50 CC A9 2C 30 2C 2C-30 2C 2C 4E 75 6E 63 61 ,P..,0,,0,,Nunca
000001C0 2C 32 2C 50 6F 72 5F 61-6E 6F 2C 52 61 72 61 6D ,2,Por_ano,Raram
000001D0 65 6E 74 65 2C 52 61 72-61 6D 65 6E 74 65 2C 4D ente,Raramente,M
000001E0 CC C1 2C 32 35 2C 4E CC-A3 6F 2C 54 65 72 72 61 ..,25,N..o,Terra
000001F0 5F 70 65 72 74 65 6E 63-65 6E 74 65 5F 61 5F 65 _pertencente_a_e
00000200 73 74 61 5F 61 6C 64 65-69 61 2C 4E 75 6E 63 61 sta_aldeia,Nunca
00000210 2C 2C 2C 4D 61 69 73 5F-6F 75 5F 6D 65 6E 6F 73 ,,,Mais_ou_menos
00000220 5F 61 5F 6D 65 73 6D 61-2C 4E CC A3 6F 2C 2C 4D _a_mesma,N..o,,M
00000230 61 69 73 5F 66 CC C1 63-69 6C 5F 64 65 5F 6F 62 ais_f..cil_de_ob
00000240 74 65 72 2C 2C 53 69 6D-2C 4E CC A3 6F 2C 2C 2C ter,,Sim,N..o,,,
00000250 2C 4E CC A3 6F 2C 53 69-6D 2C 56 61 63 61 73 2C ,N..o,Sim,Vacas,
00000260 32 20 34 2C 2C 53 69 6D-2C 31 2C 53 69 6D 2C 6A 2 4,,Sim,1,Sim,j
00000270 61 6E 65 69 72 6F 20 66-65 76 65 72 65 69 72 6F aneiro fevereiro
00000280 20 6D 61 72 CC A4 6F 20-61 62 72 69 6C 20 6D 61 mar..o abril ma
00000290 69 6F 20 6A 75 6E 68 6F-20 6A 75 6C 68 6F 20 61 io junho julho a
000002A0 67 6F 73 74 6F 2C 31 30-30 2C 53 61 63 6F 73 2C gosto,100,Sacos,
000002B0 30 2C 4D 61 69 73 5F 64-6F 5F 71 75 65 5F 75 6D 0,Mais_do_que_um
000002C0 61 5F 76 65 7A 2C 53 61-75 64 65 2C 4E CC A3 6F a_vez,Saude,N..o
000002D0 2C 4E CC A3 6F 2C 2C 2D-39 39 38 2C 53 69 6D 2C ,N..o,,-998,Sim,
000002E0 4D 75 69 74 6F 5F 73 61-74 69 73 66 65 69 74 6F Muito_satisfeito
000002F0 2C 4D 75 69 74 6F 5F 73-61 74 69 73 66 65 69 74 ,Muito_satisfeit
00000300 6F 2C 4D 75 69 74 6F 5F-73 61 74 69 73 66 65 69 o,Muito_satisfei
00000310 74 6F 2C 49 6E 73 61 74-69 73 66 65 69 74 6F 2C to,Insatisfeito,
00000320 22 4E 65 6D 5F 69 6E 73-61 74 69 73 66 65 69 74 "Nem_insatisfeit
00000330 6F 2C 5F 6E 65 6D 5F 73-61 74 69 73 66 65 69 74 o,_nem_satisfeit
00000340 6F 22 2C 49 6E 73 61 74-69 73 66 65 69 74 6F 2C o",Insatisfeito,
00000350 4D 75 69 74 6F 5F 73 61-74 69 73 66 65 69 74 6F Muito_satisfeito
00000360 2C 49 6E 73 61 74 69 73-66 65 69 74 6F 2C 49 6E ,Insatisfeito,In
00000370 73 61 74 69 73 66 65 69-74 6F 2C 53 61 74 69 73 satisfeito,Satis
00000380 66 65 69 74 6F 2C 53 61-74 69 73 66 65 69 74 6F feito,Satisfeito
00000390 2C 4E 61 5F 6D CC A9 64-69 61 2C 4D 65 6C 68 6F ,Na_m..dia,Melho
000003A0 72 5F 61 67 6F 72 61 2C-22 44 69 66 69 63 69 6C r_agora,"Dificil
000003B0 20 64 65 20 64 69 7A 65-72 2C 20 70 6F 69 73 20 de dizer, pois
000003C0 6E 61 6F 20 73 65 69 20-6F 20 71 75 65 20 6F 20 nao sei o que o
000003D0 61 6E 6F 20 6D 65 20 72-65 73 65 72 76 61 22 2C ano me reserva",
000003E0 2C 75 75 69 64 3A 66 63-32 34 66 62 38 39 2D 35 ,uuid:fc24fb89-5
000003F0 34 63 30 2D 34 39 66 34-2D 61 31 36 31 2D 35 64 4c0-49f4-a161-5d
00000400 64 34 63 63 39 62 37 39-65 39 0D 0A d4cc9b79e9..
...
Any help would be very much appreciated.
Becky

Related

Failed to run BookInfo example behind proxy server, failed calling webhook "pilot.validation.istio.io"

Issue also posted at:
https://github.com/istio/istio/issues/21195
https://www.reddit.com/r/istio/comments/f57v2a/help_failed_to_run_bookinfo_example_behind_proxy/
Problem:
Following the bookinfo example, when trying to apply bookinfo-gateway, I get the following error:
Error from server (InternalError): error when creating "samples/bookinfo/networking/bookinfo-gateway.yaml": Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
Output:
$ kubectl --v=9 apply -f samples/bookinfo/networking/bookinfo-gateway.yaml
I0216 18:01:08.548290 4904 loader.go:375] Config loaded from file: /home/user/.kube/config
I0216 18:01:08.550426 4904 round_trippers.go:423] curl -k -v -XGET -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" -H "Accept: application/com.github.proto-openapi.spec.v2#v1.0+protobuf" 'https://ha-lb-ip:6443/openapi/v2?timeout=32s'
I0216 18:01:08.600310 4904 round_trippers.go:443] GET https://ha-lb-ip:6443/openapi/v2?timeout=32s 200 OK in 49 milliseconds
I0216 18:01:08.600348 4904 round_trippers.go:449] Response Headers:
I0216 18:01:08.600355 4904 round_trippers.go:452] Accept-Ranges: bytes
I0216 18:01:08.600361 4904 round_trippers.go:452] X-Varied-Accept: application/com.github.proto-openapi.spec.v2#v1.0+protobuf
I0216 18:01:08.600366 4904 round_trippers.go:452] Content-Type: application/octet-stream
I0216 18:01:08.600371 4904 round_trippers.go:452] Etag: "DCA49D599C62F0A8DDF840BBF0F4DB11A2B0C9805F7F6CEB19F163F61CA7D40F9E7A3607B007A74CCD6DBA6565BE6E6E3085528F7FD18EDAE99BABE9702D8700"
I0216 18:01:08.600378 4904 round_trippers.go:452] Last-Modified: Sun, 16 Feb 2020 11:54:46 GMT
I0216 18:01:08.600430 4904 round_trippers.go:452] Vary: Accept-Encoding
I0216 18:01:08.600435 4904 round_trippers.go:452] Vary: Accept
I0216 18:01:08.600439 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:08 GMT
I0216 18:01:08.817775 4904 request.go:1015] Response Body:
00000000 0a 03 32 2e 30 12 15 0a 0a 4b 75 62 65 72 6e 65 |..2.0....Kuberne|
00000010 74 65 73 12 07 76 31 2e 31 37 2e 33 42 93 f7 a9 |tes..v1.17.3B...|
00000020 01 12 ae 27 0a 29 2f 61 70 69 2f 76 31 2f 77 61 |...'.)/api/v1/wa|
00000030 74 63 68 2f 6e 61 6d 65 73 70 61 63 65 73 2f 7b |tch/namespaces/{|
00000040 6e 61 6d 65 73 70 61 63 65 7d 2f 70 6f 64 73 12 |namespace}/pods.|
00000050 80 27 12 97 04 0a 07 63 6f 72 65 5f 76 31 1a 6f |.'.....core_v1.o|
00000060 77 61 74 63 68 20 69 6e 64 69 76 69 64 75 61 6c |watch individual|
00000070 20 63 68 61 6e 67 65 73 20 74 6f 20 61 20 6c 69 | changes to a li|
00000080 73 74 20 6f 66 20 50 6f 64 2e 20 64 65 70 72 65 |st of Pod. depre|
00000090 63 61 74 65 64 3a 20 75 73 65 20 74 68 65 20 27 |cated: use the '|
000000a0 77 61 74 63 68 27 20 70 61 72 61 6d 65 74 65 72 |watch' parameter|
000000b0 20 77 69 74 68 20 61 20 6c 69 73 74 20 6f 70 65 | with a list ope|
000000c0 72 61 74 69 6f 6e 20 69 6e 73 74 65 61 64 2e 2a |ration instead.*|
000000d0 1c 77 61 74 63 68 43 6f 72 65 56 31 4e 61 6d 65 |.watchCoreV1Name|
000000e0 73 70 61 63 65 64 50 6f 64 4c 69 73 74 32 10 61 |spacedPodList2.a|
000000f0 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f 6e 32 |pplication/json2|
00000100 10 61 70 70 6c 69 63 61 74 69 6f 6e 2f 79 61 6d |.application/yam|
00000110 6c 32 23 61 70 70 6c 69 63 61 74 69 6f 6e 2f 76 |l2#application/v|
00000120 6e 64 2e 6b 75 62 65 72 6e 65 74 65 73 2e 70 72 |nd.kubernetes.pr|
00000130 6f 74 6f 62 75 66 32 1d 61 70 70 6c 69 63 61 74 |otobuf2.applicat|
00000140 69 6f 6e 2f 6a 73 6f 6e 3b 73 74 72 65 61 6d 3d |ion/json;stream=|
00000150 77 61 74 63 68 32 30 61 70 70 6c 69 63 61 74 69 |watch20applicati|
00000160 6f 6e 2f 76 6e 64 2e 6b 75 62 65 72 6e 65 74 65 |on/vnd.kubernete|
00000170 73 2e 70 72 6f 74 6f 62 75 66 3b 73 74 72 65 61 |s.protobuf;strea|
00000180 6d 3d 77 61 74 63 68 3a 03 2a 2f 2a 4a 6b 0a 50 |m=watch:.*/*Jk.P|
00000190 0a 03 32 30 30 12 49 0a 47 0a 02 4f 4b 12 41 0a |..200.I.G..OK.A.|
000001a0 3f 0a 3d 23 2f 64 65 66 69 6e 69 74 69 6f 6e 73 |?.=#/definitions|
000001b0 2f 69 6f 2e 6b 38 73 2e 61 70 69 6d 61 63 68 69 |/io.k8s.apimachi|
000001c0 6e 65 72 79 2e 70 6b 67 2e 61 70 69 73 2e 6d 65 |nery.pkg.apis.me|
000001d0 74 61 2e 76 31 2e 57 61 74 63 68 45 76 65 6e 74 |ta.v1.WatchEvent|
000001e0 0a 17 0a 03 34 30 31 12 10 0a 0e 0a 0c 55 6e 61 |....401......Una|
000001f0 75 74 68 6f 72 69 7a 65 64 52 05 68 74 74 70 73 |uthorizedR.https|
00000200 6a 23 0a 13 78 2d 6b 75 62 65 72 6e 65 74 65 73 |j#..x-kubernetes|
00000210 2d 61 63 74 69 6f 6e 12 0c 12 0a 77 61 74 63 68 |-action....watch|
00000220 6c 69 73 74 0a 6a 45 0a 1f 78 2d 6b 75 62 65 72 |list.jE..x-kuber|
00000230 6e 65 74 65 73 2d 67 72 6f 75 70 2d 76 65 72 73 |netes-group-vers|
00000240 69 6f 6e 2d 6b 69 6e 64 12 22 12 20 6b 69 6e 64 |ion-kind.". kind|
00000250 3a 20 50 6f 64 0a 76 65 72 73 69 6f 6e 3a 20 76 |: Pod.version: v|
00000260 31 0a 67 72 6f 75 70 3a 20 22 22 0a 4a 82 04 0a |1.group: "".J...|
00000270 ff 03 12 fc 03 1a f9 03 12 05 71 75 65 72 79 1a |..........query.|
00000280 ce 03 61 6c 6c 6f 77 57 61 74 63 68 42 6f 6f 6b |..allowWatchBook|
00000290 6d 61 72 6b 73 20 72 65 71 75 65 73 74 73 20 77 |marks requests w|
000002a0 61 74 63 68 20 65 76 65 6e 74 73 20 77 69 74 68 |atch events with|
000002b0 20 74 79 70 65 20 22 42 4f 4f 4b 4d 41 52 4b 22 | type "BOOKMARK"|
000002c0 2e 20 53 65 72 76 65 72 73 20 74 68 61 74 20 64 |. Servers that d|
000002d0 6f 20 6e 6f 74 20 69 6d 70 6c 65 6d 65 6e 74 20 |o not implement |
000002e0 62 6f 6f 6b 6d 61 72 6b 73 20 6d 61 79 20 69 67 |bookmarks may ig|
000002f0 6e 6f 72 65 20 74 68 69 73 20 66 6c 61 67 20 61 |nore this flag a|
00000300 6e 64 20 62 6f 6f 6b 6d 61 72 6b 73 20 61 72 65 |nd bookmarks are|
00000310 20 73 65 6e 74 20 61 74 20 74 68 65 20 73 65 72 | sent at the ser|
00000320 76 65 72 27 73 20 64 69 73 63 72 65 74 69 6f 6e |ver's discretion|
00000330 2e 20 43 6c 69 65 6e 74 73 20 73 68 6f 75 6c 64 |. Clients should|
00000340 20 6e 6f 74 20 61 73 73 75 6d 65 20 62 6f 6f 6b | not assume book|
00000350 6d 61 72 6b 73 20 61 72 65 20 72 65 74 75 72 6e |marks are return|
00000360 65 64 20 61 74 20 61 6e 79 20 73 70 65 63 69 66 |ed at any specif|
00000370 69 63 20 69 6e 74 65 72 76 61 6c 2c 20 6e 6f 72 |ic interval, nor|
00000380 20 6d 61 79 20 74 68 65 79 20 61 73 73 75 6d 65 | may they assume|
00000390 20 74 68 65 20 73 65 72 76 65 72 20 77 69 6c 6c | the server will|
000003a0 20 73 65 6e 64 20 61 6e 79 20 42 4f 4f 4b 4d 41 | send any BOOKMA|
000003b0 52 4b 20 65 76 65 6e 74 20 64 75 72 69 6e 67 20 |RK event during |
000003c0 61 20 73 65 73 73 69 6f 6e 2e 20 49 66 20 74 68 |a session. If th|
000003d0 69 73 20 69 73 20 6e 6f 74 20 61 20 77 61 74 63 |is is not a watc|
000003e0 68 2c 20 74 68 69 73 20 66 69 65 6c 64 20 69 73 |h, this field is|
000003f0 20 69 67 6e 6f 72 65 64 2e 20 49 66 20 74 68 65 | ignored. If the|
00000400 20 66 65 61 74 75 72 65 20 67 61 74 65 20 57 61 | feature gate Wa|
00000410 74 63 68 42 6f 6f 6b 6d 61 72 6b 73 20 69 73 20 |tchBookmarks is |
00000420 6e 6f 74 20 65 6e 61 62 6c 65 64 20 69 6e 20 61 |not enabled in a|
00000430 70 69 73 65 72 76 65 72 2c 20 74 68 69 73 20 66 |piserver, this f|
00000440 69 65 6c 64 20 69 73 20 69 67 6e 6f 72 65 64 2e |ield is ignored.|
00000450 22 13 61 6c 6c 6f 77 57 61 74 63 68 42 6f 6f 6b |".allowWatchBook|
00000460 6d 61 72 6b 73 32 07 62 6f 6f 6c 65 61 6e a0 01 |marks2.boolean..|
00000470 01 4a ef 09 0a ec 09 12 e9 09 1a e6 09 12 05 71 |.J.............q|
00000480 75 65 72 79 1a c7 09 54 68 65 20 63 6f 6e 74 69 |uery...The conti|
00000490 6e 75 65 20 6f 70 74 69 6f 6e 20 73 68 6f 75 6c |nue option shoul|
000004a0 64 20 62 65 20 73 65 74 20 77 68 65 6e 20 72 65 |d be set when re|
000004b0 74 72 69 65 76 69 6e 67 20 6d 6f 72 65 20 72 65 |trieving more re|
000004c0 73 75 6c 74 73 20 66 72 6f 6d 20 74 68 65 20 73 |sults from the s|
000004d0 65 72 76 65 72 2e 20 53 69 6e 63 65 20 74 68 69 |erver. Since thi|
000004e0 73 20 76 61 6c 75 65 20 69 73 20 73 65 72 76 65 |s value is serve|
000004f0 72 20 64 65 66 69 6e 65 64 2c 20 63 6c 69 65 6e |r defined, clien|
00000500 74 73 20 6d 61 79 20 6f 6e 6c 79 20 75 73 65 20 |ts may only use |
00000510 74 68 65 20 63 6f 6e 74 69 6e 75 65 20 76 61 6c |the continue val|
00000520 75 65 20 66 72 6f 6d 20 61 20 70 72 65 76 69 6f |ue from a previo|
00000530 75 73 20 71 75 65 72 79 20 72 65 73 75 6c 74 20 |us query result |
00000540 77 69 74 68 20 69 64 65 6e 74 69 63 61 6c 20 71 |with identical q|
00000550 75 65 72 79 20 70 61 72 61 6d 65 74 65 72 73 20 |uery parameters |
00000560 28 65 78 63 65 70 74 20 66 6f 72 20 74 68 65 20 |(except for the |
00000570 76 61 6c 75 65 20 6f 66 20 63 6f 6e 74 69 6e 75 |value of continu|
00000580 65 29 20 61 6e 64 20 74 68 65 20 73 65 72 76 65 |e) and the serve|
00000590 72 20 6d 61 79 20 72 65 6a 65 63 74 20 61 20 63 |r may reject a c|
000005a0 6f 6e 74 69 6e 75 65 20 76 61 6c 75 65 20 69 74 |ontinue value it|
000005b0 20 64 6f 65 73 20 6e 6f 74 20 72 65 63 6f 67 6e | does not recogn|
000005c0 69 7a 65 2e 20 49 66 20 74 68 65 20 73 70 65 63 |ize. If the spec|
000005d0 69 66 69 65 64 20 63 6f 6e 74 69 6e 75 65 20 76 |ified continue v|
000005e0 61 6c 75 65 20 69 73 20 6e 6f 20 6c 6f 6e 67 65 |alue is no longe|
000005f0 72 20 76 61 6c 69 64 20 77 68 65 74 68 65 72 20 |r valid whether |
00000600 64 75 65 20 74 6f 20 65 78 70 69 72 61 74 69 6f |due to expiratio|
00000610 6e 20 28 67 65 6e 65 72 61 6c 6c 79 20 66 69 76 |n (generally fiv|
00000620 65 20 74 6f 20 66 69 66 74 65 65 6e 20 6d 69 6e |e to fifteen min|
00000630 75 74 65 73 29 20 6f 72 20 61 20 63 6f 6e 66 69 |utes) or a confi|
00000640 67 75 72 61 74 69 6f 6e 20 63 68 61 6e 67 65 20 |guration change |
00000650 6f 6e 20 74 68 65 20 73 65 72 76 65 72 2c 20 74 |on the server, t|
00000660 68 65 20 73 65 72 76 65 72 20 77 69 6c 6c 20 72 |he server will r|
00000670 65 73 70 6f 6e 64 20 77 69 74 68 20 61 20 34 31 |espond with a 41|
00000680 30 20 52 65 73 6f 75 72 63 65 45 78 70 69 72 65 |0 ResourceExpire|
00000690 64 20 65 72 72 6f 72 20 74 6f 67 65 74 68 65 72 |d error together|
000006a0 20 77 69 74 68 20 61 20 63 6f 6e 74 69 6e 75 65 | with a continue|
000006b0 20 74 6f 6b 65 6e 2e 20 49 66 20 74 68 65 20 63 | token. If the c|
000006c0 6c 69 65 6e 74 20 6e 65 65 64 73 20 61 20 63 6f |lient needs a co|
000006d0 6e 73 69 73 74 65 6e 74 20 6c 69 73 74 2c 20 69 |nsistent list, i|
000006e0 74 20 6d 75 73 74 20 72 65 73 74 61 72 74 20 74 |t must restart t|
000006f0 68 65 69 72 20 6c 69 73 74 20 77 69 74 68 6f 75 |heir list withou|
00000700 74 20 74 68 65 20 63 6f 6e 74 69 6e 75 65 20 66 |t the continue f|
00000710 69 65 6c 64 2e 20 4f 74 68 65 72 77 69 73 65 2c |ield. Otherwise,|
00000720 20 74 68 65 20 63 6c 69 65 6e 74 20 6d 61 79 20 | the client may |
00000730 73 65 6e 64 20 61 6e 6f 74 68 65 72 20 6c 69 73 |send another lis|
00000740 74 20 72 65 71 75 65 73 74 20 77 69 74 68 20 74 |t request with t|
00000750 68 65 20 74 6f 6b 65 6e 20 72 65 63 65 69 76 65 |he token receive|
00000760 64 20 77 69 74 68 20 74 68 65 20 34 31 30 20 65 |d with the 410 e|
00000770 72 72 6f 72 2c 20 74 68 65 20 73 65 72 76 65 72 |rror, the server|
00000780 20 77 69 6c 6c 20 72 65 73 70 6f 6e 64 20 77 69 | will respond wi|
00000790 74 68 20 61 20 6c 69 73 74 20 73 74 61 72 74 69 |th a list starti|
000007a0 6e 67 20 66 72 6f 6d 20 74 68 65 20 6e 65 78 74 |ng from the next|
000007b0 20 6b 65 79 2c 20 62 75 74 20 66 72 6f 6d 20 74 | key, but from t|
000007c0 68 65 20 6c 61 74 65 73 74 20 73 6e 61 70 73 68 |he latest snapsh|
000007d0 6f 74 2c 20 77 68 69 63 68 20 69 73 20 69 6e 63 |ot, which is inc|
000007e0 6f 6e 73 69 73 74 65 6e 74 20 66 72 6f 6d 20 74 |onsistent from t|
000007f0 68 65 20 70 72 65 76 69 6f 75 73 20 6c 69 73 74 |he previous list|
00000800 20 72 65 73 75 6c 74 73 20 2d 20 6f 62 6a 65 63 | results - objec|
00000810 74 73 20 74 68 61 74 20 61 72 65 20 63 [truncated 17345571 chars]
I0216 18:01:08.944681 4904 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways/bookinfo-gateway'
I0216 18:01:08.988784 4904 round_trippers.go:443] GET https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways/bookinfo-gateway 404 Not Found in 44 milliseconds
I0216 18:01:08.988834 4904 round_trippers.go:449] Response Headers:
I0216 18:01:08.988840 4904 round_trippers.go:452] Content-Length: 258
I0216 18:01:08.988844 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:08 GMT
I0216 18:01:08.988848 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:01:08.988896 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"gateways.networking.istio.io \"bookinfo-gateway\" not found","reason":"NotFound","details":{"name":"bookinfo-gateway","group":"networking.istio.io","kind":"gateways"},"code":404}
I0216 18:01:08.989774 4904 request.go:1017] Request Body: {"apiVersion":"networking.istio.io/v1alpha3","kind":"Gateway","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"networking.istio.io/v1alpha3\",\"kind\":\"Gateway\",\"metadata\":{\"annotations\":{},\"name\":\"bookinfo-gateway\",\"namespace\":\"default\"},\"spec\":{\"selector\":{\"istio\":\"ingressgateway\"},\"servers\":[{\"hosts\":[\"*\"],\"port\":{\"name\":\"http\",\"number\":80,\"protocol\":\"HTTP\"}}]}}\n"},"name":"bookinfo-gateway","namespace":"default"},"spec":{"selector":{"istio":"ingressgateway"},"servers":[{"hosts":["*"],"port":{"name":"http","number":80,"protocol":"HTTP"}}]}}
I0216 18:01:08.989839 4904 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways'
I0216 18:01:38.996165 4904 round_trippers.go:443] POST https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/gateways 500 Internal Server Error in 30006 milliseconds
I0216 18:01:38.996302 4904 round_trippers.go:449] Response Headers:
I0216 18:01:38.996315 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:01:38.996320 4904 round_trippers.go:452] Content-Length: 481
I0216 18:01:38.996349 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:38 GMT
I0216 18:01:38.996563 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded","reason":"InternalError","details":{"causes":[{"message":"failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"}]},"code":500}
I0216 18:01:38.999383 4904 round_trippers.go:423] curl -k -v -XGET -H "Accept: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/bookinfo'
I0216 18:01:39.042269 4904 round_trippers.go:443] GET https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/bookinfo 404 Not Found in 42 milliseconds
I0216 18:01:39.042304 4904 round_trippers.go:449] Response Headers:
I0216 18:01:39.042310 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:01:39.042315 4904 round_trippers.go:452] Content-Length: 256
I0216 18:01:39.042319 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:01:39 GMT
I0216 18:01:39.042352 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"virtualservices.networking.istio.io \"bookinfo\" not found","reason":"NotFound","details":{"name":"bookinfo","group":"networking.istio.io","kind":"virtualservices"},"code":404}
I0216 18:01:39.043083 4904 request.go:1017] Request Body: {"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"networking.istio.io/v1alpha3\",\"kind\":\"VirtualService\",\"metadata\":{\"annotations\":{},\"name\":\"bookinfo\",\"namespace\":\"default\"},\"spec\":{\"gateways\":[\"bookinfo-gateway\"],\"hosts\":[\"*\"],\"http\":[{\"match\":[{\"uri\":{\"exact\":\"/productpage\"}},{\"uri\":{\"prefix\":\"/static\"}},{\"uri\":{\"exact\":\"/login\"}},{\"uri\":{\"exact\":\"/logout\"}},{\"uri\":{\"prefix\":\"/api/v1/products\"}}],\"route\":[{\"destination\":{\"host\":\"productpage\",\"port\":{\"number\":9080}}}]}]}}\n"},"name":"bookinfo","namespace":"default"},"spec":{"gateways":["bookinfo-gateway"],"hosts":["*"],"http":[{"match":[{"uri":{"exact":"/productpage"}},{"uri":{"prefix":"/static"}},{"uri":{"exact":"/login"}},{"uri":{"exact":"/logout"}},{"uri":{"prefix":"/api/v1/products"}}],"route":[{"destination":{"host":"productpage","port":{"number":9080}}}]}]}}
I0216 18:01:39.043172 4904 round_trippers.go:423] curl -k -v -XPOST -H "Accept: application/json" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960" 'https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices'
I0216 18:02:09.049842 4904 round_trippers.go:443] POST https://ha-lb-ip:6443/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices 500 Internal Server Error in 30006 milliseconds
I0216 18:02:09.050031 4904 round_trippers.go:449] Response Headers:
I0216 18:02:09.050043 4904 round_trippers.go:452] Content-Type: application/json
I0216 18:02:09.050052 4904 round_trippers.go:452] Content-Length: 481
I0216 18:02:09.050059 4904 round_trippers.go:452] Date: Sun, 16 Feb 2020 12:02:09 GMT
I0216 18:02:09.050249 4904 request.go:1017] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded","reason":"InternalError","details":{"causes":[{"message":"failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"}]},"code":500}
I0216 18:02:09.051955 4904 helpers.go:203] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "error when creating \"samples/bookinfo/networking/bookinfo-gateway.yaml\": Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded",
"reason": "InternalError",
"details": {
"causes": [
{
"message": "failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"
}
]
},
"code": 500
}]
I0216 18:02:09.052104 4904 helpers.go:203] server response object: [{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "error when creating \"samples/bookinfo/networking/bookinfo-gateway.yaml\": Internal error occurred: failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded",
"reason": "InternalError",
"details": {
"causes": [
{
"message": "failed calling webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded"
}
]
},
"code": 500
}]
F0216 18:02:09.052210 4904 helpers.go:114] Error from server (InternalError): error when creating "samples/bookinfo/networking/bookinfo-gateway.yaml": Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
Error from server (InternalError): error when creating "samples/bookinfo/networking/bookinfo-gateway.yaml": Internal error occurred: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
Related Information:
Docker version 19.03.5
Kubernetes version 1.17.3
Istio version 1.4.4
Kubernetes clusters are run behind company's proxy server. CNI using kube-flannel
The docker.service.d config has proxy configured at /etc/systemd/system/docker.service.d/proxy.conf.
NO_PROXY/no_proxy is set to localhost,127.0.0.1,::1,.grameenphone.com,10.10.18.188,10.10.23.57,10.10.23.58,10.10.23.59,10.10.23.60,10.10.23.61,kubernetes.default,.validation.istio.io,.istio.io,.istio-system.svc,.svc,.istio-system,.svc.cluster.local,.cluster.local,10.244.0.0/16.
Installed ISTIO using istioctl, yaml given below:
apiVersion: install.istio.io/v1alpha2
kind: IstioControlPlane
spec:
defaultNamespace: istio-system
cni:
enabled: true
gateways:
components:
egressGateway:
enabled: false
ingressGateway:
enabled: true
enabled: true
values:
cni:
excludeNamespaces:
- istio-system
- kube-system
gateways:
istio-ingressgateway:
type: NodePort
global:
configValidation: true
grafana:
enabled: true
kiali:
enabled: true
tracing:
enabled: true
Istio install was successful, verified using gisitoctl verify-install.
Tried modifying /etc/kubernetes/manifests/kube-apiserver.yaml by adding env: section to the container, but still fails to create bookinfo-gateway.
env:
- name: http_proxy
value: http://10.10.20.107:3828
- name: https_proxy
value: http://10.10.20.107:3828
- name: no_proxy
value: localhost,127.0.0.1,::1,.grameenphone.com,10.10.18.188,10.10.23.57,10.10.23.58,10.10.23.59,10.10.23.60,10.10.23.61,kubernetes.default,.validation.istio.io,.istio.io,.istio-system.svc,.svc,.istio-system,.svc.cluster.local,.cluster.local,10.244.0.0/16,10.96.0.0/12
For sidecar injection, I'm following the manual procedure.
All Isito pods are up & running.
Log from kube-apiserver:
I0217 06:36:08.719672 1 controller.go:606] quota admission added evaluator for: deployments.apps
I0217 06:37:19.151894 1 trace.go:116] Trace[2116455659]: "Call validating webhook" configuration:istio-galley,webhook:pilot.validation.istio.io,resource:networking.istio.io/v1alpha3, Resource=gateways,subresource:,operation:CREATE,UID:de57f49e-fd19-44ea-99d7-414dfec0981f (started: 2020-02-17 06:36:49.150893916 +0000 UTC m=+6172.698922922) (total time: 30.000884409s):
Trace[2116455659]: [30.000884409s] [30.000884409s] END
W0217 06:37:19.151963 1 dispatcher.go:133] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
I0217 06:37:19.152376 1 trace.go:116] Trace[2129026394]: "Create" url:/apis/networking.istio.io/v1alpha3/namespaces/default/gateways,user-agent:kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960,client:10.10.18.188 (started: 2020-02-17 06:36:49.149255324 +0000 UTC m=+6172.697284349) (total time: 30.003096239s):
Trace[2129026394]: [30.003096239s] [30.002534955s] END
I0217 06:37:49.167492 1 trace.go:116] Trace[1063136940]: "Call validating webhook" configuration:istio-galley,webhook:pilot.validation.istio.io,resource:networking.istio.io/v1alpha3, Resource=virtualservices,subresource:,operation:CREATE,UID:2dd4b3e7-8333-4c1d-8222-dd53f8ce2db4 (started: 2020-02-17 06:37:19.166772885 +0000 UTC m=+6202.714801862) (total time: 30.000661809s):
Trace[1063136940]: [30.000661809s] [30.000661809s] END
W0217 06:37:49.167530 1 dispatcher.go:133] Failed calling webhook, failing closed pilot.validation.istio.io: failed calling webhook "pilot.validation.istio.io": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: context deadline exceeded
I0217 06:37:49.167996 1 trace.go:116] Trace[639287810]: "Create" url:/apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices,user-agent:kubectl/v1.17.3 (linux/amd64) kubernetes/06ad960,client:10.10.18.188 (started: 2020-02-17 06:37:19.165649222 +0000 UTC m=+6202.713678242) (total time: 30.002325938s):
Trace[639287810]: [30.002325938s] [30.001822425s] END
I googled/looked into other related issues & tried their solutions, but did not worked.
If there is any additional information required, please do let me know.

Seems that galley stopped or doesn't answer.
Galley provides configuration management services for Istio.
So, pls check these two steps:
1. Is galley running?
galley probe
2. If it runs, try the WA
Due to this known issue istio/istio#17162, seems that your problem is related to api-server proxy settings.
And there is a workaround
Try to use --set global.configValidation=false when installing istio
See also manual on istio validation

Can I tell GitHub (or eq.) to use ASCII to make my binary files readable?

I want to host a binary file on a web-based hosting service for git (i.e. GitHub) so I can easily see any changes made to it.
The binary file in question uses the common ASCII character encoding so that this binary
73 63 6F 70 65 20 68 75 72 72 72 20 69 6E 69 74 69 61 6C 69 7A 65 72 20 64 65 72 70 0D 0A 20 20 20 20 66 75 6E 63 74 69 6F 6E 20 64 65 72 70 20 74 61 6B 65 73 20 6E 6F 74 68 69 6E 67 20 72 65 74 75 72 6E 73 20 6E 6F 74 68 69 6E 67 0D 0A 20 20 20 20 20 20 20 20 63 61 6C 6C 20 53 65 74 53 74 61 72 74 4C 6F 63 50 72 69 6F 28 24 42 2C 24 41 2C 24 41 2C 4D 41 50 5F 4C 4F 43 5F 50 52 49 4F 5F 48 49 47 48 29 0D 0A 20 20 20 20 65 6E 64 66 75 6E 63 74 69 6F 6E 0D 0A 65 6E 64 73 63 6F 70 65
becomes this readable text (†)
scope hurrr initializer derp
function derp takes nothing returns nothing
call SetStartLocPrio($B,$A,$A,MAP_LOC_PRIO_HIGH)
endfunction
endscope
The problem is that services like GitHub will only show me the raw binary when I want to view the file in-browser (or have me download and open it in a text editor):
Right now, to have any changes made, I have to download the changed binary file, convert it to readable text, then use diff to see what changes have been made. This is tedious and loses the beautiful web interface that GitHub has.
So my question is this: Can I tell GitHub (or any equivalent service) to translate a binary file to readable text?
--
(†) For anyone interested in trivia, this is indeed vJass syntax for WarCraft III.

Find hash algorithm?

I'm trying to find out which method of hashing was used to generate certain strings. It's 32 bits (4 Bytes).
Found out it was DBJ v1. Any idea how to implement in C#? Thanks :)
Example Hashes:
00 02 B5 D5 - hash
30 - string
35 A0 FD 6A - hash
6F 66 66 65 6E 73 69 76 - string
05 7B DC 6C - hash
70 6C 61 79 65 72 20 75 73 65 64 20 6F 66 66 65 6E 73 69 76 65 20 6C 61 6E 67 75 61 67 65 - string
This is important so thank you!

How do I reassemble IP fragments with Perl's Net::Pcap::Reassemble?

I am new to Perl and trying to use the Net::Pcap::Reassemble - IP fragment reassembly for Net::Pcap. I am trying to reassemble TCP packets and "tie" the packets streams of interest and print the "tied" hex output for that data of interest. Here is the code below and and the printed output I get. The printed output is two separate Hex dumps (denoted by "Hex Payload:" string). It is apparent I am not calling Net::Pcap::Reassemble module correctly. The desire end output that I am trying to achieve is below, taken from the printed output. Can someone please point me into the right direction in using this module to achieve my desired output? Thank you.
my $user_data;
my $header;
my $packet;
my $err ='';
my $pcap = Net::Pcap::open_offline("./pcap", \$err) or die "can't open ./pcap...$err\n";
Net::Pcap::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::Reassemble::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::close($pcap);
my $ip;
my $tcp;
my $payload;
sub process_pkt
{
my ($user_data,$header, $packet) = #_;
$ip = NetPacket::IP->decode(eth_strip($packet));
$tcp = NetPacket::TCP->decode($ip->{data});
$payload = $tcp->{data};
my $hexPayload = hexdump(data => $payload, start_position => 0) if length $payload;
print "Hex Payload:". $hexPayload;
}
Output (this is what I currently get, but want to concatenate it together):
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:.*/*..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
Hex Payload: 0x0000 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0010 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0020 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0030 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0040 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0050 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0060 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0070 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0080 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0090 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep
Desired output (the above tied/concatenated together):
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:.*/*..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
0x0160 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0170 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0180 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0190 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0200 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0210 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0220 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0230 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0240 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0250 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep

You are trying to reassemble a network session, not a fragmented network packet. You should be using the module 'Net::Analysis'. It can, with some effort on your part, reassemble a complete network session. You will soon learn to hate pipe-lining.

Perl Net::Pcap hex dump packet session

I have the following information below being produced by the Net::Pcap module to print the payload of the packets of interest within a capture.
The data below is the excerpt of a Windows executable file being captured within Perl.
I would like to be able to capture all of the hex data output into one file or variable to assess the session data of the file download while retaining the integrity of the hex dump.
The problem I am having is for each packet being produced for the download of the file it obviously produces a hex dump output. This is easily seen by the output below by the string "Payload" I print per packet/hex dump output.
I want to tie all relevant data together for a given file download session. How can I do this in Perl?
Payload:HTTP/1.1 200 OK
Date: Fri, 15 Jun 2012 02:31:32 GMT
Server: Apache
Last-Modified: Sat, 10 Dec 2011 13:38:37 GMT
ETag: "dc44da-4d000-4b3bd04c7a2f1"
Accept-Ranges: bytes
Content-Length: 315392
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Content-Type: application/x-msdos-program
MZ<90>^#^C^#^#^#^D^#^#^#<FF><FF>^#^#<B8>^#^#^#^#^#^#^##^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#<F8>^#^#^#^N^_<BA>^N^#<B4> <CD>!<B8>^AL<CD>!This program cannot be run in DOS mode.^M
$^#^#^#^#^#^#^#4^TGmpu)>pu)>pu)>c}#>ru)>uyI>ru)>uy&>ku)>c}t>ru)><F3>}t>uu)>pu(>
u)>uyv><DA>u)><9C>~w>qu)>uys>qu)>Richpu)>^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#PE^#^#L^A^D^#^_R<E3>N^#^#^#^#^#^#^#^#<E0>^#^O^A^K^A^G
^#<80>^C^#^#<90>^A^#^#^#^#^#g^F^C^#^#^P^#^#^#<90>^C^#^#^##^#^#^P^#^#^#^P^#^#^D^#^#^#^#^#^#^#^D^#^#^#^#^#^#^#^# ^E^#^#^P^#^#^#^#^#^#^C^#^#^#^#^#^P^#^#^P^#^#^#^#^P^#^#^P^#^#^#^#^#^#^P^#^#^#^#^#^#^#^#^#^#^#<A8><91>^D^#P^#^#^#^#^#^E^#<C8>^T^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^##<91>^D^#H^#^#^#^#^#^#^#^#^#^#^#^#<90>^C^#<D0>^A^#^#^#^#^#^#^#^#
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#.text^#^#^#As^C^#^#^P^#^#^#<80>^C^#^#^P^#^#^#^#^#^#^#^#^#^#^#^#^#^# ^#^#`.rdata^#^#<C2>^K^A^#^#<90>^C^#^#^P^A^#^#<90>^C^#^#^#^#^#^#^#^#^#^#^#^#^##^#^##.data^#^#^#D]^#^#^#<A0>^D^#^#^P^#^#^#<A0>^D^#^#^#^#^#^#^#^#^#^#^#^#^##^#^#<C0>.rsrc^#^#^#<C8>^T^#^#^#^#^E^#^# ^#^#^#<B0>^D^#^#^#^#^#^#^#^#^#^#^#^#^##^#^##^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#Payload:^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^
Thank you Borodin for the suggestion. However, I don't know how to use this module correctly and it is apparent as I am still getting the same output. Here is the snippet of code I am using with this module and the printed hex. As you can see the printed hex is not tied together as one hex output but two seperate for the given TCP stream for which I want to tie together. Any help is appreciated.
my $user_data;
my $header;
my $packet;
my $err ='';
my $pcap = Net::Pcap::open_offline("./pcap", \$err) or die "can't open ./pcap...$err\n";
Net::Pcap::loop($pcap, -1, \&process_pkt, '');
Net::Pcap::Reassemble::loop($pcap, -1, \&rend_callback, '');
Net::Pcap::close($pcap);
my $ip;
my $tcp;
my $payload;
sub process_pkt
{
my ($user_data,$header, $packet) = #_;
$ip = NetPacket::IP->decode(eth_strip($packet));
$tcp = NetPacket::TCP->decode($ip->{data});
$payload = $tcp->{data};
my $hexPayload = hexdump(data => $payload, start_position => 0) if length $payload;
print "Hex Payload:". $hexPayload;
}
print output:
Hex Payload: 0x0000 : 47 45 54 20 2F 6D 61 63 2F 5F 62 61 73 65 5F 76 : GET./mac/_base_v
0x0010 : 31 2F 73 63 72 69 70 74 2F 6A 71 75 65 72 79 2D : 1/script/jquery-
0x0020 : 31 2E 36 2E 31 2E 6A 73 20 48 54 54 50 2F 31 2E : 1.6.1.js.HTTP/1.
0x0030 : 31 0D 0A 48 6F 73 74 3A 20 77 77 77 2E 6D 69 63 : 1..Host:.www.mic
0x0040 : 72 6F 73 6F 66 74 2E 63 6F 6D 0D 0A 55 73 65 72 : rosoft.com..User
0x0050 : 2D 41 67 65 6E 74 3A 20 4D 6F 7A 69 6C 6C 61 2F : -Agent:.Mozilla/
0x0060 : 35 2E 30 20 28 4D 61 63 69 6E 74 6F 73 68 3B 20 : 5.0.(Macintosh;.
0x0070 : 49 6E 74 65 6C 20 4D 61 63 20 4F 53 20 58 20 31 : Intel.Mac.OS.X.1
0x0080 : 30 2E 36 3B 20 72 76 3A 31 33 2E 30 29 20 47 65 : 0.6;.rv:13.0).Ge
0x0090 : 63 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 : cko/20100101.Fir
0x00A0 : 65 66 6F 78 2F 31 33 2E 30 0D 0A 41 63 63 65 70 : efox/13.0..Accep
0x00B0 : 74 3A 20 2A 2F 2A 0D 0A 41 63 63 65 70 74 2D 4C : t:./..Accept-L
0x00C0 : 61 6E 67 75 61 67 65 3A 20 65 6E 2D 75 73 2C 65 : anguage:.en-us,e
0x00D0 : 6E 3B 71 3D 30 2E 35 0D 0A 41 63 63 65 70 74 2D : n;q=0.5..Accept-
0x00E0 : 45 6E 63 6F 64 69 6E 67 3A 20 67 7A 69 70 2C 20 : Encoding:.gzip,.
0x00F0 : 64 65 66 6C 61 74 65 0D 0A 52 65 66 65 72 65 72 : deflate..Referer
0x0100 : 3A 20 68 74 74 70 3A 2F 2F 77 77 77 2E 6D 69 63 : :.http://www.mic
0x0110 : 72 6F 73 6F 66 74 2E 63 6F 6D 2F 6D 61 63 2F 72 : rosoft.com/mac/r
0x0120 : 65 6D 6F 74 65 2D 64 65 73 6B 74 6F 70 2D 63 6C : emote-desktop-cl
0x0130 : 69 65 6E 74 0D 0A 44 4E 54 3A 20 31 0D 0A 43 6F : ient..DNT:.1..Co
0x0140 : 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 2D 61 : nnection:.keep-a
0x0150 : 6C 69 76 65 0D 0A 0D 0A 00 00 00 00 00 00 00 00 : live............
Hex Payload: 0x0000 : 48 54 54 50 2F 31 2E 31 20 32 30 30 20 4F 4B 0D : HTTP/1.1.200.OK.
0x0010 : 0A 43 61 63 68 65 2D 43 6F 6E 74 72 6F 6C 3A 20 : .Cache-Control:.
0x0020 : 6D 61 78 2D 61 67 65 3D 39 30 30 0D 0A 43 6F 6E : max-age=900..Con
0x0030 : 74 65 6E 74 2D 54 79 70 65 3A 20 61 70 70 6C 69 : tent-Type:.appli
0x0040 : 63 61 74 69 6F 6E 2F 78 2D 6A 61 76 61 73 63 72 : cation/x-javascr
0x0050 : 69 70 74 0D 0A 43 6F 6E 74 65 6E 74 2D 45 6E 63 : ipt..Content-Enc
0x0060 : 6F 64 69 6E 67 3A 20 67 7A 69 70 0D 0A 4C 61 73 : oding:.gzip..Las
0x0070 : 74 2D 4D 6F 64 69 66 69 65 64 3A 20 57 65 64 2C : t-Modified:.Wed,
0x0080 : 20 30 38 20 4A 75 6E 20 32 30 31 31 20 31 38 3A : .08.Jun.2011.18:
0x0090 : 34 35 3A 34 39 20 47 4D 54 0D 0A 41 63 63 65 70 : 45:49.GMT..Accep

Take a look at Net::Pcap::Reassemble
This module performs reassembly of fragmented datagrams in libpcap
packet capture data returned by the Net::Pcap loop() function