I am trying to use mitmproxy to look at the traffic from my win32 schannel tls client. But when I try to use mitmproxy the following messages throw an "Bad HTTP request line" error with a leading 0 in the binary dump that does not exsist in the data that my client sends (I have checked with a little python server).
"CONNECT www.example.com:443 HTTP/1.0\r\n\r\n"
"HTTP/1.0 200 Connection established\r\n\r\n"
Send Tls Client Hello:
16 03 03 00 AC 01 00 00 A8 03 03 5F 80 1A 2D F6 2A 59 DE 18
69 F0 BB 3C 2D 2B 11 90 F8 8C A7 F9 D7 96 CD DC 32 88 02 22
11 90 6A 00 00 2A C0 2C C0 2B C0 30 C0 2F 00 9F 00 9E C0 24
C0 23 C0 28 C0 27 C0 0A C0 09 C0 14 C0 13 00 9D 00 9C 00 3D
00 3C 00 35 00 2F 00 0A 01 00 00 55 00 00 00 14 00 12 00 00
0F 77 77 77 2E 65 78 61 6D 70 6C 65 2E 63 6F 6D 00 0A 00 08
00 06 00 1D 00 17 00 18 00 0B 00 02 01 00 00 0D 00 1A 00 18
08 04 08 05 08 06 04 01 05 01 02 01 04 03 05 03 02 03 02 02
06 01 06 03 00 23 00 00 00 17 00 00 FF 01 00 01 00
Bad HTTP request line: b"\x00\x16\x03\x03\x00\xac\x01\x00\x00\xa8\x03\x03_\x80\x17\xbd\x1f\xf3\x8fO\xddy\xfb\xaaR\x1c\xeb\xe0sdD\xb7}|\xeb\xbes\xdf$3\xb6\xd9\ry\x00\x00*\xc0,\xc0+\xc00\xc0/\x00\x9f\x00\x9e\xc0$\xc0#\xc0(\xc0'\xc0"
Now my question: Is this just a lack of understanding in how proxys and tls work or an error from mitmproxy?
Related
I'm currently trying to extract the raw payload from an ICMP packet.
I've managed to trim it down to the format I like (without the first 5 characters on each line and without the ....... stuff).
Original format:
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 10 b4 00 00 00 00 50 4b 03 04 14 00 c.........PK....
0030 09 00 08 00 92 ac 88 51 e2 f5 38 a1 6d 70 03 00 .......Q..8.mp..
0040 94 72 03 00 08 00 1c 00 66 6c 61 67 2e 6a 70 67 .r......thing.jpg
0050 55 54 09 00 03 d3 e3 cf 5f e7 UT......_.
Scripts:
awk '{x="";x=substr($0,5,50);gsub(/ +/,"",x);print x}' nontrimmed.txt > raw.txt
tr -d "\n" < raw,txt > newraw.txt
Result:
cafe0000babedead0000beef08004500004c0001000040019b48c0a801c8b9f56302080010b400000000504b030414000900080092ac8851e2f538a16d7003009472030008001c00666c61672e6a70675554090003d3e3cf5fe7cafe0000babedead0000beef08004500004c0001000040019b48c0a801c8b9f5630208005b5000000000e3cf5f75780b000104e803000004e80300003bc....ect
However, I'd like to get a specific number of bytes every x characters - i.e this:
ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00
00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5
63 02 08 00 10 b4 00 00 00 00 50 4b 03 04 14 00
09 00 08 00 92 ac 88 51 e2 f5 38 a1 6d 70 03 00
94 72 03 00 08 00 1c 00 66 6c 61 67 2e 6a 70 67
55 54 09 00 03 d3 e3 cf 5f e7
Would become this:
504b030414000900080092ac8851e2f538a16d7003009472030008001c00666c61672e6a70675554090003d3e3cf5fe7
Instead of this:
cafe0000babedead0000beef08004500004c0001000040019b48c0a801c8b9f56302080010b400000000504b030414000900080092ac8851e2f538a16d7003009472030008001c00666c61672e6a70675554090003d3e3cf5fe7cafe0000babedead0000beef08004500004c0001000040019b48c0a801c8b9f5630208005b5000000000e3cf5f75780b000104e803000004e80300003bc....ect
But for multiple different ones of the same format:
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 10 b4 00 00 00 00 50 4b 03 04 14 00 c.........PK....
0030 09 00 08 00 92 ac 88 51 e2 f5 38 a1 6d 70 03 00 .......Q..8.mp..
0040 94 72 03 00 08 00 1c 00 66 6c 61 67 2e 6a 70 67 .r......flag.jpg
0050 55 54 09 00 03 d3 e3 cf 5f e7 UT......_.
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 5b 50 00 00 00 00 e3 cf 5f 75 78 0b c...[P......_ux.
0030 00 01 04 e8 03 00 00 04 e8 03 00 00 3b c1 7d b7 ............;.}.
0040 30 0b ce 53 1e 99 d2 3a 1b 83 4c 7c be cd ef fa 0..S...:..L|....
0050 54 86 4d 24 19 58 c5 a9 b1 4d T.M$.X...M
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 3e f4 00 00 00 00 dd 56 4c 00 11 bf c...>......VL...
0030 42 22 2a 52 86 75 01 0a e2 90 90 f5 2b ec d0 67 B"*R.u......+..g
0040 74 5a 17 70 05 b6 27 35 21 cf 98 fb a2 5e 82 a8 tZ.p..'5!....^..
0050 56 f9 05 05 3d 3e 80 3f 68 23 V...=>.?h#
Any ideas? Thanks!
Is this what you're trying to do?
$ awk -v OFS= '{$1=$NF=""; x=x $0} END{print substr(x,85)}' file
504b030414000900080092ac8851e2f538a16d7003009472030008001c00666c61672e6a70675554090003d3e3cf5fe7
The above was run against your "Original format" input file:
$ cat file
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 10 b4 00 00 00 00 50 4b 03 04 14 00 c.........PK....
0030 09 00 08 00 92 ac 88 51 e2 f5 38 a1 6d 70 03 00 .......Q..8.mp..
0040 94 72 03 00 08 00 1c 00 66 6c 61 67 2e 6a 70 67 .r......thing.jpg
0050 55 54 09 00 03 d3 e3 cf 5f e7 UT......_.
If your input file can contain multiple records then:
$ awk -v OFS= '{$1=$NF=""; $0=$0; x=x $0} !NF{print substr(x,85); x=""} END{print substr(x,85)}' file
504b030414000900080092ac8851e2f538a16d7003009472030008001c00666c61672e6a70675554090003d3e3cf5fe7
e3cf5f75780b000104e803000004e80300003bc17db7300bce531e99d23a1b834c7cbecdeffa54864d241958c5a9b14d
dd564c0011bf42222a528675010ae29090f52becd067745a177005b6273521cf98fba25e82a856f905053d3e803f6823
That second script was run against the block of 3 records under "But for multiple different ones of the same format:" at the end of your question but you didn't provide the expected output for it so idk if that's the expected output or not:
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 10 b4 00 00 00 00 50 4b 03 04 14 00 c.........PK....
0030 09 00 08 00 92 ac 88 51 e2 f5 38 a1 6d 70 03 00 .......Q..8.mp..
0040 94 72 03 00 08 00 1c 00 66 6c 61 67 2e 6a 70 67 .r......flag.jpg
0050 55 54 09 00 03 d3 e3 cf 5f e7 UT......_.
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 5b 50 00 00 00 00 e3 cf 5f 75 78 0b c...[P......_ux.
0030 00 01 04 e8 03 00 00 04 e8 03 00 00 3b c1 7d b7 ............;.}.
0040 30 0b ce 53 1e 99 d2 3a 1b 83 4c 7c be cd ef fa 0..S...:..L|....
0050 54 86 4d 24 19 58 c5 a9 b1 4d T.M$.X...M
0000 ca fe 00 00 ba be de ad 00 00 be ef 08 00 45 00 ..............E.
0010 00 4c 00 01 00 00 40 01 9b 48 c0 a8 01 c8 b9 f5 .L....#..H......
0020 63 02 08 00 3e f4 00 00 00 00 dd 56 4c 00 11 bf c...>......VL...
0030 42 22 2a 52 86 75 01 0a e2 90 90 f5 2b ec d0 67 B"*R.u......+..g
0040 74 5a 17 70 05 b6 27 35 21 cf 98 fb a2 5e 82 a8 tZ.p..'5!....^..
0050 56 f9 05 05 3d 3e 80 3f 68 23 V...=>.?h#
My managed process is suspected to have caused a BSOD at a client site. I received a full memory dump (i.e.: including kernel, physical pages only) - but still am not able to inspect my process' stacks.
After switching to my process context -
.process /p /r <MyProcAddress>
I see only -
1: kd> k
# ChildEBP RetAddr
00 b56e3b70 81f2aa5d nt!KeBugCheckEx+0x1e
01 b56e3b94 81e7b68d nt!PspCatchCriticalBreak+0x71
02 b56e3bc4 81e6dfd1 nt!PspTerminateAllThreads+0x2d
03 b56e3bf8 8d48159a nt!NtTerminateProcess+0xcd
WARNING: Stack unwind information not available. Following frames may be wrong.
04 b56e3c24 81c845e4 klif+0x7559a
05 b56e3c24 77da6bb4 nt!KiSystemServicePostCall
06 0262f34c 00220065 ntdll!KiFastSystemCallRet
07 0262f390 003e0022 0x220065
08 0262f394 0073003c 0x3e0022
09 0262f398 00730079 0x73003c
0a 0262f39c 006e003a 0x730079
0b 0262f3a0 006d0061 0x6e003a
0c 0262f3a4 00200065 0x6d0061
0d 0262f3a8 00610076 0x200065
0e 0262f3ac 0075006c 0x610076
0f 0262f3b0 003d0065 0x75006c
10 0262f3b4 00770022 0x3d0065
11 0262f3b8 006e0069 0x770022
12 0262f3bc 006f0077 0x6e0069
13 0262f3c0 00640072 0x6f0077
14 0262f3c4 00200022 0x640072
...
Which is natural for managed process. SOS extension does not work for kernel dumps.
Is there anything I can do to view the throwing managed stack? It was previously said to be 'much more difficult', but hopefully not impossible.
PS.
I'm aware of the presence of Kaspersky driver kilf.sys in the stack, and this is my personal suspect. But the question is more general - hopefully there's a way to understand what my process was doing at the time.
the stack as you posted is not correct
it appears to be overwritten or is a result of some other artefact
with such a stack details you will have a hard time deciphering
anything useful at all
the contents of stack converted to a printable range in english looks like this
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 00 3E 00 22 00 22 00 65 00 73 00 3C 00 3E 00 22 .>.".".e.s.<.>."
00000010 00 73 00 79 00 73 00 3C 00 6E 00 3A 00 73 00 79 .s.y.s.<.n.:.s.y
00000020 00 6D 00 61 00 6E 00 3A 00 20 00 65 00 6D 00 61 .m.a.n.:. .e.m.a
00000030 00 61 00 76 00 20 00 65 00 75 00 6C 00 61 00 76 .a.v. .e.u.l.a.v
00000040 00 3D 00 65 00 75 00 6C 00 77 00 22 00 3D 00 65 .=.e.u.l.w.".=.e
00000050 00 6E 00 69 00 77 00 22 00 6F 00 77 00 6E 00 69 .n.i.w.".o.w.n.i
00000060 00 64 00 72 00 6F 00 77 00 20 00 22 00 64 00 72 .d.r.o.w. .".d.r
try !analyze -v and see what is the bsod analysis results
I'm using Akka Streams to create a TCP server using akka.stream.scaladsl.TLS with client certificate authentication. I'm working on creating an echo server as a first proof of concept.
In the meantime, I'm new to Scala/Akka/Akka Streams and so I created a similar server and TCP client in Python to provide tooling in testing my work in Scala. The Python server/client are functional using client cert authentication. When connecting to the server, the client takes the following steps:
Creates and configures an SSLContext
Creates a socket using socket.create_connection()
Wraps the socket with the SSLContext using SSLContext.wrap_socket(). This creates the peer connection
Once connected, prints the server certificate
Infinite loop asking for input and sending each input to the server
I believe I have the server completed using Akka Streams and akka.stream.scaladsl.TLS, but when I attempt to connect using my Python client the client never gets past connecting to the peer using context.wrap_socket(sock, server_hostname=host). The server successfully binds the TCP connection and creates the corresponding IncomingConnection object. The client/server also never timeout (the client just sits awaiting the handshake?).
My biggest problem is that I see no information from my TLS BidiFlow, akka.stream.scaladsl.TLS. I have no idea what step in the handshake I'm stuck at, which makes troubleshooting very difficult.
Is there any way to output some information throughout the TLS handshake process? It seems as though all of the functionality is encapsulated and I don't know if there's any way to troubleshoot.
Otherwise, I'm attempting to troubleshoot with openssl and get the following:
bash$ openssl s_client -connect myserver.com:443 -state -debug
CONNECTED(00000003)
SSL_connect:before/connect initialization
write to 0x7fd914100080 [0x7fd915001000] (318 bytes => 318 (0x13E))
0000 - 16 03 01 01 39 01 00 01-35 03 03 e3 ff 5d fb 26 ....9...5....].&
0010 - 15 e3 32 89 37 e2 cb 95-f5 00 bd df 13 3d ae a6 ..2.7........=..
0020 - d7 37 db 4e 80 19 63 ad-d6 6c f1 00 00 98 cc 14 .7.N..c..l......
0030 - cc 13 cc 15 c0 30 c0 2c-c0 28 c0 24 c0 14 c0 0a .....0.,.(.$....
0040 - 00 a3 00 9f 00 6b 00 6a-00 39 00 38 ff 85 00 c4 .....k.j.9.8....
0050 - 00 c3 00 88 00 87 00 81-c0 32 c0 2e c0 2a c0 26 .........2...*.&
0060 - c0 0f c0 05 00 9d 00 3d-00 35 00 c0 00 84 c0 2f .......=.5...../
0070 - c0 2b c0 27 c0 23 c0 13-c0 09 00 a2 00 9e 00 67 .+.'.#.........g
0080 - 00 40 00 33 00 32 00 be-00 bd 00 45 00 44 c0 31 .#.3.2.....E.D.1
0090 - c0 2d c0 29 c0 25 c0 0e-c0 04 00 9c 00 3c 00 2f .-.).%.......<./
00a0 - 00 ba 00 41 c0 11 c0 07-c0 0c c0 02 00 05 00 04 ...A............
00b0 - c0 12 c0 08 00 16 00 13-c0 0d c0 03 00 0a 00 15 ................
00c0 - 00 12 00 09 00 ff 01 00-00 74 00 0b 00 04 03 00 .........t......
00d0 - 01 02 00 0a 00 3a 00 38-00 0e 00 0d 00 19 00 1c .....:.8........
00e0 - 00 0b 00 0c 00 1b 00 18-00 09 00 0a 00 1a 00 16 ................
00f0 - 00 17 00 08 00 06 00 07-00 14 00 15 00 04 00 05 ................
0100 - 00 12 00 13 00 01 00 02-00 03 00 0f 00 10 00 11 ................
0110 - 00 23 00 00 00 0d 00 26-00 24 06 01 06 02 06 03 .#.....&.$......
0120 - ef ef 05 01 05 02 05 03-04 01 04 02 04 03 ee ee ................
0130 - ed ed 03 01 03 02 03 03-02 01 02 02 02 03 ..............
SSL_connect:unknown state
At which point openssl just hangs.
The Akka TLS support uses the built in Java TLS support behind the scenes, so to get debug output for TLS you'll have to enable debugging for that. It can be done through passing a system property to the JVM when starting it like so -Djavax.net.debug=all
Ultimately I found that the ssl-config logging is very sparse and wasn't helpful to resolving my issue. It does provide some debugging information but not much. Much better for debugging the TLS handshake is to use the -Djavax.net.debug=all flag when running the JVM. However, even this provides mixed results. For example, the resulting error I received is that the server couldn't find a matching cipher suite. Eventually I resolved my issue by realizing that when creating the input streams for my keystore/truststore I was specifying my path incorrectly.
Note for anyone coming across this: if you specify your keystore and truststore incorrectly the resulting input streams will be null and SSLContext.init will happily use these and provide an error that is unrelated to the keystore/truststore! This was very difficult to troubleshoot due to the incorrect error handling in SSLContext.
I have a pcap file that contains the sendAuthenticationInfo message.
I try to parse the sccp layer from this packet using tshark
I tried the following:
tshark.exe -r filter.pcap -T fields -e sccp > parse.bin
I know what the result should be from parsing manually in wireshark, the result I get is much shorter and different from expected.
Original packet:
d4 c3 b2 a1 02 00 04 00 00 00 00 00 00 00 00 00
00 00 00 01 01 00 00 00 f0 52 7a 57 9a d7 00 00
ba 00 00 00 ba 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 08 00 45 00 00 ac 00 00 00 00 10 84
ab cc 7f 00 00 01 7f 00 00 01 0b 58 0b 59 00 00
48 23 01 f3 c7 60 00 03 00 8c 00 00 1f 7a 00 01
00 00 00 00 00 03 01 00 01 01 00 00 00 7c 02 10
00 72 00 00 04 3a 00 00 02 4e 03 02 00 04 09 80
03 0e 19 0b 12 06 00 11 04 79 52 14 02 10 07 0b
12 07 00 12 04 44 87 92 97 01 08 44 62 42 48 04
00 00 00 01 6b 1e 28 1c 06 07 00 11 86 05 01 01
01 a0 11 60 0f 80 02 07 80 a1 09 06 07 04 00 00
01 00 0e 03 6c 1a a1 18 02 01 01 02 01 38 30 10
80 08 24 05 01 77 03 84 35 f8 02 01 01 83 01 00
00 00
expected result:
09 80 03 0e 19 0b 12 06 00 11 04 79 52 14 02 10
07 0b 12 07 00 12 04 44 87 92 97 01 08 44 62 42
48 04 00 00 00 01 6b 1e 28 1c 06 07 00 11 86 05
01 01 01 a0 11 60 0f 80 02 07 80 a1 09 06 07 04
00 00 01 00 0e 03 6c 1a a1 18 02 01 01 02 01 38
30 10 80 08 24 05 01 77 03 84 35 f8 02 01 01 83
01 00
result I got:
73 63 63 70 0d 0a
Upgraded Webbit to 0.4.6 to use the new SSL support but immediately realized that all wss:// handshakes are failing silently and I don't have any errors to show for it. Chrome only reports a "success" for a response without a HTTP code or any other headers. I check server logs and it doesn't even register an "open" event.
The catch here is that any ws:// connection works great. So what could be possible problems and how can I get an error out of it? Could it be something wrong with the java keystore and SSL handshake?
Edit
I was able to find an openSSL command for a test handshake. Here's the output:
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
Edit 2
I realized I could debug this further
CONNECTED(0000016C)
SSL_connect:before/connect initialization
write to 0x1f57750 [0x1f6a730] (210 bytes => 210 (0xD2))
0000 - 16 03 01 00 cd 01 00 00-c9 03 01 4f 6b 8d 68 63 ...........Ok.hc
0010 - 99 06 08 30 93 2a 42 88-f8 f1 c4 c5 dc 89 71 0b ...0.*B.......q.
0020 - b6 04 42 4e 11 79 b4 76-6c f7 66 00 00 5c c0 14 ..BN.y.vl.f..\..
0030 - c0 0a 00 39 00 38 00 88-00 87 c0 0f c0 05 00 35 ...9.8.........5
0040 - 00 84 c0 12 c0 08 00 16-00 13 c0 0d c0 03 00 0a ................
0050 - c0 13 c0 09 00 33 00 32-00 9a 00 99 00 45 00 44 .....3.2.....E.D
0060 - c0 0e c0 04 00 2f 00 96-00 41 00 07 c0 11 c0 07 ...../...A......
0070 - c0 0c c0 02 00 05 00 04-00 15 00 12 00 09 00 14 ................
0080 - 00 11 00 08 00 06 00 03-00 ff 01 00 00 44 00 0b .............D..
0090 - 00 04 03 00 01 02 00 0a-00 34 00 32 00 01 00 02 .........4.2....
00a0 - 00 03 00 04 00 05 00 06-00 07 00 08 00 09 00 0a ................
00b0 - 00 0b 00 0c 00 0d 00 0e-00 0f 00 10 00 11 00 12 ................
00c0 - 00 13 00 14 00 15 00 16-00 17 00 18 00 19 00 23 ...............#
00d2 - <SPACES/NULS>
SSL_connect:SSLv2/v3 write client hello A
read from 0x1f57750 [0x1f6fc90] (7 bytes => 0 (0x0))
12488:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:.\ssl\s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 210 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
Edit 3
Ok I've nailed the problem to Webbit initialization, but it doesn't throw any errors so I could use some input to getting getResourceAsStream functioning properly. Here's how the server is initialized:
def startWebSocketServer(webSocketHandler:PartialFunction[WebSocketEvent, Unit]) {
val webServer = WebServers.createWebServer(port)
try {
webServer.setupSsl(getClass.getResourceAsStream("/keystore"), "webbit")
webServer.add("/", new WebSocketEventAdapter(webSocketHandler))
webServer.start
} catch {
case e => e.printStackTrace()
}
}
Unfortunately setupSsl won't output any information, and I've tried both what I thought would be the path and inserting a fake path. In either case, I can't get an error. How on earth would I properly locate the path? Thanks!
The OMFG Answer
In a hysterical twist of fate, I found the problem. This particular issue took up 48 hours of my time, but the cause was not even code related and a funny miscommunication.
So as it turns out, another developer had copied our websocket code into a new file he was working on for development. All this time we were trying to debug code in a file that wasn't even executing at run-time. So upon further investigation we scrolled to the bottom of a very long and different file, and found the webbit init code and excuted it perfectly.
Moral of the story: don't commit an incomplete file to the master branch and point everyone there for debugging ;)