My application sends an email to an Exchange mail server, mail server is configured with a third party application where it routes email to agent and agent replies to that email. Application reads agent reply from the mailbox which is used to send the email.
Email sending code is below;
Message mimeMessage = new MimeMessage(session);
mimeMessage.setFrom(new InternetAddress(from));
mimeMessage.addRecipient(Message.RecipientType.TO, new InternetAddress(to));
mimeMessage.setSubject(subject);
mimeMessage.setContent(emailText,"text/plain");
mimeMessage.setReplyTo(replyToAddress);
Transport.send(mimeMessage);
This works perfectly. When agent reply is received, Application read it as;
if (message.isMimeType("multipart/MIXED")) {
logger.info("Email MIME Type is: multipart/MIXED");
MimeMultipart multipart =(MimeMultipart)message.getContent();
logger.info("Content type = "+multipart.getContentType());
int count = multipart.getCount();
}
The content type is "multipart/mixed" but the count is 0 means there are no parts in this emails.
I need to set System property,
System.setProperty("mail.mime.multipart.allowempty", "true");
if it is not set, multipart.getCount() throws "missingBoundryException".
Why it is so ?
I can see that the agent's reply is not empty.
The email was sent with content type as text/plain, why reply type is multipart/mixed?
Is this due to any invalid formatting of email by third party application, what is the workaround?
Below is the snap of agent reply.
Below is the raw MIME content,
Received: from sociaminer.host (192.168.1.29) by thirdpartHost
(192.168.1.53) with Microsoft SMTP Server (TLS) id 14.1.218.12; Thu, 19 Jan
2017 17:06:26 +0500
To: hafiz <hafiz#bla.bla>
Message-ID: <hassan.MESSAGEID#bla.bla>
In-Reply-To: <CF72F94#bla.bla>
References: <CF72F945A#bla.bla>
Subject: Re: 1122+50
Content-Type: multipart/mixed;
boundary="----=_Part_127_14151461.1484827604583"
From: <reply#bla.bla>
Return-Path: reply#bla.bla
Date: Thu, 19 Jan 2017 17:06:26 +0500
X-MS-Exchange-Organization-AuthSource: bla.bla
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 06
X-Originating-IP: [SocialMinerIP]
MIME-Version: 1.0
------=_Part_127_14151461.1484827604583
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">Reply to 50<br>
<blockquote><hr>
<b>From:</b> hafiz <hafiz#bla.bla><br><b>Sent:</b> Thursday, January 19, 2017 5:05 PM<br><b>To:</b> testing2 <testing2#bla.bla><br><b>Subject:</b> 1122+50<br>
<html dir="ltr">
<head>
<style type="text/css" id="owaParaStyle"></style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Testing 50</div>
</body>
</html>
</blockquote>
------=_Part_127_14151461.1484827604583--
JavaMail debug output looks like below,
DEBUG: setDebug: JavaMail version 1.4.7
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: disable AUTH=PLAIN
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: trying to connect to host "Echange IP", port 143, isSSL false
* OK The Microsoft Exchange IMAP4 service is ready.
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 LOGINDISABLED STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: protocolConnect login, host=192.168.1.53, user=hafiz#bla.bla, password=<non-null>
A1 STARTTLS
A1 OK Begin TLS negotiation now.
A2 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A2 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTHENTICATE NTLM command trace suppressed
DEBUG NTLM: type 1 message: 4E 54 4C 4D 53 53 50 00 01 00 00 00 03 A2 00 00 00 00 00 00 23 00 00 00 03 00 03 00 20 00 00 00 31 39 32
DEBUG NTLM: type 3 message: 4E 54 4C 4D 53 53 50 00 03 00 00 00 18 00 18 00 68 00 00 00 18 00 18 00 80 00 00 00 00 00 00 00 40 00 00 00 22 00 22 00 40 00 00 00 06 00 06 00 62 00 00 00 00 00 00 00 98 00 00 00 01 82 00 00 68 00 61 00 66 00 69 00 7A 00 40 00 65 00 66 00 6C 00 61 00 62 00 2E 00 6C 00 6F 00 63 00 61 00 6C 00 31 00 39 00 32 00 3B 5E 2B 86 67 49 E3 01 C9 9E F2 CA ED 54 21 11 81 89 94 C6 EC E0 26 E3 BA DB E7 5A F4 CA 28 17 7C 0E 8A 08 18 B5 5A 4E 72 4F C5 7F 52 64 FA 76
DEBUG IMAP: AUTHENTICATE NTLM command result: A3 OK AUTHENTICATE completed.
A4 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+
A4 OK CAPABILITY completed.
DEBUG IMAP: AUTH: NTLM
DEBUG IMAP: AUTH: GSSAPI
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: connection available -- size: 1
A5 SELECT INBOX
* 40 EXISTS
* 0 RECENT
* FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
* OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
* OK [UNSEEN 39] Is the first unseen message
* OK [UIDVALIDITY 436] UIDVALIDITY value
* OK [UIDNEXT 46] The next unique identifier value
A5 OK [READ-WRITE] SELECT completed.
A6 SEARCH UNSEEN ALL
* SEARCH 39
A6 OK SEARCH completed.
A7 SEARCH UNSEEN ALL
* SEARCH 39
A7 OK SEARCH completed.
main INFO emailToSms.EmailReader - 1 unread emails read from inbox.
A8 STORE 39 +FLAGS (\Seen)
* 39 FETCH (FLAGS (\Seen))
A8 OK STORE completed.
A9 FETCH 39 (BODY.PEEK[HEADER])
* 39 FETCH (BODY[HEADER] {851}
MIME-Version: 1.0
Received: from HOST (IP) by HOST
(192.168.1.53) with Microsoft SMTP Server (TLS) id 14.1.218.12; Thu, 19 Jan
2017 17:06:26 +0500
To: hafiz <hafiz#bla.bla>
Message-ID: <hassan.B69E3DD110000159000004A73F57FEE3.1484827604448.cisco-ccp#bla.bla>
In-Reply-To: <CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>
References: <CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>
Subject: Re: 1122+50
Content-Type: multipart/mixed;
boundary="----=_Part_127_14151461.1484827604583"
From: <testing2#bla.bla>
Return-Path: testing2#bla.bla
Date: Thu, 19 Jan 2017 17:06:26 +0500
X-MS-Exchange-Organization-AuthSource: Expert.bla.bla
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 06
X-Originating-IP: [IP]
)
A9 OK FETCH completed.
A10 FETCH 39 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 39 FETCH (ENVELOPE ("Thu, 19 Jan 2017 17:06:26 +0500" "Re: 1122+50" ((NIL NIL "testing2" "bla.bla")) NIL NIL (("hafiz" NIL "hafiz" "bla.bla")) NIL NIL "<CF72F945A1ED2E438A53A11DA9415F65A0E981#Expert.bla.bla>" "<hassan.B69E3DD110000159000004A73F57FEE3.1484827604448.cisco-ccp#bla.bla>") INTERNALDATE "19-Jan-2017 17:06:26 +0500" RFC822.SIZE 1250)
A10 OK FETCH completed.
A11 FETCH 39 (BODYSTRUCTURE)
* 39 FETCH (BODYSTRUCTURE ("multipart" "mixed" ("boundary" "----=_Part_127_14151461.1484827604583") NIL NIL 7BIT 0 NIL NIL NIL NIL))
A11 OK FETCH completed.
DEBUG IMAP: IMAPProtocol noop
A12 NOOP
A12 OK NOOP completed.
This is a bug in Microsoft Exchange. Report this bug to Microsoft and upgrade to a newer version or newer service pack if possible in case they've already fixed it.
Exchange is returning the BODYSTRUCTURE information for the message as if it were a single part message when in fact it is a multipart message. This is a violation of the IMAP protocol spec.
You can use the workaround in the JavaMail FAQ.
Also, you might want to upgrade to a newer version of JavaMail - 1.4.7 is pretty old, the current version is 1.5.6.
Related
I'm trying to setup a WiFi Access Point with a Raspberry Pi 3B+ having 802.11w enabled.
Kernel version: Linux efb-ap-0 4.19.66-Re4son-v7+ #1 SMP Sun Aug 18 22:25:39 AEST 2019 armv7l GNU/Linux
Driver: brcmfmac
hostapd (Deb package): 2:2.9-1 armel
During the 4-Way Handshake, wpa_supplicant immediatly disconnects at the 3/4 msg, with following logs:
wlan0: WPA: IE in 3/4 msg does not match with IE in Beacon/ProbeResp (src=b8:27:eb:3b:3f:0e)
WPA: RSN IE in Beacon/ProbeResp - hexdump(len=28): 30 1a 01 00 00 0f ac 04 01 00 00 0f ac 04 01 00 00 0f ac 06 c0 00 00 00 00 0f ac 06
WPA: RSN IE in 3/4 msg - hexdump(len=26): 30 18 01 00 00 0f ac 04 01 00 00 0f ac 04 02 00 00 0f ac 02 00 0f ac 06 c0 00
Comparing 3/4 msg hexdump and Beacon hexdump via Wireshark shows that the Beacon contains the following additional fields that are not in the 3/4 msg: PMKID Count (0x00 00)+ PMKID List + Group Management Cipher Suite
(0x00 0f ac 06).
Why is the 3/4 msg not matching the Beacon ? Is this an issue in hostapd ? in driver ? in hostapd<->driver communication ?
Thanks for any information about that.
You can find below the hostapd.conf content:
interface=wlan0
driver=nl80211
logger_syslog=-1
logger_syslog_level=2
auth_algs=1
wpa_pairwise=CCMP
rsn_pairwise=CCMP
wpa=2
hw_mode=g
ieee80211w=2
ssid=XXXXXXXXXX
channel=1
wpa_key_mgmt=WPA-PSK-SHA256
wpa_passphrase=XXXXXXXXXX
And the wpa_supplicant.conf used to connect:
ctrl_interface=DIR=/var/run/
network={
ssid="XXXXXXXX"
proto=RSN
scan_ssid=1
key_mgmt=WPA-PSK-SHA256
pairwise=CCMP
psk="XXXXXXXX"
ieee80211w=2
}
Note: this thread is a duplicate from a message I had posted on hostap mailing list for which I didn't have answer: http://lists.infradead.org/pipermail/hostap/2019-November/040764.html
I'm using Akka Streams to create a TCP server using akka.stream.scaladsl.TLS with client certificate authentication. I'm working on creating an echo server as a first proof of concept.
In the meantime, I'm new to Scala/Akka/Akka Streams and so I created a similar server and TCP client in Python to provide tooling in testing my work in Scala. The Python server/client are functional using client cert authentication. When connecting to the server, the client takes the following steps:
Creates and configures an SSLContext
Creates a socket using socket.create_connection()
Wraps the socket with the SSLContext using SSLContext.wrap_socket(). This creates the peer connection
Once connected, prints the server certificate
Infinite loop asking for input and sending each input to the server
I believe I have the server completed using Akka Streams and akka.stream.scaladsl.TLS, but when I attempt to connect using my Python client the client never gets past connecting to the peer using context.wrap_socket(sock, server_hostname=host). The server successfully binds the TCP connection and creates the corresponding IncomingConnection object. The client/server also never timeout (the client just sits awaiting the handshake?).
My biggest problem is that I see no information from my TLS BidiFlow, akka.stream.scaladsl.TLS. I have no idea what step in the handshake I'm stuck at, which makes troubleshooting very difficult.
Is there any way to output some information throughout the TLS handshake process? It seems as though all of the functionality is encapsulated and I don't know if there's any way to troubleshoot.
Otherwise, I'm attempting to troubleshoot with openssl and get the following:
bash$ openssl s_client -connect myserver.com:443 -state -debug
CONNECTED(00000003)
SSL_connect:before/connect initialization
write to 0x7fd914100080 [0x7fd915001000] (318 bytes => 318 (0x13E))
0000 - 16 03 01 01 39 01 00 01-35 03 03 e3 ff 5d fb 26 ....9...5....].&
0010 - 15 e3 32 89 37 e2 cb 95-f5 00 bd df 13 3d ae a6 ..2.7........=..
0020 - d7 37 db 4e 80 19 63 ad-d6 6c f1 00 00 98 cc 14 .7.N..c..l......
0030 - cc 13 cc 15 c0 30 c0 2c-c0 28 c0 24 c0 14 c0 0a .....0.,.(.$....
0040 - 00 a3 00 9f 00 6b 00 6a-00 39 00 38 ff 85 00 c4 .....k.j.9.8....
0050 - 00 c3 00 88 00 87 00 81-c0 32 c0 2e c0 2a c0 26 .........2...*.&
0060 - c0 0f c0 05 00 9d 00 3d-00 35 00 c0 00 84 c0 2f .......=.5...../
0070 - c0 2b c0 27 c0 23 c0 13-c0 09 00 a2 00 9e 00 67 .+.'.#.........g
0080 - 00 40 00 33 00 32 00 be-00 bd 00 45 00 44 c0 31 .#.3.2.....E.D.1
0090 - c0 2d c0 29 c0 25 c0 0e-c0 04 00 9c 00 3c 00 2f .-.).%.......<./
00a0 - 00 ba 00 41 c0 11 c0 07-c0 0c c0 02 00 05 00 04 ...A............
00b0 - c0 12 c0 08 00 16 00 13-c0 0d c0 03 00 0a 00 15 ................
00c0 - 00 12 00 09 00 ff 01 00-00 74 00 0b 00 04 03 00 .........t......
00d0 - 01 02 00 0a 00 3a 00 38-00 0e 00 0d 00 19 00 1c .....:.8........
00e0 - 00 0b 00 0c 00 1b 00 18-00 09 00 0a 00 1a 00 16 ................
00f0 - 00 17 00 08 00 06 00 07-00 14 00 15 00 04 00 05 ................
0100 - 00 12 00 13 00 01 00 02-00 03 00 0f 00 10 00 11 ................
0110 - 00 23 00 00 00 0d 00 26-00 24 06 01 06 02 06 03 .#.....&.$......
0120 - ef ef 05 01 05 02 05 03-04 01 04 02 04 03 ee ee ................
0130 - ed ed 03 01 03 02 03 03-02 01 02 02 02 03 ..............
SSL_connect:unknown state
At which point openssl just hangs.
The Akka TLS support uses the built in Java TLS support behind the scenes, so to get debug output for TLS you'll have to enable debugging for that. It can be done through passing a system property to the JVM when starting it like so -Djavax.net.debug=all
Ultimately I found that the ssl-config logging is very sparse and wasn't helpful to resolving my issue. It does provide some debugging information but not much. Much better for debugging the TLS handshake is to use the -Djavax.net.debug=all flag when running the JVM. However, even this provides mixed results. For example, the resulting error I received is that the server couldn't find a matching cipher suite. Eventually I resolved my issue by realizing that when creating the input streams for my keystore/truststore I was specifying my path incorrectly.
Note for anyone coming across this: if you specify your keystore and truststore incorrectly the resulting input streams will be null and SSLContext.init will happily use these and provide an error that is unrelated to the keystore/truststore! This was very difficult to troubleshoot due to the incorrect error handling in SSLContext.
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I've encountered a serious bug in OpenSSL 1.0.1 on Ubuntu 12.04:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=665452
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=666051 <- dated Oct 3 2012!
The gist of it is that I'm able to connect to some servers but not others. Connecting to google works:
openssl s_client -connect mail.google.com:443 -debug -state -msg -CAfile /etc/ssl/certs/ca-certificates.crt
...
Protocol : TLSv1.1
Cipher : ECDHE-RSA-RC4-SHA
Session-ID: 94DB1AC8531115C501434B16A5E9B735722768581778E4FEA4D9B19988551397
Session-ID-ctx:
Master-Key: 8694BF510CD7568CBAB39ECFD32D115C511529871F3030B67A4F7AEAF957D714D3E94E4CE6117F686C975EFF21FB8708
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 100800 (seconds)
TLS session ticket:
0000 - fb 52 d6 d3 3c a8 75 e1-1f 1d f6 23 ab ce 55 44 .R..<.u....#..UD
0010 - 27 bf ad c4 7a 0d 83 c8-48 59 48 4b 39 bb 3c c7 '...z...HYHK9.<.
0020 - 01 1e ad b3 13 de 65 d4-e8 ea e4 35 89 83 55 8e ......e....5..U.
0030 - e4 d5 9f 60 58 51 33 9b-83 34 b9 35 3d 46 cb a3 ...`XQ3..4.5=F..
0040 - 35 7b 48 5d 7b 86 5c d5-a1 14 9d 8c 3e 93 eb fb 5{H]{.\.....>...
0050 - ac 78 75 72 9b d2 bc 67-f2 fa 5b 75 80 a6 31 d8 .xur...g..[u..1.
0060 - 71 15 85 7f 55 4d dc fb-b0 b5 33 db 6d 36 8c c6 q...UM....3.m6..
0070 - e8 f9 54 7a 29 69 87 2c-dd f3 c5 cf 26 55 6f 6e ..Tz)i.,....&Uon
0080 - 45 73 7a 1d e4 b3 be b2-92 3f 0b ed c4 1c a5 24 Esz......?.....$
0090 - 3c f0 ca a5 <...
Start Time: 1354063165
Timeout : 300 (sec)
Verify return code: 0 (ok)
But connecting to facebook doesn't:
openssl s_client -connect graph.facebook.com:443 -debug -state -msg -CAfile /etc/ssl/certs/ca-certificates.crt -cipher SRP-AES-256-CBC-SHA
CONNECTED(00000003)
SSL_connect:before/connect initialization
write to 0xddd2c0 [0xddd340] (64 bytes => 64 (0x40))
0000 - 16 03 01 00 3b 01 00 00-37 03 02 50 b5 5d 75 42 ....;...7..P.]uB
0010 - c2 78 55 49 b5 2e de 4f-00 a6 a8 d5 cf 10 92 44 .xUI...O.......D
0020 - 28 62 34 d6 61 5e 88 c3-68 8b 96 00 00 04 c0 20 (b4.a^..h......
0030 - 00 ff 02 01 00 00 09 00-23 00 00 00 0f 00 01 01 ........#.......
>>> TLS 1.1 [length 003b]
01 00 00 37 03 02 50 b5 5d 75 42 c2 78 55 49 b5
2e de 4f 00 a6 a8 d5 cf 10 92 44 28 62 34 d6 61
5e 88 c3 68 8b 96 00 00 04 c0 20 00 ff 02 01 00
00 09 00 23 00 00 00 0f 00 01 01
SSL_connect:unknown state
read from 0xddd2c0 [0xde28a0] (7 bytes => 7 (0x7))
0000 - 15 03 02 00 02 02 28 ......(
SSL3 alert read:fatal:handshake failure
<<< TLS 1.1 [length 0002]
02 28
SSL_connect:error in unknown state
140581179446944:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:724:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 64 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
The facebook connection either hangs after the client sends its hello buffer and never receives the server hello response, or returns with an error code if I pass in a cipher it recognizes. This happens with both -tls1 and -ssl3. I've tried every parameter to openssl I can think of.
apt-cache showpkg openssl
...
Provides:
1.0.1-4ubuntu5.5 -
1.0.1-4ubuntu5.3 -
1.0.1-4ubuntu3 -
I've also tried every parameter I can think of to curl but with no success, because it uses openssl under the hood.
I'm concerned that Ubuntu can't establish secure connections (an astounding statement, I realize). After two solid days of beating my head against this problem, I'm basically praying at this point that someone knows a workaround. I'm considering a downgrade to OpenSSL 1.0.0 or using libcurl4-dev with gnutls-dev instead. Both solutions leave a rotten taste in my mouth. Thanks in advance for any help you can provide.
P.S. all of this work is so that my server can interface with external https REST APIs. I consider this a fundamental requirement in any webserver today, no excuses.
UPDATE: Here is my output without passing a cipher. It doesn't matter if I pass -CAfile or not either:
openssl s_client -connect graph.facebook.com:443 -debug -state -msg -CAfile /etc/ssl/certs/ca-certificates.crt
CONNECTED(00000003)
SSL_connect:before/connect initialization
write to 0x14ed1a0 [0x1515bf0] (226 bytes => 226 (0xE2))
0000 - 16 03 01 00 dd 01 00 00-d9 03 02 50 b6 39 78 6a ...........P.9xj
0010 - 24 95 8e dc 62 19 37 4b-ab 77 b8 66 cd 48 ba a2 $...b.7K.w.f.H..
0020 - a1 2a f8 1d f8 c9 5d fb-9d db 84 00 00 66 c0 14 .*....]......f..
0030 - c0 0a c0 22 c0 21 00 39-00 38 00 88 00 87 c0 0f ...".!.9.8......
0040 - c0 05 00 35 00 84 c0 12-c0 08 c0 1c c0 1b 00 16 ...5............
0050 - 00 13 c0 0d c0 03 00 0a-c0 13 c0 09 c0 1f c0 1e ................
0060 - 00 33 00 32 00 9a 00 99-00 45 00 44 c0 0e c0 04 .3.2.....E.D....
0070 - 00 2f 00 96 00 41 c0 11-c0 07 c0 0c c0 02 00 05 ./...A..........
0080 - 00 04 00 15 00 12 00 09-00 14 00 11 00 08 00 06 ................
0090 - 00 03 00 ff 02 01 00 00-49 00 0b 00 04 03 00 01 ........I.......
00a0 - 02 00 0a 00 34 00 32 00-0e 00 0d 00 19 00 0b 00 ....4.2.........
00b0 - 0c 00 18 00 09 00 0a 00-16 00 17 00 08 00 06 00 ................
00c0 - 07 00 14 00 15 00 04 00-05 00 12 00 13 00 01 00 ................
00d0 - 02 00 03 00 0f 00 10 00-11 00 23 00 00 00 0f 00 ..........#.....
00e0 - 01 01 ..
>>> TLS 1.1 [length 00dd]
01 00 00 d9 03 02 50 b6 39 78 6a 24 95 8e dc 62
19 37 4b ab 77 b8 66 cd 48 ba a2 a1 2a f8 1d f8
c9 5d fb 9d db 84 00 00 66 c0 14 c0 0a c0 22 c0
21 00 39 00 38 00 88 00 87 c0 0f c0 05 00 35 00
84 c0 12 c0 08 c0 1c c0 1b 00 16 00 13 c0 0d c0
03 00 0a c0 13 c0 09 c0 1f c0 1e 00 33 00 32 00
9a 00 99 00 45 00 44 c0 0e c0 04 00 2f 00 96 00
41 c0 11 c0 07 c0 0c c0 02 00 05 00 04 00 15 00
12 00 09 00 14 00 11 00 08 00 06 00 03 00 ff 02
01 00 00 49 00 0b 00 04 03 00 01 02 00 0a 00 34
00 32 00 0e 00 0d 00 19 00 0b 00 0c 00 18 00 09
00 0a 00 16 00 17 00 08 00 06 00 07 00 14 00 15
00 04 00 05 00 12 00 13 00 01 00 02 00 03 00 0f
00 10 00 11 00 23 00 00 00 0f 00 01 01
SSL_connect:unknown state
Why are you passing -cipher SRP-AES-256-CBC-SHA when connecting to graph.facebook.com? Facebook certainly doesn't support SRP: http://srp.stanford.edu/.
Does it work if you don't pass that?
Also, can you give the IP address that you're getting? With 69.171.229.17, I can reproduce that exact ClientHello (modulo the nonce and with RC4-SHA are the only cipher save the SCSV) and I get a successful handshake.
Lastly, have you tried doing over an SSH tunnel to somewhere else? Sadly, when deploying TLS features in Chrome we have repeatedly found networking hardware that breaks TLS connections. (Although I can't think of a case where -ssl3 wouldn't fix it unless the hardware was actively trying to censor connections.)
Setting the MTU on my Ubuntu box from 1500 to 1496 (due to one of our firewalls being set too low) allows me to receive a response from the server without having to reboot (be sure to call ifconfig first and write down your original MTU which should be 1500):
sudo ifconfig eth0 mtu 1496
I discovered my MTU by pinging with successively larger buffers (add 28 bytes for UDP header):
Fails for 1472 + 28 = 1500:
ping -s 1472 facebook.com
PING facebook.com (66.220.158.16) 1472(1500) bytes of data.
...
Works for 1468 + 28 = 1496:
ping -s 1468 facebook.com
PING facebook.com (69.171.229.16) 1468(1496) bytes of data.
1476 bytes from www-slb-ecmp-06-prn1.facebook.com (69.171.229.16): icmp_req=1 ttl=242 time=30.0 ms
...
With 1496 I'm now able to curl to facebook.com:
curl -v https://facebook.com
* About to connect() to facebook.com port 443 (#0)
* Trying 66.220.152.16... connected
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-SHA
* Server certificate:
* subject: C=US; ST=California; L=Palo Alto; O=Facebook, Inc.; CN=www.facebook.com
* start date: 2012-06-21 00:00:00 GMT
* expire date: 2013-12-31 23:59:59 GMT
* subjectAltName: facebook.com matched
* issuer: O=VeriSign Trust Network; OU=VeriSign, Inc.; OU=VeriSign International Server CA - Class 3; OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
* SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: facebook.com
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Location: https://www.facebook.com/
< Content-Type: text/html; charset=utf-8
< X-FB-Debug: 3vAg1O5OG9hB/EWC+gk1Kl3WLJRGmlQDaEodirWb+i0=
< Date: Wed, 28 Nov 2012 19:52:25 GMT
< Connection: keep-alive
< Content-Length: 0
<
* Connection #0 to host facebook.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
I personally think that MTU should have absolutely nothing to do with what the user sees at the stream level with TCP so I hope the OpenSSL folks fix this. I also wish that someone would invent an automagic bug submitter for bugs that are profoundly widespread and time-sucking.
Out of curiosity, how much data is actually sent when establishing a connection to a port (using Java sockets). Is it the size of a Socket object? SocketConnection object?
Your understanding of TCP network connections seems to conflate them with electrical circuits. (Understandable, given your background.)
From a physical standpoint, there's no such thing as a connection, only data packets. Through the TCP protocol, two devices agree to establish a logical (that is, software) connection. A connection is established by a client first sending data to the remote host (SYN), the server sending data back to the client (SYN-ACK), and the client sending a final acknowledgement (ACK). All of this negotation necessarily consumes bandwidth, and when you terminate a connection, you must negotiate a completely new connection to begin sending data again.
For example, I'll connect from my machine to a local web server, 192.168.1.2:80.
First, my machine sends a TCP SYN. This sends 66 bytes over the wire: (headers deliniated with |)
0000 00 24 8c a9 4c b4 00 1e 68 66 20 79 08 00|45 00 .$..L... hf y..E.
0010 00 34 53 98 40 00 80 06 00 00 c0 a8 01 0b c0 a8 .4S.#... ........
0020 01 02|36 0a 00 50 09 ef 3a a7 00 00 00 00 80 02 ..6..P.. :.......
0030 20 00 50 c8 00 00 02 04 05 b4 01 03 03 02 01 01 .P..... ........
0040 04 02 ..
The first 14 bytes are the Ethernet frame, specifying that this packet's destination MAC address. This will typically be an upstream router, but in this case, the server happens to be on the same switch, so it's the machine's MAC address, 00:24:8c:a9:4c:b4. The source (my) MAC follows, along with the payload type (IP, 0x0800). The next 20 bytes are the IPv4 headers, followed by 32 bytes of TCP headers.
The server responds with a 62-byte SYN-ACK:
0000 00 1e 68 66 20 79 00 24 8c a9 4c b4 08 00|45 00 ..hf y.$ ..L...E.
0010 00 30 69 b9 40 00 80 06 0d b1 c0 a8 01 02 c0 a8 .0i.#... ........
0020 01 0b|00 50 36 0a d3 ae 9a 73 09 ef 3a a8 70 12 ...P6... .s..:.p.
0030 20 00 f6 9d 00 00 02 04 05 b4 01 01 04 02 ....... ......
Again, 14 bytes of Ethernet headers, 20 bytes of IP headers, and 28 bytes of TCP headers. I send an ACK:
0000 00 24 8c a9 4c b4 00 1e 68 66 20 79 08 00|45 00 .$..L... hf y..E.
0010 00 28 53 9a 40 00 80 06 00 00 c0 a8 01 0b c0 a8 .(S.#... ........
0020 01 02|36 0a 00 50 09 ef 3a a8 d3 ae 9a 74 50 10 ..6..P.. :....tP.
0030 fa f0 83 78 00 00 ...x..
14 + 20 + 20 = 54 bytes over the wire (this is the smallest possible TCP packet size, by the way – the SYN and SYN-ACK packets were larger because they included options).
This adds up to 182 bytes over the wire to establish a connection; now I can begin sending actual data to the server:
0000 00 24 8c a9 4c b4 00 1e 68 66 20 79 08 00 45|00 .$..L... hf y..E.
0010 01 9d 53 9d 40 00 80 06 00 00 c0 a8 01 0b c0 a8 ..S.#... ........
0020 01 02|36 0a 00 50 09 ef 3a a8 d3 ae 9a 74 50 18 ..6..P.. :....tP.
0030 fa f0 84 ed 00 00|47 45 54 20 2f 20 48 54 54 50 ......GE T / HTTP
0040 2f 31 2e 31 0d 0a 48 6f 73 74 3a 20 66 73 0d 0a /1.1..Ho st: fs..
...
14 Ethernet + 20 IP + 20 TCP + data, in this case HTTP.
So we can see that it costs ~182 bytes to establish a TCP connection, and an additional 162-216 bytes to terminate a TCP connection (depending on whether a 4-way FIN ACK FIN ACK or more common 3-way FIN FIN-ACK ACK termination handshake is used), adding up to nearly 400 bytes to "pulse" a connection by disconnecting and reconnecting.
Compared to the 55 bytes you'd use to send one byte of data over an already established connection, this is obviously wasteful.
What you want to do is establish one connection and then send data as-needed. If you're really bandwidth constrained, you could use UDP (which requires no handshaking at all and has an overhead of only 14 Ethernet + 20 IP + 8 UDP bytes per packet), but then you face the problem of using an unreliable transport, and having to handle lost packets on your own.
The minimum size of a TCP packet is 40 bytes. It takes three exchanged packets to create a connection, two from the client and one from the server, and four more to close it, two in each direction. The last packet in a connect exchange can also contain data, as can the first in the close exchange in each direction, which can amortize it a bit, as can combining the outgoing FIN and ACK as per #josh3736's comment below.
Upgraded Webbit to 0.4.6 to use the new SSL support but immediately realized that all wss:// handshakes are failing silently and I don't have any errors to show for it. Chrome only reports a "success" for a response without a HTTP code or any other headers. I check server logs and it doesn't even register an "open" event.
The catch here is that any ws:// connection works great. So what could be possible problems and how can I get an error out of it? Could it be something wrong with the java keystore and SSL handshake?
Edit
I was able to find an openSSL command for a test handshake. Here's the output:
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
Edit 2
I realized I could debug this further
CONNECTED(0000016C)
SSL_connect:before/connect initialization
write to 0x1f57750 [0x1f6a730] (210 bytes => 210 (0xD2))
0000 - 16 03 01 00 cd 01 00 00-c9 03 01 4f 6b 8d 68 63 ...........Ok.hc
0010 - 99 06 08 30 93 2a 42 88-f8 f1 c4 c5 dc 89 71 0b ...0.*B.......q.
0020 - b6 04 42 4e 11 79 b4 76-6c f7 66 00 00 5c c0 14 ..BN.y.vl.f..\..
0030 - c0 0a 00 39 00 38 00 88-00 87 c0 0f c0 05 00 35 ...9.8.........5
0040 - 00 84 c0 12 c0 08 00 16-00 13 c0 0d c0 03 00 0a ................
0050 - c0 13 c0 09 00 33 00 32-00 9a 00 99 00 45 00 44 .....3.2.....E.D
0060 - c0 0e c0 04 00 2f 00 96-00 41 00 07 c0 11 c0 07 ...../...A......
0070 - c0 0c c0 02 00 05 00 04-00 15 00 12 00 09 00 14 ................
0080 - 00 11 00 08 00 06 00 03-00 ff 01 00 00 44 00 0b .............D..
0090 - 00 04 03 00 01 02 00 0a-00 34 00 32 00 01 00 02 .........4.2....
00a0 - 00 03 00 04 00 05 00 06-00 07 00 08 00 09 00 0a ................
00b0 - 00 0b 00 0c 00 0d 00 0e-00 0f 00 10 00 11 00 12 ................
00c0 - 00 13 00 14 00 15 00 16-00 17 00 18 00 19 00 23 ...............#
00d2 - <SPACES/NULS>
SSL_connect:SSLv2/v3 write client hello A
read from 0x1f57750 [0x1f6fc90] (7 bytes => 0 (0x0))
12488:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:.\ssl\s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 210 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
Edit 3
Ok I've nailed the problem to Webbit initialization, but it doesn't throw any errors so I could use some input to getting getResourceAsStream functioning properly. Here's how the server is initialized:
def startWebSocketServer(webSocketHandler:PartialFunction[WebSocketEvent, Unit]) {
val webServer = WebServers.createWebServer(port)
try {
webServer.setupSsl(getClass.getResourceAsStream("/keystore"), "webbit")
webServer.add("/", new WebSocketEventAdapter(webSocketHandler))
webServer.start
} catch {
case e => e.printStackTrace()
}
}
Unfortunately setupSsl won't output any information, and I've tried both what I thought would be the path and inserting a fake path. In either case, I can't get an error. How on earth would I properly locate the path? Thanks!
The OMFG Answer
In a hysterical twist of fate, I found the problem. This particular issue took up 48 hours of my time, but the cause was not even code related and a funny miscommunication.
So as it turns out, another developer had copied our websocket code into a new file he was working on for development. All this time we were trying to debug code in a file that wasn't even executing at run-time. So upon further investigation we scrolled to the bottom of a very long and different file, and found the webbit init code and excuted it perfectly.
Moral of the story: don't commit an incomplete file to the master branch and point everyone there for debugging ;)