Frozen on "Opening STARTTLS connection to ‘smtp.qq.com:587’...done" - emacs

I referenced the answer to question sending-emails-with-emacs24-via-smtp-with-gnutls-and-extra-arguments
and set my config as
(setq user-mail-address "abst.proc.do#qq.com")
(setq user-full-name "abst.proc.do")
(require 'smtpmail)
(require 'starttls)
(setq message-send-mail-function 'smtpmail-send-it)
(defun gnutls-available-p ()
"Function redefined in order not to use built-in GnuTLS support"
nil)
(setq starttls-gnutls-program "gnutls-cli")
(setq starttls-use-gnutls t)
(setq smtpmail-stream-type 'starttls)
(setq smtpmail-smtp-server "smtp.qq.com")
(setq smtpmail-smtp-service 587) ;;587(starttls) or 465(tls/ssl)
(setq starttls-extra-arguments '("--priority" "NORMAL:%COMPAT"))
Then try to send a testing mail as
From: abst.proc.do#qq.com (abst.proc.do)
To: abst.proc.do#qq.com
Subject: Testing
Date: Thu, 23 Jan 2020 07:11:44 +0800
Message-ID: <877e1j2ij3.fsf#qq.com>
--text follows this line--
Testing
It freeze in the minibuffer with prompt
Opening STARTTLS connection to ‘smtp.qq.com:587’...done
and got a messages
Saving file /home/me/*message*-20200123-071130...
the mail was not sent.
The appreciate stmp service was checked and token placed in .authinfo
What's the problem with my configuration?
Machine: Ubuntu 19.10 Emacs 26.3
The error trace:
220 newxmesmtplogicsvrszc2.qq.com XMail Esmtp QQ Mail Server.
250-newxmesmtplogicsvrszc2.qq.com
250-PIPELINING
250-SIZE 73400320
250-STARTTLS
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN
250-MAILCOMPRESS
250 8BITMIME
Process smtpmail deleted
220 newxmesmtplogicsvrsza3.qq.com XMail Esmtp QQ Mail Server.
250-newxmesmtplogicsvrsza3.qq.com
250-PIPELINING
250-SIZE 73400320
250-STARTTLS
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN
250-MAILCOMPRESS
250 8BITMIME
220 Ready to start TLS from 106.39.174.180 to newxmesmtplogicsvrsza3.qq.com.
*** Starting TLS handshake
- Certificate type: X.509
- Got a certificate list of 3 certificates.
- Certificate[0] info:
- subject `CN=*.mail.qq.com,O=Tencent Technology (Shenzhen) Company Limited,L=shenzhen,ST=guangdong,C=CN', issuer `CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE', serial 0x41110235e23a37f3ead2dc5e, RSA key 2048 bits, signed using RSA-SHA256, activated `2019-11-11 10:32:16 UTC', expires `2020-06-03 04:00:33 UTC', pin-sha256="3gHtBaJCDCS2wo6ho7kKEE6rUA1YXM3A6R1XiFejm48="
Public Key ID:
sha1:5df67c6c87174f49828307072b0922d6da77cab0
sha256:de01ed05a2420c24b6c28ea1a3b90a104eab500d585ccdc0e91d578857a39b8f
Public Key PIN:
pin-sha256:3gHtBaJCDCS2wo6ho7kKEE6rUA1YXM3A6R1XiFejm48=
- Certificate[1] info:
- subject `CN=GlobalSign Organization Validation CA - SHA256 - G2,O=GlobalSign nv-sa,C=BE', issuer `CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE', serial 0x040000000001444ef04247, RSA key 2048 bits, signed using RSA-SHA256, activated `2014-02-20 10:00:00 UTC', expires `2024-02-20 10:00:00 UTC', pin-sha256="IQBnNBEiFuhj+8x6X8XLgh01V9Ic5/V3IRQLNFFc7v4="
- Certificate[2] info:
- subject `CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE', issuer `CN=GlobalSign Root CA,OU=Root CA,O=GlobalSign nv-sa,C=BE', serial 0x040000000001154b5ac394, RSA key 2048 bits, signed using RSA-SHA1 (broken!), activated `1998-09-01 12:00:00 UTC', expires `2028-01-28 12:00:00 UTC', pin-sha256="K87oWBWM9UZfyddvDfoxL+8lpNyoUB2ptGtn0fv6G2Q="
- Status: The certificate is trusted.
- Description: (TLS1.2)-(ECDHE-SECP256R1)-(RSA-SHA512)-(AES-128-GCM)
- Session ID: CB:34:DB:07:B8:3D:32:D8:13:BD:9B:4E:6B:5D:0A:72:1A:85:83:20:68:BB:70:8A:5A:43:15:51:3E:7D:19:BD
- Options: safe renegotiation,

I have had a number of ongoing problems with starttls, i.e. open-tls-stream, when it is using openssl in Emacs -- it is extremely hard to debug, and upgrading either emacs or openssl can cause connections to hang as you have observed -- and this usually happens far enough apart to allow one to completely forget that it happened before and how to debug and fix it again.
I've changed the definition of tls-end-of-info to solve this:
;; xxx gnutls-cli is broken, at least when used with emacs-23.3
;; (xxx "-no_ssl2" might still be required for older openssl?)
;; n.b. the "-crlf" is now necessary for Gmail, but not Cyrus
;;
(setq tls-program '("openssl s_client -connect %h:%p -ign_eof"))
(setq starttls-use-gnutls nil) ; XXX requires security/starttls be installed!!!
;;
;; XXX as of OpenSSL 1.1.1a 20 Nov 2018 there's a new ending to the noise
;; c_client prints before real data starts.
;;
;; (XXX debugging this was HARD! No clues -- emacs just "hung" without responding)
;;
(setq tls-end-of-info
"\\(^\s*Verify return code: .+
---
\\|^ Extended master secret: .+
---
\\|^- Simple Client Mode:
\\(
\\|^\\*\\*\\* Starting TLS handshake
\\)*\\)")
To help with the debugging I use a slightly hacked up version of open-tls-stream. You can find it all at my ~/.emacs.el file

Related

Error when trying to send mail over smtp with Office 365 account

An application, which obviously uses Chilkat plugin, does not correctly send E-mails anymore using an Office 365 account. After a migration from on-premise exchange to Microsoft 365, the account and server settings were changed at end of March this year. Furthermore, client authentication needed to be disabled for the mailbox. After that, sending E-mails from the application had worked. Now, it again doesn't. Yesterday, one E-mail could successfully be sent after a computer restart - the next ones failed again.
I'm stuck to find the cause of the error and also I don't know, what data exactly is sent from the application during the connection, which makes the analysis more difficult.
I found this post mentioning that entries for TLS 1.0 need to be in the registry. Those were missing, but there were entries for TLS 1.2, so I didn't assume this is an issue, espccially because the entries concerning cryptography (also mentioned in the post) did already exist. In the meantime, I have added them nonetheless.
Below is the Chilkat Log, showing that - after the server is ready - the clientHandshake2 fails with error code 0x2746.
The support person of the application he couldn't find any problem and asked concerning that error code - what i assumed he should be able to examine. When I talked to him on the phone, I found out that he doesn't know more about Chilkat than I do. He mentioned an interesting point, which need to be double checked: As far as he was informed from the user directly - or how he had understood it - the error doesn't show up, if the user chooses another "Mandant" (client) in the application and sends E-mails from there.
Can someone help identify the cause of the error?
SendEmail:
DllDate: May 25 2017
ChilkatVersion: 9.5.0.68
UnlockPrefix: WERNERMAILQ
Architecture: Little Endian; 32-bit
Language: Visual C++ 2017 (32-bit)
VerboseLogging: 0
sendEmailInner:
renderToMime:
createEmailForSending:
Auto-generating Message-ID
--createEmailForSending
renderToMime: Elapsed time: 0 millisec
--renderToMime
sendMimeInner:
ensureSmtpSession:
ensureSmtpConnection:
SmtpHost: smtp.office365.com
SmtpPort: 587
SmtpUsername: edith.beer#halten.ch
SmtpSsl: 0
StartTLS: 1
smtpConnect:
smtpHostname: smtp.office365.com
smtpPort: 587
connectionIsReady:
Need new SMTP connection
--connectionIsReady
smtpSocketConnect:
socketOptions:
SO_SNDBUF: 262144
SO_RCVBUF: 4194304
TCP_NODELAY: 1
SO_KEEPALIVE: 1
--socketOptions
--smtpSocketConnect
smtpGreeting:
readSmtpResponse:
SmtpCmdResp: 220 AM0PR02CA0163.outlook.office365.com Microsoft ESMTP MAIL Service ready at Mon, 23 Aug 2021 11:51:12 +0000
--readSmtpResponse
--smtpGreeting
startTLS:
sendCmdToSmtp:
SmtpCmdSent: EHLO EWGH-N-FV<CRLF>
--sendCmdToSmtp
readSmtpResponse:
SmtpCmdResp: 250-AM0PR02CA0163.outlook.office365.com Hello [213.221.219.37]
SmtpCmdResp: 250-SIZE 157286400
SmtpCmdResp: 250-PIPELINING
SmtpCmdResp: 250-DSN
SmtpCmdResp: 250-ENHANCEDSTATUSCODES
SmtpCmdResp: 250-STARTTLS
SmtpCmdResp: 250-8BITMIME
SmtpCmdResp: 250-BINARYMIME
SmtpCmdResp: 250-CHUNKING
SmtpCmdResp: 250 SMTPUTF8
--readSmtpResponse
sendCmdToSmtp:
SmtpCmdSent: STARTTLS<CRLF>
--sendCmdToSmtp
readSmtpResponse:
SmtpCmdResp: 220 2.0.0 SMTP server ready
--readSmtpResponse
clientHandshake:
clientHandshake2:
readHandshakeMessages:
WindowsError: Eine vorhandene Verbindung wurde vom Remotehost geschlossen.
WindowsErrorCode: 0x2746
maxToReceive: 5
Failed to receive data on the TCP socket
Failed to read beginning of SSL/TLS record.
b: 0
dbSize: 0
nReadNBytes: 0
idleTimeoutMs: 30000
--readHandshakeMessages
--clientHandshake2
--clientHandshake
Client handshake failed. (1)
connectionClosed: 0
Failed to establish TLS connection.
--startTLS
--smtpConnect
--ensureSmtpConnection
--ensureSmtpSession
--sendMimeInner
--sendEmailInner
Failed.
--SendEmail
--ChilkatLog
23.08.2021 13:48:27 -F- Beim Versand der Lohnabrechnung per Email an Mitarbeiter(in) Nadine Aeschlimann ist ein Fehler aufgetreten!
You are using an old version of Chilkat. Try testing with the latest version.
In the meantime we could solve it.
The reason was that the outgoing requests were blocked by the firewall. Strangely, a few of them went through, but not all. This is the reason why a network issue was initially not in the focus of our investigation.

How come that all mitmproxy-CA-certs have the same hash value of 8bbe0e8d?

I am using mitmproxy on two different machines. The versions are
Mitmproxy: 4.0.4
Python: 3.8.2
OpenSSL: OpenSSL 1.1.1f 31 Mar 2020
Platform: Linux-5.4.0-33-generic-x86_64-with-glibc2.29
and
Mitmproxy: 5.1.1
Python: 3.8.2
OpenSSL: OpenSSL 1.1.1g 21 Apr 2020
Platform: macOS-10.15.4-x86_64-i386-64bit
One thing, that really puzzles me: How come that the ca-certificates have the same hash value?
AFAIK, the key-pair of which the public one will go into the cert are created dynamically on installation or whenever someones deletes them in .mitmproxy.
But interestingly, both have the same hash value:
> openssl x509 -in .mitmproxy/mitmproxy-ca-cert.pem -noout -hash
8bbe0e8d
This applies actually to a few more installations i did in order to investigate this behaviour.
when I have a look at the modulus, all look different, so this seems to indicate that the keys are in fact different. But AFAIK the hash key is calculated over the key/modulus as well so I would like to know, why I find the same hash value 8bbe0e8d everywhere?
This leads to some interesting side effect:
E.g. on linux the root ca certs are usually in /etc/ssl/certs.
They are deployed there with a sensible name and in addition there is a a symlink pointing to that file.
The name of the symlink ist the hash-value of the cert followed by a sequence number. This is generated by the c_rehash tool of openssl. Normally there are no hash collisions and all sequence numbers are 0.
But in the case of a linux system containing ca-certs of two different mitmproxy-instances we have something like this
# ls -l /etc/ssl/certs/ | grep mitm
lrwxrwxrwx 1 root root 21 Jun 1 21:45 8bbe0e8d.0 -> mitmproxy-systema-ca-cert.pem
-rw-r--r-- 1 root root 1318 Jun 1 21:44 mitmproxy-systema-ca-cert.pem
lrwxrwxrwx 1 root root 21 Jun 1 22:34 8bbe0e8d.1 -> mitmproxy-systemb-ca-cert.pem
-rw-r--r-- 1 root root 1318 Jun 1 22:34 mitmproxy-systemb-ca-cert.pem
So to repeat my question:
Why is the hash value always 8bbe0e8d?
Is - contrary to my belief - the modulus not calculated into the hash
value?
are all mitmproxies using the same keys (which I hope they don´t)?
Any different reason?
Thanks in advance
Christian
Please find the relevant openssl output below:
>> openssl x509 -in mitmproxy-systema-ca-cert.pem -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 15904961119818 (0xe77298ec64a)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=mitmproxy, O=mitmproxy
Validity
Not Before: May 24 12:28:31 2020 GMT
Not After : May 26 12:28:31 2023 GMT
Subject: CN=mitmproxy, O=mitmproxy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:d3:60:2a:3a:8b:bc:9a:2c:fb:da:90:33:fa:a1:
a9:7a:96:52:e4:73:56:c8:c8:7f:8b:f8:ab:4b:e0:
55:2e:05:75:5b:55:4d:6d:58:b0:82:56:23:ac:ee:
ba:d4:4e:b0:ab:8e:52:25:2c:12:ef:fe:23:3b:f5:
0d:26:9e:cd:1e:d5:7c:5a:7b:e0:c6:6b:af:b6:b0:
cd:d1:5b:8b:12:ea:a1:d4:15:78:37:84:f2:d1:48:
61:7b:9b:c6:ec:e3:2c:41:32:72:15:15:d1:5f:7b:
87:01:40:86:6a:cf:5f:2a:0f:19:71:c5:37:08:94:
8c:4d:18:af:5d:5d:80:89:46:e9:04:23:f4:e7:84:
4e:97:ee:81:91:07:c8:18:5e:eb:64:3a:47:9e:c1:
29:50:2c:27:c7:80:35:b9:d6:ec:61:91:de:23:af:
04:7d:0c:e8:43:32:52:09:c9:34:ba:fd:98:51:ef:
78:13:2c:83:4a:e9:31:6e:d8:53:6b:12:79:44:e9:
5b:70:7a:b5:79:2e:00:a9:9f:53:f3:2f:c6:75:b0:
90:1b:00:b4:50:21:5e:fe:b5:a3:36:18:c5:42:cd:
fc:d5:33:e4:1b:c1:26:12:04:05:95:e5:99:7c:23:
2a:ea:de:f3:45:7e:3b:9d:e9:56:a5:83:07:61:e9:
dd:19
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:TRUE
Netscape Cert Type:
SSL CA
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication, E-mail Protection, Time Stamping, Microsoft Individual Code Signing, Microsoft Commercial Code Signing, Microsoft Trust List Signing, Microsoft Server Gated Crypto, Microsoft Encrypted File System, Netscape Server Gated Crypto
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Subject Key Identifier:
03:9C:EC:D3:BD:2A:C4:A8:E8:23:04:F2:AD:69:C9:2E:CF:CE:85:85
Signature Algorithm: sha256WithRSAEncryption
6d:98:36:7e:e6:2f:54:7d:7f:0a:9b:85:d5:ef:e6:c3:c7:df:
c8:c4:1b:3e:78:51:ee:48:8c:c2:0c:ac:8f:89:67:06:22:3f:
fe:05:f4:17:2b:1c:23:0e:53:1f:0e:7b:23:e1:fe:ac:9c:52:
ac:13:11:06:be:00:55:13:36:1a:47:22:29:41:79:f8:ca:8e:
2b:5a:26:57:b6:26:80:da:7d:ac:10:5f:53:b9:00:e4:d9:ed:
51:04:52:af:d0:7c:33:ce:24:6f:eb:06:d0:49:c6:da:71:25:
64:fe:66:0b:29:90:99:7f:b7:c4:3d:f9:17:5b:24:21:ae:7c:
3f:b1:33:b5:af:64:e2:bc:44:d4:41:df:35:ca:45:8a:08:61:
7a:76:8b:4c:7c:23:80:1d:87:97:29:98:78:a3:38:bf:3c:8d:
5c:79:43:64:95:77:4d:50:cb:a2:17:fd:cf:f9:9f:42:b4:d5:
20:8a:2c:12:af:9d:cd:34:b4:be:53:ad:e4:d8:33:bb:fe:7d:
a1:57:e6:cf:b7:a6:30:a2:3d:f6:8f:4d:4b:f6:2b:cc:19:df:
d2:d5:6e:25:d2:92:13:db:60:f9:6c:e4:bc:09:56:07:5a:30:
6f:89:67:1a:e4:93:52:bd:f6:89:ab:1f:71:17:6b:78:97:69:
05:46:a6:2f
-----BEGIN CERTIFICATE-----
MIIDoTCCAomgAwIBAgIGDncpjsZKMA0GCSqGSIb3DQEBCwUAMCgxEjAQBgNVBAMM
CW1pdG1wcm94eTESMBAGA1UECgwJbWl0bXByb3h5MB4XDTIwMDUyNDEyMjgzMVoX
DTIzMDUyNjEyMjgzMVowKDESMBAGA1UEAwwJbWl0bXByb3h5MRIwEAYDVQQKDAlt
aXRtcHJveHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDTYCo6i7ya
LPvakDP6oal6llLkc1bIyH+L+KtL4FUuBXVbVU1tWLCCViOs7rrUTrCrjlIlLBLv
/iM79Q0mns0e1Xxae+DGa6+2sM3RW4sS6qHUFXg3hPLRSGF7m8bs4yxBMnIVFdFf
e4cBQIZqz18qDxlxxTcIlIxNGK9dXYCJRukEI/TnhE6X7oGRB8gYXutkOkeewSlQ
LCfHgDW51uxhkd4jrwR9DOhDMlIJyTS6/ZhR73gTLINK6TFu2FNrEnlE6VtwerV5
LgCpn1PzL8Z1sJAbALRQIV7+taM2GMVCzfzVM+QbwSYSBAWV5Zl8Iyrq3vNFfjud
6Valgwdh6d0ZAgMBAAGjgdAwgc0wDwYDVR0TAQH/BAUwAwEB/zARBglghkgBhvhC
AQEEBAMCAgQweAYDVR0lBHEwbwYIKwYBBQUHAwEGCCsGAQUFBwMCBggrBgEFBQcD
BAYIKwYBBQUHAwgGCisGAQQBgjcCARUGCisGAQQBgjcCARYGCisGAQQBgjcKAwEG
CisGAQQBgjcKAwMGCisGAQQBgjcKAwQGCWCGSAGG+EIEATAOBgNVHQ8BAf8EBAMC
AQYwHQYDVR0OBBYEFAOc7NO9KsSo6CME8q1pyS7PzoWFMA0GCSqGSIb3DQEBCwUA
A4IBAQBtmDZ+5i9UfX8Km4XV7+bDx9/IxBs+eFHuSIzCDKyPiWcGIj/+BfQXKxwj
DlMfDnsj4f6snFKsExEGvgBVEzYaRyIpQXn4yo4rWiZXtiaA2n2sEF9TuQDk2e1R
BFKv0HwzziRv6wbQScbacSVk/mYLKZCZf7fEPfkXWyQhrnw/sTO1r2TivETUQd81
ykWKCGF6dotMfCOAHYeXKZh4ozi/PI1ceUNklXdNUMuiF/3P+Z9CtNUgiiwSr53N
NLS+U63k2DO7/n2hV+bPt6Ywoj32j01L9ivMGd/S1W4l0pIT22D5bOS8CVYHWjBv
iWca5JNSvfaJqx9xF2t4l2kFRqYv
-----END CERTIFICATE-----
>> openssl x509 -in mitmproxy-systemb-ca-cert.pem -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 15891076851956 (0xe73edfda8f4)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=mitmproxy, O=mitmproxy
Validity
Not Before: May 8 10:48:05 2020 GMT
Not After : May 10 10:48:05 2023 GMT
Subject: CN=mitmproxy, O=mitmproxy
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:d4:27:ef:99:12:9b:84:9d:82:a7:d1:96:e6:fe:
14:cf:a5:1a:d5:95:f5:1f:b3:25:fc:10:df:1a:f1:
20:4a:a5:e9:e9:b9:20:ba:d3:c2:88:e9:cb:fe:66:
43:5e:4a:1d:9c:39:f4:a8:64:50:51:f6:18:0b:f2:
a2:b3:da:1d:a5:0d:01:c5:bd:c0:6c:b7:a7:25:cd:
6d:d7:21:2b:ba:a8:35:b6:a4:a3:33:0d:15:8d:44:
8e:bb:70:d6:1a:9b:c2:21:09:f9:70:fc:42:8c:d6:
a9:1b:d2:d1:0c:4b:03:f2:44:ca:c7:bf:8f:8b:e2:
fe:0c:ff:99:fe:61:f2:8f:6e:26:ae:ec:60:6c:ff:
ec:51:db:3e:3c:3e:a9:32:38:61:13:52:8e:40:15:
b0:8d:f7:7b:b8:d9:11:84:d6:dc:bd:9e:12:58:5c:
03:13:d6:73:6e:95:84:5f:8d:21:72:bb:17:27:a7:
19:b4:00:43:7b:bc:2e:f2:d9:8a:68:53:0d:de:bc:
03:6c:f8:78:c9:e6:43:1f:45:1e:b0:d0:7d:3b:a7:
cc:05:f2:cb:b1:5f:9c:5f:7f:ee:f3:4e:94:99:28:
33:6f:65:eb:24:a2:44:f1:22:13:a7:71:cd:88:15:
c3:14:77:a2:3c:dc:59:6c:10:81:0f:f1:89:ef:90:
1d:b5
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints: critical
CA:TRUE
Netscape Cert Type:
SSL CA
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication, E-mail Protection, Time Stamping, Microsoft Individual Code Signing, Microsoft Commercial Code Signing, Microsoft Trust List Signing, Microsoft Server Gated Crypto, Microsoft Encrypted File System, Netscape Server Gated Crypto
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Subject Key Identifier:
FE:50:10:81:42:BA:C2:85:01:CB:D2:B4:2E:FF:F1:B3:CD:B2:63:16
Signature Algorithm: sha256WithRSAEncryption
00:d0:fe:58:df:07:90:b9:03:25:b9:0c:6d:37:e4:65:aa:0f:
f9:d4:ea:9a:42:b7:3e:0f:8f:d3:1e:c4:26:03:ff:57:5b:6f:
3d:36:fb:cd:61:4f:4a:5a:20:71:5e:96:25:b3:d2:31:4b:da:
ec:6c:6e:30:e9:0f:77:5b:fe:34:95:5d:31:2a:bf:53:b9:f4:
94:98:5c:fa:b9:c5:27:1a:7e:51:2e:dd:75:f5:c6:51:f7:8d:
69:66:77:9c:e6:0f:7c:79:1a:2f:ca:be:16:9e:45:3f:4b:ff:
49:d8:5d:37:5f:d5:2c:f4:cd:bd:06:fd:09:b0:7b:4b:2b:21:
99:40:24:0a:f6:5f:c3:9c:2f:58:f6:60:b6:b4:3c:b6:89:43:
a6:be:a0:4a:9b:d4:2d:06:b3:2c:b3:eb:c6:18:5a:e4:b1:2b:
f7:b3:7a:a6:41:96:1e:09:19:39:37:25:e0:2c:7a:31:aa:bf:
f8:1a:c2:76:9b:32:30:b7:20:28:ea:63:a9:f7:16:ba:4d:23:
a5:90:7c:0f:31:b9:cd:f8:77:64:8f:28:5f:b8:10:64:4d:08:
f8:6a:9c:45:6f:c7:28:2e:4c:2c:34:09:ef:57:ed:c6:0e:c3:
6d:db:a4:de:8c:72:30:2d:59:8d:c1:e1:2c:6d:29:89:d5:9d:
86:c3:fb:65
-----BEGIN CERTIFICATE-----
MIIDoTCCAomgAwIBAgIGDnPt/aj0MA0GCSqGSIb3DQEBCwUAMCgxEjAQBgNVBAMM
CW1pdG1wcm94eTESMBAGA1UECgwJbWl0bXByb3h5MB4XDTIwMDUwODEwNDgwNVoX
DTIzMDUxMDEwNDgwNVowKDESMBAGA1UEAwwJbWl0bXByb3h5MRIwEAYDVQQKDAlt
aXRtcHJveHkwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDUJ++ZEpuE
nYKn0Zbm/hTPpRrVlfUfsyX8EN8a8SBKpenpuSC608KI6cv+ZkNeSh2cOfSoZFBR
9hgL8qKz2h2lDQHFvcBst6clzW3XISu6qDW2pKMzDRWNRI67cNYam8IhCflw/EKM
1qkb0tEMSwPyRMrHv4+L4v4M/5n+YfKPbiau7GBs/+xR2z48PqkyOGETUo5AFbCN
93u42RGE1ty9nhJYXAMT1nNulYRfjSFyuxcnpxm0AEN7vC7y2YpoUw3evANs+HjJ
5kMfRR6w0H07p8wF8suxX5xff+7zTpSZKDNvZeskokTxIhOncc2IFcMUd6I83Fls
EIEP8YnvkB21AgMBAAGjgdAwgc0wDwYDVR0TAQH/BAUwAwEB/zARBglghkgBhvhC
AQEEBAMCAgQweAYDVR0lBHEwbwYIKwYBBQUHAwEGCCsGAQUFBwMCBggrBgEFBQcD
BAYIKwYBBQUHAwgGCisGAQQBgjcCARUGCisGAQQBgjcCARYGCisGAQQBgjcKAwEG
CisGAQQBgjcKAwMGCisGAQQBgjcKAwQGCWCGSAGG+EIEATAOBgNVHQ8BAf8EBAMC
AQYwHQYDVR0OBBYEFP5QEIFCusKFAcvStC7/8bPNsmMWMA0GCSqGSIb3DQEBCwUA
A4IBAQAA0P5Y3weQuQMluQxtN+Rlqg/51OqaQrc+D4/THsQmA/9XW289NvvNYU9K
WiBxXpYls9IxS9rsbG4w6Q93W/40lV0xKr9TufSUmFz6ucUnGn5RLt119cZR941p
Znec5g98eRovyr4WnkU/S/9J2F03X9Us9M29Bv0JsHtLKyGZQCQK9l/DnC9Y9mC2
tDy2iUOmvqBKm9QtBrMss+vGGFrksSv3s3qmQZYeCRk5NyXgLHoxqr/4GsJ2mzIw
tyAo6mOp9xa6TSOlkHwPMbnN+HdkjyhfuBBkTQj4apxFb8coLkwsNAnvV+3GDsNt
26TejHIwLVmNweEsbSmJ1Z2Gw/tl
-----END CERTIFICATE-----
For example on a freshly installed ubuntu 20.04 box or a fresh container,
issueing the following commands reproduces the issue not only for me:
apt update
apt install mitmproxy
mitmdump
<CTRL-C>
openssl x509 -in /root/.mitmproxy/mitmproxy-ca-cert.pem -hash -issuer_hash
8bbe0e8d
8bbe0e8d
Of course both hashes are the same, it is a self-signed root cert. But I find it surprising that I always get the hash value of 8bbe0e8d. Everywhere.
The answer to solve this riddle is documented in the OpenSSL man page:
-issuer_hash
outputs the "hash" of the certificate issuer name.
And as you can see in the output of your certificate the issuer of the certificate is fixed and therefore the same on each and every system mitmproxy is installed: CN=mitmproxy, O=mitmproxy
A fixed input always outputs the same hash value of course.
One Root CA certificate can have multiple child certificates. Hence all those child certificates have the same issuer and therefore are all mapped to the same hash. Therefore it is nothing unusual that multiple certificates in /etc/ssl/certs/ are mapped to the same hash value. This seems to be some sort of grouping.
Using at that point the certificate fingerprint (or the issuer certificate fingerprint) does not make much sense, because when you use /etc/ssl/certs/ usually you want to find exactly the data of this certificate. If you would already know the certificate fingerprint you also have the certificate and hence doe not have to search for the certificate data.

PHP imap_open(), OpenSSL, and no Cipher

I have moved a PHP script to another server, and now fail to login to an IMAP (TLS) postbox:
TLS/SSL failure for mail.servername.de: SSL negotiation failed
It seems that the problem is caused by OpenSSL, because when I try to connect to the Mailserver from both servers, I get a connection in one case (the mailserver asking for input), but none in the other (the connection is closed, I am back to bash):
openssl s_client -crlf -connect mail.servername.de:993
The most obvious difference is here:
verify return:1
---
<snip>
-----END CERTIFICATE-----
subject=/CN=mail.servername.de
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: DH, 1024 bits
---
SSL handshake has read 3398 bytes and written 483 bytes
Verification: OK
---
New, TLSv1.2, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
And on th other server (where no connection is made)
verify return:1
depth=0 CN = mail.servername.de
verify return:1
140410888582464:error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small:../ssl/statem/statem_clnt.c:2149:
---
<snip>
-----END CERTIFICATE-----
subject=CN = mail.servername.de
issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
---
No client certificate CA names sent
---
SSL handshake has read 3167 bytes and written 318 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Server public key is 2048 bit
On the mailserver dovecot is configured not to accept non-encrypted connections. But, I assume it already fails due to dh key too small, which seems to relate to cipher negotiation.
Now I simply fail to put the things together... Why does the SSL connection work from one server, but not from the other?
As I own the "remote end" myself, I was able to increase security. The solution is simple, and may be relevant for others as well ...
The dovecot version is 2.2.x, which is of some relevance for the DH parmaters (see Dovecot SSL configuration). In the configuration file /etc/dovecot/conf.d/10-ssl.conf you can simply add this line:
ssl_dh_parameters_length = 2048
And eventually, it may be necessary to add this here to the main configuration file /etc/dovecot/dovecot.conf at the end:
!include conf.d/*.conf
Finally, it is important not to reload, but to restart dovecot.
systemctl restart dovecot
And suddently, the weight, troubles, and frustration of several hours is gone. Great...
Further to the above, there's a change from dovecot 2.3.
ssl_dh_parameters_length is now not used, and ssl_dh must be used instead, to point to a file generated using
openssl dhparam 4096 > dh.pem
see https://doc.dovecot.org/configuration_manual/dovecot_ssl_configuration/ and scroll down to SSL Security Settings. That was the only change I had to make following the upgrade to get it to work properly again. I put the dh.pem file in /etc/dovecot, so my line in 10-ssh.conf is
ssl_dh=</etc/dovecot/dh.pem
TL;DR: your new host has a newer version of OpenSSL probably with higher security settings which prohibit connecting to the host for reasons explained below.
"dh key too small" comes from OpenSSL and because of too low security.
Things changed, and for example in newest Debian versions and with OpenSSL 1.1.1 (and I guess it is similar for newer versions), the security was enhanced.
The best and simplest explanation I have found is on Debian wiki at https://wiki.debian.org/ContinuousIntegration/TriagingTips/openssl-1.1.1
which says:
In Debian the defaults are set to more secure values by default. This
is done in the /etc/ssl/openssl.cnf config file. At the end of the
file there is:
[system_default_sect]
MinProtocol = TLSv1.2
CipherString = DEFAULT#SECLEVEL=2
This can results in errors such as:
dh key too small
ee key too small
ca md too weak
Now the possible solutions in descending order of preference:
ask the remote end to generate better "DH" values ("Server Temp Key: DH, 1024 bits"); the best explanations are at https://weakdh.org/sysadmin.html; note specifically the "Administrators should use 2048-bit or stronger Diffie-Hellman groups with "safe" primes."
configure your end specifically for this connnection to not use the OS default and lower your settings; it should be enough to set ciphers to "DEFAULT#SECLEVEL=1" in the code that does the connection
(really, really, really not recommended) change the value of SECLEVEL from 2 to 1 in the global configuration file on your end. But this impacts all connections from your host not just this one so you are lowering the global security of your system just because of one low level of security from one remote node.

Configuring gnus with gmail imap

I am trying to configure gnus to work with my gmail account. My .gnus file looks like this:
(setq gnus-select-method '(nntp "news.gwene.org"))
(setq user-full-name "George P. Burdell")
(setq user-mail-address "probablyReal#gmail.com")
(setq smtpmail-auth-credentials "~/.authinfo.epg")
(add-to-list 'gnus-secondary-select-methods
'(nnimap "gmail"
(nnimap-address "imap.gmail.com")
(nnimap-server-port 993)
(nnimap-stream ssl)
(nnimap-authinfo-file "~/.authinfo.epg")
)
)
(setq smtpmail-stream-type 'ssl
smtpmail-smtp-server "smtp.gmail.com"
smtpmail-smtp-service 465)
and my .authinfo.epg file looks like this:
machine imap.gmail.com login probablyReal#gmail.com password secret port 993
machine smtp.gmail.com login probablyReal#gmail.com password secret port 465
and the error that it throws is:
Generating the cache active file...done
Opening nnfolder server on archive...done
Opening nnimap server on gmail...
Opening connection to imap.gmail.com via tls...
Opening TLS connection to `imap.gmail.com'...
Opening TLS connection with `gnutls-cli --insecure -p 993 imap.gmail.com'...failed
Opening TLS connection with `gnutls-cli --insecure -p 993 imap.gmail.com --protocols ssl3'...failed
Opening TLS connection with `openssl s_client -connect imap.gmail.com:993 -no_ssl2 -ign_eof'...failed
Opening TLS connection to `imap.gmail.com'...failed
Unable to open server nnimap+gmail due to: Buffer *nnimap imap.gmail.com 993 *nntpd** has no process
Opening nnimap server on gmail...failed:
No new newsgroups
Checking new news...
Reading active file from gmail via nnimap...
Opening nnimap server on gmail...
Server nnimap+gmail previously determined to be down; not retrying
Opening nnimap server on gmail...failed:
Reading active file via nndraft...done
Checking new news...done
Warning: Opening nnimap server on gmail...failed: ; Server nnimap+gmail previously determined to be down; not retrying; Opening nnimap server on gmail...failed: ; Unable to open server nnimap+gmail due to: Buffer *nnimap imap.gmail.com 993 *nntpd** has no process
(sorry for wall of text)
finally, when I evaluate this:
(gnutls-available-p)
it simply prints:
nil
I'm running emacs for windows 8, I'm not using cygwin, and I really can't figure out what the problem is, even after seraching for hours.
edit: I've tried both of these (as well as combinations from both) and get the same errors:
http://blog.binchen.org/posts/notes-on-using-gnus.html
http://www.emacswiki.org/emacs/GnusGmail
My guess would be that Emacs on Windows doesn't come with the required libraries, hence (gnutls-available-p) returns nil for you. You'll have to install gnutls somewhere so that your Emacs will be able to find it, cf. the GnuTLS homepage. You probably have to put the downloaded Windows library into the directory where Emacs is looking for it's libraries.

Sending emails with emacs24 via smtp with gnutls and extra arguments

I have a rather weird problem with using sending out emails from emacs24 with my posteo email account, but everything seems to work just fine with gmail and gmx. This is the relevant part of my current .emacs configuration (it feels like I permuted it a million times with always the same results):
(require 'smtpmail)
(require 'starttls)
(setq message-send-mail-function 'smtpmail-send-it)
(setq tls-program '("gnutls-cli --priority NORMAL:%COMPAT -p %p %h"))
(setq starttls-gnutls-program "gnutls-cli --priority NORMAL:%COMPAT")
(setq starttls-use-gnutls t)
(setq smtpmail-stream-type 'starttls)
(setq smtpmail-smtp-server "posteo.de")
(setq smtpmail-debug-info t)
(setq smtpmail-debug-verb t)
(setq smtpmail-smtp-service 587) ;;587(starttls) or 465(tls/ssl) or ?
(setq starttls-extra-arguments '("--priority NORMAL:%COMPAT"))
The output in my message buffer is:
Sending via mail...
235 2.7.0 Authentication successful
gnutls.c: [0] (Emacs) fatal error: A TLS fatal alert has been received.
gnutls.c: [0] (Emacs) Received alert: Bad record MAC
smtpmail-send-command: Process smtpmail not running
and in my trace of SMTP to posteo.de buffer:
220 mail.posteo.de ESMTP Postfix
250-mail.posteo.de
250-PIPELINING
250-SIZE 76800000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
220 2.0.0 Ready to start TLS
250-mail.posteo.de
250-PIPELINING
250-SIZE 76800000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
AUTH PLAIN <omitted>
235 2.7.0 Authentication successful
Process smtpmail connection broken by remote peer
MAIL FROM:<c.bourjau#posteo.de> SIZE=281
QUIT
The problem seems to be a certificate which appears to have wrong "paddings" (I am not really sure what this is) http://gnutls.org/manual/html_node/On-Record-Padding.html. Another way to produce a similar error on the command line with this server is to do:
$ gnutls-cli --starttls -p 587 posteo.de
Resolving 'posteo.de'...
Connecting to '89.146.220.134:587'...
- Simple Client Mode:
220 mail.posteo.de ESMTP Postfix
*** Starting TLS handshake
*** Fatal error: An unexpected TLS packet was received.
*** Handshake has failed
This error is supposedly fixed if one adds the --priority NORMAL:%COMPAT to the gnutls argument which I tried to no avail (see .emacs).
So the question is: How does on treat certifcates throwing these kind of errors in emacs?
Thanks a lot in advance!
This post gave me the crucial hint: How to ask gnutls to use client certificate in emacs 24
emacs24 seems to ignore the starttls-gnutls-program variable if gnutls-available-p is not nil, which has to be force by overwriting the latter function.
My working configuration is now the following:
(require 'smtpmail)
(require 'starttls)
(setq message-send-mail-function 'smtpmail-send-it)
(defun gnutls-available-p ()
"Function redefined in order not to use built-in GnuTLS support"
nil)
(setq starttls-gnutls-program "gnutls-cli")
(setq starttls-use-gnutls t)
(setq smtpmail-stream-type 'starttls)
(setq smtpmail-smtp-server "posteo.de")
(setq smtpmail-smtp-service 587) ;;587(starttls) or 465(tls/ssl)
(setq starttls-extra-arguments '("--priority" "NORMAL:%COMPAT"))