Socket self signed certificate handshake failure - swift

Question

I am using Starscream socket library and am trying to use WSS however I am having a handshake failure. I got my self signed certificate, I converted it to a .der file. Here is the code I am trying
var socket = WebSocket(url: URL(string: "wss://192.168.1.130:6223")!, protocols: [])
override func viewDidLoad() {
super.viewDidLoad()
do
{
let urlPath = Bundle.main.path(forResource: "my_cert", ofType: "der")
let url = NSURL.fileURL(withPath: urlPath!)
let certificateData = try Data(contentsOf: url)
let certificate: SecCertificate =
SecCertificateCreateWithData(kCFAllocatorDefault, certificateData as CFData)!
var trust: SecTrust?
let policy = SecPolicyCreateBasicX509()
let status = SecTrustCreateWithCertificates(certificate, policy, &trust)
if status == errSecSuccess {
let key = SecTrustCopyPublicKey(trust!)!;
let ssl = SSLCert(key: key)
socket.security = SSLSecurity(certs: [ssl], usePublicKeys: true)
socket.delegate = self
socket.connect()
}
}catch let error as NSError
{
print(error)
}
}
So when I try to connect, I get the following error message
2017-07-07 11:06:26.590 CertificateTesting[5180:81661] CFNetwork
SSLHandshake failed (-9807) websocket is disconnected: The operation
couldn’t be completed. (OSStatus error -9807.)
The certificate should work fine, my Android colleague has tried it on his side and has had no issues. The only way I can get it working on my side is if I disable SSL validation like so
socket.disableSSLCertValidation = true
Does anyone have any experience using self signed SSL with sockets. Any information would be much appreciated.
Edit:
I called verify ssl command, it returns
➜ CertificateTesting git:(master) ✗ openssl verify -my_cert.der ca-cert.pem server-cert.pem
usage: verify [-verbose] [-CApath path] [-CAfile file] [-purpose purpose] [-crl_check] [-engine e] cert1 cert2 ...
recognized usages:
sslclient SSL client
sslserver SSL server
nssslserver Netscape SSL server
smimesign S/MIME signing
smimeencrypt S/MIME encryption
crlsign CRL signing
any Any Purpose
ocsphelper OCSP helper
Does that look okay?

Answer 1

You can give a try using the common name in the WebSocket instead of IP.
var socket = WebSocket(url: URL(string: "wss://192.168.1.130:6223")!, protocols: [])
You can verify the common name in the certificate using the command
openssl x509 -in <certificate file> -text
Validate the SSL handshake using following command
openssl s_client -host <common name mentioned in the cert> -port <port> -cert <client_cert file> -key <client_key file> -CAfile <ca_cert file>

Answer 2

in an old swift app, i had the same problem using REST API and WS exposed by a self signed certificate.
Websocket protocol is embedded in HTTP/S protocol, also the handshake.
So generate the certificate .cert:
echo -n | openssl s_client -connect yoururl:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > name_youwant.cert
Import the .cert file in the asset of the app that will use it.
In the class that estabilish the websocket connection, implement URLSessionDelegate.
And then use this logic to validate the self signed certificate:
Note: NSBundle.mainBundle().pathForResource(Config.certificate, ofType: ".cert") -> Config.certificate is a static string that indicate the name of file.
// uncomment if self signed certificate is used on the backend
public func URLSession(session: NSURLSession, didReceiveChallenge challenge: NSURLAuthenticationChallenge, completionHandler: (NSURLSessionAuthChallengeDisposition, NSURLCredential?) -> Void) {
let serverTrust = challenge.protectionSpace.serverTrust
let certificate = SecTrustGetCertificateAtIndex(serverTrust!, 0)
let certificateData = SecCertificateCopyData(certificate!)
let remoteCertificateData = certificateData as NSData
let cerPath = NSBundle.mainBundle().pathForResource(Config.certificate, ofType: ".cert")
let localCertData = NSData(contentsOfFile: cerPath!)!
if localCertData.isEqualToData(remoteCertificateData) {
completionHandler(NSURLSessionAuthChallengeDisposition.UseCredential,NSURLCredential(forTrust:
challenge.protectionSpace.serverTrust!))
}else{
completionHandler(NSURLSessionAuthChallengeDisposition.CancelAuthenticationChallenge,nil)
}
}
Hope this helps.
Regards,