Why is Swift Kitura Server Not Terminating Some Threads? - swift

Question

I am having a somewhat reproducible problem on a Swift server I'm running. This is a multi-threaded server, using Kitura. The basics are: After the server has been running for a period of time, download requests start needing retries from the client (usually three retries). The attempts from the client result in the server thread not terminating. On the server, the download problem shows up like this in the log:
[INFO] REQUEST /DownloadFile: ABOUT TO END ...
And then the request never terminates.
The relevant fragment code in my server looks like this:
// <snip>
Log.info(message: "REQUEST \(request.urlURL.path): ABOUT TO END ...")
do {
try self.response.end()
Log.info(message: "REQUEST \(request.urlURL.path): STATUS CODE: \(response.statusCode)")
} catch (let error) {
Log.error(message: "Failed on `end` in failWithError: \(error.localizedDescription); HTTP status code: \(response.statusCode)")
}
Log.info(message: "REQUEST \(request.urlURL.path): COMPLETED")
// <snip>
That is, the server clearly seems to hang on the call to end (a Kitura method). See also https://github.com/crspybits/SyncServerII/blob/master/Sources/Server/Setup/RequestHandler.swift#L105
Immediately before this issue came up last time, I observed the following in my server log:
[2017-07-12T15:31:23.302Z] [ERROR] [HTTPServer.swift:194 listen(listenSocket:socketManager:)] Error accepting client connection: Error code: 5(0x5), ERROR: SSL_accept, code: 5, reason: DH lib
[2017-07-12T15:31:23.604Z] [ERROR] [HTTPServer.swift:194 listen(listenSocket:socketManager:)] Error accepting client connection: Error code: 1(0x1), ERROR: SSL_accept, code: 1, reason: Could not determine error reason.
[2017-07-12T15:31:23.995Z] [ERROR] [HTTPServer.swift:194 listen(listenSocket:socketManager:)] Error accepting client connection: Error code: 1(0x1), ERROR: SSL_accept, code: 1, reason: Could not determine error reason.
[2017-07-12T15:40:32.941Z] [ERROR] [HTTPServer.swift:194 listen(listenSocket:socketManager:)] Error accepting client connection: Error code: 1(0x1), ERROR: SSL_accept, code: 1, reason: Could not determine error reason.
[2017-07-12T15:42:43.000Z] [VERBOSE] [HTTPServerRequest.swift:215 parsingCompleted()] HTTP request from=139.162.78.135; proto=https;
[INFO] REQUEST RECEIVED: /
[2017-07-12T16:32:38.479Z] [ERROR] [HTTPServer.swift:194 listen(listenSocket:socketManager:)] Error accepting client connection: Error code: 1(0x1), ERROR: SSL_accept, code: 1, reason: Could not determine error reason.
I am not sure where this is coming from in the sense that I'm not sure if one of my client's is generating this. I do not explicitly make requests to my server with "/". (I do occasionally see requests made to my server from clients that are not mine-- it is possible this is one of these). Note that all except one of these log messages are coming from Kitura, not directly from my code. My log message is [INFO] REQUEST RECEIVED: /.
If I was a betting man, I'd say the above errors put my server into a state where afterwards, I see this download/retry behavior.
My only solution at this point is to restart the server. From which point the issue doesn't immediately happen.
Thoughts?

Answer 1

I'm not sure if this addresses the root problem, or if it's just a work-around, but it appears to be working. With the server as stated in the question, I had been using Kitura's built-in SSL support. I have now switched to using NGINX as a front-end, and no longer use Kitura's built-in SSL support. NGINX takes care of all the HTTPS/SSL details. Since doing this (about a month ago), and with the server running for all of that intervening time, I have not experience the not terminating issue reported in this question. See also https://github.com/crspybits/SyncServerII/issues/28