How to handle stream response async - swift

I opened a stream to a server I have. When I send the text "PING" it responds with "PONG". I have successfully connected, sent messages, and received responses. I have a pretty bare bones test below.
The Problem
I want to handle the messages from the server as they come in. Currently, I send three PINGs and later I get three PONGS. The server responds with PONG immediately after but my code is not handling the response until after the main thread completes. The expected result is to see the PONG messages immediately after the PING is sent because they are being handled concurrently.
What I tried
Pretty much, what you see below. I thought "I want to handle the stream responses at the same time as sending messages so I'll need to do that on another thread". So, I put the RunLoop in another thread via GCD. That did not help....Can't figure our how to make the StreamDelegate handle its delegate method stream in a separate thread...
Current Console Result
PING
PING
PING
PONG
PONG
PONG
Desired Console Result
PING
PONG
PING
PONG
PING
PONG
The Code
import Foundation
import XCTest
class StreamTests: XCTestCase, StreamDelegate {
var inputStream: InputStream?
var outputStream: OutputStream?
let url: URL = URL(string: "http://theserver.com:4222")!
func testAsyncStream() {
self.setupStream()
let ping = "PING".data(using: String.Encoding.utf8)!
print("PING")
self.outputStream?.writeStreamWhenReady(ping)
sleep(1)
print("PING")
self.outputStream?.writeStreamWhenReady(ping)
sleep(1)
print("PING")
self.outputStream?.writeStreamWhenReady(ping)
sleep(1)
}
private func setupStream() {
guard let host = url.host, let port = url.port else { print("Failed URL parse"); return }
var readStream: Unmanaged<CFReadStream>?
var writeStream: Unmanaged<CFWriteStream>?
CFStreamCreatePairWithSocketToHost(nil, host as CFString!, UInt32(port), &readStream, &writeStream)
self.inputStream = readStream!.takeRetainedValue() as InputStream
self.outputStream = writeStream!.takeRetainedValue() as OutputStream
guard let inStream = self.inputStream, let outStream = self.outputStream else { return }
inStream.open()
outStream.open()
DispatchQueue.global(qos: .utility).sync { [weak self] in
for stream in [inStream, outStream] {
stream.delegate = self
stream.schedule(in: .current, forMode: .defaultRunLoopMode)
}
RunLoop.current.run(mode: .defaultRunLoopMode, before: Date.distantFuture)
}
}
func stream(_ aStream: Stream, handle eventCode: Stream.Event) {
switch aStream {
case inputStream!:
switch eventCode {
case [.hasBytesAvailable]:
print("PONG")
break
default:
break
}
default:
break
}
}
}

Don’t try to write unit tests for multithreaded code. It will bite you in the ass later.
The reason unit testing code that runs on multiple threads is hard is the fact that you can’t control the order of execution of threads, nor the allocated time per thread – this is an OS decision.
Thus, in order to make sure the code submitted on the other thread executes and populates the expected data, you’ll need to block the main thread, where unit tests usually runs, for an amount of time large enough to be sure the other thread finishes the work.
Now, the tricky part is to find what that amount of time should be. Make it too short and you’ll see random failures of your unit tests, make it too long and you’ll increase the duration of your unit tests more and more. And theoretically there’s no upper limit for how long will need to wait for the other thread to finish, as this is out of our control (remember, the OS decides which thread to pick up next and how much time to allocate to it).
And worse, when a unit test like this starts failing on a CI machine but it doesn’t fail on your machine, who’s to blame: the CI machine for being too slow, or your code misbehaving in some conditions that happen only on the CI machine? This ambiguity can lead to lot of time wasted in trying to figure out what the hack happens with the tested code.
Conclusion: don’t try to write unit tests for code that executes parts of its work on a different thread. The reason is simple: robust unit tests need to have control over all inputs of the tested code, and the second thread is not something it can control (unless you mock the thread dispatching, but that’s another story).
Instead, push as much logic as you can into single-thread methods, and tests those methods instead. In the end, most of the bugs occur due to incorrect business logic.

Related

Got timeout error -1001 after some HTTP requests using Swift

I create an http request using PUT to get data from the server. I got this data from the server and transform it on a PDF file.
When I run it for the first time everything runs fine, but after some calls, I start to get timeout errors.
Sometimes, I need to restart the app to be able to receive HTTP requests again.
This is the code that I use.
func callGetPdfFromEndpointUsingNSMutableURLRequest() {
if codigoBarra == "" {
messageError = "Código não localizado"
showingAlert = true
redirectToPdfView = false
showingActivityIndicator = false
return
}
let serviceRepository = ServiceRepository()
// let codigo_barra = "d152d36914313fedfbf36842a7195b723"
let json: [String: Any] = ["codigoBarra":"\(codigoBarra)"]
let request: NSMutableURLRequest = serviceRepository.clientURLRequest(endpointPesquisa, typeAutho: .basic, parms: "", body: json as Dictionary<String, AnyObject>)
print("request: \(request)")
print("request.url: \(String(describing: request.url))")
serviceRepository.put(request, retryLogin: true, completion: {isOk,msgError,httpCode,needLogin, response in
if isOk {
tratarRequisicaoPdf(response)
} else {
print("erro no request - is not ok | - httpCode: \(httpCode)")
var stringResponse:String = ""
if response != nil {
stringResponse = String(data: response as! Data, encoding: .utf8)!
} else {
stringResponse = "Sem resposta do servidor, tempo limite da solicitação foi esgotado."
}
messageError = "\(stringResponse)"
print(messageError)
showingAlert = true
redirectToPdfView = false
semaphore.signal()
}
semaphore.wait()
showingActivityIndicator = false
})
}
This error is unstable, sometimes it shows, sometimes it don't appear.
The people working on backend was not able to detect any problems.
I got the following error:
2022-05-20 15:33:15.442419-0300 CDA[2016:38068] Task <147B6F7F-E46A-47D0-A258-D6F3E5417D7E>.<1> finished with error [-1001] Error Domain=NSURLErrorDomain Code=-1001 "Esgotou-se o tempo limite da solicitação." UserInfo={_kCFStreamErrorCodeKey=-2102, NSUnderlyingError=0x7fd0e5245520 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "(null)" UserInfo={_kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <147B6F7F-E46A-47D0-A258-D6F3E5417D7E>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalDataTask <147B6F7F-E46A-47D0-A258-D6F3E5417D7E>.<1>"
), NSLocalizedDescription=Esgotou-se o tempo limite da solicitação., NSErrorFailingURLStringKey=https://myurl.sp.gov.br/gedave/api/spservicos/v1/buscaRequisicaoExame, NSErrorFailingURLKey=https://myurl.sp.gov.br/gedave/api/spservicos/v1/buscaRequisicaoExame, _kCFStreamErrorDomainKey=4}
response::: nil
erro: response is nil
httpCode: 0
What can I do to try to detect what was causing the timeout errors?
Edit:
I added the source code that can be viewed here https://swiftfiddle.com/mz2dxw6z6bda7a6t44cryncrpi.
NEW EDIT:
Answers to comments created by #Rob
Good to know about the NSUrlMutableRequest
I will try to use the 'finishAndInvalidate' in my URLSession. I didn't know about that.
My problem is unpredictable. Now I start the app and the first call got an timeout, after the second call the app works. Sometimes it starts working, but after some requests, I got a timeout
In the absence of a MCVE, there is not enough here to diagnose or reproduce the problem.
That having been said, there are some interesting clues:
You are calling wait (on a semaphore). Eliminate the DispatchSemaphore and if you want to know when the requests are done, use DispatchGroup. But when you use the dispatch group, use notify when it is done, not wait. Avoid blocking threads unnecessarily. And never block the main thread.
Your network request is performed by ServiceRepository, which you have not shared with us. But in your fiddle link, you show us unrelated URLSession code that is specifying the main queue as its delegate queue. If ServiceRepository is doing something similar, that, combined with the wait, above, could easily deadlock your code.
So, eliminate the semaphore, avoid ever calling wait (whether semaphore or dispatch group), and the deadlock risk is eliminated.
That having been said, that is only one potential timeout risk. The other scenario might be that you are simply issuing too many requests for the URLSession to run them without timing out.
If that is the case, you have a few options:
Increase the timeout threshold (either of the request or the session).
Try bumping up the timeout values and see if that mitigates the issue.
Submit uploads in just-in-time manner.
E.g., you might have a process where, rather than initiating all the uploads at once, that you issue each upload upon the completion of the prior upload. Or you can wrap the upload in a custom, asynchronous, Operation subclass (which is complicated in its own right), and then tell the operation queue to only attempt 4 at a time, or whatever. There are many techniques to tackle this, but the idea is to prevent timeouts by not even submitting the upload until you are confident that the URLSession can perform them.
Use background URLSession with file-based uploads.
If you have uploads that are so slow that they risk timing out, it begs the question of whether you really want to force the user to keep the app running in order to let the uploads finish at all. If you use a background URLSession, the background daemon will not let them timeout, but rather will upload them when it can.
Refactoring this for background URLSession is a non-trivial exercise (the first time you do it, anyway). You have to abandon completion handlers, use delegate-methods API, use file-based uploads, etc. But it is an incredibly powerful way to let uploads proceed in the background, allowing the user to leave the app, etc.
I don't see full code but you should have a look into semaphore usage - probably wrong multithread logic leads to not complete you request and your completion is hovered for a long period and causes URLDataTask to produce the timeout error

Semaphore in Swift not working for Firebase Authentication

I am trying to achieve functionality similar to Javascript/C#'s async/await. I am trying out the use of semaphores, and found that it works with URLSession in my XCode Playground.
Thus, i am now trying to perform the same thing with Firebase Authentication using the following code:
var response:String? = "test"
let semaphore = DispatchSemaphore(value: 0)
Auth.auth().createUser(withEmail: email, password: password) { (result, err) in
if err != nil {
response = "\(String(describing: err))"
}else{
response = nil
}
semaphore.signal()
}
let _ = semaphore.wait()
if response == nil{
self.transitionToHome()
}
However, the simulator freezes forever, appearing as if the semaphore.signal() never got called. Placing print statements near the semaphore.signal() didn't appear as well. I've also placed the firestore code in a DispatchQueue.global(qos: .background).async and subsequently tried to retrieve the response value in DispatchQueue.main.async but the response did not get updated as well. The code below reflects what i did:
DispatchQueue.global(qos: .background).async {
Auth.auth().createUser(withEmail: email, password: password) { (result, err) in
if err != nil {
response = "\(String(describing: err))"
}else{
response = nil
}
semaphore.signal()
}
let _ = semaphore.wait()
}
if response == nil{
self.transitionToHome()
}
While this did not freeze up the UI, the response value was not picked up after the DispatchQueue was called. I've also called the if-else block within a DispatchQueue.main.async block but that had the same result too.
Further, after waiting for a period of time, i see this error popping up in my xcode terminal:
020-01-02 01:33:25.447842+0800 das-carddeckapp[78136:10508853] Connection 4: received failure notification
2020-01-02 01:33:25.448179+0800 das-carddeckapp[78136:10508853] Connection 4: failed to connect 1:50, reason -1
2020-01-02 01:33:25.448387+0800 das-carddeckapp[78136:10508853] Connection 4: encountered error(1:50)
2020-01-02 01:33:25.457587+0800 das-carddeckapp[78136:10508853] Task <3A3720D6-7549-4C31-96A2-C88B89294821>.<1> HTTP load failed, 0/0 bytes (error code: -1009 [1:50])
I know that using completion handlers will make this work, but I want to try to get this to work before resorting to completion handlers. Any help is greatly appreciated!
The first example (where you’re calling wait on the main thread) is “deadlocking” because it is blocking the main thread with wait, which is to be signaled from the createUser closure, which also wants to run on the main thread (which you’ve blocked).
I know that semaphores is considered a bad pattern but i just wanted to understand why is it not working.
Your second example, where you are dispatching the wait to the global queue resolves the deadlock. But because you’ve dispatched that asynchronously, that means that the main thread will carry on as the global queue waits for the response. As a result, you are almost guaranteed to not have a value ready to return. (And if you attempt to change this to call the global queue synchronously, you’d just re-introduce the deadlock.)
In short, you can’t easily have the main queue wait for an asynchronous method which calls its completion handler on the main queue. That is a programmatic “Catch-22”.
But setting that aside, if this is an iOS app, it just simply a serious problem to ever block the main thread (even if you could solve the deadlock). It’s a horrible UX when an app freezes. This is especially true for unfortunate mobile users who find themselves on poor cellular connection, where it might result in a non-trivial delay. Worse, if you’re unlucky enough to block the main thread at the wrong time, the iOS watchdog process may unceremoniously kill your app.
Hey, I get it. We’ve all been there. When we first encountered asynchronous programming patterns, like network requests, it feels it would be so much more logical and intuitive if we could just wait for the response. But it should be avoided at all costs. You’ll be much happier in the long run if you stick with well-established asynchronous programming patterns, rather than fighting it.

SimplePing (macOS) used in a background thread does not work. What are the alternatives?

I'm using Simple Ping API to ping a bunch of Hosts.
My class 'PingLatencyCalculator' is a simple wrapper to SimplePing, which calculates the latency and also timesout after 1 second for each request.
let domains = ["www.amazon.com", "www.microsoft.com", "www.yahoo.com", "www.violanation.com", "www.stackoverflow.com", "www.docs.google.com", "www.microsoft.com", "www.google.com", "www.espn.in", "www.dsDSadasdsad.com"]
let latencyCalculator = PingLatencyCalculator()
for domain in domains {
latencyCalculator.pingHost(hostname: domain) { (latency) in
if let unwrappedlatency = latency {
DispatchQueue.main.async {
print("\(domain) - Your latency is \(String(describing: unwrappedlatency)))")
}
} else {
print("\(domain) could not be reached...")
}
}
}
Everything works fine. The only issue is that Apple's SimplePing API is build on C libraries and it seems their callbacks won’t work with GCD, thus this has to run in main thread.
Do you think its not a good idea to run this on the main thread?
If i run this on background thread, the callbacks are not called.
So what are my other options to ping hosts concurrently in the background?
Any suggestions would be appreciated.

Synchronous or asynchronous in swift server (Perfect)

I have created a Swift 3.0 server using the Perfect Framework. Everything works great as expected but i am trying to learn if there is a better way to do some things.
Coming from an iOS background I know to always call any blocking function in a different thread. Does this still hold when developing in the server?
For example, if I have a long blocking task (like making another request or performing a large database query), does it make a difference to call it synchronously:
routes.add(method: .get, uri: "/", handler: { request, response in
longSynchronousTask()
response.appendBody(string: "Finished")
response.completed()
})
or should I be doing this asynchronously?
routes.add(method: .get, uri: "/", handler: { request, response in
longAsynchronousTask(completion: {
response.appendBody(string: "Finished")
response.completed()
})
})
Depends on the framework. I wasn't able to find more information about perfect.org's architecture but since it claims to run on a "high-performance asynchronous networking engine", the expectation is that a thread handling a request is not supposed to block.
Most reactive frameworks (like Node.js, Vert.x) rely on one or multiple event threads that are handling requests.
If these threads block, no more requests can be handled!
Which means that longer running tasks should be run in their own threads.
There are frameworks that provide a mechanism for that (like worker-threads).
The question then becomes: What is a longer running task?
if your task does I/O in an asynchronous fashion and merely 'waits' for I/O, you can do that on the event thread.
If you do lengthy computation, you might be better off putting that into a separate thread.
yes, you can. I use the following code test, it's OK.
struct DBOperationQueue {
static let messageOperationQueue:OperationQueue = {
let operationQueue = OperationQueue()
operationQueue.maxConcurrentOperationCount = 5
return operationQueue
}()
}
func handler(request: HTTPRequest, response: HTTPResponse) {
DBOperationQueue.messageOperationQueue.addOperation {
sleep(10)
print("sleep complete")
response.setHeader(.contentType, value: "text/html")
response.appendBody(string: "<html><title>Hello, world!</title><body>Hello, world!</body></html>")
response.completed()
}
}

Pipe for repeating network requests in Reactive Cocoa 4

I want to refresh data every 15 seconds from an API using Reactive Cocoa 4. Since more than one subscriber can ask for this data at the same time, I want to have multiple subscribers to share one source of data.
My current approach is to have one Signal and share it to every instance that asks for the data. This Signal should start refreshing as soon as the first Signal is subscribed and end after the last has disposed.
SignalProducer<String, NoError> { observer, disposable in
self.disposable = self.repeatTimer.observeNext { _ in
NSLog("start network request")
observer.sendNext("result")
}
}.on(disposed: {
NSLog("disposed")
}).startWithSignal { signal, disposable1 in
self.updateSignal = signal
}
}
return (updateSignal, disposable!)
So for the first request I create and store the updateSignal and each following request will get that signal.
My first question: How can I know when the last subscriber disposed its signal? So when can I stop the requests?
My second question: I store the disposable from my repeatin network request in self.disposable which I also return to the subscriber. If the subscriber only disposes its Signal (which he got from Signal.observeNext()) the inner loop, where I log "start network request" is running endless. Do I really need to stop that Signal myself even when the outer Signal gets disposed?
Is there any nicer way or pattern for shared repeating requests?
Use the global timer function to perform work at specified intervals.
You could do something like this:
self.disposable =
timer(SomeTimeInterval onScheduler:QueueScheduler.mainQueueScheduler)
.startWithNext { _ in
//start network request here
}
But it's better if you chain your network request producer and observe the results, like this:
self.disposable =
timer(SomeTimeInterval onScheduler:QueueScheduler.mainQueueScheduler)
.flatMap(.Latest, transform { _ in
return self.networkRequestSignalProducer()
})
.start({ event in
//monitor the result of the network request
})
Note that you may not want to use the main queue like I did in this example, depending on how you've implemented your network requests.
If you want to avoid dealing with disposables, you can add a .takeUntil before .flatMap and terminate the timer with a signal