Swift - Semaphore within semaphore [duplicate] - swift

I'm entering the concurrency programming with some semaphore issues.
My function first loads data from server, analyze received info and then, if necessary, makes second request to server.
I tried different ways to make it run, none of them did it well.
My current code FOR ME seems to be correct, but on second request it just locks(maybe like a DeadLock) and the last log is "<__NSCFLocalDataTask: 0x7ff470c58c90>{ taskIdentifier: 2 } { suspended }"
Please, tell me what do I don't know. Maybe there is more elegant way to work with completions for these purposes?
Thank you in advance!
var users = [Int]()
let linkURL = URL.init(string: "https://bla bla")
let session = URLSession.shared()
let semaphore = DispatchSemaphore.init(value: 0)
let dataRequest = session.dataTask(with:linkURL!) { (data, response, error) in
let json = JSON (data: data!)
if (json["queue"]["numbers"].intValue>999) {
for i in 0...999 {
users.append(json["queue"]["values"][i].intValue)
}
for i in 1...lround(json["queue"]["numbers"].doubleValue/1000) {
let session2 = URLSession.shared()
let semaphore2 = DispatchSemaphore.init(value: 0)
let linkURL = URL.init(string: "https://bla bla")
let dataRequest2 = session2.dataTask(with:linkURL!) { (data, response, error) in
let json = JSON (data: data!)
print(i)
semaphore2.signal()
}
dataRequest2.resume()
semaphore2.wait(timeout: DispatchTime.distantFuture)
}
}
semaphore.signal()
}
dataRequest.resume()
semaphore.wait(timeout: DispatchTime.distantFuture)
P.S. Why do I do it. Server returns limited count of data. To get more, I have to use offset.

This is deadlocking because you are waiting for a semaphore on the URLSession's delegateQueue. The default delegate queue is not the main queue, but it is a serial background queue (i.e. an OperationQueue with a maxConcurrentOperationCount of 1). So your code is waiting for a semaphore on the same serial queue that is supposed to be signaling the semaphore.
The tactical fix is to make sure you're not calling wait on the same serial queue that the session's completion handlers are running on. There are two obvious fixes:
Do not use shared session (whose delegateQueue is a serial queue), but rather instantiate your own URLSession and specify its delegateQueue to be a concurrent OperationQueue that you create:
let queue = OperationQueue()
queue.name = "com.domain.app.networkqueue"
let configuration = URLSessionConfiguration.default()
let session = URLSession(configuration: configuration, delegate: nil, delegateQueue: queue)
Alternatively, you can solve this by dispatching the code with the semaphore off to some other queue, e.g.
let mainRequest = session.dataTask(with: mainUrl) { data, response, error in
// ...
DispatchQueue.global(attributes: .qosUserInitiated).async {
let semaphore = DispatchSemaphore(value: 0)
for i in 1 ... n {
let childUrl = URL(string: "https://blabla/\(i)")!
let childRequest = session.dataTask(with: childUrl) { data, response, error in
// ...
semaphore.signal()
}
childRequest.resume()
_ = semaphore.wait(timeout: .distantFuture)
}
}
}
mainRequest.resume()
For the sake of completeness, I'll note that you probably shouldn't be using semaphores to issue these requests at all, because you'll end up paying a material performance penalty for issuing a series of consecutive requests (plus you're blocking a thread, which is generally discouraged).
The refactoring of this code to do that is a little more considerable. It basically entails issuing a series of concurrent requests, perhaps use "download" tasks rather than "data" tasks to minimize memory impact, and then when all of the requests are done, piece it all together as needed at the end (triggered by either a Operation "completion" operation or dispatch group notification).

Related

URLSession.shared.dataTask Code Block Not Running

I'm trying to make a fairly simple API call in Swift but, for some reason, my dataTask code is not running. I've made sure that the .resume() is there. This code has worked in the past but, something has changed recently and I don't know what it is. The only thing I can think of is the url. I've changed the ingredients but, when putting the url into a browser, it returns JSON data normally. When running this function, I get two "Outside URLSession.shared.dataTask....." messages in a row with nothing in between, indicating that the URLSession block of code isn't running. I'm a little new to APIs so, any help would be greatly appreciated. Please let me know if there's any more information I can provide. Also, I'm on an older MacBook and am using Swift5 if that makes a difference. Thanks!
let url: URL! = URL(string: "https://api.spoonacular.com/recipes/findByIngredients?ingredients=" + ingredientString + "&apiKey=aaabbbccc111222333")
print("URL: " + url.absoluteString)
let request = URLRequest(url: url)
// Make the API call
print("Outide URLSession.shared.dataTask.....")
let session = URLSession.shared.dataTask(with: request) { data, response, error in
print("Inside URLSession.shared.dataTask.....")
DispatchQueue.main.async {
print("Inside DispatchQueue.main.async....")
if data == nil {
print("No data recieved.")
}
print("data != nil.... Moving on to JSONDecoder....")
self.model = try! JSONDecoder().decode([RecipeSearchElement].self, from: data!)
}
}
session.resume()
print("Outside URLSession.shared.dataTask.....")
Unrelated to your immediate question at hand (which I answered elsewhere), I would advise a few changes to the routine:
One should not build a URL through string interpolation. Use URLComponents. If, for example, the query parameter included a space or other character not permitted in a URL, URLComponents will percent-encode it for you. If do not percent-encode it properly, the building of the URL will fail.
I would avoid try!, which will crash the app if the server response was not what you expected. One should use try within a do-catch block, so it handles errors gracefully and will tell you what is wrong if it failed.
I would recommend renaming the URLSessionDataTask to be task, or something like that, to avoid conflating “sessions” with the “tasks” running on that session.
I would not advise updating the model from the background queue of the URLSession. Fetch and parse the response in the background queue and update the model on the main queue.
Thus:
var components = URLComponents(string: "https://api.spoonacular.com/recipes/findByIngredients")
components?.queryItems = [
URLQueryItem(name: "ingredients", value: ingredientString),
URLQueryItem(name: "apiKey", value: "aaabbbccc111222333")
]
guard let url = components?.url else {
print("Unable to build URL")
return
}
// Make the API call
let task = URLSession.shared.dataTask(with: url) { data, _, error in
DispatchQueue.main.async {
guard error == nil, let data = data else {
print("No data received:", error ?? URLError(.badServerResponse))
return
}
do {
let model = try JSONDecoder().decode([RecipeSearchElement].self, from: data)
DispatchQueue.main.async {
self.model = model
}
} catch let parseError {
print("Parsing error:", parseError, String(describing: String(data: data, encoding: .utf8)))
}
}
}
task.resume()
In a more advanced observation, I would never have a network call update the model directly. I would leave that to the caller. For example, you could use a completion handler pattern:
#discardableResult
func fetchIngredients(
_ ingredientString: String,
completion: #escaping (Result<[RecipeSearchElement], Error>) -> Void
) -> URLSessionTask? {
var components = URLComponents(string: "https://api.spoonacular.com/recipes/findByIngredients")
components?.queryItems = [
URLQueryItem(name: "ingredients", value: ingredientString),
URLQueryItem(name: "apiKey", value: "aaabbbccc111222333")
]
guard let url = components?.url else {
completion(.failure(URLError(.badURL)))
return nil
}
// Make the API call
let task = URLSession.shared.dataTask(with: url) { data, _, error in
print("Inside URLSession.shared.dataTask.....")
DispatchQueue.main.async {
guard error == nil, let data = data else {
DispatchQueue.main.async {
completion(.failure(error ?? URLError(.badServerResponse)))
}
return
}
do {
let model = try JSONDecoder().decode([RecipeSearchElement].self, from: data)
DispatchQueue.main.async {
completion(.success(model))
}
} catch let parseError {
DispatchQueue.main.async {
completion(.failure(parseError))
}
}
}
}
task.resume()
return task
}
And then the caller could do:
fetchIngredients(ingredientString) { [weak self] result in
switch result {
case .failure(let error): print(error)
case .success(let elements): self?.model = elements
}
}
This has two benefits:
The caller now knows when the model is updated, so you can update your UI at the appropriate point in time (if you want).
It maintains a better separation of responsibilities, architecturally avoiding the tight coupling of the network layer with that of the view or view model (or presenter or controller) layers.
Note, I am also returning the URLSessionTask object in case the caller would like to cancel it at a later time, but I made it an #discardableResult so that you do not have to worry about that if you are not tackling cancelation at this point.
If you (a) are reaching the “outside” message, but not seeing the “inside” message; and (b) are absolutely positive that you are reaching the resume statement, it is one of a few possibilities:
The app may be terminating before the asynchronous request has time to finish. This can happen, for example, if this is a command-line app and you are allowing the app to quit before the asynchronous request has a chance to finish. If you want a command-line app to wait for a network request to finish, you might run a RunLoop that does not exit until the network request is done.
It can also happen if you use a playground and neglect to set needsIndefiniteExecution:
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
For the sake of completeness, there are a few other, less common, possibilities:
You have some other network request whose completion handler is blocked/deadlocked, thereby preventing anything else from running on the URLSession dedicated, serial, queue.
You have thread explosion somewhere else in your code, exhausting the limited pool of worker threads, preventing other tasks/operations from being able to get an available worker thread.

Can you have synchronous but non-blocking URLSesssions?

I am utilizing URLSessions which are asynchronous in nature.
This works well when there is only one call for a session.
When I need to execute multiple (serial) calls, where the results need to be combined in the order of execution, it makes program logic painful and error prone.
I am also blocking the main thread, which isn't good.
Constraints
Moving to next task may not occur before 4 second elapses, though it can be more.
Utilizing Monterey (OS upgrade required), so using the let (data, response) = try await session.data(from: url) is not optional.
If tasks are executed faster than every 4 seconds, the result is a server side error, forcing a retry.
Task execution
Execute task -- if the task concludes in less than 4 seconds, wait
for the difference so the next task does not execute before the 4
seconds elapses.
Repeat process until all task have been completed.
Combine the results
My current process utilizes semaphores or dispatchGroups but both block the main thread.
Is there a way to get synchronous behavior without blocking the main thread?
func getDataFromInput_Sync(authToken: String, transformedText: String ) -> (Data?, URLResponse?, Error?)
{
/*
By default, transactions are asynchronous, so they return data while the rest of the program continues.
Changing the behavior to synchronous requires blocking to wait for the outcome. It affords us the
ability to manage program flow inline. This methods forces synchronous behavior and the output, which
are returned via a tuple, types: (Data?, URLResponse?, Error?)
*/
var outData : Data?
var outError : Error?
var urlResponse : URLResponse?
let targetURL = "https://..."
let urlconfig = URLSessionConfiguration.ephemeral
// set the timeout to a high number, if we are prepared to wait that long. Otherwise the session will timeout.
urlconfig.timeoutIntervalForRequest = 120
urlconfig.timeoutIntervalForResource = 120
let urlSession = URLSession(configuration: urlconfig)
// let dispatchGroup = DispatchGroup()
let semaphore = DispatchSemaphore(value: 0)
// ephermeral doesnt write cookies, cache or credentials to disk
guard let url = URL(string: targetURL),
let httpBodyData = transformedText.data(using: .utf8) else { return (nil,nil,nil) }
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.httpBody = httpBodyData
request.addValue("Token " + authToken, forHTTPHeaderField: "Authorization")
// Perform HTTP Request
let task = (urlSession).dataTask(with: request) { (data, response, error) in
guard error == nil
else {
print("we have an error: \(error!.localizedDescription)")
return
}
guard let data = data else { print("Empty data"); return }
outData = data
urlResponse = response
outError = error
// dispatchGroup.leave()
semaphore.signal()
}
task.resume()
semaphore.wait()
// dispatchGroup.enter()
// task.resume()
// dispatchGroup.wait()
return (outData, urlResponse, outError)
}
func testServerRequest()
{
let sentences = ["Example Sentence1","Example Sentence2","Example Sentence3"] //...array to process
for (_,thisString) in sentences.enumerated()
{
let timeTestPoint = Date()
let futureSecs = 4.0
let (data, urlResponse, error) = getDataFromInput_Sync(authToken: authToken, transformedText: thisString )
let elapsed = timeTestPoint.timeIntervalSinceNow * -1 // time elapsed between request and result
// act on the data received
// executing the next request before futureSecs will cause an error, so pause
let delayAmt = futureSecs - elapsed
Thread.sleep(forTimeInterval: delayAmt)
}
}
Make it in background queue, like
DispatchQueue.global(qos: .background).async {
testServerRequest()
}

Can I cancel a JSONEncoder Swift?

I have a JSONEncoder encoding a 20mb file, which takes ages to process. If the data it's processing changes, I'd like to cancel the encoding, and restart the encoding process but I can't think of a way to do this. Any ideas?
I could call JSONEncoder.encode again, but now I would have two 30 second processes running, and double the amount of memory and processor overhead.
It would be lovely to be able cancel the previous one.
EDIT: Some of you requested to see my encoder. Here's the one which I'd say causes the biggest bottleneck...
func encode(to encoder: Encoder) throws {
try autoreleasepool {
var container = encoder.container(keyedBy: CodingKeys.self)
try container.encode(brush, forKey: .brush)
if encoder.coderType == CoderType.export {
let bezierPath = try NSKeyedUnarchiver.unarchivedObject(ofClass: UIBezierPath.self, from: beziersData)
let jsonData = try UIBezierPathSerialization.data(with: bezierPath, options: UIBezierPathWritingOptions.ignoreDrawingProperties)
let bezier = try? JSONDecoder().decode(DBBezier.self, from: jsonData)
try container.encodeIfPresent(bezier, forKey: .beziersData)
} else {
try container.encodeIfPresent(beziersData, forKey: .beziersData)
}
}
}
You can use OperationQueue and add your long running task into that operation queue.
var queue: OperationQueue?
//Initialisation
if queue == nil {
queue = OperationQueue()
queue?.maxConcurrentOperationCount = 1
}
queue?.addOperation {
//Need to check the isCanceled property of the operation for stopping the ongoing execution in any case.
self.encodeHugeJSON()
}
You can also cancel the task whenever you want using the following code:
//Whenever you want to cancel the task, you can do it like this
queue?.cancelAllOperations()
queue = nil
What is an Operation Queue:
An operation queue invokes its queued Operation objects based on their
priority and readiness. After you add an operation to a queue, it
remains in the queue until the operation finishes its task. You can’t
directly remove an operation from a queue after you add it.
Reference links:
https://developer.apple.com/documentation/foundation/operationqueue
https://www.hackingwithswift.com/example-code/system/how-to-use-multithreaded-operations-with-operationqueue

How to stop DispatchGroup or OperationQueue waiting?

DispatchGroup and OperationQueue have methods wait() and waitUntilAllOperationsAreFinished() which wait for all operations in respective queues to complete.
But even when I call cancelAllOperations it just changes the flag isCancelled in every running operation and stop the queue from executing new operations. But it still waits for the operations to complete. Therefore running the operations must be stopped from the inside. But it is possible only if operation is incremental or has an inner cycle of any kind. When it's just long external request (web request for example), there is no use of isCancelled variable.
Is there any way of stopping the OperationQueue or DispatchGroup waiting for the operations to complete if one of the operations decides that all queue is now outdated?
The practical case is: mapping a request to a list of responders, and it is known that only one may answer. If it happens, queue should stop waiting for other operations to finish and unlock the thread.
Edit: DispatchGroup and OperationQueue usage is not obligatory, these are just tools I thought would fit.
OK, so I think I came up with something. Results are stable, I've just tested. The answer is just one semaphore :)
let semaphore = DispatchSemaphore(value: 0)
let group = DispatchGroup()
let queue = DispatchQueue(label: "map-reduce", qos: .userInitiated, attributes: .concurrent)
let stopAtFirst = true // false for all results to be appended into one array
let values: [U] = <some input values>
let mapper: (U) throws -> T? = <closure>
var result: [T?] = []
for value in values {
queue.async(group: group) {
do {
let res = try mapper(value)
// appending must always be thread-safe, otherwise you end up with race condition and unstable results
DispatchQueue.global().sync {
result.append(res)
}
if stopAtFirst && res != nil {
semaphore.signal()
}
} catch let error {
print("Could not map value \"\(value)\" to mapper \(mapper): \(error)")
}
}
}
group.notify(queue: queue) { // this must be declared exactly after submitting all tasks, otherwise notification fires instantly
semaphore.signal()
}
if semaphore.wait(timeout: .init(secondsFromNow: 5)) == .timedOut {
print("MapReduce timed out on values \(values)")
}

How is it possible to perform multiple Alamofire requests that are finished one after another?

I would like to perform multiple Alamofire requests. However, because of data dependency a new request should only start when the previous is finished.
I already asked a question with a more general example of an asynchronous request which was solved with OperationQueue. However, I do not succeed to achieve the same with Alamofire.
public func performAlamofireRequest(_ number: Int, success: #escaping (Int) -> Void)->Void {
Alamofire.request(String(format: "http://jsonplaceholder.typicode.com/posts/%i", number+1)) // NSURLSession dispatch queue
.responseString { response in // Completion handler at main dispatch queue?
if response.result.isSuccess {
// print("data")
} else if response.result.isFailure {
// print("error")
}
success(number) // Always leave closure in this example
}
}
To assure that requests are finished before a next request is started, I use OperationQueue as follows:
let operationQueue = OperationQueue.main
for operationNumber in 0..<4 { // Create some operations
let operation = BlockOperation(block: {
performAlamofireRequest(operationNumber) { number in
print("Operation #\(number) finished")
}
})
operation.name = "Operation #\(operationNumber)"
if operationNumber > 0 {
operation.addDependency(operationQueue.operations.last!)
}
operationQueue.addOperation(operation)
}
However, the output is:
Operation #0 finished
Operation #3 finished
Operation #2 finished
Operation #1 finished
which is clearly not correct.
How would it be possible to achieve this with Alamofire?
The issue is just the same as in the related question you posed: the operation dependencies are on finishing an operation, as documented, but you have written code where the operation exits after asynchronously dispatching a request for future execution (the operations you created and added to a queue will finish in the order set by their dependencies, but the requests will be fired concurrently by the NSURLSession underlying Alamofire).
If you need serial execution, you can for instance do the following:
// you should create an operation queue, not use OperationQueue.main here –
// synchronous network IO that would end up waiting on main queue is a real bad idea.
let operationQueue = OperationQueue()
let timeout:TimeInterval = 30.0
for operationNumber in 0..<4 {
let operation = BlockOperation {
let s = DispatchSemaphore(value: 0)
self.performAlamofireRequest(operationNumber) { number in
// do stuff with the response.
s.signal()
}
// the timeout here is really an extra safety measure – the request itself should time out and end up firing the completion handler.
s.wait(timeout: DispatchTime(DispatchTime.now, Int64(timeout * Double(NSEC_PER_SEC))))
}
operationQueue.addOperation(operation)
}
Various other solutions are discussed in connection to this question, arguably a duplicate. There's also Alamofire-Synchronous.