I am utilizing URLSessions which are asynchronous in nature.
This works well when there is only one call for a session.
When I need to execute multiple (serial) calls, where the results need to be combined in the order of execution, it makes program logic painful and error prone.
I am also blocking the main thread, which isn't good.
Constraints
Moving to next task may not occur before 4 second elapses, though it can be more.
Utilizing Monterey (OS upgrade required), so using the let (data, response) = try await session.data(from: url) is not optional.
If tasks are executed faster than every 4 seconds, the result is a server side error, forcing a retry.
Task execution
Execute task -- if the task concludes in less than 4 seconds, wait
for the difference so the next task does not execute before the 4
seconds elapses.
Repeat process until all task have been completed.
Combine the results
My current process utilizes semaphores or dispatchGroups but both block the main thread.
Is there a way to get synchronous behavior without blocking the main thread?
func getDataFromInput_Sync(authToken: String, transformedText: String ) -> (Data?, URLResponse?, Error?)
{
/*
By default, transactions are asynchronous, so they return data while the rest of the program continues.
Changing the behavior to synchronous requires blocking to wait for the outcome. It affords us the
ability to manage program flow inline. This methods forces synchronous behavior and the output, which
are returned via a tuple, types: (Data?, URLResponse?, Error?)
*/
var outData : Data?
var outError : Error?
var urlResponse : URLResponse?
let targetURL = "https://..."
let urlconfig = URLSessionConfiguration.ephemeral
// set the timeout to a high number, if we are prepared to wait that long. Otherwise the session will timeout.
urlconfig.timeoutIntervalForRequest = 120
urlconfig.timeoutIntervalForResource = 120
let urlSession = URLSession(configuration: urlconfig)
// let dispatchGroup = DispatchGroup()
let semaphore = DispatchSemaphore(value: 0)
// ephermeral doesnt write cookies, cache or credentials to disk
guard let url = URL(string: targetURL),
let httpBodyData = transformedText.data(using: .utf8) else { return (nil,nil,nil) }
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.httpBody = httpBodyData
request.addValue("Token " + authToken, forHTTPHeaderField: "Authorization")
// Perform HTTP Request
let task = (urlSession).dataTask(with: request) { (data, response, error) in
guard error == nil
else {
print("we have an error: \(error!.localizedDescription)")
return
}
guard let data = data else { print("Empty data"); return }
outData = data
urlResponse = response
outError = error
// dispatchGroup.leave()
semaphore.signal()
}
task.resume()
semaphore.wait()
// dispatchGroup.enter()
// task.resume()
// dispatchGroup.wait()
return (outData, urlResponse, outError)
}
func testServerRequest()
{
let sentences = ["Example Sentence1","Example Sentence2","Example Sentence3"] //...array to process
for (_,thisString) in sentences.enumerated()
{
let timeTestPoint = Date()
let futureSecs = 4.0
let (data, urlResponse, error) = getDataFromInput_Sync(authToken: authToken, transformedText: thisString )
let elapsed = timeTestPoint.timeIntervalSinceNow * -1 // time elapsed between request and result
// act on the data received
// executing the next request before futureSecs will cause an error, so pause
let delayAmt = futureSecs - elapsed
Thread.sleep(forTimeInterval: delayAmt)
}
}
Make it in background queue, like
DispatchQueue.global(qos: .background).async {
testServerRequest()
}
Related
I'm trying to make a fairly simple API call in Swift but, for some reason, my dataTask code is not running. I've made sure that the .resume() is there. This code has worked in the past but, something has changed recently and I don't know what it is. The only thing I can think of is the url. I've changed the ingredients but, when putting the url into a browser, it returns JSON data normally. When running this function, I get two "Outside URLSession.shared.dataTask....." messages in a row with nothing in between, indicating that the URLSession block of code isn't running. I'm a little new to APIs so, any help would be greatly appreciated. Please let me know if there's any more information I can provide. Also, I'm on an older MacBook and am using Swift5 if that makes a difference. Thanks!
let url: URL! = URL(string: "https://api.spoonacular.com/recipes/findByIngredients?ingredients=" + ingredientString + "&apiKey=aaabbbccc111222333")
print("URL: " + url.absoluteString)
let request = URLRequest(url: url)
// Make the API call
print("Outide URLSession.shared.dataTask.....")
let session = URLSession.shared.dataTask(with: request) { data, response, error in
print("Inside URLSession.shared.dataTask.....")
DispatchQueue.main.async {
print("Inside DispatchQueue.main.async....")
if data == nil {
print("No data recieved.")
}
print("data != nil.... Moving on to JSONDecoder....")
self.model = try! JSONDecoder().decode([RecipeSearchElement].self, from: data!)
}
}
session.resume()
print("Outside URLSession.shared.dataTask.....")
Unrelated to your immediate question at hand (which I answered elsewhere), I would advise a few changes to the routine:
One should not build a URL through string interpolation. Use URLComponents. If, for example, the query parameter included a space or other character not permitted in a URL, URLComponents will percent-encode it for you. If do not percent-encode it properly, the building of the URL will fail.
I would avoid try!, which will crash the app if the server response was not what you expected. One should use try within a do-catch block, so it handles errors gracefully and will tell you what is wrong if it failed.
I would recommend renaming the URLSessionDataTask to be task, or something like that, to avoid conflating “sessions” with the “tasks” running on that session.
I would not advise updating the model from the background queue of the URLSession. Fetch and parse the response in the background queue and update the model on the main queue.
Thus:
var components = URLComponents(string: "https://api.spoonacular.com/recipes/findByIngredients")
components?.queryItems = [
URLQueryItem(name: "ingredients", value: ingredientString),
URLQueryItem(name: "apiKey", value: "aaabbbccc111222333")
]
guard let url = components?.url else {
print("Unable to build URL")
return
}
// Make the API call
let task = URLSession.shared.dataTask(with: url) { data, _, error in
DispatchQueue.main.async {
guard error == nil, let data = data else {
print("No data received:", error ?? URLError(.badServerResponse))
return
}
do {
let model = try JSONDecoder().decode([RecipeSearchElement].self, from: data)
DispatchQueue.main.async {
self.model = model
}
} catch let parseError {
print("Parsing error:", parseError, String(describing: String(data: data, encoding: .utf8)))
}
}
}
task.resume()
In a more advanced observation, I would never have a network call update the model directly. I would leave that to the caller. For example, you could use a completion handler pattern:
#discardableResult
func fetchIngredients(
_ ingredientString: String,
completion: #escaping (Result<[RecipeSearchElement], Error>) -> Void
) -> URLSessionTask? {
var components = URLComponents(string: "https://api.spoonacular.com/recipes/findByIngredients")
components?.queryItems = [
URLQueryItem(name: "ingredients", value: ingredientString),
URLQueryItem(name: "apiKey", value: "aaabbbccc111222333")
]
guard let url = components?.url else {
completion(.failure(URLError(.badURL)))
return nil
}
// Make the API call
let task = URLSession.shared.dataTask(with: url) { data, _, error in
print("Inside URLSession.shared.dataTask.....")
DispatchQueue.main.async {
guard error == nil, let data = data else {
DispatchQueue.main.async {
completion(.failure(error ?? URLError(.badServerResponse)))
}
return
}
do {
let model = try JSONDecoder().decode([RecipeSearchElement].self, from: data)
DispatchQueue.main.async {
completion(.success(model))
}
} catch let parseError {
DispatchQueue.main.async {
completion(.failure(parseError))
}
}
}
}
task.resume()
return task
}
And then the caller could do:
fetchIngredients(ingredientString) { [weak self] result in
switch result {
case .failure(let error): print(error)
case .success(let elements): self?.model = elements
}
}
This has two benefits:
The caller now knows when the model is updated, so you can update your UI at the appropriate point in time (if you want).
It maintains a better separation of responsibilities, architecturally avoiding the tight coupling of the network layer with that of the view or view model (or presenter or controller) layers.
Note, I am also returning the URLSessionTask object in case the caller would like to cancel it at a later time, but I made it an #discardableResult so that you do not have to worry about that if you are not tackling cancelation at this point.
If you (a) are reaching the “outside” message, but not seeing the “inside” message; and (b) are absolutely positive that you are reaching the resume statement, it is one of a few possibilities:
The app may be terminating before the asynchronous request has time to finish. This can happen, for example, if this is a command-line app and you are allowing the app to quit before the asynchronous request has a chance to finish. If you want a command-line app to wait for a network request to finish, you might run a RunLoop that does not exit until the network request is done.
It can also happen if you use a playground and neglect to set needsIndefiniteExecution:
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
For the sake of completeness, there are a few other, less common, possibilities:
You have some other network request whose completion handler is blocked/deadlocked, thereby preventing anything else from running on the URLSession dedicated, serial, queue.
You have thread explosion somewhere else in your code, exhausting the limited pool of worker threads, preventing other tasks/operations from being able to get an available worker thread.
In a swift 2 command line tool (main.swift), I have the following:
import Foundation
print("yay")
var request = HTTPTask()
request.GET("http://www.stackoverflow.com", parameters: nil, completionHandler: {(response: HTTPResponse) in
if let err = response.error {
print("error: \(err.localizedDescription)")
return //also notify app of failure as needed
}
if let data = response.responseObject as? NSData {
let str = NSString(data: data, encoding: NSUTF8StringEncoding)
print("response: \(str)") //prints the HTML of the page
}
})
The console shows 'yay' and then exits (Program ended with exit code: 0), seemingly without ever waiting for the request to complete. How would I prevent this from happening?
The code is using swiftHTTP
I think I might need an NSRunLoop but there is no swift example
Adding RunLoop.main.run() to the end of the file is one option. More info on another approach using a semaphore here
I realize this is an old question, but here is the solution I ended on. Using DispatchGroup.
let dispatchGroup = DispatchGroup()
for someItem in items {
dispatchGroup.enter()
doSomeAsyncWork(item: someItem) {
dispatchGroup.leave()
}
}
dispatchGroup.notify(queue: DispatchQueue.main) {
exit(EXIT_SUCCESS)
}
dispatchMain()
You can call dispatchMain() at the end of main. That runs the GCD main queue dispatcher and never returns so it will prevent the main thread from exiting. Then you just need to explicitly call exit() to exit the application when you are ready (otherwise the command line app will hang).
import Foundation
let url = URL(string:"http://www.stackoverflow.com")!
let dataTask = URLSession.shared.dataTask(with:url) { (data, response, error) in
// handle the network response
print("data=\(data)")
print("response=\(response)")
print("error=\(error)")
// explicitly exit the program after response is handled
exit(EXIT_SUCCESS)
}
dataTask.resume()
// Run GCD main dispatcher, this function never returns, call exit() elsewhere to quit the program or it will hang
dispatchMain()
Don't depend on timing.. You should try this
let sema = DispatchSemaphore(value: 0)
let url = URL(string: "https://upload.wikimedia.org/wikipedia/commons/4/4d/Cat_November_2010-1a.jpg")!
let task = URLSession.shared.dataTask(with: url) { data, response, error in
print("after image is downloaded")
// signals the process to continue
sema.signal()
}
task.resume()
// sets the process to wait
sema.wait()
If your need isn't something that requires "production level" code but some quick experiment or a tryout of a piece of code, you can do it like this :
SWIFT 3
//put at the end of your main file
RunLoop.main.run(until: Date(timeIntervalSinceNow: 15)) //will run your app for 15 seconds only
More info : https://stackoverflow.com/a/40870157/469614
Please note that you shouldn't rely on fixed execution time in your architecture.
Swift 4: RunLoop.main.run()
At the end of your file
// Step 1: Add isDone global flag
var isDone = false
// Step 2: Set isDone to true in callback
request.GET(...) {
...
isDone = true
}
// Step 3: Add waiting block at the end of code
while(!isDone) {
// run your code for 0.1 second
RunLoop.main.run(until: Date(timeIntervalSinceNow: 0.1))
}
Using Swift 4, I have this code that attempts a POST request to a REST API:
spinner.startAnimation(self)
btnOk.isEnabled = false
btnCancel.isEnabled = false
attemptPost()
spinner.stopAnimation(self)
btnOk.isEnabled = true
btnCancel.isEnabled = true
The function that does this (Constants and Request are classes that I created that create the request objects and hold frequently used data):
func attemptPost() {
let url = Constants.SERVICE_URL + "account/post"
let body: [String : Any] =
["firstName": txtFirstName.stringValue,
"lastName": txtLastName.stringValue,
"email": txtEmail.stringValue,
"password": txtPassword.stringValue];
let req = Request.create(urlExtension: url, httpVerb: Constants.HTTP_POST, jsonBody: body)
let task = URLSession.shared.dataTask(with: req) { data, response, err in
guard let data = data, err == nil else {
// error
return
}
if let resp = try? JSONSerialization.jsonObject(with: data) {
// success
}
}
task.resume()
}
Since the task that does this runs asynchronously, there is no sequential way that I can update the UI once the call to attemptPost() returns. And since the UI components are on the main thread, I can't directly update the components from the task that makes the request.
In C# it works the same way; there is a BackgroundWorker class in which you can safely update the UI components to avoid a "Cross-thread operation not valid" error.
I'm trying to find an example that accomplishes more or less the same thing, in which a "wait" state is established, the task runs, and upon task completion, the main thread is notified that the task is done so that the wait state can be changed.
But I'm still having trouble understanding how this all comes together in Swift. I've looked around and seen information about the handlers that are invoked from within URLSessionDataTask and stuff about GCD, but I'm still not able to connect the dots.
And is GCD even relevant here since the URLSessionDataTask task is asynchronous to begin with?
Any help is appreciated.
If I understood correctly you might try this solution:
spinner.startAnimation(self)
btnOk.isEnabled = false
btnCancel.isEnabled = false
attemptPost { (success) in
DispatchQueue.main.async {
spinner.stopAnimation(self)
btnOk.isEnabled = true
btnCancel.isEnabled = true
}
// UI wise, eventually you can do something with 'success'
}
func attemptPost(_ completion:#escaping (Bool)->())
let url = Constants.SERVICE_URL + "account/post"
let body: [String : Any] =
["firstName": txtFirstName.stringValue,
"lastName": txtLastName.stringValue,
"email": txtEmail.stringValue,
"password": txtPassword.stringValue];
let req = Request.create(urlExtension: url, httpVerb: Constants.HTTP_POST, jsonBody: body)
let task = URLSession.shared.dataTask(with: req) { data, response, err in
guard let data = data, err == nil else {
completion(false)
return
}
if let resp = try? JSONSerialization.jsonObject(with: data) {
completion(true)
}
}
task.resume()
}
so the idea is executing from attemptPost a block which will run asynchronously into the main thread your UI stuff
In my project for OS X Command Line Tool using Swift 3.0 (Beta 2) I need to convert HTML data to String from multiple URLs. There is a problem in use of such function with many background tasks (it's not working except the main thread, so maybe there is more elegant way to control completion of all tasks and read HTML data in such tool with or without parsers that I need for Swift 3 and Mac OS X (Linux in the near future)):
func html2text (html: String, usedEncoding: String.Encoding) -> String {
let data = html.data(using: usedEncoding)!
if let htmlString = AttributedString(html: data, options: [NSDocumentTypeDocumentAttribute: NSHTMLTextDocumentType, NSCharacterEncodingDocumentAttribute: usedEncoding.rawValue], documentAttributes: nil)?.string {
return htmlString
} else {
return ""
}
}
So I read data first into an Array, waiting when all DataTasks finished and then converting it in the main thread. Also using global variable (Set of urls) to control completion of each task:
import Foundation
import WebKit
var urlArr = [String]()
var urlSet = Set<String>()
var htmlTup : [(url : String, html : String, encoding : String.Encoding)] = []
let session = URLSession.shared
For-in loop with multiple URLSession DataTasks
for myurl in urlArr {
if urlSet.insert(myurl).inserted {
print ("Loading \(myurl)...")
let inputURL = URL(string: myurl)!
let task = session.dataTask(with: inputURL, completionHandler: {mydata, response, error in
Read Encoding from HTML First
var usedEncoding = String.Encoding.utf8
if let encodingName = response!.textEncodingName {
let encoding = CFStringConvertIANACharSetNameToEncoding(encodingName)
if encoding != kCFStringEncodingInvalidId {
usedEncoding = String.Encoding(rawValue: CFStringConvertEncodingToNSStringEncoding(encoding))
}
}
Do some work with HTML String and read data into an Array
if let myString = String(data: mydata!, encoding: usedEncoding) {
htmlTup += [(url: myurl,html: myString, encoding: usedEncoding)]
}
// The end of task removing URL from Set
urlSet.remove(myurl)
})
//Run Task
task.resume()
}
}
}
Waiting for tasks to complete and convert HTML to text
while !urlSet.isEmpty {
// Do nothing
}
for (url,html,encoding) in htmlTup {
print ("Writing data from \(url)...")
print (html2text(html: html, usedEncoding: encoding))
}
Update 1: RunLoop in the main thread from this
Such code to check when each task finished:
var taskArr = [Bool]()
let task = session.dataTask(with: request) { (data, response, error) in
}
taskArr.removeLast()
}
taskArr.append(true)
task.resume()
// Waiting for tasks to complete
let theRL = RunLoop.current
while !taskArr.isEmpty && theRL.run(mode: .defaultRunLoopMode, before: .distantFuture) { }
You can't just spin in a busy loop waiting for results, because you're blocking the main run loop/thread/dispatch queue by doing that.
Instead, return at that point, thus allowing the main run loop to run. Then, in your completion handler, check to see if you've gotten all the responses you're expecting, and if so, do the stuff that you currently have after that busy wait while loop.
I'm entering the concurrency programming with some semaphore issues.
My function first loads data from server, analyze received info and then, if necessary, makes second request to server.
I tried different ways to make it run, none of them did it well.
My current code FOR ME seems to be correct, but on second request it just locks(maybe like a DeadLock) and the last log is "<__NSCFLocalDataTask: 0x7ff470c58c90>{ taskIdentifier: 2 } { suspended }"
Please, tell me what do I don't know. Maybe there is more elegant way to work with completions for these purposes?
Thank you in advance!
var users = [Int]()
let linkURL = URL.init(string: "https://bla bla")
let session = URLSession.shared()
let semaphore = DispatchSemaphore.init(value: 0)
let dataRequest = session.dataTask(with:linkURL!) { (data, response, error) in
let json = JSON (data: data!)
if (json["queue"]["numbers"].intValue>999) {
for i in 0...999 {
users.append(json["queue"]["values"][i].intValue)
}
for i in 1...lround(json["queue"]["numbers"].doubleValue/1000) {
let session2 = URLSession.shared()
let semaphore2 = DispatchSemaphore.init(value: 0)
let linkURL = URL.init(string: "https://bla bla")
let dataRequest2 = session2.dataTask(with:linkURL!) { (data, response, error) in
let json = JSON (data: data!)
print(i)
semaphore2.signal()
}
dataRequest2.resume()
semaphore2.wait(timeout: DispatchTime.distantFuture)
}
}
semaphore.signal()
}
dataRequest.resume()
semaphore.wait(timeout: DispatchTime.distantFuture)
P.S. Why do I do it. Server returns limited count of data. To get more, I have to use offset.
This is deadlocking because you are waiting for a semaphore on the URLSession's delegateQueue. The default delegate queue is not the main queue, but it is a serial background queue (i.e. an OperationQueue with a maxConcurrentOperationCount of 1). So your code is waiting for a semaphore on the same serial queue that is supposed to be signaling the semaphore.
The tactical fix is to make sure you're not calling wait on the same serial queue that the session's completion handlers are running on. There are two obvious fixes:
Do not use shared session (whose delegateQueue is a serial queue), but rather instantiate your own URLSession and specify its delegateQueue to be a concurrent OperationQueue that you create:
let queue = OperationQueue()
queue.name = "com.domain.app.networkqueue"
let configuration = URLSessionConfiguration.default()
let session = URLSession(configuration: configuration, delegate: nil, delegateQueue: queue)
Alternatively, you can solve this by dispatching the code with the semaphore off to some other queue, e.g.
let mainRequest = session.dataTask(with: mainUrl) { data, response, error in
// ...
DispatchQueue.global(attributes: .qosUserInitiated).async {
let semaphore = DispatchSemaphore(value: 0)
for i in 1 ... n {
let childUrl = URL(string: "https://blabla/\(i)")!
let childRequest = session.dataTask(with: childUrl) { data, response, error in
// ...
semaphore.signal()
}
childRequest.resume()
_ = semaphore.wait(timeout: .distantFuture)
}
}
}
mainRequest.resume()
For the sake of completeness, I'll note that you probably shouldn't be using semaphores to issue these requests at all, because you'll end up paying a material performance penalty for issuing a series of consecutive requests (plus you're blocking a thread, which is generally discouraged).
The refactoring of this code to do that is a little more considerable. It basically entails issuing a series of concurrent requests, perhaps use "download" tasks rather than "data" tasks to minimize memory impact, and then when all of the requests are done, piece it all together as needed at the end (triggered by either a Operation "completion" operation or dispatch group notification).