I would like to perform multiple Alamofire requests. However, because of data dependency a new request should only start when the previous is finished.
I already asked a question with a more general example of an asynchronous request which was solved with OperationQueue. However, I do not succeed to achieve the same with Alamofire.
public func performAlamofireRequest(_ number: Int, success: #escaping (Int) -> Void)->Void {
Alamofire.request(String(format: "http://jsonplaceholder.typicode.com/posts/%i", number+1)) // NSURLSession dispatch queue
.responseString { response in // Completion handler at main dispatch queue?
if response.result.isSuccess {
// print("data")
} else if response.result.isFailure {
// print("error")
}
success(number) // Always leave closure in this example
}
}
To assure that requests are finished before a next request is started, I use OperationQueue as follows:
let operationQueue = OperationQueue.main
for operationNumber in 0..<4 { // Create some operations
let operation = BlockOperation(block: {
performAlamofireRequest(operationNumber) { number in
print("Operation #\(number) finished")
}
})
operation.name = "Operation #\(operationNumber)"
if operationNumber > 0 {
operation.addDependency(operationQueue.operations.last!)
}
operationQueue.addOperation(operation)
}
However, the output is:
Operation #0 finished
Operation #3 finished
Operation #2 finished
Operation #1 finished
which is clearly not correct.
How would it be possible to achieve this with Alamofire?
The issue is just the same as in the related question you posed: the operation dependencies are on finishing an operation, as documented, but you have written code where the operation exits after asynchronously dispatching a request for future execution (the operations you created and added to a queue will finish in the order set by their dependencies, but the requests will be fired concurrently by the NSURLSession underlying Alamofire).
If you need serial execution, you can for instance do the following:
// you should create an operation queue, not use OperationQueue.main here –
// synchronous network IO that would end up waiting on main queue is a real bad idea.
let operationQueue = OperationQueue()
let timeout:TimeInterval = 30.0
for operationNumber in 0..<4 {
let operation = BlockOperation {
let s = DispatchSemaphore(value: 0)
self.performAlamofireRequest(operationNumber) { number in
// do stuff with the response.
s.signal()
}
// the timeout here is really an extra safety measure – the request itself should time out and end up firing the completion handler.
s.wait(timeout: DispatchTime(DispatchTime.now, Int64(timeout * Double(NSEC_PER_SEC))))
}
operationQueue.addOperation(operation)
}
Various other solutions are discussed in connection to this question, arguably a duplicate. There's also Alamofire-Synchronous.
Related
I have a func that gets a list of Players. When i fetch the players i need only to show those who belongs to the current Team so i am showing only a subset of the original list by filtering them. I don't know in advance, before making the request, how much players belong to the Team selected by the User, so i may need to do additional requests until i can display on the TableView at least 10 rows of Players. The User by pulling up from the bottom of the TableView can request more players to display. To do this i am calling a first async func request which in turn calls, inside a while, another nested async func request. Here a code to give you an idea of what i am trying to do:
let semaphore = DispatchSemaphore(value: 0)
func getTeamPlayersRequest() {
service.getTeamPlayers(...)
{
(result) in
switch result
{
case .success(let playersModel):
if let validCurrentPage = currentPageTmp ,
let validTotalPages = totalPagesTmp ,
let validNextPage = self.getTeamPlayersListNextPage()
{
while self.playersToShowTemp.count < 10 && self.currentPage < validTotalPages
{
self.currentPage = validNextPage //global var
self.fetchMorePlayers()
self.semaphore.wait() //global semaphore
}
}
case .failure(let error):
//some code...
}
})
}
private func fetchMorePlayers(){
// Completion handler of the following function is never called..
service.getTeamPlayers(requestedPage: currentPage, completion: {
(result) in
switch result
{
case .success(let playersModel):
if let validPlayerList = playersList,
let validPlayerListData = validPlayerList.data,
let validTeamModel = self.teamPlayerModel,
let validNextPage = self.getTeamPlayersListNextPage()
{
for player in validPlayerListData
{
if ( validTeamModel.id == player.team?.id)
{
self.playersToShowTemp.append(player)
}
}
}
self.currentPage = validNextPage
self.semaphore.signal() //global semaphore
case .failure(let error):
//some code...
}
}
}
I have tried both with DispatchGroup and Semaphore but i don't get it what i am doing wrong. I debugged the code and saw that the first async call get executed in a different queue (not the main queue) and a different thread. The nested async call getexecuted on a different thread but i don't know if it's the same concurrent queue of the first async call.
The completion handler of thenested call it's never called. Does anyone know why? is the self.semaphore.wait(), even if it get executed after the fetchMorePlayers() return, blocking/preventing the nested async completion handler to be called?
I am noticing through the Debugger that the completion() in the Xcode vars window has the note "swift partial apply forwarder for closure #1"
If we inline the function call in your loop, it looks something like this:
while self.playersToShowTemp.count < 10 && self.currentPage < validTotalPages
{
self.currentPage = validNextPage //global var
nbaService.getTeamPlayers(requestedPage: currentPage, completion: { ... })
self.semaphore.wait() //global semaphore
}
So nbaService.getTeamPlayers schedules a request, probably on the main DispatchQueue and immediately returns. Then you call wait on your semaphore, which blocks, probably before GCD even tries to run the task scheduled by nbaService.getTeamPlayers.
That's a problem on DispatchQueue.main, which is a serial queue. It has to be a serial queue for UI updates to work. What normally happens is on some iteration of the run loop you make a request, and return.. that bubbles back up to the run loop, which checks for more events and queued tasks. In this case, when your completion handler in getTeamPlayersRequest is waiting to be run, the run loop (via GCD) executes it for that iteration. Then you block the main thread, so the run loop can't continue. If you do need to block always do it on a different DispatchQueue, preferably a .concurrent one.
There is sometimes confusion about what .async does. It only means "run this later and right now return control back to the caller". That's all. It does not guarantee that your closure will run concurrently. It merely schedules it to be run later (possibly soon) on whatever DispatchQueue you called it on. If that queue is a serial queue, then it will be queued to run in its turn in that dispatch queue's run loop. If it's a concurrent queue (ie one you specifically set the attributes to include .concurrent). Then it will run, possibly at the same time as other tasks on that same DispatchQueue.
To avoid that instead of using a loop you can use async-chaining.
private func fetchMorePlayers(while condition: #autoclosure #escaping () -> Bool){
guard condition() else { return }
nbaService.getTeamPlayers(requestedPage: currentPage, completion: {
(result) in
switch result
{
case .success(let playersModel):
if let validPlayerList = playersList,
let validPlayerListData = validPlayerList.data,
let validTeamModel = self.teamPlayerModel,
let validNextPage = self.getTeamPlayersListNextPage()
{
for player in validPlayerListData
{
if ( validTeamModel.id == player.team?.id)
{
self.playersToShowTemp.append(player)
}
}
}
self.currentPage = validNextPage
// Chain to next call
self.fetchMorePlayers(while: condition))
case .failure(let error):
//some code...
}
}
}
Then in getTeamPlayersRequest you can do this:
func getTeamPlayersRequest() {
service.getTeamPlayers(...)
{
(result) in
switch result
{
case .success(let playersModel):
if let validCurrentPage = currentPageTmp ,
let validTotalPages = totalPagesTmp ,
let validNextPage = self.getTeamPlayersListNextPage()
{
self.currentPage = validNextPage //global var
self.fetchMorePlayers(while: self.playersToShowTemp.count < 10 && self.currentPage < validTotalPages)
}
case .failure(let error):
//some code...
}
})
}
This avoids the need to block on a semaphore, because each subsequent request happens in the completion handler of the previously completed one. The only issue is if you need for the completion handler in getTeamPlayersRequest to block while the fetchMorePlayers requests are being fetched, because now it won't you can re-introduce the semaphore. In that case the guard statement in fetchMorePlayers becomes:
guard condition() else
{
self.semaphore.signal()
return
}
That way it only signals on the last completion handler in the chain. You may need to block in a different DispatchQueue though. I think if you need to block, you probably have something about your design that needs to be reconsidered.
If you find yourself reaching for semaphores, it is almost always a mistake. Semaphores are inefficient at best, and introduce deadlock risks if misused. Semaphores should generally be avoided. (Don't get me wrong: Semaphores can be useful in some very narrow use cases, but this is not one of them.)
Use asynchronous patterns. One simple approach might be to recursively call the routine, calling the completion handler when done:
func startFetching(#escaping completion: () -> Void) {
fetchPlayers(page: 0, completion: completion)
}
private func fetchPlayers(page: Int, #escaping completion: () -> Void) {
// prepare request
// now perform request
performRequest(...) { ...
if let error = error {
completion()
return
}
...
if doesNeedMorePlayers {
fetchPlayers(page: page + 1, completion: completion)
} else {
completion()
}
}
}
Personally, I might probably add another closure to emit the players retrieved as we go along, e.g. like, if not actually, a Combine Publisher. Or if you want to update the UI all at once at the very end, just pass the players retrieved thus far as additional parameter in this recursive routine and pass the whole array back in the completion handler. But avoid globals or other state properties.
But the broader idea is to scrupulously avoid semaphores and instead embrace asynchronous patterns.
Have had trouble visualising this...
I think I need 2 threads and am not sure how to implement them.
I need to have a 2 second delay minimum in between each network retrieval requests after it has been completed.
Main Thread:
UIButton pressed --> Function adds a network retrieval request to a serial queue --> UILoop continues...
Network Thread:
Checks queue for next request --> Begins request --> Finishes request --> Waits 2 seconds --> Checks queue for next request --> Begins request --> Finishes request --> Waits 2 seconds --> Checks queue for next request --> No request --> Checks queue for next request... or Ends loop until recalled.
var networkQueue = [NetworkRequest]()
var networkQueueActive = false
#IBAction func buttonPressed(_ sender: UIButton) {
networkQueue.append(NetworkRequest(UIButton))
if networkQueueActive == false {
networkRetrieveFromQueue() // need to asynchronously call this DON'T KNOW HOW TO DO THIS
}
}
func networkRetrieveFromQueue() {
networkQueueActive = true
while !networkQueue.isEmpty {
let request = networkQueue.remove(at: 0)
// synchronous network data retrieval on this thread KNOW HOW TO DO THIS
// do something with the data retrieved KNOW HOW TO DO THIS
// wait 2 seconds DON'T KNOW HOW TO DO THIS
}
networkQueueActive = false
}
If you have
var networkQueue = [NetworkRequest]()
var networkQueueActive = false
Then, your networkRetrieveFromQueue should:
check to see if the queue is empty;
if not, grab the first item in the queue;
initiate the asynchronous request; and
in the completion handler of that asynchronous request, call networkRetrieveFromQueue again after 2 seconds
Thus
func startQueue() {
if networkQueueActive { return }
networkQueueActive = true
processNext()
}
// if queue not empty, grab first item, perform request, and call itself
// 2 seconds after prior one finishes
func processNext() {
if networkQueue.isEmpty {
networkQueueActive = false
return
}
let request = networkQueue.removeFirst()
get(request: request) {
DispatchQueue.main.asyncAfter(deadline: .now() + 2) {
self.processNext()
}
}
}
Where your "process request" might look like:
// perform asynchronous network request, with completion handler that is
// called when its done
func get(request: NetworkRequest, completionHandler: #escaping () -> Void) {
let task = URLSession.shared.dataTask(with: request.request) { data, _, error in
guard let data = data, error == nil else {
print(error ?? "Unknown error")
completionHandler()
return
}
// process successful response here
// when done, call completion handler
completionHandler()
}
task.resume()
}
Now, I don't know what your NetworkRequest looks like, but this illustrates the basic idea of how to recursively call a function in the completion handler of some asynchronous method.
DispatchGroup and OperationQueue have methods wait() and waitUntilAllOperationsAreFinished() which wait for all operations in respective queues to complete.
But even when I call cancelAllOperations it just changes the flag isCancelled in every running operation and stop the queue from executing new operations. But it still waits for the operations to complete. Therefore running the operations must be stopped from the inside. But it is possible only if operation is incremental or has an inner cycle of any kind. When it's just long external request (web request for example), there is no use of isCancelled variable.
Is there any way of stopping the OperationQueue or DispatchGroup waiting for the operations to complete if one of the operations decides that all queue is now outdated?
The practical case is: mapping a request to a list of responders, and it is known that only one may answer. If it happens, queue should stop waiting for other operations to finish and unlock the thread.
Edit: DispatchGroup and OperationQueue usage is not obligatory, these are just tools I thought would fit.
OK, so I think I came up with something. Results are stable, I've just tested. The answer is just one semaphore :)
let semaphore = DispatchSemaphore(value: 0)
let group = DispatchGroup()
let queue = DispatchQueue(label: "map-reduce", qos: .userInitiated, attributes: .concurrent)
let stopAtFirst = true // false for all results to be appended into one array
let values: [U] = <some input values>
let mapper: (U) throws -> T? = <closure>
var result: [T?] = []
for value in values {
queue.async(group: group) {
do {
let res = try mapper(value)
// appending must always be thread-safe, otherwise you end up with race condition and unstable results
DispatchQueue.global().sync {
result.append(res)
}
if stopAtFirst && res != nil {
semaphore.signal()
}
} catch let error {
print("Could not map value \"\(value)\" to mapper \(mapper): \(error)")
}
}
}
group.notify(queue: queue) { // this must be declared exactly after submitting all tasks, otherwise notification fires instantly
semaphore.signal()
}
if semaphore.wait(timeout: .init(secondsFromNow: 5)) == .timedOut {
print("MapReduce timed out on values \(values)")
}
I would like to perform a set of serial asynchronous requests in combination with Realm.
The set of requests I want to process are used to update a remote server and are determined from an array of structs containing the object type and local uuid. The related objects are fetched from a Realm database and subsequently written to the server using Alamofire.
However, fetching the Realm objects result in an error (Realm accessed from incorrect thread).
func performAsyncRequest<T: Object>(objectType: T.Type, uuid: String, failure fail: (()->Void)? = nil, success succeed: #escaping () -> Void)->Void {
let realm = try! Realm()
let dataObject = realm.objects(objectType).filter("uuid == %#", uuid).first!
let parameters = self.toJson(item: dataObject)
// ** The Realm error occurs when the Alamofire request is performed **
let urlRequest = self.getRequest(objectType: T.self, with: parameters)
self.alamoFireManager.request(urlRequest) // Perform POST request
.responseString { response in
if let status = response.response?.statusCode {
if status >= 200 && status <= 299 {
succeed() // Is not reached in combination with DispatchSemaphore
} else if status >= 400 && status <= 499 {
fail?() // Is not reached in combination with DispatchSemaphore
}
}
}
}
Edit: The code below is edited after the answer below (where a previous problem with the serial Alamofire request is solved).
In order to perform the Alamofire requests serially, OperationQueue is used in combination with DispatchSemaphore.
let operationQueue = OperationQueue()
var operation: Operation!
for requestData in requests { // requestData is a struct with the object Type and a uuid
switch requestData.objectType {
case is Object1.Type:
operation = BlockOperation(block: {
let semaphore = DispatchSemaphore(value: 0)
self.performAsyncRequest(objectType: Object1.self, uuid: requestData.uuid, failure: { error in
semaphore.signal()
}) {
semaphore.signal()
}
semaphore.wait()
})
case is Object2.Type:
// ... same as for Object1 but now for Object2
// .. and so on for other Objects
}
operationQueue.addOperation(operation)
}
As indicated in the answer below the error occurs because Realm is thread confined. However, it is unclear to me why Realm instances are passed across different threads.
With an exception breakpoint I determined that the error occurs on thread Queue: NSOperationQueue 0x… (QOS: UTILITY) (serial). This is a different thread from where BlockOperation is performed (and thus where the Realm objects are fetched). Why are the methods inside BlockOperation not performed on the same thread as NSOperationQueue?
I would appreciate any ideas to handle these problems.
Realm and Realm objects are thread confinement. You should retrieve new realm instances on each thread. Do not pass a Realm instance across other threads.
It seems the main thread stops due to waiting for the semaphore. Then Alamofire callbacks are executed on the main thread. So semaphore.signal() is never called because the main thread stops. Alamofire's response* methods can specify queue a callback is called.
I'm entering the concurrency programming with some semaphore issues.
My function first loads data from server, analyze received info and then, if necessary, makes second request to server.
I tried different ways to make it run, none of them did it well.
My current code FOR ME seems to be correct, but on second request it just locks(maybe like a DeadLock) and the last log is "<__NSCFLocalDataTask: 0x7ff470c58c90>{ taskIdentifier: 2 } { suspended }"
Please, tell me what do I don't know. Maybe there is more elegant way to work with completions for these purposes?
Thank you in advance!
var users = [Int]()
let linkURL = URL.init(string: "https://bla bla")
let session = URLSession.shared()
let semaphore = DispatchSemaphore.init(value: 0)
let dataRequest = session.dataTask(with:linkURL!) { (data, response, error) in
let json = JSON (data: data!)
if (json["queue"]["numbers"].intValue>999) {
for i in 0...999 {
users.append(json["queue"]["values"][i].intValue)
}
for i in 1...lround(json["queue"]["numbers"].doubleValue/1000) {
let session2 = URLSession.shared()
let semaphore2 = DispatchSemaphore.init(value: 0)
let linkURL = URL.init(string: "https://bla bla")
let dataRequest2 = session2.dataTask(with:linkURL!) { (data, response, error) in
let json = JSON (data: data!)
print(i)
semaphore2.signal()
}
dataRequest2.resume()
semaphore2.wait(timeout: DispatchTime.distantFuture)
}
}
semaphore.signal()
}
dataRequest.resume()
semaphore.wait(timeout: DispatchTime.distantFuture)
P.S. Why do I do it. Server returns limited count of data. To get more, I have to use offset.
This is deadlocking because you are waiting for a semaphore on the URLSession's delegateQueue. The default delegate queue is not the main queue, but it is a serial background queue (i.e. an OperationQueue with a maxConcurrentOperationCount of 1). So your code is waiting for a semaphore on the same serial queue that is supposed to be signaling the semaphore.
The tactical fix is to make sure you're not calling wait on the same serial queue that the session's completion handlers are running on. There are two obvious fixes:
Do not use shared session (whose delegateQueue is a serial queue), but rather instantiate your own URLSession and specify its delegateQueue to be a concurrent OperationQueue that you create:
let queue = OperationQueue()
queue.name = "com.domain.app.networkqueue"
let configuration = URLSessionConfiguration.default()
let session = URLSession(configuration: configuration, delegate: nil, delegateQueue: queue)
Alternatively, you can solve this by dispatching the code with the semaphore off to some other queue, e.g.
let mainRequest = session.dataTask(with: mainUrl) { data, response, error in
// ...
DispatchQueue.global(attributes: .qosUserInitiated).async {
let semaphore = DispatchSemaphore(value: 0)
for i in 1 ... n {
let childUrl = URL(string: "https://blabla/\(i)")!
let childRequest = session.dataTask(with: childUrl) { data, response, error in
// ...
semaphore.signal()
}
childRequest.resume()
_ = semaphore.wait(timeout: .distantFuture)
}
}
}
mainRequest.resume()
For the sake of completeness, I'll note that you probably shouldn't be using semaphores to issue these requests at all, because you'll end up paying a material performance penalty for issuing a series of consecutive requests (plus you're blocking a thread, which is generally discouraged).
The refactoring of this code to do that is a little more considerable. It basically entails issuing a series of concurrent requests, perhaps use "download" tasks rather than "data" tasks to minimize memory impact, and then when all of the requests are done, piece it all together as needed at the end (triggered by either a Operation "completion" operation or dispatch group notification).