How can you use Dispatch Groups to wait to call multiple functions that depend on different data? - swift

I have three variables, a, b and c. I have three asynchronous functions with completion blocks to update these variables and three more functions that do some work with only some of the data.
I'm making sure that the working functions wait until all the data is updated with a DispatchGroup.
// The Data
var a: String?
var b: String?
var c: String?
// The Update
let group = DispatchGroup()
group.enter()
updateA() {
group.leave()
}
group.enter()
updateB() {
group.leave()
}
group.enter()
updateC() {
group.leave()
}
group.wait()
// The work
doSomthingWith(a: a, b: b)
doSomethingElseWith(b: b, c: c)
doAnotherThingWith(a: a, c: c)
What I'd like to be able to do is call each work function once the parameters have been updated, rather than waiting for everything. This is only a (obviously) simplified version. There could be many more variables and functions.
I'm using Swift. Many thanks in advance.

To achieve that with dispatch groups alone you would need three
dispatch groups which are entered and left accordingly:
let abGroup = DispatchGroup()
let bcGroup = DispatchGroup()
let acGroup = DispatchGroup()
abGroup.enter()
abGroup.enter()
bcGroup.enter()
bcGroup.enter()
acGroup.enter()
acGroup.enter()
// When a is updated:
abGroup.leave()
acGroup.leave()
// When b is updated:
abGroup.leave()
bcGroup.leave()
// When c is updated:
acGroup.leave()
bcGroup.leave()
Then you can wait for the completion of each group independently
abGroup.notify(queue: .main) {
// Do something with a and b
}
bcGroup.notify(queue: .main) {
// Do something with b and c
}
acGroup.notify(queue: .main) {
// Do something with a and c
}
However, this does not scale well with more tasks and dependencies.
The better approach is to add Operations to an
OperationQueue, that allows to add arbitrary dependencies:
let queue = OperationQueue()
let updateA = BlockOperation {
// ...
}
queue.addOperation(updateA)
let updateB = BlockOperation {
// ...
}
queue.addOperation(updateB)
let updateC = BlockOperation {
// ...
}
queue.addOperation(updateC)
let doSomethingWithAandB = BlockOperation {
// ...
}
doSomethingWithAandB.addDependency(updateA)
doSomethingWithAandB.addDependency(updateB)
queue.addOperation(doSomethingWithAandB)
let doSomethingWithBandC = BlockOperation {
// ...
}
doSomethingWithBandC.addDependency(updateB)
doSomethingWithBandC.addDependency(updateC)
queue.addOperation(doSomethingWithBandC)
let doSomethingWithAandC = BlockOperation {
// ...
}
doSomethingWithAandC.addDependency(updateA)
doSomethingWithAandC.addDependency(updateC)
queue.addOperation(doSomethingWithAandC)
For asynchronous request with completion handlers you can use a
(local) dispatch group inside each block operation to wait for the
completion.
Here is a self-contained example:
import Foundation
var a: String?
var b: String?
var c: String?
let queue = OperationQueue()
let updateA = BlockOperation {
let group = DispatchGroup()
group.enter()
DispatchQueue.global().asyncAfter(deadline: .now() + 1.0, execute: {
a = "A"
group.leave()
})
group.wait()
print("updateA done")
}
queue.addOperation(updateA)
let updateB = BlockOperation {
let group = DispatchGroup()
group.enter()
DispatchQueue.global().asyncAfter(deadline: .now() + 2.0, execute: {
b = "B"
group.leave()
})
group.wait()
print("updateB done")
}
queue.addOperation(updateB)
let updateC = BlockOperation {
let group = DispatchGroup()
group.enter()
DispatchQueue.global().asyncAfter(deadline: .now() + 3.0, execute: {
c = "C"
group.leave()
})
group.wait()
print("updateC done")
}
queue.addOperation(updateC)
let doSomethingWithAandB = BlockOperation {
print("a=", a!, "b=", b!)
}
doSomethingWithAandB.addDependency(updateA)
doSomethingWithAandB.addDependency(updateB)
queue.addOperation(doSomethingWithAandB)
let doSomethingWithAandC = BlockOperation {
print("a=", a!, "c=", c!)
}
doSomethingWithAandC.addDependency(updateA)
doSomethingWithAandC.addDependency(updateC)
queue.addOperation(doSomethingWithAandC)
let doSomethingWithBandC = BlockOperation {
print("b=", b!, "c=", c!)
}
doSomethingWithBandC.addDependency(updateB)
doSomethingWithBandC.addDependency(updateC)
queue.addOperation(doSomethingWithBandC)
queue.waitUntilAllOperationsAreFinished()
Output:
updateA done
updateB done
a= A b= B
updateC done
a= A c= C
b= B c= C

Related

Swift for loop not waiting for firestore call to complete

I know firestore calls are async which means this will not work:
private func removeUserSavedMomentFromAllUsers(moment: StoryMoment, completion: #escaping () -> Void) {
guard let savedByUIDs = moment.savedByUIDs else { return }
guard let momentID = moment.id else { return }
for id in savedByUIDs {
self.userInfoCollection.document(id).collection("savedMedias").document(momentID).delete { error in
if let error = error {
print("Error removing user saved moment from UID: \(error)")
}
}
}
}
Since the loop will continue before the delete call completes (same with get requests). I have used dispatch groups in the past to solve this issue. Heres a working example:
private func removeUserSavedMomentFromAllUsers(moment: StoryMoment, completion: #escaping () -> Void) {
guard let savedByUIDs = moment.savedByUIDs else { return }
guard let momentID = moment.id else { return }
let disSemaphore = DispatchSemaphore(value: 0)
let dispatchQueue = DispatchQueue(label: "group 1")
dispatchQueue.async {
for id in savedByUIDs {
self.userInfoCollection.document(id).collection("savedMedias").document(momentID).delete { error in
if let error = error {
print("Error removing user saved moment from UID: \(error)")
} else {
disSemaphore.signal()
}
}
disSemaphore.wait()
}
}
}
But those do all the work on the background thread.
My question is: How can I use async/await in a for loop where you call firebase docs?
The code in the first part of the question does work - and works fine for small group of data. However, in general it's recommended to not call Firebase functions in tight loops.
While the question mentions DispatchQueues, we use DispatchGroups with .enter and .leave as it's pretty clean.
Given a Firebase structure
sample_data
a
key: "value"
b
key: "value"
c
key: "value"
d
key: "value"
e
key: "value"
f
key: "value"
and suppose we want to delete the d, e, and f documents. Here's the code
func dispatchGroupDelete() {
let documentsToDelete = ["d", "e", "f"]
let collection = self.db.collection("sample_data") //self.db points to my Firestore
let group = DispatchGroup()
for docId in documentsToDelete {
group.enter()
let docToDelete = collection.document(docId)
docToDelete.delete()
group.leave()
}
}
While this answer doesn't use async/await, those may not be needed for this use case
If you want to use async/await you try do this
let documentsToDelete = ["d", "e", "f"]
let collection = self.db.collection("sample_data")
for docId in documentsToDelete {
let docToDelete = collection.document(docId)
Task {
do {
try await docToDelete.delete()
} catch {
print("oops")
}
}
}

Capturing values from synchronous and asynchronous closures

Assume a simple struct like this:
struct Foo {
let value: Int
let delay: TimeInterval
}
It has 2 functions, each taking a closure as a parameter. One is synchronously called, the other asynchronously after delay:
extension Foo {
func sync(_ completion: (Int) -> Void) {
completion(value)
}
}
extension Foo {
func async(_ completion: #escaping (Int) -> Void) {
DispatchQueue.main.asyncAfter(deadline: .now() + TimeInterval(delay)) {
completion(self.value)
}
}
}
Now assume I have an array of Foo objects:
let foo1 = Foo(value: 1, delay: 1)
let foo2 = Foo(value: 2, delay: 0)
Now I want to query each of them to get the values they would supply and put those into an array. Synchronously, it can be done like this:
let syncValues = [foo1, foo2].reduce(into: []) { values, foo in foo.sync { values.append($0) } } // [1, 2]
Asynchronously, I want to do this and have foo2 report before foo1 does:
let asyncValues = [foo1, foo2].reduce(into: []) { values, foo in foo.async { values.append($0) } // [2, 1]}
However, I get an error when I try to do the asynchronous version:
Escaping closure captures 'inout' parameter 'values'
What is a good way to perform this asynchronous task?
Because it's async, you can't do this in a single synchronous statement. Assuming you understand that, using Combine would probably be easiest:
import Combine
let fooResultsPublishers = [foo1, foo2].publisher
.flatMap { foo in
Future { promise in
foo.async { promise(.success($0)) }
}
}
.collect()
To actually get the value, you need to use .sink to act on it asynchronously:
fooResultsPublishers
.sink {
print($0) // [2,1]
}
// store it in long-living property, like
// var cancellables: Set<AnyCancellable> = []
.store(in: &cancellables)
Without Combine, you'd need to use something like a DispatchGroup:
let group = DispatchGroup()
var values: [Int] = []
[foo1, foo2].forEach { foo in
group.enter()
foo.async { values.append($0); group.leave() }
}
group.notify(queue: .main) {
print(values) // [2,1]
}

Swift Completion Handler For Loop to be performed once instead of 10 times due to the loop

I I have a loop with a firestore query in it that is repeated 10 times. I need to call the (completion: block) after all the 10 queries completed; Here I have my code so that it performs the (completion: block) per query but this would be too heavy on the server and the user's phone. How can I change below to accomplish what I just described?
static func getSearchedProducts(fetchingNumberToStart: Int, sortedProducts: [Int : [String : Int]], handler: #escaping (_ products: [Product], _ lastFetchedNumber: Int?) -> Void) {
var lastFetchedNumber:Int = 0
var searchedProducts:[Product] = []
let db = Firestore.firestore()
let block : FIRQuerySnapshotBlock = ({ (snap, error) in
guard error == nil, let snapshot = snap else {
debugPrint(error?.localizedDescription)
return
}
var products = snapshot.documents.map { Product(data: $0.data()) }
if !UserService.current.isGuest {
db.collection(DatabaseRef.Users).document(Auth.auth().currentUser?.uid ?? "").collection(DatabaseRef.Cart).getDocuments { (cartSnapshot, error) in
guard error == nil, let cartSnapshot = cartSnapshot else {
return
}
cartSnapshot.documents.forEach { document in
var product = Product(data: document.data())
guard let index = products.firstIndex(of: product) else { return }
let cartCount: Int = document.exists ? document.get(DatabaseRef.cartCount) as? Int ?? 0 : 0
product.cartCount = cartCount
products[index] = product
}
handler(products, lastFetchedNumber)
}
}
else {
handler(products, lastFetchedNumber)
}
})
if lastFetchedNumber == fetchingNumberToStart {
for _ in 0 ..< 10 {
//change the fetching number each time in the loop
lastFetchedNumber = lastFetchedNumber + 1
let productId = sortedProducts[lastFetchedNumber]?.keys.first ?? ""
if productId != "" {
db.collection(DatabaseRef.products).whereField(DatabaseRef.id, isEqualTo: productId).getDocuments(completion: block)
}
}
}
}
as you can see at the very end I am looping 10 times for this query because of for _ in 0 ..< 10 :
if productId != "" {
db.collection(DatabaseRef.products).whereField(DatabaseRef.id, isEqualTo: productId).getDocuments(completion: block)
}
So I need to make the completion: block handler to be called only once instead of 10 times here.
Use a DispatchGroup. You can enter the dispatch group each time you call the async code and then leave each time it's done. Then when everything is finished it will call the notify block and you can call your handler. Here's a quick example of what that would look like:
let dispatchGroup = DispatchGroup()
let array = []
for i in array {
dispatchGroup.enter()
somethingAsync() {
dispatchGroup.leave()
}
}
dispatchGroup.notify(queue: .main) {
handler()
}

Apple Combine framework: How to execute multiple Publishers in parallel and wait for all of them to finish?

I am discovering Combine. I wrote methods that make HTTP requests in a "combine" way, for example:
func testRawDataTaskPublisher(for url: URL) -> AnyPublisher<Data, Error> {
var request = URLRequest(url: url,
cachePolicy: .useProtocolCachePolicy,
timeoutInterval: 15)
request.httpMethod = "GET"
return urlSession.dataTaskPublisher(for: request)
.tryMap {
return $0.data
}
.eraseToAnyPublisher()
}
I would like to call the method multiple times and do a task after all, for example:
let myURLs: [URL] = ...
for url in myURLs {
let cancellable = testRawDataTaskPublisher(for: url)
.sink(receiveCompletion: { _ in }) { data in
// save the data...
}
}
The code above won't work because I have to store the cancellable in a variable that belongs to the class.
The first question is: is it a good idea to store many (for example 1000) cancellables in something like Set<AnyCancellable>??? Won't it cause memory leaks?
var cancellables = Set<AnyCancellable>()
...
let cancellable = ...
cancellables.insert(cancellable) // ???
And the second question is: how to start a task when all the cancellables are finished? I was thinking about something like that
class Test {
var cancellables = Set<AnyCancellable>()
func run() {
// show a loader
let cancellable = runDownloads()
.receive(on: RunLoop.main)
.sink(receiveCompletion: { _ in }) { _ in
// hide the loader
}
cancellables.insert(cancellable)
}
func runDownloads() -> AnyPublisher<Bool, Error> {
let myURLs: [URL] = ...
return Future<Bool, Error> { promise in
let numberOfURLs = myURLS.count
var numberOfFinishedTasks = 0
for url in myURLs {
let cancellable = testRawDataTaskPublisher(for: url)
.sink(receiveCompletion: { _ in }) { data in
// save the data...
numberOfFinishedTasks += 1
if numberOfFinishedTasks >= numberOfURLs {
promise(.success(true))
}
}
cancellables.insert(cancellable)
}
}.eraseToAnyPublisher()
}
func testRawDataTaskPublisher(for url: URL) -> AnyPublisher<Data, Error> {
...
}
}
Normally I would use DispatchGroup, start multiple HTTP tasks and consume the notification when the tasks are finished, but I am wondering how to write that in a modern way using Combine.
You can run some operations in parallel by creating a collection of publishers, applying the flatMap operator and then collect to wait for all of the publishers to complete before continuing. Here's an example that you can run in a playground:
import Combine
import Foundation
func delayedPublisher<Value>(_ value: Value, delay after: Double) -> AnyPublisher<Value, Never> {
let p = PassthroughSubject<Value, Never>()
DispatchQueue.main.asyncAfter(deadline: .now() + after) {
p.send(value)
p.send(completion: .finished)
}
return p.eraseToAnyPublisher()
}
let myPublishers = [1,2,3]
.map{ delayedPublisher($0, delay: 1 / Double($0)).print("\($0)").eraseToAnyPublisher() }
let cancel = myPublishers
.publisher
.flatMap { $0 }
.collect()
.sink { result in
print("result:", result)
}
Here is the output:
1: receive subscription: (PassthroughSubject)
1: request unlimited
2: receive subscription: (PassthroughSubject)
2: request unlimited
3: receive subscription: (PassthroughSubject)
3: request unlimited
3: receive value: (3)
3: receive finished
2: receive value: (2)
2: receive finished
1: receive value: (1)
1: receive finished
result: [3, 2, 1]
Notice that the publishers are all immediately started (in their original order).
The 1 / $0 delay causes the first publisher to take the longest to complete. Notice the order of the values at the end. Since the first took the longest to complete, it is the last item.

Grand Central Dispatch for complex flow?

I have a, b, c, d, e time consuming task functions with completion handler.
There are constraints between them:
Both b & c wait for a to finish
The last task e waits for b & c & d to finish
if there is no task d, I could write code in swift like this (not tested yet)
let group = DispatchGroup()
group.enter()
a() { group.leave() }
group.wait()
group.enter()
b() { group.leave() }
group.enter()
c() { group.leave() }
group.notify(queue: .main) {
e()
}
How to add task d without waiting a to complete?
Edited on 4/30 10:00 (+8)
Code Different said
the easiest approach is to make the download function synchronous and add a warning to its documentation that it should never be called from the main thread.
So I made a version based on it. This way cannot handle the return values from concurrent calls. But it looks really like async/await. So I'm satisfied now. Thank you guys.
the async/await like part is
myQueue.async {
downloadSync("A")
downloadSync("B", isConcurrent: true)
downloadSync("C", isConcurrent: true)
downloadSync("D", 4, isConcurrent: true)
waitConcurrentJobs()
downloadSync("E")
}
And the full code is below.
let myGroup = DispatchGroup()
let myQueue = DispatchQueue(label: "for Sync/Blocking version of async functions")
func waitConcurrentJobs() {
myGroup.wait()
}
// original function (async version, no source code)
func download(_ something: String, _ seconds: UInt32 = 1, completionHandler: #escaping ()->Void = {}) {
print("Downloading \(something)")
DispatchQueue.global().async {
sleep(seconds)
print("\(something) is downloaded")
completionHandler()
}
}
// wrapped function (synced version)
// Warning:
// It blocks current thead !!!
// Do not call it on main thread
func downloadSync(
_ something: String,
_ seconds: UInt32 = 1,
isConcurrent: Bool = false
){
myGroup.enter()
download(something, seconds) { myGroup.leave() }
if !isConcurrent {
myGroup.wait()
}
}
// Now it really looks like ES8 async/await
myQueue.async {
downloadSync("A")
downloadSync("B", isConcurrent: true)
downloadSync("C", isConcurrent: true)
downloadSync("D", 4, isConcurrent: true)
waitConcurrentJobs()
downloadSync("E")
}
results
Edit: the easiest approach is to make the download function synchronous and add a warning to its documentation that it should never be called from the main thread. The pyramid of doom for async function is the reason why coroutines were proposed, by no other than Chris Lattner, Swift's creator. As of April 2018, it's not yet a formal proposal waiting for review so chances are that you won't see it in Swift 5.
An synchronous download function:
// Never call this from main thread
func download(_ something: String, _ seconds: UInt32 = 1, completionHandler: #escaping ()->Void = {}) {
let group = DispatchGroup()
print("Downloading \(something)")
group.enter()
DispatchQueue.global().async {
sleep(seconds)
print("\(something) is downloaded")
completionHandler()
group.leave()
}
group.wait()
}
And NSOperation / NSOperationQueue setup:
let opA = BlockOperation() {
self.download("A")
}
let opB = BlockOperation() {
self.download("B")
}
let opC = BlockOperation() {
self.download("C")
}
let opD = BlockOperation() {
self.download("D", 4)
}
let opE = BlockOperation() {
self.download("E")
}
opB.addDependency(opA)
opC.addDependency(opA)
opE.addDependency(opB)
opE.addDependency(opC)
opE.addDependency(opD)
let operationQueue = OperationQueue()
operationQueue.addOperations([opA, opB, opC, opD, opE], waitUntilFinished: false)
Your original effort seems very close to me. You could make a minor adjustment: make B, C, and D be the group that finishes to trigger E.
A could be another group, but since it's one task, I don't see the point. Trigger B and C when it's done.
Note that unlike some of the example code in your question and other answers, in the code below, D and A can both start right away and run in parallel.
let q = DispatchQueue(label: "my-queue", attributes: .concurrent)
let g = DispatchGroup()
func taskA() { print("A") }
func taskB() { print("B"); g.leave() }
func taskC() { print("C"); g.leave() }
func taskD() { print("D"); g.leave() }
func taskE() { print("E") }
g.enter()
g.enter()
g.enter()
q.async {
taskA()
q.async(execute: taskB)
q.async(execute: taskC)
}
q.async(execute: taskD)
g.notify(queue: q, execute: taskE)
You can use this framework to implement async/await pattern - https://github.com/belozierov/SwiftCoroutine
When you call await it doesn’t block the thread but only suspends coroutine, so you can use it in the main thread as well.
func awaitAPICall(_ url: URL) throws -> String? {
let future = URLSession.shared.dataTaskFuture(for: url)
let data = try future.await().data
return String(data: data, encoding: .utf8)
}
func load(url: URL) {
DispatchQueue.main.startCoroutine {
let result1 = try self.awaitAPICall(url)
let result2 = try self.awaitAPICall2(result1)
let result3 = try self.awaitAPICall3(result2)
print(result3)
}
}
I would like to show an alternative solution using Scala like futures:
let result = funcA().flatMap { resultA in
return [funcB(param: resultA.0),
funcC(param: resultA.1),
funcD()]
.fold(initial: [String]()) { (combined, element) in
return combined + [element]
}
}.flatMap { result in
return funcE(param: result)
}.map { result in
print(result)
}
That's it basically. It handles errors (implicitly) and is thread-safe. No Operation subclasses ;)
Note, that funcD will be called only when A completes successfully. Since funcA() can fail it would make no sense to call it. But the code can be easily adapted to make this possible as well, iff required.
Please compare this to function foo() from my other solution which uses Dispatch Groups and Dispatch Queues.
Below an example of the definitions of the async functions which each passing their result to the next one:
func funcA() -> Future<(String, String)> {
print("Start A")
let promise = Promise<(String, String)>()
DispatchQueue.global().asyncAfter(deadline: .now() + 3) {
print("Complete A")
promise.complete(("A1", "A2"))
}
return promise.future
}
func funcB(param: String) -> Future<String> {
print("Start B")
let promise = Promise<String>()
DispatchQueue.global().asyncAfter(deadline: .now() + 1) {
print("Complete B")
promise.complete("\(param) -> B")
}
return promise.future
}
func funcC(param: String) -> Future<String> {
print("Start C")
let promise = Promise<String>()
DispatchQueue.global().asyncAfter(deadline: .now() + 2) {
print("Complete C")
promise.complete("\(param) -> C")
}
return promise.future
}
func funcD() -> Future<String> {
print("Start D")
let promise = Promise<String>()
DispatchQueue.global().asyncAfter(deadline: .now() + 4) {
print("Complete D")
promise.complete("D")
}
return promise.future
}
func funcE(param: [String]) -> Future<String> {
print("Start E")
let promise = Promise<String>()
DispatchQueue.global().asyncAfter(deadline: .now() + 4) {
print("Complete E")
promise.complete("\(param) -> E")
}
return promise.future
}
Which prints this to the console:
Start A
Complete A
Start B
Start C
Start D
Complete B
Complete C
Complete D
Start E
Complete E
["A1 -> B", "A2 -> C", "D"] -> E
Hint: there are a couple of Future and Promise libraries available.