I have zipped two publishers in the function, which downloads users and vehicles with a backend API:
func fetchUserAndvehicles() {
Publishers.Zip(UserApiClient().getUser(), VehicleApiClient().getVehicles())
.eraseToAnyPublisher()
.receive(on: DispatchQueue.main)
.sink(receiveCompletion: { [weak self] completion in
switch completion {
case .failure(let error):
self?.errorHandling.showErrorAlert(error)
case .finished:
break
}
}, receiveValue: { [weak self] user, vehicles in
// store vehicles in the user object
})
.store(in: &subscriptions)
}
Each of the vehicles have an imageUrl that can be used to download an image of the vehicle. This works fine. But I would like to download the images, if any, before I store the vehicles in the user object. Is it possible to use the same combine pipeline to do this? I tried with a flatMap, but that resulted in a compile error.
The following is following the excellent answer from Cristik. It looks ok, but Xcode flags the flatMap line with No exact matches in call to instance method 'flatMap':
let vehiclesPublisher = VehicleApiClient().getVehicles()
.flatMap { vehicles in
Publishers.Zip(Just(vehicles).setFailureType(to: Error.self), Publishers.MergeMany(vehicles.map { VehicleApiClient().getImage(at: $0.url)}).collect())
}
.map {
return $0.0
}
The vehicles have an optional property that needs to be unwrapped, but that isn't the cause of the compile error.
What you need can is to
Convert the vehicles publisher to an array of image downloader publishers
Create a single publisher from a list of publishers
Wait for that publisher
Assuming you have a downloadImage(from url: URL) function somewhere, and that your Vehicle type has an imageURL property, what you have to do in order to accomplish what you need, is to
let vehiclesPublisher = VehicleApiClient().getVehicles()
.flatMap { vehicles in
Publishers.Zip(Just(vehicles).setFailureType(to: Error.self), // setFailureType dance needed
Publishers.MergeMany(vehicles.map { downloadImage(from: $0.imageURL) }).collect())
}.map {
return $0.0 // unpack the vehicles, so we have back a publisher that outputs [Vehicle]
}
Publishers.Zip(UserApiClient().getUser(), vehiclesPublisher)
// rest of the pipeline stays the same
What happens in the above code is we transform the array of vehicles to an array of image downloaders, which is waited for with the collect() operator. But we also need to retain the vehicles array, so a zip is needed, followed by an unpack of the first item returned by zip, as we don't care about the image download status, we only care for all of them to finish.
Related
This question already has answers here:
Combine framework serialize async operations
(8 answers)
Closed 1 year ago.
I am trying to chain n requests with Combine.
Let's assume I have 50 users and for each of them I need to do a single request to get a users data. I know that with flatMap you can pass one Publisher result into the next. But does that work with loops as well?
That's my function to fetch a user:
func fetchUser(for id: Int) -> AnyPublisher<User, Error> {
let url = "https://user.com/api/user/\(id)"
return URLSession.shared.dataTaskPublisher(for: url)
.mapError { $0 as Error }
.map { $0.data }
.decode(type: User.self, decoder: JSONDecoder())
.eraseToAnyPublisher()
}
So basically I need another function, which loops this over this fetchUser and returns all users in one result array. The requests should not all run at the same time, but rather start one after the previous one has finished.
For this one, a lot depends on how you want to use the User objects. If you want them to all emit as individual users as they come in, then merge is the solution. If you want to keep the array order and emit them all as an array of users once they all come in, then combineLatest is what you need.
Since you are dealing with an array, and neither merge nor combineLatest have array versions, you will need to use reduce. Here's examples:
func combine(ids: [Int]) -> AnyPublisher<[User], Error> {
ids.reduce(Optional<AnyPublisher<[User], Error>>.none) { state, id in
guard let state = state else { return fetchUser(for: id).map { [$0] }.eraseToAnyPublisher() }
return state.combineLatest(fetchUser(for: id))
.map { $0.0 + [$0.1] }
.eraseToAnyPublisher()
}
?? Just([]).setFailureType(to: Error.self).eraseToAnyPublisher()
}
func merge(ids: [Int]) -> AnyPublisher<User, Error> {
ids.reduce(Optional<AnyPublisher<User, Error>>.none) { state, id in
guard let state = state else { return fetchUser(for: id).eraseToAnyPublisher() }
return state.merge(with: fetchUser(for: id))
.eraseToAnyPublisher()
}
?? Empty().eraseToAnyPublisher()
}
Notice that in the combine case, if the array is empty, the Publisher will emit an empty array then complete. In the merge case, it will just complete without emitting anything.
Also notice that in either case if any of the Publishers fail, then the entire chain will shut down. If you don't want that, you will have to catch the errors and do something with them...
I wonder if it is possible to access values in a combine-operator-chain that are used further up in the chain.
For example, if you have an array of strings, and you download a resource with each string, is it possible to access the string itself?
Maybe a pseudo-code example will help understand better:
let strings = ["a", "b", "c"]
strings.publisher
.compactMap{ str in
URL(string: "https://someresource/\(str)")
}
.flatMap { url in
URLSession.shared.dataTaskPublisher(for: url)
}
.map { $0.data}
.map { data in
// here I would need the "str" string from above
}
Help is much appreciated
Thx, Gregor
The answer that Jake wrote and deleted is correct. I don't know why he deleted it; maybe he felt he couldn't support it with example code. But the answer is exactly right. If you need the initial str value later in the pipeline, it is up to you to keep passing it down through every step. You typically do that by passing a tuple of values at each stage, so that the string makes it far enough down the chain to be retrieved. This is a very common strategy in Combine programming.
For a simple example, take a look at the Combine code in the central section of this article:
https://www.biteinteractive.com/swift-5-5-asynchronous-looping-with-async-await/
As I say in the article:
You’ll observe that, as opposed to GCD where local variables just magically “fall through” to nested completion handlers at a lower level of scope, every step in a Combine pipeline must explicitly pass down all information that may be needed by a later step. This can result in some rather ungainly values working their way down the pipeline, often in the form of a tuple, as I’ve illustrated here.
But I don’t regard that as a problem. On the contrary, being explicit about what’s passing down the pipeline seems to me to be a gain in clarity.
To illustrate further, here's a rewrite of your pseudo-code; this is real code, you can run it and try it out:
class ViewController: UIViewController {
var storage = Set<AnyCancellable>()
override func viewDidLoad() {
super.viewDidLoad()
let strings = ["a", "b", "c"]
let pipeline = strings.publisher
.map { str -> (String, URL) in
let url = URL(string: "https://www.apeth.com/pep/manny.jpg")!
return (str, url)
}
.flatMap { (str, url) -> AnyPublisher<(String, (data: Data, response: URLResponse)), URLError> in
let sess = URLSession.shared.dataTaskPublisher(for: url)
.eraseToAnyPublisher()
let just = Just(str).setFailureType(to: URLError.self)
.eraseToAnyPublisher()
let zip = just.zip(sess).eraseToAnyPublisher()
return zip
}
.map { (str, result) -> (String, Data) in
(str, result.data)
}
.sink { comp in print(comp) } receiveValue: { (str, data) in print (str, data) }
pipeline.store(in: &storage)
}
}
That's not the only way to express the pipeline, but it does work, and it should give you a starting point.
In Combine framework there's a concept of a demand, which allows signalling backpressure to publishers.
Suppose I have a simple publisher:
let numbers = Publishers.Sequence<ClosedRange<Int>, Error>(sequence: 0...100)
I would like to download certain URLs that use these numbers as arguments. I also would like a next download to start only after a previous download has finished.
A naive approach then would look like this:
let subscription = numbers.sink(receiveCompletion: { _ in }, receiveValue: {
let url = URL(string: "https://httpbin.org/get?value=\($0)")!
URLSession.shared.dataTask(with: url) {
$0.map { print(String(data: $0, encoding: .utf8)!) }
}.resume()
})
Unfortunately, this wouldn't satisfy the requirement of waiting for a previous download to complete before starting the next one. As far as I know, sink function would return a value of type AnyCancellable, not of type Subscription. If the latter was the case, we could call the request function on the subscription with a specific demand after an upload completes.
What would be the best way to control demand of a subscription provided by sink or any other standard Combine Subscriber?
Turns out, flatMap operator takes an additional maxPublishers argument that takes a Subscribers.Demand value. In combination with the Future publisher, this allows the numbers publisher to wait until the future is able to process a given value before sending a next one.
Applying this to the original code, downloading values one after another would look like this:
enum DownloadError: Error {
case noData
}
let subscription = numbers.flatMap(maxPublishers: .max(1)) { number in
Future { promise in
let url = URL(string: "https://httpbin.org/get?value=\(number)")!
URLSession.shared.dataTask(with: url) {
switch ($0, $2) {
case let (data?, nil):
promise(.success(data))
case let (nil, error?):
promise(.failure(error))
default:
promise(.failure(DownloadError.noData))
}
}.resume()
}
}.sink(
receiveCompletion: { _ in print("errors should be handled here") },
receiveValue: { print(String(data: $0, encoding: .utf8)!) }
)
I have a requirement to request an Article.
Each Article contains an array of ArticleAsset which has various props on I need when rendering an entire article.
I do not know ahead of time how many assets exist on an article, so I must request the article and then using the assets prop dispatch X amount of request to return the value of each ArticleAsset.
At that point I should then return both the article and the array of results for my asset fetches.
For simplicity imagine in this case each asset returns an Int. So I start with this -
Article > [Article]
I would expect to end up with an tuple of the following shape (article: Article, assets: [Int])
I have attempted to recreate this as the below playground, but have been completely unsuccessful and am a little stuck.
I understand how to chain a fixed number of requests, using flatMapLatest etc but in this case I do not know the number of requests. I'm thinking I should map each ArticleAsset and return an array of Observables however I start to get very fuzzy on where to go next.
Any help would be much appreciated please and thank you.
import UIKit
import RxSwift
private let disposeBag = DisposeBag()
struct Article {
let id: UUID = UUID()
var assets: [ArticleAsset]
}
struct ArticleAsset {
let number: Int
}
let assets: [ArticleAsset] = Array(0...4).map { ArticleAsset(number: $0) }
let article = Article(assets: assets)
func fetchArticle() -> Observable<Article> {
return Observable.of(article)
}
func getArticleAsset(asset: ArticleAsset) -> Observable<Int> {
return .of(asset.number)
}
fetchArticle()
.map { art in
let assets = art.assets.map { getArticleAsset(asset: $0) }
let resp = (article: art, assets: Observable.of(assets))
return resp
}.subscribe(onNext: { resp in
// I would like my subscriber to receive (article: Article, assets: [Int])
}).disposed(by: disposeBag)
Kudos in making a compilable playground! That makes things a lot easier. What you want to do here is combine observables. In my article you will see that there are a lot of ways to do that. I think for this use-case the zip operator is best.
let articleWithAssets = fetchArticle()
.flatMap { (article) -> Observable<(article: Article, assets: [Int])> in
let articles = Observable.zip(article.assets.map { getArticleAsset(asset: $0) })
return Observable.zip(Observable.just(article), articles) { (article: $0, assets: $1) }
}
articleWithAssets
.subscribe(onNext: { resp in
// here `resp` is of type `(article: Article, assets: [Int])` as requested.
})
When fetchArticle() emits a value, the flatMaps closure will be called and it will call getArticleAsset(asset:) for each asset, wait until they are all done, combine them into a single Observable array (the articles object) and then combine that with the .just(article) observable.
Warning though, if any one asset request fails, the entire chain fails. If you don't want that, you will have to take care of that inside the { getArticleAsset(asset: $0) } block. (Maybe emit a nil or missingAsset asset instead.)
Rx has several flattening operators. flatMapLatest is not what you want here because its only going to give you the results from the last inner observable. You really want is to combine all of the ArticleAsset streams and proceed only when they are all done. So you want to merge all of your ArticleAsset requests then reduce them into an array of ArticleAsset.
fetchArticle()
.flatMap { article in
let allAssetRequests = article.assets.map {
getArticleAsset(asset: article)
}
return Observable
.merge(allAssetRequests)
.reduce([ArticleAsset]()) { array, asset in
//Combine here
}
}
You can change the reduce here to reduce into a tuple: (Article, [ArticleAsset]) and then you will have the final form of the stream that you are looking for.
I need to call a sequences of function to get all the information I need for a notification. First subscribe which opens up the session, then queryNotification to listen on all the incoming notifications, and once a notification is received, need to call getNotificationAttrs with the notificationId returned in queryNotification, then call getAppAttributes with appIdentifier returned in getNotificationAttrs and I need the combined result of queryNotification, getNotificationAttrs and getAppAttributes. How the functions look like are below:
func subscribeNotification() -> Single<Info>
func queryNotification() -> Observable<Notification>
func getNotificationAttrs(uid: UInt32, attributes: [Attribute]) -> Single<NotificationAttributes>
func getAppAttributes(appIdentifier: String, attributes: [AppAttribute]) -> Single<NotificationAppAttributes>
The tricky part is that queryNotification returns Observable and both getNotificationAttrs and getAppAttributes return Single. What I have in mind of chaining them together is like:
subscribeNotification()
.subscribe(onSuccess: { info in
queryNotification()
.flatMap({ notification in
return getNotificationAttributes(uid: notification.uid, attributes: [.appIdentifier, .content])
})
.flatMap({ notifAttrs
return getAppAttributes(appIdentifier: notifAttrs.appIdentifier, attributes: [.displayName])
})
.subscribe {
// have all the result from last two calls
}
})
Is this doable? Any direction is appreciated! Thanks!
The most obvious and IMHO correct solution is to promote your Single into an Observable. Also, I'm not a fan of the first subscribe where it is. You end up with an indentation pyramid.
I'm following your comments about needing the values from all of queryNotification(), getNotificationAttrs(did:attributes:) and getAppAttributes(appIdentifier:attributes:)...
let query = subscribeNotification()
.asObservable()
.flatMap { _ in queryNotification() }
.share(replay: 1)
let attributes = query
.flatMap { getNotificationAttrs(uid: $0.uid, attributes: [.appIdentifier, .content]) }
.share(replay: 1)
let appAttributes = attributes
.flatMap { getAppAttributes(appIdentifier: $0.appIdentifier, attributes: [.displayName]) }
Observable.zip(query, attributes, appAttributes)
.subscribe(onNext: { (query, attributes, appAttributes) in
})
The above will follow the steps you outlined and the subscribe will get called every time a new notification is emitted.
Also notice how the above reads quite a bit like synchronous code would (just with some extra wrapping.)