I wonder if it is possible to access values in a combine-operator-chain that are used further up in the chain.
For example, if you have an array of strings, and you download a resource with each string, is it possible to access the string itself?
Maybe a pseudo-code example will help understand better:
let strings = ["a", "b", "c"]
strings.publisher
.compactMap{ str in
URL(string: "https://someresource/\(str)")
}
.flatMap { url in
URLSession.shared.dataTaskPublisher(for: url)
}
.map { $0.data}
.map { data in
// here I would need the "str" string from above
}
Help is much appreciated
Thx, Gregor
The answer that Jake wrote and deleted is correct. I don't know why he deleted it; maybe he felt he couldn't support it with example code. But the answer is exactly right. If you need the initial str value later in the pipeline, it is up to you to keep passing it down through every step. You typically do that by passing a tuple of values at each stage, so that the string makes it far enough down the chain to be retrieved. This is a very common strategy in Combine programming.
For a simple example, take a look at the Combine code in the central section of this article:
https://www.biteinteractive.com/swift-5-5-asynchronous-looping-with-async-await/
As I say in the article:
You’ll observe that, as opposed to GCD where local variables just magically “fall through” to nested completion handlers at a lower level of scope, every step in a Combine pipeline must explicitly pass down all information that may be needed by a later step. This can result in some rather ungainly values working their way down the pipeline, often in the form of a tuple, as I’ve illustrated here.
But I don’t regard that as a problem. On the contrary, being explicit about what’s passing down the pipeline seems to me to be a gain in clarity.
To illustrate further, here's a rewrite of your pseudo-code; this is real code, you can run it and try it out:
class ViewController: UIViewController {
var storage = Set<AnyCancellable>()
override func viewDidLoad() {
super.viewDidLoad()
let strings = ["a", "b", "c"]
let pipeline = strings.publisher
.map { str -> (String, URL) in
let url = URL(string: "https://www.apeth.com/pep/manny.jpg")!
return (str, url)
}
.flatMap { (str, url) -> AnyPublisher<(String, (data: Data, response: URLResponse)), URLError> in
let sess = URLSession.shared.dataTaskPublisher(for: url)
.eraseToAnyPublisher()
let just = Just(str).setFailureType(to: URLError.self)
.eraseToAnyPublisher()
let zip = just.zip(sess).eraseToAnyPublisher()
return zip
}
.map { (str, result) -> (String, Data) in
(str, result.data)
}
.sink { comp in print(comp) } receiveValue: { (str, data) in print (str, data) }
pipeline.store(in: &storage)
}
}
That's not the only way to express the pipeline, but it does work, and it should give you a starting point.
Related
This question already has answers here:
Combine framework serialize async operations
(8 answers)
Closed 1 year ago.
I am trying to chain n requests with Combine.
Let's assume I have 50 users and for each of them I need to do a single request to get a users data. I know that with flatMap you can pass one Publisher result into the next. But does that work with loops as well?
That's my function to fetch a user:
func fetchUser(for id: Int) -> AnyPublisher<User, Error> {
let url = "https://user.com/api/user/\(id)"
return URLSession.shared.dataTaskPublisher(for: url)
.mapError { $0 as Error }
.map { $0.data }
.decode(type: User.self, decoder: JSONDecoder())
.eraseToAnyPublisher()
}
So basically I need another function, which loops this over this fetchUser and returns all users in one result array. The requests should not all run at the same time, but rather start one after the previous one has finished.
For this one, a lot depends on how you want to use the User objects. If you want them to all emit as individual users as they come in, then merge is the solution. If you want to keep the array order and emit them all as an array of users once they all come in, then combineLatest is what you need.
Since you are dealing with an array, and neither merge nor combineLatest have array versions, you will need to use reduce. Here's examples:
func combine(ids: [Int]) -> AnyPublisher<[User], Error> {
ids.reduce(Optional<AnyPublisher<[User], Error>>.none) { state, id in
guard let state = state else { return fetchUser(for: id).map { [$0] }.eraseToAnyPublisher() }
return state.combineLatest(fetchUser(for: id))
.map { $0.0 + [$0.1] }
.eraseToAnyPublisher()
}
?? Just([]).setFailureType(to: Error.self).eraseToAnyPublisher()
}
func merge(ids: [Int]) -> AnyPublisher<User, Error> {
ids.reduce(Optional<AnyPublisher<User, Error>>.none) { state, id in
guard let state = state else { return fetchUser(for: id).eraseToAnyPublisher() }
return state.merge(with: fetchUser(for: id))
.eraseToAnyPublisher()
}
?? Empty().eraseToAnyPublisher()
}
Notice that in the combine case, if the array is empty, the Publisher will emit an empty array then complete. In the merge case, it will just complete without emitting anything.
Also notice that in either case if any of the Publishers fail, then the entire chain will shut down. If you don't want that, you will have to catch the errors and do something with them...
I'm relatively new to the Functional Reactive programming world, and still trying to wrap my head around the concepts. I'm utilizing an SDK to make some network requests - specifically to query a remote database. The SDK returns a publisher, and I have a working pipeline that transforms that result into model objects. Here's that working pipeline:
let existingClaimQuery = "SELECT Id, Subject, CaseNumber FROM Case WHERE Status != 'Closed' ORDER BY CaseNumber DESC"
let requestForOpenCases = RestClient.shared.request(forQuery: existingClaimQuery, apiVersion: RestClient.apiVersion)
caseCancellable = RestClient.shared
.publisher(for: requestForOpenCases)
.receive(on: RunLoop.main)
.tryMap({restresponse -> [String:Any] in
let json = try restresponse.asJson() as? [String:Any]
return json ?? RestClient.JSONKeyValuePairs()
})
.map({json -> [[String:Any]] in
let records = json["records"] as? [[String:Any]]
return records ?? [[:]]
})
.map({
$0.map{(item) -> Claim in
return Claim(
id: item["Id"] as? String ?? "None Listed",
subject: item["Subject"] as? String ?? "None Listed",
caseNumber: item["CaseNumber"] as? String ?? "0"
)
}
})
.mapError{error -> Error in
print(error)
return error
}
.catch{ error in
return Just([])
}
.assign(to: \.claims, on: self)
I went to work on another section of the code, and realized I often need to do this same process - write a query, create a request for that query, and process it through a pipeline that ultimately returns a [[String:Any]].
So here's the million dollar question. What's the right way to encapsulate this pipeline such that I can re-use it without having to copy/pasta the entire pipeline all over the code base? This is my ... attempt at it, but it feels ...wrong?
class QueryStream: ObservableObject {
var query: String = ""
private var queryCancellable: AnyCancellable?
#Published var records: [[String:Any]] = [[String:Any]]()
func execute(){
let queryRequest = RestClient.shared.request(forQuery: query, apiVersion: RestClient.apiVersion)
queryCancellable = RestClient.shared.publisher(for: queryRequest)
.receive(on: RunLoop.main)
.tryMap({restresponse -> [String:Any] in
let json = try restresponse.asJson() as? [String:Any]
return json ?? [String:Any]()
})
.map({json -> [[String:Any]] in
let records = json["records"] as? [[String:Any]]
return records ?? [[:]]
})
.mapError{error -> Error in
print(error)
return error
}
.catch{ error in
return Just([])
}
.assign(to: \.records, on: self)
}
}
This still requires a pipeline to be written for each use. I feel like there should be some way to have a one off promise like pipeline that would allow for
let SomeRecords = QueryStream("Query here").execute()
Am I too n00b? overthinking it? What's the stack's wisdom?
Entire pipelines are not reusable. Publishers are reusable. When I say "publisher" I mean an initial publisher plus operators attached to it. (Remember, an operator is itself a publisher.) A publisher can exist as a property of something, so you can subscribe to it, or it can be generated for a particular case (like a particular query request) by a function.
To illustrate, here's a one-off pipeline:
let s = "https://photojournal.jpl.nasa.gov/tiff/PIA23172.tif"
let url = URL(string:s)!
let eph = URLSessionConfiguration.ephemeral
let session = URLSession(configuration: eph)
session.dataTaskPublisher(for: url)
.map {$0.data}
.replaceError(with: Data())
.compactMap { UIImage(data:$0) }
.receive(on: DispatchQueue.main)
.assign(to: \.image, on: self.iv)
.store(in:&self.storage)
That pipeline tries to download the data from a URL, tests to see whether it's image data, and if it is, turns the image data into an image and displays it in an image view in the interface.
Let's say I want to do this for various different remote images. Obviously it would be ridiculous to repeat the whole pipeline everywhere. What differs up front is the URL, so let's encapsulate the first part of the pipeline as a publisher that can be generated on demand based on the URL:
func image(fromURL url:URL) -> AnyPublisher<UIImage,Never> {
let eph = URLSessionConfiguration.ephemeral
let session = URLSession(configuration: eph)
return session.dataTaskPublisher(for: url)
.map {$0.data}
.replaceError(with: Data())
.compactMap { UIImage(data:$0) }
.receive(on: DispatchQueue.main)
.eraseToAnyPublisher()
}
Now the only thing that needs to be repeated in various places in our code is the subscriber to that publisher:
let s = "https://photojournal.jpl.nasa.gov/tiff/PIA23172.tif"
let url = URL(string:s)!
image(fromURL:url)
.map{Optional($0)}
.assign(to: \.image, on: self.iv)
.store(in:&self.storage)
You see? Elsewhere, we might have a different URL, and we might do something different with the UIImage that comes popping out of the call to image(fromURL:), and that's just fine; the bulk of the pipeline has been encapsulated and doesn't need to be repeated.
Your example pipeline's publisher is susceptible of that same sort of encapsulation and reuse.
Let me first note that I think you are dispatching to main to early in your pipeline. As far as I can tell, all of your map transforms are pure functions (no side effects or references to mutable state), so they can just as well run on the background thread and thus not block the UI.
Second, as Matt said, a Publisher is generally reusable. Your pipeline builds up a big complex Publisher, and then subscribes to it, which produces an AnyCancellable. So factor out the big complex Publisher but not the subscribing.
You can factor it out into an extension method on your RestClient for convenience:
extension RestClient {
func records<Record>(
forQuery query: String,
makeRecord: #escaping ([String: Any]) throws -> Record)
-> AnyPublisher<[Record], Never>
{
let request = self.request(forQuery: query, apiVersion: RestClient.apiVersion)
return self.publisher(for: request)
.tryMap { try $0.asJson() as? [String: Any] ?? [:] }
.map { $0["records"] as? [[String: Any]] ?? [] }
.tryMap { try $0.map { try makeRecord($0) } }
.mapError { dump($0) } // dump is a Swift standard function
.replaceError(with: []) // simpler than .catch
.eraseToAnyPublisher()
}
}
Then you can use it like this:
struct Claim {
var id: String
var subject: String
var caseNumber: String
}
extension Claim {
static func from(json: [String: Any]) -> Claim {
return .init(
id: json["Id"] as? String ?? "None Listed",
subject: json["Subject"] as? String ?? "None Listed",
caseNumber: json["CaseNumber"] as? String ?? "0")
}
}
class MyController {
var claims: [Claim] = []
var caseCancellable: AnyCancellable?
func run() {
let existingClaimQuery = "SELECT Id, Subject, CaseNumber FROM Case WHERE Status != 'Closed' ORDER BY CaseNumber DESC"
caseCancellable = RestClient.shared.records(forQuery: existingClaimQuery, makeRecord: Claim.from(json:))
.receive(on: RunLoop.main)
.assign(to: \.claims, on: self)
}
}
Note that I've put the receive(on: RunLoop.main) operator in the method that subscribes to the publisher, rather than building it in to the publisher. This makes it easy to add additional operators that run on a background scheduler before dispatching to the main thread.
UPDATE
From your comment:
In promise syntax, i could say execute run() as defined above, and .then(doSomethingWithThatData()) knowing that the doSomethingWithThatData wouldn't run until the intial work had completed successfully. I'm trying to develop a setup where I need to use this records(fromQuery:) method runs, and then (and only then) do soemthing with that data. I'm struggling with how to bolt that on to the end.
I don't know what promise implementation you're using, so it's difficult to know what your .then(doSomethingWithThatData()) does. What you've written doesn't really make much sense in Swift. Perhaps you meant:
.then { data in doSomething(with: data) }
In which case, the doSomething(with:) method cannot possibly be called until the data is available, because doSomething(with:) takes the data as an argument!
In Combine framework there's a concept of a demand, which allows signalling backpressure to publishers.
Suppose I have a simple publisher:
let numbers = Publishers.Sequence<ClosedRange<Int>, Error>(sequence: 0...100)
I would like to download certain URLs that use these numbers as arguments. I also would like a next download to start only after a previous download has finished.
A naive approach then would look like this:
let subscription = numbers.sink(receiveCompletion: { _ in }, receiveValue: {
let url = URL(string: "https://httpbin.org/get?value=\($0)")!
URLSession.shared.dataTask(with: url) {
$0.map { print(String(data: $0, encoding: .utf8)!) }
}.resume()
})
Unfortunately, this wouldn't satisfy the requirement of waiting for a previous download to complete before starting the next one. As far as I know, sink function would return a value of type AnyCancellable, not of type Subscription. If the latter was the case, we could call the request function on the subscription with a specific demand after an upload completes.
What would be the best way to control demand of a subscription provided by sink or any other standard Combine Subscriber?
Turns out, flatMap operator takes an additional maxPublishers argument that takes a Subscribers.Demand value. In combination with the Future publisher, this allows the numbers publisher to wait until the future is able to process a given value before sending a next one.
Applying this to the original code, downloading values one after another would look like this:
enum DownloadError: Error {
case noData
}
let subscription = numbers.flatMap(maxPublishers: .max(1)) { number in
Future { promise in
let url = URL(string: "https://httpbin.org/get?value=\(number)")!
URLSession.shared.dataTask(with: url) {
switch ($0, $2) {
case let (data?, nil):
promise(.success(data))
case let (nil, error?):
promise(.failure(error))
default:
promise(.failure(DownloadError.noData))
}
}.resume()
}
}.sink(
receiveCompletion: { _ in print("errors should be handled here") },
receiveValue: { print(String(data: $0, encoding: .utf8)!) }
)
I have a requirement to request an Article.
Each Article contains an array of ArticleAsset which has various props on I need when rendering an entire article.
I do not know ahead of time how many assets exist on an article, so I must request the article and then using the assets prop dispatch X amount of request to return the value of each ArticleAsset.
At that point I should then return both the article and the array of results for my asset fetches.
For simplicity imagine in this case each asset returns an Int. So I start with this -
Article > [Article]
I would expect to end up with an tuple of the following shape (article: Article, assets: [Int])
I have attempted to recreate this as the below playground, but have been completely unsuccessful and am a little stuck.
I understand how to chain a fixed number of requests, using flatMapLatest etc but in this case I do not know the number of requests. I'm thinking I should map each ArticleAsset and return an array of Observables however I start to get very fuzzy on where to go next.
Any help would be much appreciated please and thank you.
import UIKit
import RxSwift
private let disposeBag = DisposeBag()
struct Article {
let id: UUID = UUID()
var assets: [ArticleAsset]
}
struct ArticleAsset {
let number: Int
}
let assets: [ArticleAsset] = Array(0...4).map { ArticleAsset(number: $0) }
let article = Article(assets: assets)
func fetchArticle() -> Observable<Article> {
return Observable.of(article)
}
func getArticleAsset(asset: ArticleAsset) -> Observable<Int> {
return .of(asset.number)
}
fetchArticle()
.map { art in
let assets = art.assets.map { getArticleAsset(asset: $0) }
let resp = (article: art, assets: Observable.of(assets))
return resp
}.subscribe(onNext: { resp in
// I would like my subscriber to receive (article: Article, assets: [Int])
}).disposed(by: disposeBag)
Kudos in making a compilable playground! That makes things a lot easier. What you want to do here is combine observables. In my article you will see that there are a lot of ways to do that. I think for this use-case the zip operator is best.
let articleWithAssets = fetchArticle()
.flatMap { (article) -> Observable<(article: Article, assets: [Int])> in
let articles = Observable.zip(article.assets.map { getArticleAsset(asset: $0) })
return Observable.zip(Observable.just(article), articles) { (article: $0, assets: $1) }
}
articleWithAssets
.subscribe(onNext: { resp in
// here `resp` is of type `(article: Article, assets: [Int])` as requested.
})
When fetchArticle() emits a value, the flatMaps closure will be called and it will call getArticleAsset(asset:) for each asset, wait until they are all done, combine them into a single Observable array (the articles object) and then combine that with the .just(article) observable.
Warning though, if any one asset request fails, the entire chain fails. If you don't want that, you will have to take care of that inside the { getArticleAsset(asset: $0) } block. (Maybe emit a nil or missingAsset asset instead.)
Rx has several flattening operators. flatMapLatest is not what you want here because its only going to give you the results from the last inner observable. You really want is to combine all of the ArticleAsset streams and proceed only when they are all done. So you want to merge all of your ArticleAsset requests then reduce them into an array of ArticleAsset.
fetchArticle()
.flatMap { article in
let allAssetRequests = article.assets.map {
getArticleAsset(asset: article)
}
return Observable
.merge(allAssetRequests)
.reduce([ArticleAsset]()) { array, asset in
//Combine here
}
}
You can change the reduce here to reduce into a tuple: (Article, [ArticleAsset]) and then you will have the final form of the stream that you are looking for.
I need to make multiple calls.
1. Delete Document Upload
2. Image 1 & server returns URL
3. Upload Image 2 & server returns URL
4. Create Document API contains both URLs & extra
parameters.
The code which I tried to write is in RxSwift,& MVVM.
let resultOfDocumentUpdateWithDelete =
donepressed
.filter{ $0 }
.withLatestFrom(self.existingDocumentIDChangedProperty)
.flatMapLatest {id in
let deleted_document = apiClient.deleteDocument(id).asObservable().materialize()
let upload_frontImage = deleted_document
.withLatestFrom(self.frontImageNameChangedProperty)
.flatMapLatest {image in
apiClient.uploadImage(image: image!).asObservable().materialize()
}
let upload_backImage = upload_frontImage
.withLatestFrom(self.backImageChangedProperty)
.flatMapLatest {image in
apiClient.uploadImage(image: image!).asObservable().materialize()
}
let upload_document = upload_backImage
.withLatestFrom(self.parametersChangedProperty)
.flatMapLatest {parameters in
apiClient.uploadDocument(parameters: parameters)
}
return upload_document.materialize()
}
.share(replay: 1)
Make sure, two responses of server are input in last API, so all of these will be called in a sequence.
how to do in RxSwift.
This was an interesting one! The take-away here is that when you are in doubt, go ahead and make your own operator. If it turns out that you later figure out how to do the job using the built-in operators, then you can replace yours. The only thing with making your own is that they require a lot more testing.
Note, to use the below, you will have to combineLatest of your observables and then flatMap and pass their values into this function.
// all possible results from this job.
enum ProcessResult {
case success
case deleteFailure(Error)
case imageFailue(Error)
case backImageFailure(Error)
case documentFailure(Error)
}
func uploadContent(apiClient: APIClient, documentID: Int, frontImage: UIImage, backImage: UIImage, parameters: Parameters) -> Single<ProcessResult> {
// instead of trying to deal with all the materializes, I decided to turn it into a single process.
return Single.create { observer in
// each api call happens in turn. Note that there are no roll-back semantics included! You are dealing with a very poorly written server.
let deleted = apiClient.deleteDocument(id: documentID)
.asObservable()
.share()
let imagesUploaded = deleted
.flatMap { _ in Observable.zip(apiClient.uploadImage(image: frontImage).asObservable(), apiClient.uploadImage(image: backImage).asObservable()) }
.share()
let documentUploaded = imagesUploaded
.flatMap { arg -> Single<Void> in
let (frontURL, backURL) = arg
var updatedParams = parameters
// add frontURL and backURL to parameters
return apiClient.uploadDocument(parameters: updatedParams)
}
.share()
let disposable = deleted
.subscribe(onError: { observer(.success(ProcessResult.deleteFailure($0))) })
let disposable1 = imagesUploaded
.subscribe(onError: { observer(.success(ProcessResult.imageFailue($0))) })
let disposable2 = documentUploaded
.subscribe(
onNext: { observer(.success(ProcessResult.success)) },
onError: { observer(.success(ProcessResult.documentFailure($0))) }
)
return Disposables.create([disposable, disposable1, disposable2])
}
}