I may be going about this the wrong way, but I have a function with which I want to emit multiple values over time. But I don’t want it to start emitting until something is subscribed to that object. I’m coming to combine from RxSwift, so I’m basically trying to duplicated Observable.create() in the RxSwift world. The closest I have found is returning a Future, but futures only succeed or fail (so they are basically like a Single in RxSwift.)
Is there some fundamental thing I am missing here? My end goal is to make a function that processes a video file and emits progress events until it completes, then emits a URL for the completed file.
Generally you can use a PassthroughSubject to publish custom outputs. You can wrap a PassthroughSubject (or multiple PassthroughSubjects) in your own implementation of Publisher to ensure that only your process can send events through the subject.
Let's mock a VideoFrame type and some input frames for example purposes:
typealias VideoFrame = String
let inputFrames: [VideoFrame] = ["a", "b", "c"]
Now we want to write a function that synchronously processes these frames. Our function should report progress somehow, and at the end, it should return the output frames. To report progress, our function will take a PassthroughSubject<Double, Never>, and send its progress (as a fraction from 0 to 1) to the subject:
func process(_ inputFrames: [VideoFrame], progress: PassthroughSubject<Double, Never>) -> [VideoFrame] {
var outputFrames: [VideoFrame] = []
for input in inputFrames {
progress.send(Double(outputFrames.count) / Double(inputFrames.count))
outputFrames.append("output for \(input)")
}
return outputFrames
}
Okay, so now we want to turn this into a publisher. The publisher needs to output both progress and a final result. So we'll use this enum as its output:
public enum ProgressEvent<Value> {
case progress(Double)
case done(Value)
}
Now we can define our Publisher type. Let's call it SyncPublisher, because when it receives a Subscriber, it immediately (synchronously) performs its entire computation.
public struct SyncPublisher<Value>: Publisher {
public init(_ run: #escaping (PassthroughSubject<Double, Never>) throws -> Value) {
self.run = run
}
public var run: (PassthroughSubject<Double, Never>) throws -> Value
public typealias Output = ProgressEvent<Value>
public typealias Failure = Error
public func receive<Downstream: Subscriber>(subscriber: Downstream) where Downstream.Input == Output, Downstream.Failure == Failure {
let progressSubject = PassthroughSubject<Double, Never>()
let doneSubject = PassthroughSubject<ProgressEvent<Value>, Error>()
progressSubject
.setFailureType(to: Error.self)
.map { ProgressEvent<Value>.progress($0) }
.append(doneSubject)
.subscribe(subscriber)
do {
let value = try run(progressSubject)
progressSubject.send(completion: .finished)
doneSubject.send(.done(value))
doneSubject.send(completion: .finished)
} catch {
progressSubject.send(completion: .finished)
doneSubject.send(completion: .failure(error))
}
}
}
Now we can turn our process(_:progress:) function into a SyncPublisher like this:
let inputFrames: [VideoFrame] = ["a", "b", "c"]
let pub = SyncPublisher<[VideoFrame]> { process(inputFrames, progress: $0) }
The run closure is { process(inputFrames, progress: $0) }. Remember that $0 here is a PassthroughSubject<Double, Never>, exactly what process(_:progress:) wants as its second argument.
When we subscribe to this pub, it will first create two subjects. One subject is the progress subject and gets passed to the closure. We'll use the other subject to publish either the final result and a .finished completion, or just a .failure completion if the run closure throws an error.
The reason we use two separate subjects is because it ensures that our publisher is well-behaved. If the run closure returns normally, the publisher publishes zero or more progress reports, followed by a single result, followed by .finished. If the run closure throws an error, the publisher publishes zero or more progress reports, followed by a .failed. There is no way for the run closure to make the publisher emit multiple results, or emit more progress reports after emitting the result.
At last, we can subscribe to pub to see if it works properly:
pub
.sink(
receiveCompletion: { print("completion: \($0)") },
receiveValue: { print("output: \($0)") })
Here's the output:
output: progress(0.0)
output: progress(0.3333333333333333)
output: progress(0.6666666666666666)
output: done(["output for a", "output for b", "output for c"])
completion: finished
The following extension to AnyPublisher replicates Observable.create semantics by composing a PassthroughSubject. This includes cancellation semantics.
AnyPublisher.create() Swift 5.6 Extension
public extension AnyPublisher {
static func create<Output, Failure>(_ subscribe: #escaping (AnySubscriber<Output, Failure>) -> AnyCancellable) -> AnyPublisher<Output, Failure> {
let passthroughSubject = PassthroughSubject<Output, Failure>()
var cancellable: AnyCancellable?
return passthroughSubject
.handleEvents(receiveSubscription: { subscription in
let subscriber = AnySubscriber<Output, Failure> { subscription in
} receiveValue: { input in
passthroughSubject.send(input)
return .unlimited
} receiveCompletion: { completion in
passthroughSubject.send(completion: completion)
}
cancellable = subscribe(subscriber)
}, receiveCompletion: { completion in
}, receiveCancel: {
cancellable?.cancel()
})
.eraseToAnyPublisher()
}
}
Usage
func doSomething() -> AnyPublisher<Int, Error> {
return AnyPublisher<Int, Error>.create { subscriber in
// Imperative implementation of doing something can call subscriber as follows
_ = subscriber.receive(1)
subscriber.receive(completion: .finished)
// subscriber.receive(completion: .failure(myError))
return AnyCancellable {
// Imperative cancellation implementation
}
}
}
Related
I am converting some code into Combine in order to get familiar with it. I am doing fine with the easy stuff, but here it gets a little trickier. I am trying to report to the user when incoming GPS data is accurate and also whether it's stale.
So I have
let locationPublisher = PassthroughSubject<CLLocation,Never>()
private var cancellableSet: Set<AnyCancellable> = []
var status:GPSStatus = .red //enum
and in init I have
locationPublisher
.map(gpsStatus(from:)) //maps CLLocation to GPSStatus enum
.assign(to: \.gpsStatus, on: self)
.store(in: &cancellableSet)
locationPublisher.sink() { [weak self] location in
self?.statusTimer?.invalidate()
self?.setStatusTimer()
}
.store(in: &cancellableSet)
setStatusTimer()
Here is the setStatusTimer function
func setStatusTimer () {
statusTimer = Timer.scheduledTimer(withTimeInterval: 20, repeats: false) {#MainActor _ in
self.updateGPSStatus(.red)
}
}
Is there a more "Combine" way of doing this? I know there are Timer.TimerPublishers, but I'm not sure how to incorporate them?
My tendency is to think there is some kind of combineLatest with one input being the gps status publisher and the other one being some kind go publisher that fires if the upstream pub hasn't fired for x seconds.
Thanks!
This is a bit tricky. You don't need a TimerPublisher but you can use the timeout operator. The tricky part is that timeout will create a publisher that stops publishing when it times out. The question is "how do you start again". To do that, you can use the catch operator.
The solution looks like this:
import UIKit
import Combine
enum GPSStatus {
// Karma Chameleon...
case red
case gold
case green
}
func gpsStatus(from location: String) -> GPSStatus {
switch location {
case "ok":
return .green
default:
return .gold
}
}
class UnnamedLocationThingy {
let locationPublisher = PassthroughSubject<String,Never>()
var gpsStatus:GPSStatus = .red {
didSet { print("set the new value to \(String(describing: gpsStatus))") }
}
private enum LocationError: Error {
case timeout
}
private var statusProvider: AnyCancellable?
init() {
statusProvider = makeStatusProvider()
.assign(to: \.gpsStatus, on: self)
}
func makeStatusProvider() -> AnyPublisher<GPSStatus, Never> {
return locationPublisher
.map(gpsStatus(from:)) //maps CLLocation to GPSStatus enum
.setFailureType(to: LocationError.self)
.timeout(.seconds(2), scheduler: DispatchQueue.main) {
return LocationError.timeout
}
.catch { _ in
self.gpsStatus = .red;
return self.makeStatusProvider()
}.eraseToAnyPublisher()
}
}
let thingy = UnnamedLocationThingy();
Task {
for delay in [1, 1, 1, 3, 1, 1, 3, 1] {
try await Task.sleep(for: .seconds(delay))
thingy.locationPublisher.send("ok")
}
}
The heart of it is the makeStatusProvider function. This function creates a publisher that will convert published locations to GPSStatus values as long as they come in in a time limit. But if one doesn't come fast enough, it times out. I set up the timeout operator with a customError: handler so that it doesn't just terminate the publisher, but sends an error. I can catch that error in the catch operator and substitute a new publisher to replace the old one. The new publisher I substitute is a brand new publisher created by makeStatusProvider which, as we've just seen is a publisher that converts published locations into GPSStatus values until it hits a timeout...
It's a form of recursion.
I've decorated your code with enough stuff to make a Playground and added a bit of code at the end to exercise the functionality.
I have a situation where my code needs to make one network call to fetch a bunch of items, but while waiting for those to come down, another network call might fetch an update to those items. I'd love to be able to enqueue those secondary results until the first one has finished. Is there a way to accomplish that with Combine?
Importantly, I am not able to wait before making the second request. It’s actually a connection to a websocket that gets made at the same time as the first request, and the updates come over the websocket outside of my control.
Update
After examining Matt’s thorough book on Combine, I settled on .prepend(). But as Matt warned me in the comments, .prepend() doesn’t even subscribe to the other publisher until after the first one completes. This means I miss any signals sent prior to that. What I need is a Subject that enqueues values, but perhaps that’s not so hard to make. Anyway, this is where I got:
Initially I was going to use .append(), but I realized with .prepend() I could avoid keeping a reference to one of the publishers. So here’s a simplified version of what I’ve got. There might be syntax errors in this, as I’ve whittled it down from my (employer’s) code.
There’s the ItemFeed, which handles fetching a list of items and simultaneously handling item update events. The latter can arrive before the initial list of items, and thus must be sequenced via Combine to arrive after it. I attempt to do this by prepending the initial items source to the update PassthroughSubject.
Below that is an XCTestCase that simulates a lengthy initial item load, and adds an update before that load can complete. It attempts to subscribe to changes to the list of items, and tries to test that the first update is the initial 63 items, and the subsequent update is for 64 items (in this case, “update” results in adding an item).
Unfortunately, while the initial list is published, the update never arrives. I also tried removing the .output(at:) operators, but the two sinks are only called once.
After the test case sets up the delayed “fetch,” and subscribes to changes in feed.items, it calls feed.handleItemUpatedEvent. This calls ItemFeed.updateItems.send(_:), but unfortunately that is lost to oblivion.
class
ItemFeed
{
typealias InitialItemsSource = Deferred<Future<[[String : Any]], Error>>
let updateItems = PassthroughSubject<[Item], Error>()
var funnel : AnyCancellable?
#Published var items = [Item]()
init(initialItemSource inSource: InitialItemsSource)
{
// Passthrough subject each time items are updated…
var pub = self.updateItems.eraseToAnyPublisher()
// Prepend the initial items we need to fetch…
let initialItems = source.tryMap { try $0.map { try Item(object: $0) } }
pub = pub.prepend(initialItems).eraseToAnyPublisher()
// Sink on the funnel to add or update to self.items…
self.funnel =
pub.sink { inCompletion in
// Handle errors
}
receiveValue: {
self.update(items: inItems)
}
}
func handleItemUpdatedEvent(_ inItem: Item) {
self.updateItems.send([inItem])
}
func update(items inItems: [Item]) {
// Update or add inItems to self.items
}
}
class
ItemFeedTests : XCTestCase
{
func
testShouldUpdateItems()
throws
{
// Set up a mock source of items…
let source = fetchItems(named: "items", delay: 3.0) // 63 items
let expectation = XCTestExpectation(description: "testShouldUpdateItems")
expectation.expectedFulfillmentCount = 2
let feed = ItemFeed(initialItemSource: source)
let sub1 = feed.$items
.output(at: 0)
.receive(on: DispatchQueue.main)
.sink { inItems in
expectation.fulfill()
debugLog("Got first items: \(inItems.count)")
XCTAssertEqual(inItems.count, 63)
}
let sub2 = feed.$items
.output(at: 1)
.receive(on: DispatchQueue.main)
.sink { inItems in
expectation.fulfill()
debugLog("Got second items: \(inItems.count)")
XCTAssertEqual(inItems.count, 64)
}
// Send an update right away…
let item = try loadItem(named: "Item3")
feed.handleItemUpdatedEvent(item)
XCTAssertEqual(feed.items.count, 0) // Should be no items yet
// Wait for stuff to complete…
wait(for: [expectation], timeout: 10.0)
sub1.cancel() // Not necessary, but silence the compiler warning
sub2.cancel()
}
}
The accepted answer didn't actually have any code, but I was able to figure out a solution. I ended up creating a custom publisher that subscribed to the "gate publisher" and created a subscription that creates a sink for the upstream publisher. I buffer the values from upstream and emit gate publisher values based on demand until it completes, then I switch to sending the buffer downstream based on demand. The tricky part is keeping track of the upstream / gate publisher and sending demand to the right one.
After a fair bit of trial and error, I found a solution. I created a custom Publisher and Subscription that immediately subscribes to its upstream publisher and begins enqueuing elements (up to some specifiable capacity). It then waits for a subscriber to come along, and provides that subscriber with all the values up until now, and then continues providing values. Here’s a marble diagram:
I then use this in conjunction with .prepend() like so:
extension
Publisher
{
func
enqueue<P>(gatedBy inGate: P, capacity inCapacity: Int = .max)
-> AnyPublisher<Self.Output, Self.Failure>
where
P : Publisher,
P.Output == Output,
P.Failure == Failure
{
let qp = Publishers.Queueing(upstream: self, capacity: inCapacity)
let r = qp.prepend(inGate).eraseToAnyPublisher()
return r
}
}
And this is how you use it…
func
testShouldReturnAllItemsInOrder()
{
let gate = PassthroughSubject<Int, Never>()
let stream = PassthroughSubject<Int, Never>()
var results = [Int]()
let sub = stream.enqueue(gatedBy: gate)
.sink
{ inElement in
debugLog("element: \(inElement)")
results.append(inElement)
}
stream.send(3)
stream.send(4)
stream.send(5)
XCTAssertEqual(results.count, 0)
gate.send(1)
gate.send(2)
gate.send(completion: .finished)
XCTAssertEqual(results.count, 5)
XCTAssertEqual(results, [1,2,3,4,5])
sub.cancel()
}
This prints what you would expect:
element: 1
element: 2
element: 3
element: 4
element: 5
It works well because creating the .enqueue(gatedBy:) operator creates the queuing publisher qp, which immediately subscribes to stream and enqueues any values it sends. It then calls .prepend() on qp, which first subscribes to gate, and waits for it to complete. When it finishes, it then subscribes to qp, which immediately provides it with all the enqueued values, and then continues to provide it with values from the upstream publisher.
Here’s the code I finally ended up with.
//
// QueuingPublisher.swift
// Latency: Zero, LLC
//
// Created by Rick Mann on 2021-06-03.
//
import Combine
import Foundation
extension
Publishers
{
final
class
Queueing<Upstream: Publisher>: Publisher
{
typealias Output = Upstream.Output
typealias Failure = Upstream.Failure
private let upstream : Upstream
private let capacity : Int
private var queue : [Output] = [Output]()
private var subscription : QueueingSubscription<Queueing<Upstream>, Upstream>?
fileprivate var completion : Subscribers.Completion<Failure>? = nil
init(upstream inUpstream: Upstream, capacity inCapacity: Int)
{
self.upstream = inUpstream
self.capacity = inCapacity
// Subscribe to the upstream right away so we can start
// enqueueing values…
let sink = AnySubscriber { $0.request(.unlimited) }
receiveValue:
{ [weak self] (inValue: Output) -> Subscribers.Demand in
self?.relay(inValue)
return .none
}
receiveCompletion:
{ [weak self] (inCompletion: Subscribers.Completion<Failure>) in
self?.completion = inCompletion
self?.subscription?.complete(with: inCompletion)
}
inUpstream.subscribe(sink)
}
func
receive<S: Subscriber>(subscriber inSubscriber: S)
where
Failure == S.Failure,
Output == S.Input
{
let subscription = QueueingSubscription(publisher: self, subscriber: inSubscriber)
self.subscription = subscription
inSubscriber.receive(subscription: subscription)
}
/**
Return up to inDemand values.
*/
func
request(_ inDemand: Subscribers.Demand)
-> [Output]
{
let count = inDemand.min(self.queue.count)
let elements = Array(self.queue[..<count])
self.queue.removeFirst(count)
return elements
}
private
func
relay(_ inValue: Output)
{
// TODO: The Wenderlich example code checks to see if the upstream has completed,
// but I feel like want to send all the values we've gotten first?
// Save the new value…
self.queue.append(inValue)
// Discard the oldest if we’re over capacity…
if self.queue.count > self.capacity
{
self.queue.removeFirst()
}
// Send the buffer to our subscriber…
self.subscription?.dataAvailable()
}
final
class
QueueingSubscription<QP, Upstream> : Subscription
where
QP : Queueing<Upstream>
{
typealias Output = Upstream.Output
typealias Failure = Upstream.Failure
let publisher : QP
var subscriber : AnySubscriber<Output,Failure>? = nil
private var demand : Subscribers.Demand = .none
init<S>(publisher inP: QP,
subscriber inS: S)
where
S: Subscriber,
Failure == S.Failure,
Output == S.Input
{
self.publisher = inP
self.subscriber = AnySubscriber(inS)
}
func
request(_ inDemand: Subscribers.Demand)
{
self.demand += inDemand
emitAsNeeded()
}
func
cancel()
{
complete(with: .finished)
}
/**
Called by our publisher to let us know new
data has arrived.
*/
func
dataAvailable()
{
emitAsNeeded()
}
private
func
emitAsNeeded()
{
guard let subscriber = self.subscriber else { return }
let newValues = self.publisher.request(self.demand)
self.demand -= newValues.count
newValues.forEach
{
let nextDemand = subscriber.receive($0)
self.demand += nextDemand
}
if let completion = self.publisher.completion
{
complete(with: completion)
}
}
fileprivate
func
complete(with inCompletion: Subscribers.Completion<Failure>)
{
guard let subscriber = self.subscriber else { return }
self.subscriber = nil
subscriber.receive(completion: inCompletion)
}
}
}
} // extension Publishers
extension
Publisher
{
func
enqueue<P>(gatedBy inGate: P, capacity inCapacity: Int = .max)
-> AnyPublisher<Self.Output, Self.Failure>
where
P : Publisher,
P.Output == Output,
P.Failure == Failure
{
let qp = Publishers.Queueing(upstream: self, capacity: inCapacity)
let r = qp.prepend(inGate).eraseToAnyPublisher()
return r
}
}
extension
Subscribers.Demand
{
func
min(_ inValue: Int)
-> Int
{
if self == .unlimited
{
return inValue
}
return Swift.min(self.max!, inValue)
}
}
In this code I am expecting the Empty() publisher to send completion to the .sink subscriber, but no completion is sent.
func testEmpty () {
let x = XCTestExpectation()
let subject = PassthroughSubject<Int, Never>()
emptyOrSubjectPublisher(subject).sink(receiveCompletion: { completion in
dump(completion)
}, receiveValue: { value in
dump(value)
}).store(in: &cancellables)
subject.send(0)
wait(for: [x], timeout: 10.0)
}
func emptyOrSubjectPublisher (_ subject: PassthroughSubject<Int, Never>) -> AnyPublisher<Int, Never> {
subject
.flatMap { (i: Int) -> AnyPublisher<Int, Never> in
if i == 1 {
return subject.eraseToAnyPublisher()
} else {
return Empty().eraseToAnyPublisher()
}
}
.eraseToAnyPublisher()
}
Why does the emptyOrSubjectPublisher not receive the completion?
The Empty completes, but the overall pipeline does not, because the initial Subject has not completed. The inner pipeline in which the Empty is produced (the flatMap) has "swallowed" the completion. This is the expected behavior.
You can see this more easily by simply producing a Just in the flatMap, e.g. Just(100):
subject
.flatMap {_ in Just(100) }
.sink(receiveCompletion: { completion in
print(completion)
}, receiveValue: { value in
print(value)
}).store(in: &cancellables)
subject.send(1)
You know and I know that a Just emits once and completes. But although the value of the Just arrives down the pipeline, there is no completion.
And you can readily see why it works this way. It would be very wrong if we had a potential sequence of values from our publisher but some intermediate publisher, produced in a flatMap, had the power to complete the whole pipeline and end it prematurely.
(And see my https://www.apeth.com/UnderstandingCombine/operators/operatorsTransformersBlockers/operatorsflatmap.html where I make the same point.)
If the goal is to send a completion down the pipeline, it's the subject that needs to complete. For example, you could say
func emptyOrSubjectPublisher (_ subject: PassthroughSubject<Int, Never>) -> AnyPublisher<Int, Never> {
subject
.flatMap { (i: Int) -> AnyPublisher<Int, Never> in
if i == 1 {
return subject.eraseToAnyPublisher()
} else {
subject.send(completion: .finished) // <--
return Empty().eraseToAnyPublisher()
}
}
.eraseToAnyPublisher()
}
[Note, however, that your whole emptyOrSubjectPublisher is peculiar; it is unclear what purpose it is intended to serve. Returning subject when i is 1 is kind of pointless too, because subject has already published the 1 by the time we get here, and isn't going to publish anything more right now. Thus, if you send 1 at the start, you won't receive 1 as a value, because your flatMap has swallowed it and has produced a publisher that isn't going to publish.]
I have this publisher and subscribers (example code):
import Combine
let publisher = PassthroughSubject<ComplexStructOrClass, Never>()
let sub1 = publisher.sink { (someString) in
// Async work...
}
let sub2 = publisher.sink { (someString) in
// Async work, but it has to wait until sub1 has finished his work
}
So the publisher constant has 2 subscribers. When I use the method send on the publisher constant, it should send the value first to sub1 and after sub1 finished processing (with a callback or something like that), publisher should and notify sub2.
So in the comments its stated that Combine is made for this. What publisher do I need to use? A PassthroughSubject may be the wrong decision.
Usecase
I need to publish values throughout the lifetime of my app to a dynamic number of subscribers, for a few different publishers (I hope I can make a protocol). So a subscriber can be added and removed from a publisher at any given time. A subscriber look as follows:
It has a NSPersistentContainer
A callback should be made by the publisher when a new value has arrived. That process looks like:
the publisher will create a backgroundContext of the container of the subscriber, because it knows a subscriber has a container
the publisher sends the context along with the new published value to the subscriber
the publisher waits until it receives a callback of the subscriber. The subscriber shouldn't save the context, but the publisher must hold a reference to the context. The subscriber gives a callback of an enum, which has a ok case and some error cases.
When a subscriber gives a callback with an error enum case, the publisher must rollback the contexts it created for each subscriber.
When a subscriber gives a callback with the ok case, the publisher repeats step 1 till 5 for every subscriber
This step will only be reached when no subscriber gave a error enum case or there are no subscribers. The publisher will save all the contexts created by the subscribers.
Current code, no Combine
This is some code without using Combine:
// My publisher
protocol NotiPublisher {
// Type of message to send
associatedtype Notification
// List of subscribers for this publisher
static var listeners: Set<AnyNotiPublisher<Notification>> { get set }
}
// My subscriber
protocol NotificationListener: Hashable {
associatedtype NotificationType
var container: NSPersistentContainer { get }
// Identifier used to find this subscriber in the list of 'listeners' in the publisher
var identifier: Int32 { get }
var notify: ((_ notification: NotificationType, _ context: NSManagedObjectContext, #escaping CompletionHandlerAck) -> ()) { get }
}
// Type erased version of the NotificationListener and some convience methods here, can add them if desired
// In a extension of NotiPublisher, this method is here
static func notify(queue: DispatchQueue, notification: Notification, completionHander: #escaping CompletionHandlerAck) throws {
let dispatchGroup = DispatchGroup()
var completionBlocks = [SomeCompletionHandler]()
var contexts = [NSManagedObjectContext]()
var didLoop = false
for listener in listeners {
if didLoop {
dispatchGroup.wait()
} else {
didLoop = true
}
dispatchGroup.enter()
listener.container.performBackgroundTask { (context) in
contexts.append(context)
listener.notify(notification, context, { (completion) in
completionBlocks.append(completion)
dispatchGroup.leave()
})
}
}
dispatchGroup.notify(queue: queue) {
let err = completion.first(where: { element in
// Check if an error has occured
})
if err == nil {
for context in contexts {
context.performAndWait {
try! context.save()
}
}
}
completionHander(err ?? .ok(true))
}
}
This is pretty complex code, I am wondering if I can make use of the power of Combine to make this code more readable.
I wrote the following to chain async operations from a publisher using flatMap that allows you to return another publisher. I'm not a fan, and it might not meet your need to dynamically change the subs, but it might help someone:
let somePublisher = Just(12)
let anyCancellable = somePublisher.flatMap{ num in
//We can return a publisher from flatMap, so lets return a Future one because we want to do some async work
return Future<Int,Never>({ promise in
//do an async thing using dispatch
DispatchQueue.main.asyncAfter(deadline: .now() + 3, execute: {
print("flat map finished processing the number \(num)")
//now just pass on the value
promise(.success(num))
})
})
}.flatMap{ num in
//We can return a publisher from flatMap, so lets return a Future one because we want to do some async work
return Future<Int,Never>({ promise in
//do an async thing using dispatch
DispatchQueue.main.asyncAfter(deadline: .now() + 3, execute: {
print("second flat map finished processing the number \(num)")
//now just pass on the value
promise(.success(num))
})
})
}.sink { num in
print("This sink runs after the async work in the flatMap/Future publishers")
}
You can try to use serial OperationQueue to receive values. Seems like it will wait until sink completes before call another.
let queue = OperationQueue()
queue.maxConcurrentOperationCount = 1
publisher
.receive(on: queue)
.sink { _ in }
Using Apple's new Combine framework I want to make multiple requests from each element in a list. Then I want a single result from a reduction of all the the responses. Basically I want to go from list of publishers to a single publisher that holds a list of responses.
I've tried making a list of publishers, but I don't know how to reduce that list into a single publisher. And I've tried making a publisher containing a list but I can't flat map a list of publishers.
Please look at the "createIngredients" function
func createIngredient(ingredient: Ingredient) -> AnyPublisher<CreateIngredientMutation.Data, Error> {
return apollo.performPub(mutation: CreateIngredientMutation(name: ingredient.name, optionalProduct: ingredient.productId, quantity: ingredient.quantity, unit: ingredient.unit))
.eraseToAnyPublisher()
}
func createIngredients(ingredients: [Ingredient]) -> AnyPublisher<[CreateIngredientMutation.Data], Error> {
// first attempt
let results = ingredients
.map(createIngredient)
// results = [AnyPublisher<CreateIngredientMutation.Data, Error>]
// second attempt
return Publishers.Just(ingredients)
.eraseToAnyPublisher()
.flatMap { (list: [Ingredient]) -> Publisher<[CreateIngredientMutation.Data], Error> in
return list.map(createIngredient) // [AnyPublisher<CreateIngredientMutation.Data, Error>]
}
}
I'm not sure how to take an array of publishers and convert that to a publisher containing an array.
Result value of type '[AnyPublisher]' does not conform to closure result type 'Publisher'
Essentially, in your specific situation you're looking at something like this:
func createIngredients(ingredients: [Ingredient]) -> AnyPublisher<[CreateIngredientMutation.Data], Error> {
Publishers.MergeMany(ingredients.map(createIngredient(ingredient:)))
.collect()
.eraseToAnyPublisher()
}
This 'collects' all the elements produced by the upstream publishers and – once they have all completed – produces an array with all the results and finally completes itself.
Bear in mind, if one of the upstream publishers fails – or produces more than one result – the number of elements may not match the number of subscribers, so you may need additional operators to mitigate this depending on your situation.
The more generic answer, with a way you can test it using the EntwineTest framework:
import XCTest
import Combine
import EntwineTest
final class MyTests: XCTestCase {
func testCreateArrayFromArrayOfPublishers() {
typealias SimplePublisher = Just<Int>
// we'll create our 'list of publishers' here. Each publisher emits a single
// Int and then completes successfully – using the `Just` publisher.
let publishers: [SimplePublisher] = [
SimplePublisher(1),
SimplePublisher(2),
SimplePublisher(3),
]
// we'll turn our array of publishers into a single merged publisher
let publisherOfPublishers = Publishers.MergeMany(publishers)
// Then we `collect` all the individual publisher elements results into
// a single array
let finalPublisher = publisherOfPublishers.collect()
// Let's test what we expect to happen, will happen.
// We'll create a scheduler to run our test on
let testScheduler = TestScheduler()
// Then we'll start a test. Our test will subscribe to our publisher
// at a virtual time of 200, and cancel the subscription at 900
let testableSubscriber = testScheduler.start { finalPublisher }
// we're expecting that, immediately upon subscription, our results will
// arrive. This is because we're using `just` type publishers which
// dispatch their contents as soon as they're subscribed to
XCTAssertEqual(testableSubscriber.recordedOutput, [
(200, .subscription), // we're expecting to subscribe at 200
(200, .input([1, 2, 3])), // then receive an array of results immediately
(200, .completion(.finished)), // the `collect` operator finishes immediately after completion
])
}
}
I think that Publishers.MergeMany could be of help here. In your example, you might use it like so:
func createIngredients(ingredients: [Ingredient]) -> AnyPublisher<CreateIngredientMutation.Data, Error> {
let publishers = ingredients.map(createIngredient(ingredient:))
return Publishers.MergeMany(publishers).eraseToAnyPublisher()
}
That will give you a publisher that sends you single values of the Output.
However, if you specifically want the Output in an array all at once at the end of all your publishers completing, you can use collect() with MergeMany:
func createIngredients(ingredients: [Ingredient]) -> AnyPublisher<[CreateIngredientMutation.Data], Error> {
let publishers = ingredients.map(createIngredient(ingredient:))
return Publishers.MergeMany(publishers).collect().eraseToAnyPublisher()
}
And either of the above examples you could simplify into a single line if you prefer, ie:
func createIngredients(ingredients: [Ingredient]) -> AnyPublisher<CreateIngredientMutation.Data, Error> {
Publishers.MergeMany(ingredients.map(createIngredient(ingredient:))).eraseToAnyPublisher()
}
You could also define your own custom merge() extension method on Sequence and use that to simplify the code slightly:
extension Sequence where Element: Publisher {
func merge() -> Publishers.MergeMany<Element> {
Publishers.MergeMany(self)
}
}
func createIngredients(ingredients: [Ingredient]) -> AnyPublisher<CreateIngredientMutation.Data, Error> {
ingredients.map(createIngredient).merge().eraseToAnyPublisher()
}
To add on the answer by Tricky, here is a solution which retains the order of elements in the array.
It passes an index for each element through the whole chain, and sorts the collected array by the index.
Complexity should be O(n log n) because of the sorting.
import Combine
extension Publishers {
private struct EnumeratedElement<T> {
let index: Int
let element: T
init(index: Int, element: T) {
self.index = index
self.element = element
}
init(_ enumeratedSequence: EnumeratedSequence<[T]>.Iterator.Element) {
index = enumeratedSequence.offset
element = enumeratedSequence.element
}
}
static func mergeMappedRetainingOrder<InputType, OutputType>(
_ inputArray: [InputType],
mapTransform: (InputType) -> AnyPublisher<OutputType, Error>
) -> AnyPublisher<[OutputType], Error> {
let enumeratedInputArray = inputArray.enumerated().map(EnumeratedElement.init)
let enumeratedMapTransform: (EnumeratedElement<InputType>) -> AnyPublisher<EnumeratedElement<OutputType>, Error> = { enumeratedInput in
mapTransform(enumeratedInput.element)
.map { EnumeratedElement(index: enumeratedInput.index, element: $0)}
.eraseToAnyPublisher()
}
let sortEnumeratedOutputArrayByIndex: ([EnumeratedElement<OutputType>]) -> [EnumeratedElement<OutputType>] = { enumeratedOutputArray in
enumeratedOutputArray.sorted { $0.index < $1.index }
}
let transformToNonEnumeratedArray: ([EnumeratedElement<OutputType>]) -> [OutputType] = {
$0.map { $0.element }
}
return Publishers.MergeMany(enumeratedInputArray.map(enumeratedMapTransform))
.collect()
.map(sortEnumeratedOutputArrayByIndex)
.map(transformToNonEnumeratedArray)
.eraseToAnyPublisher()
}
}
Unit test for the solution:
import XCTest
import Combine
final class PublishersExtensionsTests: XCTestCase {
// MARK: - Private properties
private var cancellables = Set<AnyCancellable>()
// MARK: - Tests
func test_mergeMappedRetainingOrder() {
let expectation = expectation(description: "mergeMappedRetainingOrder publisher")
let numbers = (1...100).map { _ in Int.random(in: 1...3) }
let mapTransform: (Int) -> AnyPublisher<Int, Error> = {
let delayTimeInterval = RunLoop.SchedulerTimeType.Stride(Double($0))
return Just($0)
.delay(for: delayTimeInterval, scheduler: RunLoop.main)
.setFailureType(to: Error.self)
.eraseToAnyPublisher()
}
let resultNumbersPublisher = Publishers.mergeMappedRetainingOrder(numbers, mapTransform: mapTransform)
resultNumbersPublisher.sink(receiveCompletion: { _ in }, receiveValue: { resultNumbers in
XCTAssertTrue(numbers == resultNumbers)
expectation.fulfill()
}).store(in: &cancellables)
waitForExpectations(timeout: 5)
}
}
You can do it in one line:
.flatMap(Publishers.Sequence.init(sequence:))