Why .collect() operator in swift Combine always sends .unlimited demand regardless of the demand of upstream publisher? - swift

Question

I have been playing with Combine to understand how it works in more details and I create a custom Publisher, Subscription and Subscriber.
Here's how it looks..
The emoji beamer publisher along with subscription:
struct EmojiBeamerPublisher: Publisher {
typealias Output = String
typealias Failure = Error
private let emojis: [String] = ["👍","❤️","✅","🥰","😍","🚀","😅","🍑","🍞","🎅","❄️","🐻","👀","👄","🦷","✍️","🙏","👨‍💻","🐝","🐛","🦉","🦀","🐍","🐞","🧸"]
func receive<S>(subscriber: S) where S : Subscriber, Self.Failure == S.Failure, Self.Output == S.Input {
let subscription = EmojiBeamerSubscription(output: emojis, subscriber: subscriber)
subscriber.receive(subscription: subscription)
}
}
extension EmojiBeamerPublisher {
private final class EmojiBeamerSubscription<S: Subscriber>: Subscription where S.Input == Output, S.Failure == Failure {
var subscriber: S?
let output: [String]
init(output: [String], subscriber: S) {
self.subscriber = subscriber
self.output = output
}
func request(_ demand: Subscribers.Demand) {
Swift.print("Demand: \(demand)") // Here I receive Unlimited demand
var demand = demand
Timer.scheduledTimer(withTimeInterval: 5, repeats: true) { [weak self] timer in
guard let self = self else { return }
guard demand > 0, let subscriber = self.subscriber else {
timer.invalidate()
self.subscriber?.receive(completion: .finished)
self.cancel()
return
}
demand -= 1
demand += subscriber.receive(self.output.randomElement()! + " \(Date())")
}
}
func cancel() {
subscriber = nil
}
}
}
Here is my Custom subscriber:
final class EmojiBeamerSubscriber<Input, Failure: Error>: Subscriber, Cancellable {
var subscription: Subscription?
let receiveValue: (Input) -> Void
init(receiveValue: #escaping (Input) -> Void) {
self.receiveValue = receiveValue
}
func receive(subscription: Subscription) {
self.subscription = subscription
subscription.request(.max(3)) // Here I send only 3 as max demand
}
func receive(_ input: Input) -> Subscribers.Demand {
receiveValue(input)
return .none
}
func receive(completion: Subscribers.Completion<Failure>) {
print("Will handle later:", completion)
}
func cancel() {
self.subscription?.cancel()
self.subscription = nil
}
}
extension Publisher {
func myCustomSink(receiveValueHandler: #escaping (Self.Output) -> Void) -> AnyCancellable {
let myCustomSubscriber = EmojiBeamerSubscriber<Self.Output, Self.Failure>(receiveValue: receiveValueHandler)
subscribe(myCustomSubscriber)
return AnyCancellable(myCustomSubscriber)
}
}
As you can see on my custom subscription I request with demand .max(3) if I don't use collect everything works fine, I get an emoji beamed every 5 second after 3 I got a .finish completion.
Works fine (and sends .max(3) demand):
let emojiBeamer = EmojiBeamerPublisher()
var cancellables = Set<AnyCancellable>()
emojiBeamer
.myCustomSink { value in Swift.print("Random Emoji:: \(value)") }
.store(in: &cancellables)
However if I simply add .collect() to catch all 3 results at once in an array it just requests with .unlimited demand on my subscription, resulting in a never ending subscription because my demand will never reach zero.
Never complete (and sends unlimited demand):
let emojiBeamer = EmojiBeamerPublisher()
var cancellables = Set<AnyCancellable>()
emojiBeamer
.collect()
.myCustomSink { value in Swift.print("Random Emoji:: \(value)") }
.store(in: &cancellables)
Is there something wrong with my implementation? Or Did I misunderstood the purpose of .collect() operator?
Thank you in advance :)

Answer 1

From the documentation:
This publisher requests an unlimited number of elements from the upstream publisher and uses an unbounded amount of memory to store the received values. The publisher may exert memory pressure on the system for very large sets of elements.
So the behaviour you noticed is the correct one, as collect() sends an unlimited demand upstream.
The unlimited demand causes the demand -= 1 instruction to do nothing, so the demand > 0 check will always pass, resulting into an infinite loop that never sends the completion. You will need an extra condition to make the "collected" stream a finite one.
For infinite streams, the collect(_:) overload (the one that allows to pass a number of items to collect) performs better in regards to the demand, but still requests from upstream more elements than one might expect:
When this publisher receives a request for .max(n) elements, it requests .max(count * n) from the upstream publisher.