I created a simple Publisher from an array of paths I want to fetch from the internet. I am setting the failure type to match the DataTaskPublisher, and then I flatMap to get the new Publisher with the DataTask results. However, when I subscribe to the stream with sink, nothing gets called.
Here is my code:
import Combine
import Foundation
class NetworkManager {
var tasks = Set<AnyCancellable>()
init() {
getData()
}
func getData() {
let baseUrl = URL(string: "https://fmi.unibuc.ro")!
["/prezentare", "/cazare"].publisher
.setFailureType(to: URLError.self)
.flatMap { path -> URLSession.DataTaskPublisher in
let url = baseUrl.appendingPathComponent(path)
return URLSession.shared.dataTaskPublisher(for: url)
}
.sink(receiveCompletion: { completion in
print(completion)
}, receiveValue: { value in
print(value)
})
.store(in: &tasks)
}
}
let manager = NetworkManager()
What am I doing wrong? 🤔
Swift playgrounds finish execution when all synchronous code in them returned. However, you are executing a network request asynchronously, so you need to tell the playground to wait for the async result.
Call this before starting the network request:
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
And in the sink, you can finish execution by calling
PlaygroundPage.current.finishExecution()
Related
I have a lot (~200) urls for images, and I need to download each one, then process (resize) it, then update the cache. The thing is - I only want to have at max 3 requests at once, and since the images are heavy, I also don't want a lot of responses "hanging" waiting to be processed (and taking memory...).
TLDR I want to call the next (4th) network request only after the receiveValue in the sink is called on one of the first 3 requests... (ie after the network response & processing are both done...).
Will this flow work, and will it hold on to the waiting urls and not drop them on the floor?
Also do I need that buffer() call? I use it after seeing this answer: https://stackoverflow.com/a/67011837/2242359
wayTooManyURLsToHandleAtOnce // this is a `[URL]`
.publisher
.buffer(size: .max, prefetch: .byRequest, whenFull: .dropNewest) // NEEDED?
.flatMap(maxPublishers: .max(3)) { url in
URLSession.shared
.dataTaskPublisher(for: url)
.map { (data: Data, _) -> Picture in
Picture(from: data)
}
}
.tryCompactMap {
resizeImage(picture: $0) // takes a while and might fail
}
.receive(on: DispatchQueue.main)
.sink { completion
// handling completion...
} receiveValue: { resizedImage
self.cache.append(resizedImage)
}
.store(...)
I would use a subject. This not an optimal solution but it looks working and maybe will trigger other ideas
var cancellable: AnyCancellable?
var urls: [String] = (0...6).map { _ in "http://httpbin.org/delay/" + String((0...2).randomElement()!) }
var subject: PassthroughSubject<[String], Never> = .init()
let maxConcurrentRequests = 3
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
print(urls)
cancellable = subject
.flatMap({ urls -> AnyPublisher<[URLSession.DataTaskPublisher.Output], URLError> in
let requests = urls.map { URLSession.shared.dataTaskPublisher(for: URL.init(string: $0)!) }
return Publishers.MergeMany(requests)
.collect().eraseToAnyPublisher()
})
.print()
.sink(receiveCompletion: { completion in
print(completion)
}, receiveValue: { value in
print(value)
if self.urls.count <= self.maxConcurrentRequests {
self.urls.removeAll()
self.subject.send(completion: .finished)
} else {
self.urls.removeLast(self.maxConcurrentRequests)
self.subject.send(self.urls.suffix(self.maxConcurrentRequests))
}
})
subject.send(urls.suffix(maxConcurrentRequests))
}
I may be going about this the wrong way, but I have a function with which I want to emit multiple values over time. But I don’t want it to start emitting until something is subscribed to that object. I’m coming to combine from RxSwift, so I’m basically trying to duplicated Observable.create() in the RxSwift world. The closest I have found is returning a Future, but futures only succeed or fail (so they are basically like a Single in RxSwift.)
Is there some fundamental thing I am missing here? My end goal is to make a function that processes a video file and emits progress events until it completes, then emits a URL for the completed file.
Generally you can use a PassthroughSubject to publish custom outputs. You can wrap a PassthroughSubject (or multiple PassthroughSubjects) in your own implementation of Publisher to ensure that only your process can send events through the subject.
Let's mock a VideoFrame type and some input frames for example purposes:
typealias VideoFrame = String
let inputFrames: [VideoFrame] = ["a", "b", "c"]
Now we want to write a function that synchronously processes these frames. Our function should report progress somehow, and at the end, it should return the output frames. To report progress, our function will take a PassthroughSubject<Double, Never>, and send its progress (as a fraction from 0 to 1) to the subject:
func process(_ inputFrames: [VideoFrame], progress: PassthroughSubject<Double, Never>) -> [VideoFrame] {
var outputFrames: [VideoFrame] = []
for input in inputFrames {
progress.send(Double(outputFrames.count) / Double(inputFrames.count))
outputFrames.append("output for \(input)")
}
return outputFrames
}
Okay, so now we want to turn this into a publisher. The publisher needs to output both progress and a final result. So we'll use this enum as its output:
public enum ProgressEvent<Value> {
case progress(Double)
case done(Value)
}
Now we can define our Publisher type. Let's call it SyncPublisher, because when it receives a Subscriber, it immediately (synchronously) performs its entire computation.
public struct SyncPublisher<Value>: Publisher {
public init(_ run: #escaping (PassthroughSubject<Double, Never>) throws -> Value) {
self.run = run
}
public var run: (PassthroughSubject<Double, Never>) throws -> Value
public typealias Output = ProgressEvent<Value>
public typealias Failure = Error
public func receive<Downstream: Subscriber>(subscriber: Downstream) where Downstream.Input == Output, Downstream.Failure == Failure {
let progressSubject = PassthroughSubject<Double, Never>()
let doneSubject = PassthroughSubject<ProgressEvent<Value>, Error>()
progressSubject
.setFailureType(to: Error.self)
.map { ProgressEvent<Value>.progress($0) }
.append(doneSubject)
.subscribe(subscriber)
do {
let value = try run(progressSubject)
progressSubject.send(completion: .finished)
doneSubject.send(.done(value))
doneSubject.send(completion: .finished)
} catch {
progressSubject.send(completion: .finished)
doneSubject.send(completion: .failure(error))
}
}
}
Now we can turn our process(_:progress:) function into a SyncPublisher like this:
let inputFrames: [VideoFrame] = ["a", "b", "c"]
let pub = SyncPublisher<[VideoFrame]> { process(inputFrames, progress: $0) }
The run closure is { process(inputFrames, progress: $0) }. Remember that $0 here is a PassthroughSubject<Double, Never>, exactly what process(_:progress:) wants as its second argument.
When we subscribe to this pub, it will first create two subjects. One subject is the progress subject and gets passed to the closure. We'll use the other subject to publish either the final result and a .finished completion, or just a .failure completion if the run closure throws an error.
The reason we use two separate subjects is because it ensures that our publisher is well-behaved. If the run closure returns normally, the publisher publishes zero or more progress reports, followed by a single result, followed by .finished. If the run closure throws an error, the publisher publishes zero or more progress reports, followed by a .failed. There is no way for the run closure to make the publisher emit multiple results, or emit more progress reports after emitting the result.
At last, we can subscribe to pub to see if it works properly:
pub
.sink(
receiveCompletion: { print("completion: \($0)") },
receiveValue: { print("output: \($0)") })
Here's the output:
output: progress(0.0)
output: progress(0.3333333333333333)
output: progress(0.6666666666666666)
output: done(["output for a", "output for b", "output for c"])
completion: finished
The following extension to AnyPublisher replicates Observable.create semantics by composing a PassthroughSubject. This includes cancellation semantics.
AnyPublisher.create() Swift 5.6 Extension
public extension AnyPublisher {
static func create<Output, Failure>(_ subscribe: #escaping (AnySubscriber<Output, Failure>) -> AnyCancellable) -> AnyPublisher<Output, Failure> {
let passthroughSubject = PassthroughSubject<Output, Failure>()
var cancellable: AnyCancellable?
return passthroughSubject
.handleEvents(receiveSubscription: { subscription in
let subscriber = AnySubscriber<Output, Failure> { subscription in
} receiveValue: { input in
passthroughSubject.send(input)
return .unlimited
} receiveCompletion: { completion in
passthroughSubject.send(completion: completion)
}
cancellable = subscribe(subscriber)
}, receiveCompletion: { completion in
}, receiveCancel: {
cancellable?.cancel()
})
.eraseToAnyPublisher()
}
}
Usage
func doSomething() -> AnyPublisher<Int, Error> {
return AnyPublisher<Int, Error>.create { subscriber in
// Imperative implementation of doing something can call subscriber as follows
_ = subscriber.receive(1)
subscriber.receive(completion: .finished)
// subscriber.receive(completion: .failure(myError))
return AnyCancellable {
// Imperative cancellation implementation
}
}
}
I have a publisher which takes a network call and returns an array of IDs. I now need to call another network call for each ID to get all my data. And I want the final publisher to have the resulting object.
First network result:
"user": {
"id": 0,
"items": [1, 2, 3, 4, 5]
}
Final object:
struct User {
let id: Int
let items: [Item]
... other fields ...
}
struct Item {
let id: Int
... other fields ...
}
Handling multiple network calls:
userPublisher.flatMap { user in
let itemIDs = user.items
return Future<[Item], Never>() { fulfill in
... OperationQueue of network requests ...
}
}
I would like to perform the network requests in parallel, since they are not dependent on each other. I'm not sure if Future is right here, but I'd imagine I would then have code to do a
DispatchGroup or OperationQueue and fulfill when they're all done. Is there more of a Combine way of doing this?
Doe Combine have a concept of splitting one stream into many parallel streams and joining the streams together?
Combine offers extensions around URLSession to handle network requests unless you really need to integrate with OperationQueue based networking, then Future is a fine candidate. You can run multiple Futures and collect them at some point, but I'd really suggest looking at URLSession extensions for Combine.
struct User: Codable {
var username: String
}
let requestURL = URL(string: "https://example.com/")!
let publisher = URLSession.shared.dataTaskPublisher(for: requestURL)
.map { $0.data }
.decode(type: User.self, decoder: JSONDecoder())
Regarding running a batch of requests, it's possible to use Publishers.MergeMany, i.e:
struct User: Codable {
var username: String
}
let userIds = [1, 2, 3]
let subscriber = Just(userIds)
.setFailureType(to: Error.self)
.flatMap { (values) -> Publishers.MergeMany<AnyPublisher<User, Error>> in
let tasks = values.map { (userId) -> AnyPublisher<User, Error> in
let requestURL = URL(string: "https://jsonplaceholder.typicode.com/users/\(userId)")!
return URLSession.shared.dataTaskPublisher(for: requestURL)
.map { $0.data }
.decode(type: User.self, decoder: JSONDecoder())
.eraseToAnyPublisher()
}
return Publishers.MergeMany(tasks)
}.collect().sink(receiveCompletion: { (completion) in
if case .failure(let error) = completion {
print("Got error: \(error.localizedDescription)")
}
}) { (allUsers) in
print("Got users:")
allUsers.map { print("\($0)") }
}
In the example above I use collect to collect all results, which postpones emitting the value to the Sink until all of the network requests successfully finished, however you can get rid of the collect and receive each User in the example above one by one as network requests complete.
In Xcode 11 beta 5 or 6 my existing code that relied on URLSession.DataTaskPublisher stopped working. It seems like DataTaskPublisher is never publishing any values but I can't work out why.
I've tried with .sink and .handleEvents as subscribers. I've tested .sink with a Just publisher and confirmed it receives a value there.
I've also tried both giving the DataTaskPublisher a URL and giving it a URLRequest. I've tried a request to an API including an authorization header, as well as basic requests to google.com and apple.com. I've tried using URLSession.shared and creating a new instance of URLSession. I've also tried with and without map and decode operators.
I've used XCTest expectations to confirm that the test times out every single time, even if I give it a 4-minute timeout.
I just made a new example project and replicated the problem with the following code in the root view controller:
override func viewDidLoad() {
super.viewDidLoad()
print("view did load")
URLSession.shared.dataTaskPublisher(for: URL(string: "http://apple.com")!)
.handleEvents(receiveSubscription: { (sub) in
print(sub)
}, receiveOutput: { (response) in
print(response)
}, receiveCompletion: { (completion) in
print(completion)
}, receiveCancel: {
print("cancel")
}, receiveRequest: { (demand) in
print(demand)
})
}
The project prints "view did load" but nothing else ever prints. Any ideas about where I'm going wrong here? Thanks!
I think that there are two problems with your code, firstly you only have a publisher (handleEvent returns a publisher) and secondly that publisher goes out of scope and disappears. This works although it isn't exactly elegant.
import Combine
import SwiftUI
var pub: AnyPublisher<(data: Data, response: URLResponse), URLError>? = nil
var sub: Cancellable? = nil
var data: Data? = nil
var response: URLResponse? = nil
func combineTest() {
guard let url = URL(string: "https://apple.com") else {
return
}
pub = URLSession.shared.dataTaskPublisher(for: url)
.print("Test")
.eraseToAnyPublisher()
sub = pub?.sink(
receiveCompletion: { completion in
switch completion {
case .finished:
break
case .failure(let error):
fatalError(error.localizedDescription)
}
},
receiveValue: { data = $0.data; response = $0.response }
)
}
struct ContentView: View {
var body: some View {
Button(
action: { combineTest() },
label: { Text("Do It").font(.largeTitle) }
)
}
}
I did it in SwiftUI so that I would have less to worry about and I used 3 variables so that I could follow better. You need to use the 2 parameter sink as the publisher's error isn't Never. Finally the print() is just for test and works really well.
I would like to wrap a simple callback so that it would be able to be used as a Combine Publisher. Specifically the NSPersistentContainer.loadPersistentStore callback so I can publish when the container is ready to go.
func createPersistentContainer(name: String) -> AnyPublisher<NSPersistentContainer, Error> {
// What goes here?
// Happy path: send output NSPersistentContainer; send completion.
// Not happy path: send failure Error; send completion.
}
For instance, what would the internals of a function, createPersistentContainer given above, look like to enable me to do something like this in my AppDelegate.
final class AppDelegate: UIResponder, UIApplicationDelegate {
let container = createPersistentContainer(name: "DeadlyBattery")
.assertNoFailure()
.eraseToAnyPublisher()
// ...
}
Mostly this boils down to, how do you wrap a callback in a Publisher?
As one of the previous posters #Ryan pointed out, the solution is to use the Future publisher.
The problem of using only the Future, though, is that it is eager, which means that it starts executing its promise closure at the moment of creation, not when it is subscribed to. The answer to that challenge is to wrap it in the Deferred publisher:
func createPersistentContainer(name: String) -> AnyPublisher<NSPersistentContainer, Error> {
return Deferred {
Future<NSPersistentContainer, Error> { promise in
let container = NSPersistentContainer(name: name)
container.loadPersistentStores { _, error in
if let error = error {
promise(.failure(error))
} else {
promise(.success(container))
}
}
}
}.eraseToAnyPublisher()
}
It seems that Combine's Future is the correct tool for the job.
func createPersistentContainer(name: String) -> AnyPublisher<NSPersistentContainer, Error> {
let future = Future<NSPersistentContainer, Error> { promise in
let container = NSPersistentContainer(name: name)
container.loadPersistentStores { _, error in
if let error = error {
promise(.failure(error))
} else {
promise(.success(container))
}
}
}
return AnyPublisher(future)
}
NSPersistentContainer is just a convenience wrapper around a core data stack, you would be better off subscribing at the source:
NotificationCenter.default.publisher(for: .NSPersistentStoreCoordinatorStoresDidChange)