I have two lists of URLs that return some links to images.
The lists are passed into a future like
static func loadRecentEpisodeImagesFuture(request: [URL]) -> AnyPublisher<[RecentEpisodeImages], Never> {
return Future { promise in
print(request)
networkAPI.recentEpisodeImages(url: request)
.sink(receiveCompletion: { _ in },
receiveValue: { recentEpisodeImages in
promise(.success(recentEpisodeImages))
})
.store(in: &recentImagesSubscription)
}
.eraseToAnyPublisher()
}
Which calls:
/// Get a list of image sizes associated with a featured episode .
func featuredEpisodeImages(featuredUrl: [URL]) -> AnyPublisher<[FeaturedEpisodeImages], Error> {
let featuredEpisodesImages = featuredUrl.map { (featuredUrl) -> AnyPublisher<FeaturedEpisodeImages, Error> in
return URLSession.shared
.dataTaskPublisher(for: featuredUrl)
.map(\.data)
.decode(type: FeaturedEpisodeImages.self, decoder: decoder)
.receive(on: networkApiQueue)
.catch { _ in Empty<FeaturedEpisodeImages, Error>() }
.print("###Featured###")
.eraseToAnyPublisher()
}
return Publishers.MergeMany(featuredEpisodesImages).collect().eraseToAnyPublisher()
}
/// Get a list of image sizes associated with a recent episode .
func recentEpisodeImages(recentUrl: [URL]) -> AnyPublisher<[RecentEpisodeImages], Error> {
let recentEpisodesImages = recentUrl.map { (recentUrl) -> AnyPublisher<RecentEpisodeImages, Error> in
return URLSession.shared
.dataTaskPublisher(for: recentUrl)
.map(\.data)
.decode(type: RecentEpisodeImages.self, decoder: decoder)
.receive(on: networkApiQueue)
.catch { _ in Empty<RecentEpisodeImages, Error>() }
.print("###Recent###")
.eraseToAnyPublisher()
}
return Publishers.MergeMany(recentEpisodesImages).collect().eraseToAnyPublisher()
}
and is attached to the app state:
/// Takes an action and returns a future mapped to another action.
static func recentEpisodeImages(action: RequestRecentEpisodeImages) -> AnyPublisher<Action, Never> {
return loadRecentEpisodeImagesFuture(request: action.request)
.receive(on: networkApiQueue)
.map({ images in ResponseRecentEpisodeImages(response: images) })
.replaceError(with: RequestFailed())
.eraseToAnyPublisher()
}
It seems that:
return Publishers.MergeMany(recentEpisodes).collect().eraseToAnyPublisher()
doesn't give me a reliable downstream value as whichever response finishes last overwrites the earlier response.
I am able to log the responses of both series of requests. Both are processing the correct arrays and returning the proper json.
I would like something like:
return recentEpisodeImages
but currently this gives me the error
Cannot convert return expression of type '[AnyPublisher<RecentEpisodeImages, Error>]' to return type 'AnyPublisher<[RecentEpisodeImages], Error>'
How can I collect the values of the inner publisher and return them as
AnyPublisher<[RecentEpisodeImages], Error>
Presuming that the question is how to turn an array of URLs into an array of what you get when you download and process the data from those URLs, the answer is: turn the array into a sequence publisher, process each URL by way of flatMap, and collect the result.
Here, for instance, is how to turn an array of URLs representing images into an array of the actual images (not identically what you're trying to do, but probably pretty close):
func publisherOfArrayOfImages(urls:[URL]) -> AnyPublisher<[UIImage],Error> {
urls.publisher
.flatMap { (url:URL) -> AnyPublisher<UIImage,Error> in
return URLSession.shared.dataTaskPublisher(for: url)
.compactMap { UIImage(data: $0.0) }
.mapError { $0 as Error }
.eraseToAnyPublisher()
}.collect().eraseToAnyPublisher()
}
And here's how to test it:
let urls = [
URL(string:"http://www.apeth.com/pep/moe.jpg")!,
URL(string:"http://www.apeth.com/pep/manny.jpg")!,
URL(string:"http://www.apeth.com/pep/jack.jpg")!,
]
let pub = publisherOfArrayOfImages(urls:urls)
pub.sink { print($0) }
receiveValue: { print($0) }
.store(in: &storage)
You'll see that what pops out the bottom of the pipeline is an array of three images, corresponding to the array of three URLs we started with.
(Note, please, that the order of the resulting array is random. We fetched the images asynchronously, so the results arrive back at our machine in whatever order they please. There are ways around that problem, but it is not what you asked about.)
Related
I just started to learn Combine and therefore I can't figure out how to make a complex request to the API.
It is necessary to create an application where the user can enter the name of the company's GitHub account in the input field and get a list of open repositories and their branches.
There are two API methods:
https://api.github.com/orgs/<ORG_NAME>/repos This method returns a list of organization account repositories by name. For example, you can try to request a list of Apple's repositories https://api.github.com/orgs/apple/repos
struct for this method
struct Repository: Decodable {
let name: String
let language: String?
enum Seeds {
public static let empty = Repository(name: "", language: "")
}
}
https://api.github.com/repos/<ORG_NAME>/<REPO_NAME>/branches This method will be needed to get the branch names in the specified repository.
struct for this method
struct Branch: Decodable {
let name: String
}
As a result, I need to get an array of such structures.
struct BranchSectionModel {
var name: Repository
var branchs: [Branch]
}
For this I have two functions:
func loadRepositorys(orgName: String) -> AnyPublisher<[Repository], Never> {
guard let url = URL(string: "https://api.github.com/orgs/\(orgName)/repos" ) else {
return Just([])
.eraseToAnyPublisher()
}
return URLSession.shared.dataTaskPublisher(for: url)
.map { $0.data }
.decode(type: [Repository].self, decoder: JSONDecoder())
.replaceError(with: [])
.receive(on: RunLoop.main)
.eraseToAnyPublisher()
}
and
func loadBranchs(orgName: String, repoName: String) -> AnyPublisher<[Branch], Never> {
guard let url = URL(string: "https://api.github.com/repos/\(orgName)/\(repoName)/branches") else {
return Just([])
.eraseToAnyPublisher()
}
return URLSession.shared.dataTaskPublisher(for: url)
.map { $0.data }
.decode(type: [Branch].self, decoder: JSONDecoder())
.replaceError(with: [])
.receive(on: RunLoop.main)
.eraseToAnyPublisher()
}
Both of these functions work separately, but I don't know how to end up with an [BranchSectionModel] . I guess to use flatMap and sink, but don't understant how.
I do not understand how to combine these two requests in one thread.
When you're looking to convert one publisher into another, .map and .switchToLatest. In this case, since you're also looking to turn one publisher into many (and then back down into one), MergeMany will also be a useful tool:
loadRepositorys(orgName: orgName)
.map { repos in
Publishers.MergeMany(repos.map { repo in
loadBranchs(orgName: orgName, repoName: repo.name)
.map { branches in
BranchSectionModel(name: repo, branchs: branches)
}
})
.collect(repos.count)
}
.switchToLatest()
.sink { result in
print("---")
print(result)
}
.store(in: &cancellables)
Although I'm a big fan of Combine, I don't think it's particularly well suited to this task, compared with async/await, which will probably be a little less confusing and look cleaner. As a learning exercise, it's a great one, but if you were to tackle this problem in the real world, async/await would likely be my go-to.
I have a lot (~200) urls for images, and I need to download each one, then process (resize) it, then update the cache. The thing is - I only want to have at max 3 requests at once, and since the images are heavy, I also don't want a lot of responses "hanging" waiting to be processed (and taking memory...).
TLDR I want to call the next (4th) network request only after the receiveValue in the sink is called on one of the first 3 requests... (ie after the network response & processing are both done...).
Will this flow work, and will it hold on to the waiting urls and not drop them on the floor?
Also do I need that buffer() call? I use it after seeing this answer: https://stackoverflow.com/a/67011837/2242359
wayTooManyURLsToHandleAtOnce // this is a `[URL]`
.publisher
.buffer(size: .max, prefetch: .byRequest, whenFull: .dropNewest) // NEEDED?
.flatMap(maxPublishers: .max(3)) { url in
URLSession.shared
.dataTaskPublisher(for: url)
.map { (data: Data, _) -> Picture in
Picture(from: data)
}
}
.tryCompactMap {
resizeImage(picture: $0) // takes a while and might fail
}
.receive(on: DispatchQueue.main)
.sink { completion
// handling completion...
} receiveValue: { resizedImage
self.cache.append(resizedImage)
}
.store(...)
I would use a subject. This not an optimal solution but it looks working and maybe will trigger other ideas
var cancellable: AnyCancellable?
var urls: [String] = (0...6).map { _ in "http://httpbin.org/delay/" + String((0...2).randomElement()!) }
var subject: PassthroughSubject<[String], Never> = .init()
let maxConcurrentRequests = 3
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
print(urls)
cancellable = subject
.flatMap({ urls -> AnyPublisher<[URLSession.DataTaskPublisher.Output], URLError> in
let requests = urls.map { URLSession.shared.dataTaskPublisher(for: URL.init(string: $0)!) }
return Publishers.MergeMany(requests)
.collect().eraseToAnyPublisher()
})
.print()
.sink(receiveCompletion: { completion in
print(completion)
}, receiveValue: { value in
print(value)
if self.urls.count <= self.maxConcurrentRequests {
self.urls.removeAll()
self.subject.send(completion: .finished)
} else {
self.urls.removeLast(self.maxConcurrentRequests)
self.subject.send(self.urls.suffix(self.maxConcurrentRequests))
}
})
subject.send(urls.suffix(maxConcurrentRequests))
}
I'm attempting to use combine to chain two requests together. The code is pretty rough, but I need to call two api requests. One to get the schedule data than one for live data. I'm able to get the live data (second request) but how do I get the schedule data (first request)? I'm having a hard time understanding how to use combine to chain two requests together, this is my first need to use combine for a widget I'm working on. I'm still fresh to Swift, so my terminology may be lacking.
My last code example wasn't correct and my question was unclear. I have two publishers and the second one depends on the first one. My understanding is still unclear on how to handle the data from my first publisher as well as in .flatMap for the second data. Does it need to be ObservableObject class and have #Published variables for the data? Do I use .assign or .sink to get data from my codable data Schedule and Live? Articles seem a bit too advance for myself as they create custom extensions and changing the API data to nested types.
New example code
import Foundation
import Combine
class DataGroup {
// How to get data from Schedule and Live codable data, do I use a variable and .assign or .sink?
// Where do I put the subscriber?
func requestSchedule(_ teamID : Int) -> AnyPublisher<Schedule, Error> {
let url = URL(string: "https://statsapi.web.nhl.com/api/v1/schedule?teamId=\(teamID)")!
return URLSession
.shared.dataTaskPublisher(for: url)
.map(\.data)
.decode(type: Schedule.self, decoder: JSONDecoder())
.flatMap {self.fetchLiveFeed($0.dates.first?.games.first?.link ?? "")}
/*
.flatMap {URLSession.shared.dataTaskPublisher(for: URL(string: $0.dates.first?.games.first?.link ?? "")!)}
*/
.eraseToAnyPublisher()
}
// Remove and put into flatMap URLSession.shared.dataTaskPublisher?
func fetchLiveFeed(_ link: String) -> AnyPublisher<Live, Error> {
let url = URL(string: "https://statsapi.web.nhl.com\(link)")!
return URLSession.shared.dataTaskPublisher(for: url)
.map(\.data)
.decode(type: Live.self, decoder: JSONDecoder())
.eraseToAnyPublisher()
}
}
OLD
import Foundation
import Combine
class CombineData {
var schedule: Schedule? // Get schedule data alongside live data
var live: Live?
private var cancellables = Set<AnyCancellable>()
func fetchSchedule(_ teamID: Int, _ completion: #escaping (/* Schedule, */Live) -> Void) {
let url = URL(string: "https://statsapi.web.nhl.com/api/v1/schedule?teamId=\(teamID)")!
URLSession.shared.dataTaskPublisher(for: url)
.map { $0.data }
.decode(type: Schedule.self, decoder: JSONDecoder())
.flatMap { self.fetchLiveFeed($0.dates.first?.games.first?.link ?? "") }
.receive(on: DispatchQueue.main)
.sink(receiveCompletion: { _ in }) { data in
// How to get both schedule data and live data here?
//self.schedule = ?
self.live = data
print(data)
completion(self.schedule!, self.live!)
}.store(in: &cancellables)
}
func fetchLiveFeed(_ link: String) -> AnyPublisher<Live, Error> {
let url = URL(string: "https://statsapi.web.nhl.com\(link)")!
return URLSession.shared.dataTaskPublisher(for: url)
.map(\.data)
.decode(type: Live.self, decoder: JSONDecoder())
.eraseToAnyPublisher()
}
}
The general idea is to use a flatMap for chaining, which is what you did, but if you also need the original value, you can return a Zip publisher (with a .zip operator) that puts two results into a tuple.
One of the publishers is the second request, and the other should just emit the value. You can typically do this with Just(v), but you have to make sure that its failure type (which is Never) matches with the other publisher. You can match its failure type with .setFailureType(to:):
publisher1
.flatMap { one in
Just(one).setFailureType(to: Error.self) // error has to match publisher2
.zip(publisher2(with: one))
}
.sink(receiveCompletion: { completion in
// ...
}, receiveValue: { (one, two) in
// ...
})
Alternatively, you can use Result.Publisher which would infer the error (but might look somewhat odd):
.flatMap { one in
Result.Publisher(.success(one))
.zip(publisher2)
}
So, in your case it's going to be something like this:
URLSession.shared.dataTaskPublisher(for: url)
.map(\.data)
.decode(type: Schedule.self, decoder: JSONDecoder())
.flatMap {
Result.Publisher(.success($0))
.zip(self.fetchLiveFeed($0.dates.first?.games.first?.link ?? ""))
}
.sink(receiveCompletion: { completion in
// ...
}, receiveValue: { (schedule, live) in
// ...
})
.store(in: &cancellables)
I'm playing around with publishers in Swift/Combine, I have a function that fetches 100 records and returns them as an array.
As a test I want to return just the first two items, but it's not working as I expected it to, it always returns 100, my feeling is that it's because, the first item is an array of 100 items, if so, how do I split them out?
import UIKit
import Combine
struct Post : Decodable {
let userId: Int
let id: Int
let title: String
let body: String
}
//let url = URL(string: "https://jsonplaceholder.typicode.com/todos/1")!
let url = URL(string: "https://jsonplaceholder.typicode.com/posts")!
var subscriptions: Set<AnyCancellable> = []
func fetch() -> AnyPublisher<[Post], Never> {
return URLSession.shared.dataTaskPublisher(for: url)
.tryCompactMap{ (arg) -> [Post]? in
let (data, _) = arg
return try JSONDecoder().decode([Post].self, from: data)
}
//.print("here")
.replaceError(with: [])
.eraseToAnyPublisher()
}
fetch()
.prefix(2)
.sink(receiveCompletion: { (comp) in
print("comp: \(comp)")
}) { (res) in
print("Res: \(res.count)")
}.store(in: &subscriptions)
Update, this seems to work, not sure on the syntax though:
fetch()
.flatMap { Publishers.Sequence(sequence: $0) }
.prefix(2)
.sink(receiveCompletion: { (comp) in
print("comp: \(comp)")
}) { (res) in
print("Res: \(res)")
}.store(in: &subscriptions)
You can use map to take the full array and extract only what you need. Take a look at the following example:
[Array(0..<100)].publisher.map { array in
return Array(array[..<2])
}.sink(receiveValue: { items in
print(items)
})
This is a publisher that publishes an array with 100 values. I then use array[..<2] to create an ArraySlice that contains the first two items. This slice is then converted to Array so it's easier to use later.
The items argument received in sink is an array with just two items.
.flatMap(maxPublishers: 2)
Could be a better approach for you depending on what you want to accomplish overall.
I have a publisher which takes a network call and returns an array of IDs. I now need to call another network call for each ID to get all my data. And I want the final publisher to have the resulting object.
First network result:
"user": {
"id": 0,
"items": [1, 2, 3, 4, 5]
}
Final object:
struct User {
let id: Int
let items: [Item]
... other fields ...
}
struct Item {
let id: Int
... other fields ...
}
Handling multiple network calls:
userPublisher.flatMap { user in
let itemIDs = user.items
return Future<[Item], Never>() { fulfill in
... OperationQueue of network requests ...
}
}
I would like to perform the network requests in parallel, since they are not dependent on each other. I'm not sure if Future is right here, but I'd imagine I would then have code to do a
DispatchGroup or OperationQueue and fulfill when they're all done. Is there more of a Combine way of doing this?
Doe Combine have a concept of splitting one stream into many parallel streams and joining the streams together?
Combine offers extensions around URLSession to handle network requests unless you really need to integrate with OperationQueue based networking, then Future is a fine candidate. You can run multiple Futures and collect them at some point, but I'd really suggest looking at URLSession extensions for Combine.
struct User: Codable {
var username: String
}
let requestURL = URL(string: "https://example.com/")!
let publisher = URLSession.shared.dataTaskPublisher(for: requestURL)
.map { $0.data }
.decode(type: User.self, decoder: JSONDecoder())
Regarding running a batch of requests, it's possible to use Publishers.MergeMany, i.e:
struct User: Codable {
var username: String
}
let userIds = [1, 2, 3]
let subscriber = Just(userIds)
.setFailureType(to: Error.self)
.flatMap { (values) -> Publishers.MergeMany<AnyPublisher<User, Error>> in
let tasks = values.map { (userId) -> AnyPublisher<User, Error> in
let requestURL = URL(string: "https://jsonplaceholder.typicode.com/users/\(userId)")!
return URLSession.shared.dataTaskPublisher(for: requestURL)
.map { $0.data }
.decode(type: User.self, decoder: JSONDecoder())
.eraseToAnyPublisher()
}
return Publishers.MergeMany(tasks)
}.collect().sink(receiveCompletion: { (completion) in
if case .failure(let error) = completion {
print("Got error: \(error.localizedDescription)")
}
}) { (allUsers) in
print("Got users:")
allUsers.map { print("\($0)") }
}
In the example above I use collect to collect all results, which postpones emitting the value to the Sink until all of the network requests successfully finished, however you can get rid of the collect and receive each User in the example above one by one as network requests complete.