I've been trying to figure out how to upload multiple files (images) to AWS S3 from Swift. I asked a previous question here but did not receive any responses. Rather than try & ask the same question again, I'm hoping to find guidance on creating a retry function/wrapper for a function that fires a network request w/ a completion handler. I found this question here & have tried to model it similarly. The answers by Cristik & JustinM are written in Swift 3, so when trying to write them in Swift 4, I had to make some changes. Sadly, I'm worried I lost some logic in the conversion (I don't fully understand everything that they suggested, so some of my changes were just suggestions by Xcode). Here is what I have refactored from their question & implementing the AWS S3 Transfer Utility:
static func retryFunction(numberOfTimes: Int, task: #escaping (_ success: # escaping (String) -> (), _ failure: #escaping (Error) -> ()) -> (), success: #escaping (String) -> (), failure: #escaping (Error) -> ()) {
task({ (result) in
success(result)
}) { (error) in
if numberOfTimes > 1 {
print("retrying with \(numberOfTimes - 1)")
self.retryFunction(numberOfTimes: numberOfTimes - 1, task: task, success: success, failure: failure)
} else {
failure(error)
}
}
}
static func singleImageUploadAttempt(_ dataToUpload: Data, _ imageKey: String,
_ success: #escaping (String) -> (), _ failure: #escaping (Error) -> ()) {
let expression = AWSS3TransferUtilityUploadExpression()
var completionHandler: AWSS3TransferUtilityUploadCompletionHandlerBlock?
let transferUtility = AWSS3TransferUtility.default()
completionHandler = { (task, error) -> Void in
if let error = error {
print("failure to upload the image: \(imageKey)")
failure(error)
} else{
print("\(imageKey) uploaded successfully")
success(imageKey)
}
}
transferUtility.uploadData(dataToUpload, bucket: s3BucketName, key: imageKey, contentType: "image/png",
expression: expression, completionHandler: completionHandler).continueWith { (task) -> AnyObject! in
if let error = task.error {
print("An error occurred generating the task")
failure(error)
}
return nil;
}
}
static func uploadImages(imagesToUpload: [UIImage], complete: #escaping (String?) -> ()) {
let folderKey = UUID().uuidString
var retryArray = [String: Data]()
let imageGroup = DispatchGroup()
for i in 0..<imagesToUpload.count {
let resizedImage = compressImage(imageToReduce: imagesToUpload[i], expectedSizeInMb: 2)
let imageKey = "public/\(folderKey)/image\(i).png"
let imageData = UIImagePNGRepresentation(resizedImage!)
retryArray[imageKey] = imageData!
imageGroup.enter()
retryFunction(numberOfTimes: 3, task: { success, failure in singleImageUploadAttempt(imageData!, imageKey,
success, failure) },
success: { (keyToRemove) in
retryArray.removeValue(forKey: keyToRemove)
print("updated")
print("successfully uploaded \(imageKey)")
imageGroup.leave()
},
failure: { (err) in
print("failed to upload \(imageKey)")
imageGroup.leave()
}
)
}
imageGroup.notify(queue: DispatchQueue.main) {
if(retryArray.keys.count > 0) {
print("the images could not be uploaded")
complete(nil)
} else {
print("all images uploaded successfully")
complete(folderKey)
}
}
}
Right now, the image upload works fine when I have a connection. When I try without a connection, however, the function just waits infinitely. It calls the singleImageUploadAttempt for each image but doesn't hit the completion handler or task.error. None of my print statements are fired, and rather than hitting the function 3 times for each image (3 retries), it hits the imageGroup.notify() and then just does nothing. I apologize if I misspoke on any of this - I'm still trying to understand all of this Swift logic/syntax. I appreciate any help/suggestions, thank you!
Related
I have a situation to upload content to S3 Bucket AWS.
I am using the below code and the code is not compiled.
Please advise.
let data = // The data to upload
let expression = AWSS3TransferUtilityUploadExpression()
expression.progressBlock = {(task, progress) in DispatchQueue.main.async(execute: {
// Do something e.g. Update a progress bar.
})
}
let completionHandler = { (task, error) -> Void in
DispatchQueue.main.async(execute: {
// Do something e.g. Alert a user for transfer completion.
// On failed uploads, `error` contains the error object.
})
}
let transferUtility = AWSS3TransferUtility.default()
transferUtility.uploadData(data,
bucket: S3BucketName,
key: S3UploadKeyName,
contentType: "image/png",
expression: expression,
completionHandler: completionHandler).continueWith { (task) -> AnyObject! in
if let error = task.error {
print("Error: \(error.localizedDescription)")
}
if let _ = task.result {
// Do something with uploadTask.
}
return nil;
}
I am getting 2 below errors.
Unable to infer type of a closure parameter 'error' in the current context.
Unable to infer type of a closure parameter 'task' in the current context
I have a method execute that calls an external API with a callback that receives Result<Data?,Error>. How can I map that optional success to an unwrapped result or an Error?
func execute(then handle: #escaping (Result<Data, Error>) -> Void) {
externalAPI.retrieveData { result in
let mappedResult = result
.map {
guard let data = $0 else {
throw NSError(domain: "", code: 0, description: "error")
}
return data
}
handle(mappedResult)
}
}
This code fails with Invalid conversion from throwing function of type '(Optional<Data>) throws -> _' to non-throwing function type '(Data?) -> NewSuccess'
I was able to do this with a simple switch (below), but I was wondering if throwing a failure inside the .map is possible.
func execute(then handle: #escaping (Result<Data, Error>) -> Void) {
externalAPI.retrieveData { result in
switch result {
case .failure(let error):
handle(.failure(error))
case .success(let data):
guard let data = data else {
handle(.failure(NSError(domain: "", code: 0, description: "error")))
return
}
handle(.success(data))
}
}
}
You can convert between throws functions and functions that return Result<Success, Error> by using Result(catching:) and .get().
Here's your original map call:
.map {
guard let data = $0 else {
throw NSError(domain: "", code: 0, description: "error")
}
return data
}
Result.map takes a Result and a function that converts (Success) -> NewSuccess, and returns a Result<NewSuccess, Failure>.
Your map takes a Data (Success), and returns Result<Data, Error> (NewSuccess). So the final type, by plugging in NewSuccess is: Result<Result<Data, Error>, Error>. That's more layers than you want. You want to flatten that to just Result<Data, Error>, and that's where flatMap comes in.
Your answer shows that, but you can also pull this out into a more general-purpose tool. It only works when Failure == Error, because throws is untyped, so you can't limit it to some subset of errors. But that's what you're doing anyway. Here's tryMap:
extension Result where Failure == Error {
func tryMap<NewSuccess>(_ transform: (Success) throws -> NewSuccess) -> Result<NewSuccess, Error> {
self.flatMap { value in
Result<NewSuccess, Error> { try transform(value) }
}
}
}
With that, you can rewrite this as:
func execute(then handle: #escaping (Result<Data, Error>) -> Void) {
externalAPI.retrieveData { result in
handle(result
.tryMap {
guard let data = $0 else {
throw NSError(domain: "", code: 0, description: "error")
}
return data
})
}
}
That said, I'd probably be tempted to write it this way:
func execute(then handle: #escaping (Result<Data, Error>) -> Void) {
externalAPI.retrieveData { result in
handle(result
.flatMap { maybeData in
maybeData.map(Result.success)
?? .failure(NSError(domain: "", code: 0, description: "error"))
})
}
}
Or if I wanted someone to be able to actually read it later:
func execute(then handle: #escaping (Result<Data, Error>) -> Void) {
externalAPI.retrieveData { result in
handle(result
.flatMap {
switch $0 {
case .some(let data): return .success(data)
case .none: return .failure(NSError(domain: "", code: 0, description: "error"))
}
}
)
}
}
The advantage of this switch over yours is that you don't have to unwrap and rewrap previous failures.
Apparently, this can be done using flatmap. So in my case:
func execute(then handle: #escaping (Result<Data, Error>) -> Void) {
externalAPI.retrieveData { result in
let mappedResult = result
.flatMap { data in
Result<Data, Error> {
guard let data = data else {
throw NSError(domain: "", code: 0, description: "error")
}
return data
}
}
handle(mappedResult)
}
}
It's a little confusing, but it is working for me.
Given I have an SDK which provides the functionality below
class SDK {
static func upload(completion: #escaping (Result<String, Error>) -> Void) {
DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
completion(.success("my_value"))
}
}
}
I am able to a create a wrapper around to make its usage more functional
class CombineSDK {
func upload() -> AnyPublisher<String, Error> {
Future { promise in
SDK.upload { result in
switch result {
case .success(let key):
promise(.success(key))
case .failure(let error):
promise(.failure(error))
}
}
}.eraseToAnyPublisher()
}
}
Now I'm trying to understand how my CombineSDK.upload method should look like if the SDK upload method also provides a progress block like below:
class SDK {
static func upload(progress: #escaping (Double) -> Void, completion: #escaping (Result<String, Error>) -> Void) {
DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) {
progress(0.5)
}
DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
progress(1)
}
DispatchQueue.main.asyncAfter(deadline: .now() + 1) {
completion(.success("s3Key"))
}
}
}
We need an Output type for your publisher that represents either the progress, or the final value. So we should use an enum. Since the Foundation framework already defines a type named Progress, we'll name ours Progressable to avoid a name conflict. We might as well make it generic:
enum Progressable<Value> {
case progress(Double)
case value(Value)
}
Now we need to think about how the publisher should behave. A typical publisher, like URLSession.DataTaskPublisher, doesn't do anything until it gets a subscription, and it starts its work fresh for each subscription. The retry operator only works if the upstream publisher behaves like this.
So our publisher should behave that way, too:
extension SDK {
static func uploadPublisher() -> UploadPublisher {
return UploadPublisher()
}
struct UploadPublisher: Publisher {
typealias Output = Progressable<String>
typealias Failure = Error
func receive<S>(subscriber: S) where S : Subscriber, Self.Failure == S.Failure, Self.Output == S.Input {
<#code#>
}
}
}
Creating the publisher (by calling SDK.uploadPublisher()) doesn't start any work. We'll replace <#code#> with code to start the upload:
extension SDK {
static func uploadPublisher() -> UploadPublisher {
return UploadPublisher()
}
struct UploadPublisher: Publisher {
typealias Output = Progressable<String>
typealias Failure = Error
func receive<S>(subscriber: S) where S : Subscriber, Self.Failure == S.Failure, Self.Output == S.Input {
let subject = PassthroughSubject<Output, Failure>()
subject.receive(subscriber: subscriber)
upload(
progress: { subject.send(.progress($0)) },
completion: {
switch $0 {
case .success(let value):
subject.send(.value(value))
subject.send(completion: .finished)
case .failure(let error):
subject.send(completion: .failure(error))
}
}
)
}
}
}
Note that we call subject.receive(subscriber: subscriber) before we start the upload. This is important! What if upload calls one of its callbacks synchronously, before returning? By passing the subscriber to the subject before calling upload, we ensure that the subscriber has the chance to be notified even if upload calls its callbacks synchronously.
Note: started writing an answer that's has a largely similar intent to #robmayoff's answer, but using Deferred, so posting here for completeness.
Swift Combine only works with values and errors - there's no separate type for progress. But you can model the progress as part of the output, either as a tuple, as was suggested in another answer, or as a custom enum with both progress and result as cases, which would be my preferred approach.
class CombineSDK {
enum UploadProgress<T> {
case progress(Double)
case result(T)
}
func upload() -> AnyPublisher<UploadProgress<String>, Error> {
Deferred { () -> AnyPublisher<UploadProgress<String>, Error> in
let subject = PassthroughSubject<UploadProgress<String>, Error>()
SDK.upload(
progress: { subject.send(.progress($0)) },
completion: { r in
let _ = r.map(UploadProgress.result).publisher.subscribe(subject)
})
return subject.eraseToAnyPublisher()
}
.eraseToAnyPublisher()
}
}
EDIT
Based on #robmayoff's comment, the above solution doesn't handle synchronous case where subject.send is called before subject is returned.
The solution is largely the same, but it does introduce a small complication of having to capture these values, just in case. This can be done with Record, which will provide a temporary sink to subject
func upload() -> AnyPublisher<UploadProgress<String>, Error> {
Deferred { () -> AnyPublisher<UploadProgress<String>, Error> in
let subject = PassthroughSubject<UploadProgress<String>, Error>()
var recording = Record<UploadProgress<String>, Error>.Recording()
subject.sink(
receiveCompletion: { recording.receive(completion: $0) },
receiveValue: { recording.receive($0) })
SDK.upload(
progress: { subject.send(.progress($0)) },
completion: { r in
let _ = r.map(UploadProgress.result).publisher.subscribe(subject)
})
return Record(recording: recording).append(subject).eraseToAnyPublisher()
}
.eraseToAnyPublisher()
}
Here is possible approach
extension CombineSDK {
func upload() -> AnyPublisher<(Double, String?), Error> {
let publisher = PassthroughSubject<(Double, String?), Error>()
SDK.upload(progress: { value in
publisher.send((value, nil))
}, completion: { result in
switch result {
case .success(let key):
publisher.send((1.0, key))
publisher.send(completion: .finished)
case .failure(let error):
publisher.send(completion: .failure(error))
}
})
return publisher.eraseToAnyPublisher()
}
}
I have an array of appointments and I'm trying to grab all of the photos for these appointments from our windows azure blob storage. First, I want to get the list of blobs with the associated appointmentId so I can download and store them properly afterwards.
I'm using PromiseKit but I'm not at all sure about how to use PromiseKit in a loop:
for appointment in appointments {
// Get blobs
}
Here's my code so far. Any help is greatly appreciated!
func getBlobsPromise(appointmentId: Int32) -> Promise<[BlobDownload]> {
return Promise { seal in
var error: NSError?
var blobDownloads = [BlobDownload]()
container = AZSCloudBlobContainer(url: URL(string: containerURL)!, error: &error)
if ((error) != nil) {
print("Error in creating blob container object. Error code = %ld, error domain = %#, error userinfo = %#", error!.code, error!.domain, error!.userInfo)
seal.reject(error!)
}
let prefix: String = "AppointmentFiles/\(appointmentId)"
container?.listBlobsSegmented(with: nil, prefix: prefix, useFlatBlobListing: true, blobListingDetails: AZSBlobListingDetails(), maxResults: 150) { (error : Error?, results : AZSBlobResultSegment?) -> Void in
if error != nil {
seal.reject(error!)
}
for blob in results!.blobs!
{
let blobInfo = blob as! AZSCloudBlob
if blobInfo.blobName.lowercased().contains("jpg") || blobInfo.blobName.lowercased().contains("jpeg") {
let blobDownload: BlobDownload = BlobDownload(appointmentId: Int(jobId), blob: blobInfo)
blobDownloads.append(blobDownload)
}
}
seal.fulfill(blobDownloads)
}
}
}
That returns the blobs as expected but I want to get all of the blobs for all of the appointments before proceeding. Here's what I tried (among other things):
func getBlobsForAllJobs(appointmentIds: [Int32]) -> Promise<[BlobDownload]> {
return Promise { seal in
let count = appointmentIds.count - 1
let promises = (0..<count).map { index -> Promise<[BlobDownload]> in
return getBlobsPromise(agencyCode: agencyCode, appointmentId: appointmentIds[index])
}
when(fulfilled: promises).then({ blobDownloads in
seal.fulfill(blobDownloads)
})
}
}
EDIT 1
I solved this using a DispatchGroup and completion handler. Here's the code in case someone is interested. If there are alternate (better) ways of doing this I'd love to hear them. I'm a c# guy just getting into Swift.
func getBlobsToDownload(appointmentIds: [Int32], completion: #escaping ([BlobDownload]) -> Void) {
var myBlobsToDownload = [BlobDownload]()
let myGroup = DispatchGroup()
for apptId in appointmentIds {
myGroup.enter()
getBlobs(appointmentId: apptId) { (blobDownloads) in
print("Finished request \(apptId)")
print("Blobs fetched from apptId \(apptId) is \(blobDownloads.count)")
for blobDownload in blobDownloads {
myBlobsToDownload.append(blobDownload)
}
myGroup.leave()
}
}
myGroup.notify(queue: .main) {
print("Finished all requests.")
completion(myBlobsToDownload)
}
}
So i'm a bit lost on how to implement a retry logic when my upload request fail.
Here is my code i would like some guidance on how to do it
func startUploading(failure failure: (NSError) -> Void, success: () -> Void, progress: (Double) -> Void) {
DDLogDebug("JogUploader: Creating jog: \(self.jog)")
API.sharedInstance.createJog(self.jog,
failure: { error in
failure(error)
}, success: {_ in
success()
})
}
Here's a general solution that can be applied to any async function that has no parameters, excepting the callbacks. I simplified the logic by having only success and failure callbacks, a progress should not be that hard to add.
So, assuming that your function is like this:
func startUploading(success: #escaping () -> Void, failure: #escaping (Error) -> Void) {
DDLogDebug("JogUploader: Creating jog: \(self.jog)")
API.sharedInstance.createJog(self.jog,
failure: { error in
failure(error)
}, success: {_ in
success()
})
}
A matching retry function might look like this:
func retry(times: Int, task: #escaping(#escaping () -> Void, #escaping (Error) -> Void) -> Void, success: #escaping () -> Void, failure: #escaping (Error) -> Void) {
task(success,
{ error in
// do we have retries left? if yes, call retry again
// if not, report error
if times > 0 {
retry(times - 1, task: task, success: success, failure: failure)
} else {
failure(error)
}
})
}
and can be called like this:
retry(times: 3, task: startUploading,
success: {
print("Succeeded")
},
failure: { err in
print("Failed: \(err)")
})
The above will retry the startUploading call three times if it keeps failing, otherwise will stop at the first success.
Edit. Functions that do have other params can be simply embedded in a closure:
func updateUsername(username: String, success: #escaping () -> Void, failure: #escaping (Error) -> Void) {
...
}
retry(times: 3, { success, failure in updateUsername(newUsername, success, failure) },
success: {
print("Updated username")
},
failure: {
print("Failed with error: \($0)")
}
)
Update So many #escaping clauses in the retry function declaration might decrease its readability, and increase the cognitive load when it comes to consuming the function. To improve this, we can write a simple generic struct that has the same functionality:
struct Retrier<T> {
let times: UInt
let task: (#escaping (T) -> Void, #escaping (Error) -> Void) -> Void
func callAsFunction(success: #escaping (T) -> Void, failure: #escaping (Error) -> Void) {
let failureWrapper: (Error) -> Void = { error in
// do we have retries left? if yes, call retry again
// if not, report error
if times > 0 {
Retrier(times: times - 1, task: task)(success: success, failure: failure)
} else {
failure(error)
}
}
task(success, failureWrapper)
}
func callAsFunction(success: #escaping () -> Void, failure: #escaping (Error) -> Void) where T == Void {
callAsFunction(success: { _ in }, failure: failure)
}
}
Being callable, the struct can be called like a regular function:
Retrier(times: 3, task: startUploading)(success: { print("success: \($0)") },
failure: { print("failure: \($0)") })
, or can be circulated through the app:
let retrier = Retrier(times: 3, task: startUploading)
// ...
// sometime later
retrier(success: { print("success: \($0)") },
failure: { print("failure: \($0)") })
Here is an updated answer for swift 3. I also added a generic object in the success block so if you make an object after your network call is complete you can pass it along to the final closure. Here is the retry function:
func retry<T>(_ attempts: Int, task: #escaping (_ success: #escaping (T) -> Void, _ failure: #escaping (Error) -> Void) -> Void, success: #escaping (T) -> Void, failure: #escaping (Error) -> Void) {
task({ (obj) in
success(obj)
}) { (error) in
print("Error retry left \(attempts)")
if attempts > 1 {
self.retry(attempts - 1, task: task, success: success, failure: failure)
} else {
failure(error)
}
}
}
And here is how you would use it if you updated a user and wanted to get back a new user object with the updated info:
NetworkManager.shared.retry(3, task: { updatedUser, failure in
NetworkManager.shared.updateUser(user, success: updatedUser, error: failure) }
, success: { (updatedUser) in
print(updatedUser.debugDescription)
}) { (err) in
print(err)
}
Updated to swift 5, with Result type instead of success and failure blocks.
func retry<T>(_ attempts: Int, task: #escaping (_ completion:#escaping (Result<T, Error>) -> Void) -> Void, completion:#escaping (Result<T, Error>) -> Void) {
task({ result in
switch result {
case .success(_):
completion(result)
case .failure(let error):
print("retries left \(attempts) and error = \(error)")
if attempts > 1 {
self.retry(attempts - 1, task: task, completion: completion)
} else {
completion(result)
}
}
})
}
This is how we can use the retry function:
func updateUser(userName: String) {
retry(3, task: { (result) in
startUploadingWithResult(userName: userName, completion: result)
}) { (newResult) in
switch newResult {
case .success(let str):
print("Success : \(str)")
case .failure(let error):
print(error)
}
}
}
updateUser(userName: "USER_NAME")