Low res image taking too long to load - swift

Using Facebook Graph API, I retrieved a string URL to a 200x200 profile picture that I want to display in a UIImageView. I'm successfully able to do this, but I notice that it can take as long as 10 seconds for the image to display on the screen. Can anyone give me some pointers (no pun intended) on how to optimize it?
override func viewDidAppear(animated: Bool) {
super.viewDidAppear(animated)
NSURLSession.sharedSession().dataTaskWithURL(NSURL(string: self.profilePictureUrl)!, completionHandler: { (data, response, error) ->
Void in
self.profilePictureImageView.image = UIImage(data: data!)
self.profilePictureImageView.layer.cornerRadius = self.profilePictureImageView.frame.size.width / 2;
self.profilePictureImageView.clipsToBounds = true
dispatch_async(dispatch_get_main_queue(), { () -> Void in
self.view.addSubview(self.profilePictureImageView)
})
}).resume()
}

You should move all UIView calls (so anything you set on the UIImageView) onto the main thread as UIKit for the most part isn't thread-safe. You can instantiate the UIImage on the background thread though for performance optimization, so try this:
override func viewDidAppear(animated: Bool) {
super.viewDidAppear(animated)
let url = NSURL(string: self.profilePictureUrl)!
NSURLSession.sharedSession().dataTaskWithURL(
url,
completionHandler: { [weak self] (data, response, error) -> Void in
guard let strongSelf = self else { return }
// create the UIImage on the background thread
let image = UIImage(data: data!)
// then jump to the main thread to modify your UIImageView
dispatch_async(dispatch_get_main_queue(), { [weak self] () -> Void in
guard let strongSelf = self else { return }
let profilePictureImageView = strongSelf.profilePictureImageView
profilePictureImageView.image = image
profilePictureImageView.layer.cornerRadius = profilePictureImageView.frame.size.width / 2;
profilePictureImageView.clipsToBounds = true
strongSelf.view.addSubview(profilePictureImageView)
})
}
).resume()
}
Note also that I've weak-ified your references to self. There is no guarantee the user hasn't dismissed the view controller that is initiating this code by the time the completion routines get called so you want to make sure you're not keeping a strong reference to self. This allows the view controller to deallocate if the user dismisses it and the completion routines then return early without doing any unnecessary work.

This code is illegal:
NSURLSession.sharedSession().dataTaskWithURL(NSURL(string: self.profilePictureUrl)!, completionHandler: { (data, response, error) ->
Void in
self.profilePictureImageView.image = UIImage(data: data!)
Stop! you are setting a UIImageView's image on a background thread. No no no. UIKit is not thread-safe. You must get onto the main thread to do this. (You do eventually get onto the main thread in your code, but you are doing it too late.)

Related

Swift: Apply CIFilter to video error - unfinished AVAsynchronousVideoCompositionRequest deallocated

I'm building a video editor that lets you apply a CIFilter to a video. And it works well.
The only problem I'm facing is that when I dismiss the ViewController I get this error:
Unfinished AVAsynchronousVideoCompositionRequest deallocated - should
have called finishWithComposedVideoFrame:, finishWithError: or
finishCancelledRequest
This error doesn't make the app crash or slower, but when I try to edit another video the preview in the AVPlayer becomes black.
This is my current code:
var mutableComposition = AVMutableVideoComposition()
let exposureFilter = CIFilter.exposureAdjust()
override func viewDidLoad() {
updateComposition()
}
func updateComposition() {
mutableComposition = AVMutableVideoComposition(asset: player.currentItem!.asset, applyingCIFiltersWithHandler: { [weak self] request in
guard let self = self else {
return
}
self.exposureFilter.inputImage = request.sourceImage.clampedToExtent()
self.exposureFilter.ev = 5
let output = self.exposureFilter.outputImage!.cropped(to: request.sourceImage.extent)
request.finish(with: output, context: nil)
})
player.currentItem?.videoComposition = mutableComposition
}
If I remove the [weak self] no error it's printed, but it keeps the ViewController in memory when I dismiss it, creating an unwanted memory leak.

Autolayout images inside cells in tableview. Correct layout but only once scrolling down and back up?

Im trying to get tableview cells with auto resizing images to work. Basically I want the image width in the cell to always be the same, and the height to change in accordance with the aspect ratio of the image.
I have created a cell class, which only has outlets for a label, imageView and a NSLayoutConstraint for the height of the image. I have some async methods to download an image and set it as the image for the cell imageView. Then the completion handle gets called and I run the following code to adjust the height constraint to the correct height:
cell.cellPhoto.loadImageFromURL(url: photos[indexPath.row].thumbnailURL, completion: {
// Set imageView height to the width
let imageSize = cell.cellPhoto.image?.size
let maxHeight = ((self.tableView.frame.width-30.0)*imageSize!.height) / imageSize!.width
cell.cellPhotoHeight.constant = maxHeight
cell.layoutIfNeeded()
})
return cell
And here is the UIImageView extension I wrote which loads images:
func loadImageFromURL(url: String, completion: #escaping () -> Void) {
let url = URL(string: url)
makeDataRequest(url: url!, completion: { data in
DispatchQueue.main.async {
self.image = UIImage(data: data!)
completion()
}
})
}
And the makeDataRequest function which it calls:
func makeDataRequest(url: URL, completion: #escaping (Data?) -> Void) {
let session = URLSession.shared
let task = session.dataTask(with: url, completionHandler: { data, response, error in
if error == nil {
let response = response as? HTTPURLResponse
switch response?.statusCode {
case 200:
completion(data)
case 404:
print("Invalid URL for request")
default:
print("Something else went wrong in the data request")
}
} else {
print(error?.localizedDescription ?? "Error")
}
})
task.resume()
}
This works for all the cells out of frame, but the imageviews in the cells in the frame are small. Only when I scroll down and then back up again do they correctly size. How do I fix this? I know other people have had this issue but trying their fixes did nothing.
I had to sorta recreate the problem to understand what was going on. Basically you need to reload the tableview. I would do this when a picture finishes downloading.
In the view controller that has the table view var. Add this to the viewDidLoad() function.
override func viewDidLoad() {
super.viewDidLoad()
tableView.delegate = self
tableView.dataSource = self
//Create a notification so we can update the list from anywhere in the app. Good if you are calling this from an other class.
NotificationCenter.default.addObserver(self, selector: #selector(loadList), name: NSNotification.Name(rawValue: "loadList"), object: nil)
}
//This function updates the cells in the table view
#objc func loadList(){
//load data here
self.tableView.reloadData()
}
Now, when the photo is done downloading, you can notify the viewcontroller to reload the table view by using the following,
func loadImageFromURL(url: String, completion: #escaping () -> Void) {
let url = URL(string: url)
makeDataRequest(url: url!, completion: { data in
DispatchQueue.main.async {
self.image = UIImage(data: data!)
completion()
//This isn't the best way to do this as, if you have 25+ pictures,
//the list will pretty much freeze up every time the list has to be reloaded.
//What you could do is have a flag to check if the first 'n' number of cells
//have been loaded, and if so then don't reload the tableview.
//Basically what I'm saying is, if the cells are off the screen who cares.
NotificationCenter.default.post(name: NSNotification.Name(rawValue: "loadList"), object: nil)
}
})
}
Heres something I did to have better Async, see below.
My code as follows, I didn't do the resizing ratio thing like you did but the same idea applies. It's how you go about reloading the table view. Also, I personally don't like writing my own download code, with status code and everything. It isn't fun, why reinvent the wheel when someone else has done it?
Podfile
pod 'SDWebImage', '~> 5.0'
mCell.swift
class mCell: UITableViewCell {
//This keeps track to see if the cell has been already resized. This is only needed once.
var flag = false
#IBOutlet weak var cellLabel: UILabel!
#IBOutlet weak var cell_IV: UIImageView!
override func awakeFromNib() { super.awakeFromNib() }
}
viewController.swift (Click to see full code)
I'm just going to give the highlights of the code here.
//Set the image based on a url
//Remember this is all done with Async...In the backgorund, on a custom thread.
mCell.cell_IV.sd_setImage(with: URL(string: ViewController.cell_pic_url[row])) { (image, error, cache, urls) in
// If failed to load image
if (error != nil) {
//Set to defult
mCell.cell_IV.image = UIImage(named: "redx.png")
}
//Else we got the image from the web.
else {
//Set the cell image to the one we downloaded
mCell.cell_IV.image = image
//This is a flag to reload the tableview once the image is done downloading. I set a var in the cell class, this is to make sure the this is ONLY CALLED once. Otherwise the app will get stuck in an infinite loop.
if (mCell.flag != true){
DispatchQueue.main.asyncAfter(deadline: .now() + 0.025){ //Nothing wrong with a little lag.
NotificationCenter.default.post(name: NSNotification.Name(rawValue: "loadList"), object: nil)
mCell.flag = true
}
}
}
}

Convert images to videos in iOS without locking UI Thread (Swift)

I am trying to convert image snapshots into a video but I am facing UI Thread problems: my view controller is locked. I would like to know how to handle this because I did a lot of research and tried to detach the process into different DispatchQueues but none of them worked. So, it explains why I am not using any Queue on the code below:
class ScreenRecorder {
func renderPhotosAsVideo(callback: #escaping(_ success: Bool, _ url: URL)->()) {
var frames = [UIImage]()
for _ in 0..<100 {
let image = self.containerView.takeScreenshot()
if let imageData = UIImageJPEGRepresentation(image, 0.7), let compressedImage = UIImage(data: imageData) {
frames.append(compressedImage)
}
}
self.generateVideoUrl(frames: frames, complete: { (fileURL: URL) in
self.saveVideo(url: fileURL, complete: { saved in
print("animation video save complete")
callback(saved, fileURL)
})
})
}
}
extension UIView {
func takeScreenshot() -> UIImage {
let renderer = UIGraphicsImageRenderer(size: self.bounds.size)
let image = renderer.image { _ in
self.drawHierarchy(in: self.bounds, afterScreenUpdates: true)
}
return image
}
}
class ViewController {
let recorder = ScreenRecorder()
recorder.renderPhotoAsVideo { success, url in
if (success) {
print("ok")
} else {
self.alert(title: "Erro", message: "Nao foi possivel salvar o video", block: nil)
}
}
}
PS: I used this tutorial as reference -> http://www.mikitamanko.com/blog/2017/05/21/swift-how-to-record-a-screen-video-or-convert-images-to-videos/
It really looks like this is not possible, at least not the way you are trying to do it.
There are quite a few ways to render a UIViews content into an image, but all of them must be used from the main thread only. This applies also to the drawInHierarchy method you are using.
As you have to call it on the main thread and the method is just getting called so many times, I think this will never work in a performant way.
See profiling in Instruments:
Sources:
How to render view into image faster?
Safe way to render UIVIew to an image on background thread?

downloading and caching images from url asynchronously

I'm trying to download images from my firebase database and load them into collectionviewcells. The images download, however I am having trouble having them all download and load asynchronously.
Currently when I run my code the last image downloaded loads. However, if I update my database the collection view updates and the new last user profile image also loads in but the remainder are missing.
I'd prefer to not use a 3rd party library so any resources or suggestions would be greatly appreciated.
Here's the code that handles the downloading:
func loadImageUsingCacheWithUrlString(_ urlString: String) {
self.image = nil
// checks cache
if let cachedImage = imageCache.object(forKey: urlString as NSString) as? UIImage {
self.image = cachedImage
return
}
//download
let url = URL(string: urlString)
URLSession.shared.dataTask(with: url!, completionHandler: { (data, response, error) in
//error handling
if let error = error {
print(error)
return
}
DispatchQueue.main.async(execute: {
if let downloadedImage = UIImage(data: data!) {
imageCache.setObject(downloadedImage, forKey: urlString as NSString)
self.image = downloadedImage
}
})
}).resume()
}
I believe the solution lies somewhere in reloading the collectionview I just don't know where exactly to do it.
Any suggestions?
EDIT:
Here is where the function is being called; my cellForItem at indexpath
override func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
let cell = collectionView.dequeueReusableCell(withReuseIdentifier: userResultCellId, for: indexPath) as! FriendCell
let user = users[indexPath.row]
cell.nameLabel.text = user.name
if let profileImageUrl = user.profileImageUrl {
cell.profileImage.loadImageUsingCacheWithUrlString(profileImageUrl)
}
return cell
}
The only other thing that I believe could possibly affect the images loading is this function I use to download the user data, which is called in viewDidLoad, however all the other data downloads correctly.
func fetchUser(){
Database.database().reference().child("users").observe(.childAdded, with: {(snapshot) in
if let dictionary = snapshot.value as? [String: AnyObject] {
let user = User()
user.setValuesForKeys(dictionary)
self.users.append(user)
print(self.users.count)
DispatchQueue.main.async(execute: {
self.collectionView?.reloadData()
})
}
}, withCancel: nil)
}
Current Behavior:
As for the current behavior the last cell is the only cell that displays the downloaded profile image; if there are 5 cells, the 5th is the only one that displays a profile image. Also when I update the database, ie register a new user into it, the collectionview updates and displays the newly registered user correctly with their profile image in addition to the old last cell that downloaded it's image properly. The rest however, remain without profile images.
I know you found your problem and it was unrelated to the above code, yet I still have an observation. Specifically, your asynchronous requests will carry on, even if the cell (and therefore the image view) have been subsequently reused for another index path. This results in two problems:
If you quickly scroll to the 100th row, you are going to have to wait for the images for the first 99 rows to be retrieved before you see the images for the visible cells. This can result in really long delays before images start popping in.
If that cell for the 100th row was reused several times (e.g. for row 0, for row 9, for row 18, etc.), you may see the image appear to flicker from one image to the next until you get to the image retrieval for the 100th row.
Now, you might not immediately notice either of these are problems because they will only manifest themselves when the image retrieval has a hard time keeping up with the user's scrolling (the combination of slow network and fast scrolling). As an aside, you should always test your app using the network link conditioner, which can simulate poor connections, which makes it easier to manifest these bugs.
Anyway, the solution is to keep track of (a) the current URLSessionTask associated with the last request; and (b) the current URL being requested. You can then (a) when starting a new request, make sure to cancel any prior request; and (b) when updating the image view, make sure the URL associated with the image matches what the current URL is.
The trick, though, is when writing an extension, you cannot just add new stored properties. So you have to use the associated object API to associate these two new stored values with the UIImageView object. I personally wrap this associated value API with a computed property, so that the code for retrieving the images does not get too buried with this sort of stuff. Anyway, that yields:
extension UIImageView {
private static var taskKey = 0
private static var urlKey = 0
private var currentTask: URLSessionTask? {
get { objc_getAssociatedObject(self, &UIImageView.taskKey) as? URLSessionTask }
set { objc_setAssociatedObject(self, &UIImageView.taskKey, newValue, .OBJC_ASSOCIATION_RETAIN_NONATOMIC) }
}
private var currentURL: URL? {
get { objc_getAssociatedObject(self, &UIImageView.urlKey) as? URL }
set { objc_setAssociatedObject(self, &UIImageView.urlKey, newValue, .OBJC_ASSOCIATION_RETAIN_NONATOMIC) }
}
func loadImageAsync(with urlString: String?, placeholder: UIImage? = nil) {
// cancel prior task, if any
weak var oldTask = currentTask
currentTask = nil
oldTask?.cancel()
// reset image view’s image
self.image = placeholder
// allow supplying of `nil` to remove old image and then return immediately
guard let urlString = urlString else { return }
// check cache
if let cachedImage = ImageCache.shared.image(forKey: urlString) {
self.image = cachedImage
return
}
// download
let url = URL(string: urlString)!
currentURL = url
let task = URLSession.shared.dataTask(with: url) { [weak self] data, response, error in
self?.currentTask = nil
// error handling
if let error = error {
// don't bother reporting cancelation errors
if (error as? URLError)?.code == .cancelled {
return
}
print(error)
return
}
guard let data = data, let downloadedImage = UIImage(data: data) else {
print("unable to extract image")
return
}
ImageCache.shared.save(image: downloadedImage, forKey: urlString)
if url == self?.currentURL {
DispatchQueue.main.async {
self?.image = downloadedImage
}
}
}
// save and start new task
currentTask = task
task.resume()
}
}
Also, note that you were referencing some imageCache variable (a global?). I would suggest an image cache singleton, which, in addition to offering the basic caching mechanism, also observes memory warnings and purges itself in memory pressure situations:
class ImageCache {
private let cache = NSCache<NSString, UIImage>()
private var observer: NSObjectProtocol?
static let shared = ImageCache()
private init() {
// make sure to purge cache on memory pressure
observer = NotificationCenter.default.addObserver(
forName: UIApplication.didReceiveMemoryWarningNotification,
object: nil,
queue: nil
) { [weak self] notification in
self?.cache.removeAllObjects()
}
}
deinit {
NotificationCenter.default.removeObserver(observer!)
}
func image(forKey key: String) -> UIImage? {
return cache.object(forKey: key as NSString)
}
func save(image: UIImage, forKey key: String) {
cache.setObject(image, forKey: key as NSString)
}
}
A bigger, more architectural, observation: One really should decouple the image retrieval from the image view. Imagine you have a table where you have a dozen cells using the same image. Do you really want to retrieve the same image a dozen times just because the second image view scrolled into view before the first one finished its retrieval? No.
Also, what if you wanted to retrieve the image outside of the context of an image view? Perhaps a button? Or perhaps for some other reason, such as to download images to store in the user’s photos library. There are tons of possible image interactions above and beyond image views.
Bottom line, fetching images is not a method of an image view, but rather a generalized mechanism of which an image view would like to avail itself. An asynchronous image retrieval/caching mechanism should generally be incorporated in a separate “image manager” object. It can then detect redundant requests and be used from contexts other than an image view.
As you can see, the asynchronous retrieval and caching is starting to get a little more complicated, and this is why we generally advise considering established asynchronous image retrieval mechanisms like AlamofireImage or Kingfisher or SDWebImage. These guys have spent a lot of time tackling the above issues, and others, and are reasonably robust. But if you are going to “roll your own,” I would suggest something like the above at a bare minimum.

Setting GIF image blockes UI when updating performed in background or main thread

I'm setting a GIF image within a UIImageView using an objective-c library FLAnimatedImage. I'm currently facing an issue where the UI is being blocked while the GIF image is being loaded (touches not registered on overlay UIView which is presented in front of the image, overlay view is hidden/shown on tap). I've tried updating the image on a background thread and on a main thread, results remain the same. Is there something I'm doing wrong here? Or any alternatives? (PS. I also have a UITextView with detects links, doubt that's important)
UITableViewCell
//Main Thread blocks UI
dispatch_async(dispatch_get_main_queue(), { () -> Void in
cell.shotImg.setShotImage(self.shots[self.currentIndex])
})
//Background thread also blocks ui. See code below for background thread function
backgroundThread(0.1, completion: {
cell.shotImg.setShotImage(self.shots[self.currentIndex])
})
FLAnimatedImage Extension
extension FLAnimatedImageView{
func setRegularImage(url: String){
self.af_setImageWithURL(NSURL(string: url)!)
}
func setGIFImage(url: String){
let animatedImage = FLAnimatedImage(animatedGIFData: NSData(contentsOfURL: NSURL(string: url)!)!)
self.animatedImage = animatedImage
}
func setShotImage(shot: Shots){
var url: String!
if(shot.images.hidpi != nil){
url = shot.images.hidpi
}else{
url = shot.images.normal
}
if(shot.animated!){
self.setGIFImage(url)
}else{
self.setRegularImage(shot.images.normal)
}
}
}
Background thread
func backgroundThread(delay: Double = 0.0, background: (() -> Void)? = nil, completion: (() -> Void)? = nil) {
dispatch_async(dispatch_get_global_queue(Int(QOS_CLASS_USER_INITIATED.rawValue), 0)) {
if(background != nil){ background!(); }
let popTime = dispatch_time(DISPATCH_TIME_NOW, Int64(delay * Double(NSEC_PER_SEC)))
dispatch_after(popTime, dispatch_get_main_queue()) {
if(completion != nil){ completion!(); }
}
}
}
My guess is that the problem is this:
NSData(contentsOfURL: NSURL(string: url))
That's not how you download something (if that's what you're doing). Use NSURLSession.
Unless you have static cells, you cannot update the cell directly in a background thread as you are doing. While you scroll, the table view will reuse cells off screen for those newly appearing. By the time the background task finishes, the cell is likely to belong to another row.
Instead you should store the image in an array when the task finishes, and then call reloadData (on the main thread). Then cellForRowAtIndexPath should display the image from the array, or fetch it in the background if it hasn't been fetched yet (which will trigger the reloadData to display it when finished).