I am trying to convert image snapshots into a video but I am facing UI Thread problems: my view controller is locked. I would like to know how to handle this because I did a lot of research and tried to detach the process into different DispatchQueues but none of them worked. So, it explains why I am not using any Queue on the code below:
class ScreenRecorder {
func renderPhotosAsVideo(callback: #escaping(_ success: Bool, _ url: URL)->()) {
var frames = [UIImage]()
for _ in 0..<100 {
let image = self.containerView.takeScreenshot()
if let imageData = UIImageJPEGRepresentation(image, 0.7), let compressedImage = UIImage(data: imageData) {
frames.append(compressedImage)
}
}
self.generateVideoUrl(frames: frames, complete: { (fileURL: URL) in
self.saveVideo(url: fileURL, complete: { saved in
print("animation video save complete")
callback(saved, fileURL)
})
})
}
}
extension UIView {
func takeScreenshot() -> UIImage {
let renderer = UIGraphicsImageRenderer(size: self.bounds.size)
let image = renderer.image { _ in
self.drawHierarchy(in: self.bounds, afterScreenUpdates: true)
}
return image
}
}
class ViewController {
let recorder = ScreenRecorder()
recorder.renderPhotoAsVideo { success, url in
if (success) {
print("ok")
} else {
self.alert(title: "Erro", message: "Nao foi possivel salvar o video", block: nil)
}
}
}
PS: I used this tutorial as reference -> http://www.mikitamanko.com/blog/2017/05/21/swift-how-to-record-a-screen-video-or-convert-images-to-videos/
It really looks like this is not possible, at least not the way you are trying to do it.
There are quite a few ways to render a UIViews content into an image, but all of them must be used from the main thread only. This applies also to the drawInHierarchy method you are using.
As you have to call it on the main thread and the method is just getting called so many times, I think this will never work in a performant way.
See profiling in Instruments:
Sources:
How to render view into image faster?
Safe way to render UIVIew to an image on background thread?
Related
I'm struggling to download pictures from firebase storage to an UIImage in swift 5.
I can well upload them. When I tried to retrieve picture, the UIImage display a black screen.
here my function which return the UIImage
import UIKit
import Firebase
func getImageEvent (imagePath : String) -> UIImage? {
var myImage : UIImageView?
//Access to the storage
let storageRef = Storage.storage().reference(withPath: imagePath)
storageRef.getData(maxSize: 1 * 1024 * 1024) {(data, error) in
if let error = error {
print(error.localizedDescription)
return
}
if let data = data {
print(data.description)
myImage?.image = UIImage(data: data)
}
}
return myImage?.image
}
//Call the function
getImageEvent (imagePath :"9U4BoXgBgTTgbbJCz0zy/eventMainImage.jpg")
In the console, I can well see a value for print(data.description).
By default, there is an image in the UIImageView. When call the function, the default image is replaced by a black screen.
Could you please help me to understand the mistake ?
many thanks
There's a number of ways to go about this but a brief description of the issue first:
The return statement within the closure will execute way before that image is downloaded - Firebase functions are asynchronous and code has to be crafted in a way that allows for time to download and get data from the internet. So - don't try to return data from asynchronous functions.
Here's the code re-written with a completion handler. That handler will be called only after the image is fully downloaded.
func getImageEvent (imagePath: String, completion: #escaping(UIImage) -> Void) {
var myImage : UIImageView?
let storageRef = Storage.storage().reference(withPath: imagePath)
storageRef.getData(maxSize: 1 * 1024 * 1024) { data, error in
if let error = error {
print(error.localizedDescription)
return
}
if let data = data {
if let myImage = UIImage(data: data) {
completion(myImage)
}
}
}
}
and the key is how to call that function. Note this code awaits the data (UIImage) to be passed back to it within it's closure and lets you know that getting the image was complete.
self.getImageEvent(imagePath: "9U4BoXgBgTTgbbJCz0zy/eventMainImage.jpg", completion: { theImage in
print("got the image!")
})
You should add additional error checking in case the image was not downloaded or myImage was nil. Passing back an error message along with the nil myImage would be one option, or making the object passed back as an optional and then checking for nil within self.downloadImageAtPath would be another.
To complete the solution, below the code used to in tableView to get picture in a particular cell
getImageEvent(imagePath: myArray[indexPath.row].partyImagePath) { (image) in
cell.partyImage.image = image
}
I'm trying to download images from my firebase database and load them into collectionviewcells. The images download, however I am having trouble having them all download and load asynchronously.
Currently when I run my code the last image downloaded loads. However, if I update my database the collection view updates and the new last user profile image also loads in but the remainder are missing.
I'd prefer to not use a 3rd party library so any resources or suggestions would be greatly appreciated.
Here's the code that handles the downloading:
func loadImageUsingCacheWithUrlString(_ urlString: String) {
self.image = nil
// checks cache
if let cachedImage = imageCache.object(forKey: urlString as NSString) as? UIImage {
self.image = cachedImage
return
}
//download
let url = URL(string: urlString)
URLSession.shared.dataTask(with: url!, completionHandler: { (data, response, error) in
//error handling
if let error = error {
print(error)
return
}
DispatchQueue.main.async(execute: {
if let downloadedImage = UIImage(data: data!) {
imageCache.setObject(downloadedImage, forKey: urlString as NSString)
self.image = downloadedImage
}
})
}).resume()
}
I believe the solution lies somewhere in reloading the collectionview I just don't know where exactly to do it.
Any suggestions?
EDIT:
Here is where the function is being called; my cellForItem at indexpath
override func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
let cell = collectionView.dequeueReusableCell(withReuseIdentifier: userResultCellId, for: indexPath) as! FriendCell
let user = users[indexPath.row]
cell.nameLabel.text = user.name
if let profileImageUrl = user.profileImageUrl {
cell.profileImage.loadImageUsingCacheWithUrlString(profileImageUrl)
}
return cell
}
The only other thing that I believe could possibly affect the images loading is this function I use to download the user data, which is called in viewDidLoad, however all the other data downloads correctly.
func fetchUser(){
Database.database().reference().child("users").observe(.childAdded, with: {(snapshot) in
if let dictionary = snapshot.value as? [String: AnyObject] {
let user = User()
user.setValuesForKeys(dictionary)
self.users.append(user)
print(self.users.count)
DispatchQueue.main.async(execute: {
self.collectionView?.reloadData()
})
}
}, withCancel: nil)
}
Current Behavior:
As for the current behavior the last cell is the only cell that displays the downloaded profile image; if there are 5 cells, the 5th is the only one that displays a profile image. Also when I update the database, ie register a new user into it, the collectionview updates and displays the newly registered user correctly with their profile image in addition to the old last cell that downloaded it's image properly. The rest however, remain without profile images.
I know you found your problem and it was unrelated to the above code, yet I still have an observation. Specifically, your asynchronous requests will carry on, even if the cell (and therefore the image view) have been subsequently reused for another index path. This results in two problems:
If you quickly scroll to the 100th row, you are going to have to wait for the images for the first 99 rows to be retrieved before you see the images for the visible cells. This can result in really long delays before images start popping in.
If that cell for the 100th row was reused several times (e.g. for row 0, for row 9, for row 18, etc.), you may see the image appear to flicker from one image to the next until you get to the image retrieval for the 100th row.
Now, you might not immediately notice either of these are problems because they will only manifest themselves when the image retrieval has a hard time keeping up with the user's scrolling (the combination of slow network and fast scrolling). As an aside, you should always test your app using the network link conditioner, which can simulate poor connections, which makes it easier to manifest these bugs.
Anyway, the solution is to keep track of (a) the current URLSessionTask associated with the last request; and (b) the current URL being requested. You can then (a) when starting a new request, make sure to cancel any prior request; and (b) when updating the image view, make sure the URL associated with the image matches what the current URL is.
The trick, though, is when writing an extension, you cannot just add new stored properties. So you have to use the associated object API to associate these two new stored values with the UIImageView object. I personally wrap this associated value API with a computed property, so that the code for retrieving the images does not get too buried with this sort of stuff. Anyway, that yields:
extension UIImageView {
private static var taskKey = 0
private static var urlKey = 0
private var currentTask: URLSessionTask? {
get { objc_getAssociatedObject(self, &UIImageView.taskKey) as? URLSessionTask }
set { objc_setAssociatedObject(self, &UIImageView.taskKey, newValue, .OBJC_ASSOCIATION_RETAIN_NONATOMIC) }
}
private var currentURL: URL? {
get { objc_getAssociatedObject(self, &UIImageView.urlKey) as? URL }
set { objc_setAssociatedObject(self, &UIImageView.urlKey, newValue, .OBJC_ASSOCIATION_RETAIN_NONATOMIC) }
}
func loadImageAsync(with urlString: String?, placeholder: UIImage? = nil) {
// cancel prior task, if any
weak var oldTask = currentTask
currentTask = nil
oldTask?.cancel()
// reset image view’s image
self.image = placeholder
// allow supplying of `nil` to remove old image and then return immediately
guard let urlString = urlString else { return }
// check cache
if let cachedImage = ImageCache.shared.image(forKey: urlString) {
self.image = cachedImage
return
}
// download
let url = URL(string: urlString)!
currentURL = url
let task = URLSession.shared.dataTask(with: url) { [weak self] data, response, error in
self?.currentTask = nil
// error handling
if let error = error {
// don't bother reporting cancelation errors
if (error as? URLError)?.code == .cancelled {
return
}
print(error)
return
}
guard let data = data, let downloadedImage = UIImage(data: data) else {
print("unable to extract image")
return
}
ImageCache.shared.save(image: downloadedImage, forKey: urlString)
if url == self?.currentURL {
DispatchQueue.main.async {
self?.image = downloadedImage
}
}
}
// save and start new task
currentTask = task
task.resume()
}
}
Also, note that you were referencing some imageCache variable (a global?). I would suggest an image cache singleton, which, in addition to offering the basic caching mechanism, also observes memory warnings and purges itself in memory pressure situations:
class ImageCache {
private let cache = NSCache<NSString, UIImage>()
private var observer: NSObjectProtocol?
static let shared = ImageCache()
private init() {
// make sure to purge cache on memory pressure
observer = NotificationCenter.default.addObserver(
forName: UIApplication.didReceiveMemoryWarningNotification,
object: nil,
queue: nil
) { [weak self] notification in
self?.cache.removeAllObjects()
}
}
deinit {
NotificationCenter.default.removeObserver(observer!)
}
func image(forKey key: String) -> UIImage? {
return cache.object(forKey: key as NSString)
}
func save(image: UIImage, forKey key: String) {
cache.setObject(image, forKey: key as NSString)
}
}
A bigger, more architectural, observation: One really should decouple the image retrieval from the image view. Imagine you have a table where you have a dozen cells using the same image. Do you really want to retrieve the same image a dozen times just because the second image view scrolled into view before the first one finished its retrieval? No.
Also, what if you wanted to retrieve the image outside of the context of an image view? Perhaps a button? Or perhaps for some other reason, such as to download images to store in the user’s photos library. There are tons of possible image interactions above and beyond image views.
Bottom line, fetching images is not a method of an image view, but rather a generalized mechanism of which an image view would like to avail itself. An asynchronous image retrieval/caching mechanism should generally be incorporated in a separate “image manager” object. It can then detect redundant requests and be used from contexts other than an image view.
As you can see, the asynchronous retrieval and caching is starting to get a little more complicated, and this is why we generally advise considering established asynchronous image retrieval mechanisms like AlamofireImage or Kingfisher or SDWebImage. These guys have spent a lot of time tackling the above issues, and others, and are reasonably robust. But if you are going to “roll your own,” I would suggest something like the above at a bare minimum.
Using Facebook Graph API, I retrieved a string URL to a 200x200 profile picture that I want to display in a UIImageView. I'm successfully able to do this, but I notice that it can take as long as 10 seconds for the image to display on the screen. Can anyone give me some pointers (no pun intended) on how to optimize it?
override func viewDidAppear(animated: Bool) {
super.viewDidAppear(animated)
NSURLSession.sharedSession().dataTaskWithURL(NSURL(string: self.profilePictureUrl)!, completionHandler: { (data, response, error) ->
Void in
self.profilePictureImageView.image = UIImage(data: data!)
self.profilePictureImageView.layer.cornerRadius = self.profilePictureImageView.frame.size.width / 2;
self.profilePictureImageView.clipsToBounds = true
dispatch_async(dispatch_get_main_queue(), { () -> Void in
self.view.addSubview(self.profilePictureImageView)
})
}).resume()
}
You should move all UIView calls (so anything you set on the UIImageView) onto the main thread as UIKit for the most part isn't thread-safe. You can instantiate the UIImage on the background thread though for performance optimization, so try this:
override func viewDidAppear(animated: Bool) {
super.viewDidAppear(animated)
let url = NSURL(string: self.profilePictureUrl)!
NSURLSession.sharedSession().dataTaskWithURL(
url,
completionHandler: { [weak self] (data, response, error) -> Void in
guard let strongSelf = self else { return }
// create the UIImage on the background thread
let image = UIImage(data: data!)
// then jump to the main thread to modify your UIImageView
dispatch_async(dispatch_get_main_queue(), { [weak self] () -> Void in
guard let strongSelf = self else { return }
let profilePictureImageView = strongSelf.profilePictureImageView
profilePictureImageView.image = image
profilePictureImageView.layer.cornerRadius = profilePictureImageView.frame.size.width / 2;
profilePictureImageView.clipsToBounds = true
strongSelf.view.addSubview(profilePictureImageView)
})
}
).resume()
}
Note also that I've weak-ified your references to self. There is no guarantee the user hasn't dismissed the view controller that is initiating this code by the time the completion routines get called so you want to make sure you're not keeping a strong reference to self. This allows the view controller to deallocate if the user dismisses it and the completion routines then return early without doing any unnecessary work.
This code is illegal:
NSURLSession.sharedSession().dataTaskWithURL(NSURL(string: self.profilePictureUrl)!, completionHandler: { (data, response, error) ->
Void in
self.profilePictureImageView.image = UIImage(data: data!)
Stop! you are setting a UIImageView's image on a background thread. No no no. UIKit is not thread-safe. You must get onto the main thread to do this. (You do eventually get onto the main thread in your code, but you are doing it too late.)
I fetch a lot of images from the web, and they are all kind of sizes - they can be big, small etc..
So I can resize them when I display them in the cell but this is inefficient. It's way better to resize them after SDWebImage have download them and cache them resized, instead of storing large images on disk and resize them for every cell.
So how can I do this with SDWebImage, or I have to hack a bit onto the class?
SDWebImage developer Olivier Poitrey answered this question for me here.
You have to implement the SDWebImageManagerDelegate protocol and then set it as the shared manager's delegate like this:
SDWebImageManager.sharedManager.delegate = self;
using the imageManager:transformDownloadedImage:withURL: instance method.
More information.
Worked perfectly for me.
I had the same problem as you, and tried tweaking SDWebImage first, but ended up building my own component that solved the problem. You can take take a look at it here : https://github.com/adig/RemoteImageView
SDWebImage 3.8.2
If using UIImageView category sd_setImageWithURL. I have created another UIImageView category (extension)
func modifiedImageFromUrl(url: NSURL?) {
self.sd_setImageWithURL(url) { (image, error, cacheType, url) in
if cacheType == SDImageCacheType.None && image != nil {
dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)) {
let modifiedImage = // modify image as you want
dispatch_async(dispatch_get_main_queue()) {
SDWebImageManager.sharedManager().saveImageToCache(modifiedImage, forURL: url)
self.image = modifiedImage
}
}
}
}
}
Expansion on MaeSTRo's answer in Swift 3:
myImageView.sd_setImage(with: imageUrl){ (image, error, cacheType, url) in
guard let image = image, cacheType == .none else { return }
DispatchQueue.global(qos: .userInitiated).async {
let modifiedImage = myImageProcessor(image)
SDWebImageManager.shared().saveImage(toCache: modifiedImage, for: imageUrl)
DispatchQueue.main.async {
myImageView.image = modifiedImage
myImageView.setNeedsDisplay()
}
}
}
I'm setting a GIF image within a UIImageView using an objective-c library FLAnimatedImage. I'm currently facing an issue where the UI is being blocked while the GIF image is being loaded (touches not registered on overlay UIView which is presented in front of the image, overlay view is hidden/shown on tap). I've tried updating the image on a background thread and on a main thread, results remain the same. Is there something I'm doing wrong here? Or any alternatives? (PS. I also have a UITextView with detects links, doubt that's important)
UITableViewCell
//Main Thread blocks UI
dispatch_async(dispatch_get_main_queue(), { () -> Void in
cell.shotImg.setShotImage(self.shots[self.currentIndex])
})
//Background thread also blocks ui. See code below for background thread function
backgroundThread(0.1, completion: {
cell.shotImg.setShotImage(self.shots[self.currentIndex])
})
FLAnimatedImage Extension
extension FLAnimatedImageView{
func setRegularImage(url: String){
self.af_setImageWithURL(NSURL(string: url)!)
}
func setGIFImage(url: String){
let animatedImage = FLAnimatedImage(animatedGIFData: NSData(contentsOfURL: NSURL(string: url)!)!)
self.animatedImage = animatedImage
}
func setShotImage(shot: Shots){
var url: String!
if(shot.images.hidpi != nil){
url = shot.images.hidpi
}else{
url = shot.images.normal
}
if(shot.animated!){
self.setGIFImage(url)
}else{
self.setRegularImage(shot.images.normal)
}
}
}
Background thread
func backgroundThread(delay: Double = 0.0, background: (() -> Void)? = nil, completion: (() -> Void)? = nil) {
dispatch_async(dispatch_get_global_queue(Int(QOS_CLASS_USER_INITIATED.rawValue), 0)) {
if(background != nil){ background!(); }
let popTime = dispatch_time(DISPATCH_TIME_NOW, Int64(delay * Double(NSEC_PER_SEC)))
dispatch_after(popTime, dispatch_get_main_queue()) {
if(completion != nil){ completion!(); }
}
}
}
My guess is that the problem is this:
NSData(contentsOfURL: NSURL(string: url))
That's not how you download something (if that's what you're doing). Use NSURLSession.
Unless you have static cells, you cannot update the cell directly in a background thread as you are doing. While you scroll, the table view will reuse cells off screen for those newly appearing. By the time the background task finishes, the cell is likely to belong to another row.
Instead you should store the image in an array when the task finishes, and then call reloadData (on the main thread). Then cellForRowAtIndexPath should display the image from the array, or fetch it in the background if it hasn't been fetched yet (which will trigger the reloadData to display it when finished).