I have a grid of images that are loaded from the web, but there's a bit of lag/stutter while scrolling. I'm using an asynchronous ASIHTTPRequest to make the requests, so the download itself is happening in a separate thread, but because UIKit isn't thread-safe, once I receive the NSData response, I have to call UIImage initWithData on the main thread.
Profiling shows that, by far, the bottleneck consists of the internal PNG parsing functions invoked by UIImage initWithData. I'm interested in doing this in a background thread, so the main UI remains responsive and there's less lag.
But I'm not sure exactly how to do this. It sounds like the right direction is to use CGImageRef, since Core Graphics is thread-safe, but I only see CGImageCreateWithPNGDataProvider and CGImageCreateWithJPEGGDataProvider, whereas UIImage initWithData supports a large list of image types.
I want something that has the same functionality as UIImage initWithData but doesn't have the thread-safety issues.
You can call UIImage initWithData in a background thread safely. As a good rule of thumb, what you can't do in a background thread is alter user interface elements. In this case, you should not be setting the image property of a UIImageView, or adding a UIImageView to a superview in a background thread.
However, creating a UIImage instance is safe, and will work with no problems.
Related
I'm using a UIImagePickerController to capture a still image. I then need to do some processing work before saving different copies of the image to a Core Data store. The processing and saving work can take up to 4-8 seconds on an iPhone 4 so I'm trying to branch the work off to a background queue so the whole app and UI doesn't block.
At the root of my question is this. Is it possible to use a UIImage in a background thread so long as the UIImage object is totally confined to that thread? I found the following in apple's thread safety summary about NSImage. I'm assuming UIImage would work the same way.
NSImage Restrictions:
One thread can create an NSImage object, draw to the image buffer, and pass it off to the main thread for drawing. The underlying image cache is shared among all threads. For more information about images and how caching works, see Cocoa Drawing Guide.
Can anyone confirm this or is it just plain wrong to touch something like a UIImage object outside of the main thread. If using a confined UIImage instance is okay, then I that leads to another issue. The UIImagePickerController returns an NSDictionary which is thread safe, but within that NSDictionary is a UIImage object. Is it safe to pass that dictionary off to another thread then use the contained object to within that thread?
If the UIImage in the imagePicker info dictionary is not safe to use then any suggestions on how best to proceed?
I think I have the actual Core Data threading issues figured out. But for information I currently write and retrieve the image data using an NSValueTransformer to transform a UIImage to and from NSData within a custom NSManagedObject subclass.
I worked on some "Process an image after UIImagePickerController" code in a background thread just as you are and had no problems. Below are the steps I had success with.
Image Picker Delegate returns user image.
Assign image to the background thread class (In my case it was a #property retain'ed IVAR)
close/dismiss Image Picker
Background processing continues and creates a new UIImage after manipulating the original
new UIImage is returned to appropriate VC
Background process ends.
Does someone know of a way, or has a creative idea as to how to obtain an UIImage that is rendered from a UIWebView? The catch is, that is must be on a background thread.
I'll elaborate:
I'm trying to obtain the image from multiple UIWebViews every second+-, and display it on screen (iPhone's of course). Because rendering a layer to a CGContext is a CPU consuming job, I wouldn't like to use the main thread, so not to hang the UI.
My attempts so far were:
I tried to use renderInContext on the webView's layer to the UIGraphicalContext, but the _WebTryThreadLock error crashed the webviews.
I tried creating a CGBitmapContext, and render the webview's layer to it, but got the same result.
I tried implementing a copy method (by adding a Category) to CALayer, that deep copied all of the public properties, and sublayers. Afterwards, I tried to renderInContext the layer I copied. I got a UIImage that was partially "correct" - meaning, not all of the layers were rendered, so for example, i would get only the website header (or footer, or body, or search bar, or just a some of the frames). The UIWebview's layer consists of all sort of subclassed CALayers, so this is probably why this approached didn't work.
I tried setting the kCATransactionDisableActions in a CATransaction, but it didn't appear to change this behavior (neither of them).
I'm pretty close to giving up.
Is there a savior among you people?
UIWebView hates, and I mean really hates, having anything done to it on a background thread. UIKit is not fully thread safe. Drawing to a graphics context is (this was added in iOS 4), but not creating UIViews on a secondary thread.
Are you creating your UIWebViews off the main thread? Do you perhaps have some code to share? I would suspect your issues are being caused by the fact you're trying to perform operations to a UIWebView on a secondary thread. The drawing operation to render the view's contents as an image can happen off the main thread, but creating the view itself can't.
I've been working on this, too.
In a thread I create a view hierarchy, loop until the web-view has finished loading content.
The webview I create is inside of a UIViewController's viewdidload-- I've tried doing
if ([NSThread isMainThread] == NO) {[self performselectorOnMainThread: #selector(viewDidLoad)return;)}
And I've done the same for dealloc'ing the webView.
But that didn't work.. I've only found that we avoid UIWebView exceptions UNTIL we hit the autorelease pool...
I'm using instruments to figure out why.
Here's my attack strategy...
I'm going to perform the render operation on the main thread with an off-screen view, having a separate thread running some sort of queue to manage them. I'm worried about UI lag, so it'll have to be fairly efficient.
I'm building a UITableView similar to iPod.app's album browsing view:
I'm importing all the artists and album artworks from the iPod library on first launch. Saving everything to CoreData and getting it back into an NSFetchedResultsController. I'm reusing cell identifiers and in my cellForRowAtIndexPath: method I have this code:
Artist *artist = [fetchedResultsController objectAtIndexPath:indexPath];
NSString *identifier = #"bigCell";
SWArtistViewCell *cell = (SWArtistViewCell*)[tableView dequeueReusableCellWithIdentifier:identifier];
if (cell == nil)
cell = [[[SWArtistViewCell alloc] initWithStyle:UITableViewCellStyleDefault reuseIdentifier:identifier] autorelease];
cell.artistName = artist.artist_name;
cell.artworkImage = [UIImage imageWithData:artist.image];
[cell setNeedsDisplay];
return cell;
My SWArtistViewCell cell implements the drawRect: method to draw both the string and image:
[artworkImage drawInRect:CGRectMake(0,1,44,44)]
[artistName drawAtPoint:CGPointMake(54, 13) forWidth:200 withFont:[UIFont boldSystemFontOfSize:20] lineBreakMode:UILineBreakModeClip];
Scrolling is still choppy and I just can't figure out why. Apps like iPod and Twitter have butter smooth scrolling and yet they both draw some small image in the cell as I do.
All my views are opaque. What am I missing?
EDIT: here's what Shark says:
I'm not familiar with Shark. Any pointer as of what are these symbols related to? When I look at the trace of these, they all point to my drawRect: method, specifically the UIImage drawing.
Would it point to something else if the chokehold was the file reading? Is it definitely the drawing?
EDIT: retaining the image
I've done as pothibo suggested and added an artworkImage method to my Artist class that retains the image created with imageWithData:
- (UIImage*)artworkImage {
if(artworkImage == nil)
artworkImage = [[UIImage imageWithData:self.image] retain];
return artworkImage;
}
So now I can directly set the retained image to my TableViewCell as follow:
cell.artworkImage = artist.artworkImage;
I also set my setNeedsDisplay inside the setArtworkImage: method of my tableViewCell class. Scrolling is still laggy and Shark shows exactly the same results.
Your profiling data strongly suggests that the bottleneck is in the unpacking of your PNG images. My guess is that 58.5 % of your presented CPU time is spent unpacking PNG data (i.e. if the memcpy call is also included in the loading). Probably even more of the time is spent there, but hard to say without more data. My suggestions:
As stated before, keep loaded images in UIImage, not in NSData. This way you won't have to PNG-unpack every time you display an image.
Put the loading of your images in a worker thread, to not affect the responsiveness of the main thread (as much). Creating a worker is real easy:
[self performSelectorOnMainThread:#selector(preloadThreadEntry:) withObject:nil waitUntilDone:NO];
Preload images far ahead, like 100 rows or more (like 70 in the direction you're scrolling, keep 30 in the opposite direction). If all your images need to be 88x88 pixels on retina, 100 images would require no more than two MB.
When you profile more the calls to stuff named "png", "gz", "inflate" and so forth might not go way down your list, but they will certainly not affect the feeling of the application in such a bad way.
Only if you still have performance problems after this, I would recommend you look into scaling, and for instance loading "...#2x.png" images for retina. Good luck!
[UIImage imageWithData:] doesn't cache.
This means that CoreGraphic uncompress and process your image every single time you pass in that dataSource method.
I would change your artist's object to hold on a UIImage instead of NSData. You can always flush the image on memoryWarning if you get them a lot.
Also, I would not recommend using setNeedsDisplay inside the dataSource call, I would use that from within your cell.
SetNeedsDisplay is not a direct call to drawRect:
It only tells the os to draw the UIVIew again at the end of the runloop. You can call setNeedsDisplay 100 times in the same runloop and the OS will only call your drawRect method once.
If the delay's happening in your -drawRect, then you might want to take a look at this article: Tweetie's developer explains pretty thoroughly the method he used to get that smooth scrolling you're after. This has become a bit easier since then, though: CALayer has a shouldRasterize property that basically flattens its sublayers into a bitmap, which can then—as long as nothing changes inside the layer—give you much better performance when animating the layer around, as UITableView does when you scroll it. In this case, you'd probably apply that property to your individual UITableViewCells' layers.
My guess is that the delay is from storing images in Core Data. Core Data is usually not a good way to store large blobs of data.
A better solution would be to store the images as individual files on disk, using an album id to identify each image. Then you would setup an in memory cache to store the images in RAM for fast loading into your UIImageViews. The loading of the images from disk to RAM would ideally need to be done on a background thread (e.g. try performSelectorOnBackgroundThread) so that the I/O doesn't bog down the main thread (which will impact on your scrolling performance).
Addendum 1
If you're only calling -drawRect: once per cell load (as you should be), then the problem might be the scaling of the images. Scaling an image by drawing it in code using drawInRect will use CPU time, so an alternative approach is to scale the images as you receive them from the iPod library (if you need them in multiple sizes, save a version in each size you require). You may need to do this on a background thread when you import the data to avoid blocking the main (UI) thread.
One alternative thing to consider is that UIImageView may do it's scaling using Core Animation which would mean it is hardware accelerated (I'm not sure if this is actually the case, I'm just guessing). Switching to a UIImageView for the image would therefore get rid of the CPU burden of image scaling. You would have a slight increase in compositing overhead, but it might be the easiest way to get closer to "optimum" scrolling performance.
At this point your best bet is to use Instruments (previously Shark) to try and find bottlenecks in your code.
Hey all, here's the deal...
I've got a UIImage that is being loaded in the background via an NSURLConnection within a subclassed UIImageView. Until the data finishes downloading, a placeholder image is shown. The UIImage needs to be used by another, separate UIImageView, as well.
The problem I'm having is that if I set the second UIImageView's image property to that of the subclassed object before the download is complete, the second UIImageView never displays the downloaded image, since it's image prop is pointing to the placeholder.
Is there anyway to pass a pointer to a pointer between these two UIImageViews?
I've tried things like this:
imageView2.image = &[imageView1 image];
imageView2.image = *[imageView1 image];
But these don't seem to work. Any ideas?
Those pointers look particularly gross. The problem with doing it that way is that, even though you might be updating the data pointed to by those pointers, you are not notifying your UIImageView subclasses that the data has changed, and thus they don't know to redraw.
Instead of playing with that pointer mess, why not use Key-Value Observing instead? If you have a separate thread running that downloads the UIImage, you can just tell your UIImageViews to observe that property and when your downloader class is finished downloading all the data and has put it into the property, the views will get a notification and can then display the image.
Can't you just catch the connectionDidFinishLoading: message from the NSURLConnection and then set the second image?
I have a UIImage that I'm instantiating using imageWithData: (the data is loaded from the bundle using [NSData dataWithContentsOfFile:]).
I'm then drawing the image like so:
NSData *imageData = [NSData dataWithContentsOfFile:fileLocation];
UIImage *myImage = [UIImage imageWithData:imageData];
//These lines are superfluous from what I can tell, replacing with
//UIImage *myImage = [UIImage imagedNamed:imageName]; very soon.
[myImage drawAtPoint:CGPointMake(0,0)];
//myImage will be released at the end of the run loop
My question is this: The UIImage created is autoreleased. What happens in terms of memory when the UIImage is drawn to the view and then the UIImage is deallocated out of existence. Obviously, visually, the image is still there as it has been drawn to a context.
Does the memory usage double if the UIImage is valid AND has been drawn to the view, and then return to the same amount as if only one UIImage existed after the UIImage has been deallocated?
Now going down another route.
If I used [UIImage imageNamed:] to instantiate the image, then the UIImage class has its own image cache of sorts and will only ever hold one true instance of a particular image (regardless of how many UIImage instances are created to represent that one image).
My other question is: What happens to the image in the cache if I draw it to a context and then the UIimage is released (via autorelease at the end of the run loop)? Does the image stay in the cache an consume memory? Is it removed because no other UIImage instances are using it?
Any help would be great, thanks!
I really recommend starting with simple code here and then optimizing where you are actually having trouble. The kind of space optimizations you're describing are likely to create serious time problems (recreating the UIImage in -drawRect: for instance is very expensive). That said, let's look through the various questions:
First, as I said, be careful with doing expensive work in -drawRect:. You have little control over how often or when it gets called, and any expensive work here (like creating a new UIImage, particularly if you have to read it from disk) can seriously impact UI performance.
I'm assuming that there's a lot more to your -drawRect: then this, correct? UIImageView is optimized for what you've done here, both in speed and in memory. But if your view is much more complicated, then drawing the image yourself is better than creating lots of sub-views.
As has been noted, when you call -drawAtPoint:, the copy you make is a bitmap representation (blended with any other drawing that is done). It's unrelated to the original UIImage in terms of memory usage. You cannot trade one for the other. The memory required for the bitmap representation is a function of the view size and bit depth, and you're not going to be able to change it. It doesn't care if the UIImage exists after it's been drawn.
-imageNamed: does do caching for you and is generally a good choice. Note that it doesn't clear its cache in low-memory situations. The UIImages themselves transparently dump their underlying data if they were loaded from a file (and they dump extra representations in any case). The UIImage reference includes information about how this is done.
If you're very concerned about performance around these images (space or time), and you don't need the functionality of UIImage, you should be working with CGImages. They are not as flexible as UIImages, and they're more complex to code, but they are more efficient. That said, UIImages are good for most purposes.
At any point, when you "draw to context", the context does not hold a reference to the image. Instead, drawing will place the actual bits of pixels on the context, so it doesn't need the reference to UIImage (or anything drawn to it - NSString, NSPath etc).
Regarding imageNamed:, it will never be really released - what you get is a refernece that you don't (and shouldn't) release, but the cached image might still be there.
It really depends on where you're creating the UIImage instance itself. If this is a UIView subclass, are you creating the UIImage in -drawRect: or -initWithFrame: or somewhere else? If you load the image once (in your init or with a setter which retains, etc.) then there shouldn't be a problem. However if you're creating the image over and over again in -drawRect: then at the very least you will have a performance problem.