Using multithreading to capture image of UIView - iphone

I am using Grand Central Dispatch to queue a task to capture a UIView as an image. Everything is working fine with this except that the image capturing running on a queue is taking quite a long time.
Is there any way to speed this up or improve my technique. Here is the code to capture and scale the image, which is then set to a UIImageView's image for displaying.
(void)captureScrollViewImageForLayoverView
{
// capture big map image
CGSize size = mainView.bounds.size;
UIGraphicsBeginImageContext(size);
[[mainView layer] renderInContext:UIGraphicsGetCurrentContext()];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
// scale map
size = myLocationOverlay.bounds.size;
UIGraphicsBeginImageContext(size);
[newImage drawInRect:CGRectMake(0,0,size.width,size.height)];
UIImage *scaledImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
myLocationOverlay.imageMap.image = scaledImage;
}
And Here is the code that is queueing the task.
...
mapImageDrawQueue = dispatch_queue_create("mapImage.drawQueue", NULL);
(void)captureScrollViewImageForLayoverViewWrapper {
// multithreaded approach to draw map
dispatch_async(mapImageDrawQueue, ^{ [self captureScrollViewImageForLayoverView]; });
}

I used to load images at queue in my app with same issue you mentioned. I checked official programming guide, remembered it told that it can't guarantee the queue run immediately after you call it. So I used NSThread do the job, and it's working well:
NSThread* myThread = [[NSThread alloc] initWithTarget:self
selector:#selector(captureScrollViewImageForLayoverView)
object:nil];
[myThread start];
Remember add these lines to the function captureScrollViewImageForLayoverView:
At top : NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
At end : [pool release];
Check this doc to know more - Threading Programming Guide
Hope it helps.

Related

self.layer renderInContext:context in drawRect method gives Bad Access in ipad but not in simulator

I am working in an app and in which i do the video recording. Now what i do is capture the images and then create the video the code is as follows
- (void) drawRect:(CGRect)rect {
NSDate* start = [NSDate date];
CGContextRef context = [self createBitmapContextOfSize:self.frame.size];
NSLog(#"context value %#",context);
[self.layer renderInContext:context];
CGImageRef cgImage = CGBitmapContextCreateImage(context);
UIImage* background = [UIImage imageWithCGImage: cgImage];
CGImageRelease(cgImage);
self.currentScreen = background;
if (_recording) {
float millisElapsed = [[NSDate date] timeIntervalSinceDate:startedAt] * 1000.0;
[self writeVideoFrameAtTime:CMTimeMake((int)millisElapsed, 1000)];
}
float processingSeconds = [[NSDate date] timeIntervalSinceDate:start];
delayRemaining = (1.0 / self.frameRate) - processingSeconds;
[self performSelectorInBackground:#selector(setNeedsDisplay) withObject:nil];
}
Now the problem is that the method is called recursively and when stop recording it gives me Exc_Bad_Access.
It works fine in simulatoor bot crash in device, How can i solve this issue?
You shouldn’t be doing this in drawRect:, the only reason to override drawRect: is to do some custom drawing (which you are not). You also shouldn’t call setNeedsDisplay on a background thread, this is a GUI method which can only be called from the main thread. Also depending on how your createBitmapContextOfSize: method and your environment you might need to release your bitmap context object.

images from documents asynchronous

I need to read images from NSDocumentDirectory to multiple uiimageview async so it won't block the UI.
I know i can use perform selector in background to load a uiimage, but then how can i associate it with the dynamic uiimageview ?
One convenient way is to use blocks, something like:
[self loadFullImageAt:imagePath completion:^(UIIMage * image){
self.imageView.image = image;
}];
Where you would load the image as data (since UIImage otherwise loads the image data deferred - when you first access it). It's also a good idea to decompress the image while still in the background thread, so the main thread doesn't have to do it when we first use the image.
- (void)loadFullImageAt:(NSString *)imageFilePath completion:(MBLoaderCompletion)completion {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0), ^{
NSData *imageData = [NSData dataWithContentsOfFile:imageFilePath];
UIImage *image = nil;
if (imageData) {
image = [[[UIImage alloc] initWithData:imageData] decodedImage];
}
dispatch_async(dispatch_get_main_queue(), ^{
completion(image);
});
});
}
The callback is defined as:
typedef void (^MBLoaderCompletion)(UIImage *image);
Here's an UIImage category that implements the decompression code:
UIIMage+Decode.h
#import <UIKit/UIKit.h>
#interface UIImage (Decode)
- (UIImage *)decodedImage;
#end
UIIMage+Decode.m
#import "UIImage+Decode.h"
#implementation UIImage (Decode)
- (UIImage *)decodedImage {
CGImageRef imageRef = self.CGImage;
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(NULL,
CGImageGetWidth(imageRef),
CGImageGetHeight(imageRef),
8,
// Just always return width * 4 will be enough
CGImageGetWidth(imageRef) * 4,
// System only supports RGB, set explicitly
colorSpace,
// Makes system don't need to do extra conversion when displayed.
kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little);
CGColorSpaceRelease(colorSpace);
if (!context) return nil;
CGRect rect = (CGRect){CGPointZero,{CGImageGetWidth(imageRef), CGImageGetHeight(imageRef)}};
CGContextDrawImage(context, rect, imageRef);
CGImageRef decompressedImageRef = CGBitmapContextCreateImage(context);
CGContextRelease(context);
UIImage *decompressedImage = [[UIImage alloc] initWithCGImage:decompressedImageRef scale:self.scale orientation:self.imageOrientation];
CGImageRelease(decompressedImageRef);
return decompressedImage;
}
#end
The sample code provided here assumes that we're using ARC.
When you say "dynamic" UIImageView, are these programmatically created on a UIScrollView? on a UITableView? samfisher is quite right on the basic question, but the details differ a little based upon how you created the UIImageView (e.g. if UITableView, you need to make sure that the cell is still visible and hasn't been dequeued; if UIScrollView, even then you might want to only load the UIImageView if the image is still visible on the screen (esp if the images are large or numerous)).
But the basic idea is that you might do something like:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
UIImage *image = [self getTheImage];
// ok, now that you have the image, dispatch the update of the UI back to the main queue
dispatch_async(dispatch_get_main_queue(), ^{
// if the image view is still visible, update it
});
});
Note that you invoke the retrieval of the image on some background queue or thread, but make sure to update the UI back on the main thread!
If you're updating a scrollview, you might want to do some checking that the view is still visible, such as contemplated here or here. If you're updating a tableview, perhaps something like this which checks if the cell is still visible. It all depends upon what you're trying to do.
you can use NSThread/dispatch queue for creating threads which can create your UIImageView-s and loads up images in them.

UIImage decompression causing scrolling lag

I have this app with a full screen tableView that displays a bunch of tiny images. Those images are pulled from the web, processed on a background thread, and then saved to disk using something like:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
UIGraphicsBeginImageContextWithOptions(rect.size, YES, 0);
// code that adds some glosses, shadows, etc
UIImage *output = UIGraphicsGetImageFromCurrentImageContext();
NSData* cacheData = UIImagePNGRepresentation(output);
[cacheData writeToFile:thumbPath atomically:YES];
dispatch_async(dispatch_get_main_queue(), ^{
self.image = output; // refreshes the cell using KVO
});
});
This code is only executed the first time the cell is displayed (since after that the image is already on disk). In that case, the cell is loaded using:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
UIImage *savedImage = [UIImage imageWithContentsOfFile:thumbPath];
if(savedImage) {
dispatch_async(dispatch_get_main_queue(), ^{
self.image = savedImage; // refreshes the cell using KVO
});
}
});
My problem is that in the first case, scrolling is butter smooth. But in the 2nd case (where it's reading the image directly from disk), scrolling is super jerky, even once the image is loaded. Drawing is what's causing the lag. Using Instruments, I see copyImageBlockSetPNG, png_read_now and inflate are taking up most of the cpu (they aren't when assigning self.image to UIGraphicsGetImageFromCurrentImageContext())
I'm assuming this happens because in the first case the UIImage is a raw output of the drawing, whereas in the second case it has to decompress the PNG every time it's drawing it. I tried using JPGs instead of PNGs and I get similar results.
Is there a way to fix this? Maybe to have it only decompress the PNG the first time it gets drawn?
Your problem is that +imageWithContentsOfFile: is cached and lazy loading. If you want to do something like this, instead use this code on your background queue:
// Assuming ARC
NSData* imageFileData = [[NSData alloc] initWithContentsOfFile:thumbPath];
UIImage* savedImage = [[UIImage alloc] initWithData:imageFileData];
// Dispatch back to main queue and set image...
Now, with this code, the actual decompression of the image data will still be lazy and cost a little bit, but not nearly as much as the file access hit you're getting with the lazy loading in your code example.
Since you're still seeing a performance issue, you can also force UIImage to decompress the image on the background thread:
// Still on background, before dispatching to main
UIGraphicsBeginImageContext(CGSizeMake(100, 100)); // this isn't that important since you just want UIImage to decompress the image data before switching back to main thread
[savedImage drawAtPoint:CGPointZero];
UIGraphicsEndImageContext();
// dispatch back to main thread...
Jason's tip about pre-drawing the image to decompress it is the key, but you'll get even better performance by copying the whole image and discarding the original.
Images created at runtime on iOS seem to be better optimised for drawing than ones that have been loaded from a file, even after you've forced them to decompress. For that reason, you should load like this (it's also a good idea to put the decompressed image into an NSCache so you don't have to keep reloading it):
- (void)loadImageWithPath:(NSString *)path block:(void(^)(UIImage *image))block
{
static NSCache *cache = nil;
if (!cache)
{
cache = [[NSCache alloc] init];
}
//check cache first
UIImage *image = [cache objectForKey:path];
if (image)
{
block(image);
return;
}
//switch to background thread
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0), ^{
//load image
UIImage *image = [UIImage imageWithContentsOfFile:path];
//redraw image using device context
UIGraphicsBeginImageContextWithOptions(image.size, YES, 0);
[image drawAtPoint:CGPointZero];
image = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
//back to main thread
dispatch_async(dispatch_get_main_queue(), ^{
//cache the image
[cache setObject:image forKey:path];
//return the image
block(image);
});
});
}
Another way image decompression:
NS_INLINE void forceImageDecompression(UIImage *image)
{
CGImageRef imageRef = [image CGImage];
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(NULL, CGImageGetWidth(imageRef), CGImageGetHeight(imageRef), 8, CGImageGetWidth(imageRef) * 4, colorSpace,kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little);
CGColorSpaceRelease(colorSpace);
if (!context) { NSLog(#"Could not create context for image decompression"); return; }
CGContextDrawImage(context, (CGRect){{0.0f, 0.0f}, {CGImageGetWidth(imageRef), CGImageGetHeight(imageRef)}}, imageRef);
CFRelease(context);
}
Using:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
UIImage *image = [UIImage imageWithContentsOfFile:[NSString stringWithFormat:#"%u.jpg", pageIndex]];
forceImageDecompression(image);
dispatch_async(dispatch_get_main_queue(), ^{
[((UIImageView*)page)setImage:image];
});
}

performSelectorInBackground and performing UI Operation

In application I need to take Image out of UIView. Frame of that View is W 1800 and H 1200.
This activity takes lot time and screen stucks there unless this activity get finish.
I need to perform this activity in background,so that user can continue with other things.
What is best approach to achive this. I tried with
- (void)performSelectorInBackground:(SEL)aSelector withObject:(id)arg
But application get crash as it doesn't allow UI operation with this. If I go with
- (void)performSelectorOnMainThread:(SEL)aSelector withObject:(id)arg waitUntilDone:(BOOL)wait
screen stucks there unless this activity get finish.
I need to perform following activity in background
-(void) PrepareImageData
{
UIImage* frontViewImage = [self PreparefrontImage:[self GetSavedFrontImage]];
[self SaveImageinDocumentWithName:frontViewImage FileName:#"frontview.png"];
UIImage* rearViewImage = [self PrepareBackImage:[self GetSavedBackImage]];;
[self SaveImageinDocumentWithName:rearViewImage FileName:#"backview.png"];
}
-(UIImage*) GetSavedBackImage
{
UIImage* background;
UIImage* messageTextview;
UIImage* toTextView;
UIImage* fromTextView;
background = [self GetImageFromView:self toRect:self.frame];
CGRect rect = CGRectMake(0, 0, 1800, 1200);
UIGraphicsBeginImageContext(rect.size);
CGPoint backgroundPoint = CGPointMake(0,0);
[background drawAtPoint:backgroundPoint];
UIImage* backImage = UIGraphicsGetImageFromCurrentImageContex();
UIGraphicsEndImageContext();
return backImage;
}
- (UIImage *) GetImageFromView:(UIView *)aView toRect:(CGRect)aRect
{
CGSize pageSize = aRect.size;
UIGraphicsBeginImageContext(pageSize);
[aView.layer renderInContext:UIGraphicsGetCurrentContext()];
UIImage *image = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return image;
}
Please help me on this.
Thanks.
I think your problem lies with the sheer size of the view - an app attempting to manipulate a view this size in this way is probably quite likely to get killed by the OS for using too much memory, especially under iOS 4.x. UIViews used to be limited to 1024x1024 for a very good reason.
I'd recommend looking into ways of dividing the problem up into chunks which can be processed without requiring so much memory.

CGImage/UIImage lazily loading on UI thread causes stutter

My program displays a horizontal scrolling surface tiled with UIImageViews from left to right. Code runs on the UI thread to ensure that newly-visible UIImageViews have a freshly loaded UIImage assigned to them. The loading happens on a background thread.
Everything works almost fine, except there is a stutter as each image becomes visible. At first I thought my background worker was locking something in the UI thread. I spent a lot of time looking at it and eventually realized that the UIImage is doing some extra lazy processing on the UI thread when it first becomes visible. This puzzles me, since my worker thread has explicit code for decompressing JPEG data.
Anyway, on a hunch I wrote some code to render into a temporary graphics context on the background thread and - sure enough, the stutter went away. The UIImage is now being pre-loaded on my worker thread. So far so good.
The issue is that my new "force lazy load of image" method is unreliable. It causes intermittent EXC_BAD_ACCESS. I have no idea what UIImage is actually doing behind the scenes. Perhaps it is decompressing the JPEG data. Anyway, the method is:
+ (void)forceLazyLoadOfImage: (UIImage*)image
{
CGImageRef imgRef = image.CGImage;
CGFloat currentWidth = CGImageGetWidth(imgRef);
CGFloat currentHeight = CGImageGetHeight(imgRef);
CGRect bounds = CGRectMake(0.0f, 0.0f, 1.0f, 1.0f);
CGAffineTransform transform = CGAffineTransformIdentity;
CGFloat scaleRatioX = bounds.size.width / currentWidth;
CGFloat scaleRatioY = bounds.size.height / currentHeight;
UIGraphicsBeginImageContext(bounds.size);
CGContextRef context = UIGraphicsGetCurrentContext();
CGContextScaleCTM(context, scaleRatioX, -scaleRatioY);
CGContextTranslateCTM(context, 0, -currentHeight);
CGContextConcatCTM(context, transform);
CGContextDrawImage(context, CGRectMake(0, 0, currentWidth, currentHeight), imgRef);
UIGraphicsEndImageContext();
}
And the EXC_BAD_ACCESS happens on the CGContextDrawImage line. QUESTION 1: Am I allowed to do this on a thread other than the UI thread? QUESTION 2: What is the UIImage actually "pre-loading"? QUESTION 3: What is the official way to solve this problem?
Thanks for reading all that, any advice would be greatly appreciated!
I've had the same stuttering problem, with some help I figured out the proper solution here: Non-lazy image loading in iOS
Two important things to mention:
Don't use UIKit methods in a worker-thread. Use CoreGraphics instead.
Even if you have a background thread for loading and decompressing images, you'll still have a little stutter if you use the wrong bitmask for your CGBitmapContext. This are the options you have to choose (it's still a bit unclear to me why):
-
CGBitmapContextCreate(imageBuffer, width, height, 8, width*4, colourSpace,
kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little);
I've posted a sample project here: SwapTest, it has about the same performace as Apples' Photos app for loading/displaying images.
I used #jasamer's SwapTest UIImage category to force load my large UIImage (about 3000x2100 px) in a worker thread (with NSOperationQueue). This reduces the stutter time when setting the image into the UIImageView to an acceptable value (about 0.5 sec on iPad1).
Here is SwapTest UIImage category... thanks again #jasamer :)
UIImage+ImmediateLoading.h file
#interface UIImage (UIImage_ImmediateLoading)
- (UIImage*)initImmediateLoadWithContentsOfFile:(NSString*)path;
+ (UIImage*)imageImmediateLoadWithContentsOfFile:(NSString*)path;
#end
UIImage+ImmediateLoading.m file
#import "UIImage+ImmediateLoading.h"
#implementation UIImage (UIImage_ImmediateLoading)
+ (UIImage*)imageImmediateLoadWithContentsOfFile:(NSString*)path {
return [[[UIImage alloc] initImmediateLoadWithContentsOfFile: path] autorelease];
}
- (UIImage*)initImmediateLoadWithContentsOfFile:(NSString*)path {
UIImage *image = [[UIImage alloc] initWithContentsOfFile:path];
CGImageRef imageRef = [image CGImage];
CGRect rect = CGRectMake(0.f, 0.f, CGImageGetWidth(imageRef), CGImageGetHeight(imageRef));
CGContextRef bitmapContext = CGBitmapContextCreate(NULL,
rect.size.width,
rect.size.height,
CGImageGetBitsPerComponent(imageRef),
CGImageGetBytesPerRow(imageRef),
CGImageGetColorSpace(imageRef),
kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little
);
//kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little are the bit flags required so that the main thread doesn't have any conversions to do.
CGContextDrawImage(bitmapContext, rect, imageRef);
CGImageRef decompressedImageRef = CGBitmapContextCreateImage(bitmapContext);
UIImage* decompressedImage = [[UIImage alloc] initWithCGImage: decompressedImageRef];
CGImageRelease(decompressedImageRef);
CGContextRelease(bitmapContext);
[image release];
return decompressedImage;
}
#end
And this is how I create NSOpeationQueue and set the image on main thread...
// Loads low-res UIImage at a given index and start loading a hi-res one in background.
// After finish loading, set the hi-res image into UIImageView. Remember, we need to
// update UI "on main thread" otherwise its result will be unpredictable.
-(void)loadPageAtIndex:(int)index {
prevPage = index;
//load low-res
imageViewForZoom.image = [images objectAtIndex:index];
//load hi-res on another thread
[operationQueue cancelAllOperations];
NSInvocationOperation *operation = [NSInvocationOperation alloc];
filePath = [imagesHD objectAtIndex:index];
operation = [operation initWithTarget:self selector:#selector(loadHiResImage:) object:[imagesHD objectAtIndex:index]];
[operationQueue addOperation:operation];
[operation release];
operation = nil;
}
// background thread
-(void)loadHiResImage:(NSString*)file {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSLog(#"loading");
// This doesn't load the image.
//UIImage *hiRes = [UIImage imageNamed:file];
// Loads UIImage. There is no UI updating so it should be thread-safe.
UIImage *hiRes = [[UIImage alloc] initImmediateLoadWithContentsOfFile:[[NSBundle mainBundle] pathForResource:file ofType: nil]];
[imageViewForZoom performSelectorOnMainThread:#selector(setImage:) withObject:hiRes waitUntilDone:NO];
[hiRes release];
NSLog(#"loaded");
[pool release];
}
The UIGraphics* methods are designed to be called from the main thread only. They are probably the source of your trouble.
You can replace UIGraphicsBeginImageContext() with a call to CGBitmapContextCreate(); it's a little more involved (you need to create a color space, figure out the right sized buffer to create, and allocate it yourself). The CG* methods are fine to run from a different thread.
I'm not sure how you're initializing UIImage, but if you're doing it with imageNamed: or initWithFile: then you might be able to force it to load by loading the data yourself and then calling initWithData:. The stutter is probably due to lazy file I/O, so initializing it with a data object won't give it the option of reading from a file.
I had the same problem, even though I initialized the image using data. (I guess the data is loaded lazily, too?) I’ve succeeded to force decoding using the following category:
#interface UIImage (Loading)
- (void) forceLoad;
#end
#implementation UIImage (Loading)
- (void) forceLoad
{
const CGImageRef cgImage = [self CGImage];
const int width = CGImageGetWidth(cgImage);
const int height = CGImageGetHeight(cgImage);
const CGColorSpaceRef colorspace = CGImageGetColorSpace(cgImage);
const CGContextRef context = CGBitmapContextCreate(
NULL, /* Where to store the data. NULL = don’t care */
width, height, /* width & height */
8, width * 4, /* bits per component, bytes per row */
colorspace, kCGImageAlphaNoneSkipFirst);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), cgImage);
CGContextRelease(context);
}
#end