I've got a lovely OpenGLES code slice that renders up images for me. When I want to, I can call a function on it:
-(UIImage *)renderToImage;
That does a lot of rendering work and returns me an image. This includes the generation of FBOs, textures, etc.
Lately, I've found myself needing to enhance this. The image generation takes four seconds, so I want to pass off the work to another thread and let the app continue. This seemed simple enough. I made a method with this code:
-(void) generateRandomNewImage:(MyViewController *)evc{
UIImage * renderedImage = [self renderToImage];
NSString * fileLoc = [self writeToTempFile:renderedImage];
NSLog(#"File location:%#",fileLoc);
[evc performSelectorOnMainThread:#selector(imageGenerationComplete:) withObject:fileLoc waitUntilDone:NO];
}
Hopefully you can see the logic going on here. This method renders the image, saves it to the filesystem, and calls a method on the main thread's viewcontroller to let it know the file is ready. This code is inside my opengl renderer. It's called here, in the main thread's viewcontroller:
thread = [[NSThread alloc] initWithTarget:renderer
selector:#selector(generateRandomNewImage:)
object:self];
[thread start];
To me, that seems fine too. When I run this code, I get told in my console that my framebuffer object status were error'ed, with a status of zero. I have no idea why. As a result, I get a blank image (saving to the temp files work, by the way, I've tested them).
To test, I put all of this code into the main thread, didn't create any new threads or anything. It all worked fine. As soon as I try and pass off the image generation to another thread, I hit problems.
Using OpenGL in another thread is not that simple as that, only one thread can use a OpenGL context at a time, and your second thread doesn't have a OpenGL context, thus all OpenGL calls fail.
Solution: Create another OpenGL context for the second thread, and read this.
Related
I've been following apples example, QA1702, on how to capture images using the AVFoundation. I won't cite the code here because of space concern. A brief description of what I'm trying to achieve:
Use the iPhone camera to pass a "video" (actually a sequence of images) to a web server, and I know this is possible. However in order to be able to pass the an image using the HTTP POST as in this example, I have to save the image. not necessarily in the photos album but I wan't to be able to view the pictures there as well in debug purposes.
The apple QA1702 contains 3 methods:
- (void)setupCaptureSession
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection
//this is modified to be void as you might see, will get back to this
- (void) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer
in setupCaptureSession I start the session as in the example. the captureOutput is only running the imageFromSampleBuffer, and that's where I've added some changes:
// Create a Quartz image from the pixel data in the bitmap graphics context
CGImageRef quartzImage = CGBitmapContextCreateImage(context);
// Unlock the pixel buffer
CVPixelBufferUnlockBaseAddress(imageBuffer,0);
// Free up the context and color space
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
//library is declared in .h and is a ALAssetsLibrary
[library writeImageToSavedPhotosAlbum:quartzImage orientation:ALAssetOrientationDown completionBlock:nil];
// Release the Quartz image
CGImageRelease(quartzImage);
I've removed the creation of the UIImage and changed it to void typ since I do the writeImageToSavedPhotosAlbum: with the CGImageRef here instead.
The problem as I see it is that during the 10sec that I capture images ~150 calls to captureOutput are made, and therefor the same amount to writeImageToSavedPhotos but only ~5-10 pictures are saved. I'm aware of the memory abuse this is but since I'm not getting any warnings I can't figure out why not more images are created. and what can I do about it? Is it because, and I'm only guessing now, writeImageToSavedPhotos starts new threads and the iPhone can't handle more than a certain amount of threads. I've read something about NSOperationQueue, should I look into it?
On a side note, I use a NSTimer in setupCaptureSession:
[NSTimer scheduledTimerWithTimeInterval: 10.0 target:self selector:#selector(timerFireMethod:) userInfo:nil repeats: NO];
however I want to start it in first call to captureOutput in order to avoid time elapsing during the startup of video camera. but if I move this code line to captureOutput then timerFireMethod: is never called? any ideas?
This is solvable with NSOperationQueue, but is no longer interesting to me since writing to file is way to ineffective for most applications.
I am creating a application which displays 8 thumbnails per page and it can have n pages. Each of these thumbnails are UIViews and are added to UIScrollView. However i have implemented Paging using the Apple sample code.
The prob:
Each thumbnail(UIView) takes 150
millisecs to be created and added to
scroll view
Hence for 3 pages it takes awful
huge time to be created and added to
the UI Scrollview.
At this point the scroll view is not very respsonsive and it is very jerky and gives a bad user experience
How can i create the thumbnails and add them to UIScrollview without affecting the touch responsiveness? I want them to run independent of the main thread which is resposible for handling touch events (i suppose).
Also i would like to mention that when a thumbnail is created i trigger a Async download of the image and the delegate method is called when download is complete.
Let me know the the options i have to make this more responsive and update UI without affecting the touch operations. The page control works fine with lazy loading of the grid of thumbnails.
TIA,
Praveen S
Grand Central Dispatch is easy to use for background loading. But GCD is only for after iOS4. If you have to support iOS3, performSelectorInBackground/performSelectorOnMainThread or NSOperationQueue are helpful.
And, be careful almost UIKit classes are not thread-safe except drawing to a graphics context. For example, UIScrollView is not thread-safe, UIImage imageNamed: is not thread-safe, but UIImage imageWithContentsOfFile: is thread-safe.
dispatch_queue_t mainQueue = dispatch_get_main_queue();
dispatch_queue_t concurrentQueue =
dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
dispatch_apply([thumbnails count], concurrentQueue, ^(size_t index) {
Thumbnail *thumbnail = [thumbnails objectAtIndex:index];
thumbnail.image = [UIImage imageWithContentsOfFile:thumbnail.url];
dispatch_sync(mainQueue, ^{
/* update UIScrollView using thumbnail. It is safe because this block is on main thread. */
});
}
/* dispatch_apply waits until all blocks are done */
dispatch_async(mainQueue, ^{
/* do for all done. */
});
}
I was having a similar problem.
What i did was at an instance i kept only 3 pages in the memory and cleared remaining all.
If suppose there are 3 screens s1, s2, s3. And the user is viewing s2. Whenever he scrolls to s3 i will remove s1 and load a new page s4.
So that the users will have a better experience. And less memory will be occupied.
Whether you are using a subview or a separate ViewController for each "page" or element of the Scrollview, the jerkiness or poor performance can be helped by changing the location of your code.
Specifically the apple sample code for a scrollview with pagecontrol has something like this:
[self loadScrollViewWithPage:page - 1];
[self loadScrollViewWithPage:page];
[self loadScrollViewWithPage:page + 1];
However, that code appears in their sample in the method "scrollViewDidScroll". It's trying to do multiple heavy lifting by both scrolling and loading at the same time. Even if your images are local this is nasty.
If you move this and related code including a reference to the current page to "scrollViewDidEndDecelerating" the jerkiness of the interface is resolved because the loading happens while the scrollview is no longer moving.
I am creating a application which displays 8 thumbnails per page and it can have n pages. Each of these thumbnails are UIViews and are added to UIScrollView. However i have implemented Paging using the Apple sample code.
The prob:
Each thumbnail(UIView) takes 150
millisecs to be created and added to
scroll view
Hence for 3 pages it takes awful
huge time to be created and added to
the UI Scrollview.
At this point the scroll view is not very respsonsive and it is very jerky and gives a bad user experience
How can i create the thumbnails and add them to UIScrollview without affecting the touch responsiveness? I want them to run independent of the main thread which is resposible for handling touch events (i suppose).
Also i would like to mention that when a thumbnail is created i trigger a Async download of the image and the delegate method is called when download is complete.
Let me know the the options i have to make this more responsive and update UI without affecting the touch operations. The page control works fine with lazy loading of the grid of thumbnails.
TIA,
Praveen S
Grand Central Dispatch is easy to use for background loading. But GCD is only for after iOS4. If you have to support iOS3, performSelectorInBackground/performSelectorOnMainThread or NSOperationQueue are helpful.
And, be careful almost UIKit classes are not thread-safe except drawing to a graphics context. For example, UIScrollView is not thread-safe, UIImage imageNamed: is not thread-safe, but UIImage imageWithContentsOfFile: is thread-safe.
dispatch_queue_t mainQueue = dispatch_get_main_queue();
dispatch_queue_t concurrentQueue =
dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
dispatch_async(concurrentQueue, ^{
dispatch_apply([thumbnails count], concurrentQueue, ^(size_t index) {
Thumbnail *thumbnail = [thumbnails objectAtIndex:index];
thumbnail.image = [UIImage imageWithContentsOfFile:thumbnail.url];
dispatch_sync(mainQueue, ^{
/* update UIScrollView using thumbnail. It is safe because this block is on main thread. */
});
}
/* dispatch_apply waits until all blocks are done */
dispatch_async(mainQueue, ^{
/* do for all done. */
});
}
I was having a similar problem.
What i did was at an instance i kept only 3 pages in the memory and cleared remaining all.
If suppose there are 3 screens s1, s2, s3. And the user is viewing s2. Whenever he scrolls to s3 i will remove s1 and load a new page s4.
So that the users will have a better experience. And less memory will be occupied.
Whether you are using a subview or a separate ViewController for each "page" or element of the Scrollview, the jerkiness or poor performance can be helped by changing the location of your code.
Specifically the apple sample code for a scrollview with pagecontrol has something like this:
[self loadScrollViewWithPage:page - 1];
[self loadScrollViewWithPage:page];
[self loadScrollViewWithPage:page + 1];
However, that code appears in their sample in the method "scrollViewDidScroll". It's trying to do multiple heavy lifting by both scrolling and loading at the same time. Even if your images are local this is nasty.
If you move this and related code including a reference to the current page to "scrollViewDidEndDecelerating" the jerkiness of the interface is resolved because the loading happens while the scrollview is no longer moving.
I have a UITableView with a list of items, each having it's own image. I thought Apple's LazyTableImages sample project would be perfect to learn from, and use to implement the same kind of process of downloading images asynchronously, after the original list data is retrieved.
For the most part, it works quite well, except I did notice a subtle difference in behavior, between this sample app, and how the actual app store downloads images.
If you launch the LazyTableImages sample, then do a quick flick-scroll down, you'll see that the images do not get displayed until after the scrolling comes to a complete stop.
Now, if you do the same test with a list of items in the actual app store, you'll see that the images start displaying as soon as the new items come into view, even if scrolling hasn't stopped yet.
I'm trying to achieve these same results, but so far I'm not making any progress. Does anyone have any ideas on how to do this?
Thanks!
I'm baffled that nobody could answer this...
So, I eventually figured out how to acheive the exact same effect that is used in the actual app store, in regards to how the icons are downloaded/displayed.
Take the LazyTableImages sample project and make a few simpled modifications.
Go into the root view controller and remove all checks regarding is table scrolling and/or decelerating in cellForRowAtIndexPath
Remove all calls to loadImagesForOnScreenRows, and thus remove that method as well.
Go into IconDownload.m and change the startDownload method to not do an async image downlaod, but instead do a sync download on a background thread. Remove all the code in startDownload, and add the following, so it looks like this:
- (void)startDownload
{
NSOperationQueue *queue = [NSOperationQueue new];
NSInvocationOperation *operation = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(loadImage) object:nil];
[queue addOperation:operation];
[operation release];
[queue release];
}
Then, add a loadImage, like this:
- (void)loadImage
{
NSData *imageData = [[NSData alloc] initWithContents OfURL:[NSURL URLWithString:appRecord.imageURLString]];
self.apprecord.appIcon = [UIImage imageWithData:imageData];
[imageData release];
[self performSelectorOnMainThread:#selector(notifyMainThread) withObject:nil waitUntilDone:NO];
}
Then, add notifyMainThread like this:
- (void)notifyMainThread
{
[delegate appImageDidLoad:self.indexPathInTableView];
}
Done! Run it, and you will see the exact app store behavior, no more waiting to request image downloads until scrolling stops, and no more waiting for images to display until scrolling stops, or until user has removed their finger from the screen.
Images are downloaded as soon as the cell is ready to be displayed, and the image is displayed as soon as it is downloaded, period.
Sorry for any typos, I didn't paste this from my app, I typed it in, since I'm away from my mac right now...
Anyway, I hope this helps you all...
Check out UIScrollViewDelegate. I've implemented something like this by listening for scrollViewDidScroll:, calculating the scroll speed (by checking the contentOffset against the last recorded contentOffset, divided by the difference in time), and starting to load images once the speed drops below a certain threshold. (You could achieve something similar with UIScrollViewDelegate's scrollViewDidEndDragging:willDecelerate: as well).
Of course, you don't have to check the speed; you could just load images on UITableViewDelegate's tableView:willDisplayCell:forRowAtIndexPath: whenever you see a new cell, but I've found that if the user is flipping through tons of cells, you don't need to bother until you see that they're going to slow down to browse.
Hey folks. Trying to get started with iPhone video capture stuff, and having a terrible time getting it going.
At the moment, working with an exact copy of the sample code from this Apple Developer Q&A: qa1702; not going to re-paste it here, since it's a relatively big blob of code.
In any case, I copied and pasted that code into an otherwise-empty project, threw in a call to setupCaptureSession into a custom UIView, and figured I should be good to go:
- (void)awakeFromNib {
NSLog(#"init");
[self setupCaptureSession];
}
Unfortunately, when I run the code, my delegate never seems to be called:
- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
fromConnection:(AVCaptureConnection *)connection
{
NSLog(#"New Image!");
}
I'd expect to be seeing a flood of "New Image!" getting dumped to the log, but I'm getting absolutely nothing. I do know that all the initialization code is being run (I threw a few NSLog statements here and there within setupCaptureSession, and they all print).
So, I'd either A) like to figure out what I'm doing wrong or B) see some very simple capture code embedded into a UIView that actually works (the less code, the better). Ideally, the sample code would use the setupCaptureSession code as described in the Apple developer site above.
Note that this code is being tested on an iPhone 3GS running the newest version of iOS 4.
Thanks in advance!
I copied and pasted the same code into a new project and it works fine for me.
Here is what I did:
I created a View Based application in XCode.
I added the following frameworks:CoreMedia, CoreVideo,AVFoundation
I imported AVFoundation into my viewcontroller's header file
I specified my view controller to use the AVCaptureVideoDataOutputSampleBufferDelegate protocol in its interface definition
In the implementation I copied the code from the app document you specified in your question
I commented out this:
[self setSession:session]
I commented out this:
UIImage *image = [self imageFromSampleBuffer:sampleBuffer];
and added this:
NSLog(#"here");
In the viewDidLoad method i put
[self setupCaptureSession];
Build. Run.
The "here" gets printed repeatedly to the console.