I'm creating and loading a lot of textures (made of strings). To keep the animation running smoothly, I offload the work to a separate worker thread. It seems to work more or less exactly the way I want, but on older devices (iPhone 3GS) I sometimes notice a long (1 sec) lag. It only occurs sometimes. Now I'm wondering if I'm doing this correctly or if there is any conceptual issue. I paste the source code below.
I should also mention that I do not want to use the GLKit TextureLoader because I also want to offload the texture generating work to the other thread, not just the loading part.
In case you're wondering what I need these textures for, have a look at this video: http://youtu.be/U03p4ZhLjvY?hd=1
NSLock* _textureLock;
NSMutableDictionary* _texturesWithString;
NSMutableArray* _texturesWithStringLoading;
// This is called when I request a new texture from the drawing routine.
// If this function returns 0, it means the texture is not ready and Im not displaying it.
-(unsigned int)getTextureWithString:(NSString*)string {
Texture2D* _texture = [_texturesWithString objectForKey:string];
if (_texture==nil){
if (![_texturesWithStringLoading containsObject:string]){
[_texturesWithStringLoading addObject:string];
NSDictionary* dic = [[NSDictionary alloc] initWithObjectsAndKeys:string,#"string", nil];
NSThread* thread = [[NSThread alloc] initWithTarget:self selector:#selector(loadTextureWithDictionary:)object:dic];
thread.threadPriority = 0.01;
[thread start];
[thread release];
}
return 0;
}
return _texture.name;
}
// This is executed on a separate worker thread.
// The lock makes sure that there are not hundreds of separate threads all creating a texture at the same time and therefore slowing everything down.
// There must be a smarter way of doing that. Please let me know if you know how! ;-)
-(void)loadTextureWithOptions:(NSDictionary*)_dic{
[_textureLock lock];
EAGLContext* context = [[SharegroupManager defaultSharegroup] getNewContext];
[EAGLContext setCurrentContext: context];
NSString* string = [_dic objectForKey:#"string"];
Texture2D* _texture = [[Texture2D alloc] initWithStringModified:string];
if (_texture!=nil) {
NSDictionary* _newdic = [[NSDictionary alloc] initWithObjectsAndKeys:_texture,#"texture",string,#"string", nil];
[self performSelectorOnMainThread:#selector(doneLoadingTexture:) withObject:_newdic waitUntilDone:NO];
[_newdic release];
[_texture release];
}
[EAGLContext setCurrentContext: nil];
[context release];
[_textureLock unlock];
}
// This callback is executed on the main thread and marks adds the texture to the texture cache.
-(void)doneLoadingTextureWithDictionary:(NSDictionary*)_dic{
[_texturesWithString setValue:[_dic objectForKey:#"texture"] forKey:[_dic objectForKey:#"string"]];
[_texturesWithStringLoading removeObject:[_dic objectForKey:#"string"]];
}
The problem was that too many threads were started at the same time. Now I am using a NSOperationQueue rather than NSThreads. That allows me to set maxConcurrentOperationCount and only run one extra background thread that does the texture loading.
Related
I have the following code that attempts to load a row of thumbnails in a tableview asynchronously:
for (int i = 0; i < totalThumbnails; i++)
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0),
^{
__block GraphicView *graphicView;
__block Graphic *graphic;
dispatch_async(dispatch_get_main_queue(),
^{
graphicView = [[tableViewCell.contentView.subviews objectAtIndex:i] retain];
graphic = [[self.thumbnailCache objectForKey: [NSNumber numberWithInt:startingThumbnailIndex + i]] retain];
if (!graphic)
{
graphic = [[self graphicWithType:startingThumbnailIndex + i] retain];
[self.thumbnailCache setObject: graphic forKey:[NSNumber numberWithInt:startingThumbnailIndex + i]];
}
[graphicView setGraphic:graphic maximumDimension:self.cellDimension];
});
[graphicView setNeedsDisplay];
dispatch_async(dispatch_get_main_queue(),
^{
CGRect graphicViewFrame = graphicView.frame;
graphicViewFrame.origin.x = ((self.cellDimension - graphicViewFrame.size.width) / 2) + (i * self.cellDimension);
graphicViewFrame.origin.y = (self.cellDimension - graphicViewFrame.size.height) / 2;
graphicView.frame = graphicViewFrame;
});
[graphicView release];
[graphic release];
});
}
However when I run the code I get a bad access at this line: [graphicView setNeedsDisplay]; It's worth mentioning that the code works fine when I have it set up like this:
for (int i = 0; i < totalThumbnails; i++)
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0),
^{
dispatch_async(dispatch_get_main_queue(),
^{
//put all the code here
});
}
It works fine and the UITableView loads asynchronously when it's first called, however the scrolling is still really choppy.
So I'd like to get the to get the first bit of code to work so I can get the drawing done in the global thread instead of the main thread (which I assume will fix the choppy scrolling?).
Since iOS4 drawing is able to be done asynchronously so I don't believe that is the problem. Possibly I'm misusing the __Block type?
Anyone know how I can get this to work?
You completely misunderstand how to use GCD. Looking at your code:
__block GraphicView *graphicView;
Your variable here is not initialised to nil. It is unsafe to send messages to.
__block Graphic *graphic;
dispatch_async(dispatch_get_main_queue(),
^{
//statements
});
Your dispatch statement here returns immediately. The system works for you to spin this task off on a different thread. Before, or perhaps at the same time as, the above statements are executed we move on to the next line of execution here...
[graphicView setNeedsDisplay];
At this point graphic view may or may not have been initialised by your dispatch statement above. Most likely not as there wont have been time. As it still hasn't been initialised it points to random memory and hence trying to send messages to it causes EXC_BAD_ACCESS.
If you want to draw cell contents asynchronously (or pre-render images or whatever.) I thouroughly reccommend watching WWDC 2012 session 211 "Building Concurrent User Interfaces on iOS". They do almost exactly what you seem to be attempting to do and explain all the pitfalls you can run into.
I think the issue is because you are trying to re-draw the UIView on a working thread. You should move this:
[graphicView setNeedsDisplay];
To the main queue.
I've spent days trying every possible solution I can think of to this problem, but nothing seems to be working.
I run a background thread like this:
[MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext) {
Media *localMedia = [media inContext:localContext];
UIImage *image = localMedia.thumbnail;
dispatch_async(dispatch_get_main_queue(), ^{
[thumbnails setObject:image forKey:[NSNumber numberWithInt:i]];
[contentDict setObject:thumbnails forKey:#"MediaItems"];
[cell.entryView setNeedsDisplay];
});
}];
Or like this:
dispatch_queue_t cellSetupQueue = dispatch_queue_create("com.Journalized.SetupTimelineThumbnails", NULL);
dispatch_async(cellSetupQueue, ^{
NSManagedObjectContext *newMoc = [[NSManagedObjectContext alloc] init];
NSPersistentStoreCoordinator *coordinator = [NSManagedObjectContext contextForCurrentThread].persistentStoreCoordinator;
[newMoc setPersistentStoreCoordinator:coordinator];
NSNotificationCenter *notify = [NSNotificationCenter defaultCenter];
[notify addObserver:self
selector:#selector(mergeChanges:)
name:NSManagedObjectContextDidSaveNotification
object:newMoc];
Media *localMedia = [media inContext:[NSManagedObjectContext contextForCurrentThread];
UIImage *image = localMedia.thumbnail;
dispatch_async(dispatch_get_main_queue(), ^{
[thumbnails setObject:image forKey:[NSNumber numberWithInt:i]];
[contentDict setObject:thumbnails forKey:#"MediaItems"];
[cell.entryView setNeedsDisplay];
});
}];
Both of these give me a crash with UIImage returning as nil object, and a Cocoa Error 133000.
I've removed every other piece of background threading code, and have saved on the main thread directly before this just to make sure. Running the code above on the main thread also works, but freezes up my UI. Despite all of these efforts, the above code crashes every time.
I'm not sure what to do to make this work.
Edit:
My question, specifically, is how do I make this work without crashing? It seems to be a problem with objects from 1 context not existing in another, but how do i make them exist in another context?
Remember, the MR_inContext: method is using [NSManagedObjectContext objectWithID: ] method under the covers. You should look in there to make sure your object has:
1) Been saved prior to entering into the background context/block in your first code block
2) This method is returning something useful
I'm also not sure how you set up your thumbnail attribute. Ideally it shouldn't matter as long as you have the NSTransformable code write (there are samples on the internets that show you how to save a UIImage in core data using the transformable attribute)
Also, your code should look like this:
UIImage *image = nil;
[MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext) {
Media *localMedia = [media inContext:localContext]; //remember, this looks up an existing object. If your context is a child of a parent context that has contains this, the localContext should refresh the object from the parent.
//image processing/etc
image = localMedia.thumbnail;
} completion:^{
[thumbnails setObject:image forKey:#(i)]; //new literals are teh awesome
contentInfo[#"MediaItems"] = thumbnails; //as is the new indexer syntax (on the latest compiler)
[cell.entryView setNeedsDisplay];
}];
Fast answer:
NSManagedObjectReferentialIntegrityError = 133000,
// attempt to fire a fault pointing to an object that does not exist (we can see the store, we can't see the object)
EDIT:
It's pretty difficult to see something from the code. What is a managed object there?
I suppose the problem is that you are using temporary objects from one context in another context.
I'm using the CCScrollLayer and I wanna prepare images during player select the stages. It's simple but when I scroll stages, it tasks time(delay to load images).
so I decided to use NSThread. and I got a message "cocos2d: CCSpriteFrameCache: Trying to use file 'Level3.png' as texture" from cocos2d. then It's supposed to appear. but These images I loaded on thread doesn't to appear as I want. just nothing.
-(void) moveToStagePage:(int)page
{
...
[NSThread detachNewThreadSelector:#selector(prepareTexture:) toTarget:self withObject:[NSNumber numberWithInt:page]];
...
}
below source is the code which preparing images.(Thread)
-(void) prepareTexture:(NSNumber*)number
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
int _page = [number intValue];
NSLog(#"%d Thread start", _page);
if(loadingTexNum != 0 && (_page + 1) != loadingTexNum)
{
[[CCSpriteFrameCache sharedSpriteFrameCache] removeSpriteFramesFromFile:[NSString stringWithFormat:#"Level%d.plist", loadingTexNum]];
loadingTexNum = _page + 1;
[[CCSpriteFrameCache sharedSpriteFrameCache] addSpriteFramesWithFile:[NSString stringWithFormat:#"Level%d.plist", loadingTexNum]];
}
if(loadingTexNum == 0 && (_page + 1) != loadingTexNum)
{
loadingTexNum = _page + 1;
[[CCSpriteFrameCache sharedSpriteFrameCache] addSpriteFramesWithFile:[NSString stringWithFormat:#"Level%d.plist", loadingTexNum]];
}
[NSThread sleepForTimeInterval:10.0];
NSLog(#"%d Thread release", _page);
[pool release];
}
OpenGL doesn't support loading on multiple threads unless you pass the openGL context.
Each thread in a Mac OS X process has a single current OpenGL rendering context. Every time your application calls an OpenGL function, OpenGL implicitly looks up the context associated with the current thread and modifies the state or objects associated with that context.
OpenGL is not reentrant. If you modify the same context from multiple threads simultaneously, the results are unpredictable. Your application might crash or it might render improperly. If for some reason you decide to set more than one thread to target the same context, then you must synchronize threads by placing a mutex around all OpenGL calls to the context, such as gl* and CGL*. OpenGL commands that block—such as fence commands—do not synchronize threads.
Source
You can either use - (void) addImageAsync:(NSString *)filename target:(id) target selector:(SEL)selector; in the CCTextureCache class, calling this on your main thread.
Or you can implement your own, effectively the same as addImageAsync.
For your reference, this is how CCTextureCache achieves loading the images:
NSAutoreleasePool *autoreleasepool = [[NSAutoreleasePool alloc] init];
// textures will be created on the main OpenGL context
// it seems that in SDK 2.2.x there can't be 2 threads creating textures at the same time
// the lock is used for this purpose: issue #472
[contextLock lock];
if( auxEAGLcontext == nil ) {
auxEAGLcontext = [[EAGLContext alloc]
initWithAPI:kEAGLRenderingAPIOpenGLES1
sharegroup:[[[[CCDirector sharedDirector] openGLView] context] sharegroup]];
if( ! auxEAGLcontext )
CCLOG(#"cocos2d: TextureCache: Could not create EAGL context");
}
if( [EAGLContext setCurrentContext:auxEAGLcontext] ) {
// load / create the texture
CCTexture2D *tex = [self addImage:async.data];
// The callback will be executed on the main thread
[async.target performSelectorOnMainThread:async.selector withObject:tex waitUntilDone:NO];
[EAGLContext setCurrentContext:nil];
} else {
CCLOG(#"cocos2d: TetureCache: EAGLContext error");
}
[contextLock unlock];
[autoreleasepool release];
I believe what is happening is that the main thread tries to use the images before the prepareTexture thread has had a chance to load the textures.
If you immediately try to create sprites with the new textures, for example right in the moveToStagePage method, then that is going to fail. Your threaded texture loading method will need to flag to the other thread that it has completed loading the textures. The easiest way is to simply toggle a BOOL variable. Only when the texture loading thread signaled that the textures have been loaded should you try to create sprites etc. using these textures.
When loading textures on a separate thread you need to use [[CCTextureCache sharedTextureCache] addImageAsync:texture target:target selector:#selector(onTextureLoaded)]; (else the `GLContext gets messed up).
I have thread2 loop where i do assembly (create from raw bytes data) some UIImage
in every iteration of this loop
thread2loop()
{
//make UIIamge here
[self performSelectorOnMainThread:#selector(setUiImage) withObject:nil waitUntilDone:YES];
}
there and then i call setUIImage method on the main thread
- (void) setUiImage
{
self.imageView.image = nil;
self.imageView.image = mImage;
[mImage release];
}
it is working but the Instruments , leaks application shows to me that there are
UIImage leaks here and i do not know how to ##$! get rid of it! (im sad and little tired
and bored), help, what to do, tnx
Surround your threaded code with...
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
//threaded code....
[pool release];
Classic producer/consumer problem. Your producer thread is probably outrunning the main thread (the consumer). I'd recommend keeping a queue of images (instead of the single mImage), guarded by a lock which you enqueue images onto (from your background queue), and dequeue images from your main queue. Or you could use GCD, which makes this even easier. Instead of using mImage to hold onto the created image, you could just use a block which would retain the image and then set it on your image view in the main queue. Something like:
thread2loop() {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
while (...) {
__block id self_block = self; // (don't want to retain self in the block)
UIImage *img = [[UIImage alloc] initWithCGImage:quartzImage scale:1.0 orientation:UIImageOrientationUp];
dispatch_async(dispatch_get_main_queue(), ^{
block_self.imageView.image = img;
[img release];
});
}
[pool drain]; // release is outdated for autorelease pools
}
Warning: Doing this too much will quickly run the device out of memory and cause your app to be killed. You probably want to make sure that your use of this technique is limited to creating a small number of images.
This appears to be the the classic method for scanning images from the iPhone. I have a thread that is dispatched from the main thread to go and scan for Codes. It essentially creates a new UIImage each time then removes it.
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
{
while (![thread isCancelled]) {
#ifdef DEBUG
NSLog(#"Decoding Loop");
#endif
// [self performSelectorOnMainThread:#selector(updateImageBuffer) withObject:nil waitUntilDone:YES];
CGImageRef cgScreen = UIGetScreenImage();
UIImage *uiimage = [UIImage imageWithCGImage:cgScreen];
if (uiimage){
CGSize size = [uiimage size];
CGRect cropRect = CGRectMake(0.0, 80.0, size.width, 360); // Crop to centre of the screen - makes it more robust
#ifdef DEBUG
NSLog(#"picked image size = (%f, %f)", size.width, size.height);
#endif
[decoder decodeImage:uiimage cropRect:cropRect];
}
[uiimage release];
CGImageRelease(cgScreen);
}
}
[pool release];
the problem is that the [pool release] causes an ERROR_BAD_EXC (that old classic) and the program bombs. I'm told that there is no need to call [uiimage release] as I havent explicitly allocated a UIImage but this doesn't seem to be the case. If I take that line out, Memory usage goes through the roof and the program quits dues to lack of memory. It appears I can't have this work the way I'd like.
Is there a way to create a UIImage "in-place"? I.e, have a buffer that is written to again and again as a UIImage? I suspect that would work?
Update!
Tried executing the UIKit related calls on the main thread as follows:
-(void)performDecode:(id)arg{
// Perform the decoding in a seperate thread. This should, in theory, bounce back with a
// decoded or not decoded message. We can quit at the end of this thread.
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
{
while (![thread isCancelled]) {
#ifdef DEBUG
NSLog(#"Decoding Loop");
#endif
[self performSelectorOnMainThread:#selector(updateImageBuffer) withObject:nil waitUntilDone:YES];
if (uiimage){
CGSize size = [uiimage size];
CGRect cropRect = CGRectMake(0.0, 80.0, 320, 360); // Crop to centre of the screen - makes it more robust
#ifdef DEBUG
NSLog(#"picked image size = (%f, %f)", size.width, size.height);
#endif
[decoder decodeImage:uiimage cropRect:cropRect];
}
}
}
[pool drain];
#ifdef DEBUG
NSLog(#"finished decoding.");
#endif
}
-(void) updateImageBuffer {
CGImageRef cgScreen = UIGetScreenImage();
uiimage = [UIImage imageWithCGImage:cgScreen];
//[uiimage release];
CGImageRelease(cgScreen);
}
No joy however as EXC_BAD_ACCESS rears its ugly head when one wishes to grab the "Size" of the UIImage
As has been stated by others, you should not release the UIImage returned from imageWithCGImage: . It is autoreleased. When your pool drains, it tries sending a release message to your already-released image objects, leading to your crash.
The reason why your memory usage keeps climbing is that you only drain the autorelease pool outside of the loop. Your autoreleased objects keep accumulating inside of the loop. (By the way, you need to release your autorelease pool at the end of that method, because it is currently being leaked.) To prevent this accumulation, you could drain the pool at regular intervals within the loop.
However, I'd suggest switching to doing [[UIImage alloc] initWithCGImage:cgScreen] and then releasing the image when done. I try to avoid using autoreleased objects whereever I can within iPhone applications in order to have tighter control over memory usage and overall better performance.
UIGetScreenImage() is private and undocumented so you flat-out cannot use it. Saying that nothing about it suggests that you now own CGImageRef cgScreen so why do you release it? You also have no way of knowing if it is thread safe and so should assume it isn't. You then go on to release the IImage *uiimage which you did not init, retain or copy, so again - you don't own it. Review the docs.
[uiimage release] is definitely wrong in this context. Also, Apple stresses that all UIKit methods must be executed on the main thread. That includes UIGetScreenImage() and +[UIImage imageWithCGImage:].
Edit: So you get an exception when calling -[UIImage size] on the wrong thread. This probably shouldn't surprise you because it is not permitted.
UIImage *uiimage = [[UIImage alloc] initWithCGImage: cgScreen];
Explicitly stating that I know best when to release the object seemed to work. Virtual Memory still increases but physical now stays constant. Thanks for pointing out the UIKit Thread Safe issues though. That is a point I'd missed but seems not affect the running at this point.
Also, I should point out, Red Laser and Quickmark both use this method of scanning camera information ;)