Release textures (GLKTextureInfo objects) allocated by GLKTextureLoader - iphone

New to developing on iOS and in particular the new OpenGL related features on iOS 5, so I apologize if any of my questions are so basic.
The app I am working on is designed to receive camera frames and display them on screen via OpenGL ES (the graphic folks will take over this and add the actual OpenGL drawing about which I know very little). The application is developed XCode4, and the target is iPhone4 running iOS 5. For the moment, I used the ARC and the GLKit functionality and all is working fine except for the memory leak in loading the images as texture. The app receives a "memory warning" very soon.
Specifically, I would like to ask how to release the textures allocated by
#property(retain) GLKTextureInfo *texture;
-(void)setTextureCGImage:(CGImageRef)image
{
NSError *error;
self.texture = [GLKTextureLoader textureWithCGImage:image options:nil error:&error];
if (error)
{
NSLog(#"Error loading texture from image: %#",error);
}
}
The image is a quartz image built from the camera frame (sample code from apple). I know the problem is not in that part of the code since if I disable the assignment, the app does not receive the warning.

Super hacky solution I believe, but it seems to work:
Add the following before the assignment:
GLuint name = self.texture.name;
glDeleteTextures(1, &name);
If there's a more official way (or if this is the official way), I would appreciate if someone could let me know.

Not a direct answer, but something I noticed and it wont really fit in a comment.
If you're using GLKTextureLoader to load textures in the background to replace an existing texture, you have to delete the existing texture on the main thread. Deleting a texture in the completion handler will not work.
AFAIK this is because:
Every iOS thread requires its own EAGLContext, so the background queue has its own thread with its own context.
The completion handler is run on the queue you passed in, which is most likely not the main queue. (Else you wouldn't be doing the loading in the background...)
That is, this will leak memory.
NSDictionary *options = #{GLKTextureLoaderOriginBottomLeft:#YES};
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
[self.asyncTextureLoader textureWithContentsOfFile:#"my_texture_path.png"
options:options
queue:queue
completionHandler:^(GLKTextureInfo *texture, NSError *e){
GLuint name = self.myTexture.name;
//
// This delete textures call has no effect!!!
//
glDeleteTextures(1, &name);
self.myTexture = texture;
}];
To get around this issue you can either:
Delete the texture before the upload happens. Potentially sketchy depending on how your GL is architected.
Delete the texture on the main queue in the completion handler.
So, to fix the leak you need to do this:
//
// Method #1, delete before upload happens.
// Executed on the main thread so it works as expected.
// Potentially leaves some GL content untextured if you're still drawing it
// while the texture is being loaded in.
//
// Done on the main thread so it works as expected
GLuint name = self.myTexture.name;
glDeleteTextures(1, &name)
NSDictionary *options = #{GLKTextureLoaderOriginBottomLeft:#YES};
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
[self.asyncTextureLoader textureWithContentsOfFile:#"my_texture_path.png"
options:options
queue:queue
completionHandler:^(GLKTextureInfo *texture, NSError *e){
// no delete required, done previously.
self.myTexture = texture;
}];
or
//
// Method #2, delete in completion handler but do it on the main thread.
//
NSDictionary *options = #{GLKTextureLoaderOriginBottomLeft:#YES};
dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
[self.asyncTextureLoader textureWithContentsOfFile:#"my_texture_path.png"
options:options
queue:queue
completionHandler:^(GLKTextureInfo *texture, NSError *e){
// you could potentially do non-gl related work here, still in the background
// ...
// Force the actual texture delete and re-assignment to happen on the main thread.
dispatch_sync(dispatch_get_main_queue(), ^{
GLuint name = self.myTexture.name;
glDeleteTextures(1, &name);
self.myTexture = texture;
});
}];

Is there a way to simply replace the contents of the texture to the same GLKTextureInfo.name handle? When using glgentextures you can use the returned texture handle to load new texuture data using glteximage2d. But with GLKTextureLoader it seems that glgentextures is being called every time new texture data is loaded...

Related

IPhone: GLKTextureInfo not working using Grand Central Dispatch?

I am developing an IPhone OpengGL app using the GLKit, and using the following code to create textures:
NSRange dotRange = [textureFileName rangeOfString:#"." options:NSCaseInsensitiveSearch];
if (dotRange.location == NSNotFound){
NSLog(#"OpenGLDRawMaterial:createTextureFromFileName, incorrect file name given in inputs");
return nil;
}
GLKTextureInfo *newTexture;
NSError *error = nil; // stores the error message if we mess up
NSString *bundlepath = [[NSBundle mainBundle] pathForResource:[textureFileName substringToIndex:dotRange.location]
ofType:[textureFileName substringFromIndex:(dotRange.location+1)]];
newTexture = [GLKTextureLoader textureWithContentsOfFile:bundlepath options:[NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] forKey:GLKTextureLoaderOriginBottomLeft] error:&error];
The code works very well , as long as it works in a main thread. Every single time i attempt to make it work in a worker thread i am getting the following message:
"2013-03-04 02:09:01.528 Puppeteer[7063:1503] Error loading texture from image: Error Domain=GLKTextureLoaderErrorDomain Code=17 "The operation couldn’t be completed. (GLKTextureLoaderErrorDomain error 17.)" UserInfo=0x1c5977e0 "
The code i am using for the grand central dispatch queue is :
dispatch_queue_t backgroundQueue = dispatch_queue_create("loadPlayViewBackgroundTexture", 0);
dispatch_async(backgroundQueue, ^{
[self createTexturesForPlayView]; // method calling texture creation
dispatch_async(dispatch_get_main_queue(), ^{
});
});
dispatch_release(backgroundQueue);
If you have any insights or ideas how to fix this issue and have textures being loaded in background i would be very grateful :)
Cheers,
Stéphane
Note that in the documentation for +textureWithContentsOfFile:options:error: includes this statement:check here
This class method loads the texture into the sharegroup attached to the current context for the thread this method is called on.
When you call -textureWithContentsOfFile: from a background thread, that thread has no OpenGL context set (the current GL context is per-thread state), and so GLKit doesn't know which sharegroup to load the texture into.
But, you're making this harder than it needs to be. GLKit can already manage asynchronous texture loading itself. Look at the variant -textureWithContentsOfFile:options:queue:completionHandler:. You don't have to create your own queue at all: you can simply pass in the main queue to receive notification when the load is complete.

Feel lag when scroll and sametime load full screen image from asset in background thread

Here is the thing:
I have a scroll view, it did the lazy load for full screen image of user's photo:
[self.assetsLibrary assetForURL:[NSURL URLWithString:[[self.assets objectAtIndex:index] objectForKey:#"asset_url"]]
resultBlock:^(ALAsset *asset) {
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
CGImageRef cgImage = asset.defaultRepresentation.fullScreenImage;
UIImage *image = [UIImage imageWithCGImage:cgImage];
dispatch_async(dispatch_get_main_queue(), ^{
imageView.image = image;
});
});
}
failureBlock:^(NSError *error) {
NSLog(#"error");
}];
I know it is expensive to load full screen image, so I put it in to the background thread, but it is still lag when I do the scroll. And still lag even I change it like this:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
CGImageRef cgImage = asset.defaultRepresentation.fullScreenImage;
UIImage *image = [UIImage imageWithCGImage:cgImage];
imageView.image = image;
dispatch_async(dispatch_get_main_queue(), ^{
});
});
Obviously, nothing to do in the main queue, but it still lag until I comment the line:
// CGImageRef cgImage = asset.defaultRepresentation.fullScreenImage;
So I am so confused, is there something wrong when I used GCD?
Somebody can help me to explain it? Any thing will be helpful.
Thank you, guys.
UPDATE
To #Fogmeister : The size of the photo is the full screen size, actuel imageView size is around half. Even I comment the line: "imageView.image = image;" it is still lag. Which means it is not from the resizing. I know where the time is being taken, here: "asset.defaultRepresentation.fullScreenImage;". When I comment it, everything fine, there is no more lag.
So, which I don't understand is, I've already put it in the background thread...
Ok, finally I solved problem:
Instead of getting image directly by
asset.defaultRepresentation.fullScreenImage
I use the method from Apple's Exemple PhotosByLocation (code below) to get the image in BG thread. That works great, no more lag when scroll. But I am still confused, I don't know exactly why. So I appreciate if someone can explain it to me.
- (UIImage *)fullSizeImageForAssetRepresentation:(ALAssetRepresentation *)assetRepresentation {
UIImage *result = nil;
NSData *data = nil;
uint8_t *buffer = (uint8_t *)malloc(sizeof(uint8_t)*[assetRepresentation size]);
if (buffer != NULL) {
NSError *error = nil;
NSUInteger bytesRead = [assetRepresentation getBytes:buffer fromOffset:0 length:[assetRepresentation size] error:&error];
data = [NSData dataWithBytes:buffer length:bytesRead];
free(buffer);
}
if ([data length]) {
CGImageSourceRef sourceRef = CGImageSourceCreateWithData((__bridge CFDataRef)data, nil);
NSMutableDictionary *options = [NSMutableDictionary dictionary];
[options setObject:(id)kCFBooleanTrue forKey:(id)kCGImageSourceShouldAllowFloat];
[options setObject:(id)kCFBooleanTrue forKey:(id)kCGImageSourceCreateThumbnailFromImageAlways];
[options setObject:(id)[NSNumber numberWithFloat:640.0f] forKey:(id)kCGImageSourceThumbnailMaxPixelSize];
//[options setObject:(id)kCFBooleanTrue forKey:(id)kCGImageSourceCreateThumbnailWithTransform];
CGImageRef imageRef = CGImageSourceCreateThumbnailAtIndex(sourceRef, 0, (__bridge CFDictionaryRef)options);
if (imageRef) {
result = [UIImage imageWithCGImage:imageRef scale:[assetRepresentation scale] orientation:(UIImageOrientation)[assetRepresentation orientation]];
CGImageRelease(imageRef);
}
if (sourceRef) CFRelease(sourceRef);
}
return result;
}
You're solution taken from Apple's PhotosByLocation is actually grabbing the biggest resolution image, not the fullscreen image. IOW, it's essentially the same as calling fullResolutionImage instead of fullScreenImage. How that fixes your problem, I'm not sure. I'm struggling with the same performance issue. If I use fullScreenImage, I get lags in my scrolling. But switching to fullResolutionImage gets rid of the lags. fullResolutionImage takes about twice as long as fullScreenImage, but since this is always in the background, it shouldn't really matter how much time it takes. I suspect that fullScreenImage is returning an image that needs some sort of additional processing once it gets rendered to the screen in the main thread - hence the lag.
Do you know the actual size of the photo? What is very expensive is scrolling images that are being resized to fit the screen.
Seeing as you're already loading in a BG thread it might be worth resizing the image to the size you are displaying it at before sticking it on the screen.
You can see where the time is being taken by using the CoreAnimation tool in Instruments by profiling the app from Xcode. It will even tell you which line of code is causing the slow down and missed animation frames.
From the apple documentation:
DISPATCH_QUEUE_PRIORITY_DEFAULT
Items dispatched to the queue run at the default priority; the queue is scheduled for execution after all high priority queues have been scheduled, but before any low priority queues have been scheduled.DISPATCH_QUEUE_PRIORITY_BACKGROUNDItems dispatched to the queue run at background priority; the queue is scheduled for execution after all high priority queues have been scheduled and the system runs items on a thread whose priority is set for background status. Such a thread has the lowest priority and any disk I/O is throttled to minimize the impact on the system.
You're running it in a separate thread, but that's not necessarily a thread "in the background." A background thread loading something in my experience will be completely blocked by doing a UI update such as scrolling a UIScrollView. Have you tried using DISPATCH_QUEUE_PRIORITY_BACKGROUND?

On-demand Texture Loading with GLKit

I have an application in which I need to draw text on the screen using OpenGL textures. In my render loop I request the textures from my TextureController, which has functions such as
- (GLuint)textureForImageNamed:(NSString*)fileName;
{
// we're checking a hash.. possibly slow
NSNumber* textureHandle = [textures objectForKey:fileName];
if(!textureHandle)
textureHandle = [self loadTexture:fileName];
GLuint handle = (GLuint)[textureHandle unsignedIntValue];
return handle;
}
where loadTexture loads the texture into memory like this
- (NSNumber*)loadTexture:(NSString*)fileName
{
GLuint texID = [GLKTextureLoader textureWithCGImage:
[[UIImage imageNamed:#"star.png"] CGImage]
options:nil
error:nil].name;
NSNumber* num = [NSNumber numberWithUnsignedInt:texID];
[textures setObject:num forKey:fileName];
return num;
}
And similar functions for text:
- (GLuint)textureForTextWithString:(NSString*)text;
- (NSNumber*)loadTextureForString:(NSString*)text;
This all works fine when I have requested the image once before my render loop, but the whole point is that the text and images that need to be drawn are not known in advance. So I want to be able to call textureForTextWithString:#"fooBar" and either give me the texture if it was loaded before, or load it in -- add it to the dictionary and then give it to me. However, I get a black texture for all the textures that weren't preloaded (i.e. for which textureForTextWithString:#"fooBar" was not called before I started the render loop).
Relevant code in the render loop:
GLuint secondTex = [textures textureForTextWithString:#"NO"];
glBindTexture(GL_TEXTURE_2D, secondTex);
glBindBuffer(GL_ARRAY_BUFFER, some_vertex_buffer);
glDrawArrays(GL_POINTS, 0, num);
I am looking for a solution, or better yet -- advice on how to better handle this particular problem.
EDIT: The two files in question can be found here:
https://gist.github.com/3153940 - The main OpenGL VC
https://gist.github.com/3153941 - The texture controller

Asynchronous texture loading iPhone OpenGL ES 2

I'm creating and loading a lot of textures (made of strings). To keep the animation running smoothly, I offload the work to a separate worker thread. It seems to work more or less exactly the way I want, but on older devices (iPhone 3GS) I sometimes notice a long (1 sec) lag. It only occurs sometimes. Now I'm wondering if I'm doing this correctly or if there is any conceptual issue. I paste the source code below.
I should also mention that I do not want to use the GLKit TextureLoader because I also want to offload the texture generating work to the other thread, not just the loading part.
In case you're wondering what I need these textures for, have a look at this video: http://youtu.be/U03p4ZhLjvY?hd=1
NSLock* _textureLock;
NSMutableDictionary* _texturesWithString;
NSMutableArray* _texturesWithStringLoading;
// This is called when I request a new texture from the drawing routine.
// If this function returns 0, it means the texture is not ready and Im not displaying it.
-(unsigned int)getTextureWithString:(NSString*)string {
Texture2D* _texture = [_texturesWithString objectForKey:string];
if (_texture==nil){
if (![_texturesWithStringLoading containsObject:string]){
[_texturesWithStringLoading addObject:string];
NSDictionary* dic = [[NSDictionary alloc] initWithObjectsAndKeys:string,#"string", nil];
NSThread* thread = [[NSThread alloc] initWithTarget:self selector:#selector(loadTextureWithDictionary:)object:dic];
thread.threadPriority = 0.01;
[thread start];
[thread release];
}
return 0;
}
return _texture.name;
}
// This is executed on a separate worker thread.
// The lock makes sure that there are not hundreds of separate threads all creating a texture at the same time and therefore slowing everything down.
// There must be a smarter way of doing that. Please let me know if you know how! ;-)
-(void)loadTextureWithOptions:(NSDictionary*)_dic{
[_textureLock lock];
EAGLContext* context = [[SharegroupManager defaultSharegroup] getNewContext];
[EAGLContext setCurrentContext: context];
NSString* string = [_dic objectForKey:#"string"];
Texture2D* _texture = [[Texture2D alloc] initWithStringModified:string];
if (_texture!=nil) {
NSDictionary* _newdic = [[NSDictionary alloc] initWithObjectsAndKeys:_texture,#"texture",string,#"string", nil];
[self performSelectorOnMainThread:#selector(doneLoadingTexture:) withObject:_newdic waitUntilDone:NO];
[_newdic release];
[_texture release];
}
[EAGLContext setCurrentContext: nil];
[context release];
[_textureLock unlock];
}
// This callback is executed on the main thread and marks adds the texture to the texture cache.
-(void)doneLoadingTextureWithDictionary:(NSDictionary*)_dic{
[_texturesWithString setValue:[_dic objectForKey:#"texture"] forKey:[_dic objectForKey:#"string"]];
[_texturesWithStringLoading removeObject:[_dic objectForKey:#"string"]];
}
The problem was that too many threads were started at the same time. Now I am using a NSOperationQueue rather than NSThreads. That allows me to set maxConcurrentOperationCount and only run one extra background thread that does the texture loading.

Image which I loaded in thread is not appeared

I'm using the CCScrollLayer and I wanna prepare images during player select the stages. It's simple but when I scroll stages, it tasks time(delay to load images).
so I decided to use NSThread. and I got a message "cocos2d: CCSpriteFrameCache: Trying to use file 'Level3.png' as texture" from cocos2d. then It's supposed to appear. but These images I loaded on thread doesn't to appear as I want. just nothing.
-(void) moveToStagePage:(int)page
{
...
[NSThread detachNewThreadSelector:#selector(prepareTexture:) toTarget:self withObject:[NSNumber numberWithInt:page]];
...
}
below source is the code which preparing images.(Thread)
-(void) prepareTexture:(NSNumber*)number
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
int _page = [number intValue];
NSLog(#"%d Thread start", _page);
if(loadingTexNum != 0 && (_page + 1) != loadingTexNum)
{
[[CCSpriteFrameCache sharedSpriteFrameCache] removeSpriteFramesFromFile:[NSString stringWithFormat:#"Level%d.plist", loadingTexNum]];
loadingTexNum = _page + 1;
[[CCSpriteFrameCache sharedSpriteFrameCache] addSpriteFramesWithFile:[NSString stringWithFormat:#"Level%d.plist", loadingTexNum]];
}
if(loadingTexNum == 0 && (_page + 1) != loadingTexNum)
{
loadingTexNum = _page + 1;
[[CCSpriteFrameCache sharedSpriteFrameCache] addSpriteFramesWithFile:[NSString stringWithFormat:#"Level%d.plist", loadingTexNum]];
}
[NSThread sleepForTimeInterval:10.0];
NSLog(#"%d Thread release", _page);
[pool release];
}
OpenGL doesn't support loading on multiple threads unless you pass the openGL context.
Each thread in a Mac OS X process has a single current OpenGL rendering context. Every time your application calls an OpenGL function, OpenGL implicitly looks up the context associated with the current thread and modifies the state or objects associated with that context.
OpenGL is not reentrant. If you modify the same context from multiple threads simultaneously, the results are unpredictable. Your application might crash or it might render improperly. If for some reason you decide to set more than one thread to target the same context, then you must synchronize threads by placing a mutex around all OpenGL calls to the context, such as gl* and CGL*. OpenGL commands that block—such as fence commands—do not synchronize threads.
Source
You can either use - (void) addImageAsync:(NSString *)filename target:(id) target selector:(SEL)selector; in the CCTextureCache class, calling this on your main thread.
Or you can implement your own, effectively the same as addImageAsync.
For your reference, this is how CCTextureCache achieves loading the images:
NSAutoreleasePool *autoreleasepool = [[NSAutoreleasePool alloc] init];
// textures will be created on the main OpenGL context
// it seems that in SDK 2.2.x there can't be 2 threads creating textures at the same time
// the lock is used for this purpose: issue #472
[contextLock lock];
if( auxEAGLcontext == nil ) {
auxEAGLcontext = [[EAGLContext alloc]
initWithAPI:kEAGLRenderingAPIOpenGLES1
sharegroup:[[[[CCDirector sharedDirector] openGLView] context] sharegroup]];
if( ! auxEAGLcontext )
CCLOG(#"cocos2d: TextureCache: Could not create EAGL context");
}
if( [EAGLContext setCurrentContext:auxEAGLcontext] ) {
// load / create the texture
CCTexture2D *tex = [self addImage:async.data];
// The callback will be executed on the main thread
[async.target performSelectorOnMainThread:async.selector withObject:tex waitUntilDone:NO];
[EAGLContext setCurrentContext:nil];
} else {
CCLOG(#"cocos2d: TetureCache: EAGLContext error");
}
[contextLock unlock];
[autoreleasepool release];
I believe what is happening is that the main thread tries to use the images before the prepareTexture thread has had a chance to load the textures.
If you immediately try to create sprites with the new textures, for example right in the moveToStagePage method, then that is going to fail. Your threaded texture loading method will need to flag to the other thread that it has completed loading the textures. The easiest way is to simply toggle a BOOL variable. Only when the texture loading thread signaled that the textures have been loaded should you try to create sprites etc. using these textures.
When loading textures on a separate thread you need to use [[CCTextureCache sharedTextureCache] addImageAsync:texture target:target selector:#selector(onTextureLoaded)]; (else the `GLContext gets messed up).