I am trying to figure out how to implement a simple "undo" of last drawing action on the iPhone screen. I draw by first preparing the frame buffer:
[EAGLContext setCurrentContext:context];
glBindFramebufferOES(GL_FRAMEBUFFER_OES, viewFramebuffer);
I then prepare the vertex array and draw this way:
glVertexPointer(2, GL_FLOAT, 0, vertexBuffer);
glDrawArrays(GL_POINTS, 0, vertexCount);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, viewRenderbuffer);
[context presentRenderbuffer:GL_RENDERBUFFER_OES];
How do I simple undo this last action? There has to be a way to save previous state or an built-in OpenGL ES function, I would think.
Thanks
Late answer I know, but incase anyone else comes upon this, I'll post this anyway.
You also have the option of storing the points in an array at every touchesBegan and touchesMoved call. As in here:
[currentStroke addObject:[NSValue valueWithCGPoint:point]];
And when touchesEnded, you can move this to another mutable array, such as:
[allPoints addObject:allCurrentStroke];
Then, you can iterate through allPoints array, passing each subarray to the rendering function. This method has advantages and disadvantages over the method of storing images. First, it saves on hard drive space... however at a cost of memory. Using GL_POINTS, as you are, you will notice it will take time to redraw your image after you hit undo... however you can undo all the way to the first touch easily! So, it depends if you want speed or flexibility... If anyone has a better method to undo, please let me know!
You can grab an image from OpenGL ES context everytime you draw something and save in application's bundle as image file. This saves application's run memory.
When undo is pressed you just draw previous saved image into the context and that's it.
How to grab image from context you can find here: http://www.iphonedevsdk.com/forum/iphone-sdk-development/35281-grab-image-opengl-es-context.html
Related
I read iOS OpenGL ES Logical Buffer Loads that a performance gain can be reached by "discarding" your depth buffer after each draw cycle. I try this, but it's as my game engine is not rendering any longer. I am getting an glError 1286, or GL_INVALID_FRAMEBUFFER_OPERATION_EXT, when I try to render the next cycle.
I get the feeling I need to initialize or setup the depth buffer each cycle if I'm going to discard it, but I can't seem to find any information on this. Here is how I init the depth buffer (all buffers, actually):
// ---- GENERAL INIT ---- //
// Extract width and height.
int bufferWidth, bufferHeight;
glGetRenderbufferParameteriv(GL_RENDERBUFFER,
GL_RENDERBUFFER_WIDTH, &bufferWidth);
glGetRenderbufferParameteriv(GL_RENDERBUFFER,
GL_RENDERBUFFER_HEIGHT, &bufferHeight);
// Create a depth buffer that has the same size as the color buffer.
glGenRenderbuffers(1, &m_depthRenderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, m_depthRenderbuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT24_OES, GAMESTATE->GetViewportSize().x, GAMESTATE->GetViewportSize().y);
// Create the framebuffer object.
GLuint framebuffer;
glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_RENDERBUFFER, m_colorRenderbuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT,
GL_RENDERBUFFER, m_depthRenderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, m_colorRenderbuffer);
And here is what I'm trying to do to discard the depth buffer at the end of each draw cycle:
// Discard the depth buffer
const GLenum discards[] = {GL_DEPTH_ATTACHMENT, GL_COLOR_ATTACHMENT0};
glBindFramebuffer(GL_FRAMEBUFFER, m_depthRenderbuffer);
glDiscardFramebufferEXT(GL_FRAMEBUFFER,1,discards);
I call that immediately following all of my draw calls and...
[m_context presentRenderbuffer:GL_RENDERBUFFER];
Any ideas? Any info someone could point me to? I tried reading through Apple's guide on the subject (where I got the original idea), http://developer.apple.com/library/ios/#documentation/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/WorkingwithEAGLContexts/WorkingwithEAGLContexts.html, but it doesn't seem to work quite right for me.
Your call to glDiscardFramebufferEXT(GL_FRAMEBUFFER,1,discards) is saying that you are discarding just 1 framebuffer attachment, however your discards array includes two: GL_DEPTH_ATTACHMENT and GL_COLOR_ATTACHMENT0.
Try changing it to:
glDiscardFramebufferEXT(GL_FRAMEBUFFER, 2, discards);
In fact, you say that you are discarding these framebuffer attachments at the end of the draw cycle, but directly before [m_context presentRenderbuffer:GL_RENDERBUFFER];. You are discarding the colour renderbuffer attachment that you need in order to present the renderbuffer - perhaps try just discarding the depth attachment, as this is no longer needed at this point.
You only need to initialise your buffers once, not every draw cycle. The glDiscardFramebufferEXT() doesn't actually delete your framebuffer attachment - it is simply a hint to the API to say that the contents of the renderbuffer are not needed in that draw cycle after the discard command completes. From Apple's OpenGL ES Programming Guide for iOS:
A discard operation is defined by the EXT_discard_framebuffer
extension and is available on iOS 4.0 and later. Discard operations
should be omitted when your application is running on earlier versions
of ioS, but included whenever they are available. A discard is a
performance hint to OpenGL ES; it tells OpenGL ES that the contents of
one or more renderbuffers are not used by your application after the
discard command completes. By hinting to OpenGL ES that your
application does not need the contents of a renderbuffer, the data in
the buffers can be discarded or expensive tasks to keep the contents
of those buffers updated can be avoided.
I have an "ImageManipulator" class that performs some cropping, resizing and rotating of camera images on the iPhone.
At the moment, everything works as expected but I keep getting a few huge spikes in memory consumption which occasionally cause the app to crash.
I have managed to isolate the problem to a part of the code where I check for the current image orientation property and rotate it accordingly to UIImageOrientationUp. I then get the image from the bitmap context and save it to disk.
This is currently what I am doing:
CGAffineTransform transform = CGAffineTransformIdentity;
// Check for orientation and set transform accordingly...
transform = CGAffineTransformTranslate(transform, self.size.width, 0);
transform = CGAffineTransformScale(transform, -1, 1);
// Create a bitmap context with the image that was passed so we can perform the rotation
CGContextRef ctx = CGBitmapContextCreate(NULL, self.size.width, self.size.height,
CGImageGetBitsPerComponent(self.CGImage), 0,
CGImageGetColorSpace(self.CGImage),
CGImageGetBitmapInfo(self.CGImage));
// Rotate the context
CGContextConcatCTM(ctx, transform);
// Draw the image into the context
CGContextDrawImage(ctx, CGRectMake(0,0,self.size.height,self.size.width), self.CGImage);
// Grab the bitmap context and save it to the disk...
Even after trying to scale the image down to half or even 1/4 of the size, I am still seeing the spikes to I am wondering if there is a different / more efficient way to get the rotation done as above?
Thanks in advance for the replies.
Rog
If you are saving to JPEG, I guess an alternative approach is to save the image as-is and then set the rotation to whatever you'd like by manipulating the EXIF metadata? See for example this post. Simple but probably effective, even if you have to hold the image payload bytes in memory ;)
Things you can do:
Scale down the image even more (which you probably don't want)
Remember to release everything as soon as you finish with it
Live with it
I would choose option 2 and 3.
Image editing is very resource intensive, as it loads the entire raw uncompressed image data into the memory for processing. This is inevitable as there is absolutely no other way to modify an image other than to load the complete raw data into the memory. Having memory consumption spikes doesn't really matter unless the app receives a memory warning, in that case quickly get rid of everything before it crashes. It is very rare that you would get a memory warning, though, because my app regularly loads a single > 10 mb file into the memory and I don't get a warning, even on older devices. So you'll be fine with the spikes.
Have you tried checking for memory leaks and analyzing allocations?
If the image is still too big, try rotating the image in pieces instead of as a whole.
As Anomie mentioned, CGBitmapContextCreate creates a context. We should release that by using
CGContextRelease(ctx);
If you have any other objects created using create or copy, that should also be released. If it is CFData, then
CFRelease(cfdata);
once again I'm asking for help after quite a bit of research.
I need to create a view where the user can place an image to the background and draw lines/dots(touch events) on top of it and then save the "sketch" by pressing save button.
So after research I decide to pick up this code and build the thing on top of it because it already does half of what I want(it does the drawing).
The sample I have is using OpenGL for drawing and basically I don't care if it is OpenGL or CoreGraphics as soon as it does it.
The problem I have is how to put an image as a background of EAGLView I have in this sample code. My research gave me only suggestions for OpenGL experienced developers but not the working code snippet/solution.
If somebody can help me with this I would be very appreciate.
What I need is just a sample of how to put a UIImage to EAGLView background so then the user can draw(already have the code) on top of it and save the result.
One usually doesn't mix OpenGL with ordinary UI... views. Also drawing a background image using OpenGL is trivial:
First you need to load the Image into a texture. In GLPaint a image file is loaded as brush-texture
https://github.com/omeryavuz/glpaint/blob/master/Classes/PaintingView.m function initWithCoder
To draw a background, the first thing you draw after framebuffer clear is a fullscreen quad with that texture. If you build upong GLPaint, then the projection and modelview matrix and the vertex array state are set properly already. So it boils down to
GLfloat vert[] = {0,0, frame.size.width,0, frame.size.width,frame.size.height, 0,frame.size.height};
GLfloat tex[] = {0,0, 1,0, 1,1, 0,1};
GLuint indexes[] = {0, 1, 2, 2, 3, 0};
glBindTexture(GL_TEXTURE_2D, backgroundTexture);
glEnable(GL_TEXTURE_2D);
glVertexPointer(2, GL_FLOAT, 0, vert);
glTexCoordPointer(2, GL_FLOAT, 0, tex);
glDrawElements(GL_TRIANGLES, 2, GL_UNSIGNED_INT, indexes);
In PaintingView.m, on line 89, set eaglLayer.opaque = NO;.
In your viewController, put a UIImageView or whatever behind the paintingView.
Note: This will probably decrese performance.
Note: It might not initially work; the OpenGL layer may overwrite itself with some sort of default background color before rendering a frame. EDIT: Line 304 in PaintingView.m: Try setting the color to glClearColor(1.0, 1.0, 1.0, 0.0);. I am not sure this works, and don't have time to test this. If it doesn't work, wait till Brad Larson comes around, sees your question, and answers it perfectly ;)
I'm currently working on a project to convert a physics simulation to a video on the iPhone itself.
To do this, I'm presently using two different loops. The first loop runs in the block where the AVAssetWriterInput object polls the EAGLView for more images. The EAGLView provides the images from an array where they are stored.
The other loop is the actual simulation. I've turned off the simulation timer, and am calling the tick myself with a pre-specified time difference every time. Everytime a tick gets called, I create a new image in EAGLView's swap buffers method after the buffers have been swapped. This image is then placed in the array that AVAssetWriter polls.
There is also some miscellaneous code to make sure the array doesn't get too big
All of this works fine, but is very very slow.
Is there something I'm doing that is, conceptually, causing the entire process to be slower than it could be? Also, does anyone know of a faster way to get an image out of Open GL than glReadPixels?
Video memory is designed so, that it's fast for writing and slow for reading. That's why I perform rendering to texture. Here is the entire method that I've created for rendering the scene to texture (there are some custom containers, but I think it's pretty straightforward to replace them with your own):
-(TextureInf*) makeSceneSnapshot {
// create texture frame buffer
GLuint textureFrameBuffer, sceneRenderTexture;
glGenFramebuffersOES(1, &textureFrameBuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, textureFrameBuffer);
// create texture to render scene to
glGenTextures(1, &sceneRenderTexture);
glBindTexture(GL_TEXTURE_2D, sceneRenderTexture);
// create TextureInf object
TextureInf* new_texture = new TextureInf();
new_texture->setTextureID(sceneRenderTexture);
new_texture->real_width = [self viewportWidth];
new_texture->real_height = [self viewportHeight];
//make sure the texture dimensions are power of 2
new_texture->width = cast_to_power(new_texture->real_width, 2);
new_texture->height = cast_to_power(new_texture->real_height, 2);
//AABB2 = axis aligned bounding box (2D)
AABB2 tex_box;
tex_box.p1.x = 1 - (GLfloat)new_texture->real_width / (GLfloat)new_texture->width;
tex_box.p1.y = 0;
tex_box.p2.x = 1;
tex_box.p2.y = (GLfloat)new_texture->real_height / (GLfloat)new_texture->height;
new_texture->setTextureBox(tex_box);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, new_texture->width, new_texture->height, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2DOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_TEXTURE_2D, sceneRenderTexture, 0);
// check for completness
if(glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES) != GL_FRAMEBUFFER_COMPLETE_OES) {
new_texture->release();
#throw [NSException exceptionWithName: EXCEPTION_NAME
reason: [NSString stringWithFormat: #"failed to make complete framebuffer object %x", glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES)]
userInfo: nil];
new_texture = nil;
} else {
// render to texture
[self renderOneFrame];
}
glDeleteFramebuffersOES(1, &textureFrameBuffer);
//restore default frame and render buffers
glBindFramebufferOES(GL_FRAMEBUFFER_OES, _defaultFramebuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, _colorRenderbuffer);
glEnable(GL_BLEND);
[self updateViewport];
glMatrixMode(GL_MODELVIEW);
return new_texture;
}
Of course, if you're doing snapshots all the time, then you'd better create texture frame and render buffers only once (and allocate memory for them).
One thing to remember is that the GPU is running asynchronously from the CPU, so if you try to do glReadPixels immediately after you finish rendering, you'll have to wait for commands to be flushed to the GPU and rendered before you can read them back.
Instead of waiting synchronously, render snapshots into a queue of textures (using FBOs like Max mentioned). Wait until you've rendered a couple more frames before you deque one of the previous frames. I don't know if the iPhone supports fences or sync objects, but if so you could check those to see if rendering has finished before reading the pixels.
You could try using a CADisplayLink object to ensure that your drawing rate and your capture rate correspond to the device's screen refresh rate. You might be slowing down the execution time of the run loop by refreshing and capturing too many times per device screen refresh.
Depending on your app's goals, it might not be necessary for you to capture every frame that you present, so in your selector, you could choose whether or not to capture the current frame.
While the question isn't new, it's not answered yet so I thought I'd pitch in.
glReadPixels is indeed very slow, and therefore cannot be used to record video from an OpenGL-application without adversly affecting performance.
We did find a workaround, and have created a free SDK called Everyplay that can record OpenGL-based graphics to a video file, without performance loss. You can check it out at https://developers.everyplay.com/
I'm trying to create an offscreen framebuffer into which I can do some OpenGL drawing, and then pull the bits out manually. I'm following the instructions here, but in step 4, status is 0 instead of GL_FRAMEBUFFER_COMPLETE_OES.
If I insert a call to glGetError() after every gl call, it returns 0 (GL_NO_ERROR) every time. But, the values of variables do not change during the call. E.g.,
GLuint framebuffer;
glGenFramebuffersOES(1, &framebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
the value of framebuffer does not get altered at all (even when I change it to some arbitrary value and re-execute). It's almost like the gl calls are not actually being made. I'm linking against OpenGLES framework, and get no compile, link, or run-time errors (or warnings).
I'm at a loss as to what to do next. I've tried continuing on with my drawing, but do not see the results I expect, but at this point I can't tell whether it's because of the above error, or the conversion to a UIImage.
This sounds like you don't have an active GL context when you try to create the FBO.
Edit: For the sake of completeness, here's how to create an OpenGL ES 1.1 context and activate it:
EAGLContext* myContext = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES1];
[EAGLContext setCurrentContext: myContext];