I’m working on a kind of iPhone game where player travels through programmically generated wormhole. To draw the wormhole I chose to draw to arrays of textured vertical lines a pixel width to implement top and bottom walls of the wormhole. Every frame all the lines must be shifted left to implement the player movement and new lines must be drown in free space at right. But drawing 1000 textured rectangles every frame is killing my FPS.
And I’m looking for a solution to save all the lines that was drown at previous frame and redraw them altogether to the new shifted position.
It would be terrific if there is a way to draw textured rectangles in some kind of buffer that is bigger than screen, and then render this buffer to the screen.
I guest these are newbie questions cause I’m totally new in OpenGL.
I spent hours trying to figure this out, but haven’t succeeded. So Any help appreciated.
To expand on #Jerry's answer, I'll walk you through the steps, since you're new. First, we'll create the frame buffer object:
GLuint framebuffer;
glGenFramebuffersOES(1, &framebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
Next, we'll create the empty texture to hold our snapshot. This is just the usual OpenGL texture creation stuff, and you can modify it to fit your needs, of course. The only line to notice is the glTexImage2D line - note that instead of pixel data as the last coordinate, you can pass NULL, which creates an empty texture.
GLuint texture;
glGenTextures(1, &texture);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glBindTexture(GL_TEXTURE_2D, 0);
glDisable(GL_TEXTURE_2D);
Now we bind the texture to the frame buffer:
glFramebufferTexture2DOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_TEXTURE_2D, texture, 0);
and check to see if everything went OK:
if(glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES) != GL_FRAMEBUFFER_COMPLETE_OES)
return false; // uh oh, something went wrong
Now we're all set up to draw!
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
// do drawing here
glBindFramebufferOES(GL_FRAMEBUFFER_OES, 0);
And finally, clean up the frame buffer, if you don't need it any more:
glDeleteFramebuffersOES(1, &framebuffer);
Some caveats:
The size of the frame buffer must be a power-of-two.
You can go up to 1024x1024 on the latest iPhone, but there may be no need to have that level of detail.
Offhand I don't know the exact size that'll be available on a particular model of iPhone, but the general idea would be to use a Frame Buffer Object (FBO) to render to a texture, then you can blit pieces from that texture to the screen buffer.
Related
I have a game which runs pretty well (55-60fps) on a retina display.
I want to add a fullscreen overlay that blends with the existing scene. However, even when using a small texture, the performance hit is huge. Is there an optimization I can perform to make this useable?
If I use a 80x120 texture (the texture is rendered on the fly, which is why it's not square), I get 25-30FPS. If I make the texture smaller, performance increases, but quality is not acceptable. In general, though, the quality of the overlay is not very important (it's just lighting).
Renderer utilization is at 99%.
Even if I use a square texture from a file (.png), performance is bad.
This is how I create the texture:
[EAGLContext setCurrentContext:context];
// Create default framebuffer object.
glGenFramebuffers(1, &lightFramebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, lightFramebuffer);
// Create color render buffer and allocate backing store.
glGenRenderbuffers(1, &lightRenderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, lightRenderbuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8_OES, LIGHT_WIDTH, LIGHT_HEIGHT);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, lightRenderbuffer);
glGenTextures(1, &lightImage);
glBindTexture(GL_TEXTURE_2D, lightImage);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, LIGHT_WIDTH, LIGHT_HEIGHT, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, lightImage, 0);
And here is the rendering...
/* Draw scene... */
glBlendFunc(GL_ONE, GL_ONE);
//Switch to offscreen texture buffer
glBindFramebuffer(GL_FRAMEBUFFER, lightFramebuffer);
glBindRenderbuffer(GL_RENDERBUFFER, lightRenderbuffer);
glViewport(0, 0, LIGHT_WIDTH, LIGHT_HEIGHT);
glClearColor(ambientLight, ambientLight, ambientLight, ambientLight);
glClear(GL_COLOR_BUFFER_BIT);
/* Draw lights to texture... */
//Switch back to main frame buffer
glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);
glViewport(0, 0, framebufferWidth, framebufferHeight);
glBlendFunc(GL_DST_COLOR, GL_ZERO);
glBindTexture(GL_TEXTURE_2D, glview.lightImage);
/* Set up drawing... */
glDrawElements(GL_TRIANGLE_FAN, 4, GL_UNSIGNED_SHORT, 0);
Here are some benchmarks I took when trying to narrow down the problem. 'No blend' means I glDisable(GL_BLEND) before I draw the quad. 'No buffer switching' means I don't switch back and forth from the offscreen buffer before drawing.
(Tests using a static 256x256 .png)
No blend, No buffer switching: 52FPS
Yes blend, No buffer switching: 29FPS //disabled the glClear, which would artificially speed up the rendering
No blend, Yes buffer switching: 29FPS
Yes blend, Yes buffer switching: 27FPS
Yes buffer switching, No drawing: 46FPS
Any help is appreciated. Thanks!
UPDATE
Instead of blending the whole lightmap afterward, I ended up writing a shader to do the work on the fly. Each fragment samples and blends from the lightmap (kind of like multitexturing). At first, the performance gain was minimal, but then I used a lowp sampler2d for the light map, and then I got around 45FPS.
Here's the fragment shader:
lowp vec4 texColor = texture2D(tex, texCoordsVarying);
lowp vec4 lightColor = texture2D(lightMap, worldPosVarying);
lightColor.rgb *= lightColor.a;
lightColor.a = 1.0;
gl_FragColor = texColor * color * lightColor;
Ok I think you've run up against the limitations of the hardware. Blending a screen-sized quad over the whole scene is probably a particularly bad case for the tile-based hardware.
The PowerVR SGX (on the iPhone) is optimized for hidden surface removal, to avoid drawing things when not needed. It has low memory bandwidth because it's optimized for low power device.
So screen-sized blended quad is reading then writing every fragment on the screen. Ouch!
The glClear speed up is related - because you're telling GL you don't care about the contents of the backbuffer before rendering, which saves loading the previous contents into memory.
There's a very good overview of the iOS hardware here: http://www.imgtec.com/factsheets/SDK/POWERVR%20SGX.OpenGL%20ES%202.0%20Application%20Development%20Recommendations.1.1f.External.pdf
As for an actual solution - I would try directly rendering your overlay on the game scene.
For example, your render loop should look like:
[EAGLContext setCurrentContext:context];
// Set up game view port and render the game
InitGameViewPort();
GameRender();
// Change camera to 2d/orthographic, turn off depth write and compare
InitOverlayViewPort()
// Render overlay into same buffer
OverlayRender()
If you render to a render target on a PowerVR chip, switch to another render target and render, then switch back to any previous render target you will suffer a major performance hit. This kind of access pattern is labelled a "Logical Buffer Load" by the OpenGL ES Analyzer built into the latest Instruments.
If you switch your rendering order so that you draw your lightmap render target first, then render your scene to the main framebuffer, then do your fullscreen blend of the lightmap render target texture your performance should be much higher.
I can confirm, on iPad 1 using iOS 4.2, enable/disable GL_BLEND for one full screen quad toggled between 18 and 31 fps. In both runs, renderer utilization was 90-100%.
Even before fiddling with the texture, make sure your shader is optimized. When filling a 960x640 screen (614400 pixels) any operation in the fragment shader has a huge impact.
One good thing to create a specific version of your fragment shader for this situation. It should be something like this:
varying mediump vec2 vertexTexCoord;
uniform sampler2D texture;
void main() {
gl_FragColor = texture2D(texture, vertexTexCoord);
}
Create another program with this fragment shader and use it before drawing your big quad, then restore the normal program. The iPhone 4 is able to render about 7 full-screen, 1:1 textured quads per frame with blending, but it quickly drops to about 1 with a more sophisticated shader.
(Additionally in your case, try to render your overlay texture first, then the normal elements, then the texture over the rest. It should improve performance by a significant margin.)
iPhone OpenGL ES 2.0..
First frame, render to my framebuffer then present it (as it works by default in the template OpenGL ES application).
On the next frame, I want to use that rendered framebuffer as an input in to my shaders, while rendering to another framebuffer and presenting that 2nd framebuffer.
The next frame, I want to use framebuffer2 as input in to my shaders, while rendering to the first framebuffer again.
Repeat
How do I do this?
You should be able to set up a renderbuffer that has a texture backing it using code like the following:
// Offscreen position framebuffer object
glGenFramebuffers(1, &positionFramebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, positionFramebuffer);
glGenRenderbuffers(1, &positionRenderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, positionRenderbuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8_OES, FBO_WIDTH, FBO_HEIGHT);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, positionRenderbuffer);
// Offscreen position framebuffer texture target
glGenTextures(1, &positionRenderTexture);
glBindTexture(GL_TEXTURE_2D, positionRenderTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, FBO_WIDTH, FBO_HEIGHT, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, positionRenderTexture, 0);
Switching buffers is as simple as using code like this:
glBindFramebuffer(GL_FRAMEBUFFER, positionFramebuffer);
glViewport(0, 0, FBO_WIDTH, FBO_HEIGHT);
You can then render to that buffer and display the resulting texture by passing it into a simple shader that displays it within a rectangular piece of geometry. That texture can also be fed into a shader which renders into another similar renderbuffer that is backed by a texture, and so on.
If you need to do some CPU-based processing or readout, you can use glReadPixels() to pull in the pixels from this offscreen renderbuffer.
For examples of this, you can try out my sample applications here and here. The former does processing of video frames from a camera, with one of the settings allowing for a passthrough of the video while doing processing in an offscreen renderbuffer. The latter example renders into a cube map texture at one point, then uses that texture to do environment mapping on a teapot.
I can comfortably render a scene to a texture and map that texture back onto a framebuffer for screen display. But what if I wanted to map the texture back onto itself in order to blur it (say, at a quarter opacity in a new location). Is that possible?
The way I've done it is simply to enable the texture:
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, color_tex);
And then draw to it:
glVertexPointer(2, GL_FLOAT, 0, sv);
glTexCoordPointer(2, GL_FLOAT, 0, tcb1);
glColor4f (1.0f,1.0f,1.0f,0.25f);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
(some code omitted, obviously)
Is there anything obviously wrong with that idea? Am I being an idiot?
No, you can't write the texture to the same texture, that triggers undefined behaviour.
But you can use a technique called Ping-Pong rendering, so you draw the result of the operation into another texture, and if you need to do more processing, you write the result to the first texture.
I took the example of GLPaint... I'm trying to put a background into the "PaintingView", so you could draw over the background and finally save the image as a file..... I'm lost.
I'm loading the PNG (512x512) and try to "paint with it" at the very beginning of the program, but it's painted as 64x64 instead of 512x512...
I tried before to load is as a subview of the painting view... but then, glReadPixels doesn't work as expected (it only take in consideration the PaintingView, not the subview). Also the PaintingView doesn't have a method as initWithImage... I NEED glReadPixels work on the image (and in the modification) but i really don't know why when i load it, the texture has a 64x64 size..
The GLPaint example project uses GL_POINT_SPRITE to draw copies of the brush texture as you move the brush. On the iPhone, the glPointSize is limited to 64x64 pixels. This is a hardware limitation, and in the simulator I think you can make it larger.
It sounds like you're trying to use a GL_POINT_SPRITE method to draw your background image, and that's really not what you want. Instead, try drawing a flat, textured box that fills the screen.
Here's a bit of OpenGL code that sets up vertices and texcoords for a 2D box and then draws it:
const GLfloat verticies[] = {
0.0f, 0.0f,
1.0f, 0.0f,
0.0f, 1.0f,
1.0f, 1.0f,
};
const GLfloat texcoords[] = {
0, 0,
1, 0,
0, 1,
1, 1,
};
glVertexPointer(2, GL_FLOAT, 0, verticies);
glEnableClientState(GL_VERTEX_ARRAY);
glTexCoordPointer(2, GL_FLOAT, 0, texcoords);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
Hope that helps! Note that you need to specify the vertices differently depending on how your camera projection is set up. In my case, I set up my GL_MODELVIEW using the code below - I'm not sure how the GLPaint example does it.
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glOrtho(0, 1.0, 0, 1.0, -1, 1);
First, glReadPixels() is only going to see whatever framebuffer is associated with your current OpenGL context. That might explain why you're not getting the pixels you expect.
Second, what do you mean by the texture being rendered at a specific pixel size? I assume the texture is rendered as a quad, and then the size of that quad ought to be under your control, code-wise.
Also, check that the loading of the texture doesn't generate an OpenGL error, I'm not sure what the iPhone's limitations on texture sizes are. It's quite conceivable that 512x512 is out of range. You could of course investigate this yourself, by calling glGetIntegerv() and using the GL_MAX_TEXTURE_SIZE constant.
Normally, you'd use something like:
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glEnable(GL_BLEND);
glEnable(GL_LINE_SMOOTH);
glLineWidth(2.0f);
glVertexPointer(2, GL_FLOAT, 0, points);
glEnableClientState(GL_VERTEX_ARRAY);
glDrawArrays(GL_LINE_STRIP, 0, num_points);
glDisableClientState(GL_VERTEX_ARRAY);
It looks good in the iPhone simulator, but on the iPhone the lines get extremely thin and w/o any anti aliasing.
How do you get AA on iPhone?
One can achieve the effect of anti aliasing very cheaply using vertices with opacity 0.
Here's an image example to explain:
Comparison with AA:
You can read a paper about this here:
http://research.microsoft.com/en-us/um/people/hoppe/overdraw.pdf
You could do something along this way:
// Colors is a pointer to unsigned bytes (4 per color).
// Should alternate in opacity.
glColorPointer(4, GL_UNSIGNED_BYTE, 0, colors);
glEnableClientState(GL_COLOR_ARRAY);
// points is a pointer to floats (2 per vertex)
glVertexPointer(2, GL_FLOAT, 0, points);
glEnableClientState(GL_VERTEX_ARRAY);
glDrawArrays(GL_TRIANGLE_STRIP, 0, points_count);
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_COLOR_ARRAY);
Starting in iOS Version 4.0 you have an easy solution, it's now possible to use Antialiasing for the whole OpenGL ES scene with just a few lines of added code. (And nearly no performance loss, at least on the SGX GPU).
For the code please read the following Apple Dev-Forum Thread.
There are also some sample pictures how it looks for me on my blog.
Using http://answers.oreilly.com/topic/1669-how-to-render-anti-aliased-lines-with-textures-in-ios-4/ as a starting point, I was able to get anti-aliased lines like these:
They aren't perfect nor are they as nice as the ones that I had been drawing with Core Graphics, but they are pretty good. I am actually drawing same lines (vertices) twice - once with bigger texture and color, then with smaller texture and translucent white.
There are artifacts when lines overlap too tightly and alphas start to accumulate.
One approach around this limitation is tessellating your lines into textured triangle strips (as seen here).
The problem is that on the iPhone OpenGl renders to a frame buffer object rather than the main frame buffer and as I understand it FBO's don't support multisampling.
There are various tricks that can be done, such as rendering to another FBO at twice the display size and then relying on texture filtering to smooth things out, not something that I've tried though so can't comment on how well this works.
I remember very specifically that I tried this and there is no simple way to do this using OpenGL on the iPhone. You can draw using CGPaths and a CGContextRef, but that will be significantly slower.
Put this in your render method and setUpFrame buffer...
You will get anti-aliased appearance.
/*added*/
//[_context presentRenderbuffer:GL_RENDERBUFFER];
//Bind both MSAA and View FrameBuffers.
glBindFramebuffer(GL_READ_FRAMEBUFFER_APPLE, msaaFramebuffer);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER_APPLE, framebuffer );
// Call a resolve to combine both buffers
glResolveMultisampleFramebufferAPPLE();
// Present final image to screen
glBindRenderbuffer(GL_RENDERBUFFER, _colorRenderBuffer);
[_context presentRenderbuffer:GL_RENDERBUFFER];
/*added*/