I am new in OpenGL ES. I earlier developed games in cocoa with objective-c. Now I want to develope 3D game for iPhone by using OpenGL ES. I am at beginner stage. I am able to create triangle, square, cube, pyramid by usng OpenGL ES. But if we have any .png image with us and we have to render it in our game then what we have to do? For that we require any other tool like unity or what? I cant able to understand it exactly. or we have to do it like the GLSprite example which is given in apple.developer.com site. In that example they draw a tea pot by using one teapot.h file. teapot.h file contain some points and by using that points they plot triangle which formed a tea pot. So is this a way to draw any image. I think I am thinking in wrong direction so please guide me for this.
Thank you in advance
To draw an image you need to first define the geometry that the image can be applied to:
float w = width / 2;
float h = height / 2;
float x = 10.0f;
float y = 10.0f;
float z = 0.0f;
float scaleX = 1.0f;
float scaleY = 1.0f;
float scaleZ = 1.0f;
const GLfloat squareVertices[] = {
-w, -h,
w, -h,
-w, h,
w, h,
};
const GLfloat textureCoords[] = {
0, 0,
1, 0,
0, 1,
1, 1,
};
Then, you can apply your texture and render this geometry:
glEnable(GL_TEXTURE_2D);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glEnableClientState(GL_VERTEX_ARRAY);
// apply texture -- how to create and bind a physical image is a whole different question
glBindTexture(GL_TEXTURE_2D, yourTextureID);
glVertexPointer(2, GL_FLOAT, 0, squareVertices);
glTexCoordPointer(2, GL_FLOAT, 0, textureCoords);
glPushMatrix();
glTranslatef(x, y, z);
glScalef(scaleX, scaleY, scaleZ);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glPopMatrix();
glDisable(GL_TEXTURE_2D);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
glDisableClientState(GL_VERTEX_ARRAY);
Note that these code-snippets assume that you have setup a working view/projection.
The Crash Landing sample that used to be in the SDK is also a good place to start (apparently the audio code was broken and that's why it was removed, but the OpenGL code is still good)
You can find a link to the sample here.
Related
I have an OpenGL ES Shader fragment shader that is using the mix function to place an overlay over a videoframe. This is my shader:
#ifdef GL_ES
precision mediump float;
#endif
varying vec2 textureCoordinate;
uniform sampler2D videoFrameY;
uniform sampler2D videoFrameUV;
uniform sampler2D overlay;
const mat3 yuv2rgb = mat3(
1, 1, 1,
0, -.21482, 2.12798,
1.28033, -.38059, 0
);
void main() {
vec3 yuv;
vec4 ovr;
yuv.x = texture2D(videoFrameY, textureCoordinate).r;
yuv.yz = texture2D(videoFrameUV, textureCoordinate).rg - vec2(0.5, 0.5);
ovr = texture2D(overlay, textureCoordinate);
vec3 rgb = yuv2rgb * yuv;
gl_FragColor = mix(ovr, vec4(rgb, 1.0), ovr.a);
}
Without the overlay texture, feeding gl_FragColor this:
gl_FragColor = vec4(rgb, 1.0);
works just fine and my video is displayed. Now I'm creating my overlay texture from a CATextLayer like this:
- (void)generateOverlay {
CATextLayer *textLayer = [CATextLayer layer];
[textLayer setString:#"Sample test string"];
[textLayer setFont:(__bridge CFStringRef)#"Helvetica"];
[textLayer setFontSize:(_videoHeight / 6)];
[textLayer setAlignmentMode:kCAAlignmentLeft];
[textLayer setBounds:CGRectMake(0, 0, _videoWidth, _videoHeight)];
CGSize layerSize = textLayer.bounds.size;
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
void *imageData = malloc(layerSize.height * layerSize.width * 4);
CGContextRef context = CGBitmapContextCreate(imageData, layerSize.width, layerSize.height, 8, 4 * layerSize.width, colorspace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorspace);
CGContextClearRect(context, CGRectMake(0, 0, layerSize.width, layerSize.height));
CGContextTranslateCTM(context, 0, layerSize.height - layerSize.height);
[textLayer renderInContext:context];
glActiveTexture(GL_TEXTURE2);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, layerSize.width, layerSize.height, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageData);
CGContextRelease(context);
free(imageData);
}
The problem is the results are inverted. So therefore they look like this:
(source: c.tro.pe)
The fact is they are are inverted in two ways. First, instead of the blue being alpha'ed out and the video showing through, the text is the alpha and that is where the video is showing through. Second, the text is mirrored. The mirroring could be the results of the vertex values being used however the video is correct using the same coords. I'm sure this is a quick rearrange but I'm not sure what to tweak. Thanks!
For the mix, the mix(x, y, a) function interpolates between x and y based on a. a of 0 gives you all x, and a of 1.0 gives you all y. You're keying off of the alpha of your text layer, so for the overlay you want, you need to reverse your ordering as follows:
gl_FragColor = mix(vec4(rgb, 1.0), ovr, ovr.a);
In regards to the rotation, remember that the iOS rear cameras are mounted landscape left and the front cameras landscape right, so for a portrait orientation you need to rotate the incoming video frames. You appear to be performing that rotation in either your vertices or texture coordinates. You're going to need a second set of texture coordinates that aren't rotated to use for sampling your overlay image, or you'll need to draw your label at a matching landscape left rotation when generating its texture.
I'm trying to draw a simple circle using OpenGL ES. The problem is that the circle is stretched vertically. It looks more like an ellipse than a circle. Could someone point out where the things are going wrong?
I played around with glViewPort to fix this but was not successful. As someone else suggested here on Stackoverflow, I also tried loading a different matrix instead of the identity matrix and that doesn't work too...
Here's the code of drawFrame:
- (void)drawFrame
{
[(EAGLView *)self.view setFramebuffer];
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
GLfloat vertices[720];
for (int i = 0; i < 720; i += 2)
{
vertices[i] = (cos(degreesToRadians(i)) * 1);
vertices[i+1] = (sin(degreesToRadians(i)) * 1);
}
glVertexPointer(2, GL_FLOAT, 0, vertices);
glEnableClientState(GL_VERTEX_ARRAY);
glColor4f(1.0f, 0.0f, 0.0f, 1.0f);
glDrawArrays(GL_TRIANGLE_FAN, 0, 360);
[(EAGLView *)self.view presentFramebuffer];
}
The code you are showing will draw a perfect circle in world co-ordinates. What you need to consider is how those world co-ordinates transform into window co-ordinates i.e. pixels.
If the glViewport is set to always match the window then it's the aspect ratio of the window that will determine what you see using the code sample you have shown. If the window is square it will work i.e. you will see a perfect circle. If the window is taller than it is wide then the circle will be stretched vertically.
To preserve the perfect circle you can use a projection matrix that gives you a viewing volume of the same aspect ratio as the viewport/window. I noticed that before your first edit you had a call to glOrthof in there. Set the aspect ratio to match there and that will do the job for you. If you want a perspective projection instead of an orthographic projection then use glFrustum.
To define what I'm trying to do: I want to be able to take an arbitrary 'sprite' image from a ^2x^2 sized PNG, and display just the pixels of interest to a given x/y position on screen.
My results are the problem - major distortion - it looks awful! (Note these SS's are in iPhone sim but on real retina device they appear the same.. junky). Here is a screenshot of the source PNG in 'preview' - which looks wonderful (any variations on rendering that I describe in this question look almost exactly like the junky one)
Previously, I've asked a question about displaying a non-power-of-2 texture as a sprite using OpenGL ES 2.0 (although this applies to any OpenGL). I'm close, but I have some issues that I can't resolve. I think there are probably multiple bugs - I think there's some bug where I'm basically aliasing what I'm displaying by rendering large then squashing x2 or vice versa but I can't see it. Additionally, there are off by one errors and I cannot get a handle on them. I can't visually identify them occurring but I know for sure they're there.
I'm working in 960 x 640 landscape (on iPhone4 retina display). So I expect 0->959 moves left to right, 0->639 moves bottom to top. (And I think I'm seeing opposite of this - but that's not what this question is about)
To make things easy what I'm trying to achieve in this test case is a FULL SCREEN 960x640 display of a PNG file. Just one of them. I display a red background first so that it's obvious if I'm covering the screen or not.
Update: I realized the 'glViewport' inside of the setFramebuffer call was setting my width and height backwards. I noticed this because when I would set my geometry to draw from 0,0 to 100,100 it drew in a rectangle not a square. When I swapped these, that call does draw a square. However, using that same call, my entire screen fills with vertex range of 0,0 -> 480,320 (half 'resolution').. don't understand that. However no matter where I push on from this, I'm still not getting a good looking result
Here's my vertex shader:
attribute vec4 a_position;
attribute vec2 a_texCoord;
varying vec2 v_texCoord;
// Gives 'landscape' full screen..
mat4 projectionMatrix = mat4( 2.0/640.0, 0.0, 0.0, -1.0,
0.0, 2.0/960.0, 0.0, -1.0,
0.0, 0.0, -1.0, 0.0,
0.0, 0.0, 0.0, 1.0);
// Gives a 1/4 of screen.. (not doing 2.0/.. was suggested in previous SO Q)
/*mat4 projectionMatrix = mat4( 1.0/640.0, 0.0, 0.0, -1.0,
0.0, 1.0/960.0, 0.0, -1.0,
0.0, 0.0, -1.0, 0.0,
0.0, 0.0, 0.0, 1.0); */
// Apply the projection matrix to the position and pass the texCoord
void main()
{
gl_Position = a_position;
gl_Position *= projectionMatrix;
v_texCoord = a_texCoord;
}
Here's my fragment shader:
precision mediump float;
varying vec2 v_texCoord;
uniform sampler2D s_texture;
void main()
{
gl_FragColor = texture2D(s_texture, v_texCoord);
}
Here's my draw code:
#define MYWIDTH 960.0f
#define MYHEIGHT 640.0f
// I have to refer to 'X' as height although I'd assume I use 'Y' here..
// I think my X and Y throughout this whole block of code is screwed up
// But, I have experimented flipping them all and verifying that if they
// Are taken from the way they're set now to swapping X and Y that things
// end up being turned the wrong way. So this is a mess, but unlikely my problem
#define BG_X_ORIGIN 0.0f
// ALSO NOTE HERE: I have to put my 'dest' at 640.0f.. --- see note [1] below
#define BG_X_DEST 640.0f
#define BG_Y_ORIGIN 0.0f
// --- see note [1] below
#define BG_Y_DEST 960.0f
// These are texturing coordinates, I texture starting at '0' px and then
// I calculate a percentage of the texture to use based on how many pixels I use
// divided by the actual size of the image (1024x1024)
#define BG_X_ZERO 0.0f
#define BG_Y_USEPERCENTAGE BG_X_DEST / 1023.0f
#define BG_Y_ZERO 0.0f
#define BG_X_USEPERCENTAGE BG_Y_DEST / 1023.0f
// glViewport(0, 0, MYWIDTH, MYHEIGHT);
// See note 2.. it sets glViewport basically, provided by Xcode project template
[(EAGLView *)self.view setFramebuffer];
// Big hack just to get things going - like I said before, these could be backwards
// w/respect to X and Y
static const GLfloat backgroundVertices[] = {
BG_X_ORIGIN, BG_Y_ORIGIN,
BG_X_DEST, BG_Y_ORIGIN,
BG_X_ORIGIN, BG_Y_DEST,
BG_X_DEST, BG_Y_DEST
};
static const GLfloat backgroundTexCoords[] = {
BG_X_ZERO, BG_Y_USEPERCENTAGE,
BG_X_USEPERCENTAGE, BG_Y_USEPERCENTAGE,
BG_X_ZERO, BG_Y_ZERO,
BG_X_USEPERCENTAGE, BG_Y_ZERO
};
// Turn on texturing
glEnable(GL_TEXTURE_2D);
// Clear to RED so that it's obvious when I'm not drawing my sprite on screen
glClearColor(1.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
// Texturing parameters - these make sense.. don't think they are the issue
glActiveTexture(GL_TEXTURE0);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);//GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_NEAREST);//GL_LINEAR);
// Update attribute values.
glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, backgroundVertices);
glEnableVertexAttribArray(ATTRIB_VERTEX);
glVertexAttribPointer(ATTRIB_TEXCOORD, 2, GL_FLOAT, 0, 0, backgroundTexCoords);
glEnableVertexAttribArray(ATTRIB_TEXCOORD);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, background->textureId);
// I don't understand what this uniform does in the texture2D call in shader.
glUniform1f(uniforms[UNIFORM_SAMPLERLOC], 0);
// Draw the geometry...
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
// present the framebuffer see note [3]
[(EAGLView *)self.view presentFramebuffer];
Note [1]:
If I set BG_X_DEST to 639.0f I do not get full coverage of the 640 pixels, I get red showing through on the right hand side. But this doesn't make sense to me - I'm aiming for pixel perfect and I have to draw my sprite geometry from 0 to 640 which is 641 pixels when I only have 640!!! red line appearing with 639f instead of 640f
And if I set BG_Y_DEST to 959.0f I do not get the red line show throug.
red line top bug appearing with 958f instead of 960 or 959f
This may be a good clue as to what bug(s) I have going on.
Note: [2] - included in the OpenGL ES 2 framework by Xcode
- (void)setFramebuffer
{
if (context)
{
[EAGLContext setCurrentContext:context];
if (!defaultFramebuffer)
[self createFramebuffer];
glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
glViewport(0, 0, framebufferWidth, framebufferHeight);
}
}
Note [3]: - included in the OpenGL ES 2 framework by Xcode
- (BOOL)presentFramebuffer
{
BOOL success = FALSE;
if (context)
{
[EAGLContext setCurrentContext:context];
glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);
success = [context presentRenderbuffer:GL_RENDERBUFFER];
}
return success;
}
Note [4] - relevant image loading code (I have used PNG with and without alpha channel and actually it doesn't seem to make any difference... I also have tried to change my code up to be ARGB instead of RGBA and that's wrong - since A = 1.0 everywhere, I get a very RED image, which makes me think the RGBA is in fact valid and this code is right.): update: I have switched this texture loading to a completely different setup using CG/ImageIO calls and it looks identical to this so I assume it's not some kind of aliasing or color compression done by the image libraries (unless they both go to the same fundamental calls, which is possible..)
// Otherwise it isn't already loaded
glEnable(GL_TEXTURE_2D);
glActiveTexture(GL_TEXTURE0);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_NEAREST);//GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_NEAREST);//GL_LINEAR);
// TODO Next 2 can prob go later on..
glGenTextures(1, &(newTexture->textureId)); // generate Texture
// Use this before 'drawing' the texture to the memory...
glBindTexture(GL_TEXTURE_2D, newTexture->textureId);
NSString *path = [[NSBundle mainBundle]
pathForResource:[NSString stringWithUTF8String:newTexture->filename.c_str()] ofType:#"png"];
NSData *texData = [[NSData alloc] initWithContentsOfFile:path];
UIImage *image = [[UIImage alloc] initWithData:texData];
if (image == nil)
NSLog(#"Do real error checking here");
newTexture->width = CGImageGetWidth(image.CGImage);
newTexture->height = CGImageGetHeight(image.CGImage);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
void *imageData = malloc(newTexture->height * newTexture->width * 4 );
CGContextRef myContext = CGBitmapContextCreate
(imageData, newTexture->width, newTexture->height, 8, 4 * newTexture->width, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big );
CGColorSpaceRelease(colorSpace);
CGContextClearRect(myContext, CGRectMake(0, 0, newTexture->width, newTexture->height));
CGContextDrawImage(myContext, CGRectMake(0, 0, newTexture->width, newTexture->height), image.CGImage);
// Texture is created!
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, newTexture->width, newTexture->height, 0,
GL_RGBA, GL_UNSIGNED_BYTE, imageData);
CGContextRelease(myContext);
free(imageData);
[image release];
[texData release];
[(EAGLView *)self.view setContentScaleFactor:2.0f];
By default, iPhone windows do scaling to reach their high resolution modes. Which was destroying my image quality ..
Thanks for all the help folks
Whats the recommended way of displaying a background image in an OpenGL ES app.
UIImage added to view hierarchy underneath glview?
image drawn in OpenGL draw routine from texture in memory?
Are there any performance / memory considerations to take account of?
I like to load a texture using Apple's Texture2D class. Create a quad (two triangles, six vertices) that maps to the corners of the screen. Then you map the texture coordinates to the vertices, pass that to glDrawArrays and go. I actually have a Class that takes a frame in CG Coordinates, converts that to OpenGL Coordinates and then draws the image in the frame.
Let's say you have a struct like this:
struct {
GLfloat x; // OpenGL X Coordinate
GLfloat y; // OpenGL Y Coordinate
GLfloat z; // OpenGL Z Coordinate
GLfloat s; // Texture S Coordinate
Glfloat t; // Texture T Coordinate
} vertexData;
And an array of vertex data like this:
struct vertexData verts[6];
And you have a Texture2D object:
texture = [[Texture2D alloc] initWithImage:[UIImage imageNamed:image] filter:filter pixelFormat:pixelFormat];
So, now you have to fill in the vertex array. Assuming 3D, x range [-1, 1] and y range [-1, 1], origin for z, then you would initialize your vertices like this:
verts[0].x = verts[1].x = verts[5].x = -1.0;
verts[2].x = verts[3].x = verts[4].x = 1.0;
verts[0].y = verts[2].y = verts[4].y = 1.0;
verts[1].y = verts[3].y = verts[5].y = -1.0;
verts[0].z = verts[1].z = verts[2].z = 0.0;
verts[3].z = verts[4].z = verts[5].z = 0.0;
verts[0].s = verts[1].s = verts[5].s = tlxf;
verts[2].s = verts[3].s = verts[4].s = trxf;
verts[0].t = verts[2].t = verts[4].t = ttyf;
verts[1].t = verts[3].t = verts[5].t = tbyf;
Finally, you need to draw:
glBindTexture(GL_TEXTURE_2D, texture.name);
glVertexPointer(3, GL_FLOAT, sizeof(struct vertexData), &verts[0].x);
glTexCoordPointer(2, GL_FLOAT, sizeof(struct vertexData), &verts[0].s);
glDrawArrays(GL_TRIANGLES, 0, 6);
I'm leaving out any details about setting up OpenGL. I assume you've done that if you're getting this far.
There are tons of performance and memory constraints to think about when working in OpenGL, but I wouldn't recommend getting too worried about them just yet. Once you get it working, profile it with the OpenGL Instrument to see if have any performance issues, then deal with those.
Apple has a really good document describing best practices, and there are some useful questions on SO as well. I'm sure you will be able to search for them once you know what you're up against (if anything).
glDrawTexOES the best choice.
I'm working with the iPhone OpenGLES implementation and I wish to endlessly scroll a texture across a simple surface (two triangles making up a rectangle). This should be straightforward, but it's not something I've done before and I must be missing something. I can rotate the texture fine, but translate does not work at all. Do I have a minor implementation issue or am I doing something fundamentally wrong?
// move texture
glMatrixMode(GL_TEXTURE);
glPushMatrix();
glLoadIdentity();
// increment offset - no reset for demo purposes
wallOffset += 1.0;
// move the texture - this does not work
glTranslatef(wallOffset,wallOffset,0.0);
// rotate the texture - this does work
//glRotatef(wallOffset, 1.0, 0.0, 0.0);
glMatrixMode(GL_MODELVIEW);
glBindTexture(GL_TEXTURE_2D, WallTexture.name);
glTexCoordPointer(2, GL_FLOAT, 0, coordinates);
// simple drawing code
glNormalPointer(GL_FLOAT, 0, normals);
glVertexPointer(3, GL_FLOAT, 0, vertices);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
// push matrix back
glMatrixMode(GL_TEXTURE);
glPopMatrix();
glMatrixMode(GL_MODELVIEW);
You're incrementing your texture offset by 1.0f; but textures coordinates are considered in the range [0, 1], so you're not actually changing the texture coordinates (assuming you've enabled some sort of wrapping).
Try changing that increment (try .01f, or maybe something depending on the framerate) and see if it works. If not, then it may have something to do with the texture parameters you've got enabled.