OpenGL ES recommended way of displaying a background image - iphone

Whats the recommended way of displaying a background image in an OpenGL ES app.
UIImage added to view hierarchy underneath glview?
image drawn in OpenGL draw routine from texture in memory?
Are there any performance / memory considerations to take account of?

I like to load a texture using Apple's Texture2D class. Create a quad (two triangles, six vertices) that maps to the corners of the screen. Then you map the texture coordinates to the vertices, pass that to glDrawArrays and go. I actually have a Class that takes a frame in CG Coordinates, converts that to OpenGL Coordinates and then draws the image in the frame.
Let's say you have a struct like this:
struct {
GLfloat x; // OpenGL X Coordinate
GLfloat y; // OpenGL Y Coordinate
GLfloat z; // OpenGL Z Coordinate
GLfloat s; // Texture S Coordinate
Glfloat t; // Texture T Coordinate
} vertexData;
And an array of vertex data like this:
struct vertexData verts[6];
And you have a Texture2D object:
texture = [[Texture2D alloc] initWithImage:[UIImage imageNamed:image] filter:filter pixelFormat:pixelFormat];
So, now you have to fill in the vertex array. Assuming 3D, x range [-1, 1] and y range [-1, 1], origin for z, then you would initialize your vertices like this:
verts[0].x = verts[1].x = verts[5].x = -1.0;
verts[2].x = verts[3].x = verts[4].x = 1.0;
verts[0].y = verts[2].y = verts[4].y = 1.0;
verts[1].y = verts[3].y = verts[5].y = -1.0;
verts[0].z = verts[1].z = verts[2].z = 0.0;
verts[3].z = verts[4].z = verts[5].z = 0.0;
verts[0].s = verts[1].s = verts[5].s = tlxf;
verts[2].s = verts[3].s = verts[4].s = trxf;
verts[0].t = verts[2].t = verts[4].t = ttyf;
verts[1].t = verts[3].t = verts[5].t = tbyf;
Finally, you need to draw:
glBindTexture(GL_TEXTURE_2D, texture.name);
glVertexPointer(3, GL_FLOAT, sizeof(struct vertexData), &verts[0].x);
glTexCoordPointer(2, GL_FLOAT, sizeof(struct vertexData), &verts[0].s);
glDrawArrays(GL_TRIANGLES, 0, 6);
I'm leaving out any details about setting up OpenGL. I assume you've done that if you're getting this far.
There are tons of performance and memory constraints to think about when working in OpenGL, but I wouldn't recommend getting too worried about them just yet. Once you get it working, profile it with the OpenGL Instrument to see if have any performance issues, then deal with those.
Apple has a really good document describing best practices, and there are some useful questions on SO as well. I'm sure you will be able to search for them once you know what you're up against (if anything).

glDrawTexOES the best choice.

Related

Box2d fixture and body out of sync on retina display

I'm trying to make a cocos2d/box2d game work on iPad, iPhone and iPhone retina.
My problem is, that the fixture and body don't line up on the retina simulator, please click on screenshots below for illustration (as a new stackoverflow member, it won't allow me to post the screenshot here).
screenshot
(please disregard the different shapes, I want the 4 corners to line up)
I've done quite a bit of research on this over the last couple of days, and the closest I found was this:
link
But the solution offered there with PTM_RATIO and CC_CONTENT_SCALE_FACTOR() doesn't seem to work in my case. I think it has to do with the fact that I don't load an image from file into my sprite. Most solutions to this problem are based on loading -hd image files for the retina display, but I don't want to use files in my game at all. I basically want to draw the polygons myself at runtime,
My code looks as follows:
-(CCSprite*)addSprite
{
CGSize contextsize = CGSizeMake(200, 200);
UIGraphicsBeginImageContext(contextsize);
CGContextRef context = UIGraphicsGetCurrentContext();
CGContextFlush(context);
CGContextSetAllowsAntialiasing(context, true);
CGContextTranslateCTM(context, 0, contextsize.height);
CGContextScaleCTM(context, 1.0, -1.0);
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
CGFloat components[] = {0.0, 0.0, 1.0, 1.0};
CGColorRef color = CGColorCreate(colorspace, components);
CGContextSetStrokeColorWithColor(context, color);
UIBezierPath* aPath;
aPath = [UIBezierPath bezierPathWithArcCenter:CGPointMake(100, 100)
radius:100
startAngle:0
endAngle:1.57
clockwise:YES];
[aPath addArcWithCenter:CGPointMake(100, 100)
radius:50
startAngle:1.57
endAngle:0
clockwise:NO];
[aPath stroke];
CGContextStrokePath(context);
CGColorSpaceRelease(colorspace);
CGColorRelease(color);
UIImage *graphImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
CCTexture2D *tex = [[[CCTexture2D alloc] initWithImage:graphImage] autorelease];
CCSprite *sprite = [CCSprite spriteWithTexture:tex];
return sprite;
}
-(void) addFixture:(CCSprite *)fixsprite
{
b2Vec2 arcdots[] = {
b2Vec2(50.0f / PTM_RATIO, 0.0f / PTM_RATIO),
b2Vec2(100.0f / PTM_RATIO, 0.0f / PTM_RATIO),
b2Vec2(0.0f / PTM_RATIO, 100.0f / PTM_RATIO),
b2Vec2(0.0f / PTM_RATIO, 50.0f / PTM_RATIO)
};
b2PolygonShape p_shape;
b2FixtureDef fixtureDef;
b2BodyDef bodyDef;
bodyDef.type = b2_kinematicBody;
bodyDef.position.Set(100/PTM_RATIO, 100/PTM_RATIO);
bodyDef.userData = fixsprite;
b2Body *body = world->CreateBody(&bodyDef);
p_shape.Set(arcdots, 4);
fixtureDef.shape = &p_shape;
fixtureDef.density = 1.0f;
fixtureDef.friction = 0.3f;
body->CreateFixture(&fixtureDef);
}
And I call these functions from the main routine as follows:
CCSprite *sprite2 = [self addSprite];
sprite2.position = ccp(0, 0);
[self addChild:sprite2 z:0];
[self addFixture:sprite2];
I have these lines uncommented in the delegate file:
if( ! [director enableRetinaDisplay:YES] )
CCLOG(#"Retina Display Not supported");
Please let me know if further information is required. And please be gentle, I'm only starting to learn this. Thanks for your time.
Unless otherwise mentioned, all coordinates in cocos2d (and most of UIKit) are given in points, not pixels. On a Retina display device you still have a point resolution of 480x320 points (960x640 pixels).
From that follows: when you calculate in actual pixels, multiply or divide by CC_CONTENT_SCALE_FACTOR. If you deal with point coordinates, do nothing. Since you're rendering your own polys I assume you know whether you use actual pixel coordinates or not. If you use OpenGL directly, then you'll be working with pixel coordinates.
I'm not sure if enabling Retina display mode does anything for you if you don't use cocos2d to render your content.
Lastly, a common misunderstanding is that the Box2D world is using point coordinates and must be transformed to pixels or vice versa. Neither is the case. The Box2D world is completely oblivious to a specific coordinate system. The use of PTM_RATIO is done only to ensure that Box2D coordinates are within reasonable ranges for the Box2D engine, since it works best with objects that are 1 meter in size/diameter, and most objects should range from 0.1 to 10 meters in diameter.

Mixing Multiple Textures in OpenGL ES Shader on iOS Results in Inverse Behavior

I have an OpenGL ES Shader fragment shader that is using the mix function to place an overlay over a videoframe. This is my shader:
#ifdef GL_ES
precision mediump float;
#endif
varying vec2 textureCoordinate;
uniform sampler2D videoFrameY;
uniform sampler2D videoFrameUV;
uniform sampler2D overlay;
const mat3 yuv2rgb = mat3(
1, 1, 1,
0, -.21482, 2.12798,
1.28033, -.38059, 0
);
void main() {
vec3 yuv;
vec4 ovr;
yuv.x = texture2D(videoFrameY, textureCoordinate).r;
yuv.yz = texture2D(videoFrameUV, textureCoordinate).rg - vec2(0.5, 0.5);
ovr = texture2D(overlay, textureCoordinate);
vec3 rgb = yuv2rgb * yuv;
gl_FragColor = mix(ovr, vec4(rgb, 1.0), ovr.a);
}
Without the overlay texture, feeding gl_FragColor this:
gl_FragColor = vec4(rgb, 1.0);
works just fine and my video is displayed. Now I'm creating my overlay texture from a CATextLayer like this:
- (void)generateOverlay {
CATextLayer *textLayer = [CATextLayer layer];
[textLayer setString:#"Sample test string"];
[textLayer setFont:(__bridge CFStringRef)#"Helvetica"];
[textLayer setFontSize:(_videoHeight / 6)];
[textLayer setAlignmentMode:kCAAlignmentLeft];
[textLayer setBounds:CGRectMake(0, 0, _videoWidth, _videoHeight)];
CGSize layerSize = textLayer.bounds.size;
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
void *imageData = malloc(layerSize.height * layerSize.width * 4);
CGContextRef context = CGBitmapContextCreate(imageData, layerSize.width, layerSize.height, 8, 4 * layerSize.width, colorspace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorspace);
CGContextClearRect(context, CGRectMake(0, 0, layerSize.width, layerSize.height));
CGContextTranslateCTM(context, 0, layerSize.height - layerSize.height);
[textLayer renderInContext:context];
glActiveTexture(GL_TEXTURE2);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, layerSize.width, layerSize.height, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageData);
CGContextRelease(context);
free(imageData);
}
The problem is the results are inverted. So therefore they look like this:
(source: c.tro.pe)
The fact is they are are inverted in two ways. First, instead of the blue being alpha'ed out and the video showing through, the text is the alpha and that is where the video is showing through. Second, the text is mirrored. The mirroring could be the results of the vertex values being used however the video is correct using the same coords. I'm sure this is a quick rearrange but I'm not sure what to tweak. Thanks!
For the mix, the mix(x, y, a) function interpolates between x and y based on a. a of 0 gives you all x, and a of 1.0 gives you all y. You're keying off of the alpha of your text layer, so for the overlay you want, you need to reverse your ordering as follows:
gl_FragColor = mix(vec4(rgb, 1.0), ovr, ovr.a);
In regards to the rotation, remember that the iOS rear cameras are mounted landscape left and the front cameras landscape right, so for a portrait orientation you need to rotate the incoming video frames. You appear to be performing that rotation in either your vertices or texture coordinates. You're going to need a second set of texture coordinates that aren't rotated to use for sampling your overlay image, or you'll need to draw your label at a matching landscape left rotation when generating its texture.

logos and subtitles in reverse using ffmpeg and OpenGL in iOS 5.0

I am using ffmpeg to play video on iOS 5.0. In my app with ffmpeg decoded video frames and use OpenGL to display it.
But I have a problem I don't resolve it. Chains logos and subtitles of the video image is displayed in reverse. I think that is the problem of rendering OpenGL 2.0 or ffmpeg decoded.
Can you tell me what is wrong?, and How I can fix it?
Very thanks,
Edit: I change my prepareTExture method with this:
- (void) prepareTextureW: (GLuint) texW textureHeight: (GLuint) texH frameWidth: (GLuint) frameW frameHeight: (GLuint) frameH {
float aspect = (float)frameW/(float)frameH;
float minX=-1.f, minY=-1.f, maxX=1.f, maxY=1.f;
float scale ;
if(aspect>=(float)backingHeight/(float)backingWidth){
// Aspect ratio will retain width.
scale = (float)backingHeight / (float) frameW;
maxY = ((float)frameH * scale) / (float) backingWidth;
minY = -maxY;
} else {
// Retain height.
scale = (float) backingWidth / (float) frameW;
maxX = ((float) frameW * scale) / (float) backingHeight;
minX = -maxX;
}
if(frameTexture) glDeleteTextures(1, &frameTexture);
glEnable(GL_TEXTURE_2D);
glGenTextures(1, &frameTexture);
glBindTexture(GL_TEXTURE_2D, frameTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT );
glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT );
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, texW, texH, 0, GL_RGB, GL_UNSIGNED_SHORT_5_6_5, NULL);
verts[0] = maxX;
verts[1] = maxY;
verts[2] = minX;
verts[3] = maxY;
verts[4] = maxX;
verts[5] = minY;
verts[6] = minX;
verts[7] = minY;
float s = (float) frameW / (float) texW;
float t = (float) frameH / (float) texH;
texCoords[0] = 0.f; texCoords[1] = 1.f;
texCoords[2] = 1; texCoords[3] = 1.f;
texCoords[4] = 0.f; texCoords[5] =0;
texCoords[6] = 1; texCoords[7] =0;
mFrameH = frameH;
mFrameW = frameW;
mTexH = texH;
mTexW = texW;
maxS = s;
maxT = t;
// Just supporting one rotation direction, landscape left. Rotate Z by 90 degrees.
matSetRotZ(&rot,M_PI_2);
matMul(&mvp, &rot, &rot);
[self setupShader];
}
And now this is my result: link image
But I have a problem I don't resolve it. Chains logos and subtitles of the video image is displayed in reverse.
The whole image is mirrored, not just chain logo and subtitles. Looks like wrong texture coordinates to me. Could you please post your drawing code?
EDIT due to question update
Phew I first had to understand what you do there, it's overly complicated. Just use texture coordinates 0 and 1, don't try to outsmart yourself by calculating some s, t. Next step. Don't use a perspective projection, unless you indend to render something in perspective.
Upon your original problem. OpenGL assumes the origin of an image to be in the lower left, while most video formats put the origin into the upper left. What you did was rotating the picture, but while this will turn it upright, it will leave it mirrored. Instead you want to mirror it along the T texture coordinates, which is easily accomplished by using a negative value for t.

background with pattern texture

I tried it:
CCSprite *background = [CCSprite spriteWithSpriteFrame:frame];
background.textureRect = CGRectMake(0, 0, calcadaWidth, winSize.height);
background.position = ccp(calcadaWidth * 0.5, winSize.height * 0.5);
ccTexParams params = {GL_LINEAR, GL_LINEAR, GL_REPEAT, GL_REPEAT};
[background.texture setTexParameters:&params];
It works if the texture is regular gl size such as 64x64, 128x128...
My texture is 126x70, then, using this code I get some black space between repeats.
Cocos2D uses OpenGL and one of the limitiation of OpenGL is that if you want a texture to repeat, it must be a power of two texture.
The black spaces you are getting is where OpenGL has padded your texture up to the next power of two.

How to render image with OpenGL ES?

I am new in OpenGL ES. I earlier developed games in cocoa with objective-c. Now I want to develope 3D game for iPhone by using OpenGL ES. I am at beginner stage. I am able to create triangle, square, cube, pyramid by usng OpenGL ES. But if we have any .png image with us and we have to render it in our game then what we have to do? For that we require any other tool like unity or what? I cant able to understand it exactly. or we have to do it like the GLSprite example which is given in apple.developer.com site. In that example they draw a tea pot by using one teapot.h file. teapot.h file contain some points and by using that points they plot triangle which formed a tea pot. So is this a way to draw any image. I think I am thinking in wrong direction so please guide me for this.
Thank you in advance
To draw an image you need to first define the geometry that the image can be applied to:
float w = width / 2;
float h = height / 2;
float x = 10.0f;
float y = 10.0f;
float z = 0.0f;
float scaleX = 1.0f;
float scaleY = 1.0f;
float scaleZ = 1.0f;
const GLfloat squareVertices[] = {
-w, -h,
w, -h,
-w, h,
w, h,
};
const GLfloat textureCoords[] = {
0, 0,
1, 0,
0, 1,
1, 1,
};
Then, you can apply your texture and render this geometry:
glEnable(GL_TEXTURE_2D);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glEnableClientState(GL_VERTEX_ARRAY);
// apply texture -- how to create and bind a physical image is a whole different question
glBindTexture(GL_TEXTURE_2D, yourTextureID);
glVertexPointer(2, GL_FLOAT, 0, squareVertices);
glTexCoordPointer(2, GL_FLOAT, 0, textureCoords);
glPushMatrix();
glTranslatef(x, y, z);
glScalef(scaleX, scaleY, scaleZ);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glPopMatrix();
glDisable(GL_TEXTURE_2D);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
glDisableClientState(GL_VERTEX_ARRAY);
Note that these code-snippets assume that you have setup a working view/projection.
The Crash Landing sample that used to be in the SDK is also a good place to start (apparently the audio code was broken and that's why it was removed, but the OpenGL code is still good)
You can find a link to the sample here.