I want to implement zooming of sprites with a pinch gesture in Cocos2d.
How do I achieve it without the image getting pixelated?
I tried with vectors but with no success, I'm doomed using raster bitmap images.
Do I need the largest possible image with the highest resolution to make it look
nice?
What is the size limit for pngs in cocos2d?
What other pitfalls do I need to consider?
Yes. For example if the sprite should cover an area of 1024x1024 pixels when zoomed in to maximum, you need to create the image as 1024x1024 and set the scale property to below 1 in order to create a smaller version. If you use scale greater than 1.0 the image will always lose detail and become ever more blurred as scale increases.
There is no size limit in cocos2d, it's the devices that impose the limit. Most devices can handle 2048x2048 except 1st and 2nd generation which support only 1024x1024. You wouldn't normally support these older devices though, so 2048x2048 should be the default. Several newer devices (iPad 2+, iPhone 4S+) can use up to 4096x4096 textures.
Memory consumption. Not sure what you're trying to do, but often developers have little understanding about how much memory textures consume and what amount of memory is available. For instance, 2048x2048 as PNG with 32-Bit color consumes 16 MB of memory. Don't plan on using more than 4-5 of these, unless you're able to reduce color bit depth and use TexturePacker to be able to use the compressed .pvr.ccz format. Read my article about optimizing memory usage for more info.
I'm trying to push my FPS up on iPhone 3Gs from 30 as high as possible... and I'm running into a couple of issues and thought it would be better to ask for advice.
1) What exactly do the Renderer Utilization and Tiler Utilization columns on the OpenGL ES Instrument signify? My Tiler Utiliation percentage is extremely low, and my Renderer Utilization tends to drop during user interaction and when the app is flipped to landscape mode. I noticed that my FPS tends to drop whenever the Renderer Utilization value drops as well. My FPS dropping during landscape mode is particularly odd for me, because portrait mode and landscape mode use the exact same game logic, and textures... and landscape mode actually renders fewer vertices/triangles to boot (some parts of the UI aren't drawn at all in landscape mode).
2) I've already done most of the recommended optimizations in the ngmoco/Stanford videos, and the only things I think I can do left are changing GLfloats to GLshorts and interleaving my vertices with my texture coordinates into one array. Are any of these likely to have large effects on my FPS? It's a 2D sprite game with lots of large, detailed textures...
3) Which is a faster way to hide a polygon: setting all of its vertices to the same coordinates (essentially, reducing it to a point), or setting its alpha value to 0? I'm guessing its the former, since blending is slower in general and particularly expensive on the iPhone.
4) Currently, I'm using a 2 512x512 textures, a 1024x512 texture, and a 256x256 texture. I've sought advice on how to best manage this, and I was told not to combine them into 1 1024x1024 texture because of memory problems on the iPhone 3G. I'd like to confirm that here, because if I put everything into 1 texture, I can eradicate having to call glBindTexture repeatedly...
To #4: (a) yes, the iPhone is documented not to deal with images larger than 1024 on a side. 1024x1024 is the maximum theoretical limit, although you may run into problems if you try to push right up against the limit.
(b) all your textures don't fit into a 1024x1024; after the 1024x512 and 2 512x512s fill that space, you'll still have the 256x256 left over.
I have an Open GL ES game on the iPhone. My framerate is pretty sucky, ~20fps. Using the Xcode OpenGL ES performance tool on an iPhone 3G, it shows:
Renderer Utilization: 95% to 99%
Tiler Utilization: ~27%
I am drawing a lot of pretty large images with a lot of blending. If I reduce the number of images drawn, framerates go from ~20 to ~40, though the performance tool results stay about the same (renderer still maxed). I think I'm being limited by the fill rate of the iPhone 3G, but I'm not sure.
My questions are: How can I determine with more granularity where the bottleneck is? That is my biggest problem, I just don't know what is taking all the time. If it is fillrate, is there anything I do to improve it besides just drawing less?
I am using texture atlases. I have tried to minimize image binds, though it isn't always possible (drawing order, not everything fits on one 1024x1024 texture, etc). Every frame I do 10 image binds. This seem pretty reasonable, but I could be mistaken.
I'm using vertex arrays and glDrawArrays. I don't really have a lot of geometry. I can try to be more precise if needed. Each image is 2 triangles and I try to batch things were possible, though often (maybe half the time) images are drawn with individual glDrawArrays calls. Besides the images, I have ~60 triangles worth of geometry being rendered in ~6 glDrawArrays calls. I often glTranslate before calling glDrawArrays.
Would it improve the framerate to switch to VBOs? I don't think it is a huge amount of geometry, but maybe it is faster for other reasons?
Are there certain things to watch out for that could reduce performance? Eg, should I avoid glTranslate, glColor4g, etc?
I'm using glScissor in a 3 places per frame. Each use consists of 2 glScissor calls, one to set it up, and one to reset it to what it was. I don't know if there is much of a performance impact here.
If I used PVRTC would it be able to render faster? Currently all my images are GL_RGBA. I don't have memory issues.
One of my fullscreen textures is 256x256. Would it be better to use 480x320 so the phone doesn't have to do any scaling? Are there any other general performance advice for texture sizes?
Here is a rough idea of what I'm drawing, in this order:
1) Switch to perspective matrix.
2) Draw a full screen background image
3) Draw a full screen image with translucency (this one has a scrolling texture).
4) Draw a few sprites.
5) Switch to ortho matrix.
6) Draw a few sprites.
7) Switch to perspective matrix.
8) Draw sprites and some other textured geometry.
9) Switch to ortho matrix.
10) Draw a few sprites (eg, game HUD).
Steps 1-6 draw a bunch of background stuff. 8 draws most of the game content. 10 draws the HUD.
As you can see, there are many layers, some of them full screen and some of the sprites are pretty large (1/4 of the screen). The layers use translucency, so I have to draw them in back-to-front order. This is further complicated by needing to draw various layers in ortho and others in perspective.
I will gladly provide additional information if reqested. Thanks in advance for any performance tips or general advice on my problem!
Edit:
I added some logging to see how many glDrawArrays calls I am doing, and with how much data. I do about 20 glDrawArray calls per frame. Usually about 1 to up to 6 of these has about 40 vertices each. The rest of the calls are usually just 2 vertices (one image). I'm just using glVertexPointer and glTexCoordPointer.
Given that the Renderer Utilization is basically at 100%, that indicates that the bottleneck is filling, texturing, and blending pixels. Techniques intended to optimize vertex processing (VBOs and vertex formats) or CPU usage (draw call batching) will likely not help, as they will not speed up pixel processing.
Your best bet is to reduce the number of pixels that you are filling, and also look at different texture formats that make better use of the very limited memory bandwidth available on the first generation devices. Prefer the use of PVRTC textures wherever possible, and 16bit uncompressed textures otherwise.
Look to Apple's "Best Practices for Working with Texture Data" and "Best Practices for Working with Vertex Data" sections of the OpenGL ES Programming Guide for iPhone OS. They highly recommend (as do others) that you use PVRTC for compressing your textures, because they can offer an 8:1 or 16:1 compression ratio over your standard uncompressed textures. Aside from mipmapping, you seem to be doing the other recommended optimization of using a texture atlas.
You do not appear to be geometry-limited, because (as I discovered in this question) the Tiler Utilization statistic seems to indicate how much of a bottleneck is being caused by geometry size. However, the iPhone 3G S (and third-generation iPod touch and iPad) support hardware-accelerated VBOs, so you might give those a shot and see how they affect performance. They might not have as much of an effect as compressing textures would, but they're not hard to implement.
A big win for the (mostly for)3G will also be the texture filtering you are using. Check if you are using "TRILINEAR" filtering, and change it to "BILINEAR".
Make sure you setup the textures like this:
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
and not like this:
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
Harry
I wanted to chime in with an additional answer, change the Framebuffer backing to be a 16bit format as opposed to a 32 bit format.
if ((self = [super initWithCoder:coder]))
{
eaglLayer = (CAEAGLLayer *)self.layer;
eaglLayer.opaque = YES;
eaglLayer.drawableProperties = [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithBool:NO],
kEAGLDrawablePropertyRetainedBacking,
kEAGLColorFormatRGB565, //kEAGLColorFormatRGBA8 = large frame buff 32bit // kEAGLColorFormatRGB565 = 16bit frame buffer.
kEAGLDrawablePropertyColorFormat,
nil];
}
What woke me up to this was the XCode profiler. It kept complaining about using to large a frame buffer, eventually I found it in my init section.
http://developer.apple.com/library/ios/#documentation/iPhone/Reference/EAGLDrawable_Ref/EAGLDrawable/EAGLDrawable.html
.
That single change allowed my games on iPad, Retina, and iPods to go to 60 FPS.
I have yet to re-release them with this change as I just found it out 3 days ago :) but I do not think I plan to release at 60fps, 30fps is just fine for casual games, and I found that my sound effects cut the frame rate down, so either resample, play the sfx in another thread, or other solutions to keep that frame rate up if I decide to go with 60fps. Don't forget to discard the buffers that are not used to display..
if (fiOSver >= 4.0f) {
const GLenum discards[] = {GL_DEPTH_ATTACHMENT_OES};
glDiscardFramebufferEXT(GL_FRAMEBUFFER_OES,1,discards);
}
[m_oglContext presentRenderbuffer:GL_RENDERBUFFER_OES];
In a similar situation (porting a 2D Adventure game to the iPad). My 3GS version was running more or less locked at 60FPS, put it on iPad dropped it (and my jaw) to 20fps.
Turns out ONE of the little gotchas involved is that the PVR cards hate GL_ALPHA_TEST; on the PC that actually has a slight positive effect (especially on older intel chips), but they're death on fillrate on the iPhone. Changing that to
glDisable(GL_ALPHA_TEST);
gave me an immediate 100% boost in FPS (up to 40 FPS). Not bad for one line of code.. :)
Allan
The biggest performance killer on the iPhone platform is the number of draw calls and state changes. If you're doing more than 20ish draw calls or state changes, you're going to run into a performance wall.
Batching and texture atlases are your friend.
By my past experience with openGL ES on old windows mobile devices with processing speed around 600mhz
usually reducing the rendering window resolution increase the speed of rendering.
as my tests lately i figured out that i need performance monitoring while rendering frame by frame to collect how many fps can display with current performance and what resolution currently applied
i hope it is a good practice to keep a monitoring algorithm in place of rendering view to balance resolution and frame rate while running a game rendering engine.
depending on the amount of frame rate you wanted, you should sacrifice the rendering view resolution, to perform best and degrade gracefully on most devices with varying hardware and software performence.
you may need to control the resolution menually like explained in this article.
http://www.david-amador.com/2010/09/setting-opengl-view-for-iphone-4-retina-hi-resolution/
I'm developing on the iPhone and the majority of our game is using OpenGL ES, but there are also menus that use CGImage and Quartz in order to be displayed. In OpenGL ES, I know that no matter what image compression goes in (JPG, PNG, etc.), the data stored in memory as a texture is an 8-bit texture, unless I use PVRTC in which case I can get it to 2 or 4 bits. We've been having memory issues due to large CGImages, so my question is... what sort of optimizations and compressions do Quartz and CGImage use? I can't find the details in Apple's docs, when really I want to know if it would make a difference to put a 256-color image in, or a JPG vs a PNG, if having the dimensions at a power of 2 help, etc. Speed is unimportant, memory is the bottleneck here.
Thanks.
Quartz is uncompressed. It is for quickly compositing and rendering pixel accurate content. Once your images have been drawn into a context it doesn't matter where they came from, they take whatever that context takes per pixel for however many pixels they have (generally 4 bytes per pixel in a device if I recall correctly). The one big thing it does is premultiplies the alpha to avoid blending.
Now, some views under memory pressure can evict their contents if not displayed, and reconstitute them as needed. In those cases a CGImage from a compressed source generally ends up taking less memory, but I suspect that is not relevant in the case you described.
I've got PVR texture compression working all happy and good in my iPhone game, but I've got issues when tiling multiple textures together. Basically, I've got a very large background which is split into multiple 512x512 tiles, all PVR compressed. Then they're drawn together to look like one big background image. The way PVR works, because it doesn't know that it's supposed to be compressing the texture as if it were a really big texture - i.e. use a neighbor's tiled information to determine how to perform the PVR compression.
I can think of maybe a couple ways to do this.
1) Somehow tell the texturetool command line program to accommodate for other images that will be adjacent.
2) Use the command line program to generate a huge PVR texture that represents the whole image, then somehow split up the bytes into multiple images - probably impossible.
3) Do some kind of OpenGL ES trickiness that blends the edges nicely.
4) Do some trickiness where I have redundant information in each tile and then clip those areas when the texture is drawn (please no).
Hopefully I can do 1, 2 or 3, or there is some other well known solution.
I ended up going with option 4. I don't think this was a situation where PVRTC isn't appropriate - in fact it's almost a necessity. When I've got a total of 24 512x512 textures in memory at once (representing a very large background and foreground), putting those in uncompressed is suicide. So I simply used PVR compression as normal, then my edited a few lines of code in my tiling algorithm so that they overlap and trim on 15 pixels on each end. Voila, looks great. Took a couple days and was pretty annoying, but I think this is a good option for people who need very large tiled backgrounds on the iPhone.
My best advice, but not what you asked, is to know when PVRTC isn't appropriate. By far the simplest solution is to just not use PVRTC for those tiles. I've spent a lot of time trying to bend PVRTC to work in situations it just isn't suited for.
That said,
When using PVRTC, the texture is always assumed to tile (with itself), thus pixels on the right edge affect the pixels on the left edge (same with top & bottom). So choices 1 or 2 likely won't work.
One possibility is to add an alpha channel to the tiles and allow them to fade out around the edges so that when you overlap them, they fade into each other. Keep in mind PVRTC tends to work better with gradual alpha fades. Hard alpha edges often have artifacts in PVRTC.