opengl 1.1 messes up at launch (randomly) - iphone

I have this problem with openGL ES 1.1 on iPhone. I have made myself c++ engine that makes all the job in opengl and a view that shows the rendered content. The problem is that sometimes it works ok and sometimes (most of the times) it shows messed up view. By messed up i mean that objects that do not move appear in different locations, rotated, stretched, o ther parts of the scene is okay or invisible, there is no user interacion nor FPS (its just one frame when it breaks up). I thought it may be because my depth buffer is shitty. But i think that the overall buffer engines may be bad. Anyways these are the parts from my code.
I have view, that initializes like this:
self = [super initWithFrame:frame];
if (self) {
CAEAGLLayer* eaglLayer = (CAEAGLLayer*) super.layer;
eaglLayer.opaque = YES;
m_context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES1];
if (!m_context || ![EAGLContext setCurrentContext:m_context]) {
[self release];
return nil;
}
cplusplusEngine = CreateRenderer();
[m_context renderbufferStorage:GL_RENDERBUFFER_OES fromDrawable:eaglLayer];
cplusplusEngine ->Initialize(CGRectGetWidth(frame), CGRectGetHeight(frame));
//[self drawView: nil];
//m_timestamp = CACurrentMediaTime();
CADisplayLink* displayLink;
displayLink = [CADisplayLink displayLinkWithTarget:self selector:#selector(drawView:)];
[displayLink addToRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
[displayLink setFrameInterval:1/45];
[self loadUpTextures];
}
return self;
the draw View method looks like this:
GLint a = cplusplusengine->Render();
[m_context presentRenderbuffer:GL_RENDERBUFFER_OES];
now i create the buffer and present it, i also create buffers in engine like this:
int widthB, heightB;
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES,GL_RENDERBUFFER_WIDTH_OES, &widthB);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES,GL_RENDERBUFFER_HEIGHT_OES, &heightB); glViewport(0, 0, widthB, heightB);
// Create a depth buffer that has the same size as the color buffer.
glGenRenderbuffersOES(1, &m_depthRenderbuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_depthRenderbuffer);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_DEPTH_COMPONENT16_OES, widthB, heightB);
// Create the framebuffer object.
GLuint framebuffer;
glGenFramebuffersOES(1, &framebuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, framebuffer);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, m_colorRenderbuffer);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES,GL_RENDERBUFFER_OES,m_depthRenderbuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, m_colorRenderbuffer);
And every frame i clear color and depth buffers. Now i do get message from instruments for that draw view's line "present renderbuffer" it says "OpenGL ES performed an unnecessary logical buffer store operation. This is typically caused by not clearing buffers at the start of the rendering loop and not discarding buffers at the end of the rendering loop. If your application clears the depth buffer at the beginning of each frame, it should discard the depth buffer at the end of each frame. Please see the EXT_discard_framebuffer extension for more details." Now i am trying to work my a** off to solve this but i cannot find the solution. I may have few places in textures where this may be happening. It would be helpful to at least find out why opengl may draw messy.
P.S. I do load up textures in that view and set them in engine like this. engineTexture[index] = viewsTextureValueAt[index]; That just sets the GLuint from views texture pointer to engine texture pointer. Can i do that? It works but i don't know whether this is the case. I do get errors even if i comment out all the texture usages though.

I managed to work this out myself. It seems my buffers are all okay. My textures are also good. The error was lying in one simple "common newbie mistake". I used quite few variables to manipulate and align all my scene. It seems that when I was using those variables in objective-c without first defining the values to zero it was okay, the compiler somehow assigned 0 to them, but now, when I use c++ engine, all variables that I did not define now gets random values, this makes my application randomly crash up in different ways. For example my button alignment array was set only for last 4 buttons, first one is in 0 position so I left that number undefined so that's why that button flew off somewhere every time I launched. One time the value got 1700000+ another -0.000056+ and so.

Related

Basic Drawing of Pixels From Unsigned Short * Using OpenGLES on iOS

For my non-app store app, I've been using the private framework Core Surface to draw directly to the screen of the iPhone. However, this can be rather slow on older devices because it heavily uses the CPU to do its drawing. To fix this, I've decided to try to use OpenGLES to render the pixels to the screen.
Currently (and I have no way of changing this), I have a reference to an unsigned short * variable called BaseAddress, and essential 3rd party code accesses BaseAddress and updates it with the new pixel data.
I've set up a GLKViewController, and implemented the viewDidLoad as follows:
- (void)viewDidLoad {
[super viewDidLoad];
self.context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
if (!self.context) {
NSLog(#"Failed to create ES context");
}
[EAGLContext setCurrentContext:self.context];
GLKView *view = (GLKView *)self.view;
view.context = self.context;
glGenBuffers(1, &screenBuffer);
glBindBuffer(GL_ARRAY_BUFFER, screenBuffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(BaseAddress), BaseAddress, GL_DYNAMIC_DRAW);
}
where screenBuffer is an instance variable. In the glkView:drawInRect: method I have the following:
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect {
glDrawElements(GL_ARRAY_BUFFER, sizeof(BaseAddress)/sizeof(BaseAddress[0]), GL_UNSIGNED_SHORT, BaseAddress);
}
Unfortunately, only a black screen appears when I run the app. If I go back to using Core Surface, the app works fine. So basically, how can I draw the pixels to the screen using OpenGLES?
I think that it might be best to use a texture and for your case I'd try to find some older ES1 template for iOS devices. Basically what you need is a frame buffer and a color buffer made from your UIView layer:
glGenFramebuffers(1, &viewFramebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, viewFramebuffer);
glGenRenderbuffers(1, &viewColorBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, viewColorBuffer);
[context renderbufferStorage:GL_RENDERBUFFER fromDrawable:(CAEAGLLayer*)self.layer];
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, viewColorBuffer);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &backingWidth);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, & backingHeight);
As for projection matrix I suggest you use glOrthof(.0f, backingWidth, backingHeight, .0f, -1.0f, 1.0f); that will make your GL coordinates same as your view coordinates.
Next on some initialization generate a texture, bind it and give it dimensions of power of 2 (textureWidth = 1; while(textureWidth < backingWidth) textureWidth = textureWidth << 1;) and pass it NULL for data pointer (all in function "glTexImage2D")
Then generate vertex array for a square same as texture, from (0,0) to (textureWidth, textureHeight) and texture coordinates from (0,0) to (1,1)
When you get the data to your pointer and are ready to be pushed to texture use glTexSubImage2D to update it: You can update only a segment of a texture if you get data for it or to update a whole screen use rect (0,0,screenWidth, screenHeight)
Now just draw those 2 triangles with your texture..
Note that there is a limit on texture size: most active iOS devices 1<<11 (2048) iPad3 1<<12
Do not forget to set texture parameters when creating it: glTexParameteri
Do check for retina display and set content scale of CAEAGLLayer to 2 if needed

Least CPU intensive way to Frequently & Repeately Draw Many Views

This Is a problem that I've been leaving and coming back to for a while now. I've never really nailed the problem.
What I've been trying to do use CADisplayLink to dynamically draw pie chart style progress. My code works fine when I have 1 - 4 uiviews updating simultaneously. When I add any more than that the drawing of the pies becomes very jerky.
I want to explain what I have been trying in the hope that somebody could point out the inefficiencies and suggest a better drawing method.
I create 16 uiviews and add a CAShapeLayer subview to each one. This is where I want to draw my pie slices.
I precalcuate 360 CGPaths representing 0 to 360 degrees of a circle and store them in an array to try and improve performance.
In a master View I start a displaylink,loop through all my other views, calculate how much of a full pie it should show, then find the right path and assign it to my shapelayer.
-(void)makepieslices
{
pies=[[NSMutableArray alloc]initWithCapacity:360];
float progress=0;
for(int i=0;i<=360;i++)
{
progress= (i* M_PI)/180;
CGMutablePathRef thePath = CGPathCreateMutable();
CGPathMoveToPoint(thePath, NULL, 0.f, 0.f);
CGPathAddLineToPoint(thePath, NULL, 28, 0.f);
CGPathAddArc(thePath, NULL, 0.f,0.f, 28, 0.f, progress, NO);
CGPathCloseSubpath(thePath);
_pies[i]=thePath;
}
}
- (void)updatePath:(CADisplayLink *)dLink {
for (int idx=0; idx<[spinnydelegates count]; idx++) {
id<SyncSpinUpdateDelegate> delegate = [spinnydelegates objectAtIndex:idx];
dispatch_async(dispatch_get_global_queue(0, 0), ^{
[delegate updatePath:dLink];
});
}
}
- (void)updatePath:(CADisplayLink *)dLink {
dispatch_async(dispatch_get_global_queue(0, 0), ^{
currentarc=[engineref getsyncpercentForPad:cid pad:pid];
int progress;
progress = roundf(currentarc*360);
dispatch_async(dispatch_get_main_queue(), ^{
shapeLayer_.path = _pies[progress];
});
});
}
This technique just straight out isnt working for me when trying to simultaneously update more than 4 or 5 pies at the same time. 16 screen updates at the same time sounds like it should really not be that big of a deal for the ipad to me. So this leads me to think I doing something very very fundamentally wrong.
I'd really appreciate if somebody could tell me why this technique results in jittery screen updates and also if they could suggest a different technique that I could go an investigate that will allow me to perform 16 simultaneous shapelayer updates smoothly.
EDIT Just to give you an idea of how bad performance is, when I have all 16 pies drawing the cpu goes up to 20%
*EDIT *
This is based on studevs advice but I don't see anything been drawn. segmentLayer is a CGLayerRef as a property of my pieview.
-(void)makepies
{
self.layerobjects=[NSMutableArray arrayWithCapacity:360];
CGFloat progress=0;
CGContextRef context=UIGraphicsGetCurrentContext();
for(int i =0;i<360;i++)
{
progress= (i*M_PI)/180.0f;
CGLayerRef segmentlayer=CGLayerCreateWithContext(context, CGSizeMake(30, 30), NULL);
CGContextRef layerContext=CGLayerGetContext(segmentlayer);
CGMutablePathRef thePath = CGPathCreateMutable();
CGPathMoveToPoint(thePath, NULL, 0.f, 0.f);
CGPathAddLineToPoint(thePath, NULL, 28, 0.f);
CGPathAddArc(thePath, NULL, 0.f,0.f, 28, 0.f, progress, NO);
CGPathCloseSubpath(thePath);
[layerobjects addObject:(id)segmentlayer];
CGLayerRelease(segmentlayer);
}
}
-(void)updatePath
{
int progress;
currentarc=[engineref getsyncpercent];
progress = roundf(currentarc*360);
//shapeLayer_.path = _pies[progress];
self.pieView.segmentLayer=(CGLayerRef)[layerobjects objectAtIndex:progress];
[self.pieView setNeedsDisplay];
}
-(void)drawRect:(CGRect)rect
{
CGContextRef context=UIGraphicsGetCurrentContext();
CGContextDrawLayerInRect(context, self.bounds, segmentLayer);
}
I think one of the first things you should look to do is buffer your segments (currently represented by CGPath objects) offscreen using CGLayer objects. From the docs:
Layers are suited for the following:
High-quality offscreen rendering of drawing that you plan to reuse.
For example, you might be building a scene and plan to reuse the same
background. Draw the background scene to a layer and then draw the
layer whenever you need it. One added benefit is that you don’t need
to know color space or device-dependent information to draw to a
layer.
Repeated drawing. For example, you might want to create a
pattern that consists of the same item drawn over and over. Draw the
item to a layer and then repeatedly draw the layer, as shown in Figure
12-1. Any Quartz object that you draw repeatedly—including CGPath,
CGShading, and CGPDFPage objects—benefits from improved performance if
you draw it to a CGLayer. Note that a layer is not just for onscreen
drawing; you can use it for graphics contexts that aren’t
screen-oriented, such as a PDF graphics context.
Create a UIView subclass that draws the pie. Give it an instance variable for that pie's current progress, and override drawRect: to draw the layer representing that progress. The view needs to first get a reference the required CGLayer object, so implement a delegate with the method:
- (CGLayerRef)pieView:(PieView *)pieView segmentLayerForProgress:(NSInteger)progress context:(CGContextRef)context;
It will then become the delegate's job to return an existing CGLayerRef, or if it doesn't exist yet, create it. Since the CGLayer can only be created from within drawRect:, this delegate method should be called from PieView's drawRect: method. PieView should look something like this:
PieView.h
#import <UIKit/UIKit.h>
#import <QuartzCore/QuartzCore.h>
#class PieView;
#protocol PieViewDelegate <NSObject>
#required
- (CGLayerRef)pieView:(PieView *)pieView segmentLayerForProgress:(NSInteger)progress context:(CGContextRef)context;
#end
#interface PieView : UIView
#property(nonatomic, weak) id <PieViewDelegate> delegate;
#property(nonatomic) NSInteger progress;
#end
PieView.m
#import "PieView.h"
#implementation PieView
#synthesize delegate, progress;
- (void)drawRect:(CGRect)rect
{
CGContextRef context = UIGraphicsGetCurrentContext();
CGLayerRef segmentLayer = [delegate pieView:self segmentLayerForProgress:self.progress context:context];
CGContextDrawLayerInRect(context, self.bounds, segmentLayer);
}
#end
Your PieView's delegate (most likely your view controller) then implements:
NSString *const SegmentCacheKey = #"SegmentForProgress:";
- (CGLayerRef)pieView:(PieView *)pieView segmentLayerForProgress:(NSInteger)progress context:(CGContextRef)context
{
// First, try to retrieve the layer from the cache
NSString *cacheKey = [SegmentCacheKey stringByAppendingFormat:#"%d", progress];
CGLayerRef segmentLayer = (__bridge_retained CGLayerRef)[segmentsCache objectForKey:cacheKey];
if (!segmentLayer) { // If the layer hasn't been created yet
CGFloat progressAngle = (progress * M_PI) / 180.0f;
// Create the layer
segmentLayer = CGLayerCreateWithContext(context, layerSize, NULL);
CGContextRef layerContext = CGLayerGetContext(segmentLayer);
// Draw the segment
CGContextSetFillColorWithColor(layerContext, [[UIColor blueColor] CGColor]);
CGContextMoveToPoint(layerContext, layerSize.width / 2.0f, layerSize.height / 2.0f);
CGContextAddArc(layerContext, layerSize.width / 2.0f, layerSize.height / 2.0f, layerSize.width / 2.0f, 0.0f, progressAngle, NO);
CGContextClosePath(layerContext);
CGContextFillPath(layerContext);
// Cache the layer
[segmentsCache setObject:(__bridge_transfer id)segmentLayer forKey:cacheKey];
}
return segmentLayer;
}
So for each pie, create a new PieView and set it's delegate. When you need to update a pie, update the PieView's progress property and call setNeedsDisplay.
I'm using an NSCache here since there are a lot of graphics being stored, and it could take up a lot of memory. You could also limit the number of segments being drawn - 100 is probably plenty. Also, I agree with other comments/answers that you might try updating the views less often, as this will consume less CPU and battery power (60fps is probably not necessary).
I did some crude testing of this method on an iPad (1st gen) and managed to get well over 50 pies updating at 30fps.
dubbeat: ...CADisplayLink...
Justin: do you need to draw at the display's refresh rate?
dubbeat: The progress of the pie drawing is supposed to represent the progress of an mp3s playback progress so I guess at the displays refresh rate at a minimum.
That's much faster than is necessary, unless you're trying to display some really, really, really exotic visualizer, which is very unlikely if your spinner's radius is 28pt. Also, there's no reason to draw faster than the display's frequency.
One side effect is that your spinner's superviews may also updating at this high frequency. If you can make the spinner view opaque, then you can reduce overdrawing of superviews (and subviews if you have them).
60fps is a good number for a really fast desktop game. For an ornament/progress bar, it's far more than necessary.
Try this:
not using CADisplayLink, but the standard view system
use an NSTimer on the main run loop, begin with a frequency of 8 Hz*
adjust timer to taste
then let us know if that is adequately fast.
*the timer callback calls [spinner setNeedsDisplay]
Well, you could achieve some performance improvement by pre-assembling the background view, capturing the image of it, and then just using the image in an image view for the background. You could go further by capturing a view of the "relatively static" parts of your chart, updating that static view only when necessary.
Store your 360 circle segments as textures and use OpenGL to animate the sequences.

OpenGL-ES, iPhone and intermittent error: GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_OES (0x8CD6)

I have an app that uses OpenGL-ES and an EAGLContext within a UIView - very much like Apple's GLPaint sample code app.
It might be significant that I see this bug on my iPhone 4 but not on my iPad.
Mostly, this works very well. However, I am getting GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_OES from glCheckFramebufferStatusOES() within the createFrameBuffer method. The reason is that the backingWidth and backingHeight are both 0.
I am trying to understand the relation between )self.layer and its size - which is not (0,0) - and the values for backingWidth and backingHeight. My UIView and its CALayer both have the 'correct' size, while glGetRenderbufferParameterivOES() returns 0 for GL_RENDERBUFFER_WIDTH_OES and GL_RENDERBUFFER_HEIGHT_OES.
Here is my createFrameBuffer method - which works much of the time.
- (BOOL)createFramebuffer
{
// Generate IDs for a framebuffer object and a color renderbuffer
glGenFramebuffersOES(1, &viewFramebuffer);
glGenRenderbuffersOES(1, &viewRenderbuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, viewFramebuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, viewRenderbuffer);
// This call associates the storage for the current render buffer with the EAGLDrawable (our CAEAGLLayer)
// allowing us to draw into a buffer that will later be rendered to screen wherever the layer is (which corresponds with our view).
[context renderbufferStorage:GL_RENDERBUFFER_OES fromDrawable:(id<EAGLDrawable>)self.layer];
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, viewRenderbuffer);
//DLog(#" backing size = (%d, %d)", backingWidth, backingHeight);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth);
glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight);
DLog(#" backing size = (%d, %d)", backingWidth, backingHeight);
err = glGetError();
if (err != GL_NO_ERROR)
DLog(#"Error. glError: 0x%04X", err);
// For this sample, we also need a depth buffer, so we'll create and attach one via another renderbuffer.
glGenRenderbuffersOES(1, &depthRenderbuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, depthRenderbuffer);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_DEPTH_COMPONENT16_OES, backingWidth, backingHeight);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES, GL_RENDERBUFFER_OES, depthRenderbuffer);
if(glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES) != GL_FRAMEBUFFER_COMPLETE_OES)
{
NSLog(#"failed to make complete framebuffer object 0x%X", glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES));
return NO;
}
return YES;
}
When backingWidth and backingHeight are non-zero, then there is no error returned from glCheckFramebufferStatusOES().
I had this same problem. For me the fix was that in the opengl sample code of last year, Apple rebuilds the renderbuffer in every layoutSubviews call. Now, if you create an iPhone template opengl project, you will see that the layoutSubviews only destroys the renderbuffer. Then on every draw, if the render buffer is nil THEN create it. This is better cause when you are about to draw all CAlayers etc should be all shined up and ready to go.
I think that the render buffer in my case was trying to be built when the EagleView layer was not serviceable - i.e. in some tear down state. In any case when I changed my code to match it worked.
Also there are fewer calls to this code, which is likely faster. On startup there is a lot of scene loading and moving about, which generates 1/2 dozen layout sub view calls with my app.
Since the comments in Apple's code tend to be few and far between, the fact that there is one in the layoutsubviews call is significant:
// The framebuffer will be re-created at the beginning of the next
setFramebuffer method call.
--Tom
I had this same problem also, using Apple's OpenGL-ES sample code that does a destroyFramebuffer, createFramebuffer then drawView with the layoutSubviews function.
What you want to do is create the frame buffer in the drawView call as Tom says above, but additionally, you also want to defer the call to drawView until the layoutSubviews function returns. The way I did this was:
- (void) layoutSubviews
{
[EAGLContext setCurrentContext:context];
[self destroyFramebuffer];
// Create the framebuffer in drawView instead as needed, as before create
// would occasionally happen when the view wasn't servicable (?) and cause
// a crash. Additionally, send the drawView call to the main UI thread
// (ALA PostMessage in Win32) so that it is deferred until this function
// returns and the message loop has a chance to process other stuff, etc
// so the EAGLView will be ready to use when createFramebuffer is finally
// called and the glGetRenderbufferParameterivOES calls to get the backing
// width and height for the render buffer will always work (occasionally I
// would see them come back as zero on my old first gen phone, and this
// crashes OpenGL.)
//
// Also, using this original method, I would see memory warnings in the
// debugger console window with my iPad when rotating (not all the time,
// but pretty frequently.) These seem to have gone away using this new
// deferred method...
[self performSelectorOnMainThread:#selector(drawView)
withObject:nil
waitUntilDone:NO];
}
Ross

fastest way to draw a screen buffer on the iphone

I have a "software renderer" that I am porting from PC to the iPhone. what is the fastest way to manually update the screen with a buffer of pixels on the iphone? for instance in windows the fastest function I have found is SetDIBitsToDevice.
I don't know much about the iphone, or the libraries, and there seem to be so many layers and different types of UI elements, so I might need a lot of explanation...
for now I'm just going to constantly update a texture in opengl and render that to the screen, I very much doubt that this is going to be the best way to do it.
UPDATE:
I have tried the openGL screen sized texture method:
I got 17fps...
I used a 512x512 texture (because it needs to be a power of two)
just the call of
glTexSubImage2D(GL_TEXTURE_2D,0,0,0,512,512,GL_RGBA,GL_UNSIGNED_BYTE, baseWindowGUI->GetBuffer());
seemed pretty much responsible for ALL the slow down.
commenting it out, and leaving in all my software rendering GUI code, and the rendering of the now non updating texture, resulted in 60fps, 30% renderer usage, and no notable spikes from the cpu.
note that GetBuffer() simply returns a pointer to the software backbuffer of the GUI system, there is no re-gigging or resizing of the buffer in anyway, it is properly sized and formatted for the texture, so I am fairly certain the slowdown has nothing to do with the software renderer, which is the good news, it looks like if I can find a way to update the screen at 60, software rendering should work for the time being.
I tried doing the update texture call with 512,320 rather than 512,512 this was oddly even slower... running at 10fps, also it says the render utilization is only like 5%, and all the time is being wasted in a call to Untwiddle32bpp inside openGLES.
I can change my software render to natively render to any pixle format, if it would result in a more direct blit.
fyi, tested on a 2.2.1 ipod touch G2 (so like an Iphone 3G on steroids)
UPDATE 2:
I have just finished writting the CoreAnimation/Graphics method, it looks good, but I am a little worried about how it updates the screen each frame, basically ditching the old CGImage, creating a brand new one... check it out in 'someRandomFunction' below:
is this the quickest way to update the image? any help would be greatly appreciated.
//
// catestAppDelegate.m
// catest
//
// Created by User on 3/14/10.
// Copyright __MyCompanyName__ 2010. All rights reserved.
//
#import "catestAppDelegate.h"
#import "catestViewController.h"
#import "QuartzCore/QuartzCore.h"
const void* GetBytePointer(void* info)
{
// this is currently only called once
return info; // info is a pointer to the buffer
}
void ReleaseBytePointer(void*info, const void* pointer)
{
// don't care, just using the one static buffer at the moment
}
size_t GetBytesAtPosition(void* info, void* buffer, off_t position, size_t count)
{
// I don't think this ever gets called
memcpy(buffer, ((char*)info) + position, count);
return count;
}
CGDataProviderDirectCallbacks providerCallbacks =
{ 0, GetBytePointer, ReleaseBytePointer, GetBytesAtPosition, 0 };
static CGImageRef cgIm;
static CGDataProviderRef dataProvider;
unsigned char* imageData;
const size_t imageDataSize = 320 * 480 * 4;
NSTimer *animationTimer;
NSTimeInterval animationInterval= 1.0f/60.0f;
#implementation catestAppDelegate
#synthesize window;
#synthesize viewController;
- (void)applicationDidFinishLaunching:(UIApplication *)application {
[window makeKeyAndVisible];
const size_t byteRowSize = 320 * 4;
imageData = malloc(imageDataSize);
for(int i=0;i<imageDataSize/4;i++)
((unsigned int*)imageData)[i] = 0xFFFF00FF; // just set it to some random init color, currently yellow
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
dataProvider =
CGDataProviderCreateDirect(imageData, imageDataSize,
&providerCallbacks); // currently global
cgIm = CGImageCreate
(320, 480,
8, 32, 320*4, colorSpace,
kCGImageAlphaNone | kCGBitmapByteOrder32Little,
dataProvider, 0, false, kCGRenderingIntentDefault); // also global, probably doesn't need to be
self.window.layer.contents = cgIm; // set the UIWindow's CALayer's contents to the image, yay works!
// CGImageRelease(cgIm); // we should do this at some stage...
// CGDataProviderRelease(dataProvider);
animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:#selector(someRandomFunction) userInfo:nil repeats:YES];
// set up a timer in the attempt to update the image
}
float col = 0;
-(void)someRandomFunction
{
// update the original buffer
for(int i=0;i<imageDataSize;i++)
imageData[i] = (unsigned char)(int)col;
col+=256.0f/60.0f;
// and currently the only way I know how to apply that buffer update to the screen is to
// create a new image and bind it to the layer...???
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
cgIm = CGImageCreate
(320, 480,
8, 32, 320*4, colorSpace,
kCGImageAlphaNone | kCGBitmapByteOrder32Little,
dataProvider, 0, false, kCGRenderingIntentDefault);
CGColorSpaceRelease(colorSpace);
self.window.layer.contents = cgIm;
// and that currently works, updating the screen, but i don't know how well it runs...
}
- (void)dealloc {
[viewController release];
[window release];
[super dealloc];
}
#end
The fastest App Store approved way to do CPU-only 2D graphics is to create a CGImage backed by a buffer using CGDataProviderCreateDirect and assign that to a CALayer's contents property.
For best results use the kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Little or kCGImageAlphaNone | kCGBitmapByteOrder32Little bitmap types and double buffer so that the display is never in an inconsistent state.
edit: this should be faster than drawing to an OpenGL texture in theory, but as always, profile to be sure.
edit2: CADisplayLink is a useful class no matter which compositing method you use.
The fastest way is to use IOFrameBuffer/IOSurface, which are private frameworks.
So OpenGL seems to be the only possible way for AppStore apps.
Just to post my comment to #rpetrich's answer in the form of an answer, I will say in my tests I found OpenGL to be the fastest way. I've implemented a simple object (UIView subclass) called EEPixelViewer that does this generically enough that it should work for most people I think.
It uses OpenGL to push pixels in a wide variety of formats (24bpp RGB, 32-bit RGBA, and several YpCbCr formats) to the screen as efficiently as possible. The solution achieves 60fps for most pixel formats on almost every single iOS device, including older ones. Usage is super simple and requires no OpenGL knowledge:
pixelViewer.pixelFormat = kCVPixelFormatType_32RGBA;
pixelViewer.sourceImageSize = CGSizeMake(1024, 768);
EEPixelViewerPlane plane;
plane.width = 1024;
plane.height = 768;
plane.data = pixelBuffer;
plane.rowBytes = plane.width * 4;
[pixelViewer displayPixelBufferPlanes: &plane count: 1 withCompletion:nil];
Repeat the displayPixelBufferPlanes call for each frame (which loads the pixel buffer to the GPU using glTexImage2D), and that's pretty much all there is to it. The code is smart in that it tries to use the GPU for any kind of simple processing required such as permuting the color channels, converting YpCbCr to RGB, etc.
There is also quite a bit of logic for honoring scaling using the UIView's contentMode property, so UIViewContentModeScaleToFit/Fill, etc. all work as expected.
Perhaps you could abstract the methods used in the software renderer to a GPU shader... might get better performance. You'd need to send the encoded "video" data as a texture.
A faster method than both CGDataProvider and glTexSubImage is to use CVOpenGLESTextureCache. The CVOpenGLESTextureCache allows you to directly modify an OpenGL texture in graphics memory without re-uploading.
I used it for a fast animation view you can see here:
https://github.com/justinmeiners/image-sequence-streaming
It is a little tricky to use and I came across it after asking my own question about this topic: How to directly update pixels - with CGImage and direct CGDataProvider

How to make iPhone OpenGL ES context update straight away?

I have an OpenGL ES application for the iPhone I am developing, being a port of a 2d-oriented application from another platform. I have chosen to render the graphics using OpenGL ES for performance reasons. However, the main application runs on a separate thread (due to the original application design), so from within my app delegate I do this:
- (void) applicationDidFinishLaunching:(UIApplication *)application {
CGRect rect = [[UIScreen mainScreen] bounds];
glView = [[EAGLView alloc] initWithFrame:rect];
[window addSubview:glView];
// launch main application in separate thread
[NSThread detachNewThreadSelector:#selector(applicationMainThread) toTarget:self withObject:nil];
}
However, I notice that any calls within the applicationMainThread that try to render something to the screen do not render anything, until that thread terminates.
I set up the actual OpenGL ES context on the child application thread, not the UI thread. If I do this:
- (void) applicationMainThread {
CGRect rect = [[UIScreen mainScreen] bounds];
[glView createContext]; // creates the open GL ES context
//Initialize OpenGL states
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
glEnable(GL_TEXTURE_2D);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glMatrixMode(GL_PROJECTION);
glOrthof(0, rect.size.width, 0, rect.size.height, -1, 1);
glMatrixMode(GL_MODELVIEW);
Texture2D *tex = [[Texture2D alloc] initWithImage:[UIImage imageNamed:#"iphone_default.png"]];
glBindTexture(GL_TEXTURE_2D, [tex name]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glDisable(GL_BLEND);
[tex drawInRect:[glView bounds]];
glEnable(GL_BLEND);
[tex release];
[glView drawView];
}
Then the texture is updated to the screen pretty much immediately, as I would expect.
However, if after the [glView drawView] call I add this one line:
[NSThread sleepForTimeInterval:5.0]; // sleep for 5 seconds
Then the screen is only updated after the 5 second delay completes. This leads me to believe that the screen only updates when the thread itself terminates (need to do more testing to confirm). This means that when I substitute the actual application code, which does multiple screen updates, none of the updates actually happen (leaving a white screen) until the application thread exits, not exactly what I wanted!
So - is there any way I can get around this, or have I done something obviously wrong?
You have to be doing something obviously wrong, as multithreaded OpenGL rendering works just fine on iPhone. I can’t tell you what’s wrong with your code, but I can show you how we do it. It took me several iterations to get there, because the sample OpenGL code from Apple mashes everything together.
In the end I came up with three classes: Stage, Framebuffer and GLView. The Stage contains the game rendering logic and knows how to render itself to a framebuffer. The framebuffer class is a wrapper around the OpenGL framebuffer and renders to a renderbuffer or a EAGLDrawable. GLView is the drawable to render the framebuffer to, it contains all the OpenGL setup stuff. In the application entry point I create an OpenGL context, a GLView, a framebuffer that renders to this GLView and a Stage that renders using the framebuffer. The Stage update method runs in a separate thread and looks a bit like this:
- (void) mainLoop
{
[fbuffer bind];
[currentScene visit];
[[EAGLContext currentContext]
presentRenderbuffer:GL_RENDERBUFFER_OES];
[fbuffer unbind];
}
In plain English, it binds the framebuffer, walks the game object graph (= renders the scene), presents the framebuffer contents on the screen and unbinds the framebuffer. The presentRenderbuffer call is a bit misplaced, it belongs somewhere higher in the design – the Stage should just render into framebuffer and let you do whatever you want to do with the framebuffer. But I could not find the right place, so I just left the call there.
Otherwise I am pretty much content with the design: all the classes are simple, coherent and testable. There’s also a Scheduler class that creates the thread and calls Stage’s mainLoop as fast as possible:
- (void) loop
{
[EAGLContext setCurrentContext:context];
while (running)
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
#synchronized (context)
{
[stage mainLoop];
}
[fpsCounter update];
[pool release];
}
}
- (void) run
{
NSAssert(!running, #"Scheduler already running.");
running = YES;
[fpsCounter reset];
context = [EAGLContext currentContext];
[NSThread detachNewThreadSelector:#selector(loop)
toTarget:self withObject:nil];
}
The game update thread is synchronized using the OpenGL context so that we can be sure that we don’t corrupt the context in the main thread. (Simple rule: All drawing has to be done in the game update loop or synchronized by the GL context.)
Hope that helps.
Seems that I missed the bleeding obvious...
glViewport(0, 0, rect.size.width, rect.size.height);
glScissor(0, 0, rect.size.width, rect.size.height);
... and the updates appear as they should. I think what happened is without the viewport and scissor set on the child thread context which used a sharegroup (was set on the parent thread context), only when the child thread exited did the view update with the proper viewport, thus finally displaying my updates. Or something like that! (I'm still an OpenGLES newbie!)