OpenGL ES 2.0 iPhone - Rendering on background thread block main thread - iphone

I'm rendering OpenGL Context on a background thread with a different EAGLContext than the main thread.
I use something like this:
- (void)renderInBackground {
EAGLContext *context = [[EAGLContext] alloc] init];
[EAGLContext setCurrentContext:context];
Rendering..
}
However, even though this is performed in a background thread, when using a heavy shader, the main thread gets blocked and the UI gets stuck.
Why is the background thread blocking the main thread?
the methods are not synchronized.

You do have finite CPU and GPU resources to perform all of the rendering and UI interactions, so if you max out the GPU you will slow down everything else in your application.
That said, you should be able to render on a background thread without completely halting all UI elements. I do some fairly intense rendering in my open source Molecules application, all of it using a background GCD queue, yet you can still scroll in popovers and otherwise interact with the interface.
I describe the process I use in this answer, but the basic setup is a single-wide GCD queue that relies on a dispatch semaphore to prevent the enqueueing of additional rendering frames while one is still being processed. Threads and blocks have some overhead to them, so if they are being fired off faster than they can be processed, this can lead to resource exhaustion. The semaphore prevents this.
Wrapping all interactions with my OpenGL ES context in this queue provides lock-free usage of this shared resource, and I see a significant performance boost over simply running this all on the main thread on the multicore devices. As I said, I'm still able to interact with the UI during even heavy rendering here.

Related

Why must UIKit operations be performed on the main thread?

I am trying to understand why UI operations can't be performed using multiple threads. Is this also a requirement in other frameworks like OpenGL or cocos2d?
How about other languages like C# and javascript? I tried looking in google but people mention something about POSIX threads which I don't understand.
In Cocoa Touch, the UIApplication i.e. the instance of your application is attached to the main thread because this thread is created by UIApplicatioMain(), the entry point function of Cocoa Touch. It sets up main event loop, including the application’s run loop, and begins processing events. Application's main event loop receives all the UI events i.e. touch, gestures etc.
From docs UIApplicationMain(),
This function instantiates the application object from the principal class and instantiates the delegate (if any) from the given class and sets the delegate for the application. It also sets up the main event loop, including the application’s run loop, and begins processing events. If the application’s Info.plist file specifies a main nib file to be loaded, by including the NSMainNibFile key and a valid nib file name for the value, this function loads that nib file.
These application UI events are further forwarded to UIResponder's following the chain of responders usually like UIApplication->UIWindow->UIViewController->UIView->subviews(UIButton,etc.)
Responders handle events like button press, tap, pinch zoom, swipe etc. which get translated as change in the UI. Hence as you can see these chain of events occur on main thread which is why UIKit, the framework which contains the responders should operate on main thread.
From docs again UIKit,
For the most part, UIKit classes should be used only from an application’s main thread. This is particularly true for classes derived from UIResponder or that involve manipulating your application’s user interface in any way.
EDIT
Why drawRect needs to be on main thread?
drawRect: is called by UIKit as part of UIView's lifecycle. So drawRect: is bound to main thread. Drawing in this way is expensive because it is done using the CPU on the main thread. The hardware accelerate graphics is provided by using the CALayer technique (Core Animation).
CALayer on the other hand acts as a backing store for the view. The view will then just display cached bitmap of its current state. Any change to the view properties will result in changes in the backing store which get performed by GPU on the backed copy. However, the view still needs to provide the initial content and periodically update view. I have not really worked on OpenGL but I think it also uses layers(I could be wrong).
I have tried to answer this to the best of my knowledge. Hope that helps!
from : https://www.objc.io/issues/2-concurrency/thread-safe-class-design/
It’s a conscious design decision from Apple’s side to not have UIKit be thread-safe. Making it thread-safe wouldn’t buy you much in terms of performance; it would in fact make many things slower. And the fact that UIKit is tied to the main thread makes it very easy to write concurrent programs and use UIKit. All you have to do is make sure that calls into UIKit are always made on the main thread.
So according to this the fact that UIKit objects must be accessed on the main thread is a design decision by apple to favor performance.
C# behaves the same (see eg here: Keep the UI thread responsive). UI updates have to be done in the UI thread - most other things should be done in the background hen possible.
If that wouldn't be the case there would probably be a synchronization hell between all updates that have to be done in the UI ...
Every system, every library, needs to be concerned about thread safety and must do things to ensure thread safety, while at the same time looking after correctness and performance as well.
In the case of the iOS and MacOS X user interface, the decision was made to make the UI thread safe by only allowing UI methods to be called and executed on the main thread. And that's it.
Since there are lots of complicated things going on that would need at least serialisation to prevent total chaos from happening, I don't see very much gained from allowing UI on a background thread.
Because you want the user to be able to see the UI changes as they happen. If you were to be able to perform UI changes in a background thread and display them when complete, it would seem the app doesn't behave right.
All non-UI operations (or at least the ones that are very costly, like downloading stuff or making database queries) should take place on a background thread, whereas all UI changes must always happen on the main thread to provide as smooth of a user experience possible.
I don't know what it's like in C# for Windows Phone apps, but I would expect it to be the same. On Android the system won't even let you do things like downloading on the main thread, making you create a background thread directly.
As a rule of thumb - when you think main thread, think "what the user sees".

CADisplayLink accuracy issues with OpenGL ES on iOS

My iOS game is currently using CADisplayLink for timing of OpenGL rendering operations. I've got a GCD dispatch queue running on a second thread that issues all the OpenGL rendering and state calls to the GPU. Everything works fine, except the timing isn't perfect. I'm seeing frame skips and glitches on occasion, even when the animation interval is changed from 1 (60Hz) to 2 (30Hz).
CADisplayLink calls your selector from the RunLoop on the main thread, which means it can only fire in between other dispatched jobs and input events that the main thread is processing. (I've confirmed this is the case by logging these jobs/events). If those operations take several milliseconds, then CADisplayLink will never be perfectly accurate because it can't interrupt whatever is currently running on the main thread. As I imagine most games do, I'm running game simulation, physics, and scene culling on the main thread.
So what I'm thinking is that I should move all the game simulation and physics stuff off the main thread so that touch events and CADisplayLink can fire as close to when it should be as possible. But I'm not sure if this will solve anything and it's not a trivial amount of work.
I'm wondering, since the presentRenderBuffer call is really the thing that synchronizes your frame with the actual hardware display, maybe all I need is just a really accurate timer that can run in it's own thread (perhaps at higher priority) and trigger the rendering that way. Then I can keep everything on the main thread. It seems all CADisplayLink provides is a way to wait in the case when my code is running faster than 60Hz, so it seems to me that a similar kind of delay can be coded just as easily using a separate thread. What am I missing here?

Thread Safe (CG) Version of UIGraphicsGetImageFromCurrentImageContext?

Im trying to render PDF pages to images on a background thread, is there a thread safe way of generating an image from a pdfContext/context that doesn't not use:
UIGraphicsGetImageFromCurrentImageContext
AFAIK (almost) all Core Graphics calls should be safe to use in a background thread as long you don't draw on screen and don't share resources (such as graphics contexts) among multiple threads.

Is there a way to force Core Animation to run it's thread?

Core Animation uses a background thread to do it's job. Now the problem is this: I have a heavy calculation going on in the main thread. Core Animation immediately freezes until that calculation is done. And then it continues to finish it's animations. I remember reading in a document that CA has a low priority in processing time, meaning that whatever the main thread wants to do is high-prio and will be done more likely than any fancy animation at the same time.
I want to force Core Animation to schedule it's background thread nicely with the main thread under any circumstances. Or alternatively a separate thread that will run the heavy calculation outside the main thread. I tried that already, but CA still freezes until that's done. I expect the scheduler to switch processing time quickly between CA and that calculation.
How could CA be forced to keep on working? If things go just a bit slower than, that's fine. But most important thing is that all things keep on going from the users point of view.
You can use [CATransaction flush] to flush core animation if you don't let the runloop take its course
Don't run heavy calculations on the main thread, because they will block the UI and lead to a bad user experience. Run them in a background thread.
That said, the iPhone is a single-core system, so if a calculation is pegging the CPU in one thread, the performance of every other thread may grind to a near halt. If you can, try breaking your calculation into smaller elements and running them as NSOperations in an NSOperationQueue. If you make sure that the calculation segments aren't so small that the overhead of creating an NSOperation for them becomes too large, this might provide a means of throttling the calculation a bit so that your animations aren't being slowed down.
Core Animation tends to perform a number of calculations upfront, before an animation may run, so those may be getting slowed down by your heavy calculation thread. You might also be able to start your heavy calculation within the -animationDidStart: delegate method for your CAAnimation, making the calculation only kick off when the animation is in progress. I believe the progress of an animation uses fewer calculations than its start, so it may be better able to coexist with your heavy calculation.
I'm not sure, but you can try to call [NSThread setThreadPriority:1.0] if you running your CA in another thread.

Multi-threaded OpenGL Programming in Cocos2D-iPhone

In an attempt to create a loading bar for an iPhone game I'm developing (using Cocos2D), I wanted to use a multithreaded approach.
One thread shows a loading screen and runs the main application event loop while a new thread silently loads all the Sprites in the background (through spriteWithFile) and then adds them to a layer.
I create the new thread using NSThread's detachNewThreadSelector method (which sends updates of the loading status to the main thread via performSelectorOnMainThread).
The problem I'm facing is that any OpenGL calls (such as those found within the spriteWithFile method) in the new thread die with a BUS ERROR or memory access error of some sort. I'm assuming this is because both threads are attempting to make OpenGL calls at the same time or the new thread is unaware of the OpenGL context.
What has to be done to allow multiple threads to make OpenGL calls on the iPhone using Cocos2D-iPhone.
For the record, the new thread needs to execute the following two lines to be able to use the OpenGL API from a concurrent thread:
EAGLContext *k_context = [[[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES1 sharegroup:[[[[Director sharedDirector] openGLView] context] sharegroup]] autorelease];
[EAGLContext setCurrentContext:k_context];
This is now made obsolete by the addImageAsync method provided by the TextureMgr class in Cocos2D 0.8.x onwards that does asynchronous texture loading for you.
I want to do this too.
I'm starting from this thread.
PS: This answer is very old, now I'm not sure that asynchronous texture loading is as useful as it once was since iOS5 added "free" texture uploads via CVOpenGLESTextureCaches. Sure you still can (& should) load your assets in a secondary thread, but giving that thread an EAGLContext doesn't seem as necessary now.
Apple has some good guidelines for multithreaded OpenGL here.
Cocos2d best practices recommend against using NSTimer and I assume the same applies to threads as well. You should probably use Cocos' Timer object. This will leave the thread management to Cocos and should also let you access the correct graphics context.
HTH.