iOS testing methodology - iphone

When I write any new code I generally add two custom macros like this:
- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
NOT_IMPLEMENTED();
NOT_TESTED();
}
Where these macros are just something simple like:
#define NOT_IMPLEMENTED() do{ __asm int 3 } while(0)
I use this for two reasons: 1) I can grep my source for functions I haven't implemented or stepped through in the debugger and 2) If I hit one of these conditions it just halts the debugger at that point rather than throwing an exception like NSAssert does. This way I can look at the callstack and proceed with execution from the point of where the int 3 was hit (P.S. 'int 3' only works on intel processors).
Now here's the rub. For things like 'didFailWithError' there's almost no documentation on this message can be sent. Ideally, I'd like to build a dummy webservice that knows how to trigger this condition so I know that I'm handling this type of condition. This is just an example. If you've ever worked with the in-app purchase API there's equally nebulous error delegates that give no indication of when they might be called.
I guess what I'm asking is what do other developers do to test error delegates that are available in many of the iOS libraries? Especially for the ones that give no indication on how they can be triggered.

Related

What is "super" in Objective-C? ( self != super )?

I've read the question below, and the story SEEMS simple:
What exactly is super in Objective-C?
Yet ...
- (id) init
{
NSLog(#"self=%p, super=%p", self, super);
}
That prints out "self=0xa83dc50, super=0xbfffe8d0". The addresses are NOT THE SAME???!?!?
That second address seems like a "special value" or something. What does it mean?
Thanks to bbum for pointing out that this value is the stack address of a special struct used by the compiler to implement the "super" behavior.
I can call [super init] and the call seems to work, or at least, nothing explodes ... not immediately. Calling [((id)0xbfffe8d0) init] fails hard with EXC_BAD_ACCESS.
Then there is the REALLY WEIRD part.....
I've got a piece of code that for no explainable reason throws a "NSGenericException: collection was mutated while being enumerated" exception. Inside a DIFFERENT object (basically a wrapper that has a pointer to the NSEnumerator), commenting out the call to "[super init]" causes the exception to not happen. If I could, I'd put out a $$$ reward for an answer to THAT mind-bender.
"id sups = (id)0xbfffe8d0" ... that also leads to "collection is modified." ... WTF? Ok, so I'm posting a 2nd question for that bizzariotity ...
I originally came here with one of those "bizarre symtoms" bugs, that turned out to be entirely unrelated (typical for such things): Does casting an address to (id) have side-effects??? Is Address 0xbfffe8d0 special? (fixed: issue was with _NSCallStackArray)
However, the content above the line is still valid, and the response still excellent. Read it if you want to understand ObjC just a little bit deeper.
You are messing with the man behind the curtain and he is punishing you for it... :)
super is a bit of compiler magic, really. When you say [super doSomething], the compiler will emit a call to objc_msgSendSuper() instead of objc_msgSend(). Most of the time -- there are some special cases.
In general, you should treat super as only a target for method calls. It should never be stored anywhere and should never be considered anything but that target expression for messaging.
In fact, uses of super that involve storage should likely be flagged by the compiler.
In terms of your bug, that sounds an awful lot like there is either corruption of memory, an over-release, and/or concurrency going on. You'll need to provide more code related to the enumeration and other relevant code to deduce further.

Exception handling in entire application

I have a few doubts regarding exception handling in iPhone. Here are they:
Suppose I have a chain of methods which are being called one after the other, that is, method A calls method B, which in turn calls method C, which calls method D. Which is the best place to put my try-catch block (is it method A or B or C or D or all of them). Also, I need to display an alert to user that an exception has occured and then, I want to log this exception to my server. So, if I am writing my try - catch block in all of this methods and if an exception occurs in method D; then I think the alert will be displayed 4 times and the web service for logging will also be called 4 times (till control reaches to catch block of method A). So, should I just use #throw; in catch block of method B, C and D and write my logic in catch block of method A (top level method) or should I avoid writing try - catch at all in methods B,C and D.
I need some sort of error code from the exception (because my web service needs parameters error-code and description). Is it possible to convert an exception to error or will I need to hard-code this code?
I had read somewhere about NSSetUncaughtExceptionHandler. And I think, if I can set this handler (in appDidFinishLaunching method of app delegate) and in the handler method, if I show to user some alert and call the web service; then I need not write try - catch block in each of my methods, in each of my classes. Am I right??
If an exception has occured and I have written either a try - catch block or NSSetUncaughtExceptionHandler, then will my app continue running or it will not respond to any of the user events. (I am sure it will handle the crash. What I want to know is whether it will hang)
Someone please enlighten me on this EXCEPTION TOPIC.
0) Avoid exceptions in Cocoa. They are generally non-recoverable. You might catch them for your own error reporting, but it's generally unsafe to assume you can recover from them.
1) If you need to catch, catch it immediately. Don't write your own throws -- instead, convert it to something like an NSError and pass that around. NSError can contain all the information you need to display or send an error code as well as a localized message.
2) You cannot convert an NSException into an NSError (directly) because an NSException does not have all the properties an NSError has - it is a different data representation. For one, an error code is not available. Two, the description is not localized. The best you can do is to create an error code and domain, then use the properties you need from the NSException and store that in an NSError. This could look something like the following:
// error checking omitted
extern NSString* const MONExceptionHandlerDomain;
extern const int MONNSExceptionEncounteredErrorCode;
NSError * NewNSErrorFromException(NSException * exc) {
NSMutableDictionary * info = [NSMutableDictionary dictionary];
[info setValue:exc.name forKey:#"MONExceptionName"];
[info setValue:exc.reason forKey:#"MONExceptionReason"];
[info setValue:exc.callStackReturnAddresses forKey:#"MONExceptionCallStackReturnAddresses"];
[info setValue:exc.callStackSymbols forKey:#"MONExceptionCallStackSymbols"];
[info setValue:exc.userInfo forKey:#"MONExceptionUserInfo"];
return [[NSError alloc] initWithDomain:MONExceptionHandlerDomain code:MONNSExceptionEncounteredErrorCode userInfo:info];
}
#catch (NSException * exc) {
NSError * err = NewNSErrorFromException(exc);
...
}
If the APIs you use throw exceptions you are expected to catch and recover from (e.g. not truly exceptional cases), then yes, you could catch and attempt to continue. Unfortunately, anybody that writes exceptions in Cocoa with the intent that you will catch them probably does not understand the issues well enough to implement a solid unwind implementation (e.g. even if it produces leaks, it's not solid).
3) That's really not the time or place to display an alert. If you install a top level exception handler (via NSSetUncaughtExceptionHandler) - You should simply log a message -- then the exception handler will abort. Your app is in an unstable state -- continuing is worse than aborting. You may want to send these custom messages home, it's best to do so at the next launch of your app.
4) In the majority of cases, your app is in an unstable state and you should not continue. But, to actually answer it for those corner cases: "Yes, you can recover and continue when you catch, but you should only attempt to recover and continue when the throwing API states that recovery is supported. If the problem is beyond your control, and the problem is not exceptional (e.g. file not found), and the vendor really expects you to continue, then I would have to assume that they expect you to continue, even though it really isn't (100% safe).". Do not attempt to recover/continue from within your top level exception handler (the program will abort after it returns). If you want to be very fancy and present that immediately on OSX, another process would be best. If you are calling through a pure C++ interface, then the unwinding is well defined, and the need to catch is necessary - do continue if it is recoverable. Exceptions in C++ can be recoverable and well defined - they are also used quite extensively (including less than exceptional conditions).
(IMO...) Exceptions in ObjC should not have been introduced, and any method that throws from system or third party libraries should be deprecated. They don't unwind well, or in a well defined manner. As well, unwinding flows against normal Cocoa program flow. That means that touching any objc object's memory/relations that was in mutation at the time of the throw and which lies between the throw and the catch is as good as undefined behaviour. Problem is - you have no idea what that memory is (in most cases, and within reasonable maintenance time). C++ exceptions are well defined, and they unwind correctly (e.g. destructors are called) - but trying to continue in an ObjC context is ignoring any consequences of undefined behavior. IMO, they should only exist for ObjC++ (because C++ requires them).
In an ideal world, your ObjC programs and the libraries you use would not use exceptions (at all). Since you use libraries that do throw (including Cocoa), install a top level exception handler only if you need some special information about the error. Where the API mandates that you could expect an exception thrown due to circumstances beyond your control and you are expected to recover, then write a catch but immediately convert that logic to normal program flow (e.g. NSError) - you never need to write your own throw. -[NSArray objectAtIndex: and "object does not respond to selector" are examples of programmer errors - they should not be caught, but the program should be corrected.

How to log all methods used in iOS app

I'm taking over the development of an iPad app for a client. There's a substantial amount of work that's already been done and I'm trying to piece together how the whole thing is designed to run.
One of the things I'd like to do is log which methods get called when the app runs. I've seen a custom DTrace script that's meant to log all methods from startup, but when I run it in Instruments I get no results.
What's the best way of logging the methods?
Inspired by tc's answer to a similar question here, I put together a debug breakpoint action that will log out the class and method name for every time objc_msgSend() is triggered in your application. This works similarly to the DTrace script I described in this answer.
To enable this breakpoint action, create a new symbolic breakpoint (in Xcode 4, go to the breakpoint navigator and create a new symbolic breakpoint using the plus at the bottom left of the window). Have the symbol be objc_msgSend, set it to automatically continue after evaluating actions, and set the action to be a debugger command using the following:
printf "[%s %s]\n", (char *)object_getClassName(*(long*)($esp+4)),*(long *)($esp+8)
Your breakpoint should look something like the following:
This should log out messages like this when run against your application:
[UIApplication sharedApplication]
[UIApplication _isClassic]
[NSCFString getCString:maxLength:encoding:]
[UIApplication class]
[SLSMoleculeAppDelegate isSubclassOfClass:]
[SLSMoleculeAppDelegate initialize]
If you're wondering where I pulled the memory addresses, read this Phrack article on the Objective-C runtime internals. The memory addresses above will only work against the Simulator, so you might need to tweak this to run against applications on the iOS devices. Collin suggests the following modification in his answer to run this on a device:
printf "[%s %s]\n", (char *)object_getClassName($r0),$r1
Also, I think you'll see that logging out every method called in your application will overwhelm you with information. You might be able to use some conditions to filter this, but I don't know if this will help you to learn how your code executes.
If you are using LLDB, you will need to use the following debugger commands. These were tested in Xcode 4.6.
Device:
expr -- (void) printf("[%s %s]\n", (char *)object_getClassName($r0),$r1)
Simulator:
expr -- (void) printf("[%s %s]\n", (char *)object_getClassName(*(long*)($esp+4)), *(long *)($esp+8))
To trace the app code under Xcode 6 on device, I had to use the following debugger expression.
expr -- (void) printf("[%s %s]\n", (char *)object_getClassName($arg1),$arg2)
Brad Larson's approach can be adapted to run on the device by using the debugger command:
printf "[%s %s]\n", (char *)object_getClassName($r0),$r1
More information can be found in the Technical Note here: technotes
later xcode versions you need to call like that
expr -- (void)printf("[%s, %s]\n",(char *) object_getClassName(*(long*)($esp+4)), (char *) *(long *)($esp+8) )
If you want to limit the output to just the messages sent to one class you can add a condition like this
(int)strcmp((char*)object_getClassName($r0), "NSString")==0
If you want to log methods in the Simulator on 64 bit, use the following command instead:
expr -- (void) printf("[%s %s]\n", (char *)object_getClassName($rdi), (char *) $rsi)
Or if that doesn't work, log it this way:
The main idea is to use $rdi for the object (self), and $rsi for the selector.
A fellow developer taught me to add the same two log statements to each method. One as the first line, the other as the last line. I think he has a script that does this automatically for his projects, but the result is:
NSLog(#"<<< Entering %s >>>", __PRETTY_FUNCTION__);
NSLog(#"<<< Leaving %s >>>", __PRETTY_FUNCTION__);
At the console, this will spit out something like:
<<< Entering -[MainListTableViewController viewDidLoad] >>>
Very helpful in tracking what is going on.

Obj C: Creating a standard try-catch-block for use through the application

I am building a app with many different parts which access remote api calls (both my own, and others). There are many errors that might happen, and to exacerbate the problem, different libraries handle these errors differently.
Essentially, i would like to use the same error handling blocks for all these remote calls.
This is how i would do it with Ruby, but i am not that sure how to manipulate objective c in the same manner
//universal function to handle standard remote errors across errors
def universal_handling
begin
yield
rescue Exception => e
// handle different exceptions accordingly
// allow crash if unexpected exception
end
end
//how i would use the above block
universal_handling{ //any of my remote call here }
So, i have 2 questions (sample code very much appreciated)
How would I write the equivalent code in Objective-C? It is critical that I can use the same handling block throughout the app
In iOS dev, is this good practice?
Thanks for any help rendered! Error handling can be a major pain in the ass, so i do want to get this right early on =)
Notes:
Blocks are perfectly fine. I am not intending to support < 4.2 versions.
I read most of the articles out there, but none answers how you can use blocks to write "wrappers" for a specific set of calls.
You can do something very similar with blocks:
typedef void(^WrappableBlock)(void);
^(WrappableBlock block) {
#try {
block();
}
#catch(...)
}
//handle exception
}
}
However, it's very important to realize that the Cocoa (and CocoaTouch) libraries are not exception-safe. Throwing an exception through Cocoa frameworks will lead to all sorts of problems as the frameworks will not properly handle or clean up from the exceptions, leaving your application in a possibly inconsistent state. The correct Cocoa-style is to use NSError and return flags to indicate error conditions. This is neither better nor worse than using exceptions, just a different philosophy.
To do something similar to your universal_handling with NSError is not quite so straight forward because it will require that anything you call comply with the NSError pattern. That said:
typedef BOOL(^WrappableBlock)(NSError**);
^(WrappableBlock block, NSError **err) {
BOOL success = block(err);
if(!success) {
// handle error
}
return success;
}
would wrap any method that takes just an NSError** and returns a BOOL to indicate the presence of an error. Obviously the utility of this wrapper is limited as you'll have to wrap any interesting method in an other block to handle any other parameters. Of course, since it's the NSError** pattern, you can always just handle the errors when/where you want and pass NULL as the NSError** parameter where you don't care (ignoring the return value).
One final note: if you are using libraries that may throw exceptions, you must catch those exceptions within the scope of the library call and handle them. Do not let the exceptions propagate as they might then propagate through Cocoa framework code. Thus, the utility of the universal_handling block you propose is limited.
I would suggest you this nice cocoawithlove article that explains how to handle unhandled exceptions in Objective-C and also how to recover from an exception instead of "just" crashing. That could be a good alternative to what you are looking for.
I'm afraid that Objective-C not being as dynamic as ruby, that is a bit harder to do.
Would love to see a good block based implementation too, if that's possible, even If I'm not using them yet, because of previous version compatibility.

Usage of NSException in iPhone Apps

One of my friends asked me not to use NSException in iPhone Apps. The reason he gave was "performance bottleneck". But I am not convinced of it.
Can someone confirm me that we should restrain from using NSException in iPhone App? If you have best practices for using NSException, please provide that too.
UPDATE:
This link requests us to use Exception handling at the app level. Have someone ever done it? Please give the advantages of it and any other performance hitches it could create.
In short:
Do not use exceptions to indicate anything but unrecoverable errors
It is only appropriate to use #try/#catch to deal with unrecoverable errors. It is never appropriate to use #throw/#try/#catch to do control-flow like operations on iOS or Mac OS X. Even then, consider carefully whether you are better off using an exception to indicate an unrecoverable error or simply crashing (call abort()); a crash often leaves behind significantly more evidence.
For example, it would not be appropriate to use for catching out-of-bounds exceptions unless your goal is to catch them and somehow report the error, then -- typically -- crash or, at the least, warn the user that your app is in an inconsistent state and may lose data.
Behavior of any exception thrown through system framework code is undefined.
Can you explain the "Behavior of any
exception thrown through system
framework code is undefined." in
detail?
Sure.
The system frameworks use a design where any exception is considered to be a fatal, non-recoverable, error; a programmer error, for all intents and purposes. There are a very limited number of exceptions (heh) to this rule.
Thus, in their implementation, the system frameworks will not ensure that everything is necessarily properly cleaned up if an exception is tossed that passes through system framework code. Sine the exception is, by definition, unrecoverable, why pay the cost of cleanup?
Consider this call stack:
your-code-1()
system-code()
your-code-2()
I.e. code where your code calls into system code which calls into more of your code (a very common pattern, though the call stacks are obviously significantly deeper).
If your-code-2 throws an exception, that the exceptions passes over system-code means the behavior is undefined; system-code may or may not leave your application in an undefined, potentially crashy or data-lossy, state.
Or, more strongly: You can't throw an exception in your-code-2 with the expectation that you can catch and handle it in your-code-1.
I've used exception handling for my fairly intensive audio app with no problems whatsoever. After much reading and a bit of benchmarking and disassembly analysis, I've came to the controversial conclusion that there is no real reason not to use them (intelligently) and plenty of reason not to (NSError pointer-pointers, endless conditionals...yuck!). Most of what people are saying on the forums is just repeating the Apple Docs.
I go into quite a bit of detail in this blog post but I'll outline my findings here:
Myth 1: #try/#catch/#finally is too expensive (in terms of CPU)
On my iPhone 4, throwing and catching 1 million exceptions takes about 8.5 seconds. This equates to about only 8.5 microseconds for each. Expensive in your realtime CoreAudio thread? Maybe a bit (but you'd never throw exceptions there would you??), but an 8.5μs delay in the UIAlert telling the user there was a problem opening their file, is it going to be noticed?
Myth 2: #try Blocks Have a Cost On 32bit iOS
The Apple docs speak of "zero-cost #try blocks on 64bit" and state that 32 bit incurs a cost. A little benchmarking and disassembly analysis seems to indicate that there is zero-cost #try blocks on 32bit iOS (ARM processor) as well. Did Apple mean to say 32bit Intel?
Myth 3: It Matters that Exceptions Thrown Through Cocoa Frameworks are Undefined
Yes, they are "undefined", but what are you doing throwing through the Apple framework anyway? Of course Apple doesn't handle them for you. The whole point of implementing exception handling for recoverable errors is to handle them locally - just not every-single-line "locally".
One edge case here is with methods like NSObject:performSelectorOnMainThread:waitUntilDone:. If the later parameter is YES, this acts like a synchronous function in which case you might be forgiven for expecting the exception to bubble up to your calling scope. For example:
/////////////////////////////////////////////////////////////////////////
#pragma mark - l5CCThread
/////////////////////////////////////////////////////////////////////////
#interface l5CCThread : NSThread #end
#implementation l5CCThread
- (void)main
{
#try {
[self performSelectorOnMainThread:#selector(_throwsAnException) withObject:nil waitUntilDone:YES];
} #catch (NSException *e) {
NSLog(#"Exception caught!");
}
}
- (void)_throwsAnException { #throw [NSException exceptionWithName:#"Exception" reason:#"" userInfo:nil]; }
#end
/////////////////////////////////////////////////////////////////////////
#pragma mark - l5CCAppDelegate
/////////////////////////////////////////////////////////////////////////
#implementation l5CCAppDelegate
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
l5CCThread *thd = [[l5CCThread alloc] init];
[thd start];
return YES;
}
// ...
In this case the exception would pass "through the cocoa framework" (the main thread's run loop) missing your catch and crashing. You can easily work around this using GCD's dispatch_synch and putting in it's block argument the method call plus any exception handling.
Why Use NSException's over NSError's
Anyone who's ever done work in one of the older C-based frameworks like Core Audio knows what a chore it is checking, handling and reporting errors. The main benefit #try/#catch and NSExceptions provides is to make your code cleaner and easier to maintain.
Say you have 5 lines of code which work on a file. Each might throw one of, say, 3 different errors (eg out of disk space, read error, etc.). Instead of wrapping each line in a conditional which checks for a NO return value and then outsources the NSError pointer investigation to another ObjC method (or worse, uses a #define macro!), you wrap all 5 lines in a single #try and handle each error right there. Think of the lines you'll save!
By creating NSException subclasses, you can also easily centralise error messages, and avoid having your code littered with them. You can also easily distinguish your app's "non-fatal" exceptions from fatal programmer errors (like NSAssert). You can also avoid the need for the "name" constant (the subclass's name, is the "name").
Examples of all this and more details about the benchmarks and disassembly are on this blog post...
Exceptions plus try/catch/finally is the paradigm used by pretty much every other major language (C++, Java, PHP, Ruby, Python). Maybe its time to drop the paranoia and embrace it as well...at least in iOS.
Generally ask yourself if you're trying to signal an error, or do you actually have an exceptional condition? If the former, then it's a very bad idea regardless of any performance issue, real or perceived. If the latter, it's most definitely the right thing to do.