Occasional errors when running OCUnit application test suite on device - iphone

Question

I'm having some problems with my OCUnit test suite, when running application tests (i.e. on the device). Occasionally, I get a tricky error that seems to be thrown from the OCUnit classes. I've been trying to look through the files added to the test suites, but can't get my head around it.
I've followed the Apple documentation and looked at other tutorials when setting up my test targets, resulting in the following targets (and indented dependencies):
LogicTests
MyApp
LogicTests
ApplicationTests
MyApp
MyAppTesting
ApplicationTests
The last (MyAppTesting) is the one I run on the device to start the test suite.
And as mentioned, the tests fail to start sometimes throwing me the following output in the console:
2010-11-17 12:02:48.537 VCCTesting[12018:307] *** Assertion failure in -[SenTestClassEnumerator init], /SourceCache/OCUnit/OCUnit-1685/SourceCode/SenTestingKit/SenTestClassEnumerator.m:137
2010-11-17 12:02:48.556 VCCTesting[12018:307] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'objc_getClassList returned more classes than it should have.'
*** Call stack at first throw:
(
0 CoreFoundation 0x30897ed3 __exceptionPreprocess + 114
1 libobjc.A.dylib 0x3002f811 objc_exception_throw + 24
2 CoreFoundation 0x30897d15 +[NSException raise:format:arguments:] + 68
3 Foundation 0x349f932f -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 62
4 SenTestingKit 0x20107c29 -[SenTestClassEnumerator init] + 292
5 SenTestingKit 0x201079fd +[SenTestClassEnumerator classEnumerator] + 24
6 SenTestingKit 0x2010777d +[NSObject(SenTestRuntimeUtilities) senAllSubclasses] + 92
7 SenTestingKit 0x201067b5 +[SenTestSuite updateCache] + 28
8 SenTestingKit 0x20106753 +[SenTestSuite suiteForBundleCache] + 54
9 SenTestingKit 0x201065db +[SenTestSuite structuredTests] + 14
10 SenTestingKit 0x201065a1 +[SenTestSuite allTests] + 40
11 SenTestingKit 0x20106573 +[SenTestSuite defaultTestSuite] + 14
12 SenTestingKit 0x201057a3 +[SenTestProbe specifiedTestSuite] + 106
13 SenTestingKit 0x20105d83 +[SenTestProbe runTests:] + 94
14 Foundation 0x349c7e8d __NSFireDelayedPerform + 368
15 CoreFoundation 0x3084e7fb __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 14
16 CoreFoundation 0x3084e2ad __CFRunLoopDoTimer + 860
17 CoreFoundation 0x3081f7a5 __CFRunLoopRun + 1088
18 CoreFoundation 0x3081f277 CFRunLoopRunSpecific + 230
19 CoreFoundation 0x3081f17f CFRunLoopRunInMode + 58
20 GraphicsServices 0x31e445f3 GSEventRunModal + 114
21 GraphicsServices 0x31e4469f GSEventRun + 62
22 UIKit 0x31e51123 -[UIApplication _run] + 402
23 UIKit 0x31e4f12f UIApplicationMain + 670
24 VCCTesting 0x000234ff main + 70
25 VCCTesting 0x00002538 start + 40
)
terminate called after throwing an instance of 'NSException'
Does anyone know what could be wrong?
Thankful for any help,
Kristofer
P.S. I've posted this on the App dev forums, but haven't had any luck with it yet: https://devforums.apple.com/message/333890

Answer 1

Do you have NSZombieEnabled? I had this issue when I had NSZombieEnabled set to YES in my arguments for my unit test launch target. I've tried everything (cleaning/quitting XCode/restting simulator content).
The only solution so far was to disable zombie in my unit test launch

Answer 2

I think I've just figured out why I was getting this error as well -- perhaps the same thing is happening to you:
I have two products set up, one for internal beta testing (complete with a preference pane) and one for app store release. Both product targets were set up with the same Product Name (PRODUCT_NAME in build settings).
This worked fine in XCode 3.x, but XCode 4 was somehow trying to build the store product each time I built the beta product. I think this is due to the change in where XCode puts its derived files -- XCode 4 was storing the beta and store products at the same path, so when the beta product was built, the store output file changed and needed to be rebuilt...
Anyway... changing the beta Product Name has fixed temporarily fixed the issue... I've also changed the bundle identifier in the Info.plist so the two products have different identifiers. Now when I build one product, the other doesn't get built at the same time, but I still get the crash when the tests run :-(
My only idea now is to totally recreate the project from fresh, copying across the settings and the files to see if XCode 4 does something different. That and filing a support incident with Apple...
Cheers,
Adam

Answer 3

I am not 100% sure yet, but it seems this issue can be solved by creating different scheme for each test target.
I had this error a lot this afternoon but it hasn't happened for awhile after I created separate schemes for different test targets.

Answer 4

Just for anyone that might run into this issue like I have:
KIFTester loaded
Assertion failure in -[SenTestClassEnumerator init], /SourceCache/OCUnit/OCUnit-5315/SourceCode/SenTestingKit/SenTestClassEnumerator.m:136
Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'objc_getClassList returned more classes than it should have.'
The problem turned out to be GoogleAnalytics 3.0
Updated to 3.0.3 but still the same problem with either of these libraries
libGoogleAnalytics_debug.a
libGoogleAnalyticsServices.a
To go around it I just don't include them for testing.