Swift MVVM testing strategy and code coverage questions - swift

I've run into an issue when generating code coverage with Xcode for view models in an MVVM environment.
Our basic setup is the view controller makes requests to the view model, which in turn calls methods on a data manager that talks to web services.
I came up with what I thought was a reasonably elegant way to test the view models by creating a fake data manager that subclasses the actual data manager and overrides the function called by the VM.
The problem is that for this to work, the VM must be part of the app target and the test target. An apparent side effect of this is that code coverage is not generated for items belonging to two or more targets, even though the unit tests pass. Code coverage is enabled in the project.
Here is a excerpted view model:
import Foundation
class BoosViewModel: BaseViewModel {
convenience override init() {
self.init(dataManager: BoosDataManager(), andModel: nil)
}
func getUnlinkedBoos(_ cardType: CardType) {
(dataManager as! BoosDataManager).getUnlinkedBoos(cardType) { result, error in
...stuff happens here...
}
}
}
... and the data manager
class BoosDataManager: DataManager {
static let SharedInstance: BoosDataManager = {
var manager = BoosDataManager()
return manager
}()
func getUnlinkedBoos(_ cardType: CardType = .loyalty, completion: #escaping ((_ result: BoosModel?, _ error: NSError?) -> Void)) {
...stuff happens here...
}
}
...and the test
class BoosViewModelTests: XCTestCase {
func testGetUnlinkedBoosHappyPath() {
class FauxDataManager: BoosDataManager {
override func getUnlinkedBoos(_ cardType: CardType = .loyalty, completion: #escaping ((_ result: BoosModel?, _ error: NSError?) -> Void)) {
...stuff happens here...
}
}
let viewModel = BoosViewModel()
let dataManager = FauxDataManager()
viewModel.dataManager = dataManager
viewModel.getUnlinkedBoos(.loyalty)
XCTAssertTrue(testObserver.updated)
XCTAssertEqual(testObserver.newViewModel.getBoos().count, 1)
}
}
As I noted earlier the unit tests in this scenario complete successfully, but unit coverage does not get generated.
I have older tests where I actually created an external fake data manager class that was used by the test, the class under test is not part of the test target, and coverage works fine.
The drawback to that is that I have to create multiple data managers to handle specific cases for its returns. If I can't encapsulate the classes, I would need to create a bunch of swift data managers, one for each scenario.
That's why I came up with the internal class.
Now, the problem comes in if I remove the view model under test from the testing target. After doing this, I add #testable import BoosApp to the unit test so that the view model under test can be resolved. When I do this, I get the following error:
Could not cast value of type 'BoosTests.BoosViewModelTests.(testGetUnlinkedBoosHappyPath () -> ()).(FauxDataManager #1)' (0x11f673d18) to 'Boos.BoosDataManager' (0x10444b128).
Aug 30 20:43:01 Pay[19025] : Could not cast value of type 'BoosTests.BoosViewModelTests.(testGetUnlinkedBoosHappyPath () -> ()).(FauxDataManager #1)' (0x11f673d18) to 'Boos.BoosDataManager' (0x10444b128).
I'm not sure what I'm missing. Is there a way to make this scenario work, or am I stuck creating multiple data managers outside of the test code?

Ultimately, I figured out the main issue was that the view model and data manager had somehow gotten added to the test target. After removing them from the test target I was able to make a couple of minor changes and everything is running fine. FYI.

Related

NSXPCInterface setClasses causes "Lazily named class 0x600000xxxxxx wasn’t named by lazy name handler" on Xcode 14+ Swift #objc enum

After updating to Xcode 14 I am getting the SIGABRT crash "Lazily named class 0x600000dc6520 wasn’t named by lazy name handler". In the latest version of Xcode 13 it compiles and runs without any flaw. The only thing different from the very basic is that I use the MyAppSharedObjects to get across more complex objects between the XPC Service and the main app. Apart from that it doesn't look like anything beyond tutorial level NSXPCInterface code to me:
import MyAppScriptSandbox
import MyAppSharedObjects
import Foundation
class ScriptExecutor {
init?() {
let incomingClasses = NSSet(array: [
NSArray.self,
NSString.self,
NSValue.self,
NSNumber.self,
NSData.self,
NSDate.self,
NSNull.self,
NSURL.self,
NSUUID.self,
NSError.self,
NSDictionary.self,
ScriptSandboxReply.self,
AppleScriptError.self,
AppleScriptErrorType.self
]) as Set
let remoteInterface = NSXPCInterface(with: MyAppScriptSandboxProtocol.self)
remoteInterface.setClasses( // **** CRASH HAPPENS HERE ****
incomingClasses,
for: #selector(MyAppScriptSandboxProtocol.execute(script:withReply:)),
argumentIndex: 0,
ofReply: true
)
import Foundation
import MyAppSharedObjects
#objc
public protocol MyAppScriptSandboxProtocol {
func execute(
script: String,
withReply reply: #escaping (ScriptSandboxReply) -> Void
)
func terminate()
}
#objc(ScriptSandboxReply)
public class ScriptSandboxReply: NSObject, NSSecureCoding {
public static let supportsSecureCoding = true
public func encode(with coder: NSCoder) {
// Removed company specific code
}
required public init?(coder: NSCoder) {
// Removed company specific code
}
}
This data type was the issue:
#objc(AppleScriptErrorType)
public enum AppleScriptErrorType: Int {
case error
case noResult
case errorNorResult // This really shouldn't happen IRL
static let key = "AppleScriptErrorType"
}
After a bit of dabbing I found the issue. I started to comment out custom data types in the incomingClasses list and while the application crashed upon the first received message it stopped crashing on init. By uncommenting the custom data types one by one I finally found the culprit:
#objc(AppleScriptErrorType)
public enum AppleScriptErrorType: Int {
case error
case noResult
case errorNorResult // This really shouldn't happen IRL
static let key = "AppleScriptErrorType"
}
I have no idea why this perfectly sane enum type could not be transferred anymore after Xcode 14 dropped while it worked in Xcode 13.4.1, I think it might be an Apple bug. However I could get on with my life using the rawValue (so just the int value really) to send the value from one side and to reconstruct the AppleScriptErrorType enum on the receiving side.

How can I verify a class method is called using XCTAssert?

I have a service class, I would like to assert 2 things
A method is called
The correct params are passed to that method
Here is my class
protocol OAuthServiceProtocol {
func initAuthCodeFlow() -> Void
func renderOAuthWebView(forService service: IdentityEndpoint, queryitems: [String: String]) -> Void
}
class OAuthService: OAuthServiceProtocol {
fileprivate let apiClient: APIClient
init(apiClient: APIClient) {
self.apiClient = apiClient
}
func initAuthCodeFlow() -> Void {
}
func renderOAuthWebView(forService service: IdentityEndpoint, queryitems: [String: String]) -> Void {
}
}
Here are my tests
class OAuthServiceTests: XCTestCase {
var mockAPIClient: APIClient!
var mockURLSession: MockURLSession!
var sut: OAuthService!
override func setUp() {
mockAPIClient = APIClient()
mockAPIClient.session = MockURLSession(data: nil, urlResponse: nil, error: nil)
sut = OAuthService(apiClient: mockAPIClient)
}
func test_InitAuthCodeFlow_CallsRenderOAuthWebView() {
let renderOAuthWebViewExpectation = expectation(description: "RenderOAuthWebView")
class OAuthServiceMock: OAuthService {
override func initAuthCodeFlow() -> Void {
}
override func renderOAuthWebView(forService service: IdentityEndpoint, queryitems: [String: String]) {
renderOAuthWebViewExpectation.fulfill()
}
}
}
}
I was hoping to create a local sub class of OAuthService, assign that as my sut and call something like like sut.initAuthCodeFlow() and then assert that my expectation was fulfilled.
I believe this should satisfy point 1. However I cannot access my expectation when attempting to assign it as fulfilled as I get the following error
Class declaration cannot close over value
'renderOAuthWebViewExpectation' defined in outer scope
How can I mark this as fulfilled?
I am following a TDD approach, so I understand my OAuthService would produce a failing test at this point anyway*
I was hoping to create a local sub class of OAuthService, assign that as my sut and call something like like sut.initAuthCodeFlow() and then assert that my expectation was fulfilled.
I would strongly discourage you from using this approach. If your SUT is an instance of the subclass then your test is not truly testing OAuthService, but OAuthService mock.
Moreover, if we think of tests as a tool to:
prevent bugs when code is change
help refactoring and maintenance of the code
then I would argue that testing that calling a certain function calls another function is not a good test. That's harsh, I know, so let me unpack why that's the case.
The only thing it's testing is that initAuthCodeFlow() calls renderOAuthWebView(forService:, queryitems:) under the hood. It doesn't have any assertion on the actual behaviour of the system under test, on the outputs it produces directly or not. If I were to edit the implementation of renderOAuthWebView(forService:, queryitems:) and add some code that would crash at runtime this test would not fail.
A test like this doesn't help with keeping the codebase easy to change, because if you want to change the implementation of OAuthService, maybe by adding a parameter to renderOAuthWebView(forService:, queryitems:) or by renaming queryitems into queryItems to match the capitalization, you'll have to update both the production code and the test. In other words, the test will get in your way of refactoring -changing how the code looks without changing how it behaves- without any extra benefit.
So, how should one test OAuthService in a way that prevents bugs and helps moving fast? The trick is all in testing the behaviour instead of the implementation.
What should OAuthService do? initAuthCodeFlow() doesn't return any value, so we can check for direct outputs, but we can still check indirect outputs, side effects.
I'm making a guess here, but I from your test checking that renderOAuthWebView(forService:, queryitems:) I'd and the fact that it gets an APIClient type as input I'd say it'll present some kind of web view for a certain URL, and then make another request to the given APIClient maybe with the OAuth token received from the web view?
A way to test the interaction with APIClient is to make an assertion for the expected endpoint to be called. You can do it with a tool like OHHTTPStubs or with your a custom test double for URLSession that records the requests it gets and allows you to check them.
As for the presentation of the web view, you can use the delegate patter for it, and set a test double conforming to the delegate protocol which records whether it's called or not. Or you could test at a higher level and inspect the UIWindow in which the test are running to see if the root view controller is the one with the web view.
At the end of the day is all a matter of trade offs. The approach you've taken is not wrong, it just optimizes more towards asserting the code implementation rather than its behaviour. I hope that with this answer I showed a different kind of optimization, one biased towards the behaviour. In my experience this style of testing proves more helpful in the medium-long run.
Create a property on your mock, mutating it's value within the method you expect to call. You can then use your XCTAssertEqual to check that prop has been updated.
func test_InitAuthCodeFlow_CallsRenderOAuthWebView() {
let renderOAuthWebViewExpectation = expectation(description: "RenderOAuthWebView")
class OAuthServiceMock: OAuthService {
var renderOAuthWebViewExpectation: XCTestExpectation!
var didCallRenderOAuthWebView = false
override func renderOAuthWebView(forService service: IdentityEndpoint, queryitems: [String: String]) {
didCallRenderOAuthWebView = true
renderOAuthWebViewExpectation.fulfill()
}
}
let sut = OAuthServiceMock(apiClient: mockAPIClient)
XCTAssertEqual(sut.didCallRenderOAuthWebView, false)
sut.renderOAuthWebViewExpectation = renderOAuthWebViewExpectation
sut.initAuthCodeFlow()
waitForExpectations(timeout: 1) { _ in
XCTAssertEqual(sut.didCallRenderOAuthWebView, true)
}
}

Patterns: Singletons vs. Static vars and methods approach

I am reading a lot about the Singleton Pattern. I am currently using it to store groups of global state in my first app. I am reaching a point where I wonder which approach to implement API client classes and similar with.
Are Structs with static vars and static functions having the same issues?
To illustrate what I mean, I've tried to write the same heavily simplified and exact same(?) scenario twice.
1. A singleton being worked with by a view controller:
struct APIClientSingletonClass {
static let shared = APIClientSingletonClass()
var stateOfSomehting: Bool = true
var stateOfSomehtingMore: Bool = false
var stateNumber: CGFloat = 1234
var stateOfSomehtingComputed: CGFloat {
return stateNumber * 10
}
func convertSomethingToSomethingElse() {
// calling method in self like this:
otherMethod()
}
func otherMethod() {
// doing stuff here
}
}
// Calling APIClient from outside:
class ViewControllerTalkingToSingleton: UIViewController {
var api = APIClientSingletonClass.shared
override func viewDidLoad() {
super.viewDidLoad()
api.convertSomethingToSomethingElse()
api.stateOfSomehting = false
}
}
2. Another approach:
struct APIClientStruct {
static var stateOfSomehting: Bool = true
static var stateOfSomehtingMore: Bool = false
static var stateNumber: CGFloat = 1234
static var stateOfSomehtingComputed: CGFloat {
return stateNumber * 10
}
static func convertSomethingToSomethingElse() {
// calling method in self like this:
APIClientStruct.otherMethod()
}
static func otherMethod() {
// doing stuff here
}
}
// Calling APIClient from outside:
class ViewControllerTalkingToStruct: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
APIClientStruct.convertSomethingToSomethingElse()
APIClientStruct.stateOfSomehting = false
}
}
What do you guys think? Is approach 2 falling into the same traps that seem to make Singletons such a double-edged sword?
Any input is really appreciated!
Best from Berlin
EDIT:
This thread is pretty interesting, but I'm not sure it really relates to my question:
Difference between static class and singleton pattern?
Since there are many perspectives on this topic, let me specify:
Does my approach 2 have the same problem implications with testing and code maintainability?
A class-based singleton is the way to go, provided you accommodate for dependency injection for your tests. The way to do this is to create a single singleton for your app, called, say, DependencyManager. In your AppDelegate (or from other classes if needed), you'd create whatever controllers, network services, realm models, etc you want to hang on your DependencyManager, and then assign them to the DependencyManager. This code would be skipped by your unit tests.
Your unit tests can then access the DependencyManager (and thus instantiate the DependencyManager during first access), and populate it with mock versions of those controllers and services to whatever degree each unit test desires.
Your UIViewControllers, your MVVM view models, etc... can access the DependencyManager as a singleton, and thus get either the real controllers and services, or a mock version of them, depending on if you're running the app or unit tests.
If you're doing MVVM, I also recommend that when a UIViewController is about to create its view model class, that it first checks a special property in the DependencyManager to see if a mockViewModel exists. A single property can serve this purpose, as only one of your UIViewControllers ever would be tested at once. It'd use that property instead of creating a new view model for itself. In this way, you can mock your view models when testing each UIViewController. (There's other tricks involved to being able to prop up a single UIViewController for testing, but I won't cover that here).
Note that all of the above can work very nicely with an app that also wants to use storyboards and/or nibs. People are so down on storyboards because they can't figure out how to do dependency injection of mock services for their view controllers. Well, the above is the solution! Just make sure in your AppDelegate to load the storyboard AFTER setting up the DependencyManager. (Remove the storyboard name from your info.plist, and instantiate it yourself in AppDelegate).
I've written a few shipped apps this way, as well as some sample apps for an SDK, along with the tests. I highly recommend the approach! And be sure to write your unit tests and viewController tests either during or at least immediately after development of each such class, or you'll never get around to them!
What generally makes sinlgetons hard to test is that the singleton objects are typically always accessed directly . Because of this, you don't have a means to substitute the real singleton object (e.g. a data-store that's backed by a database) with a mock object for testing (e.g. a data-store that's backed by an easily-configurable array of predefined test data).
Using static members has the same fundamental issue. When referencing a static member directly, you don't have a means of substituting a mock object in place of the real prod implementation.
The solution to this is quite simple: don't access singleton members directly. What I do is something like this:
// An example of a dependency.
protocol DataAccessLayer {
func getData() -> [Int]
}
// A real implementation of DataAccessLayer, backed by a real production database
class ProdDB: DataAccessLayer {
static let instance = ProdDB()
private init() {}
func getData() -> [Int] {
return [1, 2, 3] // pretend this actually queries a DB
}
}
// A mcok implementation of DataAccessLayer, made for simple testing using mock data, without involving a production database.
class MockDB: DataAccessLayer {
func getData() -> [Int] {
return [1, 2, 3] // The mock *actually* hardcodes this data
}
}
// A protocol that stores all databases and services used throughout your app
protocol ServiceContextProtocol {
var dataAccessLayer: DataAccessLayer { get } // Present via protocol, either real impl or mock can go here
//var fooAPIGateway: FooAPIGateway { get }
//... add all other databases and services here
}
// The real service context, containing real databases and service gateways
class ProdServiceContext: ServiceContextProtocol {
let dataAccessLayer: DataAccessLayer = ProdDB.instance
//var fooAPIGateway: ProdFooAPIGateway { get }
//... add all other prod databases and services here
}
// A mock service context, used in testing, which provides mocked databases and service gatways
class MockServiceContext: ServiceContextProtocol {
let dataAccessLayer: DataAccessLayer = MockDB()
//var fooAPIGateway: MockFooAPIGateway { get }
//... add all other mock databases and services here
}
let debug = false // Set this true when you're running in a test context
// A global variable through which you access all other global state (databases, services, etc.)
let ServiceContext: ServiceContextProtocol = debug ? MockServiceContext() : ProdServiceContext()
// Always reference ServiceContext.dataAccessLayer, ServiceContext.fooAPIGateway, etc.
// and *never* reference ProdDB.instance of MockDB directly.
I would use a Class based Singleton. Just remember the 2 criteria for having a singleton. You want GLOBAL ACCESS and SINGLE INSTANCE in your program. There is a couple problems where struct based singleton would fail. Once you assign a struct to a new variable, Swift makes a complete copy under the hood.
Another useful snip of information can be found using this link.
What's the difference between Struct based and Class based singletons?

Is it possible to hide classes from ui tests

I have a few helper classes like UnlockedTestCase that configure my app for special scenario tests.
Theses classes show up in the test navigator without tests.
Is there a way to mark then as "not test classes" in order for them to be ignored by the test navigator?
UPDATE: Sample code:
class UnlockedTestCase: XCTestCase {
var app = XCUIApplication()
override func setUp() {
super.setUp()
continueAfterFailure = false
app.launchArguments = ["uiTesting", "unlock"]
app.launch()
}
}
A test would then be written as:
class UnlockedUITests: UnlockedTestCase {
func testButton() {
XCTAssers(app.buttons["SomeButtonInTheUnlockedState"].exists)
}
}
No, there is not a way to exclude that kind of class from the test navigator without losing the ease of defining the setUp(), as the way that it discovers test case classes is simplistic, and from the point of view of the navigator, you could add test cases to the parent/helper class at any moment since it is an XCTestCase descendant.
There is no protocol for 'helper' or 'abstract-only' classes which support inheritance in the way that you require, since inheritance from XCTestCase is required for the automatic discovery and usage of tests and test hooks like setUp().
If you really want to get rid of your helper entities from the test navigator, you could abstract them into protocols with extensions where Self: XCTestCase (to allow you access to XCTestCase's interface in the extension), have your test class conform to them, and override setUp() in the class where your tests are.
protocol UnlockableTest {}
extension UnlockableTest where Self: XCTestCase {
func unlockSetUp() {
continueAfterFailure = false
app.launchArguments = ["uiTesting", "unlock"]
app.launch()
}
}
class UnlockedUITests: XCTestCase, UnlockableTest {
var app = XCUIApplication()
override func setUp() {
super.setUp()
unlockSetUp()
}
func testButton() {
XCTAssert(app.buttons["SomeButtonInTheUnlockedState"].exists)
}
}
However, I think the simplicity and convenience of your current approach is the preferable compromise. Protocols also can't contain stored properties though, so in order to get the benefit of a stored property, you'd need to add the app property to every XCTestCase descendant too.
You can do this with Swift packages now, if your tests are in a Swift Package then you can have a helper module that contains your base classes and these will not be added to the sidebar as empty test cases.
You can mix an xcodeproj with a local to the source folder swift package to facilitate this, and in general I would personally recommend this as it improves build times and allows you to make your codebase more modular with granular access control.
Assuming the following structure:
MyTestHelperClassName: XCTestCase {
//... your helper methods
}
Remove its subclass declaration of XCTestCase
i.e the : XCTestCase part
Then it will not appear in the "Test Navigator"

Swift Privileged Helper (XPC Listener) Crashing with Illegal Instruction Error

I’ve created a Swift macOS app which uses SMJobBless to create a helper with escalated privileges. This works fine—the helper gets installed to /Library/Privileged Helper Tools and an accompanying LaunchDaemon gets created in /Library/LaunchDaemons. However, the helper is unable to start successfully. Instead, it crashes with an “Illegal instruction: 4” message.
I’ve prepared the helper to respond to XML connections by implementing the NSXPCListenerDelegate protocol. Here‘s my Helper main.swift code:
import Foundation
class HelperDelegate: NSObject, NSXPCListenerDelegate {
func listener(_ listener: NSXPCListener, shouldAcceptNewConnection newConnection: NSXPCConnection) -> Bool {
newConnection.exportedInterface = NSXPCInterface(with: HelperToolProtocol.self)
newConnection.exportedObject = HelperTool()
newConnection.resume()
return true
}
}
let delegate = HelperDelegate()
let listener = NSXPCListener.service()
listener.delegate = delegate
listener.resume()
The crash occurs on the last line, listener.resume().
I tried to launch the helper app manually from the command line (which is identical to what the LaunchDaemon does) and, again, it crashes with the above error message printed to stdout. I don’t have any more ideas on how to test this for the root cause. My implementation is more than rudimentary, following Apple’s guidlines for implementing XM services. Also, the various posts on SO regarding XML services haven’t helped me in resolving this issue. Has anyone of you tried to create a privileged helper in Swift successfully? BTW, the app is not sandboxed.
For the sake of completeness, here’s the code for the HelperTool class referenced in my HelperDelegate class above:
import Foundation
class HelperTool: NSObject, HelperToolProtocol {
func getVersion(withReply reply: (NSData?) -> ()) {
let version = Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString" as String) as? String ?? "<unknown version>"
let build = Bundle.main.object(forInfoDictionaryKey: kCFBundleVersionKey as String) as? String ?? "<unknown build>"
if let d = "v\(version) (\(build))".data(using: .utf8, allowLossyConversion: false) {
reply(d as NSData)
}
}
}
And finally the HelperToolProtocol:
import Foundation
#objc(HelperToolProtocol) protocol HelperToolProtocol {
func getVersion(withReply: (NSData?) -> ())
}
Thanks for any help!
After days of testing I finally found a solution which makes my XPC helper launch correctly and respond to any messages. The problem lies in the last three lines of the main.swift module which currently read
let listener = NSXPCListener.service()
listener.delegate = delegate
listener.resume()
which, as put in the question, make the helper crash immediately upon the very last line.
I took these lines directly from Apple’s Creating XPC Services documentation. Here’s the documentation for the NSXPCListener resume() function:
If called on the service() object, this method never returns. Therefore, you should call it as the last step inside the XPC service's main function after setting up any desired initial state and configuring the listener itself.
The solution is to not call the NSXPCListener.service() singleton object but rather instantiate a new NSXPCListener object using the init(machServiceName:)initializer passing the same Mach service name that is being used on the main app’s XPC connection. As resume() in this case would resume immediately—thus terminating the helper—you have to put it on the current run loop to have it run indeterminately. Here’s the new, working code:
let listener = NSXPCListener(machServiceName: "Privilege-Escalation-Sample.Helper")
listener.delegate = delegate
listener.resume()
RunLoop.current.run()