What if XCTestExpectation is unexpected - swift

I'm writing an XCTest unit test in Swift.
The idea is that a callback mustn't be called in a certain case.
So what I do, is
func testThatCallbackIsNotFired() {
let expectation = expectationWithDescription("A callback is fired")
// configure an async operation
asyncOperation.run() { (_) -> () in
expectation.fulfill() // if this happens, the test must fail
}
waitForExpectationsWithTimeout(1) { (error: NSError?) -> Void in
// here I expect error to be not nil,
// which would signalize that expectation is not fulfilled,
// which is what I expect, because callback mustn't be called
XCTAssert(error != nil, "A callback mustn't be fired")
}
}
When the callback is called, everything works fine: it fails with a message "A callback mustn't be fired" which is exactly what I need.
But if expectation hasn't been fulfilled, it fails and says
Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "Callback is fired".
Since a not fulfilled expectation is what I need, I don't want to have a failed test.
Do you have any suggestions what can I do to avoid this? Or, maybe, I can reach my goal in a different way? Thanks.

Use isInverted like in this post https://www.swiftbysundell.com/posts/unit-testing-asynchronous-swift-code
class DebouncerTests: XCTestCase {
func testPreviousClosureCancelled() {
let debouncer = Debouncer(delay: 0.25)
// Expectation for the closure we'e expecting to be cancelled
let cancelExpectation = expectation(description: "Cancel")
cancelExpectation.isInverted = true
// Expectation for the closure we're expecting to be completed
let completedExpectation = expectation(description: "Completed")
debouncer.schedule {
cancelExpectation.fulfill()
}
// When we schedule a new closure, the previous one should be cancelled
debouncer.schedule {
completedExpectation.fulfill()
}
// We add an extra 0.05 seconds to reduce the risk for flakiness
waitForExpectations(timeout: 0.3, handler: nil)
}
}

I had this same problem, and I am annoyed that you can't use a handler to override the timeout fail of waitForExpectationsWithTimeout. Here is how I solved it (Swift 2 syntax):
func testThatCallbackIsNotFired() {
expectationForPredicate(NSPredicate{(_, _) in
struct Holder {static let startTime = CACurrentMediaTime()}
if checkSomehowThatCallbackFired() {
XCTFail("Callback fired when it shouldn't have.")
return true
}
return Holder.startTime.distanceTo(CACurrentMediaTime()) > 1.0 // or however long you want to wait
}, evaluatedWithObject: self, handler: nil)
waitForExpectationsWithTimeout(2.0 /*longer than wait time above*/, handler: nil)
}

Related

Write xctest for function inside an operation queue

I have a function as follows which adds an operation to the operation queue, how do I test the block of code that is being added to the operation queue, I tried using an expectation by passing an analytics observer spy and check if the value is set but it does not seem to work, please point me in the right direction
func firePendingEvent() {
for analyticItem in eventQueue {
eventPriorityQueue.addOperationToQueue (operation: BlockOperation {
self.analyticsObserver?.logEvent(event: analyticItem) // write test to check the event inside this function
}, priority: .normal)
if eventQueue.count > 0 {
eventQueue.remove(at: 0)
}
}
}
The basic idea is that your test has to create an expectation:
let e = expectation(description: "testGetUsers")
And then when your asynchronous method is done, fulfill that expectation:
e.fulfill()
And then, of course, your test has to, after initiating the asynchronous tasks, wait for the expectations:
waitForExpectations(timeout: 10)
But, obviously, you need to know when your function is done. I would suggest giving it an optional completion handler:
func firePendingEvent(completion: (() -> Void)? = nil) {
let group = DispatchGroup()
for analyticItem in eventQueue {
group.enter()
eventPriorityQueue.addOperationToQueue(operation: BlockOperation {
self.analyticsObserver?.logEvent(event: analyticItem)
group.leave()
}, priority: .normal)
}
eventQueue.removeAll()
if let completion = completion {
group.notify(queue: .main, execute: completion)
}
}
(Note, I personally refrain from mutating an array as I iterate through it, so I add all the tasks to the queue and defer the removeAll until I am no longer iterating.)
Anyway, you can now write your test:
func testFirePendingEvents() {
let e = expectation(description: "testGetUsers")
foo.firePendingEvent {
XCAssert(…) // do whatever tests are appropriate
e.fulfill()
}
waitForExpectations(timeout: 10)
}
But, because that completion handler is optional, and defaults to nil, you don't have to change the rest of your code that isn't availing itself of the new completion handler.
Personally, I would add a parameter to that completion handler closure to pass back whatever value you want to test, but there wasn’t enough in the question to know what result you are testing.

How to measure performance of asynchronous method?

I've written a method loadCountries() that performs asynchronous file loading upon calling and then notifies its output that the work is done. I'm trying to measure this method's performance in my tests but I can't figure out how to handle asynchronous operations without callbacks with measure block.
I found out there are startMeasuring() and stopMeasuring() methods that allow to manually set entry and end points for the test. I tried to call the latter in my output mock:
let output = InteractorOutputMock()
output.onDisplay = { _ in
self.stopMeasuring()
}
let interactor = PremiumLocationsChooserInteractor()
interactor.output = output
measureMetrics([.wallClockTime], automaticallyStartMeasuring: false) {
self.startMeasuring()
interactor.loadCountries()
}
But the code still finishes in 0 secs. How should I approach this issue?
UPDATE:
I tried using expectations as well, but ran into problem. I can neither call it inside the measure block nor outside of it. The former looks like the following and causes the code to wait in the first iteration of measure:
let outputCalledExpectation = XCTestExpectation(description: "Output hasn't been called")
outputCalledExpectation.expectedFulfillmentCount = 10 // need to fullfill it 10 times since this is how much measure block is called
let output = InteractorOutputMock()
output.onDisplay = { _ in
self.stopMeasuring() // called only once
outputCalledExpectation.fulfill()
}
let interactor = PremiumLocationsChooserInteractor()
interactor.output = output
measureMetrics([.wallClockTime], automaticallyStartMeasuring: false) {
startMeasuring()
interactor.loadCountries()
wait(for: [outputCalledExpectation], timeout: 5.0) // then stack here
}
And if I try to move the wait method outside of the block, I get exception -stopMeasuring is only supported from a block passed to -measure...Block:
measureMetrics([.wallClockTime], automaticallyStartMeasuring: false) {
startMeasuring()
interactor.loadCountries()
}
wait(for: [outputCalledExpectation], timeout: 5.0) // get exception here
I use this extension for measuring, works like a charm for me.
extension XCTestCase {
func measureAsync(
timeout: TimeInterval = 2.0,
for block: #escaping () async throws -> Void,
file: StaticString = #file,
line: UInt = #line
) {
measureMetrics(
[.wallClockTime],
automaticallyStartMeasuring: true
) {
let expectation = expectation(description: "finished")
Task { #MainActor in
do {
try await block()
expectation.fulfill()
} catch {
XCTFail(error.localizedDescription, file: file, line: line)
expectation.fulfill()
}
}
wait(for: [expectation], timeout: timeout)
}
}
}

XCTest: Check That a Callback is Not Called

I know how to wait for a callback with expectations in XCT. However, what about testing for the inverse?
Below is an example of a test that I have as part of my tests:
manager.state = .initialized
let exp = expectation(description: "expectation")
manager.login { state, error in
exp.fulfill()
XCTAssert(state == .initialized)
XCTAssertNil(error)
}
waitForExpectations(timeout: 1)
In the next test I'm writing I would like a test to ensure that a callback is not called when my manager object's state is a specific value.
This can be done by setting isInverted = true on the XCTestExpectation object like so:
manager.state = .initialized
let exp = expectation(description: "inverted expectation")
exp.isInverted = true
manager.login { state, error in
exp.fulfill()
}
waitForExpectations(timeout: 1)

Updating UI after retrieving device settings

I want to do something simple in Swift. I have to retrieve some setting from a device and then initialize some UI controls with those settings. It may take a few seconds to complete the retrieval so I don't want the code to continue until after the retrieval (async).
I have read countless posts on many websites including this one and read many tutorials. None seem to work for me.
Also, in the interest of encapsulation, I want to keep the details within the device object.
When I run the app I see the print from the initializing method before I see the print from the method.
// Initializing method
brightnessLevel = 100
device.WhatIsTheBrightnessLevel(level: &brightnessLevel)
print("The brightness level is \(brightnessLevel)")
// method with the data retrieval code
func WhatIsTheBrightnessLevel(level brightness: inout Int) -> CResults
{
var brightness: Int
var characteristic: HMCharacteristic
var name: String
var results: CResults
var timeout: DispatchTime
var timeoutResult: DispatchTimeoutResult
// Refresh the value by querying the lightbulb
name = m_lightBulbName
characteristic = m_brightnessCharacteristic!
brightness = 100
timeout = DispatchTime.now() + .seconds(CLightBulb.READ_VALUE_TIMEOUT)
timeoutResult = .success
results = CResults()
results.SetResult(code: CResults.code.success)
let dispatchGroup = DispatchGroup()
DispatchQueue.global(qos: .userInteractive).async
{
//let dispatchGroup = DispatchGroup()
dispatchGroup.enter()
characteristic.readValue(completionHandler:
{ (error) in
if error != nil
{
results.SetResult(code: CResults.code.homeKitError)
results.SetHomeKitDescription(text: error!.localizedDescription)
print("Error in reading the brightness level for \(name): \(error!.localizedDescription)")
}
else
{
brightness = characteristic.value as! Int
print("CLightBulb: -->Read the brightness level. It is \(brightness) at " + Date().description(with: Locale.current))
}
dispatchGroup.leave()
})
timeoutResult = dispatchGroup.wait(timeout: timeout)
if (timeoutResult == .timedOut)
{
results.SetResult(code: CResults.code.timedOut)
}
else
{
print("CLightBulb: (After wait) The brightness level is \(brightness) at " + Date().description(with: Locale.current))
self.m_brightnessLevel = brightness
}
}
return(results)
}
Thank you!
If you're going to wrap an async function with your own function, it's generally best to give your wrapper function a completion handler as well. Notice the call to your completion handler. This is where you'd pass the resulting values (i.e. within the closure):
func getBrightness(characteristic: HMCharacteristic, completion: #escaping (Int?, Error?) -> Void) {
characteristic.readValue { (error) in
//Program flows here second
if error == nil {
completion(characteristic.value as? Int, nil)
} else {
completion(nil, error)
}
}
//Program flows here first
}
Then when you call your function, you just need to make sure that you're handling the results within the completion handler (i.e. closure):
getBrightness(characteristic: characteristic) { (value, error) in
//Program flows here second
if error == nil {
if let value = value {
print(value)
}
} else {
print("an error occurred: \(error.debugDescription)")
}
}
//Program flows here first
Always keep in mind that code will flow through before the async function completes. So you have to structure your code so that anything that's depending on the value or error returned, doesn't get executed before completion.

How to trigger failed test on timeout with waitForExpectations()?

I've recreated the example from here: http://www.mokacoding.com/blog/testing-callbacks-in-swift-with-xctest/.
I want to test for a timeout using waitForExpectations(). This should mimic a long running process that has timed out. To do this, I've set a sleep() command in the called function that is longer than the timeout in waitForExpectations().
However, the sleep() doesn't have any effect. The test always passes. I've tried putting sleep() before completion(true) as well but that doesn't change the outcome (i.e., passed test).
Any ideas what I'm doing run to trigger a test failure on timeout?
class SomeService {
func doSomethingAsync(completion: (_ success: Bool) -> ()) {
completion(true)
sleep(5)
}
}
In test class
let service = SomeService()
service.doSomethingAsync { (success) in
XCTAssertTrue(success, "assert is true")
expect.fulfill()
}
waitForExpectations(timeout: 3) { (error) in
if let error = error {
XCTFail("timeout errored: \(error)")
}
}
Your test passes because you are calling completion before sleep, so your expectation is being fulfilled almost immediately - before you wait for 5 seconds; while the completion block is executed asynchronously, it is likely going to finish in under a second.
If you call sleep insidecompletion then your test will fail as expected. However, your test may crash if the test is no longer running when expect.fulfill() is called since expect may no longer exist by the time it is executed, as it may have been cleaned up as soon as the test fails (about 2 seconds before the expectation will be fulfilled).
class SomeService {
func doSomethingAsync(completion: (_ success: Bool) -> ()) {
DispatchQueue.main.async {
completion(true)
}
}
}
Test:
let service = SomeService()
service.doSomethingAsync { (success) in
XCTAssertTrue(success, "assert is true")
sleep(5)
expect.fulfill()
}
waitForExpectations(timeout: 3) { (error) in
if let error = error {
XCTFail("timeout errored: \(error)")
}
}