How to trigger failed test on timeout with waitForExpectations()? - swift

I've recreated the example from here: http://www.mokacoding.com/blog/testing-callbacks-in-swift-with-xctest/.
I want to test for a timeout using waitForExpectations(). This should mimic a long running process that has timed out. To do this, I've set a sleep() command in the called function that is longer than the timeout in waitForExpectations().
However, the sleep() doesn't have any effect. The test always passes. I've tried putting sleep() before completion(true) as well but that doesn't change the outcome (i.e., passed test).
Any ideas what I'm doing run to trigger a test failure on timeout?
class SomeService {
func doSomethingAsync(completion: (_ success: Bool) -> ()) {
completion(true)
sleep(5)
}
}
In test class
let service = SomeService()
service.doSomethingAsync { (success) in
XCTAssertTrue(success, "assert is true")
expect.fulfill()
}
waitForExpectations(timeout: 3) { (error) in
if let error = error {
XCTFail("timeout errored: \(error)")
}
}

Your test passes because you are calling completion before sleep, so your expectation is being fulfilled almost immediately - before you wait for 5 seconds; while the completion block is executed asynchronously, it is likely going to finish in under a second.
If you call sleep insidecompletion then your test will fail as expected. However, your test may crash if the test is no longer running when expect.fulfill() is called since expect may no longer exist by the time it is executed, as it may have been cleaned up as soon as the test fails (about 2 seconds before the expectation will be fulfilled).
class SomeService {
func doSomethingAsync(completion: (_ success: Bool) -> ()) {
DispatchQueue.main.async {
completion(true)
}
}
}
Test:
let service = SomeService()
service.doSomethingAsync { (success) in
XCTAssertTrue(success, "assert is true")
sleep(5)
expect.fulfill()
}
waitForExpectations(timeout: 3) { (error) in
if let error = error {
XCTFail("timeout errored: \(error)")
}
}

Related

Write xctest for function inside an operation queue

I have a function as follows which adds an operation to the operation queue, how do I test the block of code that is being added to the operation queue, I tried using an expectation by passing an analytics observer spy and check if the value is set but it does not seem to work, please point me in the right direction
func firePendingEvent() {
for analyticItem in eventQueue {
eventPriorityQueue.addOperationToQueue (operation: BlockOperation {
self.analyticsObserver?.logEvent(event: analyticItem) // write test to check the event inside this function
}, priority: .normal)
if eventQueue.count > 0 {
eventQueue.remove(at: 0)
}
}
}
The basic idea is that your test has to create an expectation:
let e = expectation(description: "testGetUsers")
And then when your asynchronous method is done, fulfill that expectation:
e.fulfill()
And then, of course, your test has to, after initiating the asynchronous tasks, wait for the expectations:
waitForExpectations(timeout: 10)
But, obviously, you need to know when your function is done. I would suggest giving it an optional completion handler:
func firePendingEvent(completion: (() -> Void)? = nil) {
let group = DispatchGroup()
for analyticItem in eventQueue {
group.enter()
eventPriorityQueue.addOperationToQueue(operation: BlockOperation {
self.analyticsObserver?.logEvent(event: analyticItem)
group.leave()
}, priority: .normal)
}
eventQueue.removeAll()
if let completion = completion {
group.notify(queue: .main, execute: completion)
}
}
(Note, I personally refrain from mutating an array as I iterate through it, so I add all the tasks to the queue and defer the removeAll until I am no longer iterating.)
Anyway, you can now write your test:
func testFirePendingEvents() {
let e = expectation(description: "testGetUsers")
foo.firePendingEvent {
XCAssert(…) // do whatever tests are appropriate
e.fulfill()
}
waitForExpectations(timeout: 10)
}
But, because that completion handler is optional, and defaults to nil, you don't have to change the rest of your code that isn't availing itself of the new completion handler.
Personally, I would add a parameter to that completion handler closure to pass back whatever value you want to test, but there wasn’t enough in the question to know what result you are testing.

How to measure performance of asynchronous method?

I've written a method loadCountries() that performs asynchronous file loading upon calling and then notifies its output that the work is done. I'm trying to measure this method's performance in my tests but I can't figure out how to handle asynchronous operations without callbacks with measure block.
I found out there are startMeasuring() and stopMeasuring() methods that allow to manually set entry and end points for the test. I tried to call the latter in my output mock:
let output = InteractorOutputMock()
output.onDisplay = { _ in
self.stopMeasuring()
}
let interactor = PremiumLocationsChooserInteractor()
interactor.output = output
measureMetrics([.wallClockTime], automaticallyStartMeasuring: false) {
self.startMeasuring()
interactor.loadCountries()
}
But the code still finishes in 0 secs. How should I approach this issue?
UPDATE:
I tried using expectations as well, but ran into problem. I can neither call it inside the measure block nor outside of it. The former looks like the following and causes the code to wait in the first iteration of measure:
let outputCalledExpectation = XCTestExpectation(description: "Output hasn't been called")
outputCalledExpectation.expectedFulfillmentCount = 10 // need to fullfill it 10 times since this is how much measure block is called
let output = InteractorOutputMock()
output.onDisplay = { _ in
self.stopMeasuring() // called only once
outputCalledExpectation.fulfill()
}
let interactor = PremiumLocationsChooserInteractor()
interactor.output = output
measureMetrics([.wallClockTime], automaticallyStartMeasuring: false) {
startMeasuring()
interactor.loadCountries()
wait(for: [outputCalledExpectation], timeout: 5.0) // then stack here
}
And if I try to move the wait method outside of the block, I get exception -stopMeasuring is only supported from a block passed to -measure...Block:
measureMetrics([.wallClockTime], automaticallyStartMeasuring: false) {
startMeasuring()
interactor.loadCountries()
}
wait(for: [outputCalledExpectation], timeout: 5.0) // get exception here
I use this extension for measuring, works like a charm for me.
extension XCTestCase {
func measureAsync(
timeout: TimeInterval = 2.0,
for block: #escaping () async throws -> Void,
file: StaticString = #file,
line: UInt = #line
) {
measureMetrics(
[.wallClockTime],
automaticallyStartMeasuring: true
) {
let expectation = expectation(description: "finished")
Task { #MainActor in
do {
try await block()
expectation.fulfill()
} catch {
XCTFail(error.localizedDescription, file: file, line: line)
expectation.fulfill()
}
}
wait(for: [expectation], timeout: timeout)
}
}
}

When to call my completion block if I have multiple asynchronous calls

I'm a bit confused on how to use closures or in my case a completion block effectively. In my case, I want to call a block of code when some set of asynchronous calls have completed, to let my caller know if there was an error or success, etc.
So an example of what I'm trying to accomplish might look like the following:
// Caller
updatePost(forUser: user) { (error, user) in
if let error = error {
print(error.description)
}
if let user = user {
print("User was successfully updated")
// Do something with the user...
}
}
public func updatePost(forUser user: User, completion: #escaping (Error?, User?) -> () {
// Not sure at what point, and where to call completion(error, user)
// so that my caller knows the user has finished updating
// Maybe this updates the user in the database
someAsyncCallA { (error)
}
// Maybe this deletes some old records in the database
someAsyncCallB { (error)
}
}
So ideally, I want my completion block to be called when async block B finishes (assuming async block A finished already, I know this is a BAD assumption). But what happens in the case that async block B finishes first and async block A takes a lot longer? If I call my completion after async block B, then my caller thinks that the method has finished.
In a case like this, say I want to tell the user when updating has finished, but I only really know it has finished when both async blocks have finished. How do I tackle this or am I just using closures wrong?
I don't know if your question has been answered. What I think you are looking for is a DispatchGroup.
let dispatchGroup = DispatchGroup()
dispatchGroup.enter()
someAsyncCallA(completion: {
dispatchGroup.leave()
})
dispatchGroup.enter()
someAsyncCallB(completion: {
dispatchGroup.leave()
})
dispatchGroup.notify(queue: .main, execute: {
// When you get here, both calls are done and you can do what you want.
})
Really important note: enter() and leave() calls must balance, otherwise you crash with an exception.
try below code:
public func updatePost(forUser user: User, completion: #escaping (Error?, User?) -> () {
// Not sure at what point, and where to call completion(error, user)
// so that my caller knows the user has finished updating
// Maybe this updates the user in the database
someAsyncCallA { (error)
// Maybe this deletes some old records in the database
someAsyncCallB { (error)
completion()
}
}
}
please try updated answer below:
public func updatePost(forUser user: User, completion: #escaping (Error?, User?) -> () {
var isTaskAFinished = false
var isTaskBFinished = false
// Not sure at what point, and where to call completion(error, user)
// so that my caller knows the user has finished updating
// Maybe this updates the user in the database
someAsyncCallA { (error)
// Maybe this deletes some old records in the database
isTaskAFinished = true
if isTaskBFinished{
completion()
}
}
someAsyncCallB { (error)
isTaskBFinished = true
if isTaskAFinished{
completion()
}
}
}

What if XCTestExpectation is unexpected

I'm writing an XCTest unit test in Swift.
The idea is that a callback mustn't be called in a certain case.
So what I do, is
func testThatCallbackIsNotFired() {
let expectation = expectationWithDescription("A callback is fired")
// configure an async operation
asyncOperation.run() { (_) -> () in
expectation.fulfill() // if this happens, the test must fail
}
waitForExpectationsWithTimeout(1) { (error: NSError?) -> Void in
// here I expect error to be not nil,
// which would signalize that expectation is not fulfilled,
// which is what I expect, because callback mustn't be called
XCTAssert(error != nil, "A callback mustn't be fired")
}
}
When the callback is called, everything works fine: it fails with a message "A callback mustn't be fired" which is exactly what I need.
But if expectation hasn't been fulfilled, it fails and says
Asynchronous wait failed: Exceeded timeout of 1 seconds, with unfulfilled expectations: "Callback is fired".
Since a not fulfilled expectation is what I need, I don't want to have a failed test.
Do you have any suggestions what can I do to avoid this? Or, maybe, I can reach my goal in a different way? Thanks.
Use isInverted like in this post https://www.swiftbysundell.com/posts/unit-testing-asynchronous-swift-code
class DebouncerTests: XCTestCase {
func testPreviousClosureCancelled() {
let debouncer = Debouncer(delay: 0.25)
// Expectation for the closure we'e expecting to be cancelled
let cancelExpectation = expectation(description: "Cancel")
cancelExpectation.isInverted = true
// Expectation for the closure we're expecting to be completed
let completedExpectation = expectation(description: "Completed")
debouncer.schedule {
cancelExpectation.fulfill()
}
// When we schedule a new closure, the previous one should be cancelled
debouncer.schedule {
completedExpectation.fulfill()
}
// We add an extra 0.05 seconds to reduce the risk for flakiness
waitForExpectations(timeout: 0.3, handler: nil)
}
}
I had this same problem, and I am annoyed that you can't use a handler to override the timeout fail of waitForExpectationsWithTimeout. Here is how I solved it (Swift 2 syntax):
func testThatCallbackIsNotFired() {
expectationForPredicate(NSPredicate{(_, _) in
struct Holder {static let startTime = CACurrentMediaTime()}
if checkSomehowThatCallbackFired() {
XCTFail("Callback fired when it shouldn't have.")
return true
}
return Holder.startTime.distanceTo(CACurrentMediaTime()) > 1.0 // or however long you want to wait
}, evaluatedWithObject: self, handler: nil)
waitForExpectationsWithTimeout(2.0 /*longer than wait time above*/, handler: nil)
}

Waiting to return main block until its encapsulated children are finished

I'm quite new to asynchronous networking in Swift, and am currently using the SwiftHTTP library to run HTTP GET requests.
I've found myself in a situation whereby I run a for loop within an asynchronous GET block, which itself contains a 'secondary'/child GET block, as follows:
request.GET("https://", parameters: nil, success: { (response: HTTPResponse) in
if response.responseObject != nil {
var returnArray = [returnObject]() // ***
// responseObject is an array of 'results', thus loop through and perform a secondary GET for each 'result'
for result in response.responseObject {
secondaryRequest.GET(result["url"], parameters: nil, success: { (secondaryResponse: HTTPResponse) in
if secondaryResponse.responseObject != nil {
// *** add secondary repsonseObject to main 'returnArray', created at the top of the first block
returnArray.append(secondaryResponse.responseObject!)
}
},failure: {(error: NSError, secondaryResponse: HTTPResponse?) in
println("got an error: \(error)")
})
}
completion(returnArray) // HOW TO ONLY RETURN WHEN ALL SECONDARY LOOPS HAVE FINISHED ADDING TO THE RETURN ARRAY???
}
}
}, failure: {(error: NSError, reponse: HTTPResponse?) in
println("got an error: \(error)")
})
Hopefully it is clear enough that each secondary block adds to a returnArray that the main block returns as its completion handler. However, I wish for the main block to execute this completion (and the corresponding array) only when all secondary blocks have finished returning/adding to the returnArray.
I don't really know where to start without performing my own hacky options (like, waiting with an NSTimer —I can hear the sighs at that one! :') , or adding each secondary iteration to an external function that counts each addition until it gets to predefined loop threshold).
Many thanks for any and all guidance!
To trigger a block of code upon the completion of a series of asynchronous tasks, we generally use "dispatch groups." So create a dispatch group, "enter" the group before every request, "leave"'in every completion block, and then create a dispatch "notify" block that will be called when all of the "enter" calls are balanced by a "leave" call:
request.GET("https://", parameters: nil, success: { response in
if response.responseObject != nil {
let group = dispatch_group_create()
var returnArray = [ReturnObject]()
for result in response.responseObject {
dispatch_group_enter(group)
secondaryRequest.GET(result["url"], parameters: nil, success: { secondaryResponse in
dispatch_async(dispatch_get_main_queue()) {
if secondaryResponse.responseObject != nil {
returnArray.append(secondaryResponse.responseObject!)
}
dispatch_group_leave(group)
}
},failure: { error, secondaryResponse in
println("got an error: \(error)")
dispatch_group_leave(group)
})
}
dispatch_group_notify(group, dispatch_get_main_queue()) {
completion(returnArray)
}
}
}, failure: {error, reponse in
println("got an error: \(error)")
})
Note, since GET completion handler closures happen on background queues, you need to synchronize the handling of the returnArray. You can create a custom queue to coordinate this, or just use the main queue, as I did above.