I'm currently testing a number of classes that do network stuff like REST API calls, and a Realm database is mutated in the process. When I run all the different tests I have at once, race conditions appear (but of course, when I run them one by one, they all pass). How can I reliably make the tests pass?
I have tried to call the mentioned functions in a GCD block like this:
DispatchQueue.main.async {
self.function.start()
}
One of my tests are still failing, so I guess the above didn't work. I have enabled Thread Sanitizer and it reports, from time to time, that race conditions appear.
I can't post code, so I'm looking for conceptual solutions.
Typically some form of dependency injection. Be it an internally exposed var to the DispatchQueue, a default argument in a function with the queue, or a constructor argument. You just need some way to pass a test queue that dispatches the event when you need to.
DispatchQueue.main.async will schedule the block async to the callee on the main queue and therefore isn't guarenteed by the time you make an assertion.
Example (disclaimer: I'm typing from memory so it might not compile but it gives the idea):
// In test code.
struct TestQueue: DispatchQueue {
// make sure to impement other necessary protocol methods
func async(block: () -> Void) {
// you can even have some different behavior for when to execute the block.
// also you can pass XCTestExpectations to this TestQueue to be fulfilled if necessary.
block()
}
}
// In source code. In test, pass the Test Queue to the first argument
func doSomething(queue: DispatchQueue = DispatchQueue.main, completion: () -> Void) {
queue.async(block: completion)
}
Other methods of testing async and eliminating race conditions revolve around craftily fulfilling an XCTestExpectation.
If you have access to the completion block that is eventually invoked:
// In source
class Subject {
func doSomethingAsync(completion: () -> Void) {
...
}
}
// In test
func testDoSomethingAsync() {
let subject = Subject()
let expect = expectation(description: "does something asnyc")
subject.doSomethingAsync {
expect.fulfill()
}
wait(for: [expect], timeout: 1.0)
// assert something here
// or the wait may be good enough as it will fail if not fulfilled
}
If you don't have access to the completion block it usually means finding a way to inject or subclass a test double that you can set an XCTestExpectation on and will eventually fulfill the expectation when the async work has completed.
Related
I don't understand what's the difference between add(_) and add(_) async method. Like the below code, the MyActor has two add methods and one of them uses async keyword. They can exist at the same time. If I comment out the second add method it will output AAAA. If both add methods exist at the same time, output "BBBBB"。
actor MyActor {
var num: Int = 0
func add(_ value: Int) {
print("AAAAA")
num += value
}
func add(_ value: Int) async {
print("BBBBB")
num += value
}
}
let actor = MyActor()
Task {
await actor.add(200)
print(await actor.num)
}
Supplementary:
With the second add method commented out, I defined let actor = MyActor() outside Task and I noticed the add method signed as add(_:). If move let actor = MyActor() inside Task the add method signed as add(_:) async
The difference emerges inside the actor, for example
actor MyActor {
func addSync(_ value: Int) {
}
func addAsync(_ value: Int) async {
}
func f() {
addSync(0)
}
func g() async {
addSync(0)
await addAsync(0)
}
}
In the actor method f and g you can call addSync synchronously. While outside the actor, you need always call an actor method with the await keyword as if the method is asynchronous:
func outside() async {
let actor = MyActor()
await actor.addSync(0)
}
Async in Swift allows for structured concurrency, which will improve the readability of complex asynchronous code. Completion closures are no longer needed, and calling into multiple asynchronous methods after each other is a lot more readable.
Async stands for asynchronous and can be seen as a method attribute making it clear that a method performs asynchronous work. An example of such a method looks as follows:
func fetchImages() async throws -> [UIImage] {
// .. perform data request
}
The fetchImages method is defined as async throwing, which means that it’s performing a failable asynchronous job. The method would return a collection of images if everything went well or throws an error if something went wrong.
How async replaces closure completion callbacks
Async methods replace the often seen closure completion callbacks. Completion callbacks were common in Swift to return from an asynchronous task, often combined with a Result type parameter. The above method would have been written as followed:
func fetchImages(completion: (Result<[UIImage], Error>) -> Void) {
// .. perform data request
}
Defining a method using a completion closure is still possible in Swift today, but it has a few downsides that are solved by using async instead:
You have to make sure yourself to call the completion closure in each possible method exit. Not doing so will possibly result in an app waiting for a result endlessly.
Closures are harder to read. It’s not as easy to reason about the order of execution as compared to how easy it is with structured concurrency.
Retain cycles need to be avoided using weak references.
Implementors need to switch over the result to get the outcome. It’s not possible to use try catch statements from the implementation level.
These downsides are based on the closure version using the relatively new Result enum. It’s likely that a lot of projects still make use of completion callbacks without this enumeration:
func fetchImages(completion: ([UIImage]?, Error?) -> Void) {
// .. perform data request
}
Defining a method like this makes it even harder to reason about the outcome on the caller’s side. Both value and error are optional, which requires us to perform an unwrap in any case. Unwrapping these optionals results in more code clutter which does not help to improve readability.
In the UI class I have a method that accesses UI elements, and hence is supposed to force itself onto a main thread. Here's a minimal example of what I mean:
class SomeUI {
func doWorkOnUI() {
guard Thread.isMainThread else {
DispatchQueue.main.async {
self.doWorkOnUI()
}
return
}
print("Doing the work on UI and running on main thread")
}
}
In the tests, of course there's no problem to test the case when doWorkOnUI() is already running on main thread. I just do this:
func testWhenOnMainThread() {
let testedObject = SomeUI()
let expectation = XCTestExpectation(description: "Completed doWorkOnUI")
DispatchQueue.main.async {
testedObject.doWorkOnUI()
expectation.fulfill()
}
wait(for: [expectation], timeout: 10.0)
// Proceed to some validation
}
That is: force execution onto main thread. Wait for it to complete. Do some checks.
But how to test the opposite case, i.e. ensure that function forced itself to run on main thread when called from the background thread?
For example if I do something like:
...
DispatchQueue.global(qos: .background).async {
testedObject.doWorkOnUI()
expectation.fulfill()
}
...
I just tested that function got executed from the background thread. But I didn't explicitly check that it ran on main thread. Of course, since this function accesses UI elements, the expectation is that it crashes if not forced on main thread. So is "no crash" the only testable condition here? Is there anything better?
When there is an outer closure in the background and an inner closure on the main thread, we want two tests:
Call the outer closure. Do a wait for expectations. Wait for 0.01 seconds. Check that the expected work was performed.
Call the outer closure. This time, don't wait for expectations. Check that the work was not performed.
To use this pattern, I think you'll have to change your code so that the tests can call the outer closure directly without having to do an async dance already. This suggests that your design is too deep to be testable without some changes.
Find a way for an intermediate object to capture the closure. That is, instead of directly calling DispatchQueue.global(qos: .background).async, make a type that represents this action. Then a Test Spy version can capture the closure instead of dispatching it to the background, so that your tests can invoke it directly. Then you can test the call back to main thread using async wait.
(NOTE - I'm developing for macOS, so please ... iOS-specific advice won't help me)
What I'm trying to do:
I have an app component that performs a short task on a background thread, and then, if certain conditions are met, asynchronously sends out a notification on the main thread.
NOTE - I am not using NSNotification in my app code. I am using my own custom notification mechanism. So, any solution related to NSNotification is not applicable to me.
I'm writing a unit test for the above mentioned app component, and simply want to check if that notification was indeed sent or not. My test has to be able to wait a second or so to give the notification time to reach its subscriber/observer, before performing an assertion.
I want to be able to test both possible cases in my tests: Both are normal scenarios.
Notification was sent.
Notification was not sent.
After hours of reading several docs and code samples, I don't understand how to achieve this with expectations.
I just want to wait one second in my test. Is it really this complicated ?
sleep() doesn't work
DispatchQueue.main.asyncAfter(time) doesn't work
Timer doesn't work
Here's the app component that needs to be tested, and its unit test:
In the below code, where do I put expectation.fulfill() ???
class ComponentBeingTested {
func methodBeingTested() {
doSomeWork()
if certainConditionsAreMet {
DispatchQueue.main.async {sendOutNotification()}
}
}
}
...
class UnitTestForComponentBeingTested: XCTestCase {
let objectBeingTested = ComponentBeingTested()
func testMethodBeingTested() {
let expectation = self.expectation(description: "Notification was sent")
// Call the code being tested
objectBeingTested.methodBeingTested()
// How do I do this with expectations ??? Where does expectation.fulfill() go ?
waitForOneSecond()
XCTAssertTrue(notificationSent) // Assume the value of notificationSent is available
}
}
Here is an approach
func testMethodBeingTested() {
// create expectation
let expectation = self.expectation(description: "Notification was sent")
// set expectation condition
var notificationSent = false
let observer = NotificationCenter.default
.addObserver(forName: _Your_Notification_Name, object: nil, queue: nil) { _ in
notificationSent = true
expectation.fulfill()
}
// Call the code being tested
objectBeingTested.methodBeingTested()
// wait for expectation
self.wait(for: [expectation], timeout: 5)
XCTAssertTrue(notificationSent)
}
Check out XCTNSNotificationExpectation, which becomes fulfilled when a matching notification is posted. Different initializers are available, depending on how restrictive you want to be on the fulfilment of the expectation.
To check that the notification is not sent, set isInverted to true on the expectation object.
Then just add a call to waitForExpectations(timeout:handler:) at the end of your test.
Ok, after a lot of trial and error, this works great for me:
Description: I basically created a helper function in my test case class that contains all the boilerplate expectation/wait code. It does the following:
1 - Creates an expectation (i.e. XCTestExpectation) as a formality.
2 - Calls my (arbitrary) test case assertion code (passed in as a closure) on some global queue thread after the intended delay period. Once this assertion code has completed, the expectation is fulfilled (again, a formality).
3 - Waits on the expectation by calling XCTestCase.wait(timeout). This ensures that the main thread / run loop is kept alive while my assertion code completes on this other thread.
Then, in my test case, I simply invoke that helper function, providing it with a wait period and some code to execute (i.e. my assertions).
This way, I have a simple and expressive reusable function that hides all the excessive ugliness of expectations which I never thought necessary in the first place.
I can put this helper in a base class like MyAppTestCase: XCTestCase, so that it is available to all my test case classes.
NOTE - This solution can be enhanced and made even more generic/reusable, but as of now, this is quite sufficient for the purposes of the originally posted problem.
Solution:
class ComponentBeingTested {
func methodBeingTested() {
doSomeWork()
if certainConditionsAreMet {
DispatchQueue.main.async {sendOutNotification()}
}
}
}
...
class UnitTestForComponentBeingTested: XCTestCase {
let objectBeingTested = ComponentBeingTested()
// Helper function that uses expectation/wait to execute arbitrary
// test code (passed in as a closure) after some delay period.
func executeAfter(_ timeSeconds: Double, _ work: (#escaping () -> Void)) {
let theExpectation = expectation(description: "some expectation")
// Execute work() after timeSeconds seconds
DispatchQueue.global(qos: .userInteractive).asyncAfter(deadline: .now() + timeSeconds) {
// The call to work() will execute my test assertions
work()
// As a formality, fulfill the expectation
theExpectation.fulfill()
}
// Wait for (timeSeconds + 1) seconds to give the work() call
// some time to perform the assertions
wait(for: [theExpectation], timeout: timeSeconds + 1)
}
func testMethodBeingTested() {
// Call the code being tested
objectBeingTested.methodBeingTested()
// Call the helper function above, to do the waiting before executing
// the assertions
executeAfter(0.5) {
// Assume the value of notificationSent is computed elsewhere
// and is available to assert at this point
XCTAssertTrue(notificationSent)
}
}
}
I understand that you can not register a Realm .observe block on an object or collection if the Realm is in a write transaction.
This is easier to manage if everything is happening on the main thread however I run into this exception often because I prefer to hand my JSON parsing off to a background thread. This works great because I don't have to bog down the main thread and with Realm's beautiful notification system I can get notified of all modifications if I have already registered to listen for those changes.
Right now, if I am about to add an observation block I check to make sure my Realm is not in a write transaction like this:
guard let realm = try? Realm(), !realm.isInWriteTransaction else {
return
}
self.myToken = myRealmObject.observe({ [weak self] (change) in
//Do what ever
}
This successfully guards against this exception. However I never get a chance to re - register this token unless I get a little creative.
Does the Realm team have any code examples/ suggestions on a better pattern to avoid this exception? Any tricks I'm missing to successfully register the token?
In addition to the standard function, I do use an extension for Results to avoid this in general. This issue popped up, when our data load grew bigger and bigger.
While we do now rewrite our observe functions logic, this extension is an interims solution to avoid the crashes at a first place.
Idea is simple: when currently in a write transaction, try it again.
import Foundation
import RealmSwift
extension Results {
public func safeObserve(on queue: DispatchQueue? = nil,
_ block: #escaping (RealmSwift.RealmCollectionChange<RealmSwift.Results<Element>>) -> Void)
-> RealmSwift.NotificationToken {
// If in Write transaction, call it again
if self.realm?.isInWriteTransaction ?? false {
DispatchQueue.global().sync {
Thread.sleep(forTimeInterval: 0.1) // Better to have some delay than a crash, hm?
}
return safeObserve(on: queue, block)
}
// Aight, we can proceed to call Realms Observe function
else {
return self.observe(on: queue, block)
}
}
}
Then call it like
realmResult.safeObserve({ [weak self] (_: RealmCollectionChange<Results<AbaPOI>>) in
// Do anything
})
I’m executing some functions in a test asynchronously using a DispatchQueue like this:
let queue: DispatchQueue = DispatchQueue.global(qos: DispatchQoS.QoSClass.userInitiated)
let group: DispatchGroup = DispatchGroup()
func execute(argument: someArg) throws {
group.enter()
queue.async {
do {
// Do stuff here
group.leave()
} catch {
Log.info(“Something went wrong")
}
}
group.wait()
}
Sometimes the code inside the do block can throw errors, that I have to catch later on. Since I’m developing a test, I want it to fail, if the code inside the do block throws an error.
Is there a way to throw an error, without catching it inside the queue.async call?
You cannot throw an error, but you can return an error:
First, you need to make your calling function asynchronous as well:
func execute(argument: someArg, completion: #escaping (Value?, Error?)->()) {
queue.async {
do {
// compute value here:
...
completion(value, nil)
} catch {
completion(nil, error)
}
}
}
The completion handler takes a parameter which we could say is a "Result" containing either the value or an error. Here, what we have is a tuple (Value?, Error?), where Value is the type which is calculated by the task. But instead, you could leverage a more handy Swift Enum for this, e.g. Result<T> or Try<T> (you might want to the search the web).
Then, you use it as follows:
execute(argument: "Some string") { value, error in
guard error == nil else {
// handle error case
}
guard let value = value else {
fatalError("value is nil") // should never happen!
}
// do something with the value
...
}
Some rules that might help:
If a function calls an asynchronous function internally, it inevitable becomes an asynchronous function as well. *)
An asynchronous function should have a completion handler (otherwise, it's some sort of "fire and forget").
The completion handler must be called, no matter what.
The completion handler must be called asynchronously (with respect the the caller)
The completion handler should be called on a private execution context (aka dispatch queue) unless the function has a parameter specifying where to execute the completion handler. Never use the main thread or main dispatch queue - unless you explicitly state that fact in the docs or you intentionally want to risk dead-locks.
*) You can force it to make it synchronous using semaphores which block the calling thread. But this is inefficient and really rarely needed.
Well, you might conclude, that this looks somewhat cumbersome. Fortunately, there's help - you might look for Future or Promise which can nicely wrap this and make the code more concise and more comprehensible.
Note: In Unit Test, you would use expectations to handle asynchronous calls (see XCTest framework).
Refactor your code to use queue.sync and then throw your error from there. (Since your execute function is actually synchronous, given the group.wait() call at the last line, it shouldn't really matter.)
For instance, use this method from DispatchQueue:
func sync<T>(execute work: () throws -> T) rethrows -> T
By the way, a good idiom for leaving a DispatchGroup is:
defer { group.leave() }
as the first line of your sync/async block, which guarantees you won't accidentally deadlock when an error happens.