Is the Diffable DataSource queue created on whatever thread it we created the datasource on? - swift5

I'm trying to understand diffabledata source threading in more detail.
In my code I create a diffable data source on the main thread.
This data source uses a backing storing.
The data for the backing store comes from the following getter method.
var getData: [MyDataType] {
get {
//Sometimes this prints MainThread and sometimes it prints com.apple.uikit.datasource.diffing
LOG.DLog("Returning Data On \(currentQueueName())")
if Thread.isMainThread {
return _myDataValues
} else {
//This else statement never enters
return DispatchQueue.main.sync {
LOG.DLog("Returning Data On Main")
return _myDataValues
}
}
}
}
func currentQueueName() -> String {
let name = __dispatch_queue_get_label(nil)
guard let str = String(cString: name, encoding: .utf8) else { return ""}
return str
}
When my get method executes, sometimes the queue name is com.apple.uikit.datasource.diffing and sometimes the queue name is Main.
In the case where the queue name is com.apple.uikit.datasource.diffing I would expect my else condition to enter but it never does.
So my question is, why does the else condition in my code example ever enter?
My best guess is that the diffable serial queue is created on the same thread that the diffable datasource was created on.

Related

What happens to a pointer referenced by a closure that is effectively let go?

The following is pseudo-code to help demonstrate my question:
var dataSourceArray = [CustomClassObject]()
databaseListener { (data) in
dataSourceArray.removeAll()
let id = data.id
let imageURL = data.imageURL
let listing = CustomClassObject(id: id, image: nil)
remoteFileStorage.getImage(url: imageURL) { (image) in
listing.image = image
}
dataSourceArray.append(listing)
tableView.reloadData()
}
databaseListener is a realtime database listener that can return updates rapidly. In its completion handler is an async call that downloads an image. If databaseListener returns new data before remoteFileStorage has a chance to return from the last run, what happens to the listing pointer in the remoteFileStorage closure? The listing pointer no longer has a home in dataSourceArray (it was just purged) so can the remoteFileStorage handler safely access it anyway? My assumption is that the listing pointer is still pointing to a valid address in memory since the remoteFileStorage closure has a reference to it, but I am not certain?

Multi-threaded core data sometimes returns nil properties

I am new to core data. I have an app that uses core data as local store. Writing to and reading from core data is done by background threads. While this works generally, in rare cases are the fetched data wrong, i.e. properties of a fetched entity are nil.
To check the situation, I wrote a unit test that starts 2 async threads: One fetches continuously from core data, and the other one overwrites continuously these data by first deleting all data, and then storing new data.
This test pretty quickly provokes the error, but I have no idea why. Of course I guess this is a multi-threading problem, but I don’t see why, because fetches and deletion+writes are done in separate managed contexts of a single persistentContainer.
I am sorry that the code below is pretty long, although shortened, but I think without it one cannot identify the problem.
Any help is highly welcome!
Here is my function to fetch data:
func fetchShoppingItems(completion: #escaping (Set<ShoppingItem>?, Error?) -> Void) {
persistentContainer.performBackgroundTask { (managedContext) in
let fetchRequest: NSFetchRequest<CDShoppingItem> = CDShoppingItem.fetchRequest()
do {
let cdShoppingItems: [CDShoppingItem] = try managedContext.fetch(fetchRequest)
for nextCdShoppingItem in cdShoppingItems {
nextCdShoppingItem.managedObjectContext!.performAndWait {
let nextname = nextCdShoppingItem.name! // Here, sometimes name is nil
} // performAndWait
} // for all cdShoppingItems
completion(nil, nil)
return
} catch let error as NSError {
// error handling
completion(nil, error)
return
} // fetch error
} // performBackgroundTask
} // fetchShoppingItems
I have commented the line that sometimes crashes the test, since name is nil.
Here are my functions to store data:
func overwriteCD(shoppingItems: Set<ShoppingItem>,completion: #escaping () -> Void) {
persistentContainer.performBackgroundTask { (managedContext) in
self.deleteAllCDRecords(managedContext: managedContext, in: "CDShoppingItem")
let cdShoppingItemEntity = NSEntityDescription.entity(forEntityName: "CDShoppingItem",in: managedContext)!
for nextShoppingItem in shoppingItems {
let nextCdShoppingItem = CDShoppingItem(entity: cdShoppingItemEntity,insertInto: managedContext)
nextCdShoppingItem.name = nextShoppingItem.name
} // for all shopping items
self.saveManagedContext(managedContext: managedContext)
completion()
} // performBackgroundTask
} // overwriteCD
func deleteAllCDRecords(managedContext: NSManagedObjectContext, in entity: String) {
let deleteFetch = NSFetchRequest<NSFetchRequestResult>(entityName: entity)
let deleteRequest = NSBatchDeleteRequest(fetchRequest: deleteFetch)
deleteRequest.resultType = .resultTypeObjectIDs
do {
let result = try managedContext.execute(deleteRequest) as? NSBatchDeleteResult
let objectIDArray = result?.result as? [NSManagedObjectID]
let changes = [NSDeletedObjectsKey: objectIDArray]
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: changes as [AnyHashable: Any], into: [managedContext])
} catch let error as NSError {
// error handling
}
} // deleteAllCDRecords
func saveManagedContext(managedContext: NSManagedObjectContext) {
if !managedContext.hasChanges { return }
do {
try managedContext.save()
} catch let error as NSError {
// error handling
}
} // saveManagedContext
Are you sure that name isn't nil for all requested entities? Just use guard-let to avoid ! for optional variables. Also ! it isn't safe way to unwrap optional variable especially if you can't be sure for source of data.
The problem with my code was apparently a race condition:
While the „fetch“ thread fetched the core data records, and tried to assign the attributes to the properties, the „store“ thread deleted the records.
This apparently released the attribute objects, so that nil was stored as property.
I thought that the persistentContainer would automatically prevent this, but it does not.
The solution is to execute both background threads of the persistentContainer in a concurrent serial queue, the „fetch“ thread synchronously, and the „store“ thread asynchronously with a barrier.
So, concurrent fetches can be executed, while a store waits until all current fetches are finished.
The concurrent serial queue is defined as
let localStoreQueue = DispatchQueue(label: "com.xxx.yyy.LocalStore.localStoreQueue",
attributes: .concurrent)
EDIT:
In the following fetch and store functions, I moved the core data function persistentContainer.performBackgroundTask inside the localStoreQueue. If it were outside as in my original answer, the store code in localStoreQueue.async(flags: .barrier) would setup a new thread and thus use managedContext in another thread that it was created in, which is a core data multi-threading error.
The „fetch“ thread is modified as
localStoreQueue.sync {
self.persistentContainer.performBackgroundTask { (managedContext) in
let fetchRequest: NSFetchRequest<CDShoppingItem> = CDShoppingItem.fetchRequest()
//…
} // performBackgroundTask
} // localStoreQueue.sync
and the „store“ thread as
localStoreQueue.async(flags: .barrier) {
self.persistentContainer.performBackgroundTask { (managedContext) in
self.deleteAllCDRecords(managedContext: managedContext, in: "CDShoppingItem")
//…
} // performBackgroundTask
} // localStoreQueue.async

object content different from within method

I am encountering a strange behavior with an App I am building.
I have a struct Record and I create an instance of it from a NavigationViewController's child ViewController.
We can see it like this:
NavigationController (called TaskTabsController)
|
ViewController (called TaskFromController)
If I put a breakpoint, I can inspect and check the content of the object is what it should (conform to interface input data).
But when I call a method on that instance (the very next line), the different members have different or missing values.
The object instance is created from within TaskTabsController with something like so:
if let formVC = (viewControllers?[1] as? TaskFormViewController) {
let rec: TaskRecord? = formVC.makeTaskRecord()
// here the rec data are correct when I inspect the instance
rec?.prepareData()
// from the prepareData function, the properties are different of can't be accessed...
}
Ex: From controller, I can see my instance rec having a member task instance (of Task type) with a name property.
But from within the prepareData method, that task member can't display the name attached to it. (debugger says value unreadable or something like that)
Also, I can see a list of other objects, but in the method, their count is different...
Here is how makeTaskRecord method works: (from my TaskFormViewController)
In TaskFormFiewController I have a private property like so:
private var stepsSelected: [StepRecord] = []
That property is updated whit user actions.
The StepRecord is a struct. (so should be passed by value I think)
Next is the makeTaskRecord method in the same controller (gathering details from form elements)
func makeTaskRecord() -> TaskRecord? {
guard let current_task = task
else {
print("TaskForm.makeTaskRecord(): Can't create a record without a Task")
return nil
}
// check data validity
var errors:[String] = []
// generate StepRecords >>>
if stepsSelected.count == 0 {
errors.append("Error, no step selected")
}
// <<<
// DATA OK
if errors.isEmpty {
let taskrecord = TaskRecord(
id: record?.id,
taskId: task!.id,
userCode: Config.getUser()!.code, // global object containing login state
pictures: [],
memo: memo.text,
task: current_task,
stepRecords: stepsSelected // data here is correctly set
)
return taskrecord // but here, I can't retreive the stepRecords! (1)
}
else {
// display errors in an alert box
let errorVC = MyHelpers.makeAlert("Error with data", errors.joined(separator: "\n"))
self.present(errorVC, animated: true)
return nil
}
}
Here is how the prepareData() method looks like: (method of Record class)
func prepareData() -> [String: Any]? {
// no steps in record, data is invalid
guard
let stepRecordsJson = try? JSONEncoder().encode(self.stepRecords)
// This is where I constated that stepRecords is different (set but empty)
else {
return nil
}
// other data
var params = [
"task_id": taskId,
"user_code": userCode,
"step_records": stepRecordsJson, // this is what doesn't work well, but it's certainly related to (1) above
"memo": memo ?? ""
] as [String : Any]
// when editing, we set again the record ID
if let edit_id = self.id {
params["id"] = edit_id
}
return params
}
Note: All the data structures and Codable struct
In the sample above, at the (1), I put a breakpoint on the return line.
When checking data, I see that the "sub-struct" do not have good values:
(lldb) print current_task.name
(String) $R0 = "屋根工事"
(lldb) print taskrecord.task.name
error: Execution was interrupted, reason: EXC_BAD_ACCESS (code=1, address=0x6f).
The process has been returned to the state before expression evaluation.
The task is a struct so there should not be problems about it, but the debugger can't read the task I assigned to my taskrecord.
Is there something I am missing? Like a pointer reference of some sort?

Swift closure async order of execution

In my model have function to fetch data which expects completion handler as parameter:
func fetchMostRecent(completion: (sortedSections: [TableItem]) -> ()) {
self.addressBook.loadContacts({
(contacts: [APContact]?, error: NSError?) in
// 1
if let unwrappedContacts = contacts {
for contact in unwrappedContacts {
// handle constacts
...
self.mostRecent.append(...)
}
}
// 2
completion(sortedSections: self.mostRecent)
})
}
It's calling another function which does asynchronous loading of contacts, to which I'm forwarding my completion
The call of fetchMostRecent with completion looks like this:
model.fetchMostRecent({(sortedSections: [TableItem]) in
dispatch_async(dispatch_get_main_queue()) {
// update some UI
self.state = State.Loaded(sortedSections)
self.tableView.reloadData()
}
})
This sometimes it works, but very often the order of execution is not the way as I would expect. Problem is, that sometimes completion() under // 2 is executed before scope of if under // 1 was finished.
Why is that? How can I ensure that execution of // 2 is started after // 1?
A couple of observations:
It will always execute what's at 1 before 2. The only way you'd get the behavior you describe is if you're doing something else inside that for loop that is, itself, asynchronous. And if that were the case, you'd use a dispatch group to solve that (or refactor the code to handle the asynchronous pattern). But without seeing what's in that for loop, it's hard to comment further. The code in the question, alone, should not manifest the problem you describe. It's got to be something else.
Unrelated, you should note that it's a little dangerous to be updating model objects inside your asynchronously executing for loop (assuming it is running on a background thread). It's much safer to update a local variable, and then pass that back via the completion handler, and let the caller take care of dispatching both the model update and the UI updates to the main queue.
In comments, you mention that in the for loop you're doing something asynchronous, and something that must be completed before the completionHandler is called. So you'd use a dispatch group to do ensure this happens only after all the asynchronous tasks are done.
Note, since you're doing something asynchronous inside the for loop, not only do you need to use a dispatch group to trigger the completion of these asynchronous tasks, but you probably also need to create your own synchronization queue (you shouldn't be mutating an array from multiple threads). So, you might create a queue for this.
Pulling this all together, you end up with something like:
func fetchMostRecent(completionHandler: ([TableItem]?) -> ()) {
addressBook.loadContacts { contacts, error in
var sections = [TableItem]()
let group = dispatch_group_create()
let syncQueue = dispatch_queue_create("com.domain.app.sections", nil)
if let unwrappedContacts = contacts {
for contact in unwrappedContacts {
dispatch_group_enter(group)
self.someAsynchronousMethod {
// handle contacts
dispatch_async(syncQueue) {
let something = ...
sections.append(something)
dispatch_group_leave(group)
}
}
}
dispatch_group_notify(group, dispatch_get_main_queue()) {
self.mostRecent = sections
completionHandler(sections)
}
} else {
completionHandler(nil)
}
}
}
And
model.fetchMostRecent { sortedSections in
guard let sortedSections = sortedSections else {
// handle failure however appropriate for your app
return
}
// update some UI
self.state = State.Loaded(sortedSections)
self.tableView.reloadData()
}
Or, in Swift 3:
func fetchMostRecent(completionHandler: #escaping ([TableItem]?) -> ()) {
addressBook.loadContacts { contacts, error in
var sections = [TableItem]()
let group = DispatchGroup()
let syncQueue = DispatchQueue(label: "com.domain.app.sections")
if let unwrappedContacts = contacts {
for contact in unwrappedContacts {
group.enter()
self.someAsynchronousMethod {
// handle contacts
syncQueue.async {
let something = ...
sections.append(something)
group.leave()
}
}
}
group.notify(queue: .main) {
self.mostRecent = sections
completionHandler(sections)
}
} else {
completionHandler(nil)
}
}
}

Downloaded Data does not print in order using GCDs in Swift

I am trying to have the downloaded return message print before the message "second". Basically, once the message has been downloaded it should print and then the "second" message. Everytime the code runs, the second message prints and then the returnMessage because the return message takes a bit to download. Is it possible to allow the return message to fire after it completes and then the second message everytime the code is run?
var returnMessage: String? = ""
var downloadGroup = dispatch_group_create()
dispatch_async(utility.GlobalUtilityQueue){
dispatch_group_enter(downloadGroup)
service.executeQuery(query, completionHandler: { (ticket: GTLServiceTicket!, object: AnyObject!, error: NSError!) -> Void in
// Process the response
let json = JSON(object.JSON)
returnMessage = json["message"].string
println("\(returnMessage)") // print first
})
dispatch_group_leave(downloadGroup)
dispatch_group_notify(downloadGroup, self.utility.GlobalMainQueue) {
println("second")//should print second
}
}
The problem is that the dispatch_group_leave should be inside the completionHandler of executeQuery.
var returnMessage: String? = ""
let downloadGroup = dispatch_group_create()
dispatch_async(utility.GlobalUtilityQueue){
dispatch_group_enter(downloadGroup)
service.executeQuery(query) { ticket, object, error in
// Process the response
let json = JSON(object.JSON)
returnMessage = json["message"].string
println("\(returnMessage)") // will print first
dispatch_group_leave(downloadGroup)
}
dispatch_group_notify(downloadGroup, self.utility.GlobalMainQueue) {
println("second")//will print second
}
}
Obviously, this is not a situation where you would use dispatch group (you'd generally only do it if you were entering and leaving multiple times). Also, the outer dispatch_async is probably unnecessary (you're calling an asynchronous method, so there's no need to dispatch that to some background queue). But I assume this was more of an academic question, so hopefully this helps.