Debugging async CoreData calls - swift

Question

i am adding await and concurrency methods to my existing project and ran into some strange behaviours
which i am unable to debug because they don't happen each and every time, just randomly sometimes, somewhere down the road (inside buildDataStructureNew)
func buildDataStructureNew() async -> String {
var logComment:String = ""
let context = await (UIApplication.shared.delegate as! AppDelegate).persistentContainer.viewContext
// Step 1 get JSON data
let jsonData = try? await getRemoteFoo()
guard jsonData != nil else {...}
let feedback2 = await step2DeleteAllEntities(context: context) // Step 2
{...}
let feedback3 = await step3saveJSONtoFoo(context: context, remoteData: remoteData) // Step3
{...}
let sourceData = await step41getAllSavedFoo(context: context) // Step 4.1
{...}
let feedback42 = await step42deleteAllSingleFoos(context: context) //Step 4.2
{...}
let feedback43 = await step43splitRemoteDataIntoSingleDataFoos(context: context, sourceData: sourceData) // Step 4.3
{...}
let feedback5 = await step5createDataPacks() // Step 5
return logComment
}
as you see i executed each step with the same context, expecting it to work properly
but i started to receive fatal errors from step 4.3, which i could not explain myself..
CoreData: error: Serious application error. Exception was caught during Core Data change processing.
This is usually a bug within an observer of NSManagedObjectContextObjectsDidChangeNotification.
-[__NSCFSet addObject:]: attempt to insert nil with userInfo (null)
so i tried using a different context just for this single step
let context = await (UIApplication.shared.delegate as! AppDelegate).persistentContainer.newBackgroundContext() // just to test it
Step 3 as an example:
func step3saveJSONtoFoo(context: NSManagedObjectContext, remoteData:[remoteData]) async -> String {
var feedback:String = ""
var count:Int = 0
for i in 0...remoteFoo.count-1 {
let newFoo = remoteFoos(context: context)
newFoo.a = Int16(remoteFoo[i].a)
newFoo.b = remoteFoo[i].b
newFoo.c = remoteFoo[i].c
newFoo.Column1 = remoteFoo[i].Column1
newFoo.Column2 = remoteFoo[i].Column2
newFoo.Column3 = remoteFoo[i].Column3
newFoo.Column4 = remoteFoo[i].Column4
newFoo.Column15 = remoteFoo[i].Column4
count = i
do {
try context.save()
// print("DEBUG - - ✅ save JSON to RemoteFoo successfull")
} catch {
feedback = "\n-- !!! -- saving JSON Data to CoreData.Foos failed(Step 3), error:\(error)"
Logging.insertError(message: "error saving JSON to CoreData.Foos", location: "buildDataStructureNew")
}
}
// print("DEBUG - - ✅ Step3 \(count+1) records saved to RemoteFoo")
feedback = feedback + "\n-- ✅ \(count+1) records saved to RemoteFoo"
return feedback
}
and this solved the issue, but i then got the same error from step 5, so i added the background context to this step as well
and on first sight this solved it again
i thought this is it, but a few minutes later the method crashed on me again, but now on step 3, with again the same exact error msg..
as i understood it it has something to do with the context i use, but i don't really get what's really the issue here.
i did not have any issues with this method before, this started to happen on me when i rewrote that method as async..
right now the method is working fine without any issues, or at least i can't reproduce it at the moment, but it might come back soon..
i guess i am missing some understanding here, i hope you guys can help me out

Answer 1

The problems you're seeing are because Core Data has its own ideas about concurrency that don't directly map to any other concurrency technique you might use. Bugs that crop up inconsistently, sometimes but not always, are a classic sign of a concurrency problem.
For concurrent Core Data use, you must use either the context's perform { ... } or performAndWait { ... } with all Core Data access. Async/await is not a substitute, nor is DispatchQueue or anything else you would use for concurrency in other places in an iOS app. This includes everything that touches Core Data in any way-- fetching, saving, merging, accessing properties on a managed object, etc. You're not doing that, which is why you're having these problems.
The only exception to this is if your code is running on the main queue and you have a main-queue context.
While you're working on this you should turn on Core Data concurrency debugging by using -com.apple.CoreData.ConcurrencyDebug 1 as an argument to your app. That's described in various blog posts including this one.

Answer 2

with the help of Tom's explanations i was able to fix this issue
to make sure everything runs in the correct order,
it is important to do those core Data calls within a .perform or performAndWait call
and as step 1 is the only real async action (network API access), this is the only step marked as "async/await",
all other steps are hopefully now correctly queued via CoreData's own queue..
here's my final working code
func buildDataStructureNew() async -> String {
var logComment:String = ""
let context = await (UIApplication.shared.delegate as! AppDelegate).persistentContainer.viewContext
// Step 1 get JSON data
let jsonData = try? await getRemoteFoo()
guard jsonData != nil else {...}
let feedback2 = step2DeleteAllEntities(context: context) // Step 2
{...}
let feedback3 = step3saveJSONtoFoo(context: context, remoteData: remoteData) // Step3
{...}
let sourceData = step41getAllSavedFoo(context: context) // Step 4.1
{...}
let feedback42 = step42deleteAllSingleFoos(context: context) //Step 4.2
{...}
let feedback43 = step43splitRemoteDataIntoSingleDataFoos(context: context, sourceData: sourceData) // Step 4.3
{...}
let feedback5 = step5createDataPacks() // Step 5
return logComment
}
and again step 3 as an example
func step3saveJSONtoFoo(context: NSManagedObjectContext, remoteData:[remoteData]) -> String {
var feedback:String = ""
var count:Int = 0
for i in 0...remoteFoo.count-1 {
let newFoo = remoteFoos(context: context)
newFoo.a = Int16(remoteFoo[i].a)
newFoo.b = remoteFoo[i].b
{...}
count = i
context.performAndWait {
do { try context.save() }
catch let error {
feedback = "\n-- !!! -- saving JSON Data to CoreData.Foos failed(Step 3), error:\(error)"
Logging.insertError(message: "error saving JSON to CoreData.Foos", location: "buildDataStructureNew")
}
}
}
feedback = feedback + "\n-- ✅ \(count+1) records saved to RemoteFoo"
return feedback
}
one last note: Xcode initially told me that "performAndWait" is only available in iOS 15 and above,
that is true because apple released a new version of this function which is marked as "rethrow",
but there is also the old version of this function available and this version of "perform" is available from iOS 5 and above ;)
the trick is to not forget to use a own catch block after the do statement ;)