How to deal with concurrency on core data - swift

I've been wandering around google, stackoverflow and internet trying to understand how to work with core data and deal with the concurrency.
Consider that we have 2 tables, Events and Rooms.
An Event can have 1+ Rooms.
FunctionA - AddEvent
FunctionB - AddRoom
FunctionC - SearchRoom -> returns RoomEntity or nil
My problem, I keep getting these errors
Error Domain=NSCocoaErrorDomain Code=133020 "Could not merge changes." UserInfo={conflictList=(
"NSMergeConflict (0x10a507160) for NSManagedObject (0x1092f00c0) with objectID '0xd000000000040000 <x-coredata://A34C65BD-F9F0-4CCC-A9FB-1B1F5E48C70E/Rooms/p1>' with oldVersion = 116 and newVersion = 124 and old object snapshot = {\n location = Lisboa;\n name = \"\\U00cdndico LX\";\n} and new cached row = {\n location = Lisboa;\n name = \"\\U00cdndico LX\";\n}"
Notice the information of the Rooms is equal
my approach is the following.
1- I call the webservice once ( it brings a json with data of 3 types of Events ) These 3 all have the same json structure and share the same managedObjectContext passed by parameter
2- I create a managedObject
var managedObjectContext: NSManagedObjectContext = NSManagedObjectContext(concurrencyType: .mainQueueConcurrencyType)
managedObjectContext = persistentContainer.viewContext
managedObjectContext.parent?.mergePolicy = NSMergePolicyType.mergeByPropertyObjectTrumpMergePolicyType
3-
managedObjectContext.perform(
{
do
{
try self.deleteAllEventsFromDb()
FunctionA(eventList, managedObjectContext) -> save
FunctionA(eventList2, managedObjectContext) -> save
FunctionA(eventList3, managedObjectContext) -> save
self.DatabaseDispatchGroup.enter()
try managedObjectContext.save()
self.DatabaseDispatchGroup.leave()
completion(Result.Success(true))
}
catch let error as NSError
{
print("Could not save. \(error), \(error.userInfo)")
completion(Result.Success(false))
}
})
4- For each Event I execute the same FunctionA to create and save the data in database (managedObjectContext.insert(eventEntity)) . This will work over several tables but lets only consider Events and Rooms(FunctionB).
5- FunctionA contains functionB. Function B search for an existing Room(FunctionC->returns entity?) if it doesn't exists(nil), it creates the entity ( should I save here? )
6- If a Room exists, gets the entity and tries to update the data
Not sure if its making any difference but when I save I do these saves I do it between a dispatchGroup
DatabaseDispatchGroup.enter()
try managedObjectContext.save()
DatabaseDispatchGroup.leave()
I was using a static managedObjectContext which was used for all the database requests but now I decided to create a objectContext per function which accesses the database.
I do keep the same persistentContainer and the same DispatchGroup for all requests
private override init() {
persistentContainer = NSPersistentContainer(name: "DataModel")
persistentContainer.loadPersistentStores() { (description, error) in
if let error = error {
fatalError("Failed to load Core Data stack: \(error)")
}
}
}
It seems to me that my problem is that I am Storing the changes in memory, and always doing updates over the initial data meaning that when I execute save() the context the data is not updated for the next operation?
How/when am I suppose to execute the save function?
Thank you

Once context is being saved, global notification is being posted: ContextDidSave notification.
When using multiple contexts (and not using parent-child approach) you should use this notification and either:
Re-fetch/refresh data in case you need to update view or perform some operation on new data set (using either fetch request or refreshObjects: API).
Merge changes to other contexts (remember about thread confinement! do that only on proper context queues). (merge doc)
There are many articles about it, check for instance this tutorial
and documentation

Related

Snapshot listener, get the document name, Firebase Swift

I am adding a snapshot listener to a firebase Collection/Document/Collection to get the data as it is updated
The issue I am having is I run through a for loop and as it gets the i in the for loop I then use that string (collection name), to direct the snapshot listener, when the data comes in it will add to the data already there rather than change the data. because it doesn't touch the code to get the collection name, as far as I know.
What I need to do is to be able to add the data to a dictionary that has [String:Any], so I can have the ["collection name":name, "document name": document id, "info":document data], this is fine on first run but when data is changed it only get the data from the changed listener but I don't know how I can get it to get the collection name so I can remove that document from the dictionary before adding the new data.
func getClockOutData(completion: #escaping (_ finished: Bool) -> ()) {
for i in projectsList {
Firestore.firestore().collection("Projects").document(i).collection("TimeSheets").addSnapshotListener { (snapshot, err) in
if let err = err {
print("Tony the error was \(err.localizedDescription)")
} else {
print("Tony dataC A get is \(i)")
if let projDocs = snapshot?.documents {
print("Tony dataC get is \(i)")
for d in projDocs {
let dataG = d.data()
let dateG = d.documentID
let dataCT = ["project":i, "dateG":dateG, "inf":dataG] as [String : Any]
print("Tony dataC is \(dataCT)")
self.dataC.append(dataCT)
}
completion(true)
}
}
}
}
}
How can I get the project name (i) when the snapshot fires again with the changes?
I want that so I can create a for loop to check the project and add all the data from the project to a dict with all the same project name grouped and then run through that loop when there are changes to remove the project info before it is re appended
When you're retrieving the documents from the subcollection, no data from the parent document is retrieved.
So you will either have to load the parent document to get that information, or get the information from the context where you make this request (as likely you've loaded the parent document before as part of determining projectsList).

SwiftUI + CoreData: Insert more than 1000 entities with relationships

I have a SwiftUI project in which I'm using CoreData to save data fetched from an API into the device. I was trying to insert the entities in batches which worked fine until I realized that the relationships when inserting in batched are "untouched":
An entity Job, which has a one-to-many relationship with the entity Tag.
A one-to-one relationship with Category.
A one-to-one relationship with Type.
What I'm doing now is inserting the entities manually in a background task:
container.performBackgroundTask { context in
for job in jobs {
let jobToInsert = Job(context: context)
let type = JobType(context: context)
let category = Category(context: context)
jobToInsert.id = Int32(job.id)
....
do {
print("Inserting jobs")
try context.save()
} catch {
// log any errors
}
}
Is there any way to improve the performance by perhaps doing this in a way that I don't know? Because for the user, when they start the app, inserting the jobs one by one isn't a very nice experience because first, it takes a long time (more than 2 minutes) and second because my UI isn't automatically updated as I'm inserting the entities.
Thanks a lot in advance!
EDIT: I also see the memory increasing and after taking the screenshot and before I stopped the process, I saw the memory in 1.09 GB
EDIT 2: This is the code I used when trying to insert the jobs in batch:
private func newBatchInsertRequest(with jobs: [JobCodable]) -> NSBatchInsertRequest {
var index = 0
let total = jobs.count
let batchInsert = NSBatchInsertRequest(
entity: Job.entity()) { (managedObject: NSManagedObject) -> Bool in
guard index < total else { return true }
if let job = managedObject as? Job {
let type = JobType(context: self.container.viewContext)
let category = Category(context: self.container.viewContext)
let data: JobCodable = jobs[index]
job.id = Int32(data.id)
...
return batchInsert
Unfortunately, the relationship can't be built due probably to the context? since I'm getting Thread 13: EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0) in the line let category = Category(context: self.container.viewContext)
Are you calling save() on the context for every job that you insert? If so, I would start there — if you want to load a bunch of jobs all at once, you can insert all your Job instances, set up the relationships, and then only save() once at the very end (outside the loop).
This will not only save you time, but also make it appear as if all the objects show up at the same time to your UI. If that's not what you want, you can experiment with saving in batches within the loop — only call save() every thousand Job objects, for example.

Populating parent Reference (List) in CloudKit with child recordName when creating a child record

I'm trying to build additional functionality beyond that included in a cloudkit tutorial (https://www.raywenderlich.com/4878052-cloudkit-tutorial-getting-started) and have gotten to the point where I have a working app to view Establishments (restaurants) and Notes about them - where I've created the data in the CloudKit dashboard and pre-populated the Notes' recordName(s) in the Establishment's 'notes' Reference (List).
I'm now extending the app to allow the user to add notes and at a point where I'm able to create a new Note, and populate the Establishment's recordName (establishing the parent entity within the child record) with this:
let noteRecord = CKRecord(recordType: "Note")
noteRecord["text"] = noteText.text //grabbed from a UIAlert
if self.establishment?.name != nil { //If the establishment exists
let reference = CKRecord.Reference(recordID: self.establishment!.id, action: .deleteSelf)
noteRecord["establishment"] = reference as! CKRecordValue
}
//works fine, saves the new note as expected with the right recordName for the establishment
CKContainer.default().privateCloudDatabase.save(noteRecord) { [unowned self] record, error in
DispatchQueue.main.async {
if let error = error {
} else {
}
}
}
Now the issue I have is how to grab the recordName of this newly saved Note and append it to the Reference (list) of the Establishment.
The reason being that the way the app was built in the tutorial - when getting all the Notes for an Establishment - it uses the Reference (List) of the establishment. If folks think this is unnecessary from a data structure and just having the reference back to the parent on the child is enough, I'd be interested in the pros/cons of the approaches. As you can tell, I'm just learning!
Any ideas? Thank you!!
Please let me know if more detail would be useful
Data model for Notes - what matters is: "text" = String of the note and "establishment" = reference that holds the Establishment recordName
Data model for Establishment - what matters is "notes" which is a Reference (List) of recordNames of Note items.
GitHub repository here as well: https://github.com/SteveBlackUK/BabiFudTutorial/blob/master/BabiFud/View%20Controllers/NotesTableViewController.swift < this is the view controller I'm working on
While I still have an open question about whether you should store the parent child references in both the parent and the child (probably)...I was finally able to figure out how to append the reference to the child back in the parent record.
Reminder: we are creating a Note about a location (an Establishment), and the goal is to store the reference to the note in the Reference (list) within the Establishment in CloudKit.
//first, make sure we even have an establishment
if self.establishment?.name != nil {
//get the array of note references for the Establishment that we want to update
let noteReferences = CKRecord.Reference(record: noteRecord, action: CKRecord_Reference_Action.deleteSelf)
//store the current array of note references into notesForEstablishment
self.notesForEstablishment = self.establishment?.noteRecords
if self.notesForEstablishment != nil {
if !self.notesForEstablishment!.contains(noteReferences) {
self.notesForEstablishment!.append(noteReferences)
}
} else {
self.notesForEstablishment = [noteReferences]
self.establishment?.noteRecords = self.notesForEstablishment
}
// get the record ID for the current establishment
let establishmentRecordID = self.establishment?.id
//then fetch the establishment from CK
CKContainer.default().publicCloudDatabase.fetch(withRecordID: establishmentRecordID!) { updatedRecord, error in
if let error = error {
print("error handling to come: \(error.localizedDescription)")
} else {
// then update the record and place the array that now holds the reference to the new note into the "notes" Reference (list) in CK
updatedRecord!.setObject(self.notesForEstablishment as __CKRecordObjCValue?, forKey: "notes")
CKContainer.default().publicCloudDatabase.save(updatedRecord!) { savedRecord, error in
}
}
}
}
It could probably be much cleaner code - and any feedback welcome, but it works!

Inserting child records is slow in coredata

I have close to 7K items stored in a relation called Verse. I have another relation called Translation that needs to load 7K related items with a single call from a JSON file.
Here is my code:
let container = getContainer()
container.performBackgroundTask() { (context) in
autoreleasepool {
for row in translations{
let t = Translation(context: context)
t.text = (row["text"]! as? String)!
t.lang = (row["lang"]! as? String)!
t.contentType = "Verse"
t.verse = VerseDao.findById(row["verse_id"] as! Int16, context: context)
// this needs to make a call to the database to retrieve the approparite Verse instance.
}
}
do {
try context.save()
} catch {
fatalError("Failure to save context: \(error)")
}
context.reset()
}
Code for the findById method.
static func findById(_ id: Int16, context: NSManagedObjectContext) -> Verse{
let fetchRequest: NSFetchRequest<Verse>
fetchRequest = Verse.fetchRequest()
fetchRequest.predicate = NSPredicate(format: "verseId == %#", id)
fetchRequest.includesPropertyValues = false
fetchRequest.fetchLimit = 1
do {
let results =
try context.fetch(fetchRequest)
return results[0]
} catch let error as NSError {
print("Could not fetch \(error), \(error.userInfo)")
return Verse()
}
}
This works fine until I add the VerseDao.findById, which makes the whole process really slow because it has to make a request for each object to the Coredata database.
I did everything I could by limiting the number of fetched properties and using NSFetchedResultsController for data fetching but no luck.
I wonder if there's any way to insert child records in a more efficient way? Thanks.
Assuming your store type is persistent store type is sqlite (NSSQLiteStoreType):
The first thing you should check is whether you have an Core Data fetch index on the Verse objects verseId property. See this stack overflow answer for some introductory links on fetch indexes.
Without that, the fetch in your VerseDao.findById function may be scanning the whole database table every time.
To see if your index is working properly you may inspect the SQL queries generated by adding -com.apple.CoreData.SQLDebug 1 to the launch arguments in your Xcode scheme.
Other improvements:
Use NSManagedObjectContext.fetch or NSFetchRequest.execute (equivalent) instead of NSFetchedResultsController. The NSFetchedResultsController is typically used to bind results to a UI. In this case using it just adds overhead.
Don't set fetchRequest.propertiesToFetch, instead set fetchRequest.includesPropertyValues = false. This will avoid fetching the Verse object property values which you don't need to establish the relation to the Translation object.
Don't specify a sortDescriptor on the fetch request, this just complicates the query

How to get data in NSUserDefaults when offline from Parse

I have a project that find objects in Parse as an Array and gets it when the app finish launches. But it can't get the info when there are no network connections so i'm thinking to use NSUserDefaults and firstly save the data to NSUserDefaults and get from NSUserDefaults when it is offline.
Can somebody give me a example of logic?
this is the code i wrote and i don't know how to get it when it's offline.
the Array the i want to append the data from Parse will be [[String]].
let userDefaults = NSUserDefaults.standardUserDefaults
let query = ClassSub.query()
query.whereKey("nameOfData", containsString: "testString")
query!.findObjectsInBackgroundWithBlock { (busObjects: [PFObject]?, error: NSError?) -> Void in
if error != nil {
print(error)
}
for object in testObjects! {
let testData = object["arrayData"] as? [[String]]
for i in 0..<testData!.count {testData
testArrayappend(testData![i])
self.userDefaults.setObject(NSKeyedArchiver.archivedDataWithRootObject(testData[i]), forKey: "test-time")
}
}
}
the data type in Parse is [[String]] and it looks like [["one","two"],["one","three"],["one","four"]]
i think it's better for you to use parse ios SDK local data store. The local data store allows you to pin your parse objects into a local database and then when you are out of network you can still get the data from your local database. Another huge advantage is the saveEventually feature which allows you to save objects while you are offline and then sync them back to the server as soon as you go online.
In order to use the local data store feature you need to do the following steps:
Enable local data store in parse config
let configuration = ParseClientConfiguration {
$0.applicationId = "{PARSE_APP_ID}"
$0.server = "http://localhost:1337/parse"
$0.localDatastoreEnabled = true
}
Parse.initializeWithConfiguration(configuration)
If you want to query from the local data store (while you are offline) you just need to call an additional function before calling the findObjectsInBackground so your code should look like the following:
let query = PFQuery(className: "MyParseObjectClassName")
// if you are in offline mode then make sure the query
// will access your local data store and not to the server
if (offlineMode){
query.fromLocalDatastore()
}
query.findObjectsInBackgroundWithBlock {
(objects: [PFObject]?, error: NSError?) -> Void in
// do something with the result
}
When you are using local data store you are required to pin the objects that are fetched from your server. In order to pin an object simply call to pinInBackground() which exist under your PFObject. You can also use pinAllInBackground() to pin multiple objects to your local data store in one call. In order to pin your objects please do the following:
let query = PFQuery(className: "MyParseObjectClassName")
// if you are in offline mode then make sure the query
// will access your local data store and not to the server
if (self.offlineMode){
query.fromLocalDatastore()
}
query.findObjectsInBackgroundWithBlock {
(objects: [PFObject]?, error: NSError?) -> Void in
if (!self.offlineMode){
// pin all objects
PFObject.pinAllInBackground(objects)
// pin only first object
let obj = objects?.first
obj?.pinInBackground()
}
}
Now in order to know when you are offline or online i recommend you to use Reachability library.
This library provides you two blocks: when you are online and when you are offline. Using those blocks you can determine when your app is connected to the internet and when it doesn't. So when it is not connected to the internet you need to set the offlineMode flag to be true and from now on all the queries will work against your local database otherwise it will work against your server.