Inserting child records is slow in coredata - swift

I have close to 7K items stored in a relation called Verse. I have another relation called Translation that needs to load 7K related items with a single call from a JSON file.
Here is my code:
let container = getContainer()
container.performBackgroundTask() { (context) in
autoreleasepool {
for row in translations{
let t = Translation(context: context)
t.text = (row["text"]! as? String)!
t.lang = (row["lang"]! as? String)!
t.contentType = "Verse"
t.verse = VerseDao.findById(row["verse_id"] as! Int16, context: context)
// this needs to make a call to the database to retrieve the approparite Verse instance.
}
}
do {
try context.save()
} catch {
fatalError("Failure to save context: \(error)")
}
context.reset()
}
Code for the findById method.
static func findById(_ id: Int16, context: NSManagedObjectContext) -> Verse{
let fetchRequest: NSFetchRequest<Verse>
fetchRequest = Verse.fetchRequest()
fetchRequest.predicate = NSPredicate(format: "verseId == %#", id)
fetchRequest.includesPropertyValues = false
fetchRequest.fetchLimit = 1
do {
let results =
try context.fetch(fetchRequest)
return results[0]
} catch let error as NSError {
print("Could not fetch \(error), \(error.userInfo)")
return Verse()
}
}
This works fine until I add the VerseDao.findById, which makes the whole process really slow because it has to make a request for each object to the Coredata database.
I did everything I could by limiting the number of fetched properties and using NSFetchedResultsController for data fetching but no luck.
I wonder if there's any way to insert child records in a more efficient way? Thanks.

Assuming your store type is persistent store type is sqlite (NSSQLiteStoreType):
The first thing you should check is whether you have an Core Data fetch index on the Verse objects verseId property. See this stack overflow answer for some introductory links on fetch indexes.
Without that, the fetch in your VerseDao.findById function may be scanning the whole database table every time.
To see if your index is working properly you may inspect the SQL queries generated by adding -com.apple.CoreData.SQLDebug 1 to the launch arguments in your Xcode scheme.
Other improvements:
Use NSManagedObjectContext.fetch or NSFetchRequest.execute (equivalent) instead of NSFetchedResultsController. The NSFetchedResultsController is typically used to bind results to a UI. In this case using it just adds overhead.
Don't set fetchRequest.propertiesToFetch, instead set fetchRequest.includesPropertyValues = false. This will avoid fetching the Verse object property values which you don't need to establish the relation to the Translation object.
Don't specify a sortDescriptor on the fetch request, this just complicates the query

Related

How to deal with concurrency on core data

I've been wandering around google, stackoverflow and internet trying to understand how to work with core data and deal with the concurrency.
Consider that we have 2 tables, Events and Rooms.
An Event can have 1+ Rooms.
FunctionA - AddEvent
FunctionB - AddRoom
FunctionC - SearchRoom -> returns RoomEntity or nil
My problem, I keep getting these errors
Error Domain=NSCocoaErrorDomain Code=133020 "Could not merge changes." UserInfo={conflictList=(
"NSMergeConflict (0x10a507160) for NSManagedObject (0x1092f00c0) with objectID '0xd000000000040000 <x-coredata://A34C65BD-F9F0-4CCC-A9FB-1B1F5E48C70E/Rooms/p1>' with oldVersion = 116 and newVersion = 124 and old object snapshot = {\n location = Lisboa;\n name = \"\\U00cdndico LX\";\n} and new cached row = {\n location = Lisboa;\n name = \"\\U00cdndico LX\";\n}"
Notice the information of the Rooms is equal
my approach is the following.
1- I call the webservice once ( it brings a json with data of 3 types of Events ) These 3 all have the same json structure and share the same managedObjectContext passed by parameter
2- I create a managedObject
var managedObjectContext: NSManagedObjectContext = NSManagedObjectContext(concurrencyType: .mainQueueConcurrencyType)
managedObjectContext = persistentContainer.viewContext
managedObjectContext.parent?.mergePolicy = NSMergePolicyType.mergeByPropertyObjectTrumpMergePolicyType
3-
managedObjectContext.perform(
{
do
{
try self.deleteAllEventsFromDb()
FunctionA(eventList, managedObjectContext) -> save
FunctionA(eventList2, managedObjectContext) -> save
FunctionA(eventList3, managedObjectContext) -> save
self.DatabaseDispatchGroup.enter()
try managedObjectContext.save()
self.DatabaseDispatchGroup.leave()
completion(Result.Success(true))
}
catch let error as NSError
{
print("Could not save. \(error), \(error.userInfo)")
completion(Result.Success(false))
}
})
4- For each Event I execute the same FunctionA to create and save the data in database (managedObjectContext.insert(eventEntity)) . This will work over several tables but lets only consider Events and Rooms(FunctionB).
5- FunctionA contains functionB. Function B search for an existing Room(FunctionC->returns entity?) if it doesn't exists(nil), it creates the entity ( should I save here? )
6- If a Room exists, gets the entity and tries to update the data
Not sure if its making any difference but when I save I do these saves I do it between a dispatchGroup
DatabaseDispatchGroup.enter()
try managedObjectContext.save()
DatabaseDispatchGroup.leave()
I was using a static managedObjectContext which was used for all the database requests but now I decided to create a objectContext per function which accesses the database.
I do keep the same persistentContainer and the same DispatchGroup for all requests
private override init() {
persistentContainer = NSPersistentContainer(name: "DataModel")
persistentContainer.loadPersistentStores() { (description, error) in
if let error = error {
fatalError("Failed to load Core Data stack: \(error)")
}
}
}
It seems to me that my problem is that I am Storing the changes in memory, and always doing updates over the initial data meaning that when I execute save() the context the data is not updated for the next operation?
How/when am I suppose to execute the save function?
Thank you
Once context is being saved, global notification is being posted: ContextDidSave notification.
When using multiple contexts (and not using parent-child approach) you should use this notification and either:
Re-fetch/refresh data in case you need to update view or perform some operation on new data set (using either fetch request or refreshObjects: API).
Merge changes to other contexts (remember about thread confinement! do that only on proper context queues). (merge doc)
There are many articles about it, check for instance this tutorial
and documentation

Better approach to querying sqlite database in swift

I'm working on a word game and have bundled a complete list of english words using a sqlite database. I'm trying to find the best way to search the database for a given string to determine if it's a word.
At this point I can get the whole database out into an array:
func fetchWords() {
if let managedObjectContext = (UIApplication.shared.delegate as? AppDelegate)?.managedObjectContext {
let wordsFetch = NSFetchRequest<NSFetchRequestResult>(entityName: "EnglishWord")
do {
englishWords = try managedObjectContext.fetch(wordsFetch) as! [EnglishWord]
print(englishWords[123323].word)
//Prints "injustices"
} catch {
//error handling
}
}
}
Now. What I really want to do is pass in a given string and see if it exists as a word in my database. I have a clunky solution with predicates, e.g:
func fetchWordsToArray() {
if let managedObjectContext = (UIApplication.shared.delegate as? AppDelegate)?.managedObjectContext {
let wordsFetch = NSFetchRequest<NSFetchRequestResult>(entityName: "EnglishWord")
let searchWord = ["dastardly"]
let searchPredicate = NSPredicate(format: "word = %#", argumentArray: searchWord)
do {
englishWords = try managedObjectContext.fetch(wordsFetch) as! [EnglishWord]
let resultFilteredByPredicate = (englishWords as NSArray).filtered(using: predicate)
print(resultFilteredByPredicate)
} catch {
//error handling
}
}
}
But in order to use the filtered function I have to convert to an NSArray which means I can't work with the results directly (e.g. get resultFilteredByPredicate.word).
Also it feels like I'm probably going about this all the wrong way. Since everything has to go into an array first, I must be losing a lot of the value of using an sqlite database in the first place.
Any suggested approaches for better working with the database?
Many thanks in advance for any help!
To make the database do the filtering (which could then be optimized automatically with an index), put a predicate on the original fetch request:
let formatRequest : NSFetchRequest<Word> = Word.fetchRequest()
fetchRequest.predicate = NSPredicate(format: "word == %#", searchWord)
let fetchedResults = try context.fetch(fetchRequest) as! [Word]
But Core Data is a framework to manage your objects.
If you decide that your words are not objects but just values, you could replace Core Data with some other library that does SQL directly, and execute an SQL query instead:
SELECT * FROM EnglishWord WHERE word = ?

Fetching a one to many relationship using Core Data

iam a swift beginner, i wanna code a little app for me and i have a problem with it. My app has three entities (Templates, Records and Positions), here you can see:
Datamodel
At one part of the app i can add new Records and for that Record i add some Positions (xpos and ypos).
At a part I have a tableview were i list my Records. Now i wanna click one Record in the table and i want to get all Position-Attributes linked with the particular selected Record.
With that code i can get all xpos-Positions but how can i get a single xpos? :)
guard let moc = self.managedContext else {
return
}
let fetchRequest: NSFetchRequest<Records> = Records.fetchRequest()
do {
let searchResults = try moc.fetch(fetchRequest)
let xpos = searchResults[0].positions?.mutableArrayValue(forKey: "xpos")
print(xpos)
} catch {
print("Error with request: \(error)")
}
You want to look into using NSPredicate. NSPredicate basically lets you define the conditions which the record needs to meet to be included in the results. Think of it as a filter if you will.
fetchRequest.predicate = NSPredicate(format: "uniqueID = %#", arugments:...)

How to Save to a Custom Join Table Core Data (many-to-many) without unique predicate - ManagedObjectID (Swift)?

I will be super thankful for any help. How can I save instances to a join table without a unique identifier as a predicate? Can I use the managed object id to check if the item exists already?
I'm building an app with different exercise plans. Each plan holds many exercise, and an exercise can belong to many plans. I have structured my data model to include a custom join table so that I can query the completion status of an exercise from within one plan.
I'm sourcing my data from a json file and would like to save it to core data. I'm able to correctly save my CoreExercise, and CorePlan tables, however am having difficulty understanding how to save the instance of the object in the intermediate join table, since I'm unsure of what predicate to use.
I've written a class function to check if the instance exists, and to save it if it doesn't.
class CoreExercisePlan: NSManagedObject {
class func coreExercisesForExercisePlan(exerciseInfo: Exercise, planName: String, inManagedObjectContext context: NSManagedObjectContext) -> CoreExercisePlan? {
let request = NSFetchRequest(entityName: "CoreExercisePlan")
request.predicate = NSPredicate() // Search for ObjectID here? / How?
if let exercisePlan = (try? context.executeFetchRequest(request))?.first as? CoreExercisePlan {
print("we have this exercise plan already saved")
return exercisePlan
} else if let exercisePlan = NSEntityDescription.insertNewObjectForEntityForName("CoreExercisePlan", inManagedObjectContext: context) as? CoreExercisePlan {
exercisePlan.status = 0
exercisePlan.progress = 0
print("we are creating new object")
return exercisePlan
}
return nil
}
private func updateDatabaseWithExercisePlans(){
managedObjectContext?.performBlock {
// Array of exercises for each plan:
let coffeePlanExercises = self.coffeeExercises
let subwayPlanExercises = self.subwayExercises
for exercise in coffeePlanExercises {
_ = CoreExercisePlan.coreExercisesForExercisePlan(exercise, planName: "coffee", inManagedObjectContext: self.managedObjectContext!)
}
for exercise in subwayPlanExercises {
_ = CoreExercisePlan.coreExercisesForExercisePlan(exercise, planName: "subway", inManagedObjectContext: self.managedObjectContext!)
}
do {
try self.managedObjectContext?.save()
} catch let error {
print("printing error here: \(error)")
}
}
}
Is there a way to get the objectID of the instance in the join table, and use that as a predicate? Thanks!

How to improve performance for large datasets with Realm?

My database has 500,000 records. The tables don't have a primary key because Realm doesn't support compound primary keys. I fetch data in background thread, then I want to display it in the UI on the main thread. But since Realm objects cannot be shared across threads I cannot use the record I fetched in the background. Instead I need to refetch the record on main thread? If I fetch a record out of the 500,000 records it will block the main thread. I don't know how to deal with it. I use Realm because it said it's enough quick. If I need refetch the record many times, is it really faster than SQLite? I don't want to create another property that combine other columns as primary key because the Realm database is already bigger than a SQLite file.
#objc class CKPhraseModel: CKBaseHMMModel{
dynamic var pinyin :String!
dynamic var phrase :String = ""
class func fetchObjects(apinyin :String) -> Results<CKPhraseModel> {
let realm = Realm.createDefaultRealm()
let fetchString = generateQueryString(apinyin)
let phrases = realm.objects(self).filter(fetchString).sorted("frequency", ascending: false)
return phrases
}
func save(needTransition :Bool = true) {
if let realm = realm {
try! realm.write(needTransition) {[unowned self] in
self.frequency += 1
}
}
else {
let realm = Realm.createDefaultRealm()
if let model = self.dynamicType.fetchObjects(pinyin).filter("phrase == %#", phrase).first {
try! realm.write(needTransition) {[unowned self] in
model.frequency += self.frequency
}
}
else {
try! realm.write(needTransition) {[unowned self] in
realm.add(self)
}
}
}
}
}
then I store fetched records in Array
let userInput = "input somthing"
let phraseList = CKPhraseModel().fetchObjects(userInput)
for (_,phraseModel) in phraseList.enumerate() {
candidates.append(phraseModel)
}
Then I want to display candidates information in UI when the user clicks one of these. I will call CKPhraseModel's save function to save changes. This step is on main thread.
Realm is fast if you use its lazy loading capability, which means that you create a filter that would return your candidates directly from the Realm, because then you'd need to only retrieve only the elements you index in the results.
In your case, you copy ALL elements out. That's kinda slow, which is why you end up freezing.