This is on iOS.
I have a Core Database with about 350 000 objects. The objects (Product) have two properties: "Barcode" and "Designation". The user can search for an object by searching for the "Barcode", and the "Designation" should be returned. Everything is working fine, except it's slow. The code I use is:
NSEntityDescription *_product = [NSEntityDescription entityForName:#"Product" inManagedObjectContext:importContext];
NSFetchRequest *fetch = [[NSFetchRequest alloc]init];
[fetch setEntity:_product];
[fetch setPredicate:[NSPredicate predicateWithFormat:#"Barcode == %#",theBarcode]];
NSError *error = nil;
NSArray *results = [importContext executeFetchRequest:fetch error:&error];
NSManagedObject *object = [results objectAtIndex:0];
Since I only want to fetch one object, is there a way to speed it up?
If I load every object into an Array at the start-up I get a very slow start-up for the app and taking a lot of RAM.
Thanks in advance!
EDIT: I added [fetch setFetchLimit:1]; which speed it up a little bit. But the speed is getting slower the further down in the Database the object is.
Is the Barcode attribute indexed?
First, as #paulbailey wrote, check if Barcode is indexed.
But, if you have that many entries, and if your entry only has two properties (barcode and designation), and if you only query from the barcode side and return the designation side, using CoreData might be an overkill.
CoreData gives you a lot of object-oriented facilities with persistence to the disk, but it of course comes with a penalty.
It might be better for you to drop CoreData altogether, and use sqLite directely. There's a nice light-weight Objective-C wrapper called FMDB for that, see here.
If you want to stick to CoreData, one way to make things better is to fetch in the background thread and to show the result in the main thread, as described in this Apple document. This way the UI doesn't freeze while the database is searched.
The reason why it takes longer the further down the database the object is, is that Core Data uses a rather dull search algorithm which just places a pointer to the first object, comprehends its value to the searchitem, places the pointer to the next one and so one untill the comparison matches.
There are tons of search algorithms you can use, depending on your database (sorted/ not sorted lists, tree structure etc.) you could use Quicksearch, Hashsearches, treesearch and so on.
You might also think about setting up a SQlite database instead, which has some nice frameworks with intelligent search algorithms.
Related
So, I am trying to fetch around 2000 objects from core-data and trying to find out which would be the fastest way to fetch them.
Without NSPredicates:
When I add this block (below), the if statement takes 90% of the total time taken to execute, which is understandable, and so I comment it out.
Block {
NSError *error;
NSManagedObjectContext *context = <#Get the context#>;
// The IF block
/* if (![context save:&error])
{
NSLog(#"error %#", error);
}
*/
// The block above takes 90% of the total time of fetch
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *theEntity = [NSEntityDescription
entityForName:#"EntityName" inManagedObjectContext:context];
[fetchRequest setEntity:theEntity];
NSArray *fetchedObjects = [context executeFetchRequest:fetchRequest error:&error];
}
With NSPredicates:
I set the predicate as
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"handle == %#",handle];
and perform the same fetch as shown in Block()
It can be seen that the overall time is the exact opposite -it's around 90% faster if I use the IF BLOCK ~ 6 seconds.
Otherwise, if that block is commented, the fetching time is a LOT ~ 3 minutes. This happens only when set a predicate for the fetchRequest.
Can someone please explain ?
That is,
/*
Save + straight fetch = slow, due to saving.
Straight fetch - pretty fast.
Predicate fetch, slow.
Save + predicate fetch, much faster?
*/
(Guesswork, hopefully educated) After a save the context can perform a predicated fetch directly against the database as it knows that there are no objects whose changes have not been written to the DB yet. Without the save, it has to fetch against the DB and also any extant objects who may have unsaved changes.
When I'm trying to understand the inner workings of Core Data, it helps to turn on the SQL Debug with the following argument:
-com.apple.CoreData.SQLDebug 1
This will print out all the SQL statements that are being executed, so you could really see the difference. It's also possible (just a theory) that right after the save, the persistent store (or perhaps just indexes) are still in cache, making an indexed (predicate) fetch faster.
At any rate, the SQL Debug will show you what statements are executed and when - might lead to some optimizations in your model for indexing.
The efficiency gains have nothing to do with the way you perform your fetch.
The reason for your delays is that you are saving in the if block. This is a quite expensive operation. If you do it 2000 times it might well take several seconds or even more.
When you fetch entities from the persistent store it does not make sense to save, only if you were to modify these entities. And even then you would try to keep your calls to save to a minimum.
I am trying to insert about 400 objects in CoreData with in memory storage, but it is to long. For about 20-30 seconds. Is it normal? May be exists some special case for many insertions?
- (MyObject*) insertNew
{
NSEntityDescription* entityDescr = [MyObject entityDescription];
NSManagedObjectContext* ctx = [NSManagedObjectContext defaultContext];
MyObject* obj = [[[MyObject alloc] initWithEntity:entityDescr
insertIntoManagedObjectContext:ctx] autorelease];
obj.name = #"name";
obj.Id = #"an ID";
return obj;
}
The objects are in hierarchical relationships to each other, like a tree.
Before doing any importing with Core Data, an essential read is Core Data Programming Guide "Efficiently Importing Data".
It will potentially save you a massive amount of time, especially if the import is something that will be done repeatedly.
If this data will be imported the first time your App runs you may want to look at using a seed database, that way you can perform the slow import once during development, then it's just a matter of a small copy operation.
If your database is very large and space is at a premium you may want to look for another solution as this can use more disk space than necessary (seed database will be in the bundle and the documents directory).
I use Core Data on my iPhone app to store about 1000 objects. Each object is assigned one of 5 different groups. Over time, the objects change groups, and within each group, the order of the objects in those groups change and is stored in an array.
I then take this array and use the attribute 'name' to store it to a plist. When I relaunch the app, my AppDelegate scans each object in the database, then compares it to the 'name' attribute in my plist. When this completes, I have 5 arrays of core data objects, sorted by group, each in the original saved order. The end result is good. The time it takes to complete this task is not.
I want to make this faster. If I could save the actual core data object array to my plist, I would do that. But I can only save attributes to it.
Apple has documentation on Implementing Find-or-Create Efficiently, specifically, this code:
// get the names to parse in sorted order
NSArray *employeeIDs = [[listOfIDsAsString componentsSeparatedByString:#"\n"]
sortedArrayUsingSelector: #selector(compare:)];
// create the fetch request to get all Employees matching the IDs
NSFetchRequest *fetchRequest = [[[NSFetchRequest alloc] init] autorelease];
[fetchRequest setEntity:
[NSEntityDescription entityForName:#"Employee" inManagedObjectContext:aMOC]];
[fetchRequest setPredicate: [NSPredicate predicateWithFormat: #"(employeeID IN %#)", employeeIDs]];
// make sure the results are sorted as well
[fetchRequest setSortDescriptors: [NSArray arrayWithObject:
[[[NSSortDescriptor alloc] initWithKey: #"employeeID"
ascending:YES] autorelease]]];NSError *error = nil;
NSArray *employeesMatchingNames = [aMOC
executeFetchRequest:fetchRequest error:&error];
However, the fetchRequest sets a SortDescriptor and sorts using an 'employeeID' key. I tried this. I have an ID key attribute and can save the array index position when an object is added to a group. However, because objects are being added and removed from each group, the index of the object is constantly changing. So after each change, I would have to rescan each group and reset the index. To me, this would just move my speed problem to another part of my program.
My questions are: is there a better way to save the index position of objects in an array? or is there a different place I can store an array of core data objects? If I did the latter, though, because my app is already in the appStore, my understanding is if I add attributes or tables to the database, this can cause problems when a user upgrades.
I hope I explained this well. Any help would be appreciated, thanks.
You are making this way to complicated. The problem you want to solve i.e. ordering objects based on an attribute value, is a very common need e.g. every single tableview requires this type of ordering.
Ordering is why you can provide one or more sort descriptors to a fetch. In this case you want to order on an employeeID attribute so you just provide a sort that on that key. The array returned by the fetch will be sorted on that attribute.
When you add or remove objects, you just rerun the fetch again either directly or by using notifications. If you use a NSFetchedResultsController, this is all handled for you automatically.
If you need some kind of arbitrary ordering, say a user's list of favorites, then you need to model that arbitrary order directly in the data model. There are various means of doing so depending on the specific type of ordering needed.
Here is a good rule of thumb when designing with Core Data: If you have any other data structure to hold or order managed objects beyond the arrays returned by fetches, you've done something wrong.
Used properly, Core Data can managed the entire data model without any external structures. If you find yourself welding extra data structures onto Core Data, then you've missed something in your data model design.
On the iPhone there is no way to maintain an ordered collection other than storing an index. That property is an integer, correct?
If you add attributes what happens is you need to set up the persistantStoreCoordinator in a way that it attempts an auto-migration. If you never created another model version before though, this will be tricky. You should try that and see if you can get it to work as you are probably going to have to do it someday...
My feeling on CoreData is that the migration is tricky enough you should always keep any user generated data to the side in something like a plist also, that you can re-build a data base from. It sounds like you may already be doing that? If so, you can check when you create the persistantStoreCoordinator if it has failed, and if so then erase the database that is there, create the persistantStoreCoordinator again, and populate that database from your plists.
I'm writing a coredata based iPhone app that displays recipes. To improve performance, when displaying them in a TableView, I want to enable batching (-setFetchBatchSize:) and fetching only the "name" attribute (-setPropertiesToFetch:). When I turn on both, it doesn't work and the list is empty. As soon as I comment out one of the lines marked in the code below, it works fine.
What am I missing here? Is it impossible to have both?
NSFetchRequest *fetchRequest = [[[NSFetchRequest alloc] init] autorelease];
[fetchRequest setEntity:[NSEntityDescription entityForName:#"Rezept" inManagedObjectContext:chk_context]];
// *snip*
//BATCHING
[fetchRequest setFetchBatchSize:25];
NSDictionary *entityProperties = [[NSEntityDescription entityForName:#"Rezept" inManagedObjectContext:chk_context] propertiesByName];
//PROPERTIES
[fetchRequest setPropertiesToFetch:[NSArray arrayWithObject:[entityProperties objectForKey:#"name"]]];
I am far from a core data expert, but I got this to work successfully in my situation. I think the "conflict" between setFetchBatchSize and setPropertiesToFetch is more a side-effect of how core data works and not a bug per-se.
In my case, I did two fetches. In the first one, the result type was set to NSManagedObjectResultType and I used setFetchBatchSize to limit the amount of data actively brought into memory. In the second fetch, I populate an array of titles based on a single attribute and set the result type to NSDictionaryResultType and fetchBatchSize to 0 (infinite).
Based on testing, this scenario works perfectly. All the records in the initial fetch (with the actual managedObjects) are faulted and memory-limited by the fetchBatchSize. The second fetch returns a simple dictionary of titles. This takes much less memory than iterating through all the actual managedObjects to access the title attribute. It makes sense that the second fetch needs fetchBatchSize disabled as it returns the fully populated dictionary as a single result and batching wouldn't be appropriate.
I'm not sure I'm being 100% clear here (core data terminology is a bit arcane...) but the bottom line is that I think everything is working as intended.
Looks a lot like you found a bug in CoreData. You can verify for sure by turning on SQL logging - I'm guessing turning on both options generates slightly invalid SQL.
The option you want to use is "com.apple.CoreData.SQLDebug 1" - you can specify this from the command-line, or set the default on your program.
-Wil
Update: When I NSLog the error after
[fetchedResultsController performFetch:&error];
I get "NSUnderlyingException = The database appears corrupt. (invalid primary key);". But I don't know what that means and what it has to do with enableing both methods.
Hunter you can use com.apple.CoreData.SQLDebug 1 on the phone but not using the simulator
Is there a way to select a fixed number of random entries from a Core Data store? I am just getting started with Core Data and have been stuck on this problem for quite some time.
As a last resort, I could query a large selection of entries into memory and then randomly select a fixed number.
Also, is there a way to specify custom SQL statements to be executed on the Core Data store? I realize that this would be highly unlikely since the underlying implementation of the store could be an XML file as well.
Mmm... maybe
[[[managedObjectsContext registeredObjects] allObjects] objectsAtIndex:r]
where r is random int between 0 and the number of objects minus one? Not efficient at all but quick and easy.
EDIT: if you want to pick the random object between a selection of your objects then create a fetch request that describes your object selection and do the same as above with the query results:
[[[managedObjectsContext executeFetchRequest:request error:&error] objectAtIndex:r]
With regards to your second question, that is one of the points of Core Data, to abstract away the underlying data store. Using NSPredicate and NSExpressions to build a fetch request, or storing one in the data model is the only way to query the store.
Not sure, but I think Core Data puts data into NSSets for you. So you might be able to use -[NSSet anyObject]. I haven't tested that or anything.