I am creating and adding a number of managed objects to Core Data from a background queue. My understanding was that I could not access the context from the background thread so I was using performBlock to schedule adding to Core Data back onto the same queue that the context was created on. This works just fine ...
My question is during testing I noticed that by removing [moc performBlock:^{ ... }]; the application still performs as expected (maybe even fractions of a second faster) Do I need the performBlock? I would assume I do and its just working (for now :) in a none-threadsafe fashion, I just wanted to check to make sure my understanding is not flawed.
dispatch_queue_t backgroundQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
dispatch_async(backgroundQueue, ^{
// GET DATA
// PROCESS DATA
NSManagedObjectContext *context = [[self managedDocument] managedObjectContext];
[moc performBlock:^{
// ADD TO CORE DATA
[Core createRodInContext:context withDictionary:fuelRodDictionary];
}];
});
EDIT: Added implementation for createRodInContext ...
+ (Rod *)createRodInContext:(NSManagedObjectContext *)context withDictionary:(NSDictionary *)dictionary {
// CREATE
Rod *rod = [NSEntityDescription insertNewObjectForEntityForName:#"Rod" inManagedObjectContext:context];
// POPULATE
[neo setDataCode:[dictionary objectForKey:#"dataCode"]];
[neo setDataName:[dictionary objectForKey:#"dataName"]];
[neo setDataReference:[dictionary objectForKey:#"dataReference"]];
...
return rod;
}
In the background thread you have to use [moc performBlock:^{ ... }] to insert (and populate) a managed object in the main managed object context.
Omitting the performBlock means that you use the managed object context (which was created on the main thread) also in a different thread (which is associated with the background queue).
This might work by chance, but as soon as the main thread accesses the MOC in the same moment as your background thread, the results are unpredictable because (as you already said), a MOC is not thread-safe.
See also Concurrency Support for Managed Object Contexts
in the Core Data Release Notes for OS X v10.7 and iOS 5.0:
Confinement (NSConfinementConcurrencyType).
This is the default. You
promise that context will not be used by any thread other than the one
on which you created it.
But also for the other concurrency types (private queue, main queue), you always have to use performBlock (or performBlockAndWait) unless your code is already executing on the queue associated with the MOC.
Related
My app sometimes becomes dead , and the call stack is like below,
Update:
I've spent two days but could not handle it yet.
What I have are contexts for each thread, and one retained context for a special serial queue.
What I am wondered about is that how the deadlock comes. Why all the threads are in waiting states? Just one possibility will be much appreciated.
Thanks a lot.
That happens some times when doing work in a notification handler that may result in other notifications being sent, easiest way to avoid it is to dispatch_async the work that needs to be done.
dispatch_async(dispatch_get_current_queue(), ^{
// The work that needs to be done...
});
This usually happens when one tries to access Core Data objects on a background thread using the main-threads context OR using the same managed object context on different threads (background or main) at the same time. For more details check out Core Data concurrency rules.
So to avoid both cases, the main rule is, each thread must have its own managed object context and initialize that context exactly where it's going to be used.
For example:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
//
// Prepare your background core data context
//
if (self.privateContext == nil)
{
self.privateContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
[self.privateContext setParentContext: - main managed object context - ];
[self.privateContext setUndoManager:nil]; // this context should not manage undo actions.
}
//
// Do any Core Data requests using this thread-save context
//
.
.
.
});
I am running into intermittent, hard-to-reproduce errors on my iPhone app, so I am checking my assumptions around concurrency.
Running AFNetworking v0.10.x, I have the following network call:
[self postPath:#"/myEndPoint"
parameters:params
success:^(AFHTTPRequestOperation *request, id response)
{
AppDelegate *appDelegate = (AppDelegate*)[[UIApplication sharedApplication] delegate];
// do stuff with object context here
[appDelegate.objectContext save];
}
]
AppDelegate *appDelegate = (AppDelegate*)[[UIApplication sharedApplication] delegate];
// do other stuff with object context
[appDelegate.objectContext save];
In my AppDelegate:
-(NSManagedObjectContext*) objectContext
{
if(nil == _objectContext)
{
... set up sqlite persistent store coordinator and object model ...
_objectContext = [[NSManagedObjectContext alloc] init];
[_objectContext setPersistentStoreCoordinator:persistentStoreCoordinator];
[_objectContext setMergePolicy:NSOverwriteMergePolicy];
}
return _objectContext;
}
Is it possible, in this scenario, to end up with concurrency problems? Or, in other words, is AFNetworking's API thread-safe? I thought the NSOverwriteMergePolicy would cover me for conflicts, but crashing persists (albeit intermittently).
AFNetworking's callbacks are executed on the main thread. As a result, they are 'thread-safe', because there is only one thread that is interacting with CoreData. If you only have a single managed object things will be straightforward.
From Apple:
Tasks added to this queue are performed serially on the main thread itself. Therefore, you can use this queue as a synchronization point for work being done in other parts of your application.
There are still lots of considerations when using multi-threaded CoreData and multiple managed object contexts, and for those I refer you to rsswtmr's excellent answer, which doesn't correctly answer my question, but provides links to lots of good information.
You can't have multiple threads working on the same object context. Think through how Core Data might be part-way through changing/committing data while another change comes through on another thread. You need to create a separate context per thread and merge changes when it's safe/appropriate. The NSOverwriteMergePolicy will simply save you from having to manually handle conflicts at merge time.
Look here for a great explanation of threading Core Data access.
I have recently rewritten my Core Data driven database controller to use Grand Central Dispatch to manage fetching and importing in the background. Controller can operate on 2 NSManagedContext's:
NSManagedObjectContext *mainMoc instance variable for main thread. this contexts is used only by quick access for UI by main thread or by dipatch_get_main_queue() global queue.
NSManagedObjectContext *bgMoc for background tasks (importing and fetching data for NSFetchedresultsController for tables). This background tasks are fired ONLY by user defined queue: dispatch_queue_t bgQueue (instance variable in database controller object).
Fetching data for tables is done in background to not block user UI when bigger or more complicated predicates are performed.
Example fetching code for NSFetchedResultsController in my table view controllers:
-(void)fetchData{
dispatch_async([CDdb db].bgQueue, ^{
NSError *error = nil;
[[self.fetchedResultsController fetchRequest] setPredicate:self.predicate];
if (self.fetchedResultsController && ![self.fetchedResultsController performFetch:&error]) {
NSSLog(#"Unresolved error in fetchData %#", error);
}
if (!initial_fetch_attampted)initial_fetch_attampted = YES;
fetching = NO;
dispatch_async(dispatch_get_main_queue(), ^{
[self.table reloadData];
[self.table scrollRectToVisible:CGRectMake(0, 0, 100, 20) animated:YES];
});
});
} // end of fetchData function
bgMoc merges with mainMoc on save using NSManagedObjectContextDidSaveNotification:
- (void)bgMocDidSave:(NSNotification *)saveNotification {
// CDdb - bgMoc didsave - merging changes with main mainMoc
dispatch_async(dispatch_get_main_queue(), ^{
[self.mainMoc mergeChangesFromContextDidSaveNotification:saveNotification];
// Extra notification for some other, potentially interested clients
[[NSNotificationCenter defaultCenter] postNotificationName:DATABASE_SAVED_WITH_CHANGES object:saveNotification];
});
}
- (void)mainMocDidSave:(NSNotification *)saveNotification {
// CDdb - main mainMoc didSave - merging changes with bgMoc
dispatch_async(self.bgQueue, ^{
[self.bgMoc mergeChangesFromContextDidSaveNotification:saveNotification];
});
}
NSfetchedResultsController delegate has only one method implemented (for simplicity):
- (void)controllerDidChangeContent:(NSFetchedResultsController *)controller {
dispatch_async(dispatch_get_main_queue(), ^{
[self fetchData];
});
}
This way I am trying to follow Apple recommendation for Core Data: 1 NSManagedObjectContext per thread. I know this pattern is not completely clean for at last 2 reasons:
bgQueue not necessarily fires the same thread after suspension but since it is serial, it should not matter much (there is never 2 threads trying access bgMoc NSManagedObjectContext dedicated to it).
Sometimes table view data source methods will ask NSFetchedResultsController for info from bgMoc (since fetch is done on bgQueue) like sections count, fetched objects in section count, etc....
Event with this flaws this approach works pretty well of the 95% of application running time until ...
AND HERE GOES MY QUESTION:
Sometimes, very randomly application freezes but not crashes. It does not response on any touch and the only way to get it back to live is to restart it completely (switching back to and from background does not help).
No exception is thrown and nothing is printed to the console (I have Breakpoints set for all exception in Xcode).
I have tried to debug it using Instruments (time profiles especially) to see if there is something hard going on on main thread but nothing is showing up.
I am aware that GCD and Core Data are the main suspects here, but I have no idea how to track / debug this.
Let me point out, that this also happens when I dispatch all the tasks to the queues asynchronously only (using dispatch_async everywhere). This makes me think it is not just standard deadlock.
Is there any possibility or hints of how could I get more info what is going on? Some extra debug flags, Instruments magical tricks or build setting etc...
Any suggestions on what could be the cause are very much appreciated as well as (or) pointers to how to implement background fetching for NSFetchedResultsController and background importing in better way.
My first and very bad mistake was to fetch data for NSFetchedResultsController in the background queue.
It turned out after testing, I was way too sensitive about fetching times. I unnecessary did put fetchData execution to back thread making core data related code too complex when the longest fetch time I could generate took literally split of a second. This introduced way too much complexity and uncertainty for very small performance gain (if any).
I resigned form that by moving fetchData execution and all NSFetchedResultsControllerDelegate method to the main thread (simplified the code by removing GCD code).
When this was done I no longer needed mainMocDidSave: and unregistered from listening to the NSManagedObjectContextDidSaveNotification for main thread context.
I could also removed and unregistered DATABASE_SAVED_WITH_CHANGES notification posting.
This greatly simplified 'merging' mechanism as from this time on only background thread context merges its changes with main thread context (when saved). Let's call it one directional change notifications.
NSFetchedResultsControllerDelegate methods will be fired automatically as they pickup main thread context changes after merge.
Another important thing is to change dispatch_async to dispatch_sync in:
- (void)bgMocDidSave:(NSNotification *)saveNotification {
// CDdb - bgMoc didsave - merging changes with main mainMoc
// Previously was: dispatch_async
// dispatch_sync in this place may prevent from overlapping merging in some cases (many blocks in background queue)
dispatch_sync(dispatch_get_main_queue(), ^{
[self.mainMoc mergeChangesFromContextDidSaveNotification:saveNotification];
// !!! Extra notification NO needed anymore
});
}
Rule of thumb: SIMPLIFY and MINIMIZE amount of threads and NSManagedContexts.
I experienced that having 2 contexts is enough even for very big apps:
importContext operating in dedicated to GCD queue (MUST be serial queue). Just remember to save it at the end of the queue block's code.
mainConstext to operate on main UI thread (I call it READ context ;-) to be use when pulling data for the UI (presentation).
The DATABASE_SAVED_WITH_CHANGES notification looks a bit suspicious: Let's say bgMoc saves. Then bgMocDidSave: fires and merges the changes with the mainMoc which is fine. Then you fire a notification which in the end (I assume mainMocDidSave: fires when DATABASE_SAVED_WITH_CHANGES is raised) merges the changes back in bgMoc (which is where is originated from!). This does not sound like the right approach to me.
Also you might want to check in bgMocDidSave: that the notification originates from the bgMoc. If the mainMoc saves then changes are that bgMocDidSave: also fires.
iOS 5 introduced a new way to quickly fetch data on a background thread by initializing the MOC using NSPrivateQueueConcurrencyType and then doing the fetch in performBlock:
One of the rules of thumb of Core Data has been that you can not share a managed object between threads/queues. Is it still the case with performBlock:? Is the following:
[context performBlock:^{
// fetch request code
NSArray *results = [context executeFetchRequest:request error:nil];
dispatch_async(dispatch_get_main_queue(), ^(void) {
Class *firstObject = [results objectAtIndex:0];
// do something with firstObject
});
}];
still unacceptable since I'm sharing my results array/objects between the bg queue and the main queue? Do I still need to use the managed object IDs to do that?
When you use NSPrivateQueueConcurrencyType you need to do anything that touches that context or any object belonging to that context inside the -performBlock: method.
Your code above is illegal since you're passing those objects back to the main queue. The new API helps you in solving this, though: You create one context that's associated with the main queue, i.e. with NSMainQueueConcurrencyType:
// Assume we have these two context (They need to be set up. Assume they are.)
NSManagedObjectContext *mainMOC = [[[NSManagedObjectContext alloc] initWithConcurrencyType:NSMainQueueConcurrencyType] autorelease];
NSManagedObjectContext *backgroundMOC = [[[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType] autorelease];
// Now this can safely be called from ANY thread:
[backgroundMOC performBlock:^{
NSArray *results = [backgroundMOC executeFetchRequest:request error:nil];
for (NSManagedObject *mo in results) {
NSManagedObjectID *moid = [mo objectID];
[mainMOC performBlock:^{
NSManagedObject *mainMO = [mainMOC objectWithID:moid];
// Do stuff with 'mainMO'. Be careful NOT to use 'mo'.
}];
}
}];
This gets less confusing if you move the inner [mainMOC performBlock:] call into its own method. You may also want to pass an array of object IDs back to the main thread's context in stead of executing a block for each object ID. It depends on your needs.
As Daniel Eggert explains, this is definitely still the case. The exception is for NSMainQueueConcurrencyType, where you can also use the managed object context and objects safely on the main thread (as well as from other threads via the performBlock mechanism). The usefulness of this cannot be understated!
iOS 5 also introduced the concept of parent contexts, which also hugely simplify background operations and remove the need to worry about using notifications to propogate changes between threads.
The WWDC 2012 video "Session 214 - Core Data Best Practices" goes into a lot more detail on both subjects and is very comprehensive. The video is essential viewing for anyone using Core Data.
I have perused all related threads on SO but still confused about how to make changes to core data objects from multiple threads without having to save after every change.
I am working on an app that talks to the server constantly. The app uses Core Data for storage and NSFetchedResultsController is used in a few view controllers to fetch data from the persistence store. Usually when the user performs an action, a network request will be triggered. Before the network request is sent, usually some changes should be made to relevant Core Data objects, and upon server response, more changes would be made to those Core Data objects.
Initially all the Core Data operations were done on the main thread in the same NSManagedObjectContext. It was all well except that when network traffic is high, the app can become unresponsive for several seconds. Obviously that's not acceptable so I looked into moving some Core Data operations to run in the background.
The first approach I tried was to create an NSOperation object to process each network response. Inside the main method of the NSOperation object, I set up a dedicated MOC, make some changes, and commit the changes at the end.
- (void)main
{
#try {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
// Create a dedicated MOC for this NSOperation
NSManagedObjectContext * context = [[NSManagedObjectContext alloc] init];
[context setPersistentStoreCoordinator:[APP_DELEGATE persistentStoreCoordinator]];
// Make change to Core Data objects
// ...
// Commit the changes
NSError *error = nil;
if ([context hasChanges] && ![context save:&error]) {
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
}
// Release the MOC
[context release];
// Drain the pool
[pool drain];
}
#catch (NSException *exception) {
// Important that we don't rethrow exception here
NSLog(#"Exception: %#", exception);
}
}
The MOC on the main thread is registered for the NSManagedObjectContextDidSaveNotification.
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(backgroundContextDidSave:)
name:NSManagedObjectContextDidSaveNotification
object:nil];
So when the background context commits changes, the main MOC will be notified and then will merge in the changes:
- (void)backgroundContextDidSave:(NSNotification *)notification
{
// Make sure we're on the main thread when updating the main context
if (![NSThread isMainThread]) {
[self performSelectorOnMainThread:#selector(backgroundContextDidSave:) withObject:notification waitUntilDone:NO];
return;
}
// Merge the changes into the main context
[[self managedObjectContext] mergeChangesFromContextDidSaveNotification:notification];
}
However, as I mentioned earlier, I also need to make changes to Core Data objects from the main MOC. Each change is usually very small (e.g. update one instance variable in an object) but there can be many of them. So I really don't want to save the main MOC after every single change. But if I don't do that, I run into problems when merging changes from background MOC to the main MOC. Merge conflicts occur since both MOCs have unsaved changes. Setting the merge policy doesn't help either, since I want to keep changes from both MOCs.
One possibility is to register the background MOC with the NSManagedObjectContextDidSaveNotification as well, but that approach smells like bad design for me. And I would still need to save the main MOC after every single change.
The second approach I tried was to do all Core Data changes from a dedicated background context running on a permanent background thread.
- (NSThread *)backgroundThread
{
if (backgroundThread_ == nil) {
backgroundThread_ = [[NSThread alloc] initWithTarget:self selector:#selector(backgroundThreadMain) object:nil];
// Actually start the thread
[backgroundThread_ start];
}
return backgroundThread_;
}
// Entry point of the background thread
- (void)backgroundThreadMain
{
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
// We can't run the runloop unless it has an associated input source or a timer, so we'll just create a timer that will never fire.
[NSTimer scheduledTimerWithTimeInterval:DBL_MAX target:self selector:#selector(ignore) userInfo:nil repeats:YES];
[[NSRunLoop currentRunLoop] run];
// Create a dedicated NSManagedObjectContext for this thread.
backgroundContext_ = [[NSManagedObjectContext alloc] init];
[backgroundContext_ setPersistentStoreCoordinator:[self persistentStoreCoordinator]];
[pool drain];
}
So whenever I need to make Core Data changes from the main thread, I have to get the objectID from the main thread, and pass to the background thread to perform the change. When the background context saves, the changes will then be merged back to the main MOC.
- (void)addProduct:(Product *)product toCatalog:(Catalog *)catalog;
would change to:
- (void)addProduct:(NSManagedObjectID *)productObjectId toCatalog:(NSManagedObjectID *)catalogObjectId
{
NSArray * params = [NSArray dictionaryWithObjects:productObjectId, catalogObjectId, nil];
[self performSelector:(addProductToCatalogInBackground:) onThread:backgroundThread_ withObject:params waitUntilDone:NO];
}
But this seems so convoluted and ugly. Writing code like this seems to negate the usefulness of using Core Data in the first place. Also, I would still have to save the MOC after every single change since I can't get objectId for a new object without saving it to the datastore first.
I feel that I am missing something here. I really hope someone can shed some light on this. Thanks.
A NSManagedObjectContext is just a scratch pad. The act of saving it moves the changes in one scratch pad down to the NSPersistentStoreCoordinator and potentially down to disk. The only way one MOC can know about the changes from another MOC is through the NSPersistentStoreCoordinator. Therefore the save is required. However, saving is a LOT less expensive in the next release of iOS.
If you must be iOS4 or less compliant then saving is the only option. However you can load up the saves and do them less frequently depending on the design of your application. If you are importing data then save when the import is complete or in logical units inside of the import. There is no need to save after every entry, that is wasteful.
BTW, I would suggest using NSOperation instances instead of working with NSThread instances directly. They are easier to work with and will perform better.
Also, you do not need to wrap Objective-C code in try/catch blocks. Very few things will throw an exception; especially on iOS.
Lastly, I would suggest taking a look at my post on CIMGF about importing on a background thread.