So I need to use cora-data managed object context using multiple thread.
I found the below implementation to be pretty much what I need.
I have 2 question :
Where do I release the MOC that I allocate ? I don't want to go through every function thread in my app and to call a function to release the MOC.
I'd rather write the function, and somehow make it so it be called by every thread prior to his exit. is that possible ?
Is the below implementation enough for core data concurrency ?
-(void) StoreManagedObjectContextForCurrentThread:(NSManagedObjectContext*) context
{
[[[NSThread currentThread] threadDictionary] setObject: context forKey: AQPerThreadManagedObjectContext];
}
-(NSManagedObjectContext *) myManagedObjectContext {
NSManagedObjectContext * result = [[[NSThread currentThread] threadDictionary] objectForKey: AQPerThreadManagedObjectContext];
if ( result != nil )
return result;
NSPersistentStoreCoordinator *coordinator = [self persistentStoreCoordinator];
if (coordinator!=nil)
{
NSManagedObjectContext * moc = [[NSManagedObjectContext alloc] init];
[moc setMergePolicy: NSMergeByPropertyObjectTrumpMergePolicy];
[moc setPersistentStoreCoordinator: coordinator];
[self StoreManagedObjectContextForCurrentThread:moc];
[moc release]; // now owned by the thread dictionary
return moc;
}
return nil;
}
In the implementation you've provided, you haven't got a way to release the managed object context before the thread disappears, because its lifetime is dictated by the thread dictionary, and that object's lifetime is handled by Foundation, not by you.
If you want to manage the objects' lifetimes, you need to manage their storage. That means moving away from using -threadDictionary. One option is to implement your background work as NSOperation subclasses and keep the managed object contexts around for the lifetime of the operations, but that's likely a big change from what you currently have.
Related
Background info
I've almost completed my app. Everything was working perfectly. Then the client asked for logging in the app (i.e. various points that had to record what was done, what responses were, etc...).
The app allows the user to create "messages" which are saved into core-data. The messages are then uploaded to the server individually. The message are created on the main thread and uploaded in an NSOperation subclass on a background thread.
It is the same template for the NSOperation subclass that I have used before and works. I'm doing all the best practise stuff for multi-threaded core-data.
All this side of the app works fine.
I added the logging part of the app. I've created a singleton called MyLogManager and a CoreData entity called LogEntry. The entity is very simple, it only has a date and text.
Code
The function inside the MyLogManager is...
- (void)newLogWithText:(NSString*)text
{
NSLog(#"Logging: %#", text);
NSManagedObjectContext *context = [self context];
NSEntityDescription *entity = [NSEntityDescription entityForName:#"LogEntry" inManagedObjectContext:context];
LogEntry *logEntry = [[LogEntry alloc] initWithEntity:entity insertIntoManagedObjectContext:context];
logEntry.text = text;
logEntry.date = [NSDate date];
[self saveContext:context];
}
which in turn runs...
- (NSManagedObjectContext*)context
{
AppDelegate *appDelegate = (ThapAppDelegate*)[[UIApplication sharedApplication] delegate];
NSManagedObjectContext *managedObjectContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSConfinementConcurrencyType];
[managedObjectContext setPersistentStoreCoordinator:appDelegate.persistentStoreCoordinator];
return managedObjectContext;
}
and
- (void)saveContext:(NSManagedObjectContext*)context
{
MyAppDelegate *appDelegate = (MyAppDelegate*)[[UIApplication sharedApplication] delegate];
[[NSNotificationCenter defaultCenter] addObserver:appDelegate
selector:#selector(mergeChanges:)
name:NSManagedObjectContextDidSaveNotification
object:context];
NSError *error = nil;
if ([context hasChanges] && ![context save:&error]) {
NSLog(#"Unhandled error %#, %#", error, [error userInfo]);
abort();
}
[[NSNotificationCenter defaultCenter] removeObserver:appDelegate name:NSManagedObjectContextDidSaveNotification object:context];
}
The NSOperation main thread (well parts of it)...
- (void)main
{
//create context and retrieve NSManagedObject using the NSManagedObjectID passed in as a parameter to operation
self.message.lastSendAttempt = [NSDate date];
[self startUpload];
[self completeOperation]; //This doesn't get run because the startUpload method never returns
}
- (void)startUpload
{
[[MyLogManager sharedInstance] logSendingMessageWithURLParameters:[self.event URLParameters]]; //this is a convenience method. It just appends some extra info on the string and runs newLogWithText.
//Do some uploading stuff here...
//The operation stops before actually doing the upload when logging to CoreData.
}
The problem
My NSOperation subclass that uploads the messages (on a background thread) calls this newLogWithText function but it also updates the message it is uploading. The NSOperation uses the same methods to get and save the core-data context. (i.e. it updates the last sent date and also updates if the send was successful).
This is the first time I've tried to deal with simultaneous writes and saves to core-data.
I don't get any errors and the app carries on "working". But the operation never completes. I've tried to debug it with breakpoints but when I use breakpoints it works. Without breakpoints the operation never finishes and the upload never happens. And then it just sits there blocking the queue it is on and no other messages can be sent.
In my appDelegate (I know this isn't the ideal place for it but it's the default for a new project and I haven't changed it) the mergeChanges method is just...
- (void)mergeChanges:(NSNotification *)notification
{
[self.managedObjectContext performSelectorOnMainThread:#selector(mergeChangesFromContextDidSaveNotification:) withObject:notification waitUntilDone:NO];
}
I've tried throwing the 'newLogWithText' function off to another thread and even to the main thread with no luck.
I'm just about to try it now but change the "waitUntilDone" of the merge to YES. (Just noticed this). (This didn't work).
I'm 90% certain this is down to simultaneous writes to different contexts and the conflict resolution as it is the first time I've dealt with this. If I comment out the newLogWithText function then everything works as it should.
The only alternative at the moment is to scrap the LogEntry from core data and save the logs into an array inside NSUserDefaults but that doesn't feel right. Is there another way?
EDIT
I've changed it now so it users NSUserDefaults and it works without a problem. It just feels like a hacky solution.
I've spent days trying every possible solution I can think of to this problem, but nothing seems to be working.
I run a background thread like this:
[MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext) {
Media *localMedia = [media inContext:localContext];
UIImage *image = localMedia.thumbnail;
dispatch_async(dispatch_get_main_queue(), ^{
[thumbnails setObject:image forKey:[NSNumber numberWithInt:i]];
[contentDict setObject:thumbnails forKey:#"MediaItems"];
[cell.entryView setNeedsDisplay];
});
}];
Or like this:
dispatch_queue_t cellSetupQueue = dispatch_queue_create("com.Journalized.SetupTimelineThumbnails", NULL);
dispatch_async(cellSetupQueue, ^{
NSManagedObjectContext *newMoc = [[NSManagedObjectContext alloc] init];
NSPersistentStoreCoordinator *coordinator = [NSManagedObjectContext contextForCurrentThread].persistentStoreCoordinator;
[newMoc setPersistentStoreCoordinator:coordinator];
NSNotificationCenter *notify = [NSNotificationCenter defaultCenter];
[notify addObserver:self
selector:#selector(mergeChanges:)
name:NSManagedObjectContextDidSaveNotification
object:newMoc];
Media *localMedia = [media inContext:[NSManagedObjectContext contextForCurrentThread];
UIImage *image = localMedia.thumbnail;
dispatch_async(dispatch_get_main_queue(), ^{
[thumbnails setObject:image forKey:[NSNumber numberWithInt:i]];
[contentDict setObject:thumbnails forKey:#"MediaItems"];
[cell.entryView setNeedsDisplay];
});
}];
Both of these give me a crash with UIImage returning as nil object, and a Cocoa Error 133000.
I've removed every other piece of background threading code, and have saved on the main thread directly before this just to make sure. Running the code above on the main thread also works, but freezes up my UI. Despite all of these efforts, the above code crashes every time.
I'm not sure what to do to make this work.
Edit:
My question, specifically, is how do I make this work without crashing? It seems to be a problem with objects from 1 context not existing in another, but how do i make them exist in another context?
Remember, the MR_inContext: method is using [NSManagedObjectContext objectWithID: ] method under the covers. You should look in there to make sure your object has:
1) Been saved prior to entering into the background context/block in your first code block
2) This method is returning something useful
I'm also not sure how you set up your thumbnail attribute. Ideally it shouldn't matter as long as you have the NSTransformable code write (there are samples on the internets that show you how to save a UIImage in core data using the transformable attribute)
Also, your code should look like this:
UIImage *image = nil;
[MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext) {
Media *localMedia = [media inContext:localContext]; //remember, this looks up an existing object. If your context is a child of a parent context that has contains this, the localContext should refresh the object from the parent.
//image processing/etc
image = localMedia.thumbnail;
} completion:^{
[thumbnails setObject:image forKey:#(i)]; //new literals are teh awesome
contentInfo[#"MediaItems"] = thumbnails; //as is the new indexer syntax (on the latest compiler)
[cell.entryView setNeedsDisplay];
}];
Fast answer:
NSManagedObjectReferentialIntegrityError = 133000,
// attempt to fire a fault pointing to an object that does not exist (we can see the store, we can't see the object)
EDIT:
It's pretty difficult to see something from the code. What is a managed object there?
I suppose the problem is that you are using temporary objects from one context in another context.
I have an iPhone app that is using sqlite 3.6 (not with FMDB) to store and load data. I load the database when the app loads and uses the same database connection through the whole app.
In a background thread the app downloads some data from a webserver and writes to the database. At the same time the main thread also might need to write to the same database. This sometimes leads to EXC_BAD_ACCESS as both threads are trying to access the database.
What is the best and easiest way to be able to use the database from different threads?
This is an example that shows the problem:
sqlite3 *database;
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSString *path = [documentsDirectory stringByAppendingPathComponent:#"database.db"];
if (sqlite3_open([path UTF8String], &database) != SQLITE_OK) {
sqlite3_close(database);
return YES;
}
[NSThread detachNewThreadSelector:#selector(test) toTarget:self withObject:nil];
[self test];
return YES;
}
-(void)test {
for (int i = 0; i < 2000; i++) {
NSLog(#"%i",i);
sqlite3_exec([self getDb],"UPDATE mytable SET test=''", 0, 0, 0);
}
}
EDIT:
After willcodejavaforfood's answer below I've tried to change my code to use a separate database object (connection) for each separate thread and also added sqlite3_busy_timeout() so that sqlite will retry to write if the database is busy. Now I don't get EXC_BAD_ACCESS anymore but I've noticed that not all data get inserted. So this is not a stable solution either. It seems to be really hard to get sqlite working with threading..
My new solution with separate connections:
-(void)test {
sqlite3 *db = [self getNewDb];
for (int i = 0; i < 2000; i++) {
NSLog(#"%i",i);
sqlite3_exec(db,"UPDATE mytable SET test=''", 0, 0, 0);
}
}
- (sqlite3 *)getNewDb {
sqlite3 *newDb = nil;
if (sqlite3_open([[self getDbPath] UTF8String], &newDb) == SQLITE_OK) {
sqlite3_busy_timeout(newDb, 1000);
} else {
sqlite3_close(newDb);
}
return newDb;
}
I solved this problem by using one thread and an NSOperationQueue to insert the Data. I would give it some thought. I've never been able to get a stable System with mutliple threads, and most writes aren't that important that queuing really helps.
As per request, some more Infos:
I have a subclass of NSOperation that I instantiate with the model object I want to store.
These operations are than submitted to an extension of NSOperationsQueue that runs in a seperate thread. This custom Queue just adds a pointer to the database instance. When the operation is executed, it uses the [NSOperationsQueue currentQueue] property to access the queue and than the database. On purpose, i used non-concurrent operations (maxOperations was set to 1)
Hence, only one query (or update) is executed at a time consecutivly, completely in the background.
Obviously you need some kind of callback after you're finished.
It is possibly not the fast, but the most stable and cleanest solution i could find.
Docs:
http://developer.apple.com/library/ios/documentation/General/Conceptual/ConcurrencyProgrammingGuide/OperationObjects/OperationObjects.html
http://www.cimgf.com/2008/02/16/cocoa-tutorial-nsoperation-and-nsoperationqueue/
http://icodeblog.com/2010/03/04/iphone-coding-turbo-charging-your-apps-with-nsoperation/
As you've noticed only one thread can access an sqlite database at a time. Options to prevent simultaneous access:
Create a new database connection in each thread and rely on file locking (costly).
Turn on sqlite3_config(SQLITE_CONFIG_SERIALIZED).
Use NSLock's.
Use GCD (Grand Central Dispatch) queue's.
The first three options may cause busy waiting (one thread waiting on another to release the lock) which is wasteful.
I use option 4 because it simplifies the task of creating new queries to run in the background and has no busy waiting. It also makes sure all queries execute in the order they were added (which my code tends to assume).
dispatch_queue_t _queue = dispatch_queue_create("com.mycompany.myqueue", DISPATCH_QUEUE_SERIAL);
// Run a query in the background.
dispatch_async(_queue, ^{
...some query
// Perhaps call a completion block on the main thread when done?
dispatch_async(dispatch_get_main_queue(), ^{
//completion(results, error);
});
});
// Run a query and wait for the result.
// This will block until all previous queries have finished.
// Note that you shouldn't do this in production code but it may
// be useful to retrofit old (blocking) code.
__block NSArray *results;
dispatch_sync(_queue, ^{
results = ...
});
...use the results
dispatch_release(_queue);
In a perfect world sqlite would let you perform simultaneous reads but only one write at a time (eg. like using dispatch_barrier_async() for writes and dispatch_async() for reads).
This is all explained in the Core Data Programming Guide in the section for Concurrency.
The pattern recommended for concurrent
programming with Core Data is thread
confinement.
You should give each thread its own
entirely private managed object
context and keep their associated
object graphs separated on a
per-thread basis.
There are two possible ways to adopt
the pattern:
Create a separate managed object
context for each thread and share a
single persistent store coordinator.
This is the typically-recommended
approach.
Create a separate managed object
context and persistent store
coordinator for each thread. This
approach provides for greater
concurrency at the expense of greater
complexity (particularly if you need
to communicate changes between
different contexts) and increased
memory usage.
I've tried these two solutions and they worked perfectly. You can either use critical sections or NSOperationQueue and I prefer the first one, here is the code for both of them:
define some class "DatabaseController" and add this code to its implementation:
static NSString * DatabaseLock = nil;
+ (void)initialize {
[super initialize];
DatabaseLock = [[NSString alloc] initWithString:#"Database-Lock"];
}
+ (NSString *)databaseLock {
return DatabaseLock;
}
- (void)writeToDatabase1 {
#synchronized ([DatabaseController databaseLock]) {
// Code that writes to an sqlite3 database goes here...
}
}
- (void)writeToDatabase2 {
#synchronized ([DatabaseController databaseLock]) {
// Code that writes to an sqlite3 database goes here...
}
}
OR to use the NSOperationQueue you can use:
static NSOperationQueue * DatabaseQueue = nil;
+ (void)initialize {
[super initialize];
DatabaseQueue = [[NSOperationQueue alloc] init];
[DatabaseQueue setMaxConcurrentOperationCount:1];
}
+ (NSOperationQueue *)databaseQueue {
return DatabaseQueue;
}
- (void)writeToDatabase {
NSInvocationOperation * operation = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(FUNCTION_THAT_WRITES_TO_DATABASE) object:nil];
[operation setQueuePriority:NSOperationQueuePriorityHigh];
[[DatabaseController databaseQueue] addOperations:[NSArray arrayWithObject:operation] waitUntilFinished:YES];
[operation release];
}
these two solutions block the current thread until the writing to database is finished which you may consider in most of the cases.
I've a multi-threading application in which each thread has to do some job, but at a certain point some code needs to be executed serially (like writing into sqlite3 database), so I'm calling that code to be performed on main thread using:
[self performSelectorOnMainThread:#selector(serialJob:) withObject:object waitUntilDone:YES];
and every thing went just fine except that when that code needs some time the user interaction with the application gets disabled until that code has been finished, so is there any way to make another ONE thread that can be run on background and can be called whenever I need it just like the main one so I can replace the previous call with:
[self performSelector:#selector(serialJob:) onThread:REQUIRED_THREAD withObject:object waitUntilDone:YES];
this thread should be some class's static data member to be accessed from all over the code.
any help would be very appreciated, and many thanks in advance...
This is quite easy to do, just spawn your thread and let it run it's runloop using [[NSRunLoop currentRunLoop] run]. That's all that is required to be able to use performSelector:onThread: with a custom thread.
If you are on iOS 4 or newer you should consider using Grand Central Dispatch queues instead of threads though. The GCD APIs are much easier to use and can utilize the system resources much better.
Like Sven mentioned, look into Grand Central Dispatch.
You can create a queue like this:
dispatch_queue_t myQueue = dispatch_queue_create("com.yourcompany.myDataQueue", NULL);
Now you can call blocks on that queue:
dispatch_async(myQueue, ^{
// Your code to write to DB.
});
When you're done, don't forget to release the queue:
dispatch_release(myQueue);
Due to the my question that I need the current thread to be blocked until the database job has been finished, I've tried these two solutions and they worked perfectly. You can either use critical sections or NSOperationQueue and I prefer the first one, here is the code for both of them:
define some class "DatabaseController" and add this code to its implementation:
static NSString * DatabaseLock = nil;
+ (void)initialize {
[super initialize];
DatabaseLock = [[NSString alloc] initWithString:#"Database-Lock"];
}
+ (NSString *)databaseLock {
return DatabaseLock;
}
- (void)writeToDatabase1 {
#synchronized ([DatabaseController databaseLock]) {
// Code that writes to an sqlite3 database goes here...
}
}
- (void)writeToDatabase2 {
#synchronized ([DatabaseController databaseLock]) {
// Code that writes to an sqlite3 database goes here...
}
}
OR to use the NSOperationQueue you can use:
static NSOperationQueue * DatabaseQueue = nil;
+ (void)initialize {
[super initialize];
DatabaseQueue = [[NSOperationQueue alloc] init];
[DatabaseQueue setMaxConcurrentOperationCount:1];
}
+ (NSOperationQueue *)databaseQueue {
return DatabaseQueue;
}
- (void)writeToDatabase {
NSInvocationOperation * operation = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(FUNCTION_THAT_WRITES_TO_DATABASE) object:nil];
[operation setQueuePriority:NSOperationQueuePriorityHigh];
[[DatabaseController databaseQueue] addOperations:[NSArray arrayWithObject:operation] waitUntilFinished:YES];
[operation release];
}
these two solutions block the current thread until the writing to database is finished which you may consider in most of the cases.
Why is it that everytime I call save on my NSManagedObjectContext:
-(NSManagedObjectContext*)managedObjectContext {
NSMutableDictionary* threadDictionary = [[NSThread currentThread] threadDictionary];
NSManagedObjectContext* backgroundThreadContext = [threadDictionary objectForKey:RKManagedObjectStoreThreadDictionaryContextKey];
if (!backgroundThreadContext) {
backgroundThreadContext = [self newManagedObjectContext];
[threadDictionary setObject:backgroundThreadContext forKey:RKManagedObjectStoreThreadDictionaryContextKey];
[backgroundThreadContext release];
[[NSNotificationCenter defaultCenter] addObserver:self
selector:#selector(mergeChanges:)
name:NSManagedObjectContextDidSaveNotification
object:backgroundThreadContext];
}
return backgroundThreadContext;
}
- (NSError*)save {
NSManagedObjectContext* moc = [self managedObjectContext];
NSError *error = nil;
#try {
if (![moc save:&error]) { //breakpoint in here
//some code
}
the app seems to be waiting forever and never got back to resume it's execution? Here's what I mean in a video. Can this be possibly caused because something is wrong with the entity/relationship model?
Here's a screenshot of the leaks instruments, I don't see any leaks, but it seems that the app is allocating something that builds up:
Have you tried ditching your multi-threading code to see if it works? My guess would be that you're mixing up threads here and accessing/saving the MOC from different threads. Managing threads manually is a PITA, you should try switching to Grand Central Dispatch.
I would also give your main MOC its own accessor so you can make sure it isn't called from background threads, and have some: - (NSManagedObjectContext*)newBackgroundMOC; and - (void)saveBackgroundMOC:(NSManagedObjectContext*)context; methods to quickly create and save MOCs from background queues/threads:
dispatch_async(my_queue, ^{
NSManagedObjectContext *context = [self newBackgroundMOC]; // create context, setup didSave notification to merge with main MOC, etc
// modify context
[self saveBackgroundMOC:context]; // main MOC gets updated
});
Migrating to GCD is a bit of work, but in the long run you'll see it's much more pleasant to work with. It goes without saying that it's also the most modern and recommended by Apple way to deal with threads.