I was excited about the newly supported concurrent functions of CoreData since iOS 5.
A private queue is maintained and all save or fetch requests can be done via that queue.
However, can I set up the private queue for CoreData so that it executes request one by one?
My app is downloading news items from a number of feeds. Each time after downloading and parsing from one feed are finished, I just save the feed's items into CoreData via the private queue.
However, since I am downloading and parsing from multiple feeds simultaneously, I always have multiple groups of items, i.e., multiple save requests, for the CoreData.
Now the situation is that I guess CoreData just have a number of threads and each one is saving a group of items into the db. My UI got stuck in the mean time.
Do you think I can control the private queue so that no matter how many simultaneous save requests are, they will be done one by one?
Core Data is (probably) only using one serial queue or thread since its serial. I recently converted my app from using a serial queue I had created (app was 4.3) to use this new option in iOS 5. In all cases when you 'performBlock' the method is handled in a serial fashion. Also, you can now call '[moc performBlocK:...]' from any queue as that call is thread safe!
I believe what you want to do is have your background threads, which are most likely adding options, to use 'performBlock:' (without the wait). The block you provide is then queued and processed in a FIFO fashion. Later on, if your table wants to get objects, it can issue a 'performBlockAndWait:', or optionally your code can ask for the latest objects using performBlock, and at the end of the supplied block message back to your app the set of objects you need.
Also, I only ever save often in development builds, to verify validity. Once you are pretty sure things are working OK, you can then just perform a background save once all the data is downloaded.
EIDT: To reiterate - if you are downloading and also using images or other data while loading a viewController, and lots of things are going on, this is the WORST time to do a save. Use a timer or dispatch_after, and many seconds after everything seems stable THEN do the save.
Related
I am working on a chat client. To get new messages (or post new one) I have to perform GET (or POST) request. All new messages are stored via core data. At the moment I don't know how to implement it in most optimal way.
My thoughts:
On view controller init stage create background thread which will periodically checks for new messages (if conversation is active - with short period, if not - with period about 60 secs). If there are new messages, we store them in DB and signal delegate that there are new messages to display.
Friend suggested to use performSelector afterDelay, but I don't understand how to use it in my app.
Something else?
Thanks in advance.
Don't use performSelector afterDelay. Using NSTimer is much better (as the trigger for starting the next download). Also, use NSOperationQueue to manage your background tasks. Create yourself a custom NSOperation that you can instantiate and it will complete your request process. When you create a new operation to check for new messages, check if one is already in progress (there is no point having multiple requests in progress at the same time).
Other notes:
Make sure you consider the threading with regards to the Core Data store (having the operation call back to the main thread with the results will probably be easiest as the result data will always be relatively small).
If you have lots of messages being sent and you want to show constant status (like Skype does, showing you when someone is typing) you would need to use sockets to keep the connection alive the whole time (the cost of new connections each time would be prohibitive).
I am new to objective-c/cocoa programming. I am making an application which is to constantly sync with a server and keep its view updated.
Now in a nutshell, heres what I thought of: Initiate an NSTimer to trigger every second or two, contact the server, if there is a change, update the view. Is this a good way of doing it?
I have read elsewhere that you can have a thread running in the background which monitors the changes and updates the view. I never worked with threads before and I know they can be quite troublesome and you need a good amount of experience with memory management to get most out of them.
I have one month to get this application done. What do you guys recommend? Just use an NSTimer and do it the way I though of...or learn multithreading and get it done that way (but keep in mind my time frame).
Thanks!
I think using separate thread in this case would be too much. You need to use threads when there is some task that runs for considerable amount of time and can freeze your app for some time.
In your case do this:
Create timer and call some method (say update) every N seconds.
in update send asynchronous request to server and check for any changes.
download data using NSURLConnection delegate and parse. Note: if there is probability that you can receive a huge amount of data from server and its processing can take much time (for example parsing of 2Mb of XML data) then you do need to perform that is a separate thread.
update all listeners (appropriate view controllers for example) with processed data.
continue polling using timer.
Think about requirements. The most relevant questions, IMO, are :
does your application have to get new data while running in background?
does your application need to be responsive, that is, not sluggish when it's fetching new data?
I guess the answer to the first question is probably no. If you are updating a view depending on the data, it's only required to fetch the data when the view is visible. You cannot guarantee always fetching data in background anyway, because iOS can always just kill your application. Anyway, in your application's perspective, multithreading is not relevant to this question. Because either you are updating only in foreground or also in background, your application need no more than one thread.
Multithreading is relevant rather to the second question. If your application has to remain responsive while fetching data, then you will have to run your fetching code on a detached thread. What's more important here is, the update on the user interface (like views) must happen on the main thread again.
Learning multithreading in general is something indeed, but iOS SDK provides a lot of help. Learning how to use operation queue (I guess that's the easiest to learn, but not necessarily the easiest to use) wouldn't take many days. In a month period, you can definitely finish the job.
Again, however, think clearly why you would need multithreading.
I'm working on a Core Data-based application that has a Mac application acting as a 'server' and an iPhone as a client. Everything is going swimmingly, except I'm running into performance issues.
When the user taps an object, the server must return some objects related to that object (nothing too heavy, usually 3-4 objects) and show a UI to choose some options. This needs to be as fast as possible. The round-trip time to the server, the server pulling the data, formatting it, returning it to the client, and the client creating NSManagedObjects from the data (which cannot be optimized further) is about 200 ms. The code relating to presenting the UI (which cannot be optimized further, again) requires around 150 ms. On an iPod touch 2G running iOS 4.0, the single line of code saving the managed object context after the objects are imported is taking anywhere from 150-200 ms.
To me, this screams that I should be backgrounding the managed object context saving. However, as far as I understand it, that won't really meet my needs. If I want to save the managed object context on a background thread, then all the objects in it must have been created on a background thread in a separate managed object context, so I won't see any speed gain because it will still take 100-200 ms for the save to occur, and I'll be seeing even more overhead because I'll still need to tell my main thread to update it's managed objects from the backgrounded managed object context's save before my view controller sees that it needs to refresh itself.
Am I missing an obvious solution? Is there something about Core Data I could use in this situation that would help? I hate to throw such a general question like this out there, but I'm at a complete loss where to go from here.
Sounds like you need to move the entire server communication to a background thread. If you did that then the entire UI would be responsive no matter how long the communication with the server took.
To do this, you stand up a second NSManagedObjectContext on the background thread connected to the same NSPersistentStoreCoordinator. Then you perform your server communication on that background thread (it might even make sense to use an NSOperation) and save the changes.
Your main thread and therefore main NSManagedObjectContext listens for save notifications and when it receives one it updates the main thread and UI. This will eliminate any freezing you are seeing and the processing time becomes mostly irrelevant.
Hey, I'm working on the model layer for our app here.
Some of the requirements are like this:
It should work on iPhone OS 3.0+.
The source of our data is a RESTful Rails application.
We should cache the data locally using Core Data.
The client code (our UI controllers) should have as little knowledge about any network stuff as possible and should query/update the model with the Core Data API.
I've checked out the WWDC10 Session 117 on Building a Server-driven User Experience, spent some time checking out the Objective Resource, Core Resource, and RestfulCoreData frameworks.
The Objective Resource framework doesn't talk to Core Data on its own and is merely a REST client implementation. The Core Resource and RestfulCoreData all assume you talk to Core Data in your code and they solve all the nuts and bolts in the background on the model layer.
All looks okay so far and initially I though either Core Resource or RestfulCoreData will cover all of the above requirements, but... There's a couple of things none of them seemingly happen to solve correctly:
The main thread should not be blocked while saving local updates to the server.
If the saving operation fails the error should be propagated to the UI and no changes should be saved to the local Core Data storage.
Core Resource happens to issue all of its requests to the server when you call - (BOOL)save:(NSError **)error on your Managed Object Context and therefore is able to provide a correct NSError instance of the underlying requests to the server fail somehow. But it blocks the calling thread until the save operation finishes. FAIL.
RestfulCoreData keeps your -save: calls intact and doesn't introduce any additional waiting time for the client thread. It merely watches out for the NSManagedObjectContextDidSaveNotification and then issues the corresponding requests to the server in the notification handler. But this way the -save: call always completes successfully (well, given Core Data is okay with the saved changes) and the client code that actually called it has no way to know the save might have failed to propagate to the server because of some 404 or 421 or whatever server-side error occurred. And even more, the local storage becomes to have the data updated, but the server never knows about the changes. FAIL.
So, I'm looking for a possible solution / common practices in dealing with all these problems:
I don't want the calling thread to block on each -save: call while the network requests happen.
I want to somehow get notifications in the UI that some sync operation went wrong.
I want the actual Core Data save fail as well if the server requests fail.
Any ideas?
You should really take a look at RestKit (http://restkit.org) for this use case. It is designed to solve the problems of modeling and syncing remote JSON resources to a local Core Data backed cache. It supports an offline mode for working entirely from the cache when there is no network available. All syncing occurs on a background thread (network access, payload parsing, and managed object context merging) and there is a rich set of delegate methods so you can tell what is going on.
There are three basic components:
The UI Action and persisting the change to CoreData
Persisting that change up to the server
Refreshing the UI with the response of the server
An NSOperation + NSOperationQueue will help keep the network requests orderly. A delegate protocol will help your UI classes understand what state the network requests are in, something like:
#protocol NetworkOperationDelegate
- (void)operation:(NSOperation *)op willSendRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity;
- (void)operation:(NSOperation *)op didSuccessfullySendRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity;
- (void)operation:(NSOperation *)op encounteredAnError:(NSError *)error afterSendingRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity;
#end
The protocol format will of course depend on your specific use case but essentially what you're creating is a mechanism by which changes can be "pushed" up to your server.
Next there's the UI loop to consider, to keep your code clean it would be nice to call save: and have the changes automatically pushed up to the server. You can use NSManagedObjectContextDidSave notifications for this.
- (void)managedObjectContextDidSave:(NSNotification *)saveNotification {
NSArray *inserted = [[saveNotification userInfo] valueForKey:NSInsertedObjects];
for (NSManagedObject *obj in inserted) {
//create a new NSOperation for this entity which will invoke the appropraite rest api
//add to operation queue
}
//do the same thing for deleted and updated objects
}
The computational overhead for inserting the network operations should be rather low, however if it creates a noticeable lag on the UI you could simply grab the entity ids out of the save notification and create the operations on a background thread.
If your REST API supports batching, you could even send the entire array across at once and then notify you UI that multiple entities were synchronized.
The only issue I foresee, and for which there is no "real" solution is that the user will not want to wait for their changes to be pushed to the server to be allowed to make more changes. The only good paradigm I have come across is that you allow the user to keep editing objects, and batch their edits together when appropriate, i.e. you do not push on every save notification.
This becomes a sync problem and not one easy to solve. Here's what I'd do: In your iPhone UI use one context and then using another context (and another thread) download the data from your web service. Once it's all there go through the sync/importing processes recommended below and then refresh your UI after everything has imported properly. If things go bad while accessing the network, just roll back the changes in the non UI context. It's a bunch of work, but I think it's the best way to approach it.
Core Data: Efficiently Importing Data
Core Data: Change Management
Core Data: Multi-Threading with Core Data
You need a callback function that's going to run on the other thread (the one where actual server interaction happens) and then put the result code/error info a semi-global data which will be periodically checked by UI thread. Make sure that the wirting of the number that serves as the flag is atomic or you are going to have a race condition - say if your error response is 32 bytes you need an int (whihc should have atomic acces) and then you keep that int in the off/false/not-ready state till your larger data block has been written and only then write "true" to flip the switch so to speak.
For the correlated saving on the client side you have to either just keep that data and not save it till you get OK from the server of make sure that you have a kinnf of rollback option - say a way to delete is server failed.
Beware that it's never going to be 100% safe unless you do full 2-phase commit procedure (client save or delete can fail after the signal from the server server) but that's going to cost you 2 trips to the server at the very least (might cost you 4 if your sole rollback option is delete).
Ideally, you'd do the whole blocking version of the operation on a separate thread but you'd need 4.0 for that.
What's the preferred approach for constantly sharing data across threads when using Core Data? I am downloading a large file and want to show progress of the download in a UIProgressBar. The actual download is happening in a background thread created by NSOperation.
The download information (local path, total bytes, bytes received) is modeled as a Core Data managed object, and the actual file is stored in the Documents/ directory. One solution that came to my mind was to create a separate managed object context in the background thread and pass it the objectID and pull it up using the objectWithID: method. Whenever the background thread does a save, the main thread gets a notification and the main context merges those changes and the table view is subsequently updated.
This approach works, but the save can't be done too frequently or the UI freezes up. So the UI is updated after every X KB's of data is received where X has to be at least 500 KB for the UI to be somewhat responsive. Is there a better approach to pass the download progress data to the main thread as it is received?
EDIT: Would using KVO be of any help? If yes, do you know of any good tutorials on the topic?
I know you already built your own system, but I use ASIHTTPRequest for all my network operations. It is very robust and has tons of goodies like file resuming, saving directly to disk, upload progress monitoring, download progress monitoring, and the kitchen sink. If you dont use it, you can look at the source to see how they do it because the UI never freezes when I use the progress reporting in this framework.
Although I am going to use ASIHTTPRequest for my project, it's still good to mention my solution to the problem for completeness. It is kind of obvious, but saving the core data context as frequently as every couple of seconds is a terrible mistake.
Instead, I added a progress delegate to the download operation, which gets update notification on the main thread.
NSNumber bytesDownloaded = [NSNumber numberWithLongLong:[data length]];
[downloadDelegate performSelectorOnMainThread:#selector(updateProgress:) withObject:bytesDownloaded waitUntilDone:NO];
The important thing was to pass the download progress information to the delegate on the main thread. The delegate updates the progress, keeps accumulating changes and either saves when the download completes or at much bigger intervals.