I have an app that currently holds all state in memory. It fetches a bunch of information from a server as JSON and then holds on to the JSON values in memory. Each JSONObject can be ~300 bytes and there can be thousands of such objects.
I use this data simply to populate UITableView.
In order to better handle large amounts of aata, I modified my code to fetch data from the server and store it using CoreData. There JSON objects can be represented as simple entities, each with 3 NSString attributes and one 1 int32 attribute. I created a NSFetchedResultsController to use as the data source of the UITableView. My assumption was that this would reduce the resident memory usage of my application (I assume NSFetchedResults controllers effectively manages memory to not hold entities that aren't being displayed in the view, vs holding all my state in-memory).
For the purposes of this discussion, let's assume my app purges the CoreData store and re-fetches all data each time it runs.
When I went to measure the changes in Resident Memory and Virtual Size using the VM Tracker in Instruments, I noticed that both these values remain almost identical. Infact, the Core-Data based version of my app seems to use more memory than when I have everything entirely in-memory.
While this may be true, I don't have an intuition for why this might be so. Any explanations?
From what I have said about my app, does it sound like I don't want to bother persisting in CoreData, and why?
Core Data can use more memory as you have seen, if there is memory to be had. However, one of the benefits to Core Data is when you get into a low memory situation. When that happens then Core Data will automatically reduce its own memory footprint as much as possible.
Another thing to consider is are you letting Core Data fault these objects? If you are pulling in 1000 objects and displaying 10 of them, then the other 990 should be in a faulted state and therefore taking up less memory.
I would run through the Core Data instruments and make sure you are not fulling realizing all of these objects by accident and unintentionally causing your memory usage to be higher than it needs to be.
Update
Sounds like you are importing this data and then not flushing Core Data properly if you are not seeing any faulting going on.
Assuming you are loading this data on first launch (which I would not do btw, I would pre-load the app and avoid the plist files entirely), you want to call -reset on your NSManagedObjectContext after the load is complete so that any objects that are not being used are flushed out of memory. Then as the data is coming back into memory (on use) it will be properly faulted.
Lastly, make sure you are using a SQLite store. Otherwise this is all moot.
Related
Problem: I'm implementing aplication where user is dowloading lots of objects in xml. To reduce peak memory footprint I want to parse data to Core Data and then use batching to display them in table view. I don't want them to be saved anyhow because objects will be changing everyday but I want to let user choose its favourites object and then save them. Favourites will be displayed in different table view.
Solutions I'm thinking about:
Create two NSManagedObjectContext (one main and one tmp). Parse objects to tmp, if its marked as favourite copy object to main and save it from there.
Forget about memory and keep data in some array and save only choosen ones.
Save all using one NSManagedObjectContext and before application will terminate delete unmarket as favourites.
Any hint here would be nice. Also some code sample code with solution one because I'm new to Core Data and I may have problems. I've been looking for suitable solution for two days now.
I would suggest you forget about optimizing the memory footprint unless/until you have some hard evidence showing that this will in fact be needed. But of course it depends on what "lots of objects in xml" means exactly.
I am assuming you are not downloading millions of XML documents just to throw them away again when the application finishes (because that would be a serious waste of network resources). In contrast, an array with a few hundred or even a few thousand small XML-like data structures would most likely be nothing to worry about as far as memory is concerned.
That being said, if you would need to save both temporary and persistent data with Core Data after all, I would go for a single NSManagedObjectContext with two entities rather than two separate contexts. Or you could even have a single entity and mark certain instances as temporary data. Managing two different contexts means additional overhead while I don't really see any advantages to that approach.
Does using an NSFetchedResultsController provide any performance advantages on the iPhone over an NSArray?
I have between 4,000 and 8,000 records stored in core data and wanted to know if I should select one over the other. Is the NSFetchedResultsController just used to make code 'prettier'?
My concern is searching, and lag on keyboard button presses (as well as the issue of loading that many records into memory). Thanks!
Given your parameters, Core Data will be faster than an array especially if you make any changes to the data.
The disadvantage of an array in this case is that you have to load the entire array into memory in one go.
It might seem obvious that Core Data will be slower than more primitive methods but owing to fine tuned optimization plus the ease of integration with the rest of the API it is actually fairly hard to beat Core Data in real world apps with significant amounts of data.
Is Core Data still a good option to use for the iOS even if they data will only be quite temporary. i.e – data being sent up to a server in the cloud once within range of a network, and then never needed again on the mobile device.
You don't have to use Core Data to persist data.
If you don't want to persist the data at all, you can define an in-memory store which never writes to disk.
Core Data's true function is the management of an object graph i.e. it handles the relationships between objects. It's real advantage is the ability to automatically handle complexity.
That complexity can arise from the data objects themselves or from their needed relationships to controller or view objects. Either way, Core Data makes it easy to tie all the objects together without great gobs of custom code. Where the objects end up being persisted or even persisted at all, is really secondary.
Yes, if there could ever be a large number of records (e.g. user is overseas, doesn't have a data connection), use Core Data. The point of the Core Data abstraction is just this - if you only have 10 records max, ever, it may just use a flat file of data, or maybe a Sqlite database if more than that -- by "handing the problem" over to Core Data, you make storage decisions Apple's problem, and for free you'll get all the optimizations that Apple'll throw into the Core Data framework in the coming years.
Core Data is complex when you first look at it. Apple's API docs aren't bad, but there are a few "gotchas". If you've worked with anything like the Entity ORM framework, etc, it's really easy to pick up.
Alternatively, if you're reasonably certain that you're only going to get 5-10 records, and the data is anything that conforms to NSCoder, you could just archive it and save it, and then unarchive it when you launch the app. Also, if it's array data, a plist is pretty nice.
The approach I take is to insert entities into a NIL context and provide a base class for insertion into a valid context when I wish to persist them. Code can be found here... Temporary Core Data
How do the memory utilization and performance for XML or SQLite compare on the iPhone?
The initial data set for our application is 500 records with no more than 750 characters each.
How well would XML compare with SQLite for accessing say record 397 without going through the first 396? I know SQLite3 would have a better methods for that, but how is the memory utilization?
When dealing with XML, you'll probably need to read the entire file into memory to parse it, as well as write out the entire file when you want to save. With SQLite and Core Data, you can query the database to extract only certain records, and can write only the records that have been changed or added. Additionally, Core Data makes it easy to do batched fetching.
These limited reads and writes can make your application much faster if it is using SQLite or Core Data for its data store, particularly if you take advantage of Core Data's batched fetching. As Graham says, specific numbers on performance can only be obtained by testing under your specific circumstances, but in general XML is significantly slower for all but the smallest data sets. Memory usage can also be much greater, due to the need to load and parse records you do not need at that instant.
To find out how the memory usage for your application fares, you need to measure your application :). The Instruments tool will help you.
I have an object graph in Objective-C on the iPhone platform that I wish to persist to flash when closing the app. The graph has about 100k-200k objects and contains many loops (by design). I need to be able to read/write this graph as quickly as possible.
So far I have tried using NSCoder. This not only struggles with the loops but also takes an age and a significant amount of memory to persist the graph - possibly because an XML document is used under the covers. I have also used an SQLite database but stepping through that many rows also takes a significant amount of time.
I have considered using Core-Data but fear I will suffer the same issues as SQLite or NSCoder as I believe the backing stores to core-data will work in the same way.
So is there any other way I can handle the persistence of this object graph in a lightweight way - ideally I'd like something like Java's serialization? I've been thinking of trying Tokyo Cabinet or writing the memory occupied by bunch of C structs out to disk - but that's going to be a lot of rewrite work.
I would reccomend re-writing as c structs. I know it will be a pain, but not only will it be quick to write to disk but should perform much better.
Before anyone gets upset, I am not saying people should always use structs, but there are some situations where this is actually better for performance. Especially if you pre-allocate your memory in say 20k contiguous blocks at a time (with pointers into the block), rather than creating/allocating lots of little chunks within a repeated loop.
ie if your loop continually allocates objects, that is going to slow it down. If you have preallocated 1000 structs and just have an array of pointers (or a single pointer) then this is a large magnitude faster.
(I have had situations where even my desktop mac was too slow and did not have enough memory to cope with those millions of objects being created in a row)
Rather than rolling your own, I'd highly recommend taking another look at Core Data. Core Data was designed from the ground up for persisting object graphs. An NSCoder-based archive, like the one you describe, requires you to have the entire object graph in memory and all writes are atomic. Core Data brings objects in and out of memory as needed, and can only write the part of your graph that has changed to disk (via SQLite).
If you read the Core Data Programming Guide or their tutorial guide, you can see that they've put a lot of thought into performance optimizations. If you follow Apple's recommendations (which can seem counterintuitive, like their suggestion to denormalize your data structures at some points), you can squeeze a lot more performance out of your data model than you'd expect. I've seen benchmarks where Core Data handily beat hand-tuned SQLite for data access within databases of the size you're looking at.
On the iPhone, you also have some memory advantages when using controlling the batch size of fetches and a very nice helper class in NSFetchedResultsController.
It shouldn't take that long to build up a proof-of-principle Core Data implementation of your graph to compare it to your existing data storage methods.