Suppose I fetch a list of Realm objects and access all the data from the results (causing the data to be loaded into memory). Next, with the reference to the list of objects still around, I fetch one of those objects directly by its primary key. Am I correct to assume that, in this case, the object will be returned without having to hit the disk? What if I no longer had a reference to that original list of results? Might it still be in memory?
Just curious to understand how objects are cached internally by Realm, so I don't unnecessarily try to optimize things in my code (e.g. holding onto objects that I think I'll use again soon) if it's not needed.
Realm memory-maps the file, and only accesses it when you actually read the properties on objects. This means that recent-accessed data will still be in memory until the OS has to discard it to free up that RAM for something else, and there is no difference in terms of disk i/o from obtaining a new object from Realm and holding on to an existing object.
Related
Problem: I'm implementing aplication where user is dowloading lots of objects in xml. To reduce peak memory footprint I want to parse data to Core Data and then use batching to display them in table view. I don't want them to be saved anyhow because objects will be changing everyday but I want to let user choose its favourites object and then save them. Favourites will be displayed in different table view.
Solutions I'm thinking about:
Create two NSManagedObjectContext (one main and one tmp). Parse objects to tmp, if its marked as favourite copy object to main and save it from there.
Forget about memory and keep data in some array and save only choosen ones.
Save all using one NSManagedObjectContext and before application will terminate delete unmarket as favourites.
Any hint here would be nice. Also some code sample code with solution one because I'm new to Core Data and I may have problems. I've been looking for suitable solution for two days now.
I would suggest you forget about optimizing the memory footprint unless/until you have some hard evidence showing that this will in fact be needed. But of course it depends on what "lots of objects in xml" means exactly.
I am assuming you are not downloading millions of XML documents just to throw them away again when the application finishes (because that would be a serious waste of network resources). In contrast, an array with a few hundred or even a few thousand small XML-like data structures would most likely be nothing to worry about as far as memory is concerned.
That being said, if you would need to save both temporary and persistent data with Core Data after all, I would go for a single NSManagedObjectContext with two entities rather than two separate contexts. Or you could even have a single entity and mark certain instances as temporary data. Managing two different contexts means additional overhead while I don't really see any advantages to that approach.
I am a newbie to caching and have no idea how data is stored in caching. I have tried to read a few examples online, but everybody is providing code snippets of storing and getting data, rather than explaining how data is cached using memcache. I have read that it stores data in key, value pairs , but I am unable to understand where are those key-value pairs stored?
Also could someone explain why is data going into cache is hashed or encrypted? I am a little confused between serialising data and hashing data.
A couple of quotes from the Memcache page on Wikipedia:
Memcached's APIs provide a giant hash
table distributed across multiple
machines. When the table is full,
subsequent inserts cause older data to
be purged in least recently used (LRU)
order.
And
The servers keep the values in RAM; if
a server runs out of RAM, it discards
the oldest values. Therefore, clients
must treat Memcached as a transitory
cache; they cannot assume that data
stored in Memcached is still there
when they need it.
The rest of the page on Wikipedia is pretty informative, and it might help you get started.
They are stored in memory on the server, that way if you use the same key/value often and you know they won't change for a while you can store them in memory for faster access.
I'm not deeply familiar with memcached, so take what I have to say with a grain of salt :-)
Memcached is a separate process or set of processes that store a key-value store in-memory so they can be easily accessed later. In a sense, they provide another global scope that can be shared by different aspects of your program, enabling a value to be calculated once, and used in many distinct and separate areas of your program. In another sense, they provide a fast, forgetful database that can be used to store transient data. The data is not stored permanently, but in general it will be stored beyond the life of a particular request (it is possible for Memcached to never store your data, so every read will be a miss, but that's generally an indication that you do not have it set up correctly for your use case).
The data going into cache does not have to be hashed or encrypted (but both things can happen to the data, depending on the caching mechanism.)
Serializing data actually has nothing to do with either concept -- instead, it is the process of changing data from one format (generally one suited for in-memory storage) to another one (generally suitable for storage in a persistent medium.) Another term for this process is marshalling and unmarshalling.
I have an app that currently holds all state in memory. It fetches a bunch of information from a server as JSON and then holds on to the JSON values in memory. Each JSONObject can be ~300 bytes and there can be thousands of such objects.
I use this data simply to populate UITableView.
In order to better handle large amounts of aata, I modified my code to fetch data from the server and store it using CoreData. There JSON objects can be represented as simple entities, each with 3 NSString attributes and one 1 int32 attribute. I created a NSFetchedResultsController to use as the data source of the UITableView. My assumption was that this would reduce the resident memory usage of my application (I assume NSFetchedResults controllers effectively manages memory to not hold entities that aren't being displayed in the view, vs holding all my state in-memory).
For the purposes of this discussion, let's assume my app purges the CoreData store and re-fetches all data each time it runs.
When I went to measure the changes in Resident Memory and Virtual Size using the VM Tracker in Instruments, I noticed that both these values remain almost identical. Infact, the Core-Data based version of my app seems to use more memory than when I have everything entirely in-memory.
While this may be true, I don't have an intuition for why this might be so. Any explanations?
From what I have said about my app, does it sound like I don't want to bother persisting in CoreData, and why?
Core Data can use more memory as you have seen, if there is memory to be had. However, one of the benefits to Core Data is when you get into a low memory situation. When that happens then Core Data will automatically reduce its own memory footprint as much as possible.
Another thing to consider is are you letting Core Data fault these objects? If you are pulling in 1000 objects and displaying 10 of them, then the other 990 should be in a faulted state and therefore taking up less memory.
I would run through the Core Data instruments and make sure you are not fulling realizing all of these objects by accident and unintentionally causing your memory usage to be higher than it needs to be.
Update
Sounds like you are importing this data and then not flushing Core Data properly if you are not seeing any faulting going on.
Assuming you are loading this data on first launch (which I would not do btw, I would pre-load the app and avoid the plist files entirely), you want to call -reset on your NSManagedObjectContext after the load is complete so that any objects that are not being used are flushed out of memory. Then as the data is coming back into memory (on use) it will be properly faulted.
Lastly, make sure you are using a SQLite store. Otherwise this is all moot.
In the app I am working on now I was storing about 500 images in Core Data. I have since pulled those images out and store them in the file system now, but in the process I found that the app would crash on the device if I had an array of 500 objects with image data in them. An array with 500 object ids with the image data in those objects worked fine. The 500 objects without the image data also worked fine. I found that I got the best performance with both an array of object ids and image data stored on the filesystem instead of in core data.
The conclusion I came to was that if I had an object in an array that told Core Data I was "using" that object and Core Data would hold on to the data. Is this correct?
Short answer is yes.
The long answer is that it depends on the size of the images. The rule is:
less than 100kb store it in a main table.
less than 1mb store it in a secondary table on the other end of a relationship.
greater than 1mb, store it on disk and reference it via a file path.
So, depending on the size of your files will determine where to store them. Also, keep in mind that UIImage handles caching so you may not need to store the images in an array at all.
Update
Your question is unclear then. You do not need to store the images in an array because A) the image is being held by the cell and; B) UIImage will cache the image for you so it will not be retrieved from disk if it has been accessed lately. So you are forcing the retention of images unnecessarily.
With regard to Core Data itself, it will drop attributes out of memory as needed as well. It will automatically pull them back into memory when accessed. Core Data also caches the data so you should not see any performance issues there either as things are being moved around in memory.
In my application I need to keep track of a list of objects that are being displayed. Right now I have an NSArray with all of the NSManagedObjects. Would I be better off to store the ObjectIDs and then only request the object when I need it?
I am mainly concerned about memory at this point.
I am mainly concerned about memory at this point.
The first issue here is that you don't really understand how memory is stored. This array of yours is only holding pointers to the objects, not the objects themselves, thus holding the NSManagedObjects vs ObjectIDs is the same as the size of pointers are the same.
CoreData is pretty developed and has many internal optimizations for automatic memory handling and faulting within the Managed Object Context.
Given this info, it seems the clear choice is to use NSManagedObjects in your array solely because doing it the other way adds no benefit, and doing it this way has you write less code to retrieve them.