Okay, I can't seem to find a clear answer to this question of storage on the iPhone. My model class has several ivars and two very large (MB) mutable arrays of data that are collected from an external device and then analyzed. What I'm thinking is that you have data in the object (similar to a note or a music file) and you can save it to a permanent data "file", and then later open old data "file" and view it (no editing of old data will be done). Along side this I want another stored object that keeps track of a few key bits of information from each of the data files, and also has references to them (maybe the user could click a data point, and it would open the corresponding data file - if it still exists (it could be deleted by the user to save space)).
I see tons of advice recommending all data storage for iPhone apps should use Core Data. The thing is, except for the one side "file", there are no relationships between objects. The objects could be thought of as notes or music files, they don't care about the existence of each other, and there is only one object in existence ("loaded") at a time (either in memory with data being added to it, and to be saved later, or loaded from storage being viewed).
What is the best way to manage this? Currently a device controller (handles the device communication) creates the model, and sends data to it (the model parses and analyzes the data). But should there be some controller that handles files (or Core Data managedobject, whatever) that creates the object, and the saves its data, releases its memory and then loads a new one with stored data?
Any advice would be helpful, as the best storage examples I've found seem to be very relational (employee,boss,company) - which I can see would benefit from a database. But at the same time, manually keeping a list of files in a directory may be more work than some other method.
I can't find citation in Apple's documentation but I have read (and been told by Apple engineers) that "large" data objects are sometimes best stored outside of Core Data. The model that has been suggested places BLOBs (Binary Large OBjects) in file system with Core Data objects referencing these large objects (i.e., storing relative or absolute file paths).
So, assuming that your BLOBs are music data, then you might have your Core Data model have an entity that holds meta-data (e.g., size, time/duration, etc.) as well as reference to file that holds actual data. Your meta-data entity could also have relationships with other entities within your system. For example, you might store spectrograms for music data and have those held by a separate entity.
I wrestled with this issue for data that was being sampled from various measurement sensors. Ultimately, I decided that my data sets were small enough (in most cases) to store with Core Data as NSData properties of dedicated entity. The wrapping entity was 'dedicated' so as to avoid loading data just to display meta-data to user.
Update
I found line about BLOBs in Core Data Programming Guide at the end of the "Large Data Objects (BLOBs)" section:
It is better, however, if you are able
to store BLOBs as resources on the
filesystem, and to maintain links
(such as URLs or paths) to those
resources. You can then load a BLOB as
and when necessary.
I'd also recommend using Core Data. While Core Data does make it easy to handle relations, no one is preventing you from using Core Data to store unrelated information. There is no rule against creating models in Core Data that have no relation with each other what so ever; just don't link them together.
Core Data will handle all the reading/writing to the database, which will save you the trouble of having to parse your own files. There is a bit of a learning curve when trying to use Core Data for the first time, but once you get it running, you'll be thankful it's there.
Unless you need any kind of database access (fast queries, frequent updates, etc.), a database involves too much overhead, in terms of both coding and performance. Database features are not exactly free, so I can think of a lot of scenarios where you will get better results with less effort if you go with one of the following:
multiple files with a separately stored index file;
a large file of your own binary format.
The choice depends on the number of items, the size of items, whether their sizes are identical, how often you need to modify the data and/or the index, etc.
Related
Problem: I'm implementing aplication where user is dowloading lots of objects in xml. To reduce peak memory footprint I want to parse data to Core Data and then use batching to display them in table view. I don't want them to be saved anyhow because objects will be changing everyday but I want to let user choose its favourites object and then save them. Favourites will be displayed in different table view.
Solutions I'm thinking about:
Create two NSManagedObjectContext (one main and one tmp). Parse objects to tmp, if its marked as favourite copy object to main and save it from there.
Forget about memory and keep data in some array and save only choosen ones.
Save all using one NSManagedObjectContext and before application will terminate delete unmarket as favourites.
Any hint here would be nice. Also some code sample code with solution one because I'm new to Core Data and I may have problems. I've been looking for suitable solution for two days now.
I would suggest you forget about optimizing the memory footprint unless/until you have some hard evidence showing that this will in fact be needed. But of course it depends on what "lots of objects in xml" means exactly.
I am assuming you are not downloading millions of XML documents just to throw them away again when the application finishes (because that would be a serious waste of network resources). In contrast, an array with a few hundred or even a few thousand small XML-like data structures would most likely be nothing to worry about as far as memory is concerned.
That being said, if you would need to save both temporary and persistent data with Core Data after all, I would go for a single NSManagedObjectContext with two entities rather than two separate contexts. Or you could even have a single entity and mark certain instances as temporary data. Managing two different contexts means additional overhead while I don't really see any advantages to that approach.
I am working on an application for the iPhone (iOS 5). What I have to do is create a map by using binary data that I reveive from a server. Some issues actually work quite well:
I can connect to a server, send requests and receive binary data from it
I can interprete this data, create objects (polygons and paths) from it and draw them within a view
But now it comes to the hard part. The map that I create should be zoomable and moovable. So I have to send new requests to the server and redraw the map. This also works nicely, but the data I already received now needs to be stored, because I should not request the same data from the server twice (e.g. if I zoom out and then back in).
Finally here is my question: What would be the best way to store my data? Until now I thought about using CoreData or SQLite. Are there even better solutions? And what data should I save - the binary data or my created objects?
I hope this was understandable and you can help me with at least one of my issues...
Core data is the only way to go.
Core data is not a storage system, is an object graph and persistence framework, witch can use SQlite to store data.
If you use core data you can refactor your project and use managedObjects subclass as models.
Take a look at Core Data Programming Guide, The differences between Core Data and a Database
Edit:
From Core Data Performance
Core Data is a rich and sophisticated object graph management
framework capable of dealing with large volumes of data. The SQLite
store can scale to terabyte sized databases with billions of
rows/tables/columns. Unless your entities themselves have very large
attributes (although see “Large Data Objects (BLOBs)”) or large
numbers of properties, 10,000 objects is considered to be a fairly
small size for a data set.
It really depends on the size of your data objects and how you access them. If your objects are small, you could store them in Core Data. But, if your map data is coming as images from a bunch of URLs, I would use Core Data to store the mappings to the map image URLs and use NSURLConnection to manage the caching of your objects.
I recommend reading the Apple Core Data Programming Guide Large Data Objects (BLOBs), it discusses the size and number of objects. Some excerpts are below:
The exact definition of "small", "modest", and "large" is fluid and depends on an application's usage. A loose rule of thumb is that objects in the order of kilobytes in size are of a "modest" sized and those in the order of megabytes in size are "large" sized.
For small to modest sized BLOBs (and CLOBs), you should create a separate entity for the data and create a to-one relationship in place of the attribute.
It is better, however, if you are able to store BLOBs as resources on the filesystem, and to maintain links (such as URLs or paths) to those resources. You can then load a BLOB as and when necessary.
I'm rearching the best ways to store non standard types (string, int16 etc) on the iphone.
What I will ultimately be doing is downloading an xml file and storing values such as date, title, name, mediaurl. I've just discovered the coredata data model and I believe it would be a good candidate for storing such data so I don't have to download the xml the next time the app starts.
What I'm unsure of is the limitations (if any) of what I can store in a entity. For example one of the xml elements would hold a url to a small piece of audio (less than 1mb) and a url to an image. Would it be appropriate to store audio data , image as an attribute in an entity or should it be kept to strings and ints etc and the non standard types stored else where?
I guess what I'm really asking is, is the datamodel suitable for caching?
Ultimately what I'm seeking is a solution for storing data on the device in a location that is not tied to any one view, kinda an atomic model with everything I need that I can just dip into no matter view I'm in.
The data model is suitable for caching, but because you don't have an explicit control of the cache (you can fault a data object but it may remain in the memory), it's recommended to separate very large binary objects. Store them as resources on the filesystem, and manage their links (URLs or paths) in Core Data.
< 1MB file seems okay to be handled by Core Data, but it also depends how many of them your application uses.
Also if you do store large files in Core Data, you should use SQLite storage.
The above answer from MHC is good, but if you're storing large binary objects that don't need to be indexed (which can't be done in SQLite anyway), the recommended way is to store the actual data somewhere on the file system (say, in the NSDocumentsDirectory), and store path to the file inside the Core Data entity.
Core Data loads all parts of a fetched object into memory, which for a few instances of entities with binary data could quickly cause you to run out of memory on an iOS device.
If it's stored in the filesystem, you can lazy-load the data just when you need it.
Is Core Data still a good option to use for the iOS even if they data will only be quite temporary. i.e – data being sent up to a server in the cloud once within range of a network, and then never needed again on the mobile device.
You don't have to use Core Data to persist data.
If you don't want to persist the data at all, you can define an in-memory store which never writes to disk.
Core Data's true function is the management of an object graph i.e. it handles the relationships between objects. It's real advantage is the ability to automatically handle complexity.
That complexity can arise from the data objects themselves or from their needed relationships to controller or view objects. Either way, Core Data makes it easy to tie all the objects together without great gobs of custom code. Where the objects end up being persisted or even persisted at all, is really secondary.
Yes, if there could ever be a large number of records (e.g. user is overseas, doesn't have a data connection), use Core Data. The point of the Core Data abstraction is just this - if you only have 10 records max, ever, it may just use a flat file of data, or maybe a Sqlite database if more than that -- by "handing the problem" over to Core Data, you make storage decisions Apple's problem, and for free you'll get all the optimizations that Apple'll throw into the Core Data framework in the coming years.
Core Data is complex when you first look at it. Apple's API docs aren't bad, but there are a few "gotchas". If you've worked with anything like the Entity ORM framework, etc, it's really easy to pick up.
Alternatively, if you're reasonably certain that you're only going to get 5-10 records, and the data is anything that conforms to NSCoder, you could just archive it and save it, and then unarchive it when you launch the app. Also, if it's array data, a plist is pretty nice.
The approach I take is to insert entities into a NIL context and provide a base class for insertion into a valid context when I wish to persist them. Code can be found here... Temporary Core Data
My plan is to display a list of items alphabetically in a table view that has about 100 items. Each item has an image, a list of times and a description that the tableview will drill down to. What I am struggling with is the correct way to store and load this data. Some have told me that a plist will be too data heavy and that core data is too new. Should I just create arrays?
You're not clear about what you intend to do with this data. Plists and Core Data are both persistence formats (on disk). Arrays are an in-memory format (and can also be slapped onto disk, I suppose, if that's what you want to do, but inventing your own binary disk format is only something you should consider very rarely, and certainly not in the case you probably have).
In memory, you can probably just use an array (NSArray) and have each element perhaps be an NSDictionary of the other properties relative to that entry. That sounds like the model of your MVC design, which you can then hook up to the table view.
As far as persisting this to disk, it depends on whether 100 items is a fixed amount, a ballpark, or a minimum, etc. Plists (see NSKeyedArchiver) are great for all the data except possibly the raw image data-- you might want to keep those "to the side" as separate image files with filenames in the plist.
I don't know much about Core Data, but it's not that new, and it's not untested, so if it does what you want without much hassle, go for it.
Serialize it into an Archive using NSCoding Protocol. See Guide.
I'd use an NSArray of business objects implementing NSCoding and then just archive them.
I usually default to Core Data unless I have a compelling reason not to. (Of course, I have learned Core Data so that makes it easy for me to this.)
Core Data has the following advantages:
It has an editor in which you can create complex object graphs easily
It can generate custom classes for you data model objects.
The generated classes are easily modified.
Core Data manages the complexity of adding, deleting and saving objects.
Core Data makes persisting an object graph almost invisible.
NSFetchedResultsController is custom designed to provide data for tables.
100 objects is a small graph that Core Data can handle easily. It's a lot easier to use Core Data than it is to write custom coders to archive custom objects. For exmaple, at present, I have an app with over a dozen major entities each with two or three relationships to other entities. Hand coding all that would be a nightmare.
Core Data has something of a steep learning curve especially if you've never worked with object graphs before but if you're planning on writing a lot of Apple platform software, learning it is well worth the time.