I'm creating an iPhone app, which fetches information from a server every time it is started. However, i'm planning on using the fetched data of the last month/few months/year to calculate some averages.
I had been thinking about saving them to NSUserDefaults using dictionaries (associating a date with a value), but i just remembered there also exists something like core data. Seeing that i do not have any experience with core data, i don't know if it's better. If it wouldn't, i could save the
time i'd use learning it otherwise.
The data comes in in XML format, and i get several sets of the same response each time (for different locations on a map). The amount of sets can change, as the user can add more locations. I currently only save the raw data to the disk to load if the load fails next time it starts. However, i also want to save some specific values from that XML in a way that i can easily access it. What would be the best way to do this?
Edit: I actually also need to know how fast/efficient core data is. I'm currently passing around NSArrays with NSDictionaries for the sets of data during that session. For saving the data that last longer than the session core data is ideal, i found out that much (just need to find a nice way to associate an entity with a date), i just need some advice on the efficiency.
If you're going to be working with larger amounts of data, it's probably anyway better to give Core Data a try, it's after all not that complicated and there's a plenty of good tutorials where you can learn it. There are different settings for the storage type, you can either use a sqlite database or an xml file.
According to the guys from Apple, it should be fast and memory effective to use Core Data in contrast to self-made solutions, so it's a preferred way to go.
Core data would be easier to manipulate the data and query the data using predicates. Core Data supports dates so you can even find items in date ranges.
Related
thanks for you help.
I'm attempting to add core data to my project and I'm stuck at where and how to add the actual data into the persistent store (I'm assuming this is the place for the raw data).
I will have 1000 < objects so I don't want to use a plist approach. From my searches, there seems to be xml and csv approaches. Is there a way I can use SQL for input?
The data will not be changed by the user and the data file will be typed in by hand, so I won't need to update these files during runtime, and at this point I am not limited in any type of file - the lightest on syntax is preferred.
Thanks again for any help.
You could load your data from an xml/csv/json file and create the DB on the first lunch of your application (if the DB is not there, then read the data and create it).
A better/faster approach might be to ship your sqllite DB within your application. You can parse the file in any format you want on the simulator, create a DB with all your entities, then take it from the ApplicationData and just add it to your app as a resource.
Although I'm sure there are lighter file types that could be used, I would include a JSON file into the app bundle from which you import the initial dataset.
Update: some folks are recommending XML. NSXMLParser is almost as fast as JSONKit (but much faster than most other parsers), but the XML syntax is heavier than JSON. So an XML bundled file that holds the initial dataset would weight more than if it was in JSON.
Considering Apple considers the format of its persistent stores implementation details, shipping a prefabricated SQLite database is not a very good idea. I.e. the names of fields and tables may change between iOS versions/phones/whatever hidden variable you can think of. You should, in general, not concern yourself with how this serialization of your data is formatted.
There's a brief article about importing data on Apple's developer site: Efficiently Importing Data
You should ship initial data in whatever format you're comfortable with (XML allows you to do incremental parsing efficiently, which reduces memory footprint) and write an import routine to run if you need to import data.
Edit: With EliBud's comment in mind, I still consider the approach a bit "iffy"... The format of the SQLite database used by Core Data is not something you'd want to generate by yourself (it's weird, simply put, and still not something you should really rely on).
So you'd want to use a mock app running on the Simulator and use Core Data to create the database (as per EliBud's answer). But you'd still have to import the data into that mock-app! And while it might make sense to do this once on a "real" computer instead of a lot of times on a mobile device (i.e. copying a file is easy, importing data is hard), you're essentially using the Simulator as an administration tool.
But hey, if it works...
Okay, I can't seem to find a clear answer to this question of storage on the iPhone. My model class has several ivars and two very large (MB) mutable arrays of data that are collected from an external device and then analyzed. What I'm thinking is that you have data in the object (similar to a note or a music file) and you can save it to a permanent data "file", and then later open old data "file" and view it (no editing of old data will be done). Along side this I want another stored object that keeps track of a few key bits of information from each of the data files, and also has references to them (maybe the user could click a data point, and it would open the corresponding data file - if it still exists (it could be deleted by the user to save space)).
I see tons of advice recommending all data storage for iPhone apps should use Core Data. The thing is, except for the one side "file", there are no relationships between objects. The objects could be thought of as notes or music files, they don't care about the existence of each other, and there is only one object in existence ("loaded") at a time (either in memory with data being added to it, and to be saved later, or loaded from storage being viewed).
What is the best way to manage this? Currently a device controller (handles the device communication) creates the model, and sends data to it (the model parses and analyzes the data). But should there be some controller that handles files (or Core Data managedobject, whatever) that creates the object, and the saves its data, releases its memory and then loads a new one with stored data?
Any advice would be helpful, as the best storage examples I've found seem to be very relational (employee,boss,company) - which I can see would benefit from a database. But at the same time, manually keeping a list of files in a directory may be more work than some other method.
I can't find citation in Apple's documentation but I have read (and been told by Apple engineers) that "large" data objects are sometimes best stored outside of Core Data. The model that has been suggested places BLOBs (Binary Large OBjects) in file system with Core Data objects referencing these large objects (i.e., storing relative or absolute file paths).
So, assuming that your BLOBs are music data, then you might have your Core Data model have an entity that holds meta-data (e.g., size, time/duration, etc.) as well as reference to file that holds actual data. Your meta-data entity could also have relationships with other entities within your system. For example, you might store spectrograms for music data and have those held by a separate entity.
I wrestled with this issue for data that was being sampled from various measurement sensors. Ultimately, I decided that my data sets were small enough (in most cases) to store with Core Data as NSData properties of dedicated entity. The wrapping entity was 'dedicated' so as to avoid loading data just to display meta-data to user.
Update
I found line about BLOBs in Core Data Programming Guide at the end of the "Large Data Objects (BLOBs)" section:
It is better, however, if you are able
to store BLOBs as resources on the
filesystem, and to maintain links
(such as URLs or paths) to those
resources. You can then load a BLOB as
and when necessary.
I'd also recommend using Core Data. While Core Data does make it easy to handle relations, no one is preventing you from using Core Data to store unrelated information. There is no rule against creating models in Core Data that have no relation with each other what so ever; just don't link them together.
Core Data will handle all the reading/writing to the database, which will save you the trouble of having to parse your own files. There is a bit of a learning curve when trying to use Core Data for the first time, but once you get it running, you'll be thankful it's there.
Unless you need any kind of database access (fast queries, frequent updates, etc.), a database involves too much overhead, in terms of both coding and performance. Database features are not exactly free, so I can think of a lot of scenarios where you will get better results with less effort if you go with one of the following:
multiple files with a separately stored index file;
a large file of your own binary format.
The choice depends on the number of items, the size of items, whether their sizes are identical, how often you need to modify the data and/or the index, etc.
Is Core Data still a good option to use for the iOS even if they data will only be quite temporary. i.e – data being sent up to a server in the cloud once within range of a network, and then never needed again on the mobile device.
You don't have to use Core Data to persist data.
If you don't want to persist the data at all, you can define an in-memory store which never writes to disk.
Core Data's true function is the management of an object graph i.e. it handles the relationships between objects. It's real advantage is the ability to automatically handle complexity.
That complexity can arise from the data objects themselves or from their needed relationships to controller or view objects. Either way, Core Data makes it easy to tie all the objects together without great gobs of custom code. Where the objects end up being persisted or even persisted at all, is really secondary.
Yes, if there could ever be a large number of records (e.g. user is overseas, doesn't have a data connection), use Core Data. The point of the Core Data abstraction is just this - if you only have 10 records max, ever, it may just use a flat file of data, or maybe a Sqlite database if more than that -- by "handing the problem" over to Core Data, you make storage decisions Apple's problem, and for free you'll get all the optimizations that Apple'll throw into the Core Data framework in the coming years.
Core Data is complex when you first look at it. Apple's API docs aren't bad, but there are a few "gotchas". If you've worked with anything like the Entity ORM framework, etc, it's really easy to pick up.
Alternatively, if you're reasonably certain that you're only going to get 5-10 records, and the data is anything that conforms to NSCoder, you could just archive it and save it, and then unarchive it when you launch the app. Also, if it's array data, a plist is pretty nice.
The approach I take is to insert entities into a NIL context and provide a base class for insertion into a valid context when I wish to persist them. Code can be found here... Temporary Core Data
I have in my app a considerable amount of data that it needs to access, but will never be changed by the app. Currently I'm using this data in other applications in JSON files and SQL databases, but neither seems very straightforward to use in iOS.
I don't want to use CoreData, which provides tons of unnecessary functionality and complexity.
Would it be a good idea store the data in PropertyList file and build an accessor class? Are there any simple ways to incorporate SQLite without going the CoreData route?
You can only use plist if the amount of data is relatively small. Plist are entirely loaded into memory so you can only really use them if you can sustain all the objects created by the plist in memory at once for as long as you need them.
Core Data has a learning curve but in use it is usually less complex than SQL. In most cases the "simpler" SQL leads to more coding because you end up having to duplicate much of the functionality of Core Data to shoehorn the procedural SQL into the object-oriented API. You have to manually manage the memory use of all the data by tracking retention. You've write a lot of SQL code every time you want data. I've updated several apps from SQL to Core Data and in all cases the Core Data implementation was smaller and cleaner than the SQL.
Neither is the memory or processor "overhead" any larger. Core Data is highly optimized. In most cases, off the shelf Core Data is more efficient than hand tuned SQL. One minor sub optimization in SQL usually destroys any theoretical advantage it might have.
Of course, if you're already highly skilled at managing SQL in C then you personally might get the app to market more quickly by using SQL. However, if you're wondering what you should plan to use in general on on Apple Platforms, Core Data is almost always the answer and you should take the time to learn it.
You can just use SQLite directly without the overhead of Core Data using the SQLite C API.
Here is a tutorial I found on your use-case - simply loading some data from an SQLite database. Hope this helps.
Depending on the type of your data, the size and how often it changes, you may desire to just keep things simple and use a property list. Otherwise, using SQLite (documented in Jergason's answer) would be where I'd go. Though let me say that if you have a relatively small (less than a couple hundred) set of basic types (arrays, dictionaries, numbers, strings) that don't change frequently, then a property list will be a better choice in my opinion.
As an example to that, in one of my games, I create the levels from a single property list per difficulty. Since there are only a handful of levels per difficulty (99) and a small set of parameters for each (number of elements in play, their initial positions, mass, etc) then it makes sense, and I avoid having to deal with SQLite directly or worse yet, setting up and maintaining CoreData.
What do you mean by "best"? What kind of data?
If it's a bunch of objects, then JSON or (binary) plist aren't terrible formats, since you'll want the whole thing loaded in memory to walk the object graph. Compare space efficiency and loading performance to pick which one to use.
If it's a bunch of binary blobs, then store the blobs in a big file, memory-map the file (NSDataReadingMapped a.k.a. NSMappedRead), and use indexes into the blobs. iOS frameworks use a mixture of these (e.g. there are a lot of .pngs, but also "other.artwork" which just contains raw image data).
You can also use NSKeyedArchiver and friends if your classes implement the NSCoding protocol, but there's some object graph management overhead and the plist format it produces isn't exactly nice to work with.
is there any easy way of store XML data into core data?
Currently, my app just pulls the values from the XML file directly, however, this isn't efficient for XML files which holds over 100 entries, thus storing the data in Core Data would be the best option. XML file is called/downloaded/parsed ever time the app opens.
With the Core Data, the XML data would be downloaded ever 3600 seconds or so, and refresh the current data in the core data, to reduce the loading time when opening the app.
Any ideas on how I can do this?
Having reviewed the developer documentation, it doesn't look very tasty.
I take that you mean you have to down load an xml file, parse it and then save the data encoded in the file? You have several options for saving such data.
If the data is relatively simple and static e.g. a repeating list of items, then you might just want to use a NSArray, NSSet or NSDictionary (or some nested combination) and then just write the resulting collection to disk as a plist using the collection classes writeToFile: methods. Then when the data is needed you just use one of the initWithFile: methods. The disadvantage of this system is that you have to read the entire file back into memory to use it. This system doesn't scale for very large data sets.
If the data is complex e.g. a bunch of separate but highly interrelated chunks of data, and moderately large, then Core Data would be better.
Of course, you always have the option of writing the downloaded file straight to disk as a string if you want.