Options for Caching Generic Object Lists for iPhone? - iphone

I have an app that connects to an API and retrieves a large number of small objects at a time. These objects are returned from json-framework as a heirarchy of NSDictionary, NSArray, etc. They are basic data types, and the primary structure of most of the api calls is a list of items(which could be hundreds of items long).
I will be displaying the data in a UITableView, but I don't want all the data in memory(as I could potentially eat it all up). I also don't want to hit up the api for the objects again, as they can be cached for a few hours without any problems.
I have thought of a couple of solutions, but I am eager to hear other options. I have thought of using Core Data in various ways, but I'd like to avoid having to create and maintain entities for each type of entity returned from the API. Using Core Data like this seems like overkill. I could use core data to store archived objects, though I worry about the archiving/de-archiving overhead.
I really want a generic store that can keep a list of objects cached, and then be able to retrieve arbitrary items from the list.
The factors that I care about:
Ease of maintenence. If the api changes or we add other objects, I don't want to have a lot of places in the code to change. Storing generic objects(NSCoding compliant) is ideal.
Performance. Caching and retrieval will all be happening while the user is interacting with the app(in the background), but I don't want to consume too many resources to make the app feel slow.
Are there any existing libraries that already exist for this purpose? What options am I missing? Maybe convince me that creating Core Data entities for each data type is actually a good idea for an object cache.

In case you haven't tried out the facebook/Three20 library, they have implemented a disc/memory cache and have used it as one of the fundamental layers inside their network module. I used three20 in my app but I never used this cache layer directly, so I'm not sure about the performance with it. Still, it's a nice thing to keep in mind.
Besides, the Enormego team has implemented a stand-alone photo viewer thing. And inside this photo viewer there is a nicely-working disc-memory cache. I'm not sure whether it has anything to do with the three20 cache. But I do believe it can cache generic objects other than just images. You can try it out by yourself.

Related

What are some advantages of using Core Data? (as opposed to plist)

I am relatively new to iOS and programming, and I made an app before, but it used a plist for storage, which I saved to the documents folder. Now, I am thinking about switching over to Core Data, but it looks a little complicated, and I'm not sure if it will work for what I want. I am going to have a bunch of data which I need to graph, so I'm not sure if Core Data is best for this, as it seems that I cannot create an array type in the .xcdatamodeld file. What are some other advantages of Core Data? Is it faster? Easier to use (once you set it up)?
Update: For anyone wondering, I finished the app, and it was totally worth it to learn how to use Core Data, and it was a lot less complicated that I originally thought. Doing it with plists would have been hell. The way they go about doing it seemed a little cryptic at first but if you just start using it you will get it. The relationships are really what is awesome about it.
A few advantages off the top of my head:
Much better memory management. With a plist you must load the entire thing into memory; with Core Data only the objects you're currently using need to be loaded. Also, once objects are loaded, they're normally placeholder "fault" objects whose property data doesn't load until you need it.
Related to the above, when you have changes, you can save only the changed objects, not the entire data set.
You can read/write your model objects directly instead of converting them to/from something like an NSDictionary.
Built-in sorting of objects when you fetch them from the data store.
Rich system of predicates for searching your data set for objects of interest.
Relationships between entities are handled directly, as properties on the related objects. With a plist you would need to do something like store an object ID for a relationship, and then look up the related object.
Optional automatic validation of property values.
Data models don't use arrays, but "to-many" relationships are modeled as sets.
It's a matter of what you're saving. For simple strings, arrays, dictionaries, it's fine to use a plist. For something more complicated (data, images, non-object information) or something with to-many relationships (think relationship between song to album, or photo to photographer), then something like a more robust solution might work better like SQLite.
CoreData is an objective-c-based wrapper around SQLite. If you think you might want to something more complicated, CoreData might be the way to go.
If you need a quick tutorial, I'd check out: http://www.raywenderlich.com/934/core-data-tutorial-getting-started
This got me going and allowed me to learn the basics the workings of CoreData.
Good luck!

How to create a persistant iphone cache

So I have been doing lots of reading and found out NSCache is not persistent which is a slight problem for what I need to do. I have heard that I will need to use Core data instead... but I don't have any experience with core data so am wondering if its the only solution for persistent data.
The app will allow the user to search through a catalog, by typing in a search parameter in the code and select a distributor for their search parameter. What I want to do when the app loads is download the list of distributors and save them to a "cache" the will be persistent (till when the header I will make at some point changes and demands the app to update the cache), so that if the user turns the app of or the phone next time then open it the manufacture.
Now that I'm getting abit deeper into my app I'm getting abit lost in things for instance how would setting up a cache work with regards to NSURLConnection.
Any suggestions or code examples would be greatly appreciated..
This previous answer of mine might help you decide.
To sum it up:
If your data is small, static and low-complexity, use a collection class and write it to disk using the built-in class methods
If the data is static, low-complexity but large, SQL may be a good solution especially if you already know it.
If the data is dynamic and complex regardless of size, then Core Data is your best choice.
Viewed purely from the technical perspective, Core Data is always the best choice for iOS/MacOS API apps. Core Data is not just a persistence API, it is an API for creating the model layer of the Model-View-Controller design paradigm that the Apple API uses. It not only persist the data, but models it, validates its and provides an easy interface to the rest of the API.
If your going to be writing iOS apps, you need to eventually learn Core Data. However, it does have a learning curve and you should pick the method right now that will let you ship a usable app.
You can also check out sqlite. Here's another question that discusses getting started with sqlite on the phone: Where's the best SQLite 3 tutorial for iPhone-SDK?
The advantage to sqlite is that it is fairly easy to pick up. The downside is that you have to write queries for everything, and that can be a pain. It doesn't save objects, just data, numbers or text.

When to use CoreData in iPhone development

I've been looking into creating a new application for iOS and after my last few apps I've been tempted to use CoreData (for benefits including saving and automatic undo/redo).
I've been a little confused when trying to implement the data-model I've been given fr the project though, since it seems that CoreData seems very much much closer to a database than a data model.
Should I be using CoreData for an application that doesn't generally fit the 'large amount of data/records' description I would generally use an SQL style database for?
If it helps, the app I'm designing will be a sort of document editor, so there will be a number of objects I will need to represent (there might be embedded images, graphs/charts, hyperlinks etc within the document) and I need to create this model from an xml description.
Most of these 'items' need to implement a set of interfaces (the model was created for a Java product; I'm having difficulties seeing how inheritance and abstract interfaces can apply to CoreData), and every example I've found so far seems to add base elements (like an NSDate or String) to a simple model.
Does this sound like a candidate for CoreData, or is CoreData more of a tool for implementing a database in an application? (i.e a library system/staff database).
consider CoreData as an option once you are able to properly write the majority of the code it will replace. so once you know how to properly serialize/deserialize, write undo/redo, KVO, copying, etc.
Should I be using CoreData for an
application that doesn't generally fit
the 'large amount of data/records'
description I would generally use an
SQL style database for?
CoreData isn't restricted to large databases (at all) - it will work well with small sets, and beyond databases (binary files and documents, direct in memory use of models).
your example could benefit from CoreData. it depends on the amount of custom code you need - sometimes it is just easier to write the code if you're just using CD objects as an interface generator, and your app uses a lot of custom code/objects. to be honest, i've never used CoreData in a shipping app - i always found reasons to migrate models to existing code before then (assuming CoreData was also used during development/modeling stages).
it's a nice framework, but it shouldn't be viewed as a 'magic object generator' that will solve most problems. first, you need to understand he technologies/patterns you intend to replace with it. there is a limited number of ideal uses for it. if you can't write the code the objects depend on, don't bother using CoreData. iow - don't consider it as a replacement for initial effort because there are certainly times when it is a good choice and a bad choice - but you can't make an objective answer for your context if you don't (truly) understand what it is capable of.
One of the purposes of Core Data is managing an object graph in memory. This certainly fits your application. It can then be persisted to disk easily. Using a tool such as mogenerator allows you to use Core Data to manage the object life cycle, graph and persistence, but add your custom protocols on top.
In short, yes, you can use Core Data for non-database uses, with a bit of work to conform to the model.

How should I architect my iPhone app to talk to my website?

I'm planning my first iPhone app and I'd like to get some inputs as to how to build it, right from the start. The iPhone app is being built to be paired with a public facing web application that is already built in PHP.
I'd like the web platform to be central (data is housed in a mySQL database), and have the
iPhone clients talk to it and use REST'ful methods to perform the functions of the site
(fetching latest content, posting content, voting, account management as examples).
I'd like the clients to get a local copy of the data in a SQLite database, but refresh to get the latest version of the feed (similar to the Twitter app).
Couple of thoughts I have right now:
Use something like ASIHTTPRequest to send/recieve data to PHP files on the server listening for requests
JSON - would I be better off to send the GET/POSTS to a PHP that returns JSON objects, and work with some sort of wrapper that manages the data and communicates changes to the local SQLite database?
Am I totally off in how I should be building this thing to communicate with the web? Is
there a best practice for this?
I'd really appreciate any input on how you would architect this sort of a setup.
Thank you,
EDIT: After reading my own post again, I know it sounds like a Twitter client, but it is NOT, although it has similar features/structure of a Twitter type setup. Thanks!
As you already outlined in your plan, XML and REST are a great way to communicate with a web application. I want to suggest few details about how to actually design and build it, or what you should keep in mind.
First of all, I believe it's important to stick with MVC. I've seen people creating HTTP connections in view-controllers, controllers being NSXMLParser's delegate, controllers containing data in member variables. I've even seen UITableCells establishing HTTP connections. Don't do it!
Your model and its basic manipulation code should be as much extracted from user interface as possible. As you already have created the model in your web-application, try to recreate the entities in your iPhone project. Don't be afraid of having some simple methods in entity classes, but do not make them use external resources, especially tcp connections. As an example of methods in entity class you might have methods that formats data in specific ways (dates as an example, or returning fullname as concatenation of firstname and surname), or you can even have a method like - (void)update that would act as a wrapper to call class responsible to update the model.
Create another class for updating the model - fetching the XMLs from web-app. Do not even consider using synchronous connections, not even from a dedicated thread. Asynchronous connections with delegate is the way to go. Sometimes multiple requests need to be made to get all required data. You might want to create some kind of state-machine to keep the information about in which stage of downloading you are, and progress from stage to stage, skipping to the end if error occurs, re-executing from failed stage after some moments.
Download data somewhere temporarily, and first when you have it all, make a switch and update user interface. This helps responsiveness during launching the app - user gets to work immediately with data stored locally, while the update mechanism is downloading the new data.
If you need to download lots of files, try to download them simultaneously, if dependencies between files allow for that. This involves creating a connection per request, probably delegate instance for each of them. You can of course have only one delegate instance for all of those connections, but it gets a bit more complex to track the data. Downloading simultaneously might decrease latency considerably, making the mechanism much faster for the user.
To save the time and bandwidth, consider using HTTP's If-Modified-Since and/or ETag headers. Remember the time or tag when you requested the data the last time, and next time send it in HTTP's header. Your web-application should return HTTP code 304 if content has not been changed. iPhone app should react on this code accordingly in connection:didReceiveResponse:.
Create a dedicated class to parse the XML and update the model. You can use NSXMLParser, but if your files are not huge I strongly recommend TouchXML, it's such a pleasure to work with XML as document (it also supports XPath), instead of an event based API. You can use this parser also when files are downloaded to check their validity - re-download if parsing fails. That's when dedicated class for parsing comes handy.
If your dataset is not huge, if you do not need to persist downloaded data on iPhone forever, you probably don't need to store them in SQLite database, you can simply store them in XML format - just a simple caching. That at least might be the way for a twitter app. It gets easier that way, but for bigger data sets XML consumes lots of memory and processing power - in that case SQLite is better.
I'd suggest using Core Data, but you mention this is your first iPhone app, so I suggest you don't use it. Yet.
Do not forget about multitasking - your app can go to sleep in the middle of download, you need to cancel connections, and cleanup your update mechanisms. On app's wake-up you might want to resume the update.
Regarding the view part of the application - use Interface Builder. It might be painful in the beginning, but it pays off in the long run.
View controllers are the glue between model and views. Do not store data in there. Think twice about what to implement where, and who should call it.
This is not related to architecture of the app, but I want to remind that Objective-C is very expressive language. Code should read much like a sentence. Extend classes with protocols. As an example the other day I needed first line of a string. Sure, you can write a one-liner where you find first occurrence of a new-line, and get a substring from beginning till there. But it doesn't look right. I've added - (NSString*)firstLine into my NSString's protocol. Code looks so much better this way, it doesn't need any comments.
There are lots of things to consider in both architecture and design of any project, they both should go hand in hand. If one is causing trouble to the other, you need to adapt. Nothing is written in stone.
I'm currently working on an app that sounds similar to yours. I'd also suggest ASIHTTPRequest, and probably something like TouchJSON for JSON parsing, or extending/making a delegate of NSXMLParser if you want to parse XML.
As suggested by JosephH, depending on how your app works you may want to consider alternate authentication methods: I'd take a look at something token-based like OAuth, which has ready-made libraries for people to dig in to.
SQLite is totally viable for feed caching, although I prefer NSCoding so that you can freeze-dry your custom data structures.
As a general suggestion, make sure to spend a lot of time thinking about every use case and corner case for connections: it's easy to assume a user will only contact the server in certain ways and at certain times, and then after you throw in multitasking/incoming calls/lock screen/memory warnings, things can get hairy without any planning.
All in all, you seem to be on the right track, just make sure you plan out everything beforehand :)
Apple have a brand new in depth piece of sample code - MVCNetworking that shows in depth how to use subclasses of NSHTTPRequests and NSOperationQueues.
As others mentioned, I think you are asking the right questions and are heading in the right direction. All of the replies above are valuable advice. Here is my advice, and I hope you'll find it useful.
No matter which method/library you choose to talk to your web services, I think it's important to make a clean separation in the way you design your data model on the phone VS. the data model in your web application. You have 3 major distinctions to keep in mind for your design:
Data model on the web application (reflected by your existing mySQL database)
Since this is already there, there is not much to say about it, except that it will influence a lot your design for the following 2 parts. I suggest to make this model the 'master reference' for how your data is represented across platforms.
Data model on the iPhone app (reflected by the information you need to display in the iPhone app)
This is where the fun begins. First, you need a good understanding of what data you need to display in the phone app. So have a good, high level design of your app first (use pen and paper, draw mock-ups of each view and the interactions between them, model the navigation between your view controllers etc.). It really helps to understand the interactions between your view controllers and the various bits and pieces of data you want to show in the app. This will help you create the requirements for the data model on the phone. Based on these requirements, map the existing (web) data model to a new model, suited to your iPhone app. This new model may or may not include all tables and fields found in your web app. But the general representation of the 2 models should be very similar (e.g. relationships, data types, etc.)
Data model used to communicate between the 2 above (this is your 'data exchange protocol')
Once you have the 2 representations of your data above, you need to 'translate' from one to the other, both ways. Design your data exchange protocol to be as simple and compact as possible. You don't want to waste bytes on useless information, as transmissions over the network are costly. (As a side note, you might think of compressing the transmitted data later on, but it's just as important to have a good design from the beginning). It's probably best to begin with a protocol in which the metadata is the same as the one in your web application model (e.g. same relationships, names of tables, attributes, etc.). But remember, you'll only have to serialize/de-serialize those entities and relationships that you listed in point 2) above. So design accordingly. Your exchange protocol may also include session tokens, authentication info, a version number, or other metadata, if you need it.
Remember: your data exchange protocol is what will de-couple your web application and iPhone application models. I found that it's best to de-couple them because they may both evolve over time. The data model on the iPhone for example, may evolve a lot especially when you will find that you need to re-model some relationships or add/remove attributes from your entities in order to improve application responsiveness, or the user experience, the navigation, or whatever.
Since this is a whole concern in and by itself, well, you need to design a generic serialization/de-serialization mechanism on top of your (JSON/XML/whatever parser you choose) that is flexible enough to sustain the potential differences between your 2 data models. These differences might be: entity/attribute/relationship names, primary key identifier names, data types, attributes to ignore, and the list goes on. I would definitely implement a serializer/de-serializer utility class in the iPhone app, backed by a .plist configuration file containing all supported entities, concerns, aliases you might have. Of course, each model object should 'know' how to serialize, de-serialize itself and its relationships (i.e. the required object graph depth).
One last note, since you will end up with 2 representations of your data, you will need a way to uniquely identify an object on both sides. So for example, think of adding a uuid attribute to all data that needs to be exchanged, or use any other approach that suits your needs.
I am building an app that has similar requirements to yours, and these are the approaches I found to be best so far. Also, you might find this video useful (it inspired me a lot on how to implement some of the issues I mentioned above and is especially interesting if you're using CoreData) :
http://itunes.apple.com/ca/podcast/linkedin-important-life-lessons/id384233225?i=85092597
(see the lecture entitled "LinkedIn: Important Life Lessons on CoreData & GameKit (March 12, 2010)" )
Good luck!
It's quite a broad question, and I think you're going in the right way anyway, however I'll do my best to give some advice:
JSON, ASIHTTPRequest and POSTs to PHP scripts sound like a great way to go.
If the data is not really sensitive, I'd use http most of the time, and use https only for a login page that either sets a cookie or returns a "token" that you use in subsequent requests. (HTTPS can be quite slow over a 3G connection as the overhead in terms of number of packets to setup an SSL connection is higher than a plain TCP connection.)
You should make sure you correctly pass any data from the input to the PHP scripts to the database, to avoid any SQL injection attacks - ie. used parameterised SQL, don't create sql queries by doing "SELECT * from users where username="+$_GET['username']"
I would do this like I have done with a lot of AJAX web-page stuff. i.e.:
Have a URL on your server side package the information to be transmitted into XML format. (This can be through a CGI/PHP script or whatever). Your transmitting XML in the message body - so it's easy to human read and debug with a standard web browser.
Use the standard iPhone NSXMLParser methods to parse out the individual data fields from the XML doc, and write it back to your database. This method is equiped to both fetch the data from a URL and parse it in one call - like:
NSURL *xmlURL = [NSURL URLWithString:#"http://www.example.com/livefeed.cgi"];
NSXMLParser *myParser = [[NSXMLParser alloc] initWithContentsOfURL:xmlURL];
Walk through the data hierarchy with the NSXMLParser methods and populate your database accordingly.

ORM on iPhone. More simple than CoreData

The question is rather simple. I know that there is SQLite. There is Core Data also. But I need something in between. More object-oriented than SQLite API and simplier than Core Data.
Main points are:
I need access to stored entities only by id. No queries required.
I need to store items of a single type, it means that I can use only one table if I choose SQLite.
I want automatic object-relational conversion. Or object-storage if the storage is not relational.
I can use object archiving, but I have to implement things (NSArchiver).
But I want to write some kind of class and get persistence automatically. As it can be done with Hibernate/ActiveRecord/Core Data/etc.
Thanks.
Everything you've said you want here is completely compatible with Core Data. Apple's giving you a solution that meets your stated needs exactly, so why are you trying to avoid it?
Beyond BNRPersistence, which Alex points out, I don't think you're going to find anything that maintains object relationships, yet is simpler than Core Data on the Cocoa platforms. An object wrapper around SQLite like FMDB still requires you to manage relationships in your own code.
Maintaining relationships between objects is a non-trivial task, which is why you see so few of these frameworks out there. Core Data gets it right for many people, so there isn't that much motivation among developers to build an alternative to Apple's solution. BNRPersistence was created out of Aaron Hillegass' long-time frustration with certain aspects of Core Data, but many people (like me) are perfectly happy with the way Core Data does what it does.
You might also want to look at Core Resource, a newer framework that provides some wrappers around Core Data to make common tasks easier.
You might consider a non-Objective-C approach to serializing objects, just as XML or JSON, where you don't have to write serialization code, if you don't want to, because the framework does it for you. For example, put your objects into a key-value attribute pairing with NSDictionary (via a wrapper class or whatever) that points to another record's id key, and then encode the mess with json-framework's JSONRepresentation call. You'd probably need to do your own relationship integrity tests, but voila, instant relational database.
Take a look at BNRPersistence.