This is my first question so I will do my best to conform to the question guidelines.
I am developing an iPhone app that parses and XML feed to be displayed in a table. Parsing is not a problem but I am not sure of the best way to optimize loading times after initial run of the app.
Here is the different approaches that I am considering:
Parse the XML feed each time the application is loaded. Easy way but possibly longer loading time each run of the app.
Grab the feed and store it locally (as .xml) then parse locally. Then, each time the app is opened, make an http call to see if the feed has been changed. If not, parse locally. If so, download the new feed and parse locally. The initial loading time will be longer but could be cut down on later runs (if the feed as not been updated). This option will be beneficial if the user has a bad signal but needs to see the data.
Parse the feed and store it into a local sqlite db. Then, each time the app is opened, make an http call the see if the feed has been changed. If so, detect which objects have been added/removed and alter local db accordingly. If not, load data from local db. This might be the best option but I am not sure how long finding the changes will take.
My feed is only about 100 or so items, each with roughly 20 fields.
Initial parsing time:
Roughly 4-5sec with full bars.
Roughly 5-7sec with 3 bars.
Any insight as to which option would work best would be very much appreciated.
I think the frequency of the xml data changing should be a factor. If its only going to change once a day/week? Id load it, save it, and check for updates. If update exists download new and overwrite old.
Third solution is clearly the best and it will allow your app to work offline and start quickly. To detect the change, you can simply store a MD5 of the xml file in database and match it against the MD5 of the new XML file. If data has changed, then simply discard all previous data.
Related
Can anyone suggest the best framework, method, library I could use in iPhone (use in development, not a ready application) in order to achieve syncronization between a fileserver and a local storadge on iPhone ?
I am doing this for one of my apps and the way I've implemented it is using a Ruby & Sinatra webserver, talking to a MongoDB database. You could use any other database and webserver technology.
The basic concept is this:
Every time an object in the database is updated, the timestamp is recorded for that object.
A global, last updated timestamp is also updated.
The app contacts the webserver and asks for updates, passing along a locally stored "last updated" timestamp.
The webserver processes the request by first checking the global timestamp and making sure it is older than the app's timestamp. (This is to save having to scour the database if no changes were made to it. My model is: Big data that is not changed frequently. If you have data that changes frequently, then there is probably no benefit to this global timestamp.)
The webserver then finds every object in the database whose timestamp is newer than the app's timestamp.
The webserver packages this up inside a JSON object, and returns it to the app.
This is all RESTful in the sense that it is a stateless transaction, so the app's implementation is very simple (a simple NSURLRequest, followed by JSON decoding, followed by error handling). Now you have an array of updated objects and you can merge these with your local storage in the app.
Another nice point about this (stateless) approach is that you can run it on Heroku (for free).
From what I can tell, there is no easy way. I was looking for an rsync equivalent, but I haven't found one.
In my case, I'm manually walking the tree asking the server for differences after a certain date and I remember the last successful sync date.
Not pretty. Could spend lots of time coming up with something sophisticated.
I want to know what is the best way to store data on the iPhone from a web service.
I want the information to be stored on the device so the person doesn't need to access the web service every time he/she needs it. The currently information isn't much and contains less that 150 records. The records might update from time to time and a few new ones will be added. What is the best way to go about storing the data?
Thanks
If you use ASIHTTPRequest for your network stuff (and if you don't already, I can't sing its praises highly enough), you will find it has a cache layer built in which is perfect for situations like this.
You can activate it with a simple one line;
[ASIHTTPRequest setDefaultCache:[ASIDownloadCache sharedCache]];
And you have full control over the cache policy etc - just read the documentation.
The other simple approach of course is - on the assumption that your web service is returning JSON or XML - simply to store the response in a local file against a hash of the request parameters, then when you request the data again, you can first look to see if the file exists and if it does, return that data rather than going back to the website. You can roll your own cache policies etc too.
Since I discovered ASIHTTPRequest had a cache though, I've not needed to roll my own again.
I find that using coreData or sqllite3 is just overkill for 99% my requirements and a simple cache works very well.
If the data is relational, a Sqlite3 database would be the best storage option you have.
Also, this helps by allowing you to retrieve from the server and to update only the records that have changed, thus saving time and bandwidth.
This is the best option from a scalability point of view as well, as you stated that "current information isn't much", thus giving the impression that this is only a current situation, that may be subjected to further change, probably towards more records being added in time.
Sqite3 also gives you more control and better performance than using, for instance, Core Data. Here's an article explaining some of the details. Moreover, if you work through an Objective-C wrapper, such as FMDB, you get all the advantages without managing the complexity yourself.
I'm using MonoTouch and also System.Data to create a DataSet (just xml to those not familiar) for simple data binding. Data on my app is minimal so no need to go all out with SQLLite. The dataset use makes it easy to pass via web services for cloud sync.
I serialize the DataSet to the personal folder on save and of course read this file when the app starts up to load up the user's data. I've had issues where this file is becoming corrupt and I'm not sure why. I assume file I/O may be slow on these devices and that could be the cause, I'm not sure, but it is happening.
I'm also concerned that maybe iTunes is passing this file back and forth between the PC/MAC when the user syncs their devices with iTunes, which may be the cause of the corruption?
I want to prevent this device file from syncing with iTunes and also reliably persist it. I'm using the NSFile.Save option to save it to the device. I'm thinking since it's a text file maybe I could more safely store it in the standard user settings area instead? This would prevent it from being synced by itunes, I presume?
What is the most reliable and safe way to handle this file i/o for the xml dataset storage?
Thank you.
You're using MonoTouch. Isn't it simply a matter of calling DataSet.WriteXml() with a FileStream object ready to write to a document in your Documents folder?
That Documents folder is backed up to iTunes. It's not synced, but it helps if your user is restoring their phone (because they bricked it, lost it, whatever). It doesn't explain why it's corrupt.
The only thing that I can think of why it's corrupt is because it took too long for your app to write it. There's a limited time from the point where the user exits the app until it's closed down, to prevent apps from keeping the system hostage and deteriorate user experience.
If writing the whole dataset takes too long, you want to think about minimizing that. Perhaps you can just store the data, and not the schema. Or you can devise a way to store only the deltas on exit and reconcile when the user has loaded your app again.
You can also prevent complete loss of data by writing to a second file, and when that operation completes delete the old file and rename. That way, the next time you start up if the write operation didn't complete, the old file would still be there and the user would have only lost their more recent changes.
In any case, if your data gets too big for a simple write operation to complete, you should look at different options such as sqlite.
Your best bet is probably to just save the XML as text. It's as simple as File.WriteAllText(...) - there's no reason to go to NSFile for this. That's part of the advantage of MonoTouch :)
Regarding syncing, here's the rule:
If you keep the file in the user's documents folder (Environment.SpecialFolder.MyDocuments and Environment.SpecialFolder.Personal BOTH point to the user's doc folder), then it's going to get backed up whenever the user syncs with iTunes.
There's nothing wrong with this. It persists the data between sessions and makes it recoverable if something goes wrong with the user's phone and they need to restore from a backup. Since your question is about persisting an XML file on the phone, this is what you want.
As for the iTunes question, there's no problem with speed and syncing because your app isn't going to be running while the phone is syncing. The file will either have been saved or it won't. Any corruption that takes place is happening while your app is running.
Reasons for files getting corrupted can include:
Not saving before the user quits. You get a chance to do this.
Not gracefully handling an incoming phone call. The system warns you about this as well.
iTunes definitely isn't corrupting your file. If that were the case, iOS apps would all be broken. It could be happening on your dev machine for whatever reason, but I've never seen this happen elsewhere, and I've done quite a bit of iOS development.
If you'd like a tutorial on reading and writing files, I posted an answer in another question.
It's lengthy, but the point was to answer as many questions as I could so nobody would be left hanging or confused.
A nice thing about iOS devices is that you're back (for most apps) in the one-person-at-a-time world. You're writing apps where you don't have to worry about 5,000 people trying to use your web-based app at the same time (that's not always true, but... you get the point). As a result, you can do things that you might normally consider bad for performance, but you're unlikely to see any performance problems (as long as the file you're saving is either small enough to be saved quickly or you're saving in the background on another thread - you never want to block the main (UI) thread with a heavy IO operation).
If you have any more questions, feel free to ask.
I hope this helps :)
Lots of frameworks are read-only, but I've found that GDataXMLNode from http://code.google.com/p/gdata-objectivec-client/ works very well read/write.
Having said that, on the iPhone you'd do yourself a big favour using Core Data with a SQLLite backend. :-) Apple has done all this for you and optimized it more than any of us every will
Cheers
Nik
Consider using SQLite, I'd go for something like
http://www.taimila.com/entify/ (not tried yet)
catnap orm
or Faks sqlite-net on google code (using this in a few apps)
entify - if it does what it says it can do - looks really good.
persisting XML on the iPhone as a means to store and access data is a nause you dont want to get into. I wrote about it here http://iwayneo.blogspot.com/2010/08/festival-star-history-serialization-as.html
I am working on a regular iPhone app which pulls data from a server (XML, JSON, etc...), and I'm wondering what is the best way to implement synching data. Criteria are speed (less network data exchange), robustness (data recovery in case update fails), offline access and flexibility (adaptable when the structure of the database changes slightly, like a new column). I know it varies from app to app, but can you guys share some of your strategy/experience?
For me, I'm thinking of something like this:
1) Store Last Modified Date in iPhone
2) Upon launching, send a message like getNewData.php?lastModifiedDate=...
3) Server will process and send back only modified data from last time.
4) This data is formatted as so:
<+><data id="..."></data></+> // add this to SQLite/CoreData
<-><data id="..."></data></-> // remove this
<%><data id="..."><attribute>newValue</attribute></data></%> // new modified value
I don't want to make <+>, <->, <%>... for each attribute as well, because it would be too complicated, so probably when receive a <%> field, I would just remove the data with the specified id and then add it again (assuming id here is not some automatically auto-incremented field).
5) Once everything is downloaded and updated, I will update the Last Modified Date field.
The main problem with this strategy is: If the network goes down when I am updating something => the Last Modified Date is not yet updated => next time I relaunch the app, I will have to go through the same thing again. Not to mention potential inconsistent data. If I use a temporary table for update and make the whole thing atomic, it would work, but then again, if the update is too long (lots of data change), the user has to wait a long time until new data is available. Should I use Last-Modified-Date for each of the data field and update data gradually?
I would start by making the update routine atomic, since you'll have enough on your hands figuring out how to get the client-server communication working properly.
After that is a good time to consider tweaking it to be incremental, but only after you do some testing to figure out if it's really necessary. If you're tuning your update protocol to be as low bandwidth as possible, you might discover that even a "big" update is downloaded fast enough.
Another way to look at it is to ask yourself, how often is there going to be network trouble when an average user is doing a sync? You probably don't want to tune for unlikely scenarios.
If you are trying to optimize (minimize) the data transfer you may want to consider a different format than XML, since XML is fairly verbose. Or at least you may want to trade in XML readability for space by making each element name and attribute as small as possible, and eliminate all unnecessary whitespace.
Your basic scheme is good. The thing you need to do is to somehow make your updates idempotent so that you can restart a partially-completed transfer without risk. This is a better way to go than to try to implement some sort of true atomic commit (though you could do that too, using, eg, the SQLite database).
In our experience fairly large updates (10s of KB) can be downloaded quite rapidly, if the server is fast enough. No great need to break updates up into tiny bits. But certainly it won't hurt to try to minimize the amount of data transferred by keeping more granular info on "last update".
(And definitely you should use JSON rather than XML as your transmitted data representation.)
Wonder if you have considered using a Sync Framework to manage the synchronization. If that interests you can take a look at the open source project, OpenMobster's Sync service. You can do the following sync operations
two-way
one-way client
one-way device
bootup
Besides that, all modifications are automatically tracked and synced with the Cloud. You can have your app offline when network connection is down. It will track any changes and automatically in the background synchronize it with the cloud when the connection returns. It also provides synchronization like iCloud across multiple devices
Also, modifications in the Cloud are synched using Push notifications, so the data is always current even if it is stored locally.
In your case,
Criteria are speed (less network data exchange), robustness (data recovery in case update fails), offline access
Speed: Only the changes are sent across the network in both directions
Robustness: It stores data in a transactional store like sqlite and any failed updates are communicated in the SyncML payload. Only the successful operations are processed while the failed operations are re-tried during the next sync
Here is a link to the open source project: http://openmobster.googlecode.com
Here is a link to iPhone App Sync: http://code.google.com/p/openmobster/wiki/iPhoneSyncApp
I have created an app which displays information in a organized manner about cultural places.
The information is subject to changes, so I want it to be downloaded from the web. But not everytime. Only once in a while, because information doesn't change often.
What I want to do is, the first time the user opens the application, it downloads all data from the web. For the moment, I parse it from an xml (which is about 100Ko), and I get a NSMutableArray of "CulturalPlace" objects. but it is very slow. And what I would like to do is, to store this data locally (in case the user has an iPod touch an is not on a wifi, or if he is on EDGE and does not want to redownload all). So the user updates data only by clicking an "update button" on the top right of the screen. Otherwise it reads it from disk.
I really don't know what could be the best solution. I thought about Core Data, but I have several Tableview imbricated (Rootviewcontroller > ListofPlacesViewController > PlaceViewController) and I really cannot find good tutorial for a simple use like mine. (the iTunes "TopSongs" sample code seems too complex).
I thought also about not parsing the xml, but instead try an NSURLConnection and get a plist file. But I never managed to read anything from the local file.
So my main question is, should I keep the xml parsing method, or should I use another format to tranfert the data from the web? And what is the best way to store and read data like an NSMutableArray of custom Objects ?
Thanks in advance for your help, sorry for my approximate english.
You could use HTML5' localStorage. It's supported by Chrome and FF on the PC and Safari on Mac OS and iPhone (to the best of my knowledge). It acts like a local database. Bear in mind that if the user selects to clear all cookies (or "private settings"), your storage goes away.
You could opt to store the XML locally, and store in NSUserDefaults the date when last updated - then on app launch you can check to see if you have a new file.
ASIHTTPRequest makes it pretty easy to say "Save the contents of this URL to a file". So you'd always save the XML to a file, and always read from that file or fetch XML if it was not yet there.
In my experience XML is indeed much slower to parse than plist, even though they're technically the same thing. Fortunately, plist's are pretty easy to deal with and the API's take care of all of the archiving and de-archiving.
Once you have your data in memory, it probably wouldn't be too hard to convert it to the much faster plist representation, check out this doc for more info: http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/Archiving/Archiving.html
If your app is divided into different pages, you might also consider splitting the file into separate files, and only parsing / de-archiving the information you need when you need it (if you did this on a separate thread and displayed a UIProgressView on the main thread, the delay would probably be barely an issue to the user).