I'm using MonoTouch and also System.Data to create a DataSet (just xml to those not familiar) for simple data binding. Data on my app is minimal so no need to go all out with SQLLite. The dataset use makes it easy to pass via web services for cloud sync.
I serialize the DataSet to the personal folder on save and of course read this file when the app starts up to load up the user's data. I've had issues where this file is becoming corrupt and I'm not sure why. I assume file I/O may be slow on these devices and that could be the cause, I'm not sure, but it is happening.
I'm also concerned that maybe iTunes is passing this file back and forth between the PC/MAC when the user syncs their devices with iTunes, which may be the cause of the corruption?
I want to prevent this device file from syncing with iTunes and also reliably persist it. I'm using the NSFile.Save option to save it to the device. I'm thinking since it's a text file maybe I could more safely store it in the standard user settings area instead? This would prevent it from being synced by itunes, I presume?
What is the most reliable and safe way to handle this file i/o for the xml dataset storage?
Thank you.
You're using MonoTouch. Isn't it simply a matter of calling DataSet.WriteXml() with a FileStream object ready to write to a document in your Documents folder?
That Documents folder is backed up to iTunes. It's not synced, but it helps if your user is restoring their phone (because they bricked it, lost it, whatever). It doesn't explain why it's corrupt.
The only thing that I can think of why it's corrupt is because it took too long for your app to write it. There's a limited time from the point where the user exits the app until it's closed down, to prevent apps from keeping the system hostage and deteriorate user experience.
If writing the whole dataset takes too long, you want to think about minimizing that. Perhaps you can just store the data, and not the schema. Or you can devise a way to store only the deltas on exit and reconcile when the user has loaded your app again.
You can also prevent complete loss of data by writing to a second file, and when that operation completes delete the old file and rename. That way, the next time you start up if the write operation didn't complete, the old file would still be there and the user would have only lost their more recent changes.
In any case, if your data gets too big for a simple write operation to complete, you should look at different options such as sqlite.
Your best bet is probably to just save the XML as text. It's as simple as File.WriteAllText(...) - there's no reason to go to NSFile for this. That's part of the advantage of MonoTouch :)
Regarding syncing, here's the rule:
If you keep the file in the user's documents folder (Environment.SpecialFolder.MyDocuments and Environment.SpecialFolder.Personal BOTH point to the user's doc folder), then it's going to get backed up whenever the user syncs with iTunes.
There's nothing wrong with this. It persists the data between sessions and makes it recoverable if something goes wrong with the user's phone and they need to restore from a backup. Since your question is about persisting an XML file on the phone, this is what you want.
As for the iTunes question, there's no problem with speed and syncing because your app isn't going to be running while the phone is syncing. The file will either have been saved or it won't. Any corruption that takes place is happening while your app is running.
Reasons for files getting corrupted can include:
Not saving before the user quits. You get a chance to do this.
Not gracefully handling an incoming phone call. The system warns you about this as well.
iTunes definitely isn't corrupting your file. If that were the case, iOS apps would all be broken. It could be happening on your dev machine for whatever reason, but I've never seen this happen elsewhere, and I've done quite a bit of iOS development.
If you'd like a tutorial on reading and writing files, I posted an answer in another question.
It's lengthy, but the point was to answer as many questions as I could so nobody would be left hanging or confused.
A nice thing about iOS devices is that you're back (for most apps) in the one-person-at-a-time world. You're writing apps where you don't have to worry about 5,000 people trying to use your web-based app at the same time (that's not always true, but... you get the point). As a result, you can do things that you might normally consider bad for performance, but you're unlikely to see any performance problems (as long as the file you're saving is either small enough to be saved quickly or you're saving in the background on another thread - you never want to block the main (UI) thread with a heavy IO operation).
If you have any more questions, feel free to ask.
I hope this helps :)
Lots of frameworks are read-only, but I've found that GDataXMLNode from http://code.google.com/p/gdata-objectivec-client/ works very well read/write.
Having said that, on the iPhone you'd do yourself a big favour using Core Data with a SQLLite backend. :-) Apple has done all this for you and optimized it more than any of us every will
Cheers
Nik
Consider using SQLite, I'd go for something like
http://www.taimila.com/entify/ (not tried yet)
catnap orm
or Faks sqlite-net on google code (using this in a few apps)
entify - if it does what it says it can do - looks really good.
persisting XML on the iPhone as a means to store and access data is a nause you dont want to get into. I wrote about it here http://iwayneo.blogspot.com/2010/08/festival-star-history-serialization-as.html
Related
In Apple's Performance Tuning Guide there is a writing:
Avoid writing cache files to disk. The only exception to this rule is
when your app quits and you need to write state information that can
be used to put your app back into the same state when it is next
launched.
I'm saving a lot of cache files in Library/Cache directory, because my app deals with web services, and nobody likes the white screen. What does this statement mean? I shouldn't do this or what?
Thank you!
Well, "avoid" means "avoid if possible, because writing/reading is relatively slow". If by caching a small amount of data (I assume the definitions of the web services retrieved from somewhere?) you can improve the performance of your app's startup, by all means do it. If you are only using this data for one run of your application, and the next run will re-fetch this anyway, use an in-memory cache.
Library\Caches is basically designed to store data you fetched from somewhere which provides performance boosts when stored locally.
The text from Apple feels like more a general guideline against overusing storage if you don't need data to persist from one run of your application to another.
I just had a chat with a member of the App review team and I was told, that providing a DB 10 MB is far to large for an app to be approved. The lady told me, that the issue with my app is that I am not compliant with guideline 2.23 of the storage guidelines.
They, at least the person I talked to on the phone, claimed, that it an app copying a database on first launch of 10 MB will not be approved by Apple.
I am copying my Database which contains a set of > 50'000 records from the resource folder into the Library folder. I am segregating the databases into 2 DBs, one DB, the user is actually applying the changes, and another DB, where the user can copy data from.
Question: Is there another way instead of copying a DB from the resource dir? I decided the resource-folder to library-folder approach, as this is simply the fastest and seemed to me the most user-friendly approach. Of course I can make the user download the DBs from the internet. But this is yet another process step, which I personally would not like to undertake as an end-user.
I still don't have any specs from Apple, what the maximum allowed size of a first time launch DB is. Don't know, why Apple makes such a fuzz about this information.
Anyway, I would really appreciate your comments and possible solution approaches.
René
I may mention here, that my solution to the issue was, not copying the static database at all. I simply leave it in the bundle and read the data from there. As it is read-only data the app is fine like this. And Apple was happy. Got it approved after that change. Thanks to all for the support.
René
I had a similar issue. The correct approach to take here is to move the file to Library/Caches folder. That folder is cleaned up when there is low memory situation and also never backed up to iCloud. I have an app which has 36 MB of database file. But I dont bundle it with the app. When the app is opened for the first time The app tells the user to download the DB once. This way the app size is drastically reduced.
Any specific reason why you are bundling the db with the app?
I've been working on an app for quite a while and suddenly started to hit this error when the app tries to open a Core Data store. I hadn't made any changes to my data model or the data access code for over a month, so I don't think it can be anything that I'm doing wrong as far as interacting with Core Data. (Meaning, the URLs are ok, the call pattern is ok, etc...)
Interestingly, these are the log lines immediately before the error:
/SourceCache/GoogleMobileMaps/GoogleMobileMaps-217.2/googlenav/mac/TileStore.mm:209 unable to open /var/mobile/Library/Caches/MapTiles/MapTiles.sqlitedb: (14) unable to open database file
/SourceCache/GoogleMobileMaps/GoogleMobileMaps-217.2/googlenav/mac/TileStore.mm:155 file doesn't exist /var/mobile/Library/Caches/MapTiles/MapTiles.sqlitedb: (2)
/SourceCache/GoogleMobileMaps/GoogleMobileMaps-217.2/googlenav/mac/TileStore.mm:209 unable to open /var/mobile/Library/Caches/MapTiles/MapTiles.sqlitedb: (14) unable to open database file
/SourceCache/GoogleMobileMaps/GoogleMobileMaps-217.2/googlenav/mac/TileStore.mm:235 unable to open /var/mobile/Library/Caches/MapTiles/MapTiles.sqlitedb: tile data will not be cached
So it looks like there is "something" wrong with the sqlite layer in general. Has anybody seen this before? Is there a recovery option besides wiping my device? It's currently running 3.1.3 and I'd really hate to upgrade to 4 because it's currently my only way to test that the app will run for people who haven't upgraded.
One thing I did notice: shortly after I first hit this error, I wanted to see if any other apps were having problems. Sure enough, the iPod app had forgotten everything about me, but it was able to recover after syncing. So maybe there is some recovery mode? (Although, even if I can recover for my app, the Maps APIs might burn a lot of bandwidth if they can't cache the map tiles...)
Ryan
For what it's worth, I found the culprit and it has nothing to do with Core Data, sqlite, or the file system. The app uses a lot of small audio clips and I was pre-caching them all as AVAudioPlayers. I knew this was probably a bad idea, but it was quick and easy so I figured I'd keep doing it that way until I hit some kind of problem. (I'd put a wrapper around the players so that I could delay instantiation if required without affecting the rest of the system, which is what I'm doing now.) I just assumed the problem would show up as an audio player problem and not somewhere else seemingly totally unrelated.
I realized there must be a code error when I found the simulator also misbehaving, but in a different totally inexplicable way (keyed archives weren't being written properly). When I backed out of the most recent change I'd made (adding a new batch of audio clips), the problems vanished.
Hopefully this helps someone in the future!
I have created an app which displays information in a organized manner about cultural places.
The information is subject to changes, so I want it to be downloaded from the web. But not everytime. Only once in a while, because information doesn't change often.
What I want to do is, the first time the user opens the application, it downloads all data from the web. For the moment, I parse it from an xml (which is about 100Ko), and I get a NSMutableArray of "CulturalPlace" objects. but it is very slow. And what I would like to do is, to store this data locally (in case the user has an iPod touch an is not on a wifi, or if he is on EDGE and does not want to redownload all). So the user updates data only by clicking an "update button" on the top right of the screen. Otherwise it reads it from disk.
I really don't know what could be the best solution. I thought about Core Data, but I have several Tableview imbricated (Rootviewcontroller > ListofPlacesViewController > PlaceViewController) and I really cannot find good tutorial for a simple use like mine. (the iTunes "TopSongs" sample code seems too complex).
I thought also about not parsing the xml, but instead try an NSURLConnection and get a plist file. But I never managed to read anything from the local file.
So my main question is, should I keep the xml parsing method, or should I use another format to tranfert the data from the web? And what is the best way to store and read data like an NSMutableArray of custom Objects ?
Thanks in advance for your help, sorry for my approximate english.
You could use HTML5' localStorage. It's supported by Chrome and FF on the PC and Safari on Mac OS and iPhone (to the best of my knowledge). It acts like a local database. Bear in mind that if the user selects to clear all cookies (or "private settings"), your storage goes away.
You could opt to store the XML locally, and store in NSUserDefaults the date when last updated - then on app launch you can check to see if you have a new file.
ASIHTTPRequest makes it pretty easy to say "Save the contents of this URL to a file". So you'd always save the XML to a file, and always read from that file or fetch XML if it was not yet there.
In my experience XML is indeed much slower to parse than plist, even though they're technically the same thing. Fortunately, plist's are pretty easy to deal with and the API's take care of all of the archiving and de-archiving.
Once you have your data in memory, it probably wouldn't be too hard to convert it to the much faster plist representation, check out this doc for more info: http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/Archiving/Archiving.html
If your app is divided into different pages, you might also consider splitting the file into separate files, and only parsing / de-archiving the information you need when you need it (if you did this on a separate thread and displayed a UIProgressView on the main thread, the delay would probably be barely an issue to the user).
I am writting a django app and Iphone app, I need to keep them in sync.
Users can delete, update and create new objects in the web app, and in the iphone app.
When they get online with the iphone both app must be in sync.
Is there simple way to do this?
Thanks,
Joaquin
In general: There's no simple way. But I'll outline an approach.
If you don't care about changes being overwritten: Keep a timestamp of the most recent change to each record, and a timestamp of each sync. When syncing, you get a list of all updates on the iPhone since the last sync, and all updates on the server. You write from the iPhone to the server if the iPhone timestamp for that record is newer than the server one, and vice versa.
But you probably care. Say you've edited a note called "Where to meet up on Friday." It started out empty. Now, on the phone, you've written, "My house." Ten minutes later, your friend edits the same note on the server and writes, "The diner." Who wins out? Stack Overflow can't answer that for you; it's application-specific.
OK, so modify the approach above: if both the server version of a record and the local version have been edited since the last sync, then you have to ask the user what to do. That's the basic algorithm.
If you care a lot about changes not being overwritten, to the point that you want to merge changes to different places in the same documents, then your system will begin to approach the complexity of version control systems like Subversion or Git. Not at all simple.
There's no built in way to do this. You need to keep a server data store, and a local data store on the iPhone, and when online, check the differences manually, and see what action you should take on the server and the iPhone side (delete, update, etc.).
Sync is usually hard. I suggest you start laying out the server and iPhone data stores, and think how they relate, and how can the server or the iPhone know the status of their counterpart record, so to keep them in sync.