In Apple's Performance Tuning Guide there is a writing:
Avoid writing cache files to disk. The only exception to this rule is
when your app quits and you need to write state information that can
be used to put your app back into the same state when it is next
launched.
I'm saving a lot of cache files in Library/Cache directory, because my app deals with web services, and nobody likes the white screen. What does this statement mean? I shouldn't do this or what?
Thank you!
Well, "avoid" means "avoid if possible, because writing/reading is relatively slow". If by caching a small amount of data (I assume the definitions of the web services retrieved from somewhere?) you can improve the performance of your app's startup, by all means do it. If you are only using this data for one run of your application, and the next run will re-fetch this anyway, use an in-memory cache.
Library\Caches is basically designed to store data you fetched from somewhere which provides performance boosts when stored locally.
The text from Apple feels like more a general guideline against overusing storage if you don't need data to persist from one run of your application to another.
Related
I have a document that holds a big data structure in certain fields inside an array, it is slowing down my application due to frequent hits to read such data. am thinking on few solutions to implement but I need advice before i proceed and possibly even a better solution, here are my thoughts/questions:
would it help to cache data?
should I use memcached or redis as a caching engine and why?
would it help to read single fields from this document instead of reading it all every time?
should I do something else?!
Caching will help because it would avoid your db to be hit too often
Memcache or redis it's up to you. I prefere redis but if you already have a memcache it's fine.
If you have a cluster of servers, think if you need a centralized cache or not
Caching a full document won't help for getting a single field because you cache the result of a query without knowing what it contains.
your question need more clarification. for example how big is the data that you are speaking of is it couple of megabytes or gigabytes. All these factors change the solution. But if we consider that you have couple of megabytes and you want to prevent to call database every time the best solution is cache. How to choose a cache is also completely depends on what is your situation. If your web application runs on one server you can use the in-memory cache like ASP.Net cache which is very quick and fast for in-memory cache. this cache is stored in your heap so you can put all your object in the cache without serialization.But consider that whenever your application is restarted like most of deployments. your heap will be deleted and all the cache is cleared inside the heap.
if you have more than one server then you can start to think about an out-of-memory cache because two servers are not sharing heap memory and using all in-memory cache are useless because it duplicate the data and invalidating is nightmare. However, this is more reliable cache while it is not in the heap and in term of persistence is more than in-memory cache. But whatever you want to put in this kind of cache should be serializable while you are transferring the object over network connection. So you cannot put all your object in cache. Both Redis and memcached can be used for this purpose. Redis is more complicated with more functionality than Memcached but for your purpose memcached is quite good.
Whatever caching system you choose, approach it in a wide perspective. Design a caching system in your application while over time you need to put more things in cache. so its better to prepare everything for that time from now.
another things which is very important in cache is that whenever you set something in cache you have to consider when you are going to invalidate it.
Whether or not caching will help depends on the accession of the document. If the document is being accessed multiple times then caching will not help due to how MongoDB to memory caching actually works.
First, you need to understand your data accession patterns.
Can people give me examples of why they would use coreData in an application?
I ask this because most apps are just clients to a central server where an API of some sort gives you the information you need.
In my case I'm writing a timesheet application for a web app which has an API and I'm debating if there is any value in replicating the data structure on my server in core data(Sqlite)
e.g
Project has many timesheets
employee has many timesheets
It seems to me that I can just connect to the API on every call for lists of projects or existing timesheets for example.
I realize for some kind of offline mode you could store locally in core data but this creates way more problems because you now have a big problem with syncing that data back to the web server when you get connection again.. e.g. the project selected for a timesheet no longer exists.
Can any experienced developer shed some light on there experiences on when core data is best practice approach?
EDIT
I realise of course there is value in storing local persistance but the key value of user defaults seems to cover most applications I can think of.
You shouldn't think of CoreData simply as an SQLite database. It's not JUST an SQLite database. Sure, SQLite is an option, but there are other options as well, such as in-memory and, as of iOS5, a whole slew of custom data stores. The biggest benefit with CoreData is persistence, obviously. But even if you are using an in-memory data store, you get the benefits of a very well structured object graph, and all of the heavy lifting with regards to pulling information out of or putting information into the data store is handled by CoreData for you, without you necessarily needing to concern yourself with what is backing that data store. Sure, today you don't care too much about persistence, so you could use an in-memory data store. What happens if tomorrow, or in a month, or a year, you decide to add a feature that would really benefit from persistence? With CoreData, you simply change or add a persistent data store, and all of your methods to get information out or in remain unchanged. The overhead for that sort of addition is minimal in comparison to if you were trying to access SQLite or some other data store directly. IMHO, that's the biggest benefit: abstraction. And, in essence, abstraction is one of the most powerful things behind OOP. Granted, building the Data Model just for in-memory storage could be overkill for your app, depending on how involved the app is. But, just as a side note, you may want to consider what is faster: Requesting information from your web service every time you want to perform some action, or requesting the information once, storing it in memory, and acting on that stored value for the remainder of the session. An in-memory data store wouldn't persistent beyond that particular session.
Additionally, with CoreData you get a lot of other great features like saving, fetching, and undo-redo.
There are basically two kinds of apps. Those that provide you with local functionality (games, professional applications, navigation systems...) and those that grant access to a remote service.
Your app seems to be in the second category. If you access remote services, your users will want to access new or real-time data (you don't want to read 2 week old Facebook posts) but in some cases, local caching makes sense (e.g. reading your mails when you're on the train with unstable network).
I assume that the value of accessing cached entries when not connected to a network is pretty low for your customers (internal or external) compared to the importance of accessing real-time-data. So local storage might be not necessary at all.
If you don't have hundreds of entries in your timetable, "normal" serialization (NSCoding-protocol) might be enough. If you only access some "dashboard-data", you will be able to get along with simple request/response-caching (NSURLCache can do a lot of things...).
Core Data does make more sense if you have complex data structures which should be synchronized with a server. This adds a lot of synchronization logic to your project as well as complexity from Core Data integration (concurrency, thread-safety, in-app-conflicts...).
If you want to create a "client"-app with a server driven user experience, local storage is not necessary at all so my suggestion is: Keep it as simple as possible unless there is a real need for offline storage.
It's ideal for if you want to store data locally on the phone.
Seriously though, if you can't see a need for it for your timesheet app, then don't worry about it and don't use it.
Solving the sync problems that you would have with an "offline" mode would be detailed in your design of your app. For example - don't allow projects to be deleted. Why would you? Wouldn't you want to go back in time and look at previous data for particular projects? Instead just have a marker on the project to show it as inactive and a date/time that it was made inactive. If the data that is being synced from the device is for that project and is before the date/time that it was marked as inactive, then it's fine to sync. Otherwise display a message and the user will have to sort it.
It depends purely on your application's design whether you need to store some data locally or not, if it is a real problem or a thin GUI client around your web service. Apart from "offline" mode the other reason to cache server data on client side might be to take traffic load from your server. Just think what does it mean for your server to send every time the whole timesheet data to the client, or just the changes. Yes, it means more implementation on both side, but in some cases it has serious advantages.
EDIT: example added
You have 1000 records per user in your timesheet application and one record is cca 1 kbyte. In this case every time a user starts your application, it has to fetch ~1Mbyte data from your server. If you cache the data locally, the server can tell you that let's say two records were updated since your last update, so you'll have to download only 2 kbyte. Now you should scale up this for several tens of thousands of user and you will immediately notice the difference of the server bandwidth and CPU usage.
I want to know what is the best way to store data on the iPhone from a web service.
I want the information to be stored on the device so the person doesn't need to access the web service every time he/she needs it. The currently information isn't much and contains less that 150 records. The records might update from time to time and a few new ones will be added. What is the best way to go about storing the data?
Thanks
If you use ASIHTTPRequest for your network stuff (and if you don't already, I can't sing its praises highly enough), you will find it has a cache layer built in which is perfect for situations like this.
You can activate it with a simple one line;
[ASIHTTPRequest setDefaultCache:[ASIDownloadCache sharedCache]];
And you have full control over the cache policy etc - just read the documentation.
The other simple approach of course is - on the assumption that your web service is returning JSON or XML - simply to store the response in a local file against a hash of the request parameters, then when you request the data again, you can first look to see if the file exists and if it does, return that data rather than going back to the website. You can roll your own cache policies etc too.
Since I discovered ASIHTTPRequest had a cache though, I've not needed to roll my own again.
I find that using coreData or sqllite3 is just overkill for 99% my requirements and a simple cache works very well.
If the data is relational, a Sqlite3 database would be the best storage option you have.
Also, this helps by allowing you to retrieve from the server and to update only the records that have changed, thus saving time and bandwidth.
This is the best option from a scalability point of view as well, as you stated that "current information isn't much", thus giving the impression that this is only a current situation, that may be subjected to further change, probably towards more records being added in time.
Sqite3 also gives you more control and better performance than using, for instance, Core Data. Here's an article explaining some of the details. Moreover, if you work through an Objective-C wrapper, such as FMDB, you get all the advantages without managing the complexity yourself.
I am writing an app for iOS that uses data provided by a web service. I am using core data for local storage and persistence of the data, so that some core set of the data is available to the user if the web is not reachable.
In building this app, I've been reading lots of posts about core data. While there seems to be lots out there on the mechanics of doing this, I've seen less on the general principles/patterns for this.
I am wondering if there are some good references out there for a recommended interaction model.
For example, the user will be able to create new objects on the app. Lets say the user creates a new employee object, the user will typically create it, update it and then save it. I've seen recommendations that updates each of these steps to the server --> when the user creates it, when the user makes changes to the fields. And if the user cancels at the end, a delete is sent to the server. Another different recommendation for the same operation is to keep everything locally, and only send the complete update to the server when the user saves.
This example aside, I am curious if there are some general recommendations/patterns on how to handle CRUD operations and ensure they are sync'd between the webserver and coredata.
Thanks much.
I think the best approach in the case you mention is to store data only locally until the point the user commits the adding of the new record. Sending every field edit to the server is somewhat excessive.
A general idiom of iPhone apps is that there isn't such a thing as "Save". The user generally will expect things to be committed at some sensible point, but it isn't presented to the user as saving per se.
So, for example, imagine you have a UI that lets the user edit some sort of record that will be saved to local core data and also be sent to the server. At the point the user exits the UI for creating a new record, they will perhaps hit a button called "Done" (N.B. not usually called "Save"). At the point they hit "Done", you'll want to kick off a core data write and also start a push to the remote server. The server pus h won't necessarily hog the UI or make them wait till it completes -- it's nicer to allow them to continue using the app -- but it is happening. If the update push to server failed, you might want to signal it to the user or do something appropriate.
A good question to ask yourself when planning the granularity of writes to core data and/or a remote server is: what would happen if the app crashed out, or the phone ran out of power, at any particular spots in the app? How much loss of data could possibly occur? Good apps lower the risk of data loss and can re-launch in a very similar state to what they were previously in after being exited for whatever reason.
Be prepared to tear your hair out quite a bit. I've been working on this, and the problem is that the Core Data samples are quite simple. The minute you move to a complex model and you try to use the NSFetchedResultsController and its delegate, you bump into all sorts of problems with using multiple contexts.
I use one to populate data from your webservice in a background "block", and a second for the tableview to use - you'll most likely end up using a tableview for a master list and a detail view.
Brush up on using blocks in Cocoa if you want to keep your app responsive whilst receiving or sending data to/from a server.
You might want to read about 'transactions' - which is basically the grouping of multiple actions/changes as a single atomic action/change. This helps avoid partial saves that might result in inconsistent data on server.
Ultimately, this is a very big topic - especially if server data is shared across multiple clients. At the simplest, you would want to decide on basic policies. Does last save win? Is there some notion of remotely held locks on objects in server data store? How is conflict resolved, when two clients are, say, editing the same property of the same object?
With respect to how things are done on the iPhone, I would agree with occulus that "Done" provides a natural point for persisting changes to server (in a separate thread).
I'm using MonoTouch and also System.Data to create a DataSet (just xml to those not familiar) for simple data binding. Data on my app is minimal so no need to go all out with SQLLite. The dataset use makes it easy to pass via web services for cloud sync.
I serialize the DataSet to the personal folder on save and of course read this file when the app starts up to load up the user's data. I've had issues where this file is becoming corrupt and I'm not sure why. I assume file I/O may be slow on these devices and that could be the cause, I'm not sure, but it is happening.
I'm also concerned that maybe iTunes is passing this file back and forth between the PC/MAC when the user syncs their devices with iTunes, which may be the cause of the corruption?
I want to prevent this device file from syncing with iTunes and also reliably persist it. I'm using the NSFile.Save option to save it to the device. I'm thinking since it's a text file maybe I could more safely store it in the standard user settings area instead? This would prevent it from being synced by itunes, I presume?
What is the most reliable and safe way to handle this file i/o for the xml dataset storage?
Thank you.
You're using MonoTouch. Isn't it simply a matter of calling DataSet.WriteXml() with a FileStream object ready to write to a document in your Documents folder?
That Documents folder is backed up to iTunes. It's not synced, but it helps if your user is restoring their phone (because they bricked it, lost it, whatever). It doesn't explain why it's corrupt.
The only thing that I can think of why it's corrupt is because it took too long for your app to write it. There's a limited time from the point where the user exits the app until it's closed down, to prevent apps from keeping the system hostage and deteriorate user experience.
If writing the whole dataset takes too long, you want to think about minimizing that. Perhaps you can just store the data, and not the schema. Or you can devise a way to store only the deltas on exit and reconcile when the user has loaded your app again.
You can also prevent complete loss of data by writing to a second file, and when that operation completes delete the old file and rename. That way, the next time you start up if the write operation didn't complete, the old file would still be there and the user would have only lost their more recent changes.
In any case, if your data gets too big for a simple write operation to complete, you should look at different options such as sqlite.
Your best bet is probably to just save the XML as text. It's as simple as File.WriteAllText(...) - there's no reason to go to NSFile for this. That's part of the advantage of MonoTouch :)
Regarding syncing, here's the rule:
If you keep the file in the user's documents folder (Environment.SpecialFolder.MyDocuments and Environment.SpecialFolder.Personal BOTH point to the user's doc folder), then it's going to get backed up whenever the user syncs with iTunes.
There's nothing wrong with this. It persists the data between sessions and makes it recoverable if something goes wrong with the user's phone and they need to restore from a backup. Since your question is about persisting an XML file on the phone, this is what you want.
As for the iTunes question, there's no problem with speed and syncing because your app isn't going to be running while the phone is syncing. The file will either have been saved or it won't. Any corruption that takes place is happening while your app is running.
Reasons for files getting corrupted can include:
Not saving before the user quits. You get a chance to do this.
Not gracefully handling an incoming phone call. The system warns you about this as well.
iTunes definitely isn't corrupting your file. If that were the case, iOS apps would all be broken. It could be happening on your dev machine for whatever reason, but I've never seen this happen elsewhere, and I've done quite a bit of iOS development.
If you'd like a tutorial on reading and writing files, I posted an answer in another question.
It's lengthy, but the point was to answer as many questions as I could so nobody would be left hanging or confused.
A nice thing about iOS devices is that you're back (for most apps) in the one-person-at-a-time world. You're writing apps where you don't have to worry about 5,000 people trying to use your web-based app at the same time (that's not always true, but... you get the point). As a result, you can do things that you might normally consider bad for performance, but you're unlikely to see any performance problems (as long as the file you're saving is either small enough to be saved quickly or you're saving in the background on another thread - you never want to block the main (UI) thread with a heavy IO operation).
If you have any more questions, feel free to ask.
I hope this helps :)
Lots of frameworks are read-only, but I've found that GDataXMLNode from http://code.google.com/p/gdata-objectivec-client/ works very well read/write.
Having said that, on the iPhone you'd do yourself a big favour using Core Data with a SQLLite backend. :-) Apple has done all this for you and optimized it more than any of us every will
Cheers
Nik
Consider using SQLite, I'd go for something like
http://www.taimila.com/entify/ (not tried yet)
catnap orm
or Faks sqlite-net on google code (using this in a few apps)
entify - if it does what it says it can do - looks really good.
persisting XML on the iPhone as a means to store and access data is a nause you dont want to get into. I wrote about it here http://iwayneo.blogspot.com/2010/08/festival-star-history-serialization-as.html