I have an iPhone app syncing with a website using a webservice. The sync procedure I'm using is the following:
Send items that need to sync from iPhone to website
Add items to website MySql database
Send back items that need to sync from website to iPhone
Add items on iPhone Sqlite database
All this is done using one request to the server. Often the user just have a few items (<20) to sync but sometimes a user might have 2000 items to sync. Adding 2000 items to a MySql database can take some time. I'm using transactions and commit on the website when adding the items, something like this:
mysqlTransaction.BeginTransaction();
foreach item sent from iPhone
{
mysql.CommandText = //Sql query
mysql.ExecuteNonQuery();
}
mysqlTransaction.Commit();
Sometimes when a user have a lot of items to add, this can take a veeery long time. My idea is therefore to change the sync procedure a little bit and instead of using one large request I divide the request into chunks, maybe sending and recieiving 50 items at a time. This will cause more but faster requests. What are your ideas on this? How would you solve it? Is it the right thing to use transactions and commit on the server?
I would try to make the update from "iphone to server" and "server to iphone" concurrent and asynchronous if possible. That way the sync will be faster and the iphone app won't block waiting for the whole process to be done.
As well dividing the update into frames if the update is bigger than a given size is a good idea.
Related
I have a mongo database with a collection of objects, lets say orders.
Multiple mobile devices are saving and loading orders to that collection.
When one device saves a new order to the database, all devices should sync and download new data. I want to be able to sync every 3 seconds so everyone can have latest data at every moment.
Now my problem is how to accomplish that efficiently? I do not want to load all the data, and parse it on the client side. I want to download only the missing orders.
What would be the best practice way of achieving this?
One option is to do a differential update.
You'll need to record the time of the last update of each device in the DB and the time of each modification to the DB.
Then you only update with the changes that occurred after the last time the device updated.
Also, one last note, sync'ing every 3 seconds will drain the battery fairly quickly. Perhaps you should consider only updating a device when the device has something to send to the server. ie send a new order, and the server response contains all new orders.
I'm using Cloudant and I'm struggling to pull/replicate 600 documents from server to my iPhone. First, it's pretty slow because it has to go one-document-at-a-time, and Second Cloudant was giving me "timeouts" after the 100th-or-so REST request. (I have a ticket with Cloudant for this one, as it's unacceptable!)
I was wondering if anyone has found a way / hack to "bulk" replicate when pulling. I was thinking, perhaps it's possible to "zip up" all of the changes, send them in one file, and fast-forward the iPhone database to the last-change seq.
Any helps is great -- thanks!
Can you not hit _all_docs?include_docs=true to get everything in one shot? http://wiki.apache.org/couchdb/HTTP_Document_API#all_docs
I don't know couchcoccoa but it looks like the API supports this: http://couchbaselabs.github.com/CouchCocoa/docs/interfaceCouchDatabase.html#a49d0904f438587b988860891e8049885
Actually, why not make a view. Make a view that gives you your list and make sure your id is there. With your id, you can then go to the document and get all the rest of the required information that you need in order to update it if you need to.
There really is no reason you would ever need to hit every document individually. They have views and search2.0 for that. Keep in mind you are using a cloud based technology. This stuff is not sitting in your basement, you can't just hit it a million times per device in a few seconds and expect anyone to not notice and/or get upset (an exaggeration, yes I know).
What I do not understand is that you are trying to replicate it to an iPhone? Are you running apache and couchdb in your app? Why not just read the JSON data and throw it into a database. or just throw it into a file if it updates that much and keep overwriting it. There is so many options that are a whole lot less messy.
I am writing an app for iOS that uses data provided by a web service. I am using core data for local storage and persistence of the data, so that some core set of the data is available to the user if the web is not reachable.
In building this app, I've been reading lots of posts about core data. While there seems to be lots out there on the mechanics of doing this, I've seen less on the general principles/patterns for this.
I am wondering if there are some good references out there for a recommended interaction model.
For example, the user will be able to create new objects on the app. Lets say the user creates a new employee object, the user will typically create it, update it and then save it. I've seen recommendations that updates each of these steps to the server --> when the user creates it, when the user makes changes to the fields. And if the user cancels at the end, a delete is sent to the server. Another different recommendation for the same operation is to keep everything locally, and only send the complete update to the server when the user saves.
This example aside, I am curious if there are some general recommendations/patterns on how to handle CRUD operations and ensure they are sync'd between the webserver and coredata.
Thanks much.
I think the best approach in the case you mention is to store data only locally until the point the user commits the adding of the new record. Sending every field edit to the server is somewhat excessive.
A general idiom of iPhone apps is that there isn't such a thing as "Save". The user generally will expect things to be committed at some sensible point, but it isn't presented to the user as saving per se.
So, for example, imagine you have a UI that lets the user edit some sort of record that will be saved to local core data and also be sent to the server. At the point the user exits the UI for creating a new record, they will perhaps hit a button called "Done" (N.B. not usually called "Save"). At the point they hit "Done", you'll want to kick off a core data write and also start a push to the remote server. The server pus h won't necessarily hog the UI or make them wait till it completes -- it's nicer to allow them to continue using the app -- but it is happening. If the update push to server failed, you might want to signal it to the user or do something appropriate.
A good question to ask yourself when planning the granularity of writes to core data and/or a remote server is: what would happen if the app crashed out, or the phone ran out of power, at any particular spots in the app? How much loss of data could possibly occur? Good apps lower the risk of data loss and can re-launch in a very similar state to what they were previously in after being exited for whatever reason.
Be prepared to tear your hair out quite a bit. I've been working on this, and the problem is that the Core Data samples are quite simple. The minute you move to a complex model and you try to use the NSFetchedResultsController and its delegate, you bump into all sorts of problems with using multiple contexts.
I use one to populate data from your webservice in a background "block", and a second for the tableview to use - you'll most likely end up using a tableview for a master list and a detail view.
Brush up on using blocks in Cocoa if you want to keep your app responsive whilst receiving or sending data to/from a server.
You might want to read about 'transactions' - which is basically the grouping of multiple actions/changes as a single atomic action/change. This helps avoid partial saves that might result in inconsistent data on server.
Ultimately, this is a very big topic - especially if server data is shared across multiple clients. At the simplest, you would want to decide on basic policies. Does last save win? Is there some notion of remotely held locks on objects in server data store? How is conflict resolved, when two clients are, say, editing the same property of the same object?
With respect to how things are done on the iPhone, I would agree with occulus that "Done" provides a natural point for persisting changes to server (in a separate thread).
Say I have a TODO list iphone app, that can be edited/viewed from both a web application and the iphone application.
When on the iphone, when a user views all his todo lists, or sub-items, I would think that each time the user views a particular list it shouldn't be hitting the web applications API every-time, but rather cache locally the values and only hit the web when things change.
What strategies are there for this type of scenerio?
I agree with you in your dirty-otherwise-do-not-contact-the-server point. And I think this point is pretty straightforward and easy to implement.
However, be careful in this scenario: it gets dirty but at the same time, the device cannot reach the internet. In this scenario, I suggest you check the internet accessibility on a frequent basis (even when your app is in the background), and try to reach your server and update whenever possible.
This is a tricky problem. I'm currently working on an app that needs to perform a similar synchronization, and I haven't decided how I want to handle it yet.
You're right in that you don't want to be hitting the web repeatedly. It would slow the app down considerably. Keeping a local cache is the way to go.
One drawback is that the user could change/add an item on the web and you wouldn't see it on the phone. You'd need to have a refresh button (like in the Mail application, for example) to allow the user to get the changes.
Then you have an issue with conflict resolution. Say the same item is edited on both the phone and on the web. How does the user pick which one to keep, or do they get duplicated?
I think the best way to do this is to replicated your server's schema in CoreData. Then load a given element from the local DB, and in the background go out and check that element for updates if the device has an internet connection. You're hitting the db each time, but the user is not slowed down by the process.
You should not query the internet everytime you view the list.
But when you make updates to it, or edit it, you should update the server as well. That will make your life a whole lot simpler. That way when the user updates an item that he deleted in the web server, the server will just throw that request out...
I am writting a django app and Iphone app, I need to keep them in sync.
Users can delete, update and create new objects in the web app, and in the iphone app.
When they get online with the iphone both app must be in sync.
Is there simple way to do this?
Thanks,
Joaquin
In general: There's no simple way. But I'll outline an approach.
If you don't care about changes being overwritten: Keep a timestamp of the most recent change to each record, and a timestamp of each sync. When syncing, you get a list of all updates on the iPhone since the last sync, and all updates on the server. You write from the iPhone to the server if the iPhone timestamp for that record is newer than the server one, and vice versa.
But you probably care. Say you've edited a note called "Where to meet up on Friday." It started out empty. Now, on the phone, you've written, "My house." Ten minutes later, your friend edits the same note on the server and writes, "The diner." Who wins out? Stack Overflow can't answer that for you; it's application-specific.
OK, so modify the approach above: if both the server version of a record and the local version have been edited since the last sync, then you have to ask the user what to do. That's the basic algorithm.
If you care a lot about changes not being overwritten, to the point that you want to merge changes to different places in the same documents, then your system will begin to approach the complexity of version control systems like Subversion or Git. Not at all simple.
There's no built in way to do this. You need to keep a server data store, and a local data store on the iPhone, and when online, check the differences manually, and see what action you should take on the server and the iPhone side (delete, update, etc.).
Sync is usually hard. I suggest you start laying out the server and iPhone data stores, and think how they relate, and how can the server or the iPhone know the status of their counterpart record, so to keep them in sync.