What is the best approach to keep data of collection available offline (0 - 100 docs) and sync it on app startup if connection available?
UPD: I'm looking to setPersistenceEnabled but is there any guarantee my collection will be cached after first retrieve?
If you call setPersistenceEnabled(true) these documents will be available offline and synced when connection is available again (not necessarily at app startup). Check documentation:
https://firebase.google.com/docs/database/android/offline-capabilities
By enabling persistence, any data that the Firebase Realtime Database
client would sync while online persists to disk and is available
offline, even when the user or operating system restarts the app. This
means your app works as it would online by using the local data stored
in the cache. Listener callbacks will continue to fire for local
updates.
Also be aware that when you use this, then at app startup your listeners will be called TWICE! One - for offline (cached) data and second - online (read from firebase).
Related
As per the answer in the link below:
Does Firebase Firestore keep a local copy of snapshot
The Firestore SDK keeps a local copy of:
All data that you have an active listener for.
All pending writes.
In addition, if offline persistence is enabled,
it also keeps a local copy of data that
your code has recently read.
Does this mean that even if offline persistence is disabled on the web, onSnapshot event listeners and pending writes will still be stored locally to cache regardless?
I'm worried about sensitive data on shared computers for these listeners and pending writes that's stored locally (if they did not successfully write to the server, i.e. went offline in the process, etc.). Can malicious users access this data somehow? Is this a flaw of firestore for web apps?
If offline persistence is disabled, only pending writes are stored in the local cache.
You should assume that all data that is entered on the local device can be intercepted by a malicious actor who gets access to that device. If that is not acceptable for your use-case, consider not using a Firebase SDK for Firestore but using the REST API or gRPC API directly.
I could not find detailed information in the documentation. I have several questions regarding the offline persistence of firestore.
I understood that firestore locally caches everything and syncs back once online. My questions:
If I attach an onCompleteListener to my setDocument method it only fires when the device is online and has network access. But with offline persistence enabled, how can I detect that data has successfully been written to the cache (Is it always successful?!) - I see data is immediatly there without any listener ever triggering.
What if I wrote data to the cache while the device is offline then comes back online and everything gets synched. What if now any sort of error happens (So the onSuccessListener would contain an error, but the persistence cache already has the data). How do I know that offline and online data are ALWAYS in sync once network connection is restored on all devices?
What about race conditions? Lets say two users update a document at the "same time" while the device is offline. What happens once it comes back online?
But the most pressing question is: right now I continue with my programflow when the onSuccessListener fires, but it never does as long as the device is offline (showing an indefinete progress bar forever). I still need to continue with my program (thats why we have offline persistence) - How do I do this?
How can I detect that data has successfully been written to the cache
This is the case when the statement that write the data has completed. If writing to the local cache fails, an exception is thrown from that write statement.
You second point is hard to summarize, but:
Firestore keeps the pending writes separate from the snapshots it returns for local reads, and will update the cached snapshot correctly both for successful and for rejected writes.
If you want to know whether the snapshot you read contains any pending writes, you can check the pendingWrites field in its metadata.
What about race conditions? Let's say two users update a document at the "same time" while the device is offline. What happens once it comes back online?
The last write wins. If that's not what you need, use security rules to enforce your requirements on the server.
I'm creating a multi page application where i need to create and store transactions even when the users are offline. How do i achieve this using firestore ? Also i need some idea on how to persist the data received from the firestore locally.
You can't run transactions when offline,but if you think that your data is not changed while you are offline you can get the data from cache and update it it there using dbRef.addSnapshotListener(MetadataChanges.INCLUDE) and dbRef.update()
How to perform transactions in Firestore when the user is offline?
You cannot! Transactions are not supported for offline use, they can't be cached or saved for later. This is because a transaction absolutely requires round trip communications with the server in order to ensure that the code inside the transaction completes successfully. So you can use transaction only while online because the transactions are network dependent.
Also I need some idea on how to persist the data received from the Firestore locally.
According to the official documentation of Cloud Firestore regarding offline persistence:
For Android and iOS, offline persistence is enabled by default. To disable persistence, set the PersistenceEnabled option to false.
For the web, offline persistence is disabled by default. To enable persistence, call the enablePersistence method. Cloud Firestore's cache isn't automatically cleared between sessions. Consequently, if your web app handles sensitive information, make sure to ask the user if they're on a trusted device before enabling persistence.
Important: For the web, offline persistence is an experimental feature that is supported only by Chrome, Safari, and Firefox web browsers. Also, if a user opens multiple browser tabs that point to the same Cloud Firestore database, and offline persistence is enabled, Cloud Firestore will work correctly only in the first tab.
Edit:
The Firestore SDK for Android has a local cache that's enabled by default. So all read operations will come from the cache when there is no connectivity. So Firestore provides this feature to handle offline data. This means that if the user tries to add/delete documents while offline, every operation is added to a queue. Once the user regains the connection, every change that is made while offline will be updated on Firebase servers. In other words, all queries will be committed on the server.
Please also note that when you are offline, pending writes that have not yet been synced to the server are held in a queue. If you do too many write operations without going online to sync them, that queue will grow fast and it will not slow down only the write operations it will also slow down your read operations. So I suggest using this database for its online capabilities.
I've got a database on my server which is about 3mb big. I'd like to ship that with my iphone application.
The most important thing is that I'd like to promote changes to the database (insert, updates, deletes) to the iphone. What's the best way of doing that? I mean - what is necessary on
- the server
- the client (= iphone)
- between; how to transfer this data?
I'm pretty free in using technologies serverside; right now, I've got an sqlite-database on the server filled with the data I'd like to sync to the iphones.
How often do you need the database to be updated, and how urgent are the changes?
If the database updates are infrequent and non-urgent, I'd have the app check for a new version of the database on startup, and if it has changed, download the entire new file.
The app would always download a small metadata file from a known URL on startup. The metadata file contains an version identifier for the latest version and a location where that version of the database can be downloaded. If the version identifier has changed from the version the app already has, will download the new version. If the version identifier has not changed, or if it can't check, the app can keep using the version it has.
Pro tip: if you want to show a progress bar for the download, include the size of the database in the metadata file. Cell networks often have transparent proxies that strip out the Content-Length header from HTTP responses.
Try using web hooks.
The concept of a WebHook is simple. A
WebHook is an HTTP callback: an HTTP
POST that occurs when something
happens; a simple event-notification
via HTTP POST.
A web application implementing
WebHooks will POST a message to a URL
when certain things happen. When a web
application enables users to register
their own URLs, the users can then
extend, customize, and integrate that
application with their own custom
extensions or even with other
applications around the web. For the
user, WebHooks are a way to receive
valuable information when it happens,
rather than continually polling for
that data and receiving nothing
valuable most of the time. WebHooks
have enormous potential and are
limited only by your imagination! (No,
it can't wash the dishes. Yet.)
You can find out more on Webhooks here:
http://www.webhooks.org/ and http://webhooks.pbworks.com/
Wonder if you have considered using a Sync Framework to manage the synchronization. If that interests you can take a look at the open source project, OpenMobster's Sync service. You can do the following sync operations
two-way
one-way client
one-way device
bootup
Besides that, all modifications are automatically tracked and synced with the Cloud. You can have your app offline when network connection is down. It will track any changes and automatically in the background synchronize it with the cloud when the connection returns. It also provides synchronization like iCloud across multiple devices
Also, modifications in the Cloud are synched using Push notifications, so the data is always current even if it is stored locally.
Here is a link to the open source project: http://openmobster.googlecode.com
Here is a link to iPhone App Sync: http://code.google.com/p/openmobster/wiki/iPhoneSyncApp
I'm looking into building an application which works just as well offline as it does online. Since the application cannot communicate with the server while in offline, there is some level of synchronization which needs to take place.
What are some good tools to read about and start thinking about when planning offline operations with synchronization for your iPhone?
What tools would I have to create on my own, versus tools that apple already provides to help with this particular issue?
I've been working on an app that handles this exact behavior for the last 2 months or so. It has a small subset of functions which are online only and a large set of functionality that is offline/online.
I'm using sqlite for local storage as suggested here with a modified version of the sqlitepersistentobjects library. The base version of sqlitepersistentobjects is not thread safe so watch out if you are using it. (check out objectiverecord in: objectivesync for a thread safe alternative but be prepared to dig into the code). If you are willing to develop for the 3.0 sdk then core data is another possibility for a sqlite library.
The overall architecture is simple enough I have modeled local storage using sqlite and remote interaction using objective resource against a rails app and REST api. It can use either xml or json for data serialization.
When an object is modified locally the change is first saved to the sqlite database record for that object and then added to a queue which is serialized and stored in the local sqlite db as well. (The queue can then be processed at any time)
If there is a connection available any queued local changes are deserialized and added to an NSOperationQueue which then processes them in the background.
In order to make this all work I've subclassed NSOperation so that it can support several types of remote queue operations - create, update, delete essentially using objective resource to make the remote requests.
The nice thing about using NSOperationQueue and NSOperation is that they handle the background threading for you so I'd highly recommend having a look at the apple docs for those classes and also at the apple threading guide.
When the application loads there is a bit of remote checking done and processed in the background to pull down the latest data - although to be honest I am still changing the way this behaves a bit.
That's a quick overview of what I've had to deal with so far...hope it helps a little.
there are plenty of application on the app store which rely on both online as well as offline data
what you should really be doing is on start of your app, run a background thread (which runs silently so your user never sees any delay). this thread downloads the latest data from your server and pushes it into your local database (sqlite is the best choice)
make sure you implement some kind of data versioning so that your app only downloads data which is actually changed since last download - else you would unnecessarily be downloading the entire dataset which can be quite huge (depending upon your app requirements)
also make sure to test for internet connectivity when doing this. if no internet is available, alert the user for sure
this way you get the best of both worlds. users when away from internet can still use your app with their local sqlite data
in iphone os 3.0 apple has introduced push services - where you can simply "PUSH" your data instead of doing a "PULL" however this is not available in the current iPhone OS (2.x.x)
Push is probably not a viable option here, since the amount of data you can push is miniscule, and basically comes back to "tell my app to make a server call". We use an online/offline model in Satchel. Whenever we have to communicate with the server, we bundle that communication (a URL and possibly some POST data) and store it to a database. If we're online, we pull it right back out, send it, and when we get a valid response back, we remove the record from the database. If we're offline, those rows build up, and the next time we ARE online, they get sent out. This is not a workable model in all situations, but can be adapted to most.
In 3.0, you've got access to CoreData, which is a great data management tool. Other than that, the NSURLXXX family is your friend.
I would store all the information I gather while offline in a SQLite database. Then, on user 's request, you can SYNC all the stored information with a server using HTTP or a custom TCP/IP protocol you can come up with.
I have been using this approach on Palm OS applications for almost 10 years now, and they do work very effectively.
As far as I know, the only "tool" you will have to accomplish this is plain old OBJECTIVE-C with Cocoa Touch. Although you could use some TCP/IP C++ libraries that will make your life easier if you decide to implement your own protocol.
Wonder if you have considered using a Sync Framework to manage the synchronization. If that interests you can take a look at the open source project, OpenMobster's Sync service. You can do the following sync operations
two-way
one-way client
one-way device
bootup
Besides that, all modifications are automatically tracked and synced with the Cloud. You can have your app offline when network connection is down. It will track any changes and automatically in the background synchronize it with the cloud when the connection returns. It also provides synchronization like iCloud across multiple devices
Also, modifications in the Cloud are synched using Push notifications, so the data is always current even if it is stored locally.
Here is a link to the open source project: http://openmobster.googlecode.com
Here is a link to iPhone App Sync: http://code.google.com/p/openmobster/wiki/iPhoneSyncApp