Context:
My users need to see the same data.
Example: Admin user deletes item from customer order. Admin user is offline so change happens in cache only. Customer thinks he is still getting his product. Balance due, customer expectations, etc. obviously all out of sync.
Persistence is turned off:
await Firebase.initializeApp(options: DefaultFirebaseOptions.currentPlatform);
FirebaseFirestore.instance.settings = const Settings(persistenceEnabled: false);
Why can I still delete a document locally, or write to cache?
Are there any other options? Is this setting working as it should?
The only two workarounds I can find from reading articles and other questions is creating callables to cloud functions, and using a transaction - this is obviously not as fast. And is using transactions this way good practice?
I just don't want to be doing something else and find out I am doing something wrong with this cache setting as its not working as expected:
https://firebase.flutter.dev/docs/firestore/usage/
"This functionality is enabled by default, however it can be disabled if needed."
What I would like is for it to just throw if cache is false.
Thank you
Setting persistenceEnabled to false disabled the persistent cache that Firestore uses to caching data and keep pending writes between page/app reloads. But even when this is set to false, the Firestore SDK will keep all pending writes and data you have active listeners on in memory. This allows it to work in the situation of spotty network connections, which is quite common.
There is no way to disable the in-memory cache of pending writes.
Possible workarounds are to either detect the no-network situation yourself, and disable the functionality you don't want to allow, or (as you said already) use a mechanism that inherently requires a connection, such as a transaction or calling a Cloud Function.
All of these lead to a worse user experience though, so I recommend considering whether disabling the operation is really needed, or if there's a way to work with Firestore's model rather than against it.
Related
I could not find detailed information in the documentation. I have several questions regarding the offline persistence of firestore.
I understood that firestore locally caches everything and syncs back once online. My questions:
If I attach an onCompleteListener to my setDocument method it only fires when the device is online and has network access. But with offline persistence enabled, how can I detect that data has successfully been written to the cache (Is it always successful?!) - I see data is immediatly there without any listener ever triggering.
What if I wrote data to the cache while the device is offline then comes back online and everything gets synched. What if now any sort of error happens (So the onSuccessListener would contain an error, but the persistence cache already has the data). How do I know that offline and online data are ALWAYS in sync once network connection is restored on all devices?
What about race conditions? Lets say two users update a document at the "same time" while the device is offline. What happens once it comes back online?
But the most pressing question is: right now I continue with my programflow when the onSuccessListener fires, but it never does as long as the device is offline (showing an indefinete progress bar forever). I still need to continue with my program (thats why we have offline persistence) - How do I do this?
How can I detect that data has successfully been written to the cache
This is the case when the statement that write the data has completed. If writing to the local cache fails, an exception is thrown from that write statement.
You second point is hard to summarize, but:
Firestore keeps the pending writes separate from the snapshots it returns for local reads, and will update the cached snapshot correctly both for successful and for rejected writes.
If you want to know whether the snapshot you read contains any pending writes, you can check the pendingWrites field in its metadata.
What about race conditions? Let's say two users update a document at the "same time" while the device is offline. What happens once it comes back online?
The last write wins. If that's not what you need, use security rules to enforce your requirements on the server.
I am getting very high counts of Entity Writes in my firestore database.
Write permission in most of the paths are restricted, done from back-end server using admin SDK. Only a very few paths have write access- specifically only to the users who are (authenticated & registered & joined and approved in a specific group), so even though the ways to abuse are apparently thin, yet hard to specifically identify.
Only way I see- is to execute Cloud Functions on every write, and have the function log the paths somewhere to analyze. But that introduces further costs and complexity.
Is there any way/recommendation to monitor/profile where (i.e.- path) and who (UID or any identity) are performing the writes? There are tools to do such for RTDB, bu't can't find anything for Firestore.
I am also wondering if there is any way to restrict ip/users automatically in case of abuse (i.e.- high rate of read/write)?
What I'm currently doing is going to firestore console => menu usage => view usage
and I see something like this:
It's not the same as the profiler, but better than nothing.
I'm also keeping an eye on the video on the link below to see if someone provides an answer. People are asking for the profiler too.
https://www.youtube.com/watch?v=9CObBsjk6Tc
PWA application storage (IndexedDB) isn't able to provide data persistence.
In case that PWA is pinned to home screen it is possible to clear all application data from browser by clearing browsing history.
It might be unclear for users that cleaning browser data can affect pinned application and unsynchronised data will be lost.
Is there any way to avoid this?
The only way I see for now - turn back to native apps.
The clear storage mechanism in browsers is to put the user in control of their device.
This is why you as an application should never (native or web) assume your cached assets are cached.
If it is absolutely important to you to make sure you have core assets and data persisted then you need to have some sort of integrity check when the service worker initiates. That way you can restore cached state in case the application goes offline.
You also need to realize the operating system, looking at you iOS, will purge data when it feels like it (think when the available disk space gets critical), which takes you out fo control. It does this for native apps too as far as I know.
I do not know a way around that. The function in Chrome to "clear storage" (for example) does exactly that. I suppose it is reasonable for a user to be able to remove any data from their own device, but I agree it is not a good situation for the developer.
This is not possible.
The Storage API provides a StorageManager.persist() method to request the user explicit permission to persist data until deleted by the user itself:
if (navigator.storage && navigator.storage.persist)
navigator.storage.persist().then(function(persistent) {
if (persistent)
console.log("Storage will not be cleared except by explicit user action");
else
console.log("Storage may be cleared by the UA under storage pressure.");
});
If the local storage is running out of space, the User Agent will start automatically pruning cached resourced except the ones set as "persistent". However if the user itself chooses to clear the local data, there is no way to prevent this.
As far as I am aware, there is no event you can intercept in order to detect a browser clear action from the user.
See API reference doc :
https://developer.mozilla.org/en-US/docs/Web/API/StorageManager/persist
I'm creating a multi page application where i need to create and store transactions even when the users are offline. How do i achieve this using firestore ? Also i need some idea on how to persist the data received from the firestore locally.
You can't run transactions when offline,but if you think that your data is not changed while you are offline you can get the data from cache and update it it there using dbRef.addSnapshotListener(MetadataChanges.INCLUDE) and dbRef.update()
How to perform transactions in Firestore when the user is offline?
You cannot! Transactions are not supported for offline use, they can't be cached or saved for later. This is because a transaction absolutely requires round trip communications with the server in order to ensure that the code inside the transaction completes successfully. So you can use transaction only while online because the transactions are network dependent.
Also I need some idea on how to persist the data received from the Firestore locally.
According to the official documentation of Cloud Firestore regarding offline persistence:
For Android and iOS, offline persistence is enabled by default. To disable persistence, set the PersistenceEnabled option to false.
For the web, offline persistence is disabled by default. To enable persistence, call the enablePersistence method. Cloud Firestore's cache isn't automatically cleared between sessions. Consequently, if your web app handles sensitive information, make sure to ask the user if they're on a trusted device before enabling persistence.
Important: For the web, offline persistence is an experimental feature that is supported only by Chrome, Safari, and Firefox web browsers. Also, if a user opens multiple browser tabs that point to the same Cloud Firestore database, and offline persistence is enabled, Cloud Firestore will work correctly only in the first tab.
Edit:
The Firestore SDK for Android has a local cache that's enabled by default. So all read operations will come from the cache when there is no connectivity. So Firestore provides this feature to handle offline data. This means that if the user tries to add/delete documents while offline, every operation is added to a queue. Once the user regains the connection, every change that is made while offline will be updated on Firebase servers. In other words, all queries will be committed on the server.
Please also note that when you are offline, pending writes that have not yet been synced to the server are held in a queue. If you do too many write operations without going online to sync them, that queue will grow fast and it will not slow down only the write operations it will also slow down your read operations. So I suggest using this database for its online capabilities.
I am using iCloud with Core Data, based on the SQLite "Library-style" application design as specified by Apple. While the basic functionality works very well, I am concerned about the transaction logs and how they are managed.
While the database for my app is not large, it is very active and the core data stack is saved many times while the app is in use. I have noticed that a new transaction log is created for every core data save. The end result is that I have a TON of transaction logs and they take up much more space than the actual database.
1) Do the transaction logs ever get automatically pruned / coalesced, or will they continue to grow indefinitely, eventually numbering in the thousands and taking up many megabytes? It seems that the only way to manually purge the transaction logs and recreate a .baseline archive would be to disable and then re-enable iCloud (removing the ubiquity container and starting fresh). But this is obviously not a good solution.
2) My current architecture saves the core data stack often, even for minor changes. In general, this makes sense as my database is small and saving often ensures that the database file is always up-to-date. However, given the above issues with transaction logs, I am thinking that I should perhaps minimize saves to the database. Maybe doing so on a timed basis and/or on app transition states.
3) Even if I minimize the number of transaction logs by reducing how often I save the database, there seems to be an issue here, as the logs will continue to grow in number over time. Eventually the benefit of the "transaction log" design will become a burden in terms of the amount of iCloud storage used and the initial iCloud sync as a new device is added.
As Apple has provided very sparse information on iCloud and almost nothing in the form of "best practices", I would appreciate any insight from the community.
I filed a radar on this issue and received the following reply. They noted that it should be working correctly in iOS 5.1, though I have not yet verified this myself.
A clarification for any who might misunderstand the following. The transaction logs will be cleaned up by the core data internals. This is NOT something that should be performed by the application itself.
Engineering has provided the following feedback regarding this issue:
The transaction logs are intended to be deleted after all the active
peers have had a chance to read them, and they exceed a threshold of
space consumed. There was a previous issue that prevented the devices
from correctly doing so.