I am retrieving a lot of documents from Firestore, it can take a lot of time depending on the network connection of the user.
I would like to display a progres bar.
I didn't find any documentation on this
Here is my code :
final databaseReference = Firestore.instance;
databaseReference.collection("XXX").getDocuments().then((QuerySnapshot snapshot) {
Map<String, Transac.Transaction> map = new Map<String, Transac.Transaction>();
snapshot.documents.forEach((f) {
//doing something
});
I would like to have a percentage of the data loaded. For example :
Loading (58%)
Thank you.
Firestore doesn't expose any progress indicator for the documents within a query. By the time your then callback gets called, all documents have been loaded on the client and are present in the QuerySnapshot.
The usual way to deal with this is to show a "spinner" instead of a progress bar, and then hide that spinner as the first code inside your then callback.
What you can do instead is just call getDocument() for each document individually, and track their individual completions. You should track them in the order that you issued them, as the requests are pipelined over a single connection. This would give you a rough sense of progress, if you're requesting lot of documents.
Related
I am trying to get the download urls from 300 files inside the firebase storage :
items.forEach((element) {
setUrls(element.id);
});
void setUrls(String id) async {
String imageurl = await storageRef.child('id').getDownloadURL();
saveUrl(imageurl, id);
}
but it takes a lot of time to get all the files urls, how can i speed it up ?
Tried to call method to excute all 300 files at the same time, but still very slow, it gets one file at a time.
You could use the .listAll() method which buffers results in memory. You can even do this recursively if you need to go additional layers. Then call forEach like you did. Honestly not sure what kind of time this would save but it might help.
I would create a separate Firestore collection which stores just the id and URL of each image. Be sure to call getDownloadURL and save the result when images are added or changed. Then create a custom index to increase read speed. Just query that collection and loop the results.
I'm working with Sembast nowadays and was wondering If there's any way to create a stream of data that could get me all the values inside the DB. My requirement is to setup a listener on that stream so that whenever the data change is triggered, I could do something with it.
Documentation on Sembast is pretty limited and I'm now sure how I can do this. Usually I use the .find method to fetch all the values from within my db. I've been using a stringMapFactory to store my records.
Can we do this ? Any help would be really appreciated.
Sorry for the poor documentation.
It is quite similar to firestore. You can listen to all changes in a store
// Track every store changes
var query = store.query();
var subscription = query.onSnapshots(db).listen((snapshots) {
// snapshots always contains the list of all records
// ...
});
Basically you have a query on the store (with or without filter) that you can query or listen for changes.
See https://pub.dev/documentation/sembast/latest/sembast/QueryRef/onSnapshots.html
https://github.com/tekartik/sembast.dart/blob/master/sembast/doc/change_listener.md
If you use Hive as db, may use Hive Box as listenable.
ValueListenableBuilder<Box<YOUR_BOX_MODEL>>(
valueListenable: box.listenable(),
builder: (context,value,child){}
)
I'm using Riverpod and I have a relatively simple provider that I'm using to get the number of pending writes from Firestore. This is basically used to help give users feedback as to whether or not their data is fully synced.
final pendingUploadsCounterProvider =
FutureProvider.autoDispose.family<int, List<Report>>((ref, reports) async {
final db = ref.read(firebaseServiceProvider).db;
final reportIds = reports.map((e) => e.id).toList();
final documents = await Future.wait([
for (final name in kReportTypes)
db
.collection(name)
.where('report_id', whereIn: reportIds)
.get(GetOptions(source: Source.cache))
]);
return documents.where((event) => event.metadata.hasPendingWrites).length;
});
As you can see, all I'm doing is reading some data from the Firestore cache and then filtering that list based on the hasPendingWrites property.
I'm attempting to use this provider on multiple screens as I need to provide the user feedback on their sync status in multiple locations.
When I use this provider on a single screen only, every time I enter the screen, the provider is triggered and I get an up to date status.
When I use this provider on multiple screens, the first time I enter a screen where it is used, the provider is triggered. Subsequently, entering any other screen where this provider is used, the provider is no longer triggered and therefore, the results are out of date.
I thought by using autoDispose, the provider would be disposed of when I leave the screen, but that doesn't appear to be the case.
I'm sure that there's a disconnect with my understanding of riverpod and how this works and hopeful someone can shed some light on my misunderstanding.
Thank you!
the provider will not be disposed while the widget in the tree you need to remove it from the back stack to be disposed
in your case, you need to reload it on other screens
by calling
context.refresh(pendingUploadsCounterProvider);
I have been using onSnapshot successfully to alert my code to changes in underlying data, as in
// Set up to listen for changes to the "figures" collection, that is,
// someone has created a new figure that we will want to list on the screen.
setFiguresListener: function () {
// `figuresCR` is a collection reference defined elsewhere
return this.figuresCR.onSnapshot((iFigs) => {
iFigs.forEach((fSnap) => {
const aFigure = figureConverter.fromFirestore(fSnap, null);
const dbid = aFigure.guts.dbid; // ID of the "figure" in the database
nos2.theFigures[dbid] = aFigure; // update the local copy of the data
});
nos2.ui.update();
console.log(` Listener gets ${iFigs.size} figures`);
});
But I now read about on in the docs. It explains:
[The on() function] Listens for data changes at a particular location.
This is the primary way to read data from a Database. Your callback
will be triggered for the initial data and again whenever the data
changes. Use off( )to stop receiving updates. See Retrieve Data on
the Web for more details.
The syntax is a bit different, and on() seems to do much the same as onSnapshot().
So what is the real difference? Should we be using on() instead of onSnapshot()?
on() is an operation for reading from Firebase Realtime Database. That's a completely different database with different APIs than Firestore. They have essentially no overlap. There is no on() operation with Firestore.
If you're working with Firestore, ignore all the documentation about Realtime Database, and stick to using onSnapshot() for getting realtime updates.
Other tyros who fall into this tar pit: in the API doc pages, you might think that since firestore is a database under firebase, you could look for help under firebase.database. But no: look only in the next section, firebase.firestore.
What am I trying to accomblish?
I am currently facing a bunch of problems implementing a real time updated infinite scrolling list with the firestore backend.
In my application I want to display comments (like in e.g. YouTube or other social media sites) to the user. Since the number of comments in a collection might be quite big, I see an option to paginate the collection, while receiving real time updates based on snapshots. So I initially load x comments with the option to load up to x more items whenever the user presses a button. In the image below x = 3.
The standard solution
Based on other SO questions I figured out that one is supposed to use the .limit() and the .startAfter() methods to implement such behaviour.
So the first page is loaded as:
query = this
.collection
.orderBy('date', descending: true)
.limit(pageSize);
query.snapshots().map((QuerySnapshot snap) {
lastVisible = snap.documents.last;
// convert the DocumentSnapshot into model object
});
All additional pages are loaded with the following code:
query = this.collection
.orderBy('date', descending: true)
.startAfterDocument(lastVisible)
.limit(pageSize);
Furthermore, I'd like to add that this code is located in a repository class which is used with the BLoC pattern similar to the code shown in Felix Angelov's Flutter Todos Tutorial.
While Felix uses a simple flutter list to show the items, I have a list of pages showing comments based on the data provided by their BLoCs. Note that each BLoC accesses a shared repository (parts of the repository code is shown below).
The Problem with the standard solution
With the code shown above I see multiple problems:
If a comment is inserted in the middle of the ordered collection (how is not of importance), the added comment is shown because of the Stream provided by the snapshot. However, another comment that already existed is not longer shown because of the .limit() operator in the query. One could increase the limit by one but I'm not sure how to edit a snapshot query. In the case that editing a snapshot query is not possible, one could create a new (and bigger) query, but that would cost additional reads.
Similar to 1., if a comment in the middle is deleted, the snapshot will return a list which does not longer contain the deleted comment, however another comment (which is already covered by a different page) appears. E.g., in the scenario shown in the image above 5 comments are loaded. Assuming that comment 3 is deleted, comment 2 will show twice.
Improving the standard solution
Based on these two problems discussed above, I decided that the solution is not sufficient and I implemented a solution which first loads x items by obtaining two "interval" documents. Then a query which fetches the required items in an interval using .startAtDocument() and .endAtDocument() is created, which eliminates the .limit() operator.
DocumentSnapshot pageStartDocument;
DocumentSnapshot pageEndDocument;
Future<Stream<List<Comment>>> comments() async {
// This fetches the first and next Document as initialization
// (maybe should be implemented in constructor)
if (pageStartDocument == null) {
Query query = collection
.orderBy('date', descending: true)
.limit(pageSize);
QuerySnapshot snap = await query.getDocuments();
pageStartDocument = snap.documents.first;
pageEndDocument = snap.documents.last;
} else {
Query query = collection
.orderBy('date', descending: true)
.startAfterDocument(pageEndDocument)
.limit(pageSize);
QuerySnapshot snap = await query.getDocuments();
pageStartDocument = snap.documents.first;
pageEndDocument = snap.documents.last;
}
// This fetches a subcollection of elements from the collection
// with the tradeof of double the reads
Query query = this
.collection
.orderBy('date', descending: true)
.startAtDocument(pageStartDocument)
.endAtDocument(pageEndDocument);
return query.snapshots().asyncMap((QuerySnapshot snap) async {
// convert the QuerySnapshot into model objects
});
As commented in the code, this solution has the following drawback:
Since a query is required to obtain the pageStartDocument and pageEndDocument, the number of reads is doubled, because all the data is read again when the second query is created. The performance impact might be neglectable because I believe the data is cashed, however having 2x database read cost can be significant.
Question:
Since I am not only implementing pagination but also real time updates (with collection insertions), the .limit() operator seems to be not working in my case.
How does one implement a pagination with real time updates (without double reads)?
Side Notes:
I watched how Todd Kerpelman devoures a massive gummy bear while explaining pagination, but in the video it seems to be not so trivial (and a point was made that a tradeoff might be necessary).
If further code from my side is required please say so in the comments.
For the scenario of comments it does not really makes sense that an item is inserted into the middle of the (sorted) collection. However I would like to understand how it should be implemented if the scenario requires such a feature.
this may come as a very late answer. The OP probably won't need help anymore, however for anyone who should stumble on this I wrote a tutorial with a solution that partly solve this:
the Bloc keep a list of stream subscription to keep trace of realtime updates to the list.
however concerning the insertion problem, since when you will have paginated streams based on a document cursor, upon insertion or deletion you necessarily need to reset your pagination stream subscriptions unless it is the last page.
Hence my solution around it was to update the list when modifications occur but reset it when insertions or deletions occur.
Here is the link to the tutorial :
https://link.medium.com/2SPf2Qsbsgb