first,I added an animation, after animation completed I called below code line but it's consuming too much time.
getData()async{
DateTime now = DateTime.now();
await Firestore.instance.collection('Table_Name').getDocuments().then((QuerySnapshot snapshot){
print("length ${snapshot.documents.length}");
for(int i = 0 ; i< snapshot.documents.length; i++){
bool isToday = snapshot.documents[i].data['CreatedBy'].toString().split(" ")[0]==DateFormat("yyyy-MM-dd").format(now);
print("isTOday $isToday");
if(isToday && snapshot.documents[i].data['GainStatus']== "0"){
setState(() {
giftDocumentID = snapshot.documents[i].documentID;
});
break;
}
}
winOrLose(now);
print("giftDocumentID: $giftDocumentID");
});
}
This Above codes taking approx 10 min to retrieve data from firestore, Why is it too lazy?
It got solved by testing it on another device and running smoothly without any much time consumption.
Actually my first device's memory has filled, so that's why, taking too much time.
But I'm still confuse, what the relation between device's memory space and firestorm database..!
The relation between firestore data and device's memory is cache. When data is loaded for the first time, it's from firebase's servers (and thus costs you read requests), then firestore saves the data in the cache memory for future use. So slow retrieval of the data must be a Device OS problem (or a power issue).
You can remove caching feature from firebase data but I won't recommend that.
Also as suggested earlier you should not load all data at once especially if the data is going to be fetched a lot or data gets changed very often. The solution for that is pagination.
This Video explains Flutter Pagination nicely:
https://www.youtube.com/watch?v=coR4Y-DkrLc
Related
I have data around 100 000 (each object consists 10 - 15 fields). It's crashing the application while trying to insert such data. I am bit confused whether choosing hive for this purpose was correct. I need to sync the data on regular basis. Is Hive compatible for storing such huge data. Currently I am trying to insert complete data at once. Is it a right choice to Use Hive or should i shift to other sources like sqflite or should I split the data while inserting.
I faced the same problem , after using app for a while app crashes suddenly due to some weird issue . Fixed this problem by this approach in main startup function of flutter app .
if (!Hive.isAdapterRegistered(77)) Hive.registerAdapter(AdapterName());
try {
if (!Hive.isBoxOpen('BoxName'))
await Hive.openBox<Type>('BoxName');
} catch (error) {
await Hive.deleteBoxFromDisk('BoxName');
await Hive.openBox('BoxName');
}
Client-side, I'm using a listener to detect if the "notifications" collection of the user changes. The App calls a Cloud Function that retrieves the last three unread notifications and the total number of unread notifications.
In my App, I have this:
Listener
firestore.collection("users")
.doc(uid)
.collection("notifications")
.snapshots().listen((QuerySnapshot querySnapshot) {
NotificationsPreviewModel notificationsPreview =
await _cloudFunctionsService.getNotificationsPreview(doctor.id)
})
Cloud Function
exports.getNotificationsPreview = functions.https.onCall(async (data, context) => {
const userId = data.userId;
let notifications = [];
const notificationsDocuments = await db
.collection("users")
.doc(userId)
.collection("notifications")
.orderBy("dateTime", "desc")
.get();
notifications = notificationsDocuments.docs.map((rawNotification) =>
rawNotification.data()).filter((element) => element.unread == true);
const notificationsNumber = notifications.length;
notifications = notifications.slice(0, 3);
return { "notifications": notifications, "notificationsNumber": notificationsNumber };
});
The Cloud Function gets called only when a change is detected, so it shouldn't return old data.
The error appears only the first time the Cloud Function is called from the App's start, but not always. The following calls don't generate the error.
How can I solve this? For now, I've added a delay of 500ms, and it works perfectly, but it's not a real solution.
Based on your description, it sounds like you see some form of latency while collecting the data from Firestore. Retrieving data from the Cloud takes time, and a delay of 500ms is not excessive.
I am not familiar with Flutter enough to comment on your code. However, according to the documentation for Java:
By default, get() attempts to provide up-to-date data when possible by waiting for data from the server, but it may return cached data or fail if you are offline and the server cannot be reached. This behavior can be altered via the Source parameter.
Source:
By providing a Source value, these methods can be configured to fetch results only from the server, only from the local cache, or attempt to fetch results from the server and fall back to the cache (which is the default).
If you are online, get() checks the server for the latest data, which can take between 300ms and 1500ms depending on several factors. For example, where is your Firestore instance located in comparison to your Cloud Function and client? Try adjusting the delay and see if you can identify the timing.
There are also some soft limits you should be aware of as this might also impact your timings for how quickly you can retrieve the data. There is a maximum sustained write rate to a document of 1 per second. Sustaining a write rate above once per second increases latency and causes contention errors.
As for the documentation:
When you set a listener, Cloud Firestore sends your listener an initial snapshot of the data, and then another snapshot each time the document changes.
It seems that you are initially receiving the snapshot of the data, and then the following updates, as expected.
You can check some possible solutions to this in this post.
I have been using onSnapshot successfully to alert my code to changes in underlying data, as in
// Set up to listen for changes to the "figures" collection, that is,
// someone has created a new figure that we will want to list on the screen.
setFiguresListener: function () {
// `figuresCR` is a collection reference defined elsewhere
return this.figuresCR.onSnapshot((iFigs) => {
iFigs.forEach((fSnap) => {
const aFigure = figureConverter.fromFirestore(fSnap, null);
const dbid = aFigure.guts.dbid; // ID of the "figure" in the database
nos2.theFigures[dbid] = aFigure; // update the local copy of the data
});
nos2.ui.update();
console.log(` Listener gets ${iFigs.size} figures`);
});
But I now read about on in the docs. It explains:
[The on() function] Listens for data changes at a particular location.
This is the primary way to read data from a Database. Your callback
will be triggered for the initial data and again whenever the data
changes. Use off( )to stop receiving updates. See Retrieve Data on
the Web for more details.
The syntax is a bit different, and on() seems to do much the same as onSnapshot().
So what is the real difference? Should we be using on() instead of onSnapshot()?
on() is an operation for reading from Firebase Realtime Database. That's a completely different database with different APIs than Firestore. They have essentially no overlap. There is no on() operation with Firestore.
If you're working with Firestore, ignore all the documentation about Realtime Database, and stick to using onSnapshot() for getting realtime updates.
Other tyros who fall into this tar pit: in the API doc pages, you might think that since firestore is a database under firebase, you could look for help under firebase.database. But no: look only in the next section, firebase.firestore.
I have to increment a value in firebase firestore and the data is look like this
{
"key":{
"name":"Apple",
"quantity":12
}
}
I need to increment quantity by one. For this I use this Function.
FirebaseFirestore.instance.runTransaction((transaction) async {
transaction.set(
documentReference,
{
"key":{
"quantity" : FieldValue.increment(1)
}
},
SetOptions(merge: true));
return transaction;
});
This is working fine when the app is connected with network. But when the app goes offline, the value just increment by 1 when the app is connected again. Though several times quantity has increased in offline but it just increment by 1 when the connection is back.
The problem is that you are using a transaction for that, the way transactions work is by getting the current value of a field to determine the new value and the Firebase clients don't persist transactions across app restarts because the concept of transactions doesn't work well when a user is not connected.
Therefore the way transactions work makes it impossible for your current code to increment more than one time when offline. So in order to achieve what you want it would be better to take this out of a transaction/increment approach and possibly rethink the structure of your app to accomodate this counter in a different way, like a list of keys and a background function that gets the list when the app gets back online and increments it, for example.
What am I trying to accomblish?
I am currently facing a bunch of problems implementing a real time updated infinite scrolling list with the firestore backend.
In my application I want to display comments (like in e.g. YouTube or other social media sites) to the user. Since the number of comments in a collection might be quite big, I see an option to paginate the collection, while receiving real time updates based on snapshots. So I initially load x comments with the option to load up to x more items whenever the user presses a button. In the image below x = 3.
The standard solution
Based on other SO questions I figured out that one is supposed to use the .limit() and the .startAfter() methods to implement such behaviour.
So the first page is loaded as:
query = this
.collection
.orderBy('date', descending: true)
.limit(pageSize);
query.snapshots().map((QuerySnapshot snap) {
lastVisible = snap.documents.last;
// convert the DocumentSnapshot into model object
});
All additional pages are loaded with the following code:
query = this.collection
.orderBy('date', descending: true)
.startAfterDocument(lastVisible)
.limit(pageSize);
Furthermore, I'd like to add that this code is located in a repository class which is used with the BLoC pattern similar to the code shown in Felix Angelov's Flutter Todos Tutorial.
While Felix uses a simple flutter list to show the items, I have a list of pages showing comments based on the data provided by their BLoCs. Note that each BLoC accesses a shared repository (parts of the repository code is shown below).
The Problem with the standard solution
With the code shown above I see multiple problems:
If a comment is inserted in the middle of the ordered collection (how is not of importance), the added comment is shown because of the Stream provided by the snapshot. However, another comment that already existed is not longer shown because of the .limit() operator in the query. One could increase the limit by one but I'm not sure how to edit a snapshot query. In the case that editing a snapshot query is not possible, one could create a new (and bigger) query, but that would cost additional reads.
Similar to 1., if a comment in the middle is deleted, the snapshot will return a list which does not longer contain the deleted comment, however another comment (which is already covered by a different page) appears. E.g., in the scenario shown in the image above 5 comments are loaded. Assuming that comment 3 is deleted, comment 2 will show twice.
Improving the standard solution
Based on these two problems discussed above, I decided that the solution is not sufficient and I implemented a solution which first loads x items by obtaining two "interval" documents. Then a query which fetches the required items in an interval using .startAtDocument() and .endAtDocument() is created, which eliminates the .limit() operator.
DocumentSnapshot pageStartDocument;
DocumentSnapshot pageEndDocument;
Future<Stream<List<Comment>>> comments() async {
// This fetches the first and next Document as initialization
// (maybe should be implemented in constructor)
if (pageStartDocument == null) {
Query query = collection
.orderBy('date', descending: true)
.limit(pageSize);
QuerySnapshot snap = await query.getDocuments();
pageStartDocument = snap.documents.first;
pageEndDocument = snap.documents.last;
} else {
Query query = collection
.orderBy('date', descending: true)
.startAfterDocument(pageEndDocument)
.limit(pageSize);
QuerySnapshot snap = await query.getDocuments();
pageStartDocument = snap.documents.first;
pageEndDocument = snap.documents.last;
}
// This fetches a subcollection of elements from the collection
// with the tradeof of double the reads
Query query = this
.collection
.orderBy('date', descending: true)
.startAtDocument(pageStartDocument)
.endAtDocument(pageEndDocument);
return query.snapshots().asyncMap((QuerySnapshot snap) async {
// convert the QuerySnapshot into model objects
});
As commented in the code, this solution has the following drawback:
Since a query is required to obtain the pageStartDocument and pageEndDocument, the number of reads is doubled, because all the data is read again when the second query is created. The performance impact might be neglectable because I believe the data is cashed, however having 2x database read cost can be significant.
Question:
Since I am not only implementing pagination but also real time updates (with collection insertions), the .limit() operator seems to be not working in my case.
How does one implement a pagination with real time updates (without double reads)?
Side Notes:
I watched how Todd Kerpelman devoures a massive gummy bear while explaining pagination, but in the video it seems to be not so trivial (and a point was made that a tradeoff might be necessary).
If further code from my side is required please say so in the comments.
For the scenario of comments it does not really makes sense that an item is inserted into the middle of the (sorted) collection. However I would like to understand how it should be implemented if the scenario requires such a feature.
this may come as a very late answer. The OP probably won't need help anymore, however for anyone who should stumble on this I wrote a tutorial with a solution that partly solve this:
the Bloc keep a list of stream subscription to keep trace of realtime updates to the list.
however concerning the insertion problem, since when you will have paginated streams based on a document cursor, upon insertion or deletion you necessarily need to reset your pagination stream subscriptions unless it is the last page.
Hence my solution around it was to update the list when modifications occur but reset it when insertions or deletions occur.
Here is the link to the tutorial :
https://link.medium.com/2SPf2Qsbsgb