Can a firestore snapshot contain x number of changed documents - google-cloud-firestore

I want to ask about Firestore
Reading the docs
When a document is added, removed or modified then I get a signal about that event here in this code from the doc:
db.collection("cities").where("state", "==", "CA")
.onSnapshot((snapshot) => {
snapshot.docChanges().forEach((change) => {
if (change.type === "added") {
console.log("New city: ", change.doc.data());
}
if (change.type === "modified") {
console.log("Modified city: ", change.doc.data());
}
if (change.type === "removed") {
console.log("Removed city: ", change.doc.data());
}
});
});
When I start this listener will I then get all document matching "state", "==", "CA" even if it's 100.000 of them? Will they come all at once or in batches?
after the above initial all (100.000) will I after that always get one(1) document, like when a doc is modified, added or removed, or can there be like a batch collapsing latency from firestore so I will get 1-to-many in the snapshot?

When you first run the query, you'll get everything that matches that query, with a change.type === "added". Then you will receive changes as they are made, one by one (unless someone writes a batch at once).
The way to manage this is to add a filter to the collection. For example, you may want to sort the collection by a date field or a name field. Then limit the results to a manageable number and paginate.
db.collection("cities")
.where("state", ">=", "CA")
.orderBy("state")
.limit(50)
.onSnapshot((snapshot) => {
snapshot.docChanges().forEach((change) => {
if (change.type === "added") {
console.log("New city: ", change.doc.data());
}
if (change.type === "modified") {
console.log("Modified city: ", change.doc.data());
}
if (change.type === "removed") {
console.log("Removed city: ", change.doc.data());
}
});
});
Don't forget to add an unsubscribe so that you can remove the listener

You get only the documents that were added/modified/removed however the documentation clearly states the first time when the listener is set up it will return all the matching documents documents.
The first query snapshot contains added events for all existing
documents that match the query. This is because you're getting a set
of changes that bring your query snapshot current with the initial
state of the query. This allows you, for instance, to directly
populate your UI from the changes you receive in the first query
snapshot, without needing to add special logic for handling the
initial state.
So they do come all along. After that they'll be only the changes i.e. the amount of documents being changed.
The initial state can come from the server directly, or from a local
cache. If there is state available in a local cache, the query
snapshot will be initially populated with the cached data, then
updated with the server's data when the client has caught up with the
server's state.
Now with that being said you may be charged for it if the data is coming from the server and not from the local cache.

Related

How to update a firebase document field based on a value from another document in other collection? Flutter

I have a number in the usersnumber collection (counter document), and I want to take that number and put it in the users collection in the number field (as you can see in the photo). Is there any way I can get the data from usersnumber and update the users > collection > document > number?
I expect having the number 10 from the usersnumber collection in the collection users > document > number: 0
a transaction is a set of read and write operations on one or more
documents.
Using the Cloud Firestore client libraries, you can group multiple
operations into a single transaction. Transactions are useful when you
want to update a field's value based on its current value, or the
value of some other field.
https://firebase.google.com/docs/firestore/manage-data/transactions#transactions
import { runTransaction } from "firebase/firestore";
try {
await runTransaction(db, async (transaction) => {
const sfDoc = await transaction.get(sfDocRef);
if (!sfDoc.exists()) {
throw "Document does not exist!";
}
const newPopulation = sfDoc.data().population + 1;
transaction.update(sfDocRef, { population: newPopulation });
});
console.log("Transaction successfully committed!");
} catch (e) {
console.log("Transaction failed: ", e);
}

Dart/Flutter stream to a field/data in firebase [duplicate]

How can I listen to a specific field change with firestore js sdk ?
In the documentation, they only seem to show how to listen for the whole document, if any of the "SF" field changes, it will trigger the callback.
db.collection("cities").doc("SF")
.onSnapshot(function(doc) {
console.log("Current data: ", doc && doc.data());
});
You can't. All operations in Firestore are on an entire document.
This is also true for Cloud Functions Firestore triggers (you can only receive an entire document that's changed in some way).
If you need to narrow the scope of some data to retrieve from a document, place that in a document within a subcollection, and query for that document individually.
As Doug mentioned above, the entire document will be received in your function. However, I have created a filter function, which I named field, just to ignore document changes when those happened in fields that I am not interested in.
You can copy and use the function field linked above in your code. Example:
export const yourCloudFunction = functions.firestore
.document('/your-path')
.onUpdate(
field('foo', 'REMOVED', (change, context) => {
console.log('Will get here only if foo was removed');
}),
);
Important: The field function is not avoiding your function to be executed if changes happened in other fields, it will just ignore when the change is not what you want. If your document is too big, you should probably consider Doug's suggestion.
Listen for the document, then set a conditional on the field you're interesting in:
firebase.firestore().collection('Dictionaries').doc('Spanish').collection('Words').doc(word).collection('Pronunciations').doc('Castilian-female-IBM').onSnapshot(function(snapshot) {
if (snapshot.data().audioFiles) { // eliminates an error message
if (snapshot.data().audioFiles.length === 2) {
audioFilesReady++;
if (audioFilesReady === 3) {
$scope.showNextWord();
}
}
}
}, function(error) {
console.error(error);
});
I'm listening for a document for a voice (Castilian-female-IBM), which contains an array of audio files, in webm and mp3 formats. When both of those audio files have come back asynchronously then snapshot.data().audioFiles.length === 2. This increments a conditional. When two more voices come back (Castilian-male-IBM and Latin_American-female-IBM) then audioFilesReady === 3 and the next function $scope.showNextWord() fires.
Just out of the box what I do is watching before and after with the before and after method
const clientDataBefore = change.before.data();
console.log("Info database before ", clientDataBefore);
const clientDataAfter = change.after.data();
console.log("Info database after ", clientDataAfter );
For example now you should compare the changes for a specific field and do some actions or just return it.
Some more about before.data() and after.data() here

Latency compensation and Firestore transactions

I have an onSnapshot keeping track of the documents in a collection:
db.collection('/.../').onSnapshot(querySnapshot=> mylocalvariable = querySnapshot.docs)
Now, I want to select the first (in some order) element of this collection of documents that my user has not yet handled. When a user is done handling a document, I use a transaction to update the document according to the user's needs (transaction is better for me than .update() because I might have multiple users changing different parts of the document).
The problem is that unlike a .update (which would update mylocalvariable immediately), it seems like the transaction finishes without updating mylocalvariable. So, when I go to grab the "next" document, it just grabs the same document, because the function runs before the variable gets updated.
Code sample:
db.collection('/mycollection').onSnapshot(querySnapshot=> mylocalvariable = querySnapshot.docs)
function selectnextrecord(){
nextrecord = mylocalvariable.find(x=>!x.data().done)
console.log(nextrecord)
//expected: Get something different than the current record
//observed: This is being run with old data, so it returns the same record that I currently have with the old data.
}
let nextrecord;
selectnextrecord();
function submitchanges(){
let sfDocRef = db.collection('/mycollection').doc(nextrecord.id);
return db.runTransaction(function(transaction) {
return transaction.get(sfDocRef).then(function(sfDoc) {
if (!sfDoc.exists) {
throw "Document does not exist!";
}
transaction.update(sfDocRef, {done:true});
});
}).then(function() {
selectnextrecord();
}).catch(function(error) {
console.log("Transaction failed: ", error);
});
}```
After going through the documentation, I think this is expected behavior.
Do not modify application state inside of your transaction functions. Doing so will introduce concurrency issues, because transaction functions can run multiple times and are not guaranteed to run on the UI thread. Instead, pass information you need out of your transaction functions
In any case, you could filter the documents that are not done with .where() and then place your transaction inside a foreach:
db.collection('cities')
.where("done", "==", true)
.get()
.then(snapshot => {
snapshot.forEach(doc => {
return db.runTransaction(function(transaction) {
return transaction.get(sfDocRef).then(function(sfDoc) {
if (!sfDoc.exists) {
throw "Document does not exist!";
}
transaction.update(sfDocRef, {done:true});
});
}).catch(function(error) {
console.log("Transaction failed: ", error);
});
})
})

Get number of documents in a big collection in Cloud Firestore

I know this question was already asked but I'm being specific about my case: I've got a large database (approximately 1 million documents inside the collection users).
I wanna get the exact number of documents inside users. I'm trying this:
export const count_users = functions.https.onRequest((request, response) => {
corsHandler(request, response, () => {
db.collection('users').select().get().then(
(snapshot) => response.json(snapshot.docs.length)
)
.catch(function(error) {
console.error("[count_users] Error counting users: ", error);
response.json("Failed");
});
});
});
Although it seems right, it takes forever to give me a result. I'm not allowed to add or remove documents from the database.
Is there any possible approach for getting this quantity?

mongodb: collection.update inside cursor.each resolves before update is completed

First of all, I'm using mongodb-promise as a wrapper to MongoClient.
I need to fetch all records from a collection "people" that matches specific criteria and then update each of them.
For that I have this code to find all people:
return db.collection('people')
.then( (collection) => {
// Store reference to collection for future use
peopleCollection = collection;
return collection.find({a:1})
})
An then invoke this to update each record:
.then( (people) => {
// Process each people
return people.each( (person) => {
person.b = 2;
// Where peopleCollection is a reference to my collection
return peopleCollection.update({_id: person._id}, person)
})
})
I then add another promise chain to fetch all people where b != 2 and I find many records and I counted them. But when I execute this script repeatedly, the count decreases which means mongo is still updating other records when the promise has already resolved. What am I missing here?
Maybe:
.then( (people) => {
// Process each people
return people.each( (person) => {
// Where peopleCollection is a reference to my collection
return peopleCollection.update({_id: person._id}, {$set:{b:2}})
})
})