I am trying to paginate data with Firestore, and would be ordering data on columns where duplicates are expected, if the pagination happens to be among those values its expected that it won't work correctly.
I can work around this issue by using StartAfter based on Document ID which will be always be unique.
One way I can accomplish this is passing id of the last document to server side rest api request. This would require two steps, i.e. to fetch the DocumentSnapshot using the DocumentId and constructing the query based on it
var lastSnapshot = fetchSnapshot(id);
citiesRef.OrderBy("Population").StartAfter(lastSnapshot);
Other approach is to persist the DocumentId in the document while creation.This would require two steps each time when the document is created, one to create and the other to update immediately with Id generated (As I don't see a way to persist DocumentId during creation itself)
citiesRef.OrderBy("Population").StartAfter(lastId);
Which one of these is a good approach to follow, either to fetch DocumentSnapshot and not to persist id into the document, or perform two operations by persisting the DocumentId in the first place and using it as key for StartAfter.
Decided to go with Option 1, instead of persisting Document ID in the Document itself as in Option 2.
Related
I have a firestore DB where I'm storing polls in one collection and responses to polls in another collection. I want to get a document from the poll collection that isn't referenced in the responses collection for a particular user.
The naive approach would be to get all of the poll documents and all of the responses filtered by user ID then filter the polls on the client side. The problem is that there may be quite a few polls and responses so those queries would have to pull down a lot of data.
So my question is, is there a way to structure my data so that I can query for polls that haven't been completed by a user without having to pull down the collections in their entirety? Or more generally, is there some pattern to use when you need to query for documents in one collection that aren't referenced by another?
The documents in each of the collections look something like this:
Polls:
{
question: string;
answers: Answer[];
}
Responses:
{
userId: string;
pollId: string;
answerId: string;
}
Anyhelp would be much appreciated!
Queries in Firestore can only return documents from one collection (or from all collections with the same name) and can only contain conditions on the data that they actually return.
Since there's no way to filter based on a condition in some other documents, you'll need to include the information that you want to filter on in the polls documents.
For example, you could include a completionCount field in each poll document, that you initially set to 0, and then update only every poll completion. With that in place, the query becomes a simple query on the completionCount field of the polls collection.
For a specific user I'd actually add all polls to their profile document, and remove them from there. Duplicating data is usually the easiest (and sometimes only) way to implement use-cases such as this.
If you're worried about having to add each new poll to each new user profile when it is created, you can also query all polls on their creation timestamp when you next load a user profile and perform that sync at that moment.
load user profile,
check when they were last active,
query for new polls,
add them to user profile.
I'm currently working on an application where users can create groups and invite others in it.
I would like people in the same group to be able to see their first and last names.
To do that, I have a collection named Users where each of the users have a document contains all their personnal data, like first and last names, phone, position , ...
I have also another collection named Groups, where all of my groups are stored, with their name, and an array contaning the ID of the members.
When an user open the app, a first request is done for request his groups (he recieve the groups names and the arrays of members). After, if he want to know the user in a certain group, another request is done for search only the first and last name of all the members.
So, I imagine that there is a query that will return me only the fields that I would like to retrieve, and that there is a rule allowing a potential hacker to be refused access to the entire user document except if the user is the owner of the document.
// For retrieving my user's groups
Stream<List<Group>?> get organizations {
return firestore
.collection('Groups')
.where('members', arrayContains: this.uid)
.snapshots()
.map(_groupsFromSnapshot);
}
// For retrieving names of the members of a group
Stream<List<Member>?> getMembers(Group group){
return firestore
.collection('Users')
// and i dont know what to do here ...
}
With the Client SDKs and the Flutter plugin it is not possible to get only a subset of the fields of a Document. When you fetch a Document you get it with all its fields.
If you want to get only a subset of the fields of a document, you can implements the two following approaches:
Denormalize your data: You create another collection which contains documents that only contain the fields you want to expose. You need to synchronize the two collections (the Users collection, which is the "master", and the new collection): for that it's quite common to use a Cloud Function. Note also that it's a good idea to use the same documentID for the linked documents in the two collections.
Use the Firestore REST API to fetch the data: With the REST API you can use a DocumentMask when you fetch one document with the get method or a Projection when you query a Collection. The DocumentMask or the Projection will "restrict a get operation on a document to a subset of its fields". You can use the http package for calling the API from your Flutter app.
HOWEVER, the second approach is not valid if you want to protect the other users data: a malicious user could call the Firestore REST API with the same request but without a DocumentMask or a Projection. In other words, this approach is interesting if you just want to minimize the network traffic, not if you want to keep secret certain fields of a document.
So, for your specific use case, you need to go for the first solution.
I would like to create two queries, with pagination option. On the first one I would like to get the first ten records and the second one I would like to get the other all records:
.startAt(0)
.limit(10)
.startAt(9)
.limit(null)
Can anyone confirm that above code is correct for both condition?
Firestore does not support index or offset based pagination. Your query will not work with these values.
Please read the documentation on pagination carefully. Pagination requires that you provide a document reference (or field values in that document) that defines the next page to query. This means that your pagination will typically start at the beginning of the query results, then progress through them using the last document you see in the prior page.
From CollectionReference:
offset(offset) → {Query}
Specifies the offset of the returned results.
As Doug mentioned, Firestore does not support Index/offset - BUT you can get similar effects using combinations of what it does support.
Firestore has it's own internal sort order (usually the document.id), but any query can be sorted .orderBy(), and the first document will be relative to that sorting - only an orderBy() query has a real concept of a "0" position.
Firestore also allows you to limit the number of documents returned .limit(n)
.endAt(), .endBefore(), .startAt(), .startBefore() all need either an object of the same fields as the orderBy, or a DocumentSnapshot - NOT an index
what I would do is create a Query:
const MyOrderedQuery = FirebaseInstance.collection().orderBy()
Then first execute
MyOrderedQuery.limit(n).get()
or
MyOrderedQuery.limit(n).get().onSnapshot()
which will return one way or the other a QuerySnapshot, which will contain an array of the DocumentSnapshots. Let's save that array
let ArrayOfDocumentSnapshots = QuerySnapshot.docs;
Warning Will Robinson! javascript settings is usually by reference,
and even with spread operator pretty shallow - make sure your code actually
copies the full deep structure or that the reference is kept around!
Then to get the "rest" of the documents as you ask above, I would do:
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).get()
or
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).onSnapshot()
which will start AFTER the last returned document snapshot of the FIRST query. Note the re-use of the MyOrderedQuery
You can get something like a "pagination" by saving the ordered Query as above, then repeatedly use the returned Snapshot and the original query
MyOrderedQuery.startAfter(ArrayOfDocumentSnapshots[n-1]).limit(n).get() // page forward
MyOrderedQuery.endBefore(ArrayOfDocumentSnapshots[0]).limit(n).get() // page back
This does make your state management more complex - you have to hold onto the ordered Query, and the last returned QuerySnapshot - but hey, now you're paginating.
BIG NOTE
This is not terribly efficient - setting up a listener is fairly "expensive" for Firestore, so you don't want to do it often. Depending on your document size(s), you may want to "listen" to larger sections of your collections, and handle more of the paging locally (Redux or whatever) - Firestore Documentation indicates you want your listeners around at least 30 seconds for efficiency. For some applications, even pages of 10 can be efficient; for others you may need 500 or more stored locally and paged in smaller chucks.
I am new to NoSQL and I'm trying to figure out a good way to represent my data. I have a series of workers that need to request vacations via mobile app.
When I try to write a Firebase query with Flutter, I can do this:
Firestore.instance
.collection("ferie_permessi")
.document("worker1#test.com")
.snapshot();
It works but there are two main errors:
If I try to create another collection called "Woker info" I cannot use worker1#test.com as document ID as it already esists;
I have to sort data client side because firestore doesn't give me the possibility (with this setup I've made).
I'm quite sure that this structure isn't good at all. Each worker needs to have 2 lists: vacations and other. What is wrong?
My guess is that I should move worker1#test.com together with vacations and other so that I can make a query of this kind:
Firestore.instance
.collection("ferie_permessi")
.where("user", "==", "worker1#test.com)
.snapshot();
But now the id? Is an automatic one good?
I had a chance to recently explore creating an app using a firebase-firestore. A couple of things will help here:
Yes, the autogenerated id is good since it is unique, for example, you can have a collections vacation_requests, users you can then use that user_id as a document in vaccation_requests -> user_id -> vacations, instead of using email as a document key.
Or
You can do it like this collections users, vacation_requests, and requests.
store user details in users.
store requests in requests with from and to dates.
store reference of User and Request in vaccation_requests.
Hope this helps!
I have a collection in mongodb that stores activities of customers like product_view, added_to_cart etc with productId. I need this data to display products to my customer when he visits next.
Right now I am thinking to store all data of a customer in a single document,such as with customer_id as key and corresponding activities in array like product_view activities in product_view array etc.This will be fast to fetch for me as all data of a customer will be in one key only, but my consideration is that data size will go on increasing always this way. Moreover I may need to check say last 50-100 activities of a customer only. For that too I need to fetch the entire document.
What will be the best way to store this data. Request for data will be very very frequent. How can I manage response time ?
Your question answers itself. Have customer activity as separate collection with reference to customerId. Any time customer visits, you know customerId, hence can apply filter/aggregate operations to get whatever you want.
This way you can do paginated fetch of customer activities.