https://firebase.google.com/docs/firestore/use-rest-api#making_rest_calls
Hi,
I want to fetch all the documents from my collection using REST for reporting purposes.
I tried using list method in API explorer but I am only getting max 30 documents at a time and for next page I have to use the nextPageToken.
I have even tried giving the pageSize to 100, even then it is returning only 30 documents as it is for maximum number of documents to return. Is there any way I can fetch all documents?
I have around 3-4k simple documents.
The example here works for me: https://stackoverflow.com/a/48889822/2441655
Example:
https://firestore.googleapis.com/v1/projects/YOUR_PROJECT/databases/(default)/documents/YOUR_DOC_PATH?pageSize=300
You can use paging by finding the "nextPageToken" at the end of the json, then inserting it like so:
https://firestore.googleapis.com/v1/projects/YOUR_PROJECT/databases/(default)/documents/YOUR_DOC_PATH?pageSize=300&pageToken=NEXT_PAGE_TOKEN_HERE
However it still limits the max pageSize to 300 for me. (odd that it limits it to 30 for you)
Related
Using mongocxx driver (c++ project).
Working on a mongodb query to paginate some results from a query. I'm trying to return the first 10 results, while also informing whether or not there are more results to fetch with another query (using an offset) - so as to inform the recipient if there are more documents to fetch. The results are stored in a std::vector after the db find query.
Is there any elegant way to do this, preferably without returning all the result documents and then comparing the vector size to the specified page limit?
Current query (without specifics):
db.collection.find({"<some_field>" : <some value>}).limit(10);
This, however, will not inform whether or not any documents were removed, in the case that exactly 10 results were found.
Currently I'm simply returning the full vector of results and looping through it, breaking if the loop goes over 10 iterations (and setting a "more_items" bool to true).
You have 2 ways to do this:
Count all documents found by query:
db.collection.count({"<some_field>" : <some value>});
And then if there is more documents than you need (10 in here) - you can set "more_items" bool to true
Find and set limit to +1 (11 in here):
db.collection.find({"<some_field>" : <some value>}).limit(11);
That way you find 11 documents or less.
If you find 11 documents - this indicates that you have more documents than 10 (actual limit). If you find less than 11 - then you don't have documents to reach actual limit.
I have a document structure which looks something like this:
{
...
"groupedFieldKey": "groupedFieldVal",
"otherFieldKey": "otherFieldVal",
"filterFieldKey": "filterFieldVal"
...
}
I am trying to fetch all documents which are unique with respect to groupedFieldKey. I also want to fetch otherField from ANY of these documents. This otherFieldKey has minor changes from one document to another, but I am comfortable with getting ANY of these values.
SELECT DISTINCT groupedFieldKey, otherField
FROM bucket
WHERE filterFieldKey = "filterFieldVal";
This query fetches all the documents because of the minor variations.
SELECT groupedFieldKey, maxOtherFieldKey
FROM bucket
WHERE filterFieldKey = "filterFieldVal"
GROUP BY groupFieldKey
LETTING maxOtherFieldKey= MAX(otherFieldKey);
This query works as expected, but is taking a long time due to the GROUP BY step. As this query is used to show products in UI, this is not a desired behaviour. I have tried applying indexes, but it has not given fast results.
Actual details of the records:
Number of records = 100,000
Size per record = Approx 10 KB
Time taken to load the first 10 records: 3s
Is there a better way to do this? A way of getting DISTINCT only on particular fields will be good.
EDIT 1:
You can follow this discussion thread in Couchbase forum: https://forums.couchbase.com/t/getting-distinct-on-the-basis-of-a-field-with-other-fields/26458
GROUP must materialize all the documents. You can try covering index
CREATE INDEX ix1 ON bucket(filterFieldKey, groupFieldKey, otherFieldKey);
I have used the call
GroupId?fields=feed{full_picture,place,from,comments{message},message,likes.limit(0).summary(true)}
to get the stated data for my application. But this only returns 25 feeds/posts. I also tried and limit number &limit=number (number like 1000) appending to the above. Yet it returns only 25. How to get all the posts?
Search for "Traversing Paged Results" in the docs:
https://developers.facebook.com/docs/graph-api/using-graph-api#reading
https://developers.facebook.com/docs/php/howto/example_pagination_basic
Some API calls have a max limit, so paging would be the only reliable way to get all items.
I am looking for something like FOUND_ROWS() in mysql select query in algolia results as I need to keep a track of how many total results to expect. Is there someway to get this in Algolia?
The proper way to obtain the number of results is to access the nbHits value which is available in the JSON response of every search call.
I am trying to use limit along with since and until in Graph Api call. In between a certain time interval, I have 2 uploaded photos. The query looks like this:
me/photos/uploaded?since=2009-12-31T18%3A30%3A00.000Z&until=2010-12-31T18%3A30%3A00.000Z&limit=200
This query has limit set as 200 , and it correctly returns the 2 photos. But the problem arises when I have a smaller value of limit, lets say 100. So basically a query:
me/photos/uploaded?since=2009-12-31T18%3A30%3A00.000Z&until=2010-12-31T18%3A30%3A00.000Z&limit=100
return me an empty json. Even though there are only two photos, a limit of 100 returns null. Any insights?