Is there any way I can fetch all the pokemon's data at once? Currently, I only see the option to fetch with a limit:
Passing -1 to the limit parameter will return all the records
Related
I am using skip & limit for mongodb C# driver to fetch tickets batchwise like below,
var data = db.collectionName.Find({}).Skip(1000).Limit(500).ToList()
Data fetching is happening as expected. Need confirmation on whether Sort() is mandatory for Skip & limit methods like below ? or sort will be handled by mongodb if not specified
var data = db.collectionName.Find({}).Sort("{_id:1}").Skip(1000).Limit(500).ToList()
I have removed Sort from query to improve time taken to complete fetch operation.
No, Sort() is not mandatory for Skip() and Limit() methods. You can use them directly like you are using in your query:
var data = db.collectionName.Find({}).Skip(1000).Limit(500).ToList()
To know more about default sort order, refer to below link:
https://stackoverflow.com/questions/11599069/how-does-mongodb-sort-records-when-no-sort-order-is-specified#:~:text=When%20we%20run%20a%20Mongo,objects%20in%20forward%20natural%20order.
https://firebase.google.com/docs/firestore/use-rest-api#making_rest_calls
Hi,
I want to fetch all the documents from my collection using REST for reporting purposes.
I tried using list method in API explorer but I am only getting max 30 documents at a time and for next page I have to use the nextPageToken.
I have even tried giving the pageSize to 100, even then it is returning only 30 documents as it is for maximum number of documents to return. Is there any way I can fetch all documents?
I have around 3-4k simple documents.
The example here works for me: https://stackoverflow.com/a/48889822/2441655
Example:
https://firestore.googleapis.com/v1/projects/YOUR_PROJECT/databases/(default)/documents/YOUR_DOC_PATH?pageSize=300
You can use paging by finding the "nextPageToken" at the end of the json, then inserting it like so:
https://firestore.googleapis.com/v1/projects/YOUR_PROJECT/databases/(default)/documents/YOUR_DOC_PATH?pageSize=300&pageToken=NEXT_PAGE_TOKEN_HERE
However it still limits the max pageSize to 300 for me. (odd that it limits it to 30 for you)
I've got a special case of query that I need to perform as optimized as possible,
Problem:
I have a collection of messages, each message has a field of groupID and I need to get last message of each groupID, but I don't want to perform one query for each of my groups, instead I wan't to give an array of groupIDs and get an array of Messages
Solutions so far:
I managed to come up with two soloutions
1. perform query for each groupID which works fine takes about 200 ms to complete but performs many requests on MongoDB
2. use Aggregate to groups messages based on GroupID and and then select first of each group which is really slow and takes about 4000 ms
Code
var filter = Builders<Message>.Filter.Eq("GroupID", groupID);
var sort = Builders<Message>.Sort.Descending("Date");
return await MessagesCollection.Find(filter).Sort(sort).Limit(1).FirstAsync();
what I'm looking for is a way to batch queries or do one query that can return first of mesage of given groupID, any idea??
any help will be appreciated
thanx in advance
I am using MongoTool runner to import the data from mongoDB to Hadoop mapreduce jobs. Due to the size of the data i am getting OutOfMemoryError. So i want to limit the number of records i fetch in a batch fashion.
MongoConfigUtil.setQuery()
can set only the query but i cannot set the size to limit the number of records fetched. What i am looking for is something like
MongoConfigUtil.setBatchSize()
and then
MongoConfigUtil.getNextBatch()
something like that.
Kindly suggest.
You can use the setLimit method of the class MongoInputSplit, passing the number of document that you want to fetch.
myMongoInputSplitObj = new MongoInputSplit(*param*)
myMongoInputSplitObj.setLimit(100)
MongoConfigUtil setLimit
Allow users to set the limit on MongoInputSplits (HADOOP-267).
Let's say i put a limit and skip on the MongoDB query...I want to know the total results if there was not a limit on there.
Of course, I could do this the shitty way...which is to query twice.
In MongoDB the default behavior of count() is to ignore skip and limit and count the number of results in the entire original query. So running count will give you exactly what you want.
Passing a Boolean true to count or calling size instead would give you a count WITH skip or limit.
There is no way to get the count without executing the query twice.