How to get paginated/sliced data of subdocument array in mongo collection? - mongodb

Question

I have a collection like this:
{
"_id" : ObjectId("51f4ad560364f5490ccebe26"),
"fiTpcs" : [
"uuid1",
"uuid2",
"uuid3",
"uuid4",
"uuid5"
],
"fiTpcsCnt" : 5
}
The list of fiTpcs is long and can go to hundreds later. When I retrieve my collection, I want to get a limited list of fiTpcs, say 20 at a time and fire separate requests to get subsequent data from fiTpcs. I just want to ensure that the queries don't get slow later when I have a lot more data. Is there a way to do it in mongodb? until now, I have been doing
db.userext.find({"_id" : ObjectId("51f4ad560364f5490ccebe26")}).pretty();
which always gets me the full fiTpcs array. I am using java driver with Spring and a solution using Spring/java would also be fine. Please note - if the solution requires mongo to scan through the whole fiTpcs array and then slice a part of it, it doesn't really add any performance benefits, that is not what I am looking for.

Answer 1

I may not understand your question in full depth, but seems like $slice is the droid your are looking for:
> db.page.find()
{ "_id" : ObjectId("51f4ad560364f5490ccebe26"), "fiTpcs" : [ "uuid1", "uuid2", "uuid3", "uuid4", "uuid5" ], "fiTpcsCnt" : 2 }
> db.page.find({}, {"fiTpcs" : {$slice : 3}})
{ "_id" : ObjectId("51f4ad560364f5490ccebe26"), "fiTpcs" : [ "uuid1", "uuid2", "uuid3" ], "fiTpcsCnt" : 2 }
> db.page.find({}, {"fiTpcs" : {$slice : [1,3]}})
{ "_id" : ObjectId("51f4ad560364f5490ccebe26"), "fiTpcs" : [ "uuid2", "uuid3", "uuid4" ], "fiTpcsCnt" : 2 }

Answer 2

After a couple of days of thinking/trying various options, this is what I did finally. I modified my document like this:
{
"_id" : ObjectId("51f4ad560364f5490ccebe26"),
"page" : 1, //1 is the default
"slug" : "some-unique-string-identifier"
"fiTpcs" : [
"uuid1", //these could be long text, like a long comment/essay
"uuid2",
"uuid3",
"uuid4",
"uuid5"
],
"fiTpcsCnt" : 5
}
I keep a "pageCount" and "totalFiTpcsCnt" in memcached. I have set MAX_FITPCSCNT = 500 (500 for now, experimental). When I create a new document of type userext, I set the page value to 1.
If I have to push a new object to fiTpcs array:
1) check if "totalFiTpcsCnt" is a multiple of 500. If yes, create a new document of type userext with the same slug, fiTpcsCnt as 0 and fiTpcs array as null.
2) update the last userext - query by slug and "pageCount", push to fiTpcs. Evict cache for "pageCount" and "totalFiTpcsCnt".
Whenever I need my userext document, I always take just the first page. This way I'll never need to query for more than 500 objects of type fiTpcs at a time and I will still have totalFiTpcsCnt always updated in memcached.