I have a capped collection. When I call stats() on it I get following output:
/* 0 */
{
"ns" : "log_db.access_logs",
"count" : 42088,
"size" : 13602632,
"avgObjSize" : 323,
"storageSize" : 100003840,
"numExtents" : 1,
"nindexes" : 2,
"lastExtentSize" : 100003840,
"paddingFactor" : 1,
"systemFlags" : 1,
"userFlags" : 0,
"totalIndexSize" : 3932656,
"indexSizes" : {
"_id_" : 1389920,
"apikey_1_ts_1" : 2542736
},
"capped" : true,
"max" : NumberLong(9223372036854775807),
"ok" : 1
}
I don't know what 'max' parameter is showing. I created this collection using db.runCommand('convertToCapped' ... ) construct and I had put the 'size' parameter to be 100 million. But here there's no mention of it (size)
Can someone explain the meaning of 'max' field here and also how to find correct size of a capped collection.
I had this exact question once, since it was not in the documentation and this is the answer I got from a 10gen (MongoDB Inc) employee mongodb capped collection is not using all available space :
The max property is an (optional) maximum number of documents to allow in the capped collection (see db.createCollection()). If you don't specify a max value, it will be set to MAXINT (in your example, the maximum positive value for a signed Int64). Since space for capped collections is always preallocated, the size limit takes precedence over the max number of documents.
Related
About the statistics of a collection
From MongoDB: The Definite Guide, 2ed, by Kristina Chodorow, 2013,
which I guess uses MongoDB 2.4.0, it says that "storageSize" is
greater than "size"
For seeing information about a whole collection, there is a stats function:
> db.boards.stats()
{
"ns" : "brains.boards",
"count" : 12,
"size" : 32292,
"avgObjSize" : 2691,
"storageSize" : 270336,
"numExtents" : 3,
"nindexes" : 2,
"lastExtentSize" : 212992,
"paddingFactor" : 1.0099999999999825,
"flags" : 1,
"totalIndexSize" : 16352,
"indexSizes" : {
"_id_" : 8176,
"username_1_slug_1" : 8176
},
"ok" : 1
}
"size" is what you’d get if you called Object.bsonsize() on each
element in the collection and added up all the sizes: it’s the actual
number of bytes the document in the collection are taking up.
Equivalently, if you take the "avgObjSize" and multiply it by "count",
you’ll get "size".
As mentioned above, a total count of the documents’ bytes leaves out
some important space a collection uses: the padding around each
document and the indexes. "storage Size" not only includes those, but
also empty space that has been set aside for the collection but not
yet used. Collections always have empty space at the “end” so that new
documents can be added quickly.
On my local computer, I experiment with MongoDB 3.2 to get the
statistics of a collection, and find that "storageSize" is smaller
than "size"
> db.c20160712.stats(1024)
{
"ns" : "TVFS.c20160712",
"count" : 2305,
"size" : 231,
"avgObjSize" : 102,
"storageSize" : 80,
...
Do the meanings of "storageSize" and "size" change in MongoDB 3.2 from 2.x?
If yes, what do they mean in 3.2?
Thanks.
Wat is the minimal document size in MongoDb.
If I insert the following Json {"_id",1234} or {"_id":NumberLong(1),"g":"1383,5,2000,1"} or {"_id":ObjectId("5652e26c1a8014b040aacc54"),"v" : "3,123"}.
All three documents takes up 48 bytes. Is 48 bytes the minimum size of the document?
MongoDB supports dynamic schema i.e documents within same collection might have different schema.
So there is no fixed minimal size of document in MongoDB.
To determine size of document in MongoDB execute the following code snippet
db.test.stats()
Following is response of stats() method
{
"ns" : "pcat.test",
"count" : 2,
"size" : 736,
"avgObjSize" : 368,
"numExtents" : 1,
"storageSize" : 8192,
"lastExtentSize" : 8192.0000000000000000,
"paddingFactor" : 1.0000000000000000,
"paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",
"userFlags" : 1,
"capped" : false,
"nindexes" : 2,
"indexDetails" : {},
"totalIndexSize" : 16352,
"indexSizes" : {
"_id_" : 8176,
"accounts.platform_id_1_accounts.id_1" : 8176
},
"ok" : 1.0000000000000000
}
From above mentioned output key avgObjSize consists of avg document size (in bytes) across collection.
Alternative you can determine size of individual document in collection using following code snippet
Object.bsonsize(db.collection.find({ "_id" :1}))
But there is a restriction to maximum size of document in MongoDB.
The maximum BSON document size is 16 megabytes. The maximum document
size helps ensure that a single document cannot use excessive amount
of RAM or, during transmission, excessive amount of bandwidth.
I've noticed a strange behaviour of MongoDB and I'm try to guess what can be the problem:
I've a MongoDB with a lot of documents inside a collection. I've run the following query:
db.mydocuments.count({_id:{$lte:new ObjectId("549010c9e4b06c2f044f27f4")}});
The result is 66.579.389 documents
Than I ran the following:
db.mydocuments.count();
and surprisingly I've got the following total: 32.606.242
How this can be? how the whole count of a collection can be less than a count with a query?At least it need to be equal to the query count.
The db.mydocument.stats() is:
{
"ns" : "mydb.documents.photos",
"count" : 32606242,
"size" : 76109891776,
"avgObjSize" : 2334,
"storageSize" : 164665658240,
"numExtents" : 97,
"nindexes" : 1,
"lastExtentSize" : 2146426864,
"paddingFactor" : 1,
"systemFlags" : 0,
"userFlags" : 0,
"totalIndexSize" : 1944138336,
"indexSizes" : {
"_id_" : 1944138336
},
"ok" : 1
}
.stats() returns you amount of allocated records as well as average object size, extents etc.
This call is not supposed to get exact information about your collection (like a number of documents).
Example:
you inserted 100 docs, then deleted 99. The collection is still allocated to handle 100 docs.
Do not rely on .stats() call unless you need it for informational purposes only.
In your case, we have exact opposite behavior. May be, broken index.
I am looking at the output of db.system.profile.stats() and I'm curious about what the max field means in the returned document (running mongodb 2.2.2).
Here's an example:
> db.system.profile.stats()
{
"ns" : "mydb.system.profile",
"count" : 2476,
"size" : 1012284,
"avgObjSize" : 408.83844911147014,
"storageSize" : 1052672,
"numExtents" : 2,
"nindexes" : 0,
"lastExtentSize" : 4096,
"paddingFactor" : 1,
"systemFlags" : 0,
"userFlags" : 0,
"totalIndexSize" : 0,
"indexSizes" : {
},
"capped" : true,
"max" : 2147483647,
"ok" : 1
}
There is no mention of max on the official mongodb documentation of db.collection.stats().
Perhaps it has something to do with the fact that system.profile is a capped collection. Although max is definitely not the maximum size of the capped collection because (1) the max shown is a huge number and (2) my collection doesn't get larger than 2500 or so documents and the total size is much less than this.
Any thoughts?
Thanks,
Kevin
max is an optional setting for a capped collection to also limit the number of documents in the collection, instead of just limiting by number of bytes (size).
See docs here.
I have 57M documents in my mongodb collection, which is 19G of data.
My indexes are taking up 10G. Does this sound normal or could I be doing something very wrong! My primary key is 2G.
{
"ns" : "myDatabase.logs",
"count" : 56795183,
"size" : 19995518140,
"avgObjSize" : 352.0636272974065,
"storageSize" : 21217578928,
"numExtents" : 39,
"nindexes" : 4,
"lastExtentSize" : 2146426864,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 10753999088,
"indexSizes" : {
"_id_" : 2330814080,
"type_1_playerId_1" : 2999537296,
"type_1_time_-1" : 2344582464,
"type_1_tableId_1" : 3079065248
},
"ok" : 1
}
The index size is determined by the number of documents being indexed, as well as the size of the key (compound keys store more information and will be larger). In this case, the _id index divided by the number of documents is 40 bytes, which seems relatively reasonable.
If you run db.collection.getIndexes(), you can find the index version. If {v : 0}, the index was created prior to mongo 2.0, in which case you should upgrade to {v:1}. This process is documented here: http://www.mongodb.org/display/DOCS/Index+Versions