I'm using a mongodb capped collection which is tailable. I want to set the size of it to a maximum value as I don't want to really have the oldest records removed according to the FIFO rule.
I want the data to pesist for as long as possible whilst keeping the features of a capped collection.
Thanks
You can make the capped collection as big as you want; just set the size parameter of create_collection to a value big enough to not run out of space.
Like this:
db.create_collection('captest', capped=True, size=20000000000)
You can create the capped collection using size (in bytes), or max (maximum number of documents to keep), or both in this case:
pymongo ->
self.db.create_collection('system_health_log', capped=True, size=5242880, max=5000)
mongo shell ->
db.createCollection("system_health_log", { capped : true, size : 5242880, max : 5000 } )
See more here on Capped Collections.
Related
I want to store my data in a collection that holds a maximum of ~10000 past records before they're deleted automatically from the database, like a FIFO queue of 10k records.
I have been looking at capped collections, but apart from the maximum number of records, they also require the maximum size of the collection. Now, I don't really know how much size 10k records are going to occupy. If I set a size that can't hold 10k records, I'll have lesser than I need. On the other hand, if I set an upper limit on the size, space is going to be wasted because mongo allocates the space beforehand.
What I can do is get dummy records, but I don't know how to check the size of each document.
Does anyone know of a method that can set an upper limit solely on the number of documents in my collection? Using the latest version on mongo out right now (v 3.6.3)
I'd personally push in 10000 actual real documents into a mongodb collection then call stats on the collection
db.test.stats()
Then use the size property:
> db.test.stats()
{
"ns" : "test.test",
"size" : 28609,
"count" : 1018,
"avgObjSize" : 28,
You may also want to add some padding, say 10% to that number:
cap col size = size * 1.1
I am using mongodb for one of my application.
We are fetching large amount of records from the db.
We are facing following issue when we fetch large number of documents from db.
aggregation result exceeds maximum document size
Any option to set this max limit?
The documentation states that:
If you do not specify the cursor option or store the results in a
collection, the aggregate command returns a single BSON document that
contains a field with the result set. As such, the command will
produce an error if the total size of the result set exceeds the BSON
Document Size limit.
Earlier versions of the aggregate command can only return a single
BSON document that contains the result set and will produce an error
if the if the total size of the result set exceeds the BSON Document
Size limit.
The maximum BSON document size is 16 megabytes.
That actually is the problem that you have now.
Is it possible to have a second Collection that is exactly the same as the first Collection, but with a limit and sort operator applied to it?
If the first Collection have 1000+ records and have new records being added, the second Collection will have all the new records, but limited to N newest records (sort by timestamp field).
The reason for doing this is to overcome a limitation in my database driver that does not have sort and limit implemented yet.
It sounds like what you want is a capped collection.
A capped collection is automatically limited to a fixed amount of records. When they are full and you add a new document, the oldest document is deleted. All records are guaranteed to be in insertion order when you query from the collection without an explicit sort. The biggest limitation is that documents in a capped collection can not be updated when that would increase their size.
Capped collections need to be created explicitely with the createCollection function. This shell command would create a capped collection limited to 1000 documents:
db.createCollection( "your_collection_name", { capped: true, size: 1000 } );
When you want to convert an already existing collection to a capped collection, you can use the convertToCapped database command:
db.runCommand({"convertToCapped": "your_existing_collection_name", size: 1000});
Which is the limit of a mongodb array field?
subdocuments have a limit of 4mb-16mb(it depends of the version). Does an array has the same limits?
The only limit is the document size. Whatever Document (or ReferenceDocument) contains the array cannot exceed that size. See limits.
I will execute a big query so i want to know what is the maximum length of a mongodb query ?
The maximum size of a document, which is what you are constructing when you create a query is 16MB. You can see that, and other limits here:
http://docs.mongodb.org/manual/reference/limits/