I want to keep track of the documents removed from capped collection. Is there any way that I can know the when an document has been removed from the capped collection ? I want to maintain a list of the documents removed from the capped collection. Please help me.
Related
I'm trying to understand what is the capped collection is, specifically in context of MongoDB, and what would be the difference in compare with queue?
Capped collection will remove oldest document when it reaches it limit, so that could be an issue if there is a need to process ALL documents from capped collection.
from mongo: Capped collections work in a way similar to circular
buffers: once a collection fills its allocated space, it makes room
for new documents by overwriting the oldest documents in the
collection.
comparing to queue:
queue will not remove records when full (it could throw an exception
like out of memory)
queue can remove record when dequeued - in capped collection you need to delete it on your own
capped collection cleanup: if capped collection size is 40 documents - then when 41st document is added -> the 1st entry is removed
I think this the most important things - any comments welcome!
CAPPED collection in mongodb is implementation of circular buffer.
From official documentation
Capped collections are fixed-size collections that support high-throughput operations that insert and retrieve documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.
Can we save new record in decending order in MongoDB? So that the first saved document will be returned last in a find query. I do not want to use $sort, so data should be presaved in decending order.
Is it possible?
According to above mentioned description ,as an alternative solution if you do not need to use $sort, you need to create a Capped collection which maintains order of insertion of documents into MongoDB collection
For more detailed description regarding Capped collections in MongoDB please refer the documentation mentioned in following URL
https://docs.mongodb.org/manual/core/capped-collections/
But please note that capped collections are fixed size collections hence it will automatically flush old documents in case when collection size exceeds size of capped collection
The order of the records is not guaranteed by MongoDB unless you add a $sort operator. Even if the records happen to be ordered on disk, there is no guarantee that MongoDB will always return the records in the same order. MongoDB does quite a bit of work under the hood and as your data grows in size, the query optimiser may pick a different execution plan and return the data in a different order.
When I try to remove a document in collection in MongoDB . It didn't remove because the collection is capped . My question is why? And is there a solution or other function can remove the document in this case?
No You cannot delete documents from a capped collection. And there is no possible workaround. The only thing you can do is drop() the collection.
You can not use remove function for capped collection. There are some restrictions on capped collections. Also you can refer this document for capped collection in mongoDB.
Which is the better way to remove all entries from a mongodb collection?
db.collection.remove({})
or
db.collection.drop()
Remove all documents in a collection
db.collection.remove({})
Will only remove all the data in the collection but leave any indexes intact. If new documents are added to the collection they will populate the existing indexes.
Drop collection and all attached indexes
db.collection.drop()
Will drop the collection and all indexes. If the collection is recreated then the indexes will also need to be re-created.
From your question, if you only want to remove all the entities from a collection then using db.collection.remove({}) would be better as that would keep the Collection intact with all indexes still in place.
From a speed perspective, the drop() command is faster.
db.collection.drop() will delete to whole collection (very fast) and all indexes on the collection.
db.collection.remove({}) will delete all entries but the collection and the indexes will still exists.
So there is a difference in both operations. The first one is faster but it will delete more meta information. If you want to ceep them, you should not use it.
I wonder if capped collections keep indexes for expired documents?
Removing documents from normal collection keeps indexes.
Capped collections remove documents by timer and do not allow db.collection.remove() at all.
I could not find any word in docs what happens with indexes for capped collections and would appreciate any help from ones who know.
TL;DR: The only way to remove documents from a capped collection is to drop the entire collection, that will also remove the indexes themselves from the collection.
I wonder if capped collections keep indexes for expired documents?
No. Documents that are no longer stored never remain in the index.
Removing documents from normal collection keeps indexes.
This is a bit misleading. Removing all documents from a normal collection by using db.collection.remove() removes both the documents from the collection and also deletes those documents from the index. It does not, however, remove the indexes of the collection, i.e. once you add new documents they are being added to the respective indexes again (i.e. removing the index itself is different from deleting documents from the index).
Capped collections remove documents by timer and do not allow db.collection.remove() at all.
The TTL-feature you linked has nothing to do with capped collections, in fact, the documentation says:
You cannot create a TTL index on a capped collection, because MongoDB cannot remove documents from a capped collection.
A collection with a TTL index does allow db.collection.remove.
A capped collection, on the other hand, has a fixed size (in terms of data size) and the oldest documents of the collection are automatically overwritten once the collection is full. This is not based on time, but purely on the size of the collection. Capped collections are always kept in insertion order (natural order).
Since the only way to remove documents from a capped collection is to drop the entire collection, that will also remove the indexes themselves from the collection.