We've been facing issues where quite a lot of documents are getting deleted from our mongodb instance. I've confirmed that it doesnt happen because of our code since we only have the find and replaceOne(with crupdate:true) API's used.
I ran the command db.serverStatus and under the opcounters section i can see that there are 1929 deletes. The mongodb.log doesn't give much information and i dont see anything related to deletion of documents.
Right now ours is a single node mongodb instance and so i think the opLog is not enabled, which would have probably helped in knowing when the deletes took place.
Is there any other way to debug this or is there anything specific in the mongodb.log i should be looking for?
Related
For the last week and a half for some reason my Mongodb collections all get dumped. I can't find a reason why this is happening there doesn't seem to be a real pattern to when/why the collections get dumped. Does anyone have any insight? I'm running Mongodb version 2.6.12.
you can read this artical https://snyk.io/blog/mongodb-hack-and-secure-defaults/
also you can have a look on MongoDB database deleted automatically it should solve your problem.
I cloned 2 of my collections from localhost to a remote location on MongoLab platform yesterday. I was trying to debug my (MEAN stack) application (with WebStorm IDE) and i realized one of those collections have no data in it. Well, there were 7800 documents this morning...
I am pretty much the only one who works on the database and especially with this collection. I didn't run any query to remove all of the documents from this collection. In mongolab's website there is a button says 'delete all documents from collection'. I am pretty sure I didn't hit that button. I asked my team mates; no one even opened that web page today.
Assuming that my team is telling the truth and I didn't remove everything and have a black out...
Is there a way to find out what happened?
And, is there a way to keep a query history (like unix command-line history) for mongo database that runs on a remote server? And if yes, how?
So, I am just curious about what happened. Also note that I don't have any DBA responsibilities or experience in that field.
MongoDB replica sets have a special collection called oplog. This collection stores all write operations for all databases in that replica set.
Here are instructions on how to access oplog in Mongolab:
Accessing the MongoDB oplog
Here is a query that will find all delete operations:
use local
db.oplog.rs.find({"op": "d", "ns" : "db_name.collection_name"})
To reduce my existing collection size, I have removed some unwanted fields from the documents from the collection. After and before I ran the collection stats to check the size, but it never changed.
I am missing some thing (like update anything) to reflect the reduced size in my stats, Please advice me.
I am running this in my local PC, not having any other nodes.
Thank you.
This is correct behavior.
The only time mongo releases disk space is when you drop a database or do a repair. (see here)
prettier explanation here
Basically, mongodb keeps any space it has allocated unless and until you drop a database or do a repair. It does this because adding space is not efficient, it is time-consuming. So once it has it, it keeps it and uses the blank space created until there is none left and then it gets more.
As mentioned by Daniel F in a comment on the question, this worked for me in the mongo shell:
use your-database-name
db.runCommand({ compact: "your-collection-name" })
I know this is an old question but I'll add that here just in case. Read carefully the documentation on compact as it causes the node it is run on to stop taking most of the traffic until it is done.
All previous ideas have their usecases I'll add another one that might be the best fit. If your able to query the subset that you want to leave from the collection and it is a lot smaller than the entire collection and you have tons of other collections there that you don't want to wait on restoring from scratch then you can write the sub query to another collection, apply indexes and when done plan a short downtime if necessary and switch the collections.
My program uses MongoDB and It needs to log all insert, update and delete operations and prints it to user. I look at the mongo oplog and it seems it provides the necessary functionality. As in advantage, it allows to see all changes made outside of my program. I see one problem that it is capped and deletes old entries, but I want to keep all history. Though I think it can be solved by setting a large oplog size and I hope the space will not be an issue.
Are there other problems with this approach?
One of MongoDB's solutions architects has written a blog post about this. Essentially you can query the oplog and copy the data to a new collection. I guess that means its not a terrible idea though I can't comment on whether it's a good idea.
Bear with me as I'm pretty new to MongoDB and mongoose has been my only interaction with it. So apologize ahead of time if I'm not properly separating the two technologies or understanding either.
I have a collection that is basically a log. Each log item is the result of a status check on a specific server that I have in my system. When I delete a server I'd like to also remove all the associated log entries. Whats the most efficient way to do this? I know I can find all the log items with a specific server id then iterate through each and remove it but that seems really inefficient. I've read through most of the mongo documentation and it looks like findAndModify is the best way to do this but I can't figure out syntax for this with mongoose. Any chance someone could help me out?
I'm using the latest version on mongoose at the moment, 1.7.2
Maybe I'm missing something here, but why not just do
db.logs.remove({serverid: deletedServerId})
?