My server went down from an electrical failure and for a few horrifying seconds, I thought I'd lost all MongoDB data. I then realized that when the server restarted, mongo automatically restarted without the --dbpath option.
What I can't figure out is why, even though my mongodb.conf has the dbpath set to /var/lib/mongodb, mongo looked for the db files in /data/db on startup. Can anyone shed some light on this?
Thanks!
/data/db is the default path that mongod will look for data files in if it is started with no options. Does your startup script instruct mongod to load the correct config file? If not, that would explain this behavior.
When was the last time you updated Mongod on your system? and how did you update it?
Depending how you installed / updated Mongod it could happen that either the /etc/init.d/mongo script or the /etc/mongo.conf files could be overwritten.
Or somebody edited those files a long time ago, and this is the first time Mongo got restarted, and now it looks like there's suddenly a change.
I haven't seen either of those two things happen in a long time (but if you're using Gentoo, you would probably see this happen)
What OS do you use?
It's a good idea to keep backups of those files...
You should definitely ensure that your init script for Mongo includes the full pathname to the data directory. Here is a snippet of what we use in our production deployment:
ulimit -s unlimited
MONGO_USER=mongo
MONGO_HOME=/opt/mongo
MONGO_DATA=/san2/data
MONGO_LOGS=/home/mongo/logs
start() {
su $MONGO_USER -c "$MONGO_HOME/bin/mongod --master --fork --logpath $MONGO_LOGS/mongodb.log --logappend --dbpath $MONGO_DATA --maxConns 2400"
}
Check the startup options in /etc/default/mongodb most likely those will be set to the default path of /data/db instead of the one you selected when you started it up manually.
Related
I am using Ubuntu 14.04 LTS and I have installed MongoDB 3.2.1. I had various problems with it that I fixed by either finding help from the internet (thanks Google) or by uninstalling and re-installing MongoDb.
One persistent problem that I cannot fix, unless I re-install, is by running mongod on the terminal. Currently my mongodb installation is working properly. With sudo service mongod start I start mongodb (I commented out start on xxxx line at /etc/init/mongod.conf so it doesn't auto start.) and with sudo service mongod stop I can stop it normally atm. And it correctly uses /var/lib/mongodb path for saving my collections.
If however after I stop mongodb with sudo service mongod stop I hit, on the terminal, mongod then mongodb breaks. I have gotten all kinds of errors like
Unusable mongod.lock. I have gotten around it, in a previous install, by doing sudo mongod but that was not a proper solution.
/data/db folder not found !!. Why look for it in the first place? The /etc/mongod.conf specifies the dbpath /var/lib/mongodb which is also the default when mongodb gets installed !!?? I have gotten around it as well with the mongod --dbpath /var/lib/mongodb option.
I think there was one more error that I don't remember but I also fixed/gotten_around it by finding solutions online.
I know that by re-installing and by never running mongod I can - for now? - not face those problems again (which looks a much better solution that the "workarounds" I did whenever those problems arose. I am wondering however what it is that is going wrong when I type mongod instead of sudo service mongod start ?!
In case somebody tries to replicate the problem know that I ve done only two modifications on my system after installing:
Commenting out the start on xxxx line at /etc/init/mongod.conf
Disabling transparent hugepages as described on the answer here.
I don't think anyone of those should interfere with my installation.
Can anyone help me understand what is going on? Aren't those commands supposed to do the same thing??
Thanks for your help.
Ok, I will try to answer.
First, unless you know what you are doing, you should not start mongod manually.
In general (a bit simplified), calling [sudo] service mongod start, you instruct the system to read the according file in /etc/init and start the executable according to the configuration described in said file.
When you started mongod by hand, however, you actually called the mongod binary, the server software itself – while the name is the same as the service, the two commands have few things in common. The binary does not use the /etc/mongod.conf by default, falling back to its default values for the various settings. Actually, you can see that the config file is explicitly defined in the init script. This is why mongod tried to find /data/db.
You can find said binary by issuing
which mongod
Regarding the lock file: When mongod is started by the system, user root actually assumes the effective user id of mongod (or mongodb I don't know for Ubuntu of the top of my head). When you tried to start it from your user id, you do not have the privileges to overwrite the lock file. When you used sudo mongod afterwards, you assumed the effective user id of root which on the other hand is allowed to overwrite said file. However, mongod will the run as root which is a security no-no.
An init script defines which environment to use, which user to run under and a lot of other stuff. Unless you really know what you are doing, you should not even fiddle with them, much less skip it.
And now, with the finger up
sudo is not the UNIX way of saying "I mean it!"
It has security implications, and you should be very aware of those implications before using it – aka read the man pages of every command you issue before you use it until you have at least a fact based idea of what the command is doing.
And again: unless you really know (as opposed to assume) what you are doing, do not fiddle with the system configuration.
I have MongoDB installed and working on my computer (Using Ubuntu 14.04 for the operating system).
The problem is, every time I go to start it, it takes me messing with it for a few minutes before I can get it to successfully start. Usually, some combination of running
mongod --dbpath ~/data/db
or
rm -rf ~/data/db/mongod.lock
or finding and killing the running but broken isntance with
ps wuax | grep mongo
kill <pid>
is what it takes to get it working again.
What would cause this? Every time I google this issue, it's all the same answers to just get it fixed, but not permanently make it start and stop correctly, just to fix it when it happens incorrectly.
i had the same issue of having to specifying --dbpath every time so i just created the two folders /data/db in my root directory and now had just to type mongod to start MongoDB. Note that this requires superuser privileges. Hope it helps
I try to shutdown mongoD in order to make a backup. However when I send service mongod stop it totally failed Stopping mongod:[FAILED]
I took a look on mongodb logs and there is nothing about shutdown...
I've got important data on this DB and i don't want to kill anything, so as i'm a beginner in mongoDB, I prefer to ask here what to do before destroy everything.
I was thinking about kill the process of MongoD, make my backup with mongodump and restart MongoD, but i'm not sure that it will work.
Thanks for helping.
For my MongoDB 3.0 install, the problem with stopping was caused by a comment field in the /etc/mongod.conf file:
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
This caused a problem with the awk command in /etc/init.d/mongod that defines the PIDFILEPATH since the comment text became part of the path, which then meant the mongo_killproc would not work.
I fixed this by editing the mongod.conf to move the comment to a separate line and now all is good (I did not have to change the quotes around $PIDFILEPATH either)
I had the same issue. In my case it worked after editing the mongod init script. You need to remove the quotes from $PIDFILE in the stop function definition. You might need to kill the mongo process first in order for it to work.
Hope this helps.
In my case it was "PIDFILEPATH" in the stop function and I removed the quotes
I think killing processes of mongodb would definitely work
sudo killall -15 mongod
Ever since I changed the dbpath in /etc/mongodb.conf, MongoDB has not been starting automatically, nor using the new dbpath. Prior to the change, MongoDB would be running when the computer started and I was able to simply run the command mongo to get into the console or start my Ruby on Rails server with no issues.
After I made the modification (in order to switch to a new drive with more space), the only way I can get everything to work is by manually running the command mongod --config /etc/mongodb.conf. If I don't run that, it doesn't seem like the service is running and running without the --config option give me the following error: ERROR: dbpath (/data/db/) does not exist. even though the config file says nothing about data/db.
Some other notes:
In addition to changing /etc/mongodb.conf, I moved all files out of /var/lib/mongodb and into /home/nick/appdev/mongodb.
I changed the owner and group from root to nick. Tried changing it back, but it didn't seem to fix anything.
I'm running Ubuntu 12.10 Beta 1 and Mongo 2.2.0 with Ruby on Rails 3.2.8
A late follow up on the above question...
I had a similar issue after moving the db to an ebs on ec2.
It turns out that just running mongod still directs the dbpath to /data/db/ (which exists).
The /etc/mongodb.conf is completely ignored unless specifically directed to.
I manage to work around this by using the directive --config or just the --dbpath(both work)
But was left wondering where does mongod takes it defaults from...?!
I was unable to locate and override these defaults anywhere.
Anyone ?
Note:
I am really annoyed by this behaviour of mongod...This is just bad design,and bad documentation.
It turns out that I needed to set the owner and group to mongodb. When I transferred the files to the new directory, I had set the owner and group to my user account nick and also tried root, neither of which worked.
To do so, here are the following commands:
sudo chown mongodb /home/nick/appdev/mongodb -R
sudo chgrp mongodb /home/nick/appdev/mongodb -R
To confirm that it worked, you can check the file permissions with:
ls -l /home/nick/appdev/mongodb
After checking all permission in the data, journal and log folders as suggested, my problem was solved by giving permission to a lock file in the /tmp folder
sudo chown mongod:mongod mongodb-27017.sock
I was running it as a AWS Amazon Linux instance. I figured that out by executing as the mongod user as below, and then, researching the error code. It might be useful for other troubleshooting.
sudo -S -u mongod mongod -f /etc/mongod.conf
MongoDB 1.6 is very old and the latest production version is 2.2, which contains a large amount of bug fixes and enhancements since 1.6.
Am I correct that you haven't installed 1.6 via a package manager such as yum or aptitude? I don't believe there are packages for 1.6 at present afaik. Therefore, mongod is behaving correctly as you have not started MongoDB with a control script.
Please see this link on configuration file options.
UPDATE: this was fixed after Meteor v0.4 (2012). For historical purposes:
Excerpt from du:
2890768 ./Code/Meteor/QuarterTo/.meteor/local/db/journal
2890772 ./Code/Meteor/QuarterTo/.meteor/local/db
2890776 ./Code/Meteor/QuarterTo/.meteor/local
2890788 ./Code/Meteor/QuarterTo/.meteor
2890804 ./Code/Meteor/QuarterTo
I merely ask because it was in my Dropbox and pushed me over my limit.
When meteor run is executed, it starts mongodb with default mongo settings, so it creates (massive) prealloc files in .meteor/local/db/journal.
There is no obvious way to disable this behavior. What I have done as a workaround is change the file app/lib/mongo_runner.js and add a --nojournal parameter that gets passed to mongodb at startup.
I created an issue for this: https://github.com/meteor/meteor/issues/15
Maybe you can use smallfiles=true parameter for mongoDB? It will create smallest prealloc files
You can turn off preallocation by passing the --noprealloc arg to mongod. The downside is that there will be pauses each time a new storage file needs to be allocated. Depending on the filesystem you are using (e.g., ext3 vs. ext4), this could result in noticeable latency for a user.
The commands that work for me are:
stop mongodb instance if it is running
sudo service mongod stop
create new mongodb instance without requiring 3+GB preallocated space and use smallfiles.
mongod --noprealloc --smallfiles
If you are getting “ERROR: dbpath (/data/db) does not exist.” when running 2,
then run these commands before 2.
sudo mkdir -p /data/db/
sudo chown `id -u` /data/db