MongoDB 3.2.9 fails to start as a service on CentOS - mongodb

I am trying to start mongod as a service on CentOS:
mongod --version
db version v3.2.9
git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
distmod: rhel70
distarch: x86_64
target_arch: x86_64
CentOS Linux release 7.2.1511 (Core)
/etc/selinux/config:
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
mongod.conf:
systemLog:
destination: file
path: "/mnt/log/mongod.log"
logAppend: true
storage:
dbPath: "/mnt/data"
engine: wiredTiger
journal:
enabled: true
processManagement:
fork: true
pidFilePath: "/var/run/mongodb/mongod.pid"
net:
# bindIp: 127.0.0.1
port: 27017
replication:
replSetName: XXXX
security:
keyFile: "/usr/mongodb/mongodb-keyfile"
The issue states:
Starting mongod (via systemctl): Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.
[FAILED]
journalctl -xe
...
Sep 20 13:10:55 ip-10-0-231-19.localdomain systemd[1]: Unit mongod.service entered failed state.
Sep 20 13:10:55 ip-10-0-231-19.localdomain systemd[1]: mongod.service failed.
Sep 20 13:10:55 ip-10-0-231-19.localdomain polkitd[9717]: Unregistered Authentication Agent for unix-process:26567:1630816 (system bus name :1.27, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Sep 20 13:11:04 ip-10-0-231-19.localdomain sudo[26587]: centos : TTY=pts/1 ; PWD=/home/centos ; USER=root ; COMMAND=/bin/vi /etc/mongod.conf
The server starts using:
sudo mongod -f /etc/mongod.conf
I know that there was a bug opened regarding similar issue but the fix might be available in 3.2.7 already and besides it was related to the scenario when SELINUX=enforcing...
Please support.

Recently encountered a similar problem, but on FC26; here, the problem could be traced back to SELINUX and specifically the permissions on the custom dbPath directory.
After trying many things, I found that the following sequence helped resolve my problem:
MONGODB_DATADIR=/mnt/data
sudo chown -R mongodb:mongodb $MONGODB_DATADIR
sudo chmod -R 700 $MONGODB_DATADIR
SELinux settings:
sudo semanage port -a -t mongod_port_t -p tcp 27017
sudo semanage fcontext -a -t mongod_var_lib_t ${MONGODB_DATADIR}
sudo semanage fcontext -a -t mongod_var_lib_t ${MONGODB_DATADIR}/mongod.lock
sudo restorecon -v ${MONGODB_DATADIR}
You probably also have to add mongod_log_t to the custom log file location, /mnt/log/mongod.log, I did not modify that one on my end.

Actually, for me the fix was:
sudo chown -R mongod:mongod $MONGODB_DATADIR

Related

MongoDB impossible to connect when start as a service

I did install mongodb(3.2.18) on my ubuntu server ( 16.04 ) named automationTaskV3 . It work when i'm working on my server, but when i try to connect from my computer it failed only when mongodb is start as a service.
For example if i start mongod this way :
sudo mongod
it will work i can access my dbs
But if i do :
sudo service mongod start
it will be impossible for me to connect from my computer.
Otherwise the mongod service status seems ok :
sudo service mongod status
● mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2017-11-30 17:34:46 UTC; 8min ago
Docs: https://docs.mongodb.org/manual
Main PID: 17712 (mongod)
CGroup: /system.slice/mongod.service
└─17712 /usr/bin/mongod --quiet --auth --config /etc/mongod.conf
Nov 30 17:34:46 automationTaskV3 systemd[1]: Started High-performance, schema-free document-oriented database.
Nov 30 17:38:09 automationTaskV3 systemd[1]: Started High-performance, schema-free document-oriented database.
Nov 30 17:39:29 automationTaskV3 systemd[1]: Started High-performance, schema-free document-oriented database.
For the moment i'm trying to access without credentials.
Here my mongod.service file :
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual
[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
and here my mongod.conf file
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
#processManagement:
Is it possible that's a user right problem ? It seems strange as i follow the steps from thoses tutorials :
https://www.linode.com/docs/databases/mongodb/install-mongodb-on-ubuntu-16-04
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
As I said in the comment that its the IP binding issue. So you can even try this solution.
You can access the 27017 port of your remote machine by using ssh tunnelling as it is most the easiest way and it's also a secure way.
ssh -L 27017:localhost:27017 username#remoteaddress.com
This command says forward 27017 port of your local machine to address localhost:27017 of your remote machine.
You will be prompted for the password (of remote machine) if you don't have ssh keys set up. Now you might get prompt for ssh of your remote machine don't close it unless you want to close port forwarding, open a new tab of your terminal or Robomongo you can now connect to localhost:27017. So it's like MongoDB is installed on your local machine but actually, it is serving from the remote machine using ssh.
To know more about ssh tunneling read here : https://www.ssh.com/ssh/tunneling

mongod stuck on fork with no error message

I was unable to run mongod as a background process. There were no errors and the program got stuck after printing the following message:
about to fork child process, waiting until server is ready for connections.
Here is the content of the specified mongodb.conf:
# See http://www.mongodb.org/display/DOCS/File+Based+Configuration for format details
# Run mongod --help to see a list of options
bind_ip = 127.0.0.1
port = 27027
quiet = true
fork = true
logpath = /home/mljli/proj/log/mongod.log
dbpath = /home/mljli/proj/db
logRotate = reopen
logappend = true
noscripting = true
noauth = true
Port 27027 was not used according to the output of netstat. dbpath exists and logpath is not a directory. I also tried the following simple configuration and it didn't work either:
$ mongod --fork --logpath mongod.log
about to fork child process, waiting until server is ready for connections.
Using sudo does not help.
My MongoDB version:
db version v3.4.3
git version: f07437fb5a6cca07c10bafa78365456eb1d6d5e1
OpenSSL version: OpenSSL 1.1.0e 16 Feb 2017
allocator: tcmalloc
modules: none
build environment:
distarch: x86_64
target_arch: x86_64

Mongodb 3.4 on 64bit Ubuntu 16.04 mongodb-27017.sock ERROR

There are a few posts related Mongodb 3.2, but when I attempted to install 3.4 the startup does not occur. Instead an unusual error appears in the log file (shown at the very end of the listing).
Is this perhaps because of an older version that was installed?
I did do an apt-get remove and purge.
Can anybody suggest why the error appears?
I followed the steps below, as outlined here,
$ sudo apt-get remove mongodb*
$ sudo apt-get purge mongodb*
$ sudo rm -rf /usr/lib/mongodb
$ sudo rm -rf /usr/log/mongodb
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
$ echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
$ sudo apt-get update
$ sudo apt-get install -y mongodb-org
..
..
E: There were unauthenticated packages and -y was used without --allow-unauthenticated
$ sudo apt-get install mongodb-org # removed -y to get it going
..
..
Setting up mongodb-org (3.4.2) ...
$ sudo chown -R mongodb:mongodb /var/log/mongodb
$ sudo chown -R mongodb:mongodb /var/lib/mongodb # modified ownership in view of what is stated on the mongodb installation note
$ # created the following file
$ cat /etc/systemd/system/mongod.service
[Unit]
Description=MongoDB Database Service 3.4
Wants=network.target
After=network.target
[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
StandardOutput=syslog
StandardError=syslog
[Install]
WantedBy=multi-user.target
# startup
$ sudo systemctl daemon-reload
$ sudo service mongod start
# nothing happens, I then viewed the log file and the following error appears:
$ cat /var/log/mongodb/mongod.log
2017-02-08T22:08:26.155+0000 I CONTROL [main] ***** SERVER RESTARTED *****
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] MongoDB starting : pid=11384 port=27017 dbpath=/var/lib/mongodb 64-bit host=12g
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] db version v3.4.2
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] allocator: tcmalloc
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] modules: none
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] build environment:
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] distmod: ubuntu1604
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] distarch: x86_64
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] target_arch: x86_64
2017-02-08T22:08:26.159+0000 I CONTROL [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log", quiet: true } }
2017-02-08T22:08:26.178+0000 E NETWORK [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
That means the mongod process was started by root user.So you can just remove the sock file or run this command to fix this issue :
sudo chown `whoami` /tmp/mongodb-*.sock
Make sure that the port you assign needs to be accessible and not blocked by your firewall and then just restart the mongod process.
The waiting for connections message in the console output indicates
that the mongod.exe process is running successfully.
You can check out Install MongoDB Community Edition on Ubuntu.
Hope this helps.
#mcgraddy, thanks, that was helpful indeed. Creating the directory and copying the service file was the answer.
$ sudo cp /etc/systemd/system/mongod.service /etc/systemd/system/mongodb.service
$ sudo mkdir -p /data/db
$ sudo chown -R $USER:$USER /data
I had to change ownership for /data.
Starting the service and viewing status shows everything is running as expected:
$ sudo service mongod start
$ sudo systemctl status mongodb
● mongodb.service - MongoDB Database Service
Loaded: loaded (/etc/systemd/system/mongodb.service; enabled; vendor preset:
Active: active (running) since Thu 2017-02-09 13:26:07 GMT; 9min ago
Main PID: 1184 (mongod)
CGroup: /system.slice/mongodb.service
└─1184 /usr/bin/mongod --quiet --config /etc/mongod.conf
Feb 09 13:26:07 systemd[1]: Started MongoDB Database Service.

Mongo Restart Error -- /var/run/mongodb/mongod.pid exists

I just started a new AWS Linux AMI 2016.09.1(HVM) and downloaded the latest Mongodb release. I am using MongoDB version 3.0.14. I updated /etc/mongod.conf to the following,
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
# network interfaces
net:
port: 27017
# bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on
all interfaces.
#security:
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
I run Mongodb for the first time using service,
sudo service mongod start
Everything works fine. When I stop Mongodb and restart,
sudo service mongod stop
sudo service mongod restart
I get the following error,
Error starting mongod. /var/run/mongodb/mongod.pid exists.
Here is what is in the logfile, /var/log/mongodb/mongod.log,
2017-01-16T14:11:16.869+0000 I CONTROL ***** SERVER RESTARTED *****
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] MongoDB starting :
pid=2820 port=27017 dbpath=/var/lib/mongo 64-bit hos
t=ip-10-0-0-91
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] db version v3.0.14
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] git version:
08352afcca24bfc145240a0fac9d28b978ab77f3
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] build info: Linux
ip-10-63-190-181 3.4.43-43.43.amzn1.x86_64 #1 SMP Mon
May 6 18:04:41 UTC 2013 x86_64 BOOST_LIB_VERSION=1_49
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] allocator: tcmalloc
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] options: { config:
"/etc/mongod.conf", net: { port: 27017 }, processMan
agement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" },
storage: { dbPath: "/var/lib/mongo", journal: { enabled: t
rue } }, systemLog: { destination: "file", logAppend: true, path:
"/var/log/mongodb/mongod.log" } }
2017-01-16T14:11:16.893+0000 E NETWORK [initandlisten] Failed to unlink
socket file /tmp/mongodb-27017.sock errno:1 Operation
not permitted
2017-01-16T14:11:16.893+0000 I - [initandlisten] Fatal Assertion
28578
2017-01-16T14:11:16.893+0000 I - [initandlisten]
***aborting after fassert() failure
I have seen several blogs posted about this problem with solutions but none of them seem to work. Spent all day Sunday trying to figure this one out. Never had this problem before running an earlier AWS instance and Mongo.
Ok, found the problem. I deleted the /var/run/mongodb/mongod.pid file as suggested by Dave Maples above. However, MongoDB would still not run. I looked at the log file again and now saw it barking about this file,
/tmp/mongodb-27017.sock
It looked like it did not have the correct permissions. I tried,
sudo chown mongod:mongod /tmp/mongodb-27017.sock
And it worked fine. Now when I start/stop/restart MongoDB multiple times I no longer have the problem. I think the file permissions caused MongoDB to not clean itself up properly and left /var/run/mongodb/mongod.pid file in the directory. This caused the "Error starting mongod. /var/run/mongodb/mongod.pid exists" error. However, this was not the real problem. The real problem was the /tmp/ permissions. Looking at the log file after I deleted the mongod.pid file and restarting Mongod showed the permissions problem. So here is what fixed the problem,
sudo rm /var/run/mongodb/mongod.pid
sudo chown mongod:mongod /tmp/mongodb-27017.sock
I bet if I just tried "sudo chown mongod:mongod /tmp/mongodb-27017.sock" right after installing MongoDB on the new server before running MongoDB would have avoided the problem. Wasted a perfectly good Florida sunny day on this one.
Ok, found a better solution to what I had in my first answer. My previous answer simply eliminated the symptoms. I saw that several other people had to chown various other directories. I believe the problem is that mongod does not restart properly and leaves a number of directories incorrectly set up as it fails. I suspect multiple threads are running so what directory that gets incorrectly set up is somewhat random. I am using Amazon's AWS AMI Linux release and installing MongoDB 3.4. I found a very good MongoDB install script at
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-amazon/
So here is the fix. I tested it several times and it worked each time. Install mongo as you normally would. Use the above script if helpful. Before you start mongod execute the following command,
sudo chkconfig mongod on
You will note that the above install script highlights chkconfig but at the bottom of the script where it might not get seen. This will ensure that you can successfully restart mongod each time. This should work on other linux versions.
you might change the ip in configuration file:
vim /etc/mongod.conf
# bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on
all interfaces.
bindIp: xxx.xxx.xxx.xxx
You have another option in case of the assignation of user and group dosen't work but this depends directly of the state of sock file.
I moved the mongodb-27017.sock and renamed it, remove the .pid file and start the service and with this the process works
sudo mv /tmp/mongodb-27017.sock /tmp/mongodb-27017.sock2
sudo rm /var/run/mongodb/mongod.pid
sudo service mongod start
Try Rebooting the server , connecting to the server and run the command again. Worked fine for me.
After stopping the server, you are trying to restart. You can only start the server. That's why you are getting the stopping error. And also no need to worry about this error.

mongod doesn't stop (docker debian:wheezy)

I have the following Dockerfile:
FROM debian:wheezy
RUN apt-get update
RUN apt-get install -y apt-utils
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
RUN echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.2 main" | tee /etc/apt/sources.list.d/mongodb-org-3.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
Then I build the image:
docker build -t my/image .
and try to start and stop the server:
$ docker run -it my/image
root#35ba8193f979:/# service mongod start
[ ok ] Starting database: mongod.
root#35ba8193f979:/# service mongod stop
[FAIL] Stopping database: mongod failed!
As you can see, mongod fails to stop. I can verify this by running ps and grepping for the mongod process; it's still there.
Here's the logs:
root#fce8d9638ce4:/# cat /var/log/mongodb/mongod.log
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] MongoDB starting : pid=19 port=27017 dbpath=/var/lib/mongodb 64-bit host=fce8d9638ce4
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] db version v3.2.9
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] allocator: tcmalloc
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] modules: none
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] build environment:
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] distmod: debian71
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] distarch: x86_64
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] target_arch: x86_64
2016-09-20T05:01:02.874+0000 I CONTROL [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-09-20T05:01:02.893+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-09-20T05:01:03.087+0000 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongodb/diagnostic.data'
2016-09-20T05:01:03.087+0000 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-09-20T05:01:03.156+0000 I NETWORK [initandlisten] waiting for connections on port 27017
I've googled for a solution already; most of the posts I find are very old, or don't describe the same problem as mine. Any help is greatly appreciated.
There appears to be an issue with service mongod stop since internally it calls a command which looks similar to:
start-stop-daemon --stop --pidfile /var/run/mongod.pid --user mongodb --exec /usr/bin/mongod
but the start-stop-daemon command somehow cannot find the executable /usr/bin/mongod, so it failed to stop the process. You can see the service script in /etc/init.d/mongod.
The solution that I found to be working is to call it without --exec directly from the command line:
start-stop-daemon --stop --pidfile /var/run/mongod.pid
and that seems to be able to stop the service properly.
I believe this issue is not limited to MongoDB, since I found multiple references about services not able to be stopped.