Can't start mongodb through systemctl, error logs won't show up - mongodb

I cannot boot MongoDB via systemctl.
When I try, I get an error, but nothing comes through the log.
When I try and run it manually via $ /usr/bin/mongod --config /etc/mongod.conf no error message comes through the log.
When I turn off logging in mongod.conf, an error message comes out that I don't have rights to the directory with the data (which is to be expected).
However, I can boot the process with sudo such as:
$ sudo /usr/bin/mongod --config /etc/mongod.conf, the process runs but it is tied to my terminal so if my session closes so does MongoDB. Hence why I am trying to run it with systemctl.
Then when I try and run with sudo in my service such as:
ExecStart=/usr/bin/sudo /usr/bin/mongod --config /etc/mongod.conf
Then it doesn't start and no error log comes through.
My conf file:
# mongod.conf
# for documentation of all options, see:
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
# how the process runs
processManagement:
timeZoneInfo: /usr/share/zoneinfo
My service:
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
Restart=always
User=mongodb
Group=mongodb
EnvironmentFile=-/etc/default/mongod
ExecStart=/usr/bin/sudo /usr/bin/mongod --config /etc/mongod.conf
PIDFile=/var/run/mongodb/mongod.pid
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for mongod as specified in
# https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings
So it becomes an impossible question. When I try and boot it through systemctl no error log comes through /var/log/mongodb/mongod.log. If I run it manually I get an error saying that I don't have access to the data folder since the data folder is owned by the mongodb user, which is expected.
What can I do to see what is causing my service above to fail? I don't want to use sudo in my ExecStart command but whether I do or I don't I get the same problem: the service fails to start and no logs come through my Mongo log file.

Check of your service is failed
systemctl is-failed mongodb.service
Get failure logs
Journalctl is a utility for querying and displaying logs from journald, systemd's logging service.
get latest 20 lines of logs which will you failure details
journalctl | grep mongod | tail -n20
Use journalctl -u service and tail to get top 20 logs
journalctl -u mongodb.service | tail -n20
Get list of all failed services
systemctl list-units --failed
Read - https://www.cyberciti.biz/faq/systemd-systemctl-list-all-failed-units-services-on-linux/

Related

mongod service failed to restarted, 27017 port not in use - even getting address already in use error

I am not able to start mongo db service, if I try to start mongod using systemctl restart mongod getting error
Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.
Logs: vi /var/log/mongo/mongod.log
In log file, if I see below message, it looks like port is already in use
exception in initAndListen std::exception: listen: Address already in use, terminating
netstat -apt| grep "27017"
No process id.
Below is mongod status:
I am not getting any solution, your help will be greatly appreciated. Thanks.
Trying starting mongodb on different port and with different data directory, that might be helpful, use below command to start with different port and different data directory
mongod --port 27027 --dbpath </path/to/your/data/directory>
If this work then some process might be using 27017 or or you might have mongod running on background, so first check the process id and manually kill it as below
ps -Aef | grep 27017
or
ps -Aef | grep mongod
Find the process id and kill it
sudo kill -9 <processId>
Then try to restart it as you are doing:
systemctl restart mongod
or:
/etc/init.d/mongodb restart
Hope this help you out.
One issue was the lock file used by MongoDB might have an incorrect ownership:
-rw-r--r-- 1 root root 0 Mar 16 02:27 /var/lib/mongo/mongod.lock
Suggest you to simply removed it which will allow MongoDB to recreate it properly. This file is only needed by MongoDB if it's currently running, and ideally the file should only exist if it's running too, since it only contains a copy of the current process ID which MongoDB is running under.
The second issue will be the configuration file itself: /etc/mongod.conf.
MongoDB expects the file to be in YAML format where apparently blank spaces matter. I have share the sample configuration file which worked in our server.
Basically, each indentation should be made using two spaces for any directives underneath their parent directive.
Additinal notes:
For example, the "net:" heading is not indented because it's the top of its configuration tier, as where "port:" and "bindIp:" are each indented with 2 spaces since "net:" is their parent directive.
The "ssl:" directive is also indented with two spaces since its parent is "net:", but since "mode:", "CAFile:" and "PEMKeyFile:" along belong to "ssl:" they go underneath it and again each take 2 additional spaces for further indentation for a total of 4 spaces.
Finally the following mongo restart will work:
systemctl restart mongod
or
systemctl restart mongod.service
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0 # Listen to local interface only, comment to listen on all interfaces.
ssl:
mode: preferSSL
CAFile: /etc/ssl/mongodb/RootCA/root-ca.pem
PEMKeyFile: /etc/ssl/mongodb/host.omkieitsolutions.com.pem
security:
authorization: enabled
clusterAuthMode: keyFile
#operationProfiling:
#replication:
"/etc/mongod.conf" 50L, 990C

MongoDB impossible to connect when start as a service

I did install mongodb(3.2.18) on my ubuntu server ( 16.04 ) named automationTaskV3 . It work when i'm working on my server, but when i try to connect from my computer it failed only when mongodb is start as a service.
For example if i start mongod this way :
sudo mongod
it will work i can access my dbs
But if i do :
sudo service mongod start
it will be impossible for me to connect from my computer.
Otherwise the mongod service status seems ok :
sudo service mongod status
● mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2017-11-30 17:34:46 UTC; 8min ago
Docs: https://docs.mongodb.org/manual
Main PID: 17712 (mongod)
CGroup: /system.slice/mongod.service
└─17712 /usr/bin/mongod --quiet --auth --config /etc/mongod.conf
Nov 30 17:34:46 automationTaskV3 systemd[1]: Started High-performance, schema-free document-oriented database.
Nov 30 17:38:09 automationTaskV3 systemd[1]: Started High-performance, schema-free document-oriented database.
Nov 30 17:39:29 automationTaskV3 systemd[1]: Started High-performance, schema-free document-oriented database.
For the moment i'm trying to access without credentials.
Here my mongod.service file :
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual
[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
and here my mongod.conf file
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
#processManagement:
Is it possible that's a user right problem ? It seems strange as i follow the steps from thoses tutorials :
https://www.linode.com/docs/databases/mongodb/install-mongodb-on-ubuntu-16-04
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
As I said in the comment that its the IP binding issue. So you can even try this solution.
You can access the 27017 port of your remote machine by using ssh tunnelling as it is most the easiest way and it's also a secure way.
ssh -L 27017:localhost:27017 username#remoteaddress.com
This command says forward 27017 port of your local machine to address localhost:27017 of your remote machine.
You will be prompted for the password (of remote machine) if you don't have ssh keys set up. Now you might get prompt for ssh of your remote machine don't close it unless you want to close port forwarding, open a new tab of your terminal or Robomongo you can now connect to localhost:27017. So it's like MongoDB is installed on your local machine but actually, it is serving from the remote machine using ssh.
To know more about ssh tunneling read here : https://www.ssh.com/ssh/tunneling

installing and running mongodb in codeanywhere

i used this code:
$ mkdir data
$ echo 'mongod --bind_ip=$0.0.0.0 --dbpath=data --nojournal "$#"' > mongod
$ chmod a+x mongod
i have made a file called mongodb that contains this:
​ mongod --bind_ip=$0.0.0.0 --dbpath=data --nojournal "$#"
but when i run
./mongod
i get this and it doesn't work:
https://i.stack.imgur.com/L9224.png
Assuming you are using one of the recent versions of Mongo.
Try this:
Step1- Make a directory, say database, its path being /home/cabox/database
Step2- In the same level as the directory database, create a file named mongod.log, by using touch /home/cabox/mongod.log
Step3- Create a configuration file for our database, mongod.conf using
touch /home/cabox/mongod.conf
Step4- Now open this file using vi and put following lines in it
systemLog:
destination: file
path: "/home/cabox/mongod.log"
logAppend: true
processManagement:
fork: true
storage:
dbPath: "/home/cabox/database"
journal:
enabled: true
engine: "mmapv1"
mmapv1:
smallFiles: true
net:
bindIp: 0.0.0.0
Step5- Save this file and start mongod by running:
mongod --config /home/cabox/mongod.conf
Step6- Start the shell: mongo
Hopefully everything should be working fine by now. The problem with your method was that mongod was using wired Tiger storage engine by default, which causes space related problems in containers like codeanywhere which have a small disk size.
P.S - In the conf file, make sure you indent using spaces and not tabs.
You can shutdown the server by using:
mongo --eval "db.getSiblingDB('admin').shutdownServer()"
To check whether it has been successfully started or shutdown:
netstat -ntulp|grep 27017

Mongo Restart Error -- /var/run/mongodb/mongod.pid exists

I just started a new AWS Linux AMI 2016.09.1(HVM) and downloaded the latest Mongodb release. I am using MongoDB version 3.0.14. I updated /etc/mongod.conf to the following,
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
# network interfaces
net:
port: 27017
# bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on
all interfaces.
#security:
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
I run Mongodb for the first time using service,
sudo service mongod start
Everything works fine. When I stop Mongodb and restart,
sudo service mongod stop
sudo service mongod restart
I get the following error,
Error starting mongod. /var/run/mongodb/mongod.pid exists.
Here is what is in the logfile, /var/log/mongodb/mongod.log,
2017-01-16T14:11:16.869+0000 I CONTROL ***** SERVER RESTARTED *****
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] MongoDB starting :
pid=2820 port=27017 dbpath=/var/lib/mongo 64-bit hos
t=ip-10-0-0-91
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] db version v3.0.14
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] git version:
08352afcca24bfc145240a0fac9d28b978ab77f3
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] build info: Linux
ip-10-63-190-181 3.4.43-43.43.amzn1.x86_64 #1 SMP Mon
May 6 18:04:41 UTC 2013 x86_64 BOOST_LIB_VERSION=1_49
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] allocator: tcmalloc
2017-01-16T14:11:16.872+0000 I CONTROL [initandlisten] options: { config:
"/etc/mongod.conf", net: { port: 27017 }, processMan
agement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" },
storage: { dbPath: "/var/lib/mongo", journal: { enabled: t
rue } }, systemLog: { destination: "file", logAppend: true, path:
"/var/log/mongodb/mongod.log" } }
2017-01-16T14:11:16.893+0000 E NETWORK [initandlisten] Failed to unlink
socket file /tmp/mongodb-27017.sock errno:1 Operation
not permitted
2017-01-16T14:11:16.893+0000 I - [initandlisten] Fatal Assertion
28578
2017-01-16T14:11:16.893+0000 I - [initandlisten]
***aborting after fassert() failure
I have seen several blogs posted about this problem with solutions but none of them seem to work. Spent all day Sunday trying to figure this one out. Never had this problem before running an earlier AWS instance and Mongo.
Ok, found the problem. I deleted the /var/run/mongodb/mongod.pid file as suggested by Dave Maples above. However, MongoDB would still not run. I looked at the log file again and now saw it barking about this file,
/tmp/mongodb-27017.sock
It looked like it did not have the correct permissions. I tried,
sudo chown mongod:mongod /tmp/mongodb-27017.sock
And it worked fine. Now when I start/stop/restart MongoDB multiple times I no longer have the problem. I think the file permissions caused MongoDB to not clean itself up properly and left /var/run/mongodb/mongod.pid file in the directory. This caused the "Error starting mongod. /var/run/mongodb/mongod.pid exists" error. However, this was not the real problem. The real problem was the /tmp/ permissions. Looking at the log file after I deleted the mongod.pid file and restarting Mongod showed the permissions problem. So here is what fixed the problem,
sudo rm /var/run/mongodb/mongod.pid
sudo chown mongod:mongod /tmp/mongodb-27017.sock
I bet if I just tried "sudo chown mongod:mongod /tmp/mongodb-27017.sock" right after installing MongoDB on the new server before running MongoDB would have avoided the problem. Wasted a perfectly good Florida sunny day on this one.
Ok, found a better solution to what I had in my first answer. My previous answer simply eliminated the symptoms. I saw that several other people had to chown various other directories. I believe the problem is that mongod does not restart properly and leaves a number of directories incorrectly set up as it fails. I suspect multiple threads are running so what directory that gets incorrectly set up is somewhat random. I am using Amazon's AWS AMI Linux release and installing MongoDB 3.4. I found a very good MongoDB install script at
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-amazon/
So here is the fix. I tested it several times and it worked each time. Install mongo as you normally would. Use the above script if helpful. Before you start mongod execute the following command,
sudo chkconfig mongod on
You will note that the above install script highlights chkconfig but at the bottom of the script where it might not get seen. This will ensure that you can successfully restart mongod each time. This should work on other linux versions.
you might change the ip in configuration file:
vim /etc/mongod.conf
# bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on
all interfaces.
bindIp: xxx.xxx.xxx.xxx
You have another option in case of the assignation of user and group dosen't work but this depends directly of the state of sock file.
I moved the mongodb-27017.sock and renamed it, remove the .pid file and start the service and with this the process works
sudo mv /tmp/mongodb-27017.sock /tmp/mongodb-27017.sock2
sudo rm /var/run/mongodb/mongod.pid
sudo service mongod start
Try Rebooting the server , connecting to the server and run the command again. Worked fine for me.
After stopping the server, you are trying to restart. You can only start the server. That's why you are getting the stopping error. And also no need to worry about this error.

unable to launch config server for mongo shard

I am trying to get the config server running for mongo shard using the following command and config file:
mongod --config /etc/mongod2.conf
mongod2.config
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb2
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
processManagement:
pidFilePath: /srv/mongodb/db2.pid
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27019
bindIp: 127.0.0.1
processManagement:
fork: true
#security:
# authorization: enabled
replication:
replSetName: configReplSet
sharding:
clusterRole: configsvr
When I run the command above, I get the following output (with no log output in the mongo logs):
about to fork child process, waiting until server is ready for connections.
forked process: 14246
ERROR: child process failed, exited with error number 1
I am running mongo v3.2.2 on ubuntu v14. The port 27019 is not bound by another process.
The issue was caused since I never created a separate data directory. I saw better error messages when I ran the following:
mongod --dbpath /var/lib/mongodb2 --port 27019
This command gave better error messages (it showed that the data directory was not present etc.). This helped solve the issue and I was able to run the config server (output below):
2016-06-29T12:21:30.216-0400 I CONTROL [initandlisten] MongoDB starting : pid=16060 port=27019 dbpath=/var/lib/mongodb2 64-bit host=abcdefghi
2016-06-29T12:21:30.216-0400 I CONTROL [initandlisten] db version v3.2.0