I'm using MongoDB with replication on azure and I have attached an SSD disk to MongoDB node and mounted on a specific path.
And I changed MongoDB data and logs path to specific path in MongoDB.conf(/opt/bitnami/mongodb/conf/mongodb.conf) file.
But when I restart MongoDB server using sudo service bitnami restart command it gives me error like
ERROR Unable to start com.bitnami.mongodb: Cannot find pid file
'/opt/bitnami/mong...b.pid'.
enter image description here
bitnami#mymongodb0:/tmp$ sudo service bitnami status
● bitnami.service - LSB: Bitnami Init Script
Loaded: loaded (/etc/init.d/bitnami)
Active: failed (Result: exit-code) since Fri 2017-08-04 05:49:58 UTC; 2min 15s ago
Process: 26654 ExecStop=/etc/init.d/bitnami stop (code=exited, status=0/SUCCESS)
Process: 92099 ExecStart=/etc/init.d/bitnami start (code=exited, status=1/FAILURE)
Aug 04 05:46:53 mymongodb0 bitnami[92099]: 2017-08-04T05:46:53.376Z - info: Saving configuration info to disk
Aug 04 05:46:53 mymongodb0 bitnami[92099]: 2017-08-04T05:46:53.987Z - info: Performing service start operation for mongodb
Aug 04 05:49:58 mymongodb0 bitnami[92099]: nami ERROR Unable to start com.bitnami.mongodb: Cannot find pid file '/opt/bitnami/mong...b.pid'.
Aug 04 05:49:58 mymongodb0 bitnami[92099]: 2017-08-04T05:49:58.249Z - error: Unable to perform start operation nami command exited wi... code 1
Aug 04 05:49:58 mymongodb0 systemd[1]: bitnami.service: control process exited, code=exited status=1
Aug 04 05:49:58 mymongodb0 systemd[1]: Failed to start LSB: Bitnami Init Script.
Aug 04 05:49:58 mymongodb0 systemd[1]: Unit bitnami.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.
root#mymongodb0:/# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 30G 1.8G 27G 7% /
udev 10M 0 10M 0% /dev
tmpfs 3.2G 8.4M 3.2G 1% /run
tmpfs 7.9G 0 7.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/sdd1 1007G 272M 956G 1% /data
/dev/sdb1 32G 48M 30G 1% /mnt/resource
/opt/bitnami/mongodb/conf/mongodb.conf
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /data/db
journal:
enabled: true
#engine:
#mmapv1:
#smallFiles: true
#wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /data/logs/mongodb.log
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0
unixDomainSocket:
enabled: true
pathPrefix: /opt/bitnami/mongodb/tmp
# replica set options
replication:
replSetName: replicaset
# process management options
processManagement:
fork: false
pidFilePath: /opt/bitnami/mongodb/tmp/mongodb.pid
/dev/sdc1 50G 33M 50G 1% /bitnami
tmpfs 1.6G 0 1.6G 0% /run/user/1000
After a long talk, the root reason is /data not give enough permissions.
chown mongo:mongo -R /data
/data directory needs mongo user and group.
Follow-up for an older image: bitnami/mongodb:3.2.7-r5
I mounted a directory /Users/sb/mongodata on my host to :/bitnami/mongodb and had to manually create this directory structure:
- conf
mongodb.conf
- data
- db
- logs
- tmp
Then chmod -R g+rwX /Users/sb/mongodata
Related
I am using ubuntu16.04 on VPS.
It has been migrated to another company's VPS and is in use.
nginix works fine and my node application works fine.
When the mongodb service is run as systemlctl stat
The following error code was shown and it became Active:Failed.
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2022-03-18 08:57:04 UTC; 3s ago
Docs: https://docs.mongodb.org/manual
Process: 3558 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=217/USER)
Main PID: 3558 (code=exited, status=217/USER)
Mar 18 08:57:04 user systemd[1]: Started MongoDB Database Server.
Mar 18 08:57:04 user systemd[1]: mongod.service: Main process exited, code=exited, status=217/USER
Mar 18 08:57:04 user systemd[1]: mongod.service: Unit entered failed state.
Mar 18 08:57:04 user systemd[1]: mongod.service: Failed with result 'exit-code'.
Below is my mongod.conf setting.
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
# how the process runs
processManagement:
timeZoneInfo: /usr/share/zoneinfo
#security:
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options:
#auditLog:
#snmp:
In my system
mongod.conf Path
/etc/mongod.conf
mongodb DB path
/var/lib/mongodb
I'd appreciate it if you could let me know what I need to check.
The problem is probably with the account mongo is trying to launch with in your systemd mongod.service file. Check the user and group defined in the [Service] block are valid.
I tried to install mongoDB on Centos8, but when I run the command
systemctl status mongod.service
I get this error:
● mongod.service - MongoDB Database Server
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Sat 2020-08-01 14:26:53 CEST; 11min ago
Docs: https://docs.mongodb.org/manual
Process: 1875 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=14)
Process: 1873 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 1871 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 1869 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)
Aug 01 14:26:53 db.localhost systemd[1]: Starting MongoDB Database Server...
Aug 01 14:26:53 db.localhost mongod[1875]: about to fork child process, waiting until server is ready for>
Aug 01 14:26:53 db.localhost mongod[1875]: forked process: 1877
Aug 01 14:26:53 db.localhost mongod[1875]: ERROR: child process failed, exited with 14
Aug 01 14:26:53 db.localhost mongod[1875]: To see additional information in this output, start without th>
Aug 01 14:26:53 db.localhost systemd[1]: mongod.service: Control process exited, code=exited status=14
Aug 01 14:26:53 db.localhost systemd[1]: mongod.service: Failed with result 'exit-code'.
Aug 01 14:26:53 db.localhost systemd[1]: Failed to start MongoDB Database Server.
I tried to check privileges for the folders: /var/lib/mongo and /var/log/mongodb
#/var/lib/
drwxr-xr-x. 4 mongod mongod 4096 Aug 1 14:44 mongo
#/var/log/
drwxr-xr-x. 2 mongod mongod 50 Aug 1 14:14 mongodb
In some other posts, people told to try this command:
sudo chown -R mongodb:mongodb /var/lib/mongodb/
but in don't have the user mongodb and I get the error: invalid user: 'mongodb:mongodb'!
In my /etc/passwd file the only user for mongo is this:
mongod:x:994:992:mongod:/var/lib/mongo:/bin/false
What's the problem?
Why I cannot run mongod.service?
Thanks for your support.
Added more informations:
systemctl cat mongod
[root#db tmp]# systemctl cat mongod
# /usr/lib/systemd/system/mongod.service
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongod
Group=mongod
Environment="OPTIONS=-f /etc/mongod.conf"
EnvironmentFile=-/etc/sysconfig/mongod
ExecStart=/usr/bin/mongod $OPTIONS
ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb
ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb
ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb
PermissionsStartOnly=true
PIDFile=/var/run/mongodb/mongod.pid
Type=forking
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for mongod as specified in
# https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings
[Install]
WantedBy=multi-user.target
mongod.conf
[root#db tmp]# cat /etc/mongod.conf
# ..
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1 # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.
#security:
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
/usr/bin/mongod -f /etc/mongod.conf
[root#db mongodb]# /usr/bin/mongod -f /etc/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 1959
child process started successfully, parent exiting
For some kind of reason, mongod want the folder /data/db and ignore the file mongod.conf
After I created these folders, if I run mongod command (with sudo), programme start "correctly".
But if I reboot the system, the service fails on boot and continue to have problems.
My Problem
Hi there! When I tried to start and see status I've got an error.
skozurak#danny:~$ service mongod start
skozurak#danny:~$ service mongod status
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2020-07-14 17:54:33 CEST; 3s ago
Docs: https://docs.mongodb.org/manual
Process: 77503 ExecStart=/usr/bin/mongod --auth --config /etc/mongod.conf (code=exited, status=2)
Main PID: 77503 (code=exited, status=2)
Jul 14 17:54:33 danny systemd[1]: Started MongoDB Database Server.
Jul 14 17:54:33 danny mongod[77503]: Error parsing YAML config file: yaml-cpp: error at line 32, col>
Jul 14 17:54:33 danny mongod[77503]: try '/usr/bin/mongod --help' for more information
Jul 14 17:54:33 danny systemd[1]: mongod.service: Main process exited, code=exited, status=2/INVALID>
Jul 14 17:54:33 danny systemd[1]: mongod.service: Failed with result 'exit-code'.
lines 1-12/12 (END)
What I did
Tried to reinstall
Tried to install different versions
Then I added the service to be able to run on server startup.
skozurak#danny:~$ systemctl enable mongod.service
I think that problem is in the config file. But I do not know how to fix it there.
skozurak#danny:~$ sudo nano /etc/mongod.conf
# mongod.conf
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
# how the process runs
processManagement:
timeZoneInfo: /usr/share/zoneinfo
security:
authorization: enabled
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options:
#auditLog:
#snmp:
I just changed one line of config as I saw here
skozurak#danny:~$ sudo nano /lib/systemd/system/mongod.service
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target
[Service]
User=mongodb
Group=mongodb
EnvironmentFile=-/etc/default/mongod
ExecStart=/usr/bin/mongod --auth --config /etc/mongod.conf #i changed this line
PIDFile=/var/run/mongodb/mongod.pid
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false
# Recommended limits for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings
[Install]
WantedBy=multi-user.target
Your error lies here:
security:
authorization: enabled
You can't use any other indentation than exactly 2 spaces.
I try to start service with mongo and get an error:
* mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2019-04-12 12:55:29 MSK; 9s ago
Docs: https://docs.mongodb.org/manual
Process: 15162 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=exited, status=1/FAILURE)
Main PID: 15162 (code=exited, status=1/FAILURE)
Apr 12 12:55:29 mx systemd[1]: Started MongoDB Database Server.
Apr 12 12:55:29 mx systemd[1]: mongod.service: Main process exited, code=exited, status=1/FAILURE
Apr 12 12:55:29 mx systemd[1]: mongod.service: Failed with result 'exit-code'.
If I start mongod by manual, all works good.
mongod.service:
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
[Service]
User=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
[Install]
WantedBy=multi-user.target
mongod.config:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
# how the process runs
processManagement:
timeZoneInfo: /usr/share/zoneinfo
I've set permissions for /var/lib/mongodb and /var/log/mongodb/mongod.log
chown mongodb:mongodb /var/lib/mongodb -R
chown mongodb:mongodb /var/log/mongodb/mongod.log -R
What's wrong? What should I do? Any ideas?
I've solved my porblem. The solution is very simple, but I had a lot of headace.
Free disk space. I have no anought free disk space in my server. Thats it!
I hope this will be helpfull for anybody...
My MongoDB gets stuck and returning the following error:
2019-01-28T18:28:53.419+0000 E STORAGE [WTCheckpointThread] WiredTiger error (28) [1548700133:419188][1:0x7feecb0ae700], file:WiredTiger.wt, WT_SESSION.checkpoint: /data/db/WiredTiger.turtle.set: handle-open: open: No space left on device
2019-01-28T18:28:53.419+0000 E STORAGE [WTCheckpointThread] WiredTiger error (22) [1548700133:419251][1:0x7feecb0ae700], file:WiredTiger.wt, WT_SESSION.checkpoint: WiredTiger.wt: the checkpoint failed, the system must restart: Invalid argument
2019-01-28T18:28:53.419+0000 E STORAGE [WTCheckpointThread] WiredTiger error (-31804) [1548700133:419260][1:0x7feecb0ae700], file:WiredTiger.wt, WT_SESSION.checkpoint: the process must exit and restart: WT_PANIC: WiredTiger library panic
2019-01-28T18:28:53.419+0000 F - [WTCheckpointThread] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 361
2019-01-28T18:28:53.419+0000 F - [WTCheckpointThread]
***aborting after fassert() failure
2019-01-28T18:28:53.444+0000 F - [WTCheckpointThread] Got signal: 6 (Aborted).
However, my disk has space:
df -h
Filesystem Size Used Avail Use% Mounted on
udev 992M 0 992M 0% /dev
tmpfs 200M 5.7M 195M 3% /run
/dev/xvda1 39G 26G 14G 66% /
tmpfs 1000M 1.1M 999M 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1000M 0 1000M 0% /sys/fs/cgroup
tmpfs 200M 0 200M 0% /run/user/1000
df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 253844 322 253522 1% /dev
tmpfs 255835 485 255350 1% /run
/dev/xvda1 5120000 5090759 29241 100% /
tmpfs 255835 10 255825 1% /dev/shm
tmpfs 255835 3 255832 1% /run/lock
tmpfs 255835 16 255819 1% /sys/fs/cgroup
tmpfs 255835 4 255831 1% /run/user/1000
And this would be my docker-compose:
version: "3"
services:
# MariaDB
mariadb:
container_name: mariadb
image: mariadb
ports: ['3306:3306']
restart: always
volumes:
- /home/ubuntu/mysql:/var/lib/mysql
environment:
- "MYSQL_ROOT_PASSWORD=PasswordGoesHere"
command:
# - --memory=1536M
- --wait_timeout=28800
- --innodb_buffer_pool_size=1g
- --innodb_buffer_pool_instances=4
# - --innodb_buffer_pool_chunk_size=1073741824
# APACHE
apache:
container_name: apache
image: apache-php7.1
ports: ['80:80', '443:443']
restart: always
entrypoint: tail -f /dev/null
volumes:
- /home/ubuntu/apache2/apache-config:/etc/apache2/sites-available/
- /home/ubuntu/apache2/www:/var/www/html/
# MONGODB
mongodb:
container_name: mongodb
image: mongo
ports: ['27017:27017']
restart: always
command:
- --auth
volumes:
- /home/ubuntu/moongodb:/data/db
Would it be a problem with my docker-compose.yml? Because I'm using the physical disk and not virtual. I can run the applications and after 1-2 hours the mongo will fail again.
Clean docker cache - volumes & containers
for me it works:
docker system prune
and then
docker volume prune
or in one line:
docker system prune --volumes
To see all volumes : docker volume ls
To show docker disk usage: docker system df
If you are running this in centos/RHEL/Amazon Linux you should know that the devicemapper has major issues with releasing inodes in Docker.
Even if you prune the entire docker system, it will still hang on to a lot of inodes, the only way to really solve this is to basically implode docker:
service docker stop
rm -rf /var/lib/docker
service docker start
This should release all your inodes.
I've spent a lot of time on this, Docker really only fully supports Ubuntu overlay2, and the devicemapper, although works, is technically not supported.
It looks like 100% of your inodes are in use (from the df -i output). Try looking for dangling volumes and cleaning them up. Also, it would be a good idea to make sure the docker daemon is using a production-grade storage driver (about storage drivers, choosing a storage driver).