When I try to do a mongodump :
mongodump -u aaa -p abc123 --authenticationDatabase admin -d TestDb --gzip --out /var/backups/dump-25-05-22/mybackup.gz
inside my mongodb pod (kubectl exec -t <pod_name> --bash> I am getting an error :
Failed: error dumping metadata: error creating directory for metadata file /var/backups/...: permission denied
For context I do not have access to the host machine where the k8s instance is running, I can only do kubectl commands so the issue can only be fixed from K8S side.
These are the current permissions on the mount path (/var/backups) :
ls -la
drwxr-xr-x 2 root root 4096 Jun 11 2021 .
drwxr-xr-x 1 root root 4096 Jun 11 2021 ..
I have also tried the following :
1.
sudo chmod 777 -R /var/backups/
bash: sudo: command not found
chmod 777 -R /var/backups/
error changing permissions of '/var/backups/': Operation not permitted
Tried to enable sudo mode (Requires password and I dont know it)
su -
Having done a lot of digging around this the best solution I have is to tweak the mongo Dockerfile and build my own custom image.
The only feasible solution I found is at this link and it says "Configure mongo to run as root via editing the mongo config and copying it during the docker build process. This assumes you're using a docker file to build that image. Then it will have no problem accessing the attached volume."
My question(s):
Given an example Dockerfile as below how would I make the so-called mongod user run as root ? :
FROM mongo:3.2
COPY entrypoint.sh /root/entrypoint.sh
RUN chown -R mongodb:mongodb /var/log /data/db
USER mongodb
ENTRYPOINT ["/root/entrypoint.sh"]
At which level is this mongodb user accessible ? I have checked the users in the mongodb container but they is no entry for such (Unless Im missing something obvious) :
cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats
I'm using the postgres docker image, and after months of using databases running in docker images, now I'm getting the behaviour where after a certain period of time, they simply just hang. I can exec with bin/bash but can't do anything with postgres at all; the commands don't return and the containers can't be brought down. Even docker kill -s SIGKILL <container_id> doesn't work; needs a reboot of the docker server to stop them.
The only smoking gun I can see is the message:
WARNING: could not open statistics file "pg_stat_tmp/global.stat": Operation not permitted
on all containers. Anyone has any ideas I'd be really appreciative as this is killing things at the moment.
This is happening due to a user permission mismatch in the docker container.
Listing the relevant files in the container:
$ docker exec <container> ls -l /var/lib/postgresql/data/pg_stat_tmp
-rw------- 1 root root [...] db_0.stat
-rw------- 1 root root [...] db_1.stat
-rw------- 1 root root [...] db_2.stat
-rw------- 1 postgres postgres [...] global.stat
we can see that all the db_*.stat files are owned by root:root, while global.stat is owned by postgres:postgres.
Checking the docker user gives us:
$ docker exec <container> whoami
root
So, we'd like all of these files to be owned by the postgres user.
Luckily, this is quite easy! Just set user to postgres, and restart!
In a dockerfile:
USER postgres
Using docker-compose:
services:
postgres:
image: postgres:13
user: postgres
As Norling and Dharman's answer did not work for me, I've tried another way: leaving the temporary file in container - and that worked. I've changed with inline command the Postgresql config:
postgresdb:
image: 'postgres'
command: postgres -c stats_temp_directory=/tmp
.
.
.
I am using official mongodb docker FROM mongo:3.2. In the entrypoint.sh I am restarting the Mongodb with replica mode. Mongodb process is owned by root user. Is there any way I can start the container by non root user and able to restart the mongodb in the replica set mode. Right now I am getting the following error.
2017-10-27T20:08:23.888+0000 I STORAGE [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /data/db/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
My docker file is
FROM mongo:3.2
COPY entrypoint.sh /root/entrypoint.sh
ENTRYPOINT ["/root/entrypoint.sh"] here
Thanks,
Using docker-compose:
version: '3.5'
services:
yourmongo:
image: mongo:xenial
container_name: yourmongo
restart: always
user: 1000:1000
volumes:
- ./yourmongo-folder:/data/db
1000:1000 are the chosen values for uid:gid (user-id:group-id)
EDIT: don't forget to add the folder with the correct permissions: mkdir yourmongo-folder && chown 1000:1000 yourmongo-folder (This must be done on the Docker host machine BEFORE bringing up the Docker service).
You can use the USER command available in dockerfile, when using ENTRYPOINT, CMD, RUN instructions to start/execute the process with the required user.
Syntax:
USER <user>[:<group>] or
USER <UID>[:<GID>]
Example:
FROM mongo:3.2
COPY entrypoint.sh /root/entrypoint.sh
USER deploy:root
ENTRYPOINT ["/root/entrypoint.sh"]
Update From user2010672: (working solution)
FROM mongo:3.2
COPY entrypoint.sh /root/entrypoint.sh
RUN chown -R mongodb:mongodb /var/log /data/db
USER mongodb
ENTRYPOINT ["/root/entrypoint.sh"]
Thanks user2010672
You can customize user on docker run command using the flag --user with user:group format
I'm having problems lauching mongod as a service:
How is it possible that it works when I do sudo mongod -f /etc/mongod.conf but when launching it with sudo service mongod start I get an error in the log
Assertion: 28595:13: Permission denied src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 267
I'm running mongodb on ubuntu 16
I followed exactly the instructions in the mongodb documentation for installation of that version, so is this a bug? Any suggestions how to solve this are appreciated.
Additional information:
The mongodb service startup script looks like this and runs it as user mongodb, could this be connected to the error?
lib/systemd/system/mongodb.service:
[Unit]
Description=MongoDB Database Service
Wants=network.target
After=network.target
[Service]
ExecStart=/usr/bin/mongod --config /etc/mongod.conf
ExecReload=/bin/kill -HUP $MAINPID
Restart=always
User=mongodb
Group=mongodb
StandardOutput=syslog
StandardError=syslog
[Install]
WantedBy=multi-user.target
I'm having problems lauching mongod as a service: How is it possible that it works when I do sudo mongod -f /etc/mongod.conf but when launching it with sudo service mongod start I get an error in the log
The sudo command starts mongod with root permissions (aka superuser access). If you run mongod as a service the user and group are configured in the service definition (mongodb for both in your example).
There is no need to run the mongod process as the root user, and this is strongly discouraged as per the common security practice of Principle of least privilege.
If you want to test a configuration from the command-line, you could use sudo to run with a specified user instead of the default (root) user.
For example:
sudo -u mongodb mongod -f /etc/mongod.conf
In general, it's best to use a service configuration rather than running mongod manually. With manual invocation you will also have to remember to include parameters like the config file path (as there is no default config path). Without a configuration file, mongod also uses default options such as a dbPath of /data/db.
Assertion: 28595:13: Permission denied src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 267
The likely cause of your permission errors is having previously started mongod as the root user. Some directories and files may now be owned by the root user, so the mongodb user cannot access those. Your specific error relates to accessing files in the data directory (i.e. the configured storage.dbPath in mongod.conf).
Assuming you haven't changed the default paths in your mongod.conf file, you should be able to recursively adjust permissions to match what the mongod.service definition expects.
First, ensure you have stopped your mongod instance if it is currently running.
Then, recursively adjust permissions to the expected user and group:
# storage.dbPath
sudo chown -R mongodb:mongodb /var/lib/mongodb
# systemLog.path
sudo chown -R mongodb:mongodb /var/log/mongodb
Now you should be able to start mongod as a service. If the service fails to start, there should be further detail in the mongod log file (assuming the log file is writable by the mongodb service user).
Have same problem.
What been in /var/log/mongodb/mongod.log:
2017-05-13T13:46:41.152+0700 E STORAGE [initandlisten] WiredTiger error (13) [1494658001:152518][15821:0x7fb843803cc0], connection: /var/lib/mongodb/journal/WiredTigerPreplog.0000000002: file-remove: unlink: Permission denied
2017-05-13T13:46:41.159+0700 I - [initandlisten] Assertion: 28595:13: Permission denied src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 267
So wee see that something can't remove file "WiredTigerPreplog.0000000002" in /var/lib/mongodb/journal/
So id just gave permissions, i just did:
sudo chmod 764 /var/lib/mongodb/journal/
If not help, try:
sudo chown -R mongodb:mongodb /var/lib/mongodb/ && sudo chmod 764 /var/lib/mongodb/journal/
There are three set-ups that triggers this kind of problem :
MongoDB installation is configured to create database files at a given path and this path does not exist on your current system. This path is called dbpath in mongo.
In your case, check if /data/db exist. If it doesn't or if it is empty, mongod is trying the wrong dbpath. You need to find it, it's usually under /var/lib/mongodb.
Once you found it there's two thing you can do. First, copy all the file from there to /data/db. Second, change your dbpath under the mongod.conf file, which is located (in linux) at /etc/mongod.conf. Make sure to start mongod with the --config specifying the configuration file.
MongoDB does not have the permission to read one or more files or directories corresponding to its dbpath.
chown mongodb:mongodb dbpath -R.
MongoDB is missing WiredTiger.wt . This can happen if you remove files under the dbpath or if there's a device failure. We do it for testing a recovery strategy for example.
If you're sure dbpath is correct and that there's no instance of WiredTiger.wt there. Your database is broken. There are no ways to ensure integrity if you lose this file. Reinstall mongodb by :
sudo apt-get purge mongodb-org*
sudo rm -r dbpath
sudo apt-get install mongodb-org
Edit :
Or copy dbpath from one of your replicas.
I had same problem in my mongod.log:
2021-09-16T16:06:43.782+0200 F STORAGE [initandlisten] Reason: 13: Permission denied
2021-09-16T16:06:43.782+0200 F - [initandlisten] Fatal Assertion 28595 at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 789
So I looked in my dbPath folder (specified in mongodb config file /etc/mongod.conf, section storage:dbPath) and found that same file was owned by root:
WiredTiger.turtle
WiredTigerLAS.wt
journal/WiredTigerLog.0000000112
From dbPath folder, using the command below, I change owner and group to mongodb:
sudo chown mongodb:mongodb WiredTiger.turtle WiredTigerLAS.wt journal/WiredTigerLog.0000000112
After this, I can start mongodb server as
"service mongod start"
and check its status as
"service mongod status" with output:
mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2021-09-16 16:19:47 CEST; 54min ago
Docs: https://docs.mongodb.org/manual
Main PID: 8459 (mongod)
CGroup: /system.slice/mongod.service
└─8459 /usr/bin/mongod --config /etc/mongod.conf
Sep 16 16:19:47 svi5-ubu16 systemd[1]: Started MongoDB Database Server.
Note that above commands are working as normal user, not as root.
My configuration: Ubuntu 16.04.2 LTS, MongoDB shell version v4.2.1.
Other file or folder owned by mongodb (user and group):
/var/log/mongodb
/var/lib/mongodb
/tmp/mongodb-27028.sock
I spent a lot of time in solving this issue, thank you very match to every one has posted here and guided me to solution.
I wish append a comment to the previous answer, but unfortunately I cannot yet.
I completely agree with the explanation of Stennie. It is exactly what was happened to me.
I've always run mongod as a service but today, because some changes that I've made, I've tried to run the process using sudo mongod --auth --dbpath /data/mongodb/.. to test authorizations and db changing location.
After that the mongod service didn't run anymore, due to this permissions problem.
I've to say that the command sudo chown -R mongodb:mongodb /data/mongodb/ didn't immediately fix the problem as expected. I've had to reboot several times, remove the mongod.lock file under /data/mongodb/, reissue the sudo chown command again.. and finally everything gone well.
I have a similar issue but with custom log path and data dir. updating the owner and group access to them did not fix the issue for running as a service
updating the group and owner to mongodb:mongodb works on running mongod by itself as stated earlier
sudo -u mongodb mongod -f /etc/mongod.conf
or
sudo mongod -f /etc/mongod.conf
If you find yourself with the same problem on CentOS, but the permissions look correct, it could be because of SELinux policies. On my system, MongoDB files use a specific SELinux context. The mongod server failed to start, outputting the same permissions errors, until I corrected the SELinux contexts.
If it doesn't exist, create the MongoDB data folder (either /var/lib/mongo or /var/lib/mongodb, check your /etc/mongod.conf file):
$ mkdir -p /var/lib/mongodb
Then try to restore the SELinux contexts:
$ restorecon -v /var/lib/mongodb
$ restorecon -v /var/lib/mongodb/*
If that still doesn't work, try to apply the contexts directly:
$ chcon system_u:object_r:mongod_var_lib_t:s0 /var/lib/mongodb
$ chcon system_u:object_r:mongod_var_lib_t:s0 /var/lib/mongodb/*
Verify the contexts are correct:
$ ls -lZ -d /var/lib/mongodb
drwxr-xr-x. mongod mongod system_u:object_r:mongod_var_lib_t:s0 /var/lib/mongodb
sudo chmod -R 666 /var/lib/mongodb
is another solution...
This docker-compose.yml:
services:
database:
image: mongo:3.2
ports:
- "27017"
command: "mongod --dbpath=/usr/database"
networks:
- backend
volumes:
- dbdata:/usr/database
volumes:
dbdata:
results in this error (snipped):
database_1 | 2016-11-28T06:30:29.864+0000 I STORAGE [initandlisten] exception in initAndListen: 98 Unable to create/open lock file: /usr/database/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
Ditto for just trying to run the command in a container using that image directly:
$ docker run -v /usr/database mongo:3.2 mongod --dbpath=/usr/database
But, if I run /bin/bash when starting the container, and THEN start mongo, we're OK:
$ docker run -it -v /usr/database mongo:3.2 /bin/bash
root#8aab722fad89:/# mongod --dbpath=/usr/database
Based on the output, the difference seems to be that in the second scenario, the command is run as root.
So, my questions are:
Why does the /bin/bash method work, when the others do not?
How can I replicate that reason, in the docker-compose?
Note: On OSX, since that seems to effect whether you can mount a host directory as a volume for Mongo to use - not that I'm doing that.
To clarify, this image hub.docker.com/_/mongo is an official MongoDB docker image from DockerHub, but NOT an official docker image from MongoDB.
Now to answer your questions,
Why does the /bin/bash method work, when the others do not?
This answer is based on Dockerfile v3.2. First to point out that your volume mount command -v /usr/database , is essentially creating a directory in the container with the root ownership permission.
Your command below failed with permission denied because the the docker image is running the command as user mongodb (see this dockerfile line) . As the directory /usr/database is owned by root.
$ docker run -v /usr/database mongo:3.2 mongod --dbpath=/usr/database
While if you execute below /bin/bash then manually run mongod:
$ docker run -it -v /usr/database mongo:3.2 /bin/bash
Your are logged in as root and executing mongod as root, and it has the permission to create database files in /usr/database/.
Also, if you're executing the line below, it works because you're pointing to a directory /data/db which the permission has been corrected for user mongodb (see this dockerfile line)
$ docker run -v db:/data/db mongo:3.2
How can I replicate that reason, in the docker-compose?
The easiest solution is to use command: "mongod --dbpath=/data/db" because the permission ownership has been corrected in the Dockerfile.
If you are intending to use a host volume, you probably would have to add mongodb user on your host OSX and change appropriate directories permission. Modifying the permission ownership of a volume mount is outside the scope of docker-compose.