Currently looking at deploying mongo in a container. So far my file looks like,
############################################################
# Dockerfile to build Mongo Containers
# Based on Ubuntu
############################################################
# Set the base image to Ubuntu
FROM ubuntu:14.04
# File Author / Maintainer
MAINTAINER Maintaner felix001
# Create repo file
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
RUN echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.0.list
# Update the default application repository sources list
RUN apt-get update && apt-get install -y \
mongodb-org \
vim
# Create the MongoDB data directory
RUN mkdir -p /data/db
# Expose port 27017 from the container to the host
EXPOSE 27017
# Set usr/bin/mongod as the dockerized entry-point application
ENTRYPOINT ["/usr/bin/mongod"]
However I need to lockdown mongo so you need a password to perform any admin actions and also create a database/user. So my question is 2 fold,
What is the best method for securing ? So far I have,
vim /etc/mongod.conf
+ auth = true
use admin
db.createUser({ user:"admin", pwd:"secretpassword", roles: ["dbAdminAnyDatabase","clusterAdmin"]})
use example
db.createUser({ user:"user1", pwd:"abc123", roles:["readWrite"] })
What is the best method for adding this to a Dockerfile ?
Thanks,
I would not recommend "baking" credentials into your image. Every database you create will then have the same password.
For an example take a look at the tutum mongodb image, where the container runs a script set the password on startup:
https://hub.docker.com/r/tutum/mongodb/
https://github.com/tutumcloud/mongodb/tree/master/3.0
Finally, I refer you to a related question concerning the use of environment variables for passwords:
Specifying superuser PostgreSQL password for a Docker Container
Here is how I secured my MongoDB docker container.
Following step-by-step process will guide you to implement the security.
Solution-1 : Using Environment Variable
A simple solution is to use environment variables when docker run command
$ docker run -d -p 27017:27017 --name mongodb -v /var/docker/mongo/data:/data/db \
-e MONGO_INITDB_ROOT_USERNAME=superuser \
-e MONGO_INITDB_ROOT_PASSWORD=Strong_Password \
mongo:4.4.2
When you would try to access using MongoDB client like MongoDB-Compass, then you would have to supply the username and password to access the database.
Solution-2 : Create User in MongoDB Database (Recommended approach)
Create docker container using auth option
$ docker run --name <container_name> --restart=always -d -p 27017:27017 mongo mongod --auth
Bash into the container
$ sudo docker exec -it <container_name> bash
Connect to local mongo instance
# mongo
Create the first admin user
> use admin
> db.createUser({
user: 'user',
pwd: 'StrongPassword',
roles: [{ role: 'dbOwner', db:'admin'}]
})
Exit the mongo shell
> exit
Exit the container
# exit
Now you can connect with the username and password. Remember to use --authenticationDatabase "admin"
mongo -u "user" -p "StrongPassword" YOURHOSTIP --authenticationDatabase "admin"
You can also connect to mongo container via MongoDB-Compass. I connected using the following connection string. Check your string carefully if you couldn't connect
mongodb://user:*****#IP:27017/?authSource=admin&compressors=zlib&readPreference=primary&gssapiServiceName=mongodb&appname=MongoDB%20Compass&ssl=false
Ref: https://gist.github.com/davideicardi/f2094c4c3f3e00fbd490#file-mongo-docker-bash-L23
Related
Good evening.
I'm noobie in docker and try to learn it a little bit. Currently writing simple java application integrated with mongodb, but I stuck on dockerfile. Basically the problem is with mongodb start. Here is my docker file:
FROM debian:buster-slim
# Install necessary libs
RUN apt-get update && apt-get install -y apt-utils wget gnupg gnupg2 curl
# Install mongodb
RUN wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | apt-key add -
RUN echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
RUN systemctl enable mongod.service
RUN service mongod start
# Install jre 11
RUN apt-get install -y openjdk-11-jre
Here is the terminal output (only last step):
Setting up mongodb-org-shell (4.2.1) ...
Setting up mongodb-org-tools (4.2.1) ...
Setting up mongodb-org-mongos (4.2.1) ...
Setting up mongodb-org (4.2.1) ...
Removing intermediate container 7491080bfe9f
---> bbcf5b2ccb13
Step 7/11 : RUN service mongod start
---> Running in 46a66989ade2
mongod: unrecognized service
The command '/bin/sh -c service mongod start' returned a non-zero code: 1
Funny think is that I followed an official mongodb installation guide:
Mongodb installation on debian
During installation on 'real' debian/ubuntu machine it works.
It also doesn't work when tried to build docker image from official mongodb image from docker hub, I mean FROM mongo:4.2-bionic
After login to container and try to run mongo it returns:
root#8cc1d270a262:~# mongo
MongoDB shell version v4.2.0
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
2019-10-23T20:39:44.728+0000 E QUERY [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:341:17
#(connect):2:6
2019-10-23T20:39:44.729+0000 F - [main] exception: connect failed
2019-10-23T20:39:44.729+0000 E - [main] exiting with code 1
I expected that, cause mongo is unable to stand... Somehow.
Any ideas?
So, it seems when trying to follow the instructions to install MongoDB on Debian the SysVInit files are not created and error message mongod: unrecognized service. So a basic question: Does a docker container really need daemon control with either SysVInit or systemd? I don't think it really needs it, and my reason is because the container itself has a single purpose - to host the database. The container should always have the database engine running. With this philosophy in mind, I altered the Dockerfile to include an ENTRYPOINT that starts the mongod instead of relying on any daemon management system.
In order for the MongoDB database to be available outside the container I adjusted the mongod.conf file to bind to all network adapters by using bindIp: 0.0.0.0 instead of bindIp: 127.0.0.1. I also expose port 27017 in the Dockerfile. This means if you have MongoDB installed and running on the host computer using the default port 27107 that process will need to be halted to yield the port to the Docker container.
I was getting some errors in the container around the debconf stuff so I set it non-interactive as well. The installation of java was giving me fits, so I commented it out. If you need java on this container this will still need to be worked out.
Dockerfile:
FROM debian:buster-slim
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
# Install necessary libs
RUN apt-get update && apt-get install -y apt-utils wget gnupg gnupg2 curl
# Install mongodb
RUN wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | apt-key add -
RUN echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-4.2.list
RUN apt-get update
RUN apt-get install -y mongodb-org
# BIND TO ALL ADAPTERS IN CONTAINER
RUN sed -i "s,\\(^[[:blank:]]*bindIp:\\) .*,\\1 0.0.0.0," /etc/mongod.conf
# Install jre 11
# RUN "apt-get install -y openjdk-11-jre"
EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod", "-f", "/etc/mongod.conf"]
Build:
To build the Docker image issue the following command...
docker build --tag mongodb .
(notice the period in the command - it is required).
Run:
To create a docker container, use the run command.
docker run --publish 27017:27017 --name mongodb -d mongodb
Notice the --publish to map host port 27017 to container port 27017. Notice the --name to name the container for easier reference if we need to get a bash shell inside the container. Run -d for detached mode so it runs in the background, and finally refer to the image named mongodb.
Connect:
Assuming MongoDB is installed on the host too the mongo shell binary will be available. Issue a mongo shell command...
mongo
No other parameters are needed. The installation of MongoDB in the container does not have authorization enabled and so does not ask for a username or password. The default port of 27107 is used by the container and mapped by the docker engine. Localhost is used by default.
Get BASH shell of container:
If you want to get a BASH shell inside of the container issue the following command...
docker exec -it mongodb bash
Try to run mongodb docker container and connect to it using mongo client before building custom images:
docker run --name some-mongo -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -d mongo
docker exec -it some-mongo bash
mongo -u mongoadmin -p secret --authenticationDatabase admin
I'm new to Docker and I need help setting up my web app.
So I'm able to get my mongo container up and running - it is also the latest version - by running docker run -d --name mongo1 mongo.
I then launch the web-app with linking mongoDB by running docker run -it --link mongo1:mongo --name webapp1 webapp
It then errors out on the build: Error: connect ECONNREFUSED 127.0.0.1:27017
Here's the build that should be exposing the ports:
FROM node:latest
RUN mkdir -p /webapp /home/nodejs && \
groupadd -r nodejs && \
useradd -r -g nodejs -d /home/nodejs -s /sbin/nologin nodejs && \
chown -R nodejs:nodejs /home/nodejs
WORKDIR /webapp
COPY package.json typings.json /webapp/
RUN npm install
COPY . /webapp
RUN chown -R nodejs:nodejs /webapp
USER nodejs
EXPOSE 3000
EXPOSE 27017
CMD [ "npm", "start" ]
What am I missing?
EDIT: This like may be useful, alongside cml.co's answer: http://www.ifdattic.com/how-to-mongodb-nodejs-docker/
Your nodejs app should connect to mongodb using mongo:27017 (as you set mongo as the mongodb alias inside container --link mongo1:mongo) and not localhost:27017 neither127.0.0.1:27017.
So, check mongodb connection url.
By the way, exposing port 27017 seems to be not necessary in webapp container
Is there an easy way to clear a mongodb database running in docker?
Or possibly delete it all and create it again?
There are a number of ways:
The same way you would normally clear a mongodb database. You're running it in Docker, but it is a real mongodb after all. See this, and this.
If you've mounted your database's data as a volume on the host system using -v on the docker run command, you can just clear this folder out.
If you haven't, the data lives in the container. You can remove the container and recreate it (docker rm container_name). Or run a shell in the container and remove the data from the container's filesystem (docker exec -it container_name bash).
Regarding the last option, you shouldn't be in this scenario because your data should live on the host system and your container should be disposable.
docker exec -it mongodb_container bash -c "mongo db-name --eval 'db.dropDatabase()'"
I had a case with slightly different requirements:
docker-compose
authentication needed
bash variable expansion (the container was running with environemt variables MONGO_USER & MONGO_PASSWORD already set)
So, for anyone who might need a handy one-liner to authenticate and drop a mongo db with docker, here it is 👇:
docker-compose exec mongo_container_name /bin/bash -c 'mongo database_name -u $MONGO_USER -p $MONGO_PASSWORD --authenticationDatabase admin --eval "db.dropDatabase();"'
And if you add a Makefile to the "mix", it's exactly the same but it needs $$ to the variables:
db-reset:
docker-compose exec mongo_container_name /bin/bash -c 'mongo database_name -u $$MONGO_USER -p $$MONGO_PASSWORD --authenticationDatabase admin --eval "db.dropDatabase();"'
I'm running postgres inside a docker container to limit the amount of system resources it has access to. I'm having some trouble understanding how to make the data persistent. I've read the following articles:
https://www.andreagrandi.it/2015/02/21/how-to-create-a-docker-image-for-postgresql-and-persist-data/
http://container42.com/2013/12/16/persistent-volumes-with-docker-container-as-volume-pattern/
Which suggest using a data only container, and then having my postgres container link to it. What I'm failing to understand is; what's the advantage to this? As far as I can tell, if for some reason the docker-machine shut down (for example; moving it to a different physical machine), the data only container stops running, and all of it's contents are lost? I've tried creating a volume in the postgres container, but it doesn't actually seem to save anything to the disk.
Here's my docker file. What am I doing wrong?
FROM ubuntu
MAINTAINER Andrew Broadbent <andrew.broadbent#manchester.ac.uk>
# Add the PostgreSQL PGP key to verify their Debian packages.
# It should be the same key as https://www.postgresql.org/media/keys/ACCC4CF8.asc
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8
# Add PostgreSQL's repository. It contains the most recent stable release
# of PostgreSQL, ``9.3``.
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main" > /etc/apt/sources.list.d/pgdg.list
# Install ``python-software-properties``, ``software-properties-common`` and PostgreSQL 9.3
# There are some warnings (in red) that show up during the build. You can hide
# them by prefixing each apt-get statement with DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y python-software-properties software-properties-common postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
# Note: The official Debian and Ubuntu images automatically ``apt-get clean``
# after each ``apt-get``
# Run the rest of the commands as the ``postgres`` user created by the ``postgres-9.3`` package when it was ``apt-get installed``
USER postgres
# Create a PostgreSQL role named ``docker`` with ``docker`` as the password and
# then create a database `docker` owned by the ``docker`` role.
# Note: here we use ``&&\`` to run commands one after the other - the ``\``
# allows the RUN command to span multiple lines.
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'docker';" &&\
createdb -O docker docker
# Complete configuration
USER root
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/9.3/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/9.3/main/postgresql.conf
# Expose the PostgreSQL port
EXPOSE 5432
# Add VOLUMEs to allow backup of config, logs and databases
RUN mkdir -p /var/run/postgresql && chown -R postgres /var/run/postgresql
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
# Set the default command to run when starting the container
USER postgres
CMD ["/usr/lib/postgresql/9.3/bin/postgres", "-D", "/var/lib/postgresql/9.3/main", "-c", "config_file=/etc/postgresql/9.3/main/postgresql.conf"]
This one answers your question about data container:
docker mounting volumes on host
Regarding to your dockerfile, I would suggest you either:
1) use data container pattern
2) mount the volume to host machine by specifying: docker run -v [host-path]:[container-path] ..., so that data will be kept at one place in your host and will not be lost after the container is removed.
Ref: https://docs.docker.com/engine/tutorials/dockervolumes/#/mount-a-host-directory-as-a-data-volume
I deployed my app on a Ubuntu server using mup deploy (https://github.com/arunoda/meteor-up) with the option "setupMongo": true in the mup.json file.
Everything works fine, and I would like to save the mongoDB database daily to FTP or S3, or to set a mongoDB replica to another server (to avoid copying the whole database every time, but it seems more complicated).
If deployed with mup, you are in luck.
You can find the steps here: https://github.com/xpressabhi/mup-data-backup
Here are the steps again:
MongoDB Data Backup deployed via mup
These commands run well only if meteor deployed with mup tool. Mup creates docker for mongodb hence taking backup becomes easy with these commands.
Backup
Take backup of running app data from docker then copy to local folder out of docker.
docker exec -it mongodb mongodump --archive=/root/mongodump.gz --gzip
docker cp mongodb:/root/mongodump.gz mongodump_$(date +%Y-%m-%d_%H-%M-%S).gz
Copy backup to server
Move data to another server/local machine or a backup location
scp /path/to/dumpfile root#serverip:/path/to/backup
Delete old data from meteor deployment
Get into mongo console running in docker then drop current database before getting new data.
docker exec -it mongodb mongo appName
db.runCommand( { dropDatabase: 1 } )
Restore data to meteor docker
docker cp /path/to/dumpfile mongodb:/root/mongodump.gz
docker exec -it mongodb mongorestore --archive=/root/mongodump.gz --gzip
The best way is to mongodump it.
Assuming its running on the mup instance itself since it only listens to 127.0.0.1 you would have to ssh in and use mongodump.
If you simply run it:
mongodump
It will create a directory dump containing your backup.
If you want to do this remotely you would have to edit /etc/mongodb.conf to ensure it binds globally, you will have to create users though since it will be publicly accessible. Then set auth to true.
You could then mongodump from your own machine (you can download the mongodump binary from mongodb.org):
./mongodump --host <your server ip address> --username <username> --password <password>
This answer is inspired by:
sheharyar.me/blog/regular-mongo-backups-using-cron
It uses a script to: mongodump -> tar -> wput (ftp)
First, create a bash script:
#!/bin/bash
MONGO_DATABASE="your_db_name"
APP_NAME="your_app_name"
MONGO_HOST="127.0.0.1"
MONGO_PORT="27017"
TIMESTAMP=`date +%F-%H%M`
MONGODUMP_PATH="/usr/bin/mongodump"
BACKUPS_DIR="/home/username/backups/$APP_NAME"
BACKUP_NAME="$APP_NAME-$TIMESTAMP"
# mongo admin --eval "printjson(db.fsyncLock())"
# $MONGODUMP_PATH -h $MONGO_HOST:$MONGO_PORT -d $MONGO_DATABASE
$MONGODUMP_PATH -d $MONGO_DATABASE
# mongo admin --eval "printjson(db.fsyncUnlock())"
mkdir -p $BACKUPS_DIR
mv dump $BACKUP_NAME
tar -zcvf $BACKUPS_DIR/$BACKUP_NAME.tgz $BACKUP_NAME
rm -rf $BACKUP_NAME
wput $BACKUP_NAME.tgz ftp://login:password#ftp.domain.com/backups/
Save it as mongo_backup.sh and run:
chmod +x mongo_backup.sh
bash mongo_backup.sh
sudo su
crontab -e
And enter this new line:
00 00 * * * /bin/bash /home/username/scripts/mongo_backup.sh
That's it.