I create docker-compose with two services: python api and mongodb, after starting first-time sudo docker-compose up it creates a file for mongo db inside \data, after sudo docker-compose down and again sudo docker-compose up mongo can't access files in \data.
Obviously something wrong with permissions, but I don't know what exactly.
But if I just stop containers without sudo docker-compose down and sudo docker-compose up again everything is ok.
So everything goes wrong after down
systemd service:
[Unit]
Description=RestAPI imp
Requires=docker.service
After=docker.service
[Service]
Type=simple
WorkingDirectory=/home/entrant/myserv
ExecStart=/usr/local/bin/docker-compose up
ExecStop=/usr/local/bin/docker-compose down
Restart=on-failure
RestartSec=10
KillMode=process
[Install]
WantedBy=multi-user.target
docker-compose.yml
version: '3.5'
services:
web_dev:
build: .
ports:
- "8080:8080"
volumes:
- .:/app
environment:
- ENV=PROD
- DATABASE_URL=mongodb://mongodb:27017/myserv?authSource=admin&replicaSet=myrepl
depends_on:
- mongodb
command: deploy/wait-for-it.sh mongodb:27017 -- gunicorn -b 0.0.0.0:8080 index:api.app -w 9
mongodb:
image: mongo:4.0.12-xenial
container_name: "mongodb"
environment:
- MONGO_INITDB_DATABASE=myserv
- MONGO_DATA_DIR=/data/db
- MONGO_LOG_DIR=/dev/null
volumes:
- ./data/db:/data/db
ports:
- 27017:27017
command: bash -c "
mongod --fork --replSet myrepl --bind_ip_all --smallfiles --logpath=/dev/null
&& mongo --eval 'rs.initiate()'
&& mongod --shutdown
&& mongod --replSet myrepl --bind_ip_all --smallfiles --logpath=/dev/null
"
networks:
default:
name: web_dev
Related
I want to create a Docker container with an instance of Mongo. In particular, I would like to create a replica set with only one node (since I'm interested in transactions and they are only available for replica sets).
Dockerfile
FROM mongo
RUN echo "rs.initiate();" > /docker-entrypoint-initdb.d/replica-init.js
CMD ["--replSet", "rs0"]
docker-compose.yml
version: "3"
services:
db:
build:
dockerfile: Dockerfile
context: .
ports:
- "27017:27017"
If I use the Dockerfile alone everything is fine, while if I use docker-compose it does not work: in fact if I then log to the container I got prompted as rs0:OTHER> instead of rs0:PRIMARY>.
I consulted these links but the solutions proposed are not working:
https://github.com/docker-library/mongo/issues/246#issuecomment-382072843
https://github.com/docker-library/mongo/issues/249#issuecomment-381786889
This is the compose file I have used for a while now for local development. You can remove the keyfile pieces if you don't need to connect via SSL.
version: "3.8"
services:
mongodb:
image : mongo:4
container_name: mongodb
hostname: mongodb
restart: on-failure
environment:
- PUID=1000
- PGID=1000
- MONGO_INITDB_ROOT_USERNAME=mongo
- MONGO_INITDB_ROOT_PASSWORD=mongo
- MONGO_INITDB_DATABASE=my-service
- MONGO_REPLICA_SET_NAME=rs0
volumes:
- mongodb4_data:/data/db
- ./:/opt/keyfile/
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 10s
start_period: 30s
command: "--bind_ip_all --keyFile /opt/keyfile/keyfile --replSet rs0"
volumes:
mongodb4_data:
It uses Docker's health check (with a startup delay) to sneak in the rs.initiate() if it actually needs it after it's already running.
To create a keyfile.
Mac:
openssl rand -base64 741 > keyfile
chmod 600 keyfile
Linux:
openssl rand -base64 756 > keyfile
chmod 600 keyfile
sudo chown 999 keyfile
sudo chgrp 999 keyfile
The top answer stopped working for me in later MongoDB and Docker versions. Particularly because rs.initiate().ok would throw an error if the replica set was already initiated, causing the whole command to fail. In addition, connecting from another container was failing because the replica set's sole member had some random host, which wouldn't allow the connection. Here's my new docker-compose.yml:
services:
web:
# ...
environment:
DATABASE_URL: mongodb://root:root#db/?authSource=admin&tls=false
db:
build:
context: ./mongo/
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: root
ports:
- '27017:27017'
volumes:
- data:/data/db
healthcheck:
test: |
test $$(mongosh --quiet -u root -p root --eval "try { rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'db' }] }).ok } catch (_) { rs.status().ok }") -eq 1
interval: 10s
start_period: 30s
volumes:
data:
Inside ./mongo/, I have a custom Dockerfile that looks like:
FROM mongo:6
RUN echo "password" > /keyfile \
&& chmod 600 /keyfile \
&& chown 999 /keyfile \
&& chgrp 999 /keyfile
CMD ["--bind_ip_all", "--keyFile", "/keyfile", "--replSet", "rs0"]
This Dockerfile is suitable for development, but you'd definitely want a securely generated and persistent keyfile to be mounted in production (and therefore strike the entire RUN command).
You still need to issue replSetInitiate even if there's only one node in the RS.
See also here.
I had to do something similar to build tests around ChangeStreams which are only available when running mongo as a replica set. I don't remember where I pulled this from, so I can't explain it in detail but it does work for me. Here is my setup:
Dockerfile
FROM mongo:5.0.3
RUN echo "rs.initiate({'_id':'rs0', members: [{'_id':1, 'host':'127.0.0.1:27017'}]});" > "/docker-entrypoint-initdb.d/init_replicaset.js"
RUN echo "12345678" > "/tmp/key.file"
RUN chmod 600 /tmp/key.file
RUN chown 999:999 /tmp/key.file
CMD ["mongod", "--replSet", "rs0", "--bind_ip_all", "--keyFile", "/tmp/key.file"]
docker-compose.yml
version: '3.7'
services:
mongo:
build: .
restart: always
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u admin -p pass --quiet) -eq 1
interval: 10s
start_period: 30s
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: pass
MONGO_INITDB_DATABASE: test
Run docker-compose up and you should be good.
Connection String: mongodb://admin:pass#localhost:27017/test
Note: You shouldn't use this in production obviously, adjust the key "12345678" in the Dockerfile if security is a concern.
If you just need single node replica set of MongoDB via docker-compose.yml you can simply use this:
mongodb:
image: mongo:5
restart: always
command: ["--replSet", "rs0", "--bind_ip_all"]
ports:
- 27018:27017
healthcheck:
test: mongo --eval "rs.initiate()"
start_period: 5s
This one works fine for me:
version: '3.4'
services:
ludustack-db:
container_name: ludustack-db
command: mongod --auth
image: mongo:latest
hostname: mongodb
ports:
- '27017:27017'
env_file:
- .env
environment:
- MONGO_INITDB_ROOT_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
- MONGO_INITDB_DATABASE=${MONGO_INITDB_DATABASE}
- MONGO_REPLICA_SET_NAME=${MONGO_REPLICA_SET_NAME}
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 60s
start_period: 60s
I'm trying to deploy a minimal mongodb cluster into my development swarm environment like this:
Up to now:
version: "3.3"
networks:
net:
driver: overlay
services:
data1:
image: mongo:3.6
container_name: data1
command: mongod --shardsvr --replSet datars --smallfiles --port 27017
expose:
- 27017
networks:
- net
cfg1:
image: mongo:3.6
container_name: cfg1
command: mongod --configsvr --replSet cfgrs --smallfiles --port 27017
expose:
- 27017
networks:
- net
mongos1:
image: mongo:3.4
container_name: mongos1
command: mongos --configdb cfgrs/cfg1:27017
expose:
- 27017
networks:
- net
Into my config server, I'm getting these message:
2019-09-06T09:22:15.693+0000 I SHARDING [shard registry reload] Periodic reload of shard registry failed :: caused by :: 134 could not get updated shard list from config server due to Read concern majority reads are currently not possible.; will retry after 30s,
Any ideas?
I have several docker containers , one of them is Mongodb official image,
Here's part of my docker-compose.yml file
version: '3'
services:
mongo:
image: mongo
container_name: mongo01
# command: ["mongod", "-f", "/etc/mongo/mongod.conf"]
volumes:
- ./data/mongodata:/data/db
# - ./config/mongo:/etc/mongo
restart: always
ports:
- "27017:27017"
I could access to mongo service from the host ( my system) but according to the mongo new security policy there is config for limit access to mongo just form 127.0.0.1,I know it , it's
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
if I could push the mongo image read my custom config I could resolve the problem, but I tried to
to mount a custom config file - ./config/mongo:/etc/mongo and then run mongod with command: ["mongod", "-f", "/etc/mongo/mongod.conf"] but didn't work.
it seems mongod starting in container as process 1 and try to run it with custom command not works, even when I tried to shutdown the mongod in container with mongod --shutdown it shutdown the whole container.( I wanted to stop the mongod and then rerun it with mongod --bind_ip_all )
So the problem is how we can change the mongo image config file ?
The mongo docker image already has an ENTRYPOINT set and it basically is mongod, so in your command (CMD) you can add extra arguments to mongod
simple docker run
docker run -d mongo --bind_ip_all
or with compose
version: '3'
services:
mongo:
image: mongo
command: ["--bind_ip_all"]
ports:
- "27017:27017"
The entrypoint for the official mongo image already contains a step to add --bind_ip_all as long as you don't explicitly bind a specific IP:
# MongoDB 3.6+ defaults to localhost-only binding
haveBindIp=
if _mongod_hack_have_arg --bind_ip "$#" || _mongod_hack_have_arg --bind_ip_all "$#"; then
haveBindIp=1
elif _parse_config "$#" && jq --exit-status '.net.bindIp // .net.bindIpAll' "$jsonConfigFile" > /dev/null; then
haveBindIp=1
fi
if [ -z "$haveBindIp" ]; then
# so if no "--bind_ip" is specified, let's add "--bind_ip_all"
set -- "$#" --bind_ip_all
fi
I would like to start this MongoDB Replica Set:
version: "3"
services:
mongo1:
image: mongo
ports:
- 27017:27017
command: mongod --replSet rs0
mongo2:
image: mongo
ports:
- 27018:27017
command: mongod --replSet rs0
mongo3:
image: mongo
ports:
- 27019:27017
command: mongod --replSet rs0
Wait for those to come up, then access the Mongo shell via terminal:
docker exec -it mongo1 mongo
Then in Mongo shell do:
rs.initiate({"_id":"rs0","members":[{"_id":0,"host":"mongo1:27017"},{"_id":1,"host":"mongo2:27017"},{"_id":2,"host":"mongo3:27017"}]})
Mongo also allows mongo --eval "rs.initiate(..)", which may make things easier.
My question is how do I run this command after mongo1, mongo2, mongo3 are up?
You can do this, I recently had to run mongo --repair then run the MongoDB itself and after the MongoDB is up I needed to add my user to the DB, you can easily change things to run commands only after all three MongoDBs are up.
Possible docker-compose.yml:
version: "2"
services:
mongo:
container_name: mongo
restart: on-failure:10
image: mongo
environment:
- MONGO_INITDB_ROOT_USERNAME=<user>
- MONGO_INITDB_ROOT_PASSWORD=<pass>
- MONGO_INITDB_DATABASE=db
volumes:
- ./data:/data/db
ports:
- "27017:27017"
command: bash -c "mongod --repair && mongod"
mongoClient:
image: mongo
container_name: mongoClient
links:
- mongo
volumes:
- ./deployment_scripts:/deployment_scripts
command:
- /deployment_scripts/add_user.sh
depends_on:
- mongo
app:
container_name: app
restart: always
build: .
volumes:
- .:/usr/src/app
ports:
- "3000:3000"
depends_on:
- mongoClient
links:
- mongo
My /deployment_scripts/add_user.sh script wait for MongoDB to be up:
until mongo --host mongo --eval "print(\"waited for connection\")"
do
sleep 1
done
// you can add more MongoDB waits here
echo "Adding user to MongoDB..."
mongo --host mongo --eval "db.createUser({ user: \"<user>\", pwd: \"<pass>\", roles: [ { role: \"root\", db: \"admin\" } ] });"
echo "User added."
Note that you can address all three of your MongoDBs by replacing --host mongo with your --host mongo1 --host mongo2 and --host mongo3. You'll use this for both of the eval commands in the script.
Credit to this SO answer https://stackoverflow.com/a/45060399/4295037 that I used (until mongo ...).
I assume you are using Oficial Mongo image, that image is configured with:
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["mongod"]
If you check the docker-entrypoint.sh you will notice you could run any command you want by overwriting the CMD.
So, you can do for each mongo container
$ docker run -d mongo
9bf0473d491a2d7ae821bcf10ed08cd49678d28e46344589622bd9440a6aca65
$ docker ps -q
9bf0473d491a
$ docker exec -ti 9bf0473d491a mongo --eval "rs.initiate(.....)"
MongoDB shell version v3.6.5
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.5
{
"ok" : 0,
"errmsg" : "This node was not started with the replSet option",
"code" : 76,
"codeName" : "NoReplicationEnabled"
}
Notice that the errmsg is just because in my example the rs.initiate() is empty, it will work for you with the right config.
I'm new to Docker, and I'm trying the simplest of setups with docker-compose, but don't succeed to connect to Mongodb.
My docker-compose.local.yaml file:
version: "2"
services:
posts-api:
build:
dockerfile: Dockerfile.local
context: ./
volumes:
- ".:/app"
ports:
- "6820:6820"
depends_on:
- mongodb
mongodb:
image: mongo:3.5
ports:
- "27018:27018"
command: mongod --port 27018
My Docker file:
FROM node:7.8.0
MAINTAINER Livefeed 'project.livefeed#gmail.com'
RUN mkdir /app
VOLUME /app
WORKDIR /app
ADD package.json yarn.lock ./
RUN eval rm -rf node_modules && \
yarn
ADD server.js .
RUN mkdir config src
ADD config config/
ADD src src/
EXPOSE 6820
EXPOSE 27018
CMD yarn run local
In server.js I try to connect with:
mongoose.connect('mongodb://localhost:27018');
I also tried:
mongoose.connect('mongodb://mongodb:27018');
To run docker-compose:
docker-compose -f docker-compose.local.yaml up --build
And I receive the error:
connection error: { MongoError: failed to connect to server [localhost:27018] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27018]
What am I missing?
In server.js use mongodb instead of localhost:
mongoose.connect('mongodb://mongodb:27018');
Because containers in the same network can communicate using their service name.
Bear in mind that each container and your host have their own localhost. Each localhost is a different host: container A, container B, your host (each one has its own network interface).
Edit:
Be sure to get your mongo up:
docker-compose logs mongodb
docker-compose ps
Sometimes it doesn't get up because of disk space.
Edit 2:
With newer versions of mongo, you need to specify to listen to all interfaces too:
command: mongod --port 27018 --bind_ip_all
I think, that you should add links option in your config. Like this:
ports:
- "6820:6820"
depends_on:
- mongodb
links:
- mongodb
update
As I promised
version: '2.1'
services:
pm2:
image: keymetrics/pm2-docker-alpine:6
restart: always
container_name: pm2
volumes:
- ./pm2:/app
links:
- redis_db
- db
environment:
REDIS_CONNECTION_STRING: redis://redis_db:6379
nginx:
image: firesh/nginx-lua
restart: always
volumes:
- ./nginx:/etc/nginx
- /var/run/docker.sock:/tmp/docker.sock:ro
ports:
- 80:80
links:
- pm2
s3: # mock for development
image: lphoward/fake-s3:latest
redis_db:
container_name: redis_db
image: redis
ports:
- 6379:6379
db: # for scorebig-syncer
image: mysql:5.7
ports:
- 3306:3306