Docker compose for MongoDB ReplicaSet - mongodb

I have been trying to dockerize my spring boot application which depends on redis, kafka and mongodb.
Following is the docker-compose.yml:
version: '3.3'
services:
my-service:
image: my-service
build:
context: ../../
dockerfile: Dockerfile
restart: always
container_name: my-service
environment:
KAFKA_CONFLUENT_BOOTSTRAP_SERVERS: kafka:9092
MONGO_HOSTS: mongodb:27017
REDIS_HOST: redis
REDIS_PORT: 6379
volumes:
- /private/var/log/my-service/:/var/log/my-service/
ports:
- 8080:8090
- 1053:1053
depends_on:
- redis
- kafka
- mongodb
portainer:
image: portainer/portainer
command: -H unix:///var/run/docker.sock
restart: always
container_name: portainer
ports:
- 9000:9000
- 9001:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
redis:
image: redis
container_name: redis
restart: always
ports:
- 6379:6379
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookeeper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
container_name: kafka
environment:
KAFKA_CREATE_TOPICS: "cms.entity.change:1:1" # topic:partition:replicas
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_PORT: 9092
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- "zookeeper"
mongodb:
image: mongo:latest
container_name: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME:
MONGO_INITDB_ROOT_PASSWORD:
ports:
- 27017:27017
volumes:
- ./data/db:/data/db
The issue is that this starts up mongo as a STANDALONE instance. So the APIs in my service that persist data are failing as mongo needs to start as a REPLICA_SET.
How can I edit my docker-compose file to start mongo as a REPLICA_SET?

I had the same issue and ended up on this stackoverflow post.
We had a requirement of using official mongoDB docker image (https://hub.docker.com/_/mongo ) and couldn't use bitnami as suggested in Vahid's answer.
This answer isn't exactly what's needed by the question asked and coming in 6 months too late; but it should give directions to someone who need to use the mongoDb standalone replicaset throw away instance for integration testing purpose. If you need to use it in PROD then you'll have to provide environment variables for volumes and auth as per Vahid's answer.
version: '3.7'
services:
mongodb:
image: mongo:latest
container_name: myservice-mongodb
networks:
- myServiceNetwork
expose:
- 27017
command: --replSet singleNodeReplSet
mongodb-replicaset:
container_name: mongodb-replicaset-helper
depends_on:
- mongodb
networks:
- myServiceNetwork
image: mongo:latest
command: bash -c "sleep 5 && mongo --host myservice-mongodb --port 27017 --eval \"rs.initiate()\" && sleep 2 && mongo --host myservice-mongodb --port 27017 --eval \"rs.status()\" && sleep infinity"
my-service:
depends_on:
- mongodb-replicaset
image: myserviceimage
container_name: myservicecontainer
networks:
- myServiceNetwork
environment:
myservice__Database__ConnectionString: mongodb://myservice-mongodb:27017/?connect=direct&replicaSet=singleNodeReplSet&readPreference=primary
myservice__Database__Name: myserviceDb
networks:
myServiceNetwork:
driver: bridge
NOTE: Please look at the way how connection string is passed as env variable to the service depending on mongo replicaset instance. You'd have to ensure that the name used in setting up the mongodb replicaset (in my case singleNodeReplicaSet) is passed on to the service depending on it.

Edited:
my previous answer was far wrong so I changed it. I managed to make it work using 'bitnami/mongodb:4.0'. Not sure if that would help you or not, but maybe it gives you some idea. They have a docker-compose file ready for replicaset mode.
version: '3'
services:
mdb-primary:
image: 'bitnami/mongodb:4.0'
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-primary
mdb-secondary:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-secondary
mdb-arbiter:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-arbiter
mongo-cli:
image: 'bitnami/mongodb:latest'
don't forget to add volumes and map it to /bitnami on the primary node
the last container, mongo-cli is for testing purposes. So you can connect to the replicaset using the cli, there is an argument about that here if you like to read about it.
$ docker-compose exec mongo-cli bash
$ mongo "mongodb://mdb-primary:27017/test?replicaSet=replicaset"

Related

How to use data from MongoDB in docker container on Typesense in the same container?

Description
I have a container that has a mongodb, mongo-express and typesense. I need to set that typesense use data from the mongodb inside the container. How can i do this?
I already used and configured the replica set.
Steps to reproduce
This is my docker compose file:
version: "3.9"
services:
database:
image : 'mongo'
container_name: 'container-name'
environment:
- PUID=1000
- PGID=1000
volumes:
- ./database:/data/db
entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
ports:
- 27017:27017
restart: unless-stopped
mongo-express:
image: mongo-express:0.54
ports:
- "8081:8081"
environment:
ME_CONFIG_MONGODB_SERVER: database
depends_on:
- database
typesense:
image: typesense/typesense:0.21.0
ports:
- "8108:8108"
environment:
TYPESENSE_API_KEY: xyz
TYPESENSE_DATA_DIR: /data/typesense
TYPESENSE_ENABLE_CORS: "true"
volumes:
- ./typesense:/data/typesense
When i call the API colletions/data, i get a response and on the end i see num_documents: 0
Github answer

How to connect remotely to Mongodb running on Docker-compose?

How to change this script so that I can connect remotely to my Mongodb running in docker-compose, from different machines (that are not connected to the same network/internet provider).
I want to allow all remote connections.
I don't care about security matters as it's just for practice purposes!
docker-compose.yaml script file:
version: "3.8"
services:
mongodb:
image: mongo
container_name: mongodb
ports:
- 27017:27017
volumes:
- data:/data
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=admin
mongo-express:
image: mongo-express
container_name: mongo-express
restart: always
ports:
- 8081:8081
environment:
- ME_CONFIG_MONGODB_ADMINUSERNAME=admin
- ME_CONFIG_MONGODB_ADMINPASSWORD=admin
- ME_CONFIG_MONGODB_SERVER=mongodb
volumes:
data: {}
networks:
default:
name: mongodb_network
I solved my issue by migrating my data to Atlas cloud.mongodb.com
Answer update: To provide more info as suggested by #nuhkoca, This is the video tutorial to create a mongodb atlas: https://www.youtube.com/watch?v=xrc7dIO_tXk&t=15s And this is the link to the db from my resources file in my Springboot backend api:
server.port=<Port_number>
spring.data.mongodb.uri=<Link_to_mongodb_atlas>

Intergrate elasticsearch with multiple mongodb in docker-compose

I have a implemented a microservice architecture with several servers and databases. I have installed elasticsearch with docker and when I do docker-compose up, everything seems to run fine.
However I would like to integrate the elasticsearch with the several databases (2 mongodb in this sample below) in the system. How do I synch the two mongodb in two different containers with elasticsearch so that I can search them?
client:
container_name: client
stdin_open: true
build:
context: ./client
dockerfile: Dockerfile
restart: always
volumes:
- './client:/app'
ports:
- '1000:3000'
environment:
- NODE_ENV=development
- CHOKIDAR_USEPOLLING=true
weatherdb:
container_name: weather-db
image: mongo
restart: always
ports:
- '2002:27017'
volumes:
- ./weather_service/weather_db:/data/db
networks:
- backend
weather-service:
container_name: weather-service
build: ./weather_service
restart: always
ports:
- "1002:3000"
depends_on:
- weatherdb
links:
- elasticsearch
networks:
- backend
newsdb:
container_name: news-db
image: mongo
restart: always
ports:
- '2003:27017'
volumes:
- ./news_service/news_db:/data/db
networks:
- backend
news-service:
container_name: news-service
build: ./news_service
restart: always
ports:
- "1003:3000"
depends_on:
- newsdb
links:
- elasticsearch
networks:
- backend
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0
container_name: elasticsearch
restart: always
ports:
- 9200:9200
- 9300:9300
environment:
ES_JAVA_OPTS: '-Xms512m -Xmx512m'
network.bind_host: 0.0.0.0
network.host: 0.0.0.0
discovery.type: single-node
volumes:
- ./elasticsearch/esdata:/usr/share/elasticsearch/data
networks:
- backend
Its very simple to just add a elasticsearch docker section in any docker-compose file and start it, all these are independent docker containers and as long as their exposed port on host is not interfering each other and you have the correct configuration in place it should work.
Please refer elasticsearch multi-docker installation using docker file for more info.
NOTE: You have not mentioned what exact issue you are facing, you have mentioned everything ie all docker containers are running file, so please explain in detail what exactly you are trying to solve

AdminMongo with docker-compose doesn't work

I'm trying to access to my mongo database on docker with adminmongo.
Here's my docker-compose.yml
version: '3'
services:
mongo:
image: mongo
volumes:
- ~/data:/data/db
restart: always
expose:
- 6016
adminmongo:
image: mrvautin/adminmongo
expose:
- 1234
links:
- mongo:mongo
When i do a docker-compose up everything works fine, adminmongo also return me this : adminmongo_1_544d9a6f954c | adminMongo listening on host: http://localhost:1234
But when i go to localhost:1234 my navigator is telling me this page doesn't exist.
Here's what a docker ps return me :
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9c27d4a89254 mrvautin/adminmongo "/bin/sh -c 'node ap…" 38 seconds ago Up 33 seconds 1234/tcp iris_adminmongo_1_544d9a6f954c
2a7496a8c56a mongo "docker-entrypoint.s…" 40 minutes ago Up 38 seconds 6016/tcp, 27017/tcp iris_mongo_1_7f00356a3adc
I've found 2 issues here:
1st: Exposing a port is not enough. expose is just documentation, you need to publish (bind) a port to the host to be reachable. This is how it's done:
ports:
- 1234:1234
2nd: you have to configure adminmongo to listen to 0.0.0.0 because by default it starts listening on 127.0.0.1 and this makes it accessible only inside the container itself. From the documentation page you've included in your question, the Configuration section states that this can be done by passing an environment variable:
All above parameters are usable through the environment which makes it very handy to when using adminMongo as a docker container! just run docker run -e HOST=yourchoice -e PORT=1234 ...
Since you are using docker-compose, this is done by the following:
environment:
- HOST=0.0.0.0
Working example:
version: '3'
services:
mongo:
image: mongo
volumes:
- ~/data:/data/db
restart: always
expose:
- 6016
adminmongo:
image: mrvautin/adminmongo
ports:
- 1234:1234
environment:
- HOST=0.0.0.0
Example of docker-compose works :
version: '3'
services:
server:
container_name: docker_api_web_container
image: docker_api_web
build: .
volumes:
- ./src:/usr/src/node-app/src
- ./package.json:/usr/src/node-app/package.json
environment:
- ENV=DEVELOPMENT
- PORT=4010
ports:
- '9000:4010'
depends_on:
- 'mongo'
mongo:
container_name: docker_mongo_container
image: 'mongo'
ports:
- '27017:27017'
adminmongo:
container_name: docker_adminmongo_container
image: mrvautin/adminmongo
links: ['mongo:mongo']
environment:
- HOST=0.0.0.0
ports:
- '1234:1234'
You have to expose your service to the outside world like this:
version: '3'
services:
mongo:
image: mongo
volumes:
- ~/data:/data/db
restart: always
adminmongo:
image: mrvautin/adminmongo
ports:
- 1234:1234
Now you can access your adminmongo by http://localhost:1234.
And you don't have to use links here.Since compose creates a network and joins all services in the compose files. You can access other containers with their service names.

How to connect to localhost postgres database from docker container?

I'm configured my project to docker. I have database that have been used in non-docker period and now I want to connect my docker-compose db service to it. But when I write docker-compose up - existing database not used - new one created instead (I suspect, docker container simply doesn't see the database). If I do nonsense please let me know. Maybe I shoud migrate my server db into container.
Here is my docker-compose.yml:
services:
db:
restart: always
image: postgres:latest
environment:
- POSTGRES_DB=mydb
- POSTGRES_PASSWORD=p#ssw0rd
- POSTGRES_USER=root
ports:
- "5432:5432"
volumes:
# We'll mount the 'postgres-data' volume into the location Postgres stores it's data:
- postgres-data:/var/lib/postgresql/data
web:
build: .
command: bash -c "python manage.py collectstatic --noinput && ./manage.py migrate && ./run_gunicorn.sh"
volumes:
- .:/code
- /static:/static
ports:
- 443:443
depends_on:
- db
nginx:
restart: always
image: nginx:latest
ports:
- 80:80
volumes:
- ./misc/nginx.conf:/etc/nginx/conf.d/default.conf
- /static:/static
depends_on:
- web
I think, the canonic approach is to have your DB engine running in container while storing the data on the persistent storage (map the volume to your hard disk).
So I would use the Postgres in docker as ServerDB, as you suggested.
If you only want your application connect to the external database, declare it as an external host:
version: '2'
services:
web:
build: .
command: bash -c "python manage.py collectstatic --noinput && ./manage.py migrate && ./run_gunicorn.sh"
volumes:
- .:/code
- /static:/static
ports:
- 443:443
extra_hosts:
- "db:192.168.1.2"
nginx:
restart: always
image: nginx:latest
ports:
- 80:80
volumes:
- ./misc/nginx.conf:/etc/nginx/conf.d/default.conf
- /static:/static
depends_on:
- web
Just be sure your application reference the database as db and replace the ip I put there with your host ip.
Regards