docker-compose MongoDB replica set data loss after some period of time - mongodb

Below is the docker-compose.yml I have used.
MongoDB container got created, after some 6 to 8 hours of time the databases inside the mongo are getting cleared of automatically.
Even after the manual restart I could not able to get my data back.
I have followed two approaches, and still could not able to figure out what's the issue.
Should I need to declare volumes at the top of the compose file like in approach 1 or provide user specific directory "./data/mongo-1" like in approach 2
Adding the below will it help?
volumes:
mongo_data:
driver: local # not working
external: true # will it work?
Approach-1: Single node replica set
version: '3.8'
volumes:
mongo_data:
mongodb:
hostname: mongodb
container_name: mongodb
image: mongo:latest
environment:
MONGO_INITDB_DATABASE: moviebooking
MONGO_REPLICA_SET_NAME: rs0
volumes:
- ./mongo-initdb.d:/docker-entrypoint-initdb.d
- mongo_data:/data/db
expose:
- 27017
ports:
- "27017:27017"
restart: unless-stopped
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.slaveOk().ok || rs.status().ok" | mongo --quiet) -eq 1
interval: 10s
start_period: 30s
entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
Approach-2: Multi node replica set
version: '3.8'
services:
mongo1:
image: mongo:4.2
container_name: mongo1
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "27017"]
volumes:
- ./data/mongo-1:/data/db
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate({_id:'my-replica-set',members:[{_id:0,host:\"mongo1:27017\"},{_id:1,host:\"mongo2:27018\"},{_id:2,host:\"mongo3:27019\"}]}).ok || rs.status().ok" | mongo --port 27017 --quiet) -eq 1
interval: 10s
start_period: 30s
mongo2:
image: mongo:4.2
container_name: mongo2
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "27018"]
volumes:
- ./data/mongo-2:/data/db
ports:
- 27018:27018
mongo3:
image: mongo:4.2
container_name: mongo3
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "27019"]
volumes:
- ./data/mongo-3:/data/db
ports:
- 27019:27019

In my case , db was hacked by some one.
Adding volume mount will work, making it as external in compose will ask you to create a volume externally before running the compose file. Which I feel is a safe way instead of asking compose to decide the volume location.
you are welcome :)

Related

How to connect to MongoDB replica set?

I want to connect to MongoDB cluster using
mongodb://localhost:27017
It shows me an error
getaddrinfo ENOTFOUND messenger-mongodb-1
This is my docker-compose.yml file
version: '3'
services:
messenger-mongodb-1:
container_name: messenger-mongodb-1
image: mongo:6.0.3
command: mongod --replSet messenger-mongodb-replica-set --bind_ip_all
ports:
- 27017:27017
networks:
- messenger-mongodb-cluster
volumes:
- messenger-mongodb-1-data:/data/db
depends_on:
- messenger-mongodb-2
- messenger-mongodb-3
healthcheck:
test: test $$(echo "rs.initiate({_id:\"messenger-mongodb-replica-set\",members:[{_id:0,host:\"messenger-mongodb-1\"},{_id:1,host:\"messenger-mongodb-2\"},{_id:2,host:\"messenger-mongodb-3\"}]}).ok || rs.status().ok" | mongo --quiet) -eq 1
interval: 10s
start_period: 30s
messenger-mongodb-2:
container_name: messenger-mongodb-2
image: mongo:6.0.3
command: mongod --replSet messenger-mongodb-replica-set --bind_ip_all
ports:
- 27018:27017
networks:
- messenger-mongodb-cluster
environment:
- MONGO_INITDB_DATABASE=messenger-db
volumes:
- messenger-mongodb-2-data:/data/db
messenger-mongodb-3:
container_name: messenger-mongodb-3
image: mongo:6.0.3
command: mongod --replSet messenger-mongodb-replica-set --bind_ip_all
ports:
- 27019:27017
networks:
- messenger-mongodb-cluster
environment:
- MONGO_INITDB_DATABASE=messenger-db
volumes:
- messenger-mongodb-3-data:/data/db
networks:
messenger-mongodb-cluster:
volumes:
messenger-mongodb-1-data:
messenger-mongodb-2-data:
messenger-mongodb-3-data:
I run it like
docker-compose up -d
How can I connect to my replica set?
I want to use it for the local development of my node.js application
My operating system is Windows 11 Pro

MongoDB replicaSet on three different machines

I have the following Docker Compose file, and I wish to persist data on three machines (different hostname).
The file as-is works OK, but on the same machine (it creates one primary and two replicas).
My questions:
How can I set the Docker Compose file up in such way that the primary is on hostname primary-1.com and the replicas are on backup-1.com and backup-2.com.
Should this docker-compose run on each hostname?
My docker-compose file:
version: "3.8"
services:
mongo1:
image: mongo:5
container_name: mongo1
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "30001"]
volumes:
- ./data/mongo-1:/data/db
ports:
- 30001:30001
healthcheck:
test: test $$(echo "rs.initiate({_id:'my-replica-set',members:[{_id:0,host:\"mongo1:30001\"},{_id:1,host:\"mongo2:30002\"},{_id:2,host:\"mongo3:30003\"}]}).ok || rs.status().ok" | mongo --port 30001 --quiet) -eq 1
interval: 10s
start_period: 30s
mongo2:
image: mongo:5
container_name: mongo2
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "30002"]
volumes:
- ./data/mongo-2:/data/db
ports:
- 30002:30002
mongo3:
image: mongo:5
container_name: mongo3
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "30003"]
volumes:
- ./data/mongo-3:/data/db
ports:
- 30003:30003

How to use data from MongoDB in docker container on Typesense in the same container?

Description
I have a container that has a mongodb, mongo-express and typesense. I need to set that typesense use data from the mongodb inside the container. How can i do this?
I already used and configured the replica set.
Steps to reproduce
This is my docker compose file:
version: "3.9"
services:
database:
image : 'mongo'
container_name: 'container-name'
environment:
- PUID=1000
- PGID=1000
volumes:
- ./database:/data/db
entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
ports:
- 27017:27017
restart: unless-stopped
mongo-express:
image: mongo-express:0.54
ports:
- "8081:8081"
environment:
ME_CONFIG_MONGODB_SERVER: database
depends_on:
- database
typesense:
image: typesense/typesense:0.21.0
ports:
- "8108:8108"
environment:
TYPESENSE_API_KEY: xyz
TYPESENSE_DATA_DIR: /data/typesense
TYPESENSE_ENABLE_CORS: "true"
volumes:
- ./typesense:/data/typesense
When i call the API colletions/data, i get a response and on the end i see num_documents: 0
Github answer

How to connect to a mongoDb replica created with Docker compose from mongo express and mongo-Compass

Problem:
I have created a mongo Db replica set like this using docker-compose.
questionsdb:
image: mongo:latest
container_name: questionsdb
hostname: questionsdb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=mi1234
volumes:
- mongo_data:/data/dbadmin
ports:
- 27017:27017
- 9229:9229
entrypoint: [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"]
#networks:
# - custom-
questionsdb1:
image: mongo:latest
container_name: questionsdb1
hostname: questionsdb1
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=mi1234
volumes:
- mongo_data1:/data/dbadmin
ports:
- 27018:27017
- 9230:9229
entrypoint: [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"]
#networks:
# - custom-network
questionsdb2:
image: mongo:latest
container_name: questionsdb2
hostname: questionsdb2
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=mi1234
volumes:
- mongo_data2:/data/dbadmin
expose:
- "27017"
ports:
- 27019:27017
- 9231:9229
entrypoint: [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"]
#networks:
# - custom-network
And my mongo-express container configuration is like this.
mongo-express:
image: mongo-express
container_name: mongo-express
restart: always
ports:
- 8111:8081
environment:
- ME_CONFIG_MONGODB_SERVER=questionsdb
- ME_CONFIG_MONGODB_ADMINUSERNAME=admin
- ME_CONFIG_MONGODB_ADMINPASSWORD=mi1234
- ME_CONFIG_BASICAUTH_USERNAME=admin#mi.com
- ME_CONFIG_BASICAUTH_PASSWORD=mi#1234
#networks:
# - custom-network
And I try to connect to it through mongo Compass like this.
mongodb://admin:mi1234#localhost:27017,localhost:27018,localhost:27019/admin?replicaSet=rsmongo
But both mongo-express and mongo compass was failed with giving me authentication failed.
This is the error I can see in the docker container.
{"t":{"$date":"2021-08-16T01:57:15.391+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn219","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"admin","authenticationDatabase":"admin","remote":"172.18.0.15:59638","extraInfo":{},"error":"UserNotFound: Could not find user \"admin\" for db \"admin\""}}
Can someone help me to solve this. I tried a lot to find out a solution to this but I was unable to do so. Thank you
According to the error message no admin user has been created in your DB during the container startup.
This could be explained by your custom entrypoint [ "/usr/bin/mongod", "--replSet", "rsmongo", "--bind_ip_all"] that overrides the official one. Whereas the root user creation work seems to be achieved by this script.
Try to execute your docker-compose stack with only one mongo instance and without the entrypoint statement.

Docker: Wait for the Mongodb ReplicaSet

everyone. I have an odd problem (who hasn't?)
I have this docker-compose file:
version: '3.4'
services:
ludustack-web:
container_name: ludustack-web
image: ${DOCKER_REGISTRY-}ludustack-web
build:
context: .
dockerfile: LuduStack.Web/Dockerfile
networks:
- ludustack-network
ports:
- '80:80'
- '443:443'
depends_on:
- 'ludustack-db'
ludustack-db:
container_name: ludustack-db
command: mongod --auth
image: mongo:latest
hostname: mongodb
networks:
- ludustack-network
ports:
- '27017:27017'
env_file:
- .env
environment:
- MONGO_INITDB_ROOT_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
- MONGO_INITDB_DATABASE=${MONGO_INITDB_DATABASE}
- MONGO_REPLICA_SET_NAME=${MONGO_REPLICA_SET_NAME}
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 60s
start_period: 60s
command: ["--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
networks:
ludustack-network:
driver: bridge
The problem is the web application only waits for the mongodb container to be ready, not the replica set itself. So, when the application starts, it crashes because the replicaset is not ready yet. Right after the crash, it logs the replicaset continuing its job:
Any tips on how to make the web application wait the replicaset to be ready?
The application did wait, for 30 seconds. You can increase the timeout by adjusting serverSelectionTimeoutMS URI option or through language-specific means.