I want to connect to MongoDB cluster using
mongodb://localhost:27017
It shows me an error
getaddrinfo ENOTFOUND messenger-mongodb-1
This is my docker-compose.yml file
version: '3'
services:
messenger-mongodb-1:
container_name: messenger-mongodb-1
image: mongo:6.0.3
command: mongod --replSet messenger-mongodb-replica-set --bind_ip_all
ports:
- 27017:27017
networks:
- messenger-mongodb-cluster
volumes:
- messenger-mongodb-1-data:/data/db
depends_on:
- messenger-mongodb-2
- messenger-mongodb-3
healthcheck:
test: test $$(echo "rs.initiate({_id:\"messenger-mongodb-replica-set\",members:[{_id:0,host:\"messenger-mongodb-1\"},{_id:1,host:\"messenger-mongodb-2\"},{_id:2,host:\"messenger-mongodb-3\"}]}).ok || rs.status().ok" | mongo --quiet) -eq 1
interval: 10s
start_period: 30s
messenger-mongodb-2:
container_name: messenger-mongodb-2
image: mongo:6.0.3
command: mongod --replSet messenger-mongodb-replica-set --bind_ip_all
ports:
- 27018:27017
networks:
- messenger-mongodb-cluster
environment:
- MONGO_INITDB_DATABASE=messenger-db
volumes:
- messenger-mongodb-2-data:/data/db
messenger-mongodb-3:
container_name: messenger-mongodb-3
image: mongo:6.0.3
command: mongod --replSet messenger-mongodb-replica-set --bind_ip_all
ports:
- 27019:27017
networks:
- messenger-mongodb-cluster
environment:
- MONGO_INITDB_DATABASE=messenger-db
volumes:
- messenger-mongodb-3-data:/data/db
networks:
messenger-mongodb-cluster:
volumes:
messenger-mongodb-1-data:
messenger-mongodb-2-data:
messenger-mongodb-3-data:
I run it like
docker-compose up -d
How can I connect to my replica set?
I want to use it for the local development of my node.js application
My operating system is Windows 11 Pro
Related
I am using to following docker-compose file to setup my sharded MongoDB which has 3 shards, 3 config servers and 2 router instances:
version: "3.5"
services:
mongorsn1:
container_name: mongors1n1
hostname: mongors1n1
image: mongo
command: mongod --shardsvr --replSet mongors1 --dbpath /data/db --port 27017
ports:
- 27017:27017
expose:
- "27017"
environment:
TERM: xterm
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/data1:/data/db
mongors1n2:
container_name: mongors1n2
hostname: mongors1n2
image: mongo
command: mongod --shardsvr --replSet mongors1 --dbpath /data/db --port 27017
ports:
- 27027:27017
expose:
- "27017"
environment:
TERM: xterm
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/data2:/data/db
mongors1n3:
container_name: mongors1n3
hostname: mongors1n3
image: mongo
command: mongod --shardsvr --replSet mongors1 --dbpath /data/db --port 27017
ports:
- 27037:27017
expose:
- "27017"
environment:
TERM: xterm
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/data3:/data/db
mongocfg1:
container_name: mongocfg1
image: mongo
command: mongod --configsvr --replSet mongors1conf --dbpath /data/db --port 27017
environment:
TERM: xterm
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/config1:/data/db
mongocfg2:
container_name: mongocfg2
image: mongo
command: mongod --configsvr --replSet mongors1conf --dbpath /data/db --port 27017
environment:
TERM: xterm
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/config2:/data/db
mongocfg3:
container_name: mongocfg3
image: mongo
command: mongod --configsvr --replSet mongors1conf --dbpath /data/db --port 27017
environment:
TERM: xterm
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/config3:/data/db
mongos1:
container_name: mongos1
image: mongo
hostname: mongos1
depends_on:
- mongocfg1
- mongocfg2
command: mongos --configdb mongors1conf/mongocfg1:27017,mongocfg2:27017,mongocfg3:27017 --port 27017
ports:
- 27019:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
mongos2:
container_name: mongos2
image: mongo
hostname: mongos2
depends_on:
- mongocfg1
- mongocfg2
command: mongos --configdb mongors1conf/mongocfg1:27017,mongocfg2:27017,mongocfg3:27017 --port 27017
ports:
- 27020:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
As you can see it is up and running correctly.
docker exec -it mongos1 bash -c "echo 'sh.status()' | mongo "
MongoDB shell version v4.4.6
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("f00e3ac7-e114-40b3-b7f8-0e5cf9c56e10") }
MongoDB server version: 4.4.6
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("609e21145937100b3f006e20")
}
shards:
{ "_id" : "mongors1", "host" : "mongors1/mongors1n1:27017,mongors1n2:27017,mongors1n3:27017", "state" : 1 }
active mongoses:
"4.4.6" : 2
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
bye
Now here is the docker-compose file I use to connect Orion and the IoT Agent to my MongoDB shards.
version: "3.5"
services:
orion:
image: atos/smbt/orion:2.5.2
hostname: orion
container_name: orion
restart: unless-stopped
ports:
- "1026:1026"
entrypoint:
- /usr/bin/contextBroker
- -fg
- -multiservice
- -ngsiv1Autocast
- -corsOrigin
- __ALL
command: -dbhost mongors1n1:27017,mongors1n2:27027,mongors1n3:27037 -rplSet mongors1 -logLevel ERROR -noCache
healthcheck:
test: curl --fail -s http://orion:1026/version || exit 1
networks:
- fiware_mongo_sharding_default
iot-agent:
image: atos/smbt/iotagent-ul:1.15.0
hostname: iot-agent
container_name: iot-agent
restart: unless-stopped
expose:
- "4041"
- "7896"
ports:
- "4041:4041"
- "7896:7896"
environment:
- IOTA_CB_HOST=orion
- IOTA_CB_PORT=1026
- IOTA_NORTH_PORT=4041
- IOTA_REGISTRY_TYPE=mongodb
- IOTA_LOG_LEVEL=ERROR
- IOTA_TIMESTAMP=true
- IOTA_CB_NGSI_VERSION=v2
- IOTA_AUTOCAST=true
- IOTA_MONGO_HOST=mongors1n1
- IOTA_MONGO_PORT=27017
- IOTA_MONGO_DB=iotagentul
- IOTA_HTTP_PORT=7896
- IOTA_PROVIDER_URL=http://iot-agent:4041
networks:
- fiware_mongo_sharding_default
networks:
fiware_mongo_sharding_default:
external: true
Orion is connected to MongoDB but with the IoT Agent I get the following error message.
time=2021-05-14T07:51:51.086Z | lvl=ERROR | corr=28202651-30e1-48dc-88a9-53de603e7e6d | trans=28202651-30e1-48dc-88a9-53de603e7e6d | op=IoTAgentNGSI.DbConn | from=n/a | srv=n/a | subsrv=n/a | msg=MONGODB-001: Error trying to connect to MongoDB: MongoNetworkError: failed to connect to server [mongos1:27019] on first connect [MongoNetworkError: connect ECONNREFUSED 172.20.0.9:27019] | comp=IoTAgent
I have tried different things such as:
- IOTA_MONGO_HOST=mongos1
- IOTA_MONGO_PORT=27019
but to no avail unfortunately. Is there something I am doing wrong?
Thanks in advance for your help.
I found the solution myself. What I was doing wrong is that the IoT Agent should be NOT be connected with a Sharded MongoDB as it only expects one Mongo instance. That instance is used to persist and retrieve IoT device created when you provision a sensor via the IoT Agent. Device measurement are sent to the Orion Context Broket which will persist into the Sharded MongoDB. Hope it can help somebody else.
Below is the docker-compose.yml I have used.
MongoDB container got created, after some 6 to 8 hours of time the databases inside the mongo are getting cleared of automatically.
Even after the manual restart I could not able to get my data back.
I have followed two approaches, and still could not able to figure out what's the issue.
Should I need to declare volumes at the top of the compose file like in approach 1 or provide user specific directory "./data/mongo-1" like in approach 2
Adding the below will it help?
volumes:
mongo_data:
driver: local # not working
external: true # will it work?
Approach-1: Single node replica set
version: '3.8'
volumes:
mongo_data:
mongodb:
hostname: mongodb
container_name: mongodb
image: mongo:latest
environment:
MONGO_INITDB_DATABASE: moviebooking
MONGO_REPLICA_SET_NAME: rs0
volumes:
- ./mongo-initdb.d:/docker-entrypoint-initdb.d
- mongo_data:/data/db
expose:
- 27017
ports:
- "27017:27017"
restart: unless-stopped
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.slaveOk().ok || rs.status().ok" | mongo --quiet) -eq 1
interval: 10s
start_period: 30s
entrypoint: [ "/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0" ]
Approach-2: Multi node replica set
version: '3.8'
services:
mongo1:
image: mongo:4.2
container_name: mongo1
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "27017"]
volumes:
- ./data/mongo-1:/data/db
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate({_id:'my-replica-set',members:[{_id:0,host:\"mongo1:27017\"},{_id:1,host:\"mongo2:27018\"},{_id:2,host:\"mongo3:27019\"}]}).ok || rs.status().ok" | mongo --port 27017 --quiet) -eq 1
interval: 10s
start_period: 30s
mongo2:
image: mongo:4.2
container_name: mongo2
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "27018"]
volumes:
- ./data/mongo-2:/data/db
ports:
- 27018:27018
mongo3:
image: mongo:4.2
container_name: mongo3
command: ["--replSet", "my-replica-set", "--bind_ip_all", "--port", "27019"]
volumes:
- ./data/mongo-3:/data/db
ports:
- 27019:27019
In my case , db was hacked by some one.
Adding volume mount will work, making it as external in compose will ask you to create a volume externally before running the compose file. Which I feel is a safe way instead of asking compose to decide the volume location.
you are welcome :)
I have tried to run mongodb replicaSet in local with mongoldb-community in my Mac I follow mongodb doc I can run it by this command
mongod --port 27017 --dbpath /usr/local/var/mongodb --replSet rs0 --bind_ip localhost,127.0.0.1
but it doesn't run on background, so every time I want to start replica set mongodb I should run that command, before I run it I should stop mongo first, then on the next tab console I should run mongo --eval "rs.initiate()" to create to replicaSet again
here is my docker compose:
version: "3.7"
services:
mongodb_container:
image: mongo:latest
ports:
- 27017:27017
volumes:
- mongodb_data_container:/data/db
volumes:
mongodb_data_container:
how to convert that into docker-compose ? is it possible ?
can I do docker exec CONTAINER_ID [commands] ? to run command mongo like above , but must stop the mongodb run in that docker ?
You can have a mongodb replica-set with this docker-compose services:
mongodb-primary:
image: "bitnami/mongodb:4.2"
user: root
volumes:
- ./mongodb-persistence/bitnami:/bitnami
networks:
- parse_network
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_REPLICA_SET_KEY=123456789
- MONGODB_ROOT_USERNAME=admin-123
- MONGODB_ROOT_PASSWORD=password-123
- MONGODB_USERNAME=admin-123
- MONGODB_PASSWORD=password-123
- MONGODB_DATABASE=my_database
ports:
- 27017:27017
mongodb-secondary:
image: "bitnami/mongodb:4.2"
depends_on:
- mongodb-primary
environment:
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_REPLICA_SET_KEY=123456789
- MONGODB_PRIMARY_HOST=mongodb-primary
- MONGODB_PRIMARY_PORT_NUMBER=27017
- MONGODB_PRIMARY_ROOT_USERNAME=admin-123
- MONGODB_PRIMARY_ROOT_PASSWORD=password-123
networks:
- parse_network
ports:
- 27027:27017
mongodb-arbiter:
image: "bitnami/mongodb:4.2"
depends_on:
- mongodb-primary
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-arbiter
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_HOST=mongodb-primary
- MONGODB_PRIMARY_PORT_NUMBER=27017
- MONGODB_PRIMARY_ROOT_PASSWORD=password-123
- MONGODB_REPLICA_SET_KEY=123456789
networks:
- parse_network
ports:
- 27037:27017
networks:
parse_network:
driver: bridge
ipam:
driver: default
volumes:
mongodb_master_data:
driver: local
I have been trying to dockerize my spring boot application which depends on redis, kafka and mongodb.
Following is the docker-compose.yml:
version: '3.3'
services:
my-service:
image: my-service
build:
context: ../../
dockerfile: Dockerfile
restart: always
container_name: my-service
environment:
KAFKA_CONFLUENT_BOOTSTRAP_SERVERS: kafka:9092
MONGO_HOSTS: mongodb:27017
REDIS_HOST: redis
REDIS_PORT: 6379
volumes:
- /private/var/log/my-service/:/var/log/my-service/
ports:
- 8080:8090
- 1053:1053
depends_on:
- redis
- kafka
- mongodb
portainer:
image: portainer/portainer
command: -H unix:///var/run/docker.sock
restart: always
container_name: portainer
ports:
- 9000:9000
- 9001:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
redis:
image: redis
container_name: redis
restart: always
ports:
- 6379:6379
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookeeper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
container_name: kafka
environment:
KAFKA_CREATE_TOPICS: "cms.entity.change:1:1" # topic:partition:replicas
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_PORT: 9092
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- "zookeeper"
mongodb:
image: mongo:latest
container_name: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME:
MONGO_INITDB_ROOT_PASSWORD:
ports:
- 27017:27017
volumes:
- ./data/db:/data/db
The issue is that this starts up mongo as a STANDALONE instance. So the APIs in my service that persist data are failing as mongo needs to start as a REPLICA_SET.
How can I edit my docker-compose file to start mongo as a REPLICA_SET?
I had the same issue and ended up on this stackoverflow post.
We had a requirement of using official mongoDB docker image (https://hub.docker.com/_/mongo ) and couldn't use bitnami as suggested in Vahid's answer.
This answer isn't exactly what's needed by the question asked and coming in 6 months too late; but it should give directions to someone who need to use the mongoDb standalone replicaset throw away instance for integration testing purpose. If you need to use it in PROD then you'll have to provide environment variables for volumes and auth as per Vahid's answer.
version: '3.7'
services:
mongodb:
image: mongo:latest
container_name: myservice-mongodb
networks:
- myServiceNetwork
expose:
- 27017
command: --replSet singleNodeReplSet
mongodb-replicaset:
container_name: mongodb-replicaset-helper
depends_on:
- mongodb
networks:
- myServiceNetwork
image: mongo:latest
command: bash -c "sleep 5 && mongo --host myservice-mongodb --port 27017 --eval \"rs.initiate()\" && sleep 2 && mongo --host myservice-mongodb --port 27017 --eval \"rs.status()\" && sleep infinity"
my-service:
depends_on:
- mongodb-replicaset
image: myserviceimage
container_name: myservicecontainer
networks:
- myServiceNetwork
environment:
myservice__Database__ConnectionString: mongodb://myservice-mongodb:27017/?connect=direct&replicaSet=singleNodeReplSet&readPreference=primary
myservice__Database__Name: myserviceDb
networks:
myServiceNetwork:
driver: bridge
NOTE: Please look at the way how connection string is passed as env variable to the service depending on mongo replicaset instance. You'd have to ensure that the name used in setting up the mongodb replicaset (in my case singleNodeReplicaSet) is passed on to the service depending on it.
Edited:
my previous answer was far wrong so I changed it. I managed to make it work using 'bitnami/mongodb:4.0'. Not sure if that would help you or not, but maybe it gives you some idea. They have a docker-compose file ready for replicaset mode.
version: '3'
services:
mdb-primary:
image: 'bitnami/mongodb:4.0'
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-primary
mdb-secondary:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-secondary
mdb-arbiter:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-arbiter
mongo-cli:
image: 'bitnami/mongodb:latest'
don't forget to add volumes and map it to /bitnami on the primary node
the last container, mongo-cli is for testing purposes. So you can connect to the replicaset using the cli, there is an argument about that here if you like to read about it.
$ docker-compose exec mongo-cli bash
$ mongo "mongodb://mdb-primary:27017/test?replicaSet=replicaset"
I'm running this docker-compose.yml on my mac over a fresh Docker for Mac environment.
Gist here!
version: '2'
services:
replica1:
image: mongo:3.0
container_name: mongo1
ports:
- "27017:27017"
volumes:
- ./mongodata/replica1:/data/db
command: mongod --smallfiles --replSet "mrmtx"
networks:
- mongo_cluster
replica2:
image: mongo:3.0
container_name: mongo2
ports:
- "27017:27017"
volumes:
- ./mongodata/replica2:/data/db
command: mongod --smallfiles --replSet "mrmtx"
networks:
- mongo_cluster
replica3:
image: mongo:3.0
container_name: mongo3
ports:
- "27017:27017"
volumes:
- ./mongodata/replica3:/data/db
command: mongod --smallfiles --replSet "mrmtx"
networks:
- mongo_cluster
networks:
mongo_cluster:
driver: overlay
I'm getting this error:
Cannot start service replica2: b'Could not attach to network mongo-rplset_mongo_cluster: rpc error: code = PermissionDenied desc = network mongo-rplset_mongo_cluster not manually attachable'