Docker with MongoDB (3.4) Replica Shard - mongodb

Using docker-compose to bring up a development mongo sharded cluster. Development because each replica set and the config are single instances:
Here is the docker-compose file:
version: '2'
services:
rs1_n1:
image: mongo:3.4
command: mongod --shardsvr --replSet rs1
rs2_n1:
image: mongo:3.4
command: mongod --shardsvr --replSet rs2
config_n1:
image: mongo:3.4
command: mongod --configsvr --dbpath /data/db
mongos_n1:
image: mongo:3.4
command: mongos --configdb rs1/config_n1:27017,
No issue bringing up the replica sets and the config. The router (mongos) is failing with: Failed global initialization: FailedToParse: Empty host component parsing HostAndPort from ""
As far as I can tell from the docs, I am passing the <replicaset_name>/ip:port argument correctly. Is some additional information necessary because I am doing all of this within docker-compose?

default configsvr port is 27019.
try this instead:
version: '2'
services:
rs1_n1:
image: mongo:3.4
command: mongod --shardsvr --replSet rs1
rs2_n1:
image: mongo:3.4
command: mongod --shardsvr --replSet rs2
config_n1:
image: mongo:3.4
command: mongod --configsvr --dbpath /data/db
mongos_n1:
image: mongo:3.4
command: mongos --configdb rs1/config_n1:27019
or
version: '2'
services:
rs1_n1:
image: mongo:3.4
command: mongod --shardsvr --replSet rs1
rs2_n1:
image: mongo:3.4
command: mongod --shardsvr --replSet rs2
config_n1:
image: mongo:3.4
command: mongod --configsvr --dbpath /data/db --port 27017
mongos_n1:
image: mongo:3.4
command: mongos --configdb rs1/config_n1:27017
You may also be interested in the following image khezen/mongo

Related

sharding mogodb using docker container :docker-compose MongoDB Error: network error while attempting to run command 'isMaster' on host

I am trying sharding mongodbs. I have replicated the config server and also shards.But while connecting using router I get "Error: network error while attempting to run command 'isMaster' on host '127.0.0.1:60001'". I tried this but still getting same error. I am new to mongodb
I am using docker containers for each mongo instance
Here is .yaml file
""""
version: '3'
services:
cfgsvr1:
container_name: cfgsvr1
image: mongo
command: mongod --configsvr --replSet cfgrs --port 27017 --dbpath /data/db
ports:
- 40001:27017
volumes:
- cfgsvr1:/data/db
cfgsvr2:
container_name: cfgsvr2
image: mongo
command: mongod --configsvr --replSet cfgrs --port 27017 --dbpath /data/db
ports:
- 40002:27017
volumes:
- cfgsvr2:/data/db
cfgsvr3:
container_name: cfgsvr3
image: mongo
command: mongod --configsvr --replSet cfgrs --port 27017 --dbpath /data/db
ports:
- 40003:27017
volumes:
- cfgsvr3:/data/db
volumes:
cfgsvr1: {}
cfgsvr2: {}
cfgsvr3: {}
version: '3'
services:
mongos:
container_name: mongos
image: mongo
command: mongos --bind_ip_all --port 27017 --configdb cfgrs/cfgsvr3:27017,cfgsvr2:27017,cfgsvr1:27017
ports:
- 60001:27017
version: '3'
services:
shard1svr1:
container_name: shard1svr1
image: mongo
command: mongod --shardsvr --replSet shard1rs --port 27017 --dbpath /data/db
ports:
- 50001:27017
volumes:
- shard1svr1:/data/db
shard1svr2:
container_name: shard1svr2
image: mongo
command: mongod --shardsvr --replSet shard1rs --port 27017 --dbpath /data/db
ports:
- 50002:27017
volumes:
- shard1svr2:/data/db
shard1svr3:
container_name: shard1svr3
image: mongo
command: mongod --shardsvr --replSet shard1rs --port 27017 --dbpath /data/db
ports:
- 50003:27017
volumes:
- shard1svr3:/data/db
volumes:
shard1svr1: {}
shard1svr2: {}
shard1svr3: {}
version: '3'
services:
shard2svr1:
container_name: shard2svr1
image: mongo
command: mongod --shardsvr --replSet shard2rs --port 27017 --dbpath /data/db
ports:
- 50004:27017
volumes:
- shard2svr1:/data/db
shard2svr2:
container_name: shard2svr2
image: mongo
command: mongod --shardsvr --replSet shard2rs --port 27017 --dbpath /data/db
ports:
- 50005:27017
volumes:
- shard2svr2:/data/db
shard2svr3:
container_name: shard2svr3
image: mongo
command: mongod --shardsvr --replSet shard2rs --port 27017 --dbpath /data/db
ports:
- 50006:27017
volumes:
- shard2svr3:/data/db
volumes:
shard2svr1: {}
shard2svr2: {}
shard2svr3: {}
""""
my ubuntu version is 18.04
my dbversion is 3.6.3
my shell version is 3.6.3
logs
mongos | 2019-12-19T09:14:24.527+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cfgsvr3:27017
mongos | 2019-12-19T09:14:24.527+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cfgsvr1:27017
mongos | 2019-12-19T09:14:24.536+0000 W NETWORK [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set cfgrs
mongos | 2019-12-19T09:14:25.027+0000 I CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cfgsvr2:27017
mongos | 2019-12-19T09:14:25.033+0000 W NETWORK [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set cfgrs
mongos | 2019-12-19T09:14:25.539+0000 W NETWORK [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set cfgrs
Found error as I was replicating config server without " configsvr: true,"
and
sh.addShard("shard_replica/shard1,shard2,shard3")
sh.addShard("shard_replica2/shard4,shard5,shard6")
version: '2'
services:
shard1:
container_name: shard1
image: mongo
command: mongod --shardsvr --replSet shard_replica --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
ports:
- 40001:27017
environment:
TERM: xterm
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/data1:/data/db
shard2:
container_name: shard2
image: mongo
command: mongod --shardsvr --replSet shard_replica --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 40002:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/data2:/data/db
shard3:
container_name: shard3
image: mongo
command: mongod --shardsvr --replSet shard_replica --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 40003:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/data3:/data/db
cfg1:
container_name: cfg1
image: mongo
command: mongod --configsvr --replSet conf_replica --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 50001:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/config1:/data/db
cfg2:
container_name: cfg2
image: mongo
command: mongod --configsvr --replSet conf_replica --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
ports:
- 50002:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/config2:/data/db
cfg3:
container_name: cfg3
image: mongo
command: mongod --configsvr --replSet conf_replica --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 50003:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/config3:/data/db
shard4:
container_name: shard4
image: mongo
command: mongod --shardsvr --replSet shard_replica2 --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
ports:
- 30001:27017
environment:
TERM: xterm
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/data4:/data/db
shard5:
container_name: shard5
image: mongo
command: mongod --shardsvr --replSet shard_replica2 --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 30002:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/data5:/data/db
shard6:
container_name: shard6
image: mongo
command: mongod --shardsvr --replSet shard_replica2 --dbpath /data/db --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 30003:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro
- /home/Chinmay/Documents/Chinmay/learn-mongodb/new/data6:/data/db
mongos:
container_name: mongos
image: mongo
depends_on:
- cfg1
- cfg2
- cfg3
- shard1
- shard2
- shard3
- shard4
- shard5
- shard6
command: mongos --configdb conf_replica/cfg1:27017,cfg2:27017,cfg3:27017 --port 27017 --bind_ip 0.0.0.0
environment:
TERM: xterm
ports:
- 27019:27017
expose:
- "27017"
volumes:
- /etc/localtime:/etc/localtime:ro

Mongodb: Deploy a minimal mongodb cluster into docker swarm

I'm trying to deploy a minimal mongodb cluster into my development swarm environment like this:
Up to now:
version: "3.3"
networks:
net:
driver: overlay
services:
data1:
image: mongo:3.6
container_name: data1
command: mongod --shardsvr --replSet datars --smallfiles --port 27017
expose:
- 27017
networks:
- net
cfg1:
image: mongo:3.6
container_name: cfg1
command: mongod --configsvr --replSet cfgrs --smallfiles --port 27017
expose:
- 27017
networks:
- net
mongos1:
image: mongo:3.4
container_name: mongos1
command: mongos --configdb cfgrs/cfg1:27017
expose:
- 27017
networks:
- net
Into my config server, I'm getting these message:
2019-09-06T09:22:15.693+0000 I SHARDING [shard registry reload] Periodic reload of shard registry failed :: caused by :: 134 could not get updated shard list from config server due to Read concern majority reads are currently not possible.; will retry after 30s,
Any ideas?

can't access docker-compose volume after restart my service systemd

I create docker-compose with two services: python api and mongodb, after starting first-time sudo docker-compose up it creates a file for mongo db inside \data, after sudo docker-compose down and again sudo docker-compose up mongo can't access files in \data.
Obviously something wrong with permissions, but I don't know what exactly.
But if I just stop containers without sudo docker-compose down and sudo docker-compose up again everything is ok.
So everything goes wrong after down
systemd service:
[Unit]
Description=RestAPI imp
Requires=docker.service
After=docker.service
[Service]
Type=simple
WorkingDirectory=/home/entrant/myserv
ExecStart=/usr/local/bin/docker-compose up
ExecStop=/usr/local/bin/docker-compose down
Restart=on-failure
RestartSec=10
KillMode=process
[Install]
WantedBy=multi-user.target
docker-compose.yml
version: '3.5'
services:
web_dev:
build: .
ports:
- "8080:8080"
volumes:
- .:/app
environment:
- ENV=PROD
- DATABASE_URL=mongodb://mongodb:27017/myserv?authSource=admin&replicaSet=myrepl
depends_on:
- mongodb
command: deploy/wait-for-it.sh mongodb:27017 -- gunicorn -b 0.0.0.0:8080 index:api.app -w 9
mongodb:
image: mongo:4.0.12-xenial
container_name: "mongodb"
environment:
- MONGO_INITDB_DATABASE=myserv
- MONGO_DATA_DIR=/data/db
- MONGO_LOG_DIR=/dev/null
volumes:
- ./data/db:/data/db
ports:
- 27017:27017
command: bash -c "
mongod --fork --replSet myrepl --bind_ip_all --smallfiles --logpath=/dev/null
&& mongo --eval 'rs.initiate()'
&& mongod --shutdown
&& mongod --replSet myrepl --bind_ip_all --smallfiles --logpath=/dev/null
"
networks:
default:
name: web_dev

docker-compose MongoDB Error: network error while attempting to run command 'isMaster' on host

I`m trying to connect to mongodb in docker container from outside with mongo, but I get the following error.
MongoDB shell version v3.6.5
connecting to: mongodb://127.0.0.1:27017
2018-05-30T14:54:13.950+0200 E QUERY [thread1] Error: network error while attempting to run command 'isMaster' on host '127.0.0.1:27017' :
connect#src/mongo/shell/mongo.js:251:13
#(connect):1:6
exception: connect failed
When I bash into the Container everything is fine.
It was working maybe half a year ago but now it won't.
I think it's a problem with docker-compose, but I don't get it.
The versions of the shell and server are the same.
For testing this is my docker-compose.yaml
version: '3'
networks:
backend:
external:
name: my-mongo-cluster
services:
## Config Servers
config01:
image: mongo
command: mongod --port 27017 --configsvr --replSet configserver --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
config02:
image: mongo
command: mongod --port 27017 --configsvr --replSet configserver --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
config03:
image: mongo
command: mongod --port 27017 --configsvr --replSet configserver --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
## Shards
shard01a:
image: mongo
command: mongod --port 27018 --shardsvr --replSet shard01 --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
shard01b:
image: mongo
command: mongod --port 27018 --shardsvr --replSet shard01 --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
shard02a:
image: mongo
command: mongod --port 27019 --shardsvr --replSet shard02 --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
shard02b:
image: mongo
command: mongod --port 27019 --shardsvr --replSet shard02 --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
shard03a:
image: mongo
command: mongod --port 27020 --shardsvr --replSet shard03 --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
shard03b:
image: mongo
command: mongod --port 27020 --shardsvr --replSet shard03 --noprealloc --smallfiles --oplogSize 16
volumes:
- ./scripts:/scripts
networks:
- backend
## Router
router:
image: mongo
command: mongos --port 27017 --configdb configserver/config01:27017,config02:27017,config03:27017
ports:
- "27017:27017"
volumes:
- ./scripts:/scripts
networks:
- backend
depends_on:
- config01
- config02
- config03
- shard01a
- shard01b
- shard02a
- shard02b
- shard03a
- shard03b
I had this same issue
I resolved it by adding following keyword to mongos: --bind_ip_all
so
change:
mongos --port 27017 --configdb configserver/config01:27017,config02:27017,config03:27017
to:
mongos --bind_ip_all --port 27017 --configdb configserver/config01:27017,config02:27017,config03:27017
I've had a similar issue. In my case a script was first starting up docker and then connecting to it using mongo.exe.
The solution was to introduce a 3 second sleep before mongo.exe connects and starts executing queries. Another workaround was not binding docker ip to specific local ip and allowing access from all ips. However that was not an option for me.

Cannot reach any nodes for set mongo config server

I want to deploy mongo cluster containing 1 mongo shard (replica-set), 1 configserver (replica-set) and 1 router via docker-compose. I have a problem that it's "Unable to reach primary for set mongoconfigserver".
I found an issue about it:
Unable to launch mongos, but in that case there was no replica-set for config-server, while I have required "--replSet mongoconfigserver" option in commands.
Here is my docker-compose.yml:
version: '3'
services:
# Config servers for metadata
mongoconfigserver-01:
container_name: mongoconfigserver-01
image: mongo
command: mongod --configsvr --replSet mongoconfigserver --dbpath /data/db --port 27017
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/config1:/data/db
mongoconfigserver-02:
container_name: mongoconfigserver-02
image: mongo
command: mongod --configsvr --replSet mongoconfigserver --dbpath /data/db --port 27017
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/config2:/data/db
mongoconfigserver-03:
container_name: mongoconfigserver-03
image: mongo
command: mongod --configsvr --replSet mongoconfigserver --dbpath /data/db --port 27017
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/config3:/data/db
# First shard
mongo-shard-01a:
image: mongo
command: mongod --port 27018 --replSet mongo-shard-01 --dbpath /data/db
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/data1:/data/db
ports:
- 27017:27017
mongo-shard-01b:
image: mongo
command: mongod --port 27018 --replSet mongo-shard-01 --dbpath /data/db
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/data2:/data/db
ports:
- 27027:27017
mongo-shard-01c:
image: mongo
command: mongod --port 27018 --replSet mongo-shard-01 --dbpath /data/db
volumes:
- /etc/localtime:/etc/localtime:ro
- /mongo_cluster/data3:/data/db
ports:
- 27037:27017
# Mongo router
mongo-router-01:
container_name: mongo-router-01
image: mongo
depends_on:
- mongoconfigserver-01
- mongoconfigserver-02
- mongoconfigserver-03
- mongo-shard-01a
- mongo-shard-01b
- mongo-shard-01c
command: mongos --configdb mongoconfigserver/mongoconfigserver-01:27017,mongoconfigserver-02:27017,mongoconfigserver-03:27017
volumes:
- /etc/localtime:/etc/localtime:ro
and logs:
mongo-router-01 | 2018-03-14T22:46:40.857+0100 I NETWORK [ReplicaSetMonitor-TaskExecutor-0] Successfully connected to mongoconfigserver-02:27017 (1 connections now open to mongoconfigserver-02:27017 with a 5 second timeout)
mongo-router-01 | 2018-03-14T22:46:40.857+0100 I NETWORK [shard registry reload] Successfully connected to mongoconfigserver-03:27017 (1 connections now open to mongoconfigserver-03:27017 with a 5 second timeout)
mongo-router-01 | 2018-03-14T22:46:40.857+0100 I NETWORK [monitoring keys for HMAC] Successfully connected to mongoconfigserver-01:27017 (1 connections now open to mongoconfigserver-01:27017 with a 5 second timeout)
mongo-router-01 | 2018-03-14T22:46:40.858+0100 W NETWORK [monitoring keys for HMAC] Unable to reach primary for set mongoconfigserver
mongo-router-01 | 2018-03-14T22:46:41.359+0100 W NETWORK [mongosMain] Unable to reach primary for set mongoconfigserver
mongo-router-01 | 2018-03-14T22:46:41.359+0100 I NETWORK [mongosMain] Cannot reach any nodes for set mongoconfigserver. Please check network connectivity and the status of the set. This has happened for 2 checks in a row.
mongo-router-01 | 2018-03-14T22:46:41.860+0100 W NETWORK [mongosMain] Unable to reach primary for set mongoconfigserver
Can anybody help with that?
Set the following parameters in the configuration server.
use admin
config = {
_id : "configs",
members : [
{_id : 0, host : "ip:21000" },
{_id : 1, host : "ip:21000" },
{_id : 2, host : "ip:21000" }
]
}
rs.initiate(config);