Implement docker mongodb replica set config settings inside .yml file - mongodb

I have set a MongoDB replica set which is running properly.
But I want to run the config settings inside the .yml, and not initiating inside a replica set node.
by config settings I mean:
1.
config = {
"_id": "comments",
"members": [
{
"_id": 0,
"host": "node1:27017"
},
{
"_id": 1,
"host": "node2:27017"
},
{
"_id": 2,
"host": "node3:27017"
}
]
}
and the below:
2.
rs.initiate(config)

So i figured out a way, hope it helps.Below i will post my .yml file, rsinit file and rs.sh file, all the file should be placed on the same location in order to work, rest all config are written anyways.
.yml file:
version: '3.7'
services:
mongo1:
hostname: mongo1
container_name: localmongo1
image: mongo
expose:
- 27017
ports:
- 27017:27017
restart: always
entrypoint: ["/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0"]
# volumes:
# - /data/db/mongotest:/data/db # This is where your volume will persist. e.g. VOLUME-DIR = ./volumes/mongodb
mongo2:
hostname: mongo2
container_name: localmongo2
image: mongo
expose:
- 27017
ports:
- 27018:27017
restart: always
entrypoint: ["/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0"]
mongo3:
hostname: mongo3
container_name: localmongo3
image: mongo
expose:
- 27017
ports:
- 27019:27017
restart: always
entrypoint: ["/usr/bin/mongod", "--bind_ip_all", "--replSet", "rs0"]
rsinit:
build:
context: .
dockerfile: rsinit
depends_on:
- mongo1
- mongo2
- mongo3
entrypoint: ["sh", "-c", "rs.sh"]
rsinit(normal text file):
FROM mongo
ADD rs.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/rs.sh
rs.sh file:
!/bin/bash
echo "prepare rs initiating"
check_db_status() {
mongo1=$(mongo --host mongo1 --port 27017 --eval "db.stats().ok" | tail -n1 | grep -E '(^|\s)1($|\s)')
mongo2=$(mongo --host mongo2 --port 27017 --eval "db.stats().ok" | tail -n1 | grep -E '(^|\s)1($|\s)')
mongo3=$(mongo --host mongo3 --port 27017 --eval "db.stats().ok" | tail -n1 | grep -E '(^|\s)1($|\s)')
if [[ $mongo1 == 1 ]] && [[ $mongo2 == 1 ]] && [[ $mongo3 == 1 ]]; then
init_rs
else
check_db_status
fi
}
init_rs() {
ret=$(mongo --host mongo1 --port 27017 --eval "rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'mongo1:27017' }, { _id: 1, host: 'mongo2:27017' }, { _id: 2, host: 'mongo3:27017' } ] })" > /dev/null 2>&1)
}
check_db_status > /dev/null 2>&1
echo "rs initiating finished"
exit 0

Related

ECONNREFUSED 127.0.0.1:27017 - Compass <> MongoDB Replication Set connection challenges

I'm testing using docker compose for a replicated set on my local machine and am having trouble with getting Compass connected and functioning with the local set. Everything has deployed fine and I can see the exposed ports listening on the host machine. I just keep getting ECONNREFUSED.
Here is the Docker Compose file.
services:
mongo1:
hostname: mongo1
container_name: mongo1
image: mongo:latest
networks:
- mongo-network
expose:
- 27017
ports:
- 30001:27017
volumes:
- md1:/data/db
restart: always
command: mongod --replSet mongo-net
mongo2:
hostname: mongo2
container_name: mongo2
image: mongo:latest
networks:
- mongo-network
expose:
- 27017
ports:
- 30002:27017
volumes:
- md2:/data/db
restart: always
command: mongod --replSet mongo-net
mongo3:
hostname: mongo3
container_name: mongo3
image: mongo:latest
networks:
- mongo-network
expose:
- 27017
ports:
- 30003:27017
volumes:
- md3:/data/db
restart: always
command: mongod --replSet mongo-net
mongoinit:
image: mongo
restart: "no"
depends_on:
- mongo1
- mongo2
- mongo3
command: >
sh -c "mongosh --host mongo1:27017 --eval
'
db = (new Mongo("localhost:27017")).getDB("test");
config = {
"_id" : "mongo-net",
"members" : [
{
"_id" : 0,
"host" : "localhost:27017",
"priority": 1
},
{
"_id" : 1,
"host" : "localhost:27017",
"priority": 2
},
{
"_id" : 2,
"host" : "localhost:27017",
"priority": 3
}
]
};
rs.initiate(config);
'"
networks:
mongo-network:
volumes:
md1:
md2:
md3:
The containers and replication set deploy fine and are communicating. I can see the defined ports exposed and listening.
My issue is trying to use Compass to connect to the replicated set. I get ECONNREFUSED.
What's odd is I can actually see the client connecting on the primary logs, but get no other information about what the connection was refused/DCed.
{
"attr": {
"connectionCount": 7,
"connectionId": 129,
"remote": "192.168.32.1:61508",
"uuid": "f78c3803-fc7c-4655-9b77-94329bd41a7d"
},
"c": "NETWORK",
"ctx": "listener",
"id": 22943,
"msg": "Connection accepted",
"s": "I",
"t": {
"$date": "2022-11-02T18:57:20.032+00:00"
}
}
{
"attr": {
"client": "conn129",
"doc": {
"application": {
"name": "MongoDB Compass"
},
"driver": {
"name": "nodejs",
"version": "4.8.1"
},
"os": {
"architecture": "arm64",
"name": "darwin",
"type": "Darwin",
"version": "22.1.0"
},
"platform": "Node.js v16.5.0, LE (unified)|Node.js v16.5.0, LE (unified)"
},
"remote": "192.168.32.1:61508"
},
"c": "NETWORK",
"ctx": "conn129",
"id": 51800,
"msg": "client metadata",
"s": "I",
"t": {
"$date": "2022-11-02T18:57:20.034+00:00"
}
}
{
"attr": {
"connectionCount": 6,
"connectionId": 129,
"remote": "192.168.32.1:61508",
"uuid": "f78c3803-fc7c-4655-9b77-94329bd41a7d"
},
"c": "NETWORK",
"ctx": "conn129",
"id": 22944,
"msg": "Connection ended",
"s": "I",
"t": {
"$date": "2022-11-02T18:57:20.044+00:00"
}
}

Cannot connect with Mongo Compass to docerized mongoDB replica sets

I have the following docker-compose.yaml file:
version: '3.8'
services:
mongo_launcher:
container_name: mongo_launcher
image: mongo:5.0.8
restart: on-failure
networks:
- mongo_network
volumes:
- ./docker/mongo-setup.sh:/scripts/mongo-setup.sh
entrypoint: ['sh', '/scripts/mongo-setup.sh']
mongo_replica_1:
container_name: mongo_replica_1
image: mongo:5.0.8
ports:
- 27017:27017
restart: always
entrypoint:
[
'/usr/bin/mongod',
'--bind_ip_all',
'--replSet',
'dbrs',
'--dbpath',
'/data/db',
'--port',
'27017',
]
volumes:
- ./.volumes/mongo/replica1:/data/db
- ./.volumes/mongo/replica1/configdb:/data/configdb
networks:
- mongo_network
mongo_replica_2:
container_name: mongo_replica_2
image: mongo:5.0.8
ports:
- 27018:27018
restart: always
entrypoint:
[
'/usr/bin/mongod',
'--bind_ip_all',
'--replSet',
'dbrs',
'--dbpath',
'/data/db',
'--port',
'27018',
]
volumes:
- ./.volumes/mongo/replica2:/data/db
- ./.volumes/mongo/replica2/configdb:/data/configdb
networks:
- mongo_network
mongo_replica_3:
container_name: mongo_replica_3
image: mongo:5.0.8
ports:
- 27019:27019
restart: always
entrypoint:
[
'/usr/bin/mongod',
'--bind_ip_all',
'--replSet',
'dbrs',
'--dbpath',
'/data/db',
'--port',
'27019',
]
volumes:
- ./.volumes/mongo/replica3:/data/db
- ./.volumes/mongo/replica3/configdb:/data/configdb
networks:
- mongo_network
networks:
mongo_network:
driver: bridge
Note that the first service will use the mongo-setup.sh script:
#!/bin/bash
MONGODB_REPLICA_1=mongo_replica_1
MONGODB_REPLICA_2=mongo_replica_2
MONGODB_REPLICA_3=mongo_replica_3
echo "************ [ Waiting for startup ] **************" ${MONGODB_REPLICA_1}
until curl http://${MONGODB_REPLICA_1}:27017/serverStatus\?text\=1 2>&1 | grep uptime | head -1; do
printf '.'
sleep 1
done
echo "************ [ Startup completed ] **************" ${MONGODB_REPLICA_1}
mongosh --host ${MONGODB_REPLICA_1}:27017 <<EOF
var cfg = {
"_id": "dbrs",
"protocolVersion": 1,
"version": 1,
"members": [
{
"_id": 1,
"host": "${MONGODB_REPLICA_1}:27017",
"priority": 3
},
{
"_id": 2,
"host": "${MONGODB_REPLICA_2}:27018",
"priority": 2
},
{
"_id": 3,
"host": "${MONGODB_REPLICA_3}:27019",
"priority": 1
}
],settings: {chainingAllowed: true}
};
rs.initiate(cfg, { force: true });
rs.reconfig(cfg, { force: true });
rs.secondaryOk();
db.getMongo().setReadPref('nearest');
db.getMongo().setSecondaryOk();
EOF
When I run docker-compose up -d, it succeeds and when I run docker ps I get:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0c8f6d916e4a mongo:5.0.8 "/usr/bin/mongod --b…" 11 seconds ago Up 10 seconds 27017/tcp, 0.0.0.0:27018->27018/tcp mongo_replica_2
c59cb625362e mongo:5.0.8 "/usr/bin/mongod --b…" 11 seconds ago Up 10 seconds 0.0.0.0:27017->27017/tcp mongo_replica_1
cb61093e4dd0 mongo:5.0.8 "/usr/bin/mongod --b…" 11 seconds ago Up 10 seconds 27017/tcp, 0.0.0.0:27019->27019/tcp mongo_replica_3
But when I try to connect one of the replica sets with Mongo Compass I get the following error:
So it seems as the containers are all running fine, why do I get this issue? Please advice..
I could solve the issue by appending the following content to my /etc/hosts file:
127.0.0.1 mongo_replica_1
127.0.0.1 mongo_replica_2
127.0.0.1 mongo_replica_3
Could anyone come up with more elegant way?
I tried to bypass the /etc/hosts solution, so I edited docker-compose.yaml file and I applied extra_hosts:
version: '3.8'
services:
mongo_launcher:
container_name: mongo_launcher
image: mongo:5.0.8
restart: on-failure
networks:
- mongo_network
volumes:
- ./docker/mongo-setup.sh:/scripts/mongo-setup.sh
entrypoint: ['sh', '/scripts/mongo-setup.sh']
mongo_replica_1:
container_name: mongo_replica_1
image: mongo:5.0.8
ports:
- 27017:27017
extra_hosts:
- 'localhost:0.0.0.0'
restart: always
entrypoint:
[
'/usr/bin/mongod',
'--bind_ip_all',
'--replSet',
'dbrs',
'--dbpath',
'/data/db',
'--port',
'27017',
]
volumes:
- ./.volumes/mongo/replica1:/data/db
- ./.volumes/mongo/replica1/configdb:/data/configdb
networks:
- mongo_network
mongo_replica_2:
container_name: mongo_replica_2
image: mongo:5.0.8
ports:
- 27018:27018
extra_hosts:
- 'localhost:0.0.0.0'
restart: always
entrypoint:
[
'/usr/bin/mongod',
'--bind_ip_all',
'--replSet',
'dbrs',
'--dbpath',
'/data/db',
'--port',
'27018',
]
volumes:
- ./.volumes/mongo/replica2:/data/db
- ./.volumes/mongo/replica2/configdb:/data/configdb
networks:
- mongo_network
mongo_replica_3:
container_name: mongo_replica_3
image: mongo:5.0.8
ports:
- 27019:27019
extra_hosts:
- 'localhost:0.0.0.0'
restart: always
entrypoint:
[
'/usr/bin/mongod',
'--bind_ip_all',
'--replSet',
'dbrs',
'--dbpath',
'/data/db',
'--port',
'27019',
]
volumes:
- ./.volumes/mongo/replica3:/data/db
- ./.volumes/mongo/replica3/configdb:/data/configdb
networks:
- mongo_network
networks:
mongo_network:
driver: bridge
Then I edited the mongo-setup.sh to:
#!/bin/bash
echo "************ [ Waiting for startup ] **************"
until curl http://${MONGODB_REPLICA_1}:27017/serverStatus\?text\=1 2>&1 | grep uptime | head -1; do
printf '.'
sleep 1
done
echo "************ [ Startup completed ] **************"
mongosh --host localhost:27017 <<EOF
var cfg = {
"_id": "dbrs",
"protocolVersion": 1,
"version": 1,
"members": [
{
"_id": 1,
"host": "localhost:27017",
"priority": 3
},
{
"_id": 2,
"host": "localhost:27018",
"priority": 2
},
{
"_id": 3,
"host": "localhost:27019",
"priority": 1
}
],settings: {chainingAllowed: true}
};
rs.initiate(cfg, { force: true });
rs.reconfig(cfg, { force: true });
rs.secondaryOk();
db.getMongo().setReadPref('nearest');
db.getMongo().setSecondaryOk();
EOF
And I removed the edits I made in /etc/hosts file. I don't get an error like before, I do get timeout:

Unable to connect to Mongodb container from flask container

#Docker compose file
version: "3.4" # optional since v1.27.0
services:
flaskblog:
build:
context: flaskblog
dockerfile: Dockerfile
image: flaskblog
restart: unless-stopped
environment:
APP_ENV: "prod"
APP_DEBUG: "False"
APP_PORT: 5000
MONGODB_DATABASE: flaskdb
MONGODB_USERNAME: flaskuser
MONGODB_PASSWORD:
MONGODB_HOSTNAME: mongodbuser
volumes:
- appdata:/var/www
depends_on:
- mongodb
networks:
- frontend
- backend
mongodb:
image: mongo:4.2
#container_name: mongodb
restart: unless-stopped
command: mongod --auth
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD:
MONGO_INITDB_DATABASE: flaskdb
MONGODB_DATA_DIR: /data/db2
MONDODB_LOG_DIR: /dev/null
volumes:
- mongodbdata:/data/db2
#- ./mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js:ro
networks:
- backend
networks:
backend:
driver: bridge
volumes:
mongodbdata:
driver: local
appdata:
driver: local
#Flask container file
FROM python:3.8-slim-buster
#python:3.6-stretch
LABEL MAINTAINER="Shekhar Banerjee"
ENV GROUP_ID=1000 \
USER_ID=1000 \
SECRET_KEY="4" \
EMAIL_USER="dankml.com" \
EMAIL_PASS="nzw"
RUN mkdir /home/logix3
WORKDIR /home/logix3
ADD . /home/logix3/
RUN pip install -r requirements.txt
RUN pip install gunicorn
RUN groupadd --gid $GROUP_ID www
RUN useradd --create-home -u $USER_ID --shell /bin/sh --gid www www
USER www
EXPOSE 5000/tcp
#CMD ["python","run.py"]
CMD [ "gunicorn", "-w", "4", "--bind", "127.0.0.1:5000", "run:app"]
These are my docker-compose file and Dockerfile for an application . The container for MongoDB and Flask are working fine individually, but when I try to run them together, I do not get any response on the localhost, There are no error messages in the logs or the containers
Curl in the system gives this :
$ curl -i http://127.0.0.1:5000
curl: (7) Failed to connect to 127.0.0.1 port 5000: Connection refused
Can anyone suggest how do I debug this ??
**Only backend network is functional
[
{
"Name": "mlspace_backend",
"Id": "e7e37cad0058330c99c55ffea2b98281e0c6526763e34550db24431b30030b77",
"Created": "2022-05-15T22:52:25.0281001+05:30",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.20.0.0/16",
"Gateway": "172.20.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"12c50f5f70d18b4c7ffc076177b59ff063a8ff81c4926c8cae0bf9e74dc2fc83": {
"Name": "mlspace_mongodb_1",
"EndpointID": "8278f672d9211aec9b539e14ae4eeea5e8f7aaef95448f44aab7ec1a8c61bb0b",
"MacAddress": "02:42:ac:14:00:02",
"IPv4Address": "172.20.0.2/16",
"IPv6Address": ""
},
"75cde7a34d6b3c4517c536a06349579c6c39090a93a017b2c280d987701ed0cf": {
"Name": "mlspace_flaskblog_1",
"EndpointID": "20489de8841d937f01768170a89f74c37ed049d241b096c8de8424c51e58704c",
"MacAddress": "02:42:ac:14:00:03",
"IPv4Address": "172.20.0.3/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {
"com.docker.compose.network": "backend",
"com.docker.compose.project": "mlspace",
"com.docker.compose.version": "1.23.2"
}
}
]
I checked the logs of the containers. no errors found, flask engine seem to be running but the site wont run , even curl won't given any output

Sequelize not work in docker (Node + Postgres)

Even with containers connected to the same network, when executing the migration on sequelize, the error 'getaddrinfo ENOTFOUND' is returned.
If I remove the host address in the 'database.js' settings, the migration runs but the CRUD routines stop running returning the error "connect ECONNREFUSED 127.0.0.1:5432"
If I point to the 'postgresdb' container in the 'database.js' settings, the CRUD routines execute but the sequelize migration does not.
Help me please
Fragment of docker network inspect in bridge
"Containers": {
"35467ab419c3632c4c0cfe57e972bd94c7de0a5818e37fdae6eb82a25381ceab": {
"Name": "api",
"EndpointID": "d54d95b94bec543fa831526d1abb99346045efc4cc5f4425d5d59b200ece3e62",
"MacAddress": "02:42:ac:1a:00:05",
"IPv4Address": "172.26.0.5/16",
"IPv6Address": ""
},
"7e028bcd80948fd61d14ff87437b963af31291220b7adbc8861fb98a2171f04a": {
"Name": "postgresdb",
"EndpointID": "e1d55e0fe8658a142880da27eea12a47d73f6cce472eeff38322ed4d6a60e8ad",
"MacAddress": "02:42:ac:1a:00:03",
"IPv4Address": "172.26.0.3/16",
"IPv6Address": ""
},
My docker-compose.yml
version: '3'
networks:
api:
driver: bridge
services:
api:
container_name: api
depends_on:
- postgresdb
- mongodb
- redisdb
links:
- postgresdb
environment:
POSTGRES_HOST: postgresdb
build: .
volumes:
- .:/home/node/api
command: yarn dev
networks:
- api
ports:
- 3333:3333
postgresdb:
image: postgres:alpine
container_name: postgresdb
environment:
- POSTGRES_USER=docker
- POSTGRES_PASSWORD=docker
- POSTGRES_DB=postdb
networks:
- api
ports:
- "5432:5432"
mongodb:
image: mongo
container_name: mongodb
networks:
- api
ports:
- "27017:27017"
redisdb:
image: redis
container_name: redisdb
networks:
- api
ports:
- "6379:6379"
My database.js file
require('dotenv/config');
module.exports = {
dialect: process.env.DB_DIALECT,
host: process.env.POSTGRES_HOST,
username: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD,
database: process.env.POSTGRES_DB,
define: {
timestamps: true,
underscored: true,
underscoredAll: true,
},
};

Mongodb data is not seeding using docker compose

My directory structure:
root
- mongo_seed
- Dockerfile
- init.json
- docker-compose.yml
- Dockerfile
Docker compose file
version: "3"
services:
web:
container_name: "hgbackend"
build: .
image: tahashin/hgbackend:v2
ports:
- "3000:3000"
links:
- mongodb
depends_on:
- mongodb
mongodb:
image: mongo:latest
container_name: "mongodb"
ports:
- "27017:27017"
mongo_seeding:
build: ./mongo_seed .
volumes:
- ./config/db-seed:/data
links:
- mongodb
depends_on:
- mongodb
docker file under mongo_seed directory
FROM mongo:latest
COPY init.json /init.json
CMD mongoimport --host mongodb --db alifhala --collection honcollection --type json --file /init.json --jsonArray
mongodb test data file init.json
[
{
"name": "Joe Smith",
"email": "jsmith#gmail.com",
"age": 40,
"admin": false
},
{
"name": "Jen Ford",
"email": "jford#gmail.com",
"age": 45,
"admin": true
}
]
After running docker-compose up in windows powershell database and collection is not creating and data is not dumping. After running mongo query in docker it is showing only 3 databases: local, admin, config
Check this Answer
https://stackoverflow.com/a/48179360/1124364
mongo_seeding:
build: ./mongo_seed .
Change it to
build:mongo_seed/.