I am trying to run a MongoDB container on a DigitalOcean droplet and so created a new dir with mkdir -pv mongodb/data then inside mongodb I created a new file with touch mongodb.conf then edited it with nano mongodb.conf so that it looks like this:
systemLog:
destination: file
path: "/var/log/mongodb/mongod.log"
logAppend: true
storage:
journal:
enabled: true
processManagement:
fork: true
net:
bindIp: 0.0.0.0
port: 27017
setParameter:
enableLocalhostAuthBypass: false
I then used the same process to create a file docker-compose.yml Which looks like:
version: "3.8"
services:
mongodb:
image: mongo:latest
container_name: mongodb
environment:
- PUID=1000
- PGID=1000
volumes:
- /mongodb/mongod.conf:/etc/mongod.conf
- /mongodb/data:/data/db
command: ["mongod", "--config", "/etc/mongod.conf"]
ports:
- 0.0.0.0:27017:27017
This makes the filesystem look like:
.
├── mongodb
│ ├── data
│ ├── docker-compose.yml
│ └── mongod.conf
When I then try running docker-compose up it gives the error:
mongodb | grep: /etc/mongo/mongod.conf: Is a directory
mongodb | error: unexpected "js-yaml.js" output while parsing config:
mongodb | undefined
mongodb exited with code 1
I have also tried changing the command to: command: ["mongod", "-f", "/etc/mongod.conf"] but this gives a similar error:
mongodb | Error opening config file: Is a directory
mongodb | try 'mongod --help' for more information
mongodb exited with code 2
Any help to solve this would be appreciated.
I did it following way:
version: "3.8"
services:
mongodb:
image: mongo:latest
container_name: mongodb
environment:
- PUID=1000
- PGID=1000
volumes:
- /mongodb/cnf:/etc/mongo/
- /mongodb/data:/data/db
command: ["mongod", "--config", "/etc/mongo/mongod.conf"]
ports:
- 0.0.0.0:27017:27017
Then just put your mongod.conf in /mongodb/cnf on host machine
Related
I am trying to connect to a MongoDB replica set using pymongo, but I keep getting the error: pymongo.errors.ServerSelectionTimeoutError: No replica set members match selector. In the error message it's also specified that my topology type is ReplicaSetNoPrimary, which is odd, as connecting with mongo bash shows a clear primary.
Note that the replica set works fine and is usable via mongo bash on the master node.
Also, I have added firewall rules to allow both inbound and outbound traffic on the specified ports, just to make sure this isn't the issue.
I am using docker-compose for the cluster. The file:
version: "3.9"
services:
mongo-master:
image: mongo:latest
container_name: mongo_master
volumes:
- ./data/master:/data/db
ports:
- 27017:27017
command: mongod --replSet dbrs & mongo --eval rs.initiate(`cat rs_config.json`)
stdin_open: true
tty: true
mongo-slave-1:
image: mongo:latest
container_name: mongo_slave_1
volumes:
- ./data/slave_1:/data/db
ports:
- 27018:27017
command: mongod --replSet dbrs
stdin_open: true
tty: true
mongo-slave-2:
image: mongo:latest
container_name: mongo_slave_2
volumes:
- ./data/slave_2:/data/db
ports:
- 27019:27017
command: mongod --replSet dbrs
stdin_open: true
tty: true
The rs_config.json file used above:
{
"_id" : "dbrs",
"members" : [
{
"_id" : 0,
"host" : "mongo_master:27017",
"priority" : 10
},
{
"_id" : 1,
"host" : "mongo_slave_1:27017"
},
{
"_id" : 2,
"host" : "mongo_slave_2:27017"
}
]
}
The error raises on the last line here:
self.__client = MongoClient(["localhost:27017", "localhost:27018", "localhost:27019"], replicaset="dbrs")
self.__collection = self.__client[self.__db_name][collection.value]
self.__collection.insert_one(dictionary_object)
I ommitted some code for brevity, but you can assume all class attributes and dictionary_object are well defined according to pymongo docs.
Also please note that I have tried many different ways to initialize MongoClient, including a connection string (as in the docs), and the connect=False optional parameter as advised in some blogs. The issue persists...
Edit: I tried adding "mongo_master" to my etc/hosts file pointing at 127.0.0.1 and changing the connection string from localhost to that, and it works with the replica set. This is a bad workaround but maybe can help in figuring out a solution.
Thanks in advance for any help!
To get a connection to a MongoDB replicaset from an external client, you must be able to resolve the hostnames from the local client.
https://docs.mongodb.com/manual/tutorial/deploy-replica-set/#connectivity
Ensure that network traffic can pass securely between all members of the set and all clients in the network.
So, add the following to your /etc/hosts file:
127.0.0.1 mongodb-1
127.0.0.1 mongodb-2
127.0.0.1 mongodb-3
To be able to connect both internally and externally, you will need to run each MongoDB service on different ports.
The following script will initiate a 3-node MongoDB replicaset and run a test client. I recommend using the Bitnami image as it takes care of the replset initiation for you. (Borrowing heavily from this configuration)
#!/bin/bash
PROJECT_NAME=replset_test
MONGODB_VERSION=4.4
PYTHON_VERSION=3.9.6
PYMONGO_VERSION=4.0.1
cd "$(mktemp -d)" || exit
cat << EOF > Dockerfile
FROM python:${PYTHON_VERSION}-slim-buster
COPY requirements.txt /tmp/
RUN pip install -r /tmp/requirements.txt
COPY ${PROJECT_NAME}.py .
CMD [ "python", "./${PROJECT_NAME}.py" ]
EOF
cat << EOF > requirements.txt
pymongo==${PYMONGO_VERSION}
EOF
cat << EOF > ${PROJECT_NAME}.py
from pymongo import MongoClient
connection_string = 'mongodb://root:password123#mongodb-1:27017,mongodb-2:27018,mongodb-3:27019/mydatabase?authSource=admin&replicaSet=replicaset'
client = MongoClient(connection_string)
db = client.db
db['mycollection'].insert_one({'a': 1})
record = db['mycollection'].find_one()
if record is not None:
print(f'{__file__}: MongoDB connection working using connection string "{connection_string}"')
EOF
cp ${PROJECT_NAME}.py ${PROJECT_NAME}_external.py
cat << EOF > docker-compose.yaml
version: '3.9'
services:
mongodb-1:
image: docker.io/bitnami/mongodb:${MONGODB_VERSION}
ports:
- 27017:27017
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-1
- MONGODB_PORT_NUMBER=27017
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=password123
- MONGODB_REPLICA_SET_KEY=replicasetkey123
volumes:
- 'mongodb_master_data:/bitnami/mongodb'
mongodb-2:
image: docker.io/bitnami/mongodb:${MONGODB_VERSION}
ports:
- 27018:27018
depends_on:
- mongodb-1
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-2
- MONGODB_PORT_NUMBER=27018
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_INITIAL_PRIMARY_HOST=mongodb-primary
- MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password123
- MONGODB_REPLICA_SET_KEY=replicasetkey123
mongodb-3:
image: docker.io/bitnami/mongodb:${MONGODB_VERSION}
ports:
- 27019:27019
depends_on:
- mongodb-1
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-3
- MONGODB_PORT_NUMBER=27019
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_INITIAL_PRIMARY_HOST=mongodb-primary
- MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD=password123
- MONGODB_REPLICA_SET_KEY=replicasetkey123
${PROJECT_NAME}:
container_name: ${PROJECT_NAME}
build: .
depends_on:
- mongodb-1
- mongodb-2
- mongodb-3
volumes:
mongodb_master_data:
driver: local
EOF
docker rm --force $(docker ps -a -q --filter name=mongo) 2>&1 > /dev/null
docker rm --force $(docker ps -a -q --filter name=${PROJECT_NAME}) 2>&1 > /dev/null
docker-compose up --build -d
python ${PROJECT_NAME}.py
docker ps -a -q --filter name=${PROJECT_NAME}
docker logs $(docker ps -a -q --filter name=${PROJECT_NAME})
If all is ok you will get an output confirming both internal and external connectivity:
/tmp/tmp.QM9tQPE8Dj/replset_test.py: MongoDB connection working using connection string "mongodb://root:password123#mongodb-1:27017,mongodb-2:27018,mongodb-3:27019/mydatabase?authSource=admin&replicaSet=replicaset"
d53e8c41ad20
//./replset_test.py: MongoDB connection working using connection string "mongodb://root:password123#mongodb-1:27017,mongodb-2:27018,mongodb-3:27019/mydatabase?authSource=admin&replicaSet=replicaset"
I have FastAPI app running in docker docker container. It works well except only one thing
The app doesn't reload if any changes. The changes applied only if i restart the container. But i wonder why it doesn't reload app if i put in command --reload flag?
I understand that docker itself do not reload if some changes in code. But app must be if flag --reload in command .
If I misunderstand, please advise how to achieve what i want. Thanks
main.py
from typing import Optional
import uvicorn
from fastapi import FastAPI
app = FastAPI()
#app.get("/")
def read_root():
return {"Hello": "World"}
#app.get("/items/{item_id}")
def read_item(item_id: int, q: Optional[str] = None):
return {"item_id": item_id, "q": q}
if __name__ == '__main__':
uvicorn.run(app, host="0.0.0.0", port=8000, reload=True)
docker-compose.yml
version: "3"
services:
web:
build: .
restart: always
command: bash -c "uvicorn main:app --host 0.0.0.0 --port 8000 --reload"
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
db:
image: postgres
ports:
- "50009:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=test_db
this works for me
version: "3.9"
services:
people:
container_name: people
build: .
working_dir: /code/app
command: uvicorn main:app --host 0.0.0.0 --reload
environment:
DEBUG: 1
volumes:
- ./app:/code/app
ports:
- 8008:8000
restart: on-failure
this is my directory structure
.
├── Dockerfile
├── Makefile
├── app
│ └── main.py
├── docker-compose.yml
└── requirements.txt
make sure working_dir and volumes section's - ./app:/code/app match
example run:
docker-compose up --build
...
Attaching to people
people | INFO: Will watch for changes in these directories: ['/code/app']
Are you starting the container with docker compose up? This is working for me with hot reload at http://127.0.0.1.
version: "3.9"
services:
bff:
container_name: bff
build: .
working_dir: /code/app
command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload
environment:
DEBUG: 1
volumes:
- .:/code
ports:
- "80:8000"
restart: on-failure
Also, I don't have your final two lines, if __name__ == etc., in my app. Not sure if that would change anything.
I found this solution that worked for me, in this answer.
In the watchfiles documentation it is understood that the detection relies on file system notifications, and I think that via docker its events are not launched when using a volume.
Notify will fall back to file polling if it can't use file system
notifications
So you have to tell watchfiles to force the polling, that's what you did in your test python script with the parameter force_polling and that's why it works:
for changes in watch('/code', force_polling=True):
Fortunately in the documentation we are given the possibility to force the polling via the environment variable WATCHFILES_FORCE_POLLING. Add this environment variable to your docker-compose.yml and auto-reload will work:
services:
fastapi-dev:
image: myimagename:${TAG:-latest}
build:
context: .
volumes:
- ./src:/code
- ./static:/static
- ./templates:/templates
restart: on-failure
ports:
- "${HTTP_PORT:-8080}:80"
environment:
- WATCHFILES_FORCE_POLLING=true
I want to create a Docker container with an instance of Mongo. In particular, I would like to create a replica set with only one node (since I'm interested in transactions and they are only available for replica sets).
Dockerfile
FROM mongo
RUN echo "rs.initiate();" > /docker-entrypoint-initdb.d/replica-init.js
CMD ["--replSet", "rs0"]
docker-compose.yml
version: "3"
services:
db:
build:
dockerfile: Dockerfile
context: .
ports:
- "27017:27017"
If I use the Dockerfile alone everything is fine, while if I use docker-compose it does not work: in fact if I then log to the container I got prompted as rs0:OTHER> instead of rs0:PRIMARY>.
I consulted these links but the solutions proposed are not working:
https://github.com/docker-library/mongo/issues/246#issuecomment-382072843
https://github.com/docker-library/mongo/issues/249#issuecomment-381786889
This is the compose file I have used for a while now for local development. You can remove the keyfile pieces if you don't need to connect via SSL.
version: "3.8"
services:
mongodb:
image : mongo:4
container_name: mongodb
hostname: mongodb
restart: on-failure
environment:
- PUID=1000
- PGID=1000
- MONGO_INITDB_ROOT_USERNAME=mongo
- MONGO_INITDB_ROOT_PASSWORD=mongo
- MONGO_INITDB_DATABASE=my-service
- MONGO_REPLICA_SET_NAME=rs0
volumes:
- mongodb4_data:/data/db
- ./:/opt/keyfile/
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 10s
start_period: 30s
command: "--bind_ip_all --keyFile /opt/keyfile/keyfile --replSet rs0"
volumes:
mongodb4_data:
It uses Docker's health check (with a startup delay) to sneak in the rs.initiate() if it actually needs it after it's already running.
To create a keyfile.
Mac:
openssl rand -base64 741 > keyfile
chmod 600 keyfile
Linux:
openssl rand -base64 756 > keyfile
chmod 600 keyfile
sudo chown 999 keyfile
sudo chgrp 999 keyfile
The top answer stopped working for me in later MongoDB and Docker versions. Particularly because rs.initiate().ok would throw an error if the replica set was already initiated, causing the whole command to fail. In addition, connecting from another container was failing because the replica set's sole member had some random host, which wouldn't allow the connection. Here's my new docker-compose.yml:
services:
web:
# ...
environment:
DATABASE_URL: mongodb://root:root#db/?authSource=admin&tls=false
db:
build:
context: ./mongo/
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: root
ports:
- '27017:27017'
volumes:
- data:/data/db
healthcheck:
test: |
test $$(mongosh --quiet -u root -p root --eval "try { rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'db' }] }).ok } catch (_) { rs.status().ok }") -eq 1
interval: 10s
start_period: 30s
volumes:
data:
Inside ./mongo/, I have a custom Dockerfile that looks like:
FROM mongo:6
RUN echo "password" > /keyfile \
&& chmod 600 /keyfile \
&& chown 999 /keyfile \
&& chgrp 999 /keyfile
CMD ["--bind_ip_all", "--keyFile", "/keyfile", "--replSet", "rs0"]
This Dockerfile is suitable for development, but you'd definitely want a securely generated and persistent keyfile to be mounted in production (and therefore strike the entire RUN command).
You still need to issue replSetInitiate even if there's only one node in the RS.
See also here.
I had to do something similar to build tests around ChangeStreams which are only available when running mongo as a replica set. I don't remember where I pulled this from, so I can't explain it in detail but it does work for me. Here is my setup:
Dockerfile
FROM mongo:5.0.3
RUN echo "rs.initiate({'_id':'rs0', members: [{'_id':1, 'host':'127.0.0.1:27017'}]});" > "/docker-entrypoint-initdb.d/init_replicaset.js"
RUN echo "12345678" > "/tmp/key.file"
RUN chmod 600 /tmp/key.file
RUN chown 999:999 /tmp/key.file
CMD ["mongod", "--replSet", "rs0", "--bind_ip_all", "--keyFile", "/tmp/key.file"]
docker-compose.yml
version: '3.7'
services:
mongo:
build: .
restart: always
ports:
- 27017:27017
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u admin -p pass --quiet) -eq 1
interval: 10s
start_period: 30s
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: pass
MONGO_INITDB_DATABASE: test
Run docker-compose up and you should be good.
Connection String: mongodb://admin:pass#localhost:27017/test
Note: You shouldn't use this in production obviously, adjust the key "12345678" in the Dockerfile if security is a concern.
If you just need single node replica set of MongoDB via docker-compose.yml you can simply use this:
mongodb:
image: mongo:5
restart: always
command: ["--replSet", "rs0", "--bind_ip_all"]
ports:
- 27018:27017
healthcheck:
test: mongo --eval "rs.initiate()"
start_period: 5s
This one works fine for me:
version: '3.4'
services:
ludustack-db:
container_name: ludustack-db
command: mongod --auth
image: mongo:latest
hostname: mongodb
ports:
- '27017:27017'
env_file:
- .env
environment:
- MONGO_INITDB_ROOT_USERNAME=${MONGO_INITDB_ROOT_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${MONGO_INITDB_ROOT_PASSWORD}
- MONGO_INITDB_DATABASE=${MONGO_INITDB_DATABASE}
- MONGO_REPLICA_SET_NAME=${MONGO_REPLICA_SET_NAME}
healthcheck:
test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
interval: 60s
start_period: 60s
I'm new to Docker, and I'm trying the simplest of setups with docker-compose, but don't succeed to connect to Mongodb.
My docker-compose.local.yaml file:
version: "2"
services:
posts-api:
build:
dockerfile: Dockerfile.local
context: ./
volumes:
- ".:/app"
ports:
- "6820:6820"
depends_on:
- mongodb
mongodb:
image: mongo:3.5
ports:
- "27018:27018"
command: mongod --port 27018
My Docker file:
FROM node:7.8.0
MAINTAINER Livefeed 'project.livefeed#gmail.com'
RUN mkdir /app
VOLUME /app
WORKDIR /app
ADD package.json yarn.lock ./
RUN eval rm -rf node_modules && \
yarn
ADD server.js .
RUN mkdir config src
ADD config config/
ADD src src/
EXPOSE 6820
EXPOSE 27018
CMD yarn run local
In server.js I try to connect with:
mongoose.connect('mongodb://localhost:27018');
I also tried:
mongoose.connect('mongodb://mongodb:27018');
To run docker-compose:
docker-compose -f docker-compose.local.yaml up --build
And I receive the error:
connection error: { MongoError: failed to connect to server [localhost:27018] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27018]
What am I missing?
In server.js use mongodb instead of localhost:
mongoose.connect('mongodb://mongodb:27018');
Because containers in the same network can communicate using their service name.
Bear in mind that each container and your host have their own localhost. Each localhost is a different host: container A, container B, your host (each one has its own network interface).
Edit:
Be sure to get your mongo up:
docker-compose logs mongodb
docker-compose ps
Sometimes it doesn't get up because of disk space.
Edit 2:
With newer versions of mongo, you need to specify to listen to all interfaces too:
command: mongod --port 27018 --bind_ip_all
I think, that you should add links option in your config. Like this:
ports:
- "6820:6820"
depends_on:
- mongodb
links:
- mongodb
update
As I promised
version: '2.1'
services:
pm2:
image: keymetrics/pm2-docker-alpine:6
restart: always
container_name: pm2
volumes:
- ./pm2:/app
links:
- redis_db
- db
environment:
REDIS_CONNECTION_STRING: redis://redis_db:6379
nginx:
image: firesh/nginx-lua
restart: always
volumes:
- ./nginx:/etc/nginx
- /var/run/docker.sock:/tmp/docker.sock:ro
ports:
- 80:80
links:
- pm2
s3: # mock for development
image: lphoward/fake-s3:latest
redis_db:
container_name: redis_db
image: redis
ports:
- 6379:6379
db: # for scorebig-syncer
image: mysql:5.7
ports:
- 3306:3306
I have installed MongoDB in Linux Ubuntu through the docker image.
I have set the parameters in the YAML file like below to implement the authentication for mongodb. But when I up the server using the command docker-compose up -d, I'm getting error like
"Unsupported config option for 'db' service: 'security'".
How can I resolve the issue?
docker-compose.yaml:
db:
image: mongo:2.6.4
command: ["mongod", "--smallfiles"]
expose: "27017"
ports: "27017:27017"
security: keyFile: "/home/ubuntu/dockerlab/keyfile.key"
authorization: "enabled"
security and authorization are not a keyword for the docker-compose YAML file, so take them out of there.
If that file key file needs copying into the container, you should put something like:
FROM: mongo:2.6.4
ADD /home/ubuntu/dockerlab/keyfile.key /tmp
ENV AUTH yes
in a Dockerfile.
And change in the docker-compose.yml file:
image: mongo:2.6.4
into
build: .
and the command value into
command: ["mongod", "--smallfiles", "--keyFile /tmp/keyfile.key" ]
Alternatively you can use a volume entry in your docker-compose.yml to map the keyfile.key into your container, and instead of the ENV in the Dockerfile you add , "--auth" to sequence that is the value for command. Then you can continue to use the image stanza and leave out the Dockerfile altogether:
db:
image: mongo:2.6.4
command: ["mongod", "--smallfiles", "--auth", "--keyFile /tmp/keyfile.key" ]
expose: "27017"
ports: "27017:27017"
volumes:
- /home/ubuntu/dockerlab/keyfile.key: /tmp