How to change this script so that I can connect remotely to my Mongodb running in docker-compose, from different machines (that are not connected to the same network/internet provider).
I want to allow all remote connections.
I don't care about security matters as it's just for practice purposes!
docker-compose.yaml script file:
version: "3.8"
services:
mongodb:
image: mongo
container_name: mongodb
ports:
- 27017:27017
volumes:
- data:/data
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=admin
mongo-express:
image: mongo-express
container_name: mongo-express
restart: always
ports:
- 8081:8081
environment:
- ME_CONFIG_MONGODB_ADMINUSERNAME=admin
- ME_CONFIG_MONGODB_ADMINPASSWORD=admin
- ME_CONFIG_MONGODB_SERVER=mongodb
volumes:
data: {}
networks:
default:
name: mongodb_network
I solved my issue by migrating my data to Atlas cloud.mongodb.com
Answer update: To provide more info as suggested by #nuhkoca, This is the video tutorial to create a mongodb atlas: https://www.youtube.com/watch?v=xrc7dIO_tXk&t=15s And this is the link to the db from my resources file in my Springboot backend api:
server.port=<Port_number>
spring.data.mongodb.uri=<Link_to_mongodb_atlas>
Related
When trying to spin up a docker container with docker-compose,
I get the following error message:
(node:8) [MONGODB DRIVER] Warning: Current Server Discovery
and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
mongo-express | Could not connect to database using connectionString: mongodb://127.0.0.1:27017/"
I noticed that mongo-express always adds a double quote to the end of my connection string, is this intentional or did I mess something up in my docker-compose file?
The file:
version: '3.8'
services:
mongodb:
image: mongo:latest
container_name: mongodb
restart: unless-stopped
networks:
- backbone
expose:
- 27017
ports:
- 27017:27017
env_file:
- ./.env
command: [--auth]
environment:
- MONGO_INITDB_ROOT_USERNAME=${DB_USERNAME}
- MONGO_INITDB_ROOT_PASSWORD=${DB_PASSWORD}
volumes:
- ./data:/data/db
mongo-express:
image: mongo-express
container_name: mongo-express
restart: unless-stopped
networks:
- backbone
ports:
- 8081:8081
environment:
- ME_CONFIG_MONGODB_SERVER=127.0.0.1
- ME_CONFIG_MONGODB_URL="mongodb://127.0.0.1:27017"
- ME_CONFIG_BASICAUTH_USERNAME=${DB_USERNAME}
- ME_CONFIG_BASICAUTH_PASSWORD=${DB_PASSWORD}
networks:
backbone:
driver: bridge
I already tried updating my version of docker/docker-compose and mongodb.
How do I resolve the error?
Or is there a workaround I could use?
In a container, localhost refers to the container itself. So when Mongo Express tries to connect to the database at localhost, it looks for the database inside the Mongo Express container.
Docker compose let's you refer to containers using their container names, so you should change your Mongo Express environment variables to
- ME_CONFIG_MONGODB_SERVER=mongodb
- ME_CONFIG_MONGODB_URL="mongodb://mongodb:27017"
I have a implemented a microservice architecture with several servers and databases. I have installed elasticsearch with docker and when I do docker-compose up, everything seems to run fine.
However I would like to integrate the elasticsearch with the several databases (2 mongodb in this sample below) in the system. How do I synch the two mongodb in two different containers with elasticsearch so that I can search them?
client:
container_name: client
stdin_open: true
build:
context: ./client
dockerfile: Dockerfile
restart: always
volumes:
- './client:/app'
ports:
- '1000:3000'
environment:
- NODE_ENV=development
- CHOKIDAR_USEPOLLING=true
weatherdb:
container_name: weather-db
image: mongo
restart: always
ports:
- '2002:27017'
volumes:
- ./weather_service/weather_db:/data/db
networks:
- backend
weather-service:
container_name: weather-service
build: ./weather_service
restart: always
ports:
- "1002:3000"
depends_on:
- weatherdb
links:
- elasticsearch
networks:
- backend
newsdb:
container_name: news-db
image: mongo
restart: always
ports:
- '2003:27017'
volumes:
- ./news_service/news_db:/data/db
networks:
- backend
news-service:
container_name: news-service
build: ./news_service
restart: always
ports:
- "1003:3000"
depends_on:
- newsdb
links:
- elasticsearch
networks:
- backend
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0
container_name: elasticsearch
restart: always
ports:
- 9200:9200
- 9300:9300
environment:
ES_JAVA_OPTS: '-Xms512m -Xmx512m'
network.bind_host: 0.0.0.0
network.host: 0.0.0.0
discovery.type: single-node
volumes:
- ./elasticsearch/esdata:/usr/share/elasticsearch/data
networks:
- backend
Its very simple to just add a elasticsearch docker section in any docker-compose file and start it, all these are independent docker containers and as long as their exposed port on host is not interfering each other and you have the correct configuration in place it should work.
Please refer elasticsearch multi-docker installation using docker file for more info.
NOTE: You have not mentioned what exact issue you are facing, you have mentioned everything ie all docker containers are running file, so please explain in detail what exactly you are trying to solve
I'm trying to setup a Flask <-> MongoDB <-> mongo_express application. I have 3 containers defined in my docker-compose.yml, and I start them successfully. However, while the Mongo part is OK (I can access the DB via the express api at localhost:8081), Flask can't access the DB.
What I'm looking for:
I want to be able to send requests from host machine (or any other
in the network) to Flask (running on 0.0.0.0:5000), from Flask to DB
(running on localhost:27017, accessed using pymongo wrapper).
Also, I want to be able to have access to mongo_express (on
localhost:8081) and from it to the DB [This part is already
working!]
In order to debug it, I removed the Flask container from the docker-compose.yml, restarted, and run it locally, and voila! everything works (meaning my pytest runs are ok - I have a test where I send a request to the Flask server, which in turn, uses the pymongo wrapper to access the DB, and return data from it to the client).
I guess my network configuration is flawed, but I don't understand where.
Here is my docker-compose.yml (Flask is commented out, since it is currently running locally):
version: "3.8"
services:
# mgmt_server:
# build: ./server # Dockerfile just copies py files, installs pip requirements, and runs "python server.py"
# container_name: mgmt_server
# restart: always
# networks:
# - backend # Connect to flask
# ports:
# - "5000:5000"
mongo:
image: mongo:latest
container_name: mongodb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=pass
volumes:
- /data/db:/data/db
networks:
- backend # Connect to flask
- frontend # Connect to mongo_express
ports:
- "27017:27017"
mongo-express:
image: mongo-express:latest
container_name: mongo_express
restart: always
environment:
- ME_CONFIG_MONGODB_SERVER=mongo
- ME_CONFIG_MONGODB_PORT=27017
- ME_CONFIG_MONGODB_ENABLE_ADMIN=true
- ME_CONFIG_MONGODB_AUTH_DATABASE=admin
- ME_CONFIG_MONGODB_ADMINUSERNAME=root
- ME_CONFIG_MONGODB_ADMINPASSWORD=pass
# Uncomment if a secure login via browser is required
# - ME_CONFIG_BASICAUTH_USERNAME=root
# - ME_CONFIG_BASICAUTH_PASSWORD=pass
links:
- mongo
networks:
- frontend # Connect to mongo_express
ports:
- 8081:8081
networks:
backend:
driver: bridge
frontend:
driver: bridge
$ docker network ls # When mgmt_server is run locally
NETWORK ID NAME DRIVER SCOPE
10577560a149 bridge bridge local
a037a11e12bb host host local
b153eea9db12 node_mgmt_backend bridge local
c8e1b58ffb44 node_mgmt_frontend bridge local
4f7b75b5695a none null local
The problem was that Flask tried to access Mongo on localhost. When running Flask on the host, this is OK, but when Flask is containerized, it gets its own ip, and so does Mongo, and localhost just doesn't point to the right place.
Editing the docker-compose.yml networks, and redirecting pymongo client to the updated Mongo IP fixed the issue:
version: "3.8"
services:
mgmt_server:
build: ./server
container_name: mgmt_server
restart: always
ports:
- "5000:5000"
networks:
app_net:
ipv4_address: 172.16.238.2
mongo:
image: mongo:latest
container_name: mongodb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=pass
volumes:
- /data/db:/data/db
ports:
- "27017:27017"
networks:
app_net:
ipv4_address: 172.16.238.3
mongo-express:
image: mongo-express:latest
container_name: mongo_express
restart: always
environment:
- ME_CONFIG_MONGODB_SERVER=mongo
- ME_CONFIG_MONGODB_PORT=27017
- ME_CONFIG_MONGODB_ENABLE_ADMIN=true
- ME_CONFIG_MONGODB_AUTH_DATABASE=admin
- ME_CONFIG_MONGODB_ADMINUSERNAME=root
- ME_CONFIG_MONGODB_ADMINPASSWORD=pass
# Uncomment if a secure login via browser is required
# - ME_CONFIG_BASICAUTH_USERNAME=root
# - ME_CONFIG_BASICAUTH_PASSWORD=pass
links:
- mongo
ports:
- 8081:8081
networks:
app_net:
ipv4_address: 172.16.238.4
networks:
app_net:
ipam:
driver: default
config:
- subnet: "172.16.238.0/24"
This was helpful, but eventually I found this to be more informative.
I have been trying to dockerize my spring boot application which depends on redis, kafka and mongodb.
Following is the docker-compose.yml:
version: '3.3'
services:
my-service:
image: my-service
build:
context: ../../
dockerfile: Dockerfile
restart: always
container_name: my-service
environment:
KAFKA_CONFLUENT_BOOTSTRAP_SERVERS: kafka:9092
MONGO_HOSTS: mongodb:27017
REDIS_HOST: redis
REDIS_PORT: 6379
volumes:
- /private/var/log/my-service/:/var/log/my-service/
ports:
- 8080:8090
- 1053:1053
depends_on:
- redis
- kafka
- mongodb
portainer:
image: portainer/portainer
command: -H unix:///var/run/docker.sock
restart: always
container_name: portainer
ports:
- 9000:9000
- 9001:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
redis:
image: redis
container_name: redis
restart: always
ports:
- 6379:6379
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookeeper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
container_name: kafka
environment:
KAFKA_CREATE_TOPICS: "cms.entity.change:1:1" # topic:partition:replicas
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_PORT: 9092
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- "zookeeper"
mongodb:
image: mongo:latest
container_name: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME:
MONGO_INITDB_ROOT_PASSWORD:
ports:
- 27017:27017
volumes:
- ./data/db:/data/db
The issue is that this starts up mongo as a STANDALONE instance. So the APIs in my service that persist data are failing as mongo needs to start as a REPLICA_SET.
How can I edit my docker-compose file to start mongo as a REPLICA_SET?
I had the same issue and ended up on this stackoverflow post.
We had a requirement of using official mongoDB docker image (https://hub.docker.com/_/mongo ) and couldn't use bitnami as suggested in Vahid's answer.
This answer isn't exactly what's needed by the question asked and coming in 6 months too late; but it should give directions to someone who need to use the mongoDb standalone replicaset throw away instance for integration testing purpose. If you need to use it in PROD then you'll have to provide environment variables for volumes and auth as per Vahid's answer.
version: '3.7'
services:
mongodb:
image: mongo:latest
container_name: myservice-mongodb
networks:
- myServiceNetwork
expose:
- 27017
command: --replSet singleNodeReplSet
mongodb-replicaset:
container_name: mongodb-replicaset-helper
depends_on:
- mongodb
networks:
- myServiceNetwork
image: mongo:latest
command: bash -c "sleep 5 && mongo --host myservice-mongodb --port 27017 --eval \"rs.initiate()\" && sleep 2 && mongo --host myservice-mongodb --port 27017 --eval \"rs.status()\" && sleep infinity"
my-service:
depends_on:
- mongodb-replicaset
image: myserviceimage
container_name: myservicecontainer
networks:
- myServiceNetwork
environment:
myservice__Database__ConnectionString: mongodb://myservice-mongodb:27017/?connect=direct&replicaSet=singleNodeReplSet&readPreference=primary
myservice__Database__Name: myserviceDb
networks:
myServiceNetwork:
driver: bridge
NOTE: Please look at the way how connection string is passed as env variable to the service depending on mongo replicaset instance. You'd have to ensure that the name used in setting up the mongodb replicaset (in my case singleNodeReplicaSet) is passed on to the service depending on it.
Edited:
my previous answer was far wrong so I changed it. I managed to make it work using 'bitnami/mongodb:4.0'. Not sure if that would help you or not, but maybe it gives you some idea. They have a docker-compose file ready for replicaset mode.
version: '3'
services:
mdb-primary:
image: 'bitnami/mongodb:4.0'
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-primary
mdb-secondary:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-secondary
mdb-arbiter:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-arbiter
mongo-cli:
image: 'bitnami/mongodb:latest'
don't forget to add volumes and map it to /bitnami on the primary node
the last container, mongo-cli is for testing purposes. So you can connect to the replicaset using the cli, there is an argument about that here if you like to read about it.
$ docker-compose exec mongo-cli bash
$ mongo "mongodb://mdb-primary:27017/test?replicaSet=replicaset"
Here is my docker compose file:
version: "3.3"
services:
test:
image: test
networks:
- mongo_net
ports:
- 4000:80
depends_on:
- mongodb
links:
- mongodb
mongodb:
image: mongo:latest
networks:
- mongo_net
ports:
- 27017:27017
volumes:
- local_data:/data/db
volumes:
local_data:
networks:
mongo_net:
driver: bridge
The 'test' image cannot find the 'mongodb' instance.
My assumption is that the 'links' section would connect the two, but it is not happening.
What am I missing?
for your compose file try just using depends_on. links is deprecated and maybe thats why you are currently getting issues since this is a V3 compose file.