Dockerized flask server is not respnsive from within the container, but is responsive when is run on host - mongodb

I'm trying to setup a Flask <-> MongoDB <-> mongo_express application. I have 3 containers defined in my docker-compose.yml, and I start them successfully. However, while the Mongo part is OK (I can access the DB via the express api at localhost:8081), Flask can't access the DB.
What I'm looking for:
I want to be able to send requests from host machine (or any other
in the network) to Flask (running on 0.0.0.0:5000), from Flask to DB
(running on localhost:27017, accessed using pymongo wrapper).
Also, I want to be able to have access to mongo_express (on
localhost:8081) and from it to the DB [This part is already
working!]
In order to debug it, I removed the Flask container from the docker-compose.yml, restarted, and run it locally, and voila! everything works (meaning my pytest runs are ok - I have a test where I send a request to the Flask server, which in turn, uses the pymongo wrapper to access the DB, and return data from it to the client).
I guess my network configuration is flawed, but I don't understand where.
Here is my docker-compose.yml (Flask is commented out, since it is currently running locally):
version: "3.8"
services:
# mgmt_server:
# build: ./server # Dockerfile just copies py files, installs pip requirements, and runs "python server.py"
# container_name: mgmt_server
# restart: always
# networks:
# - backend # Connect to flask
# ports:
# - "5000:5000"
mongo:
image: mongo:latest
container_name: mongodb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=pass
volumes:
- /data/db:/data/db
networks:
- backend # Connect to flask
- frontend # Connect to mongo_express
ports:
- "27017:27017"
mongo-express:
image: mongo-express:latest
container_name: mongo_express
restart: always
environment:
- ME_CONFIG_MONGODB_SERVER=mongo
- ME_CONFIG_MONGODB_PORT=27017
- ME_CONFIG_MONGODB_ENABLE_ADMIN=true
- ME_CONFIG_MONGODB_AUTH_DATABASE=admin
- ME_CONFIG_MONGODB_ADMINUSERNAME=root
- ME_CONFIG_MONGODB_ADMINPASSWORD=pass
# Uncomment if a secure login via browser is required
# - ME_CONFIG_BASICAUTH_USERNAME=root
# - ME_CONFIG_BASICAUTH_PASSWORD=pass
links:
- mongo
networks:
- frontend # Connect to mongo_express
ports:
- 8081:8081
networks:
backend:
driver: bridge
frontend:
driver: bridge
$ docker network ls # When mgmt_server is run locally
NETWORK ID NAME DRIVER SCOPE
10577560a149 bridge bridge local
a037a11e12bb host host local
b153eea9db12 node_mgmt_backend bridge local
c8e1b58ffb44 node_mgmt_frontend bridge local
4f7b75b5695a none null local

The problem was that Flask tried to access Mongo on localhost. When running Flask on the host, this is OK, but when Flask is containerized, it gets its own ip, and so does Mongo, and localhost just doesn't point to the right place.
Editing the docker-compose.yml networks, and redirecting pymongo client to the updated Mongo IP fixed the issue:
version: "3.8"
services:
mgmt_server:
build: ./server
container_name: mgmt_server
restart: always
ports:
- "5000:5000"
networks:
app_net:
ipv4_address: 172.16.238.2
mongo:
image: mongo:latest
container_name: mongodb
restart: always
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=pass
volumes:
- /data/db:/data/db
ports:
- "27017:27017"
networks:
app_net:
ipv4_address: 172.16.238.3
mongo-express:
image: mongo-express:latest
container_name: mongo_express
restart: always
environment:
- ME_CONFIG_MONGODB_SERVER=mongo
- ME_CONFIG_MONGODB_PORT=27017
- ME_CONFIG_MONGODB_ENABLE_ADMIN=true
- ME_CONFIG_MONGODB_AUTH_DATABASE=admin
- ME_CONFIG_MONGODB_ADMINUSERNAME=root
- ME_CONFIG_MONGODB_ADMINPASSWORD=pass
# Uncomment if a secure login via browser is required
# - ME_CONFIG_BASICAUTH_USERNAME=root
# - ME_CONFIG_BASICAUTH_PASSWORD=pass
links:
- mongo
ports:
- 8081:8081
networks:
app_net:
ipv4_address: 172.16.238.4
networks:
app_net:
ipam:
driver: default
config:
- subnet: "172.16.238.0/24"
This was helpful, but eventually I found this to be more informative.

Related

How to connect remotely to Mongodb running on Docker-compose?

How to change this script so that I can connect remotely to my Mongodb running in docker-compose, from different machines (that are not connected to the same network/internet provider).
I want to allow all remote connections.
I don't care about security matters as it's just for practice purposes!
docker-compose.yaml script file:
version: "3.8"
services:
mongodb:
image: mongo
container_name: mongodb
ports:
- 27017:27017
volumes:
- data:/data
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=admin
mongo-express:
image: mongo-express
container_name: mongo-express
restart: always
ports:
- 8081:8081
environment:
- ME_CONFIG_MONGODB_ADMINUSERNAME=admin
- ME_CONFIG_MONGODB_ADMINPASSWORD=admin
- ME_CONFIG_MONGODB_SERVER=mongodb
volumes:
data: {}
networks:
default:
name: mongodb_network
I solved my issue by migrating my data to Atlas cloud.mongodb.com
Answer update: To provide more info as suggested by #nuhkoca, This is the video tutorial to create a mongodb atlas: https://www.youtube.com/watch?v=xrc7dIO_tXk&t=15s And this is the link to the db from my resources file in my Springboot backend api:
server.port=<Port_number>
spring.data.mongodb.uri=<Link_to_mongodb_atlas>

Intergrate elasticsearch with multiple mongodb in docker-compose

I have a implemented a microservice architecture with several servers and databases. I have installed elasticsearch with docker and when I do docker-compose up, everything seems to run fine.
However I would like to integrate the elasticsearch with the several databases (2 mongodb in this sample below) in the system. How do I synch the two mongodb in two different containers with elasticsearch so that I can search them?
client:
container_name: client
stdin_open: true
build:
context: ./client
dockerfile: Dockerfile
restart: always
volumes:
- './client:/app'
ports:
- '1000:3000'
environment:
- NODE_ENV=development
- CHOKIDAR_USEPOLLING=true
weatherdb:
container_name: weather-db
image: mongo
restart: always
ports:
- '2002:27017'
volumes:
- ./weather_service/weather_db:/data/db
networks:
- backend
weather-service:
container_name: weather-service
build: ./weather_service
restart: always
ports:
- "1002:3000"
depends_on:
- weatherdb
links:
- elasticsearch
networks:
- backend
newsdb:
container_name: news-db
image: mongo
restart: always
ports:
- '2003:27017'
volumes:
- ./news_service/news_db:/data/db
networks:
- backend
news-service:
container_name: news-service
build: ./news_service
restart: always
ports:
- "1003:3000"
depends_on:
- newsdb
links:
- elasticsearch
networks:
- backend
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.4.0
container_name: elasticsearch
restart: always
ports:
- 9200:9200
- 9300:9300
environment:
ES_JAVA_OPTS: '-Xms512m -Xmx512m'
network.bind_host: 0.0.0.0
network.host: 0.0.0.0
discovery.type: single-node
volumes:
- ./elasticsearch/esdata:/usr/share/elasticsearch/data
networks:
- backend
Its very simple to just add a elasticsearch docker section in any docker-compose file and start it, all these are independent docker containers and as long as their exposed port on host is not interfering each other and you have the correct configuration in place it should work.
Please refer elasticsearch multi-docker installation using docker file for more info.
NOTE: You have not mentioned what exact issue you are facing, you have mentioned everything ie all docker containers are running file, so please explain in detail what exactly you are trying to solve

Docker: Unable to connect one container with another on same network

I've a following docker-compose.yaml file:
version: "3"
services:
# Database
database:
image: mongo
container_name: database
restart: always
volumes: ["dbdata:/data/db"]
environment:
- MONGO_INITDB_ROOT_USERNAME=myusername
- MONGO_INITDB_ROOT_PASSWORD=mypassword
ports:
- 7000:27017
command: mongod
networks:
- webapp
# Server
server:
container_name: server
restart: always
build: ./server
volumes: ["./server:/var/www", "/var/www/node_modules"]
ports:
- "9000:3000"
depends_on:
- database
networks:
- webapp
networks:
webapp:
driver: bridge
volumes:
dbdata:
Now, I'm able to connect with my mongo database from my local windows machine through mongodb client but my server container is not able connect with database container.
I'm using following connection URI on my server to establish a connection:
mongodb://myusername:mypassword#database:7000/mydatabase
It must be noted that on my local machine I'm using the same URI to connect successfully with the database, the only difference is that I'm using localhost instead of database (container name) on my local machine.
Can you please tell me what I'm doing wrong here?

How to connect to Mongo DB which is in remote server to my Docker?

Hi I have an application where I am using docker-compose. I have large amount of data in the remote mongo DB server which is running on port 28107. How can I connect from my docker-compose to this remote server?
Below is my docker-compose.yml file:
version: '3'
services:
myapp:
# container_name: myapp
restart: always
build: .
ports:
- '52000:52000'
# - '8080:8080'
# - '4300:4300'
# - '4301:4301'
environment:
- MONGO_URL=mongodb://test:test#ip_address:28107/test
# command: ["./wait-for-it.sh", "mongo:28107", "--", "npm", "start"]
links:
- redis
- mongo
mongo:
# container_name: myapp-mongo
image: 'mongo:latest'
ports:
- '28107:28107'
# - '27017:27017'
volumes:
# - ~/Downloads/db_dump_09_01_2020:/data/db
- /data/db
# - /data/configdb
# command: mongod --auth
redis:
# container_name: myapp-redis
restart: always
image: 'redis:4.0.11'
# command: ["redis-server", "--appendonly", "yes"]
depends_on:
- helper
sysctls:
- net.core.somaxconn=511
ports:
- '6379:6379'
helper:
image: alpine
command: sh -c "echo never > /sys/kernel/mm/transparent_hugepage/enabled"
privileged: true
In the above code, in the environment parameter, I have mentioned the remote mongo DB server url. I have all the data in that url. I don't want to export that data in my localhost and mount it in my docker container but instead, I would like to directly link my docker container to that remote mongo DB server.
How can I do it ? I am new to the docker concepts.
Seems like you are trying to connect to an external MongoDb server on a network different from your docker network. Then you shouldn't need.
mongo:
# container_name: myapp-mongo
image: 'mongo:latest'
ports:
- '28107:28107'
# - '27017:27017'
volumes:
# - ~/Downloads/db_dump_09_01_2020:/data/db
- /data/db
# - /data/configdb
# command: mongod --auth
You should only need to provide the necessary environment settings of the remote MongoDb server to myapp.

Making migrations for hasura container and running console

I want to make a developer offline space to develop my database with hasura
I know the existence of the container with the tag cli-migrations, but the command:
hasura-cli console
doesn't work for accessing outside of the container.
My configuration for the docker-compose.yml is:
version: '3'
services:
hasura:
environment:
- HASURA_GRAPHQL_DATABASE_URL=postgres://[some pass]:[some user]#db:5432/[some db]
- HASURA_GRAPHQL_ENABLE_CONSOLE=false
image: hasura/graphql-engine:v1.0.0-rc.1.cli-migrations
container_name: hasura
volumes:
- ./hasura-migrations:/hasura-migrations
networks:
- hasura-db
ports:
- "8081:8080"
- "8082:8081"
restart: always
command: hasura-cli console --console-port 8081 --no-browser
db:
environment:
- POSTGRES_USER=[some user]
- POSTGRES_PASSWORD=[some pass]
- POSTGRES_DB=[some db]
image: postgres:11.4-alpine
container_name: db
restart: always
networks:
- hasura-db
networks:
hasura-db:
There is a Pull request in the hasura graphql project for this issue, but is not merged.
I'm looking for a workaround for this pull request now.
I found a workaround for this!
If I install hasura cli in my machine and use
hasura console --console-port 8080 --endpoint http://127.0.0.1:8081
I can connect to the hasura api and run a console locally.
this is my updated docker-compose.yml
version: '3'
services:
hasura:
environment:
- HASURA_GRAPHQL_DATABASE_URL=postgres://[some pass]:[some user]#db:5432/[some db]
- HASURA_GRAPHQL_ENABLE_CONSOLE=false
image: hasura/graphql-engine:v1.0.0-rc.1.cli-migrations
container_name: hasura
volumes:
- ./hasura-migrations:/hasura-migrations
networks:
- hasura-db
ports:
- "8081:8080"
restart: always
db:
environment:
- POSTGRES_USER=[some user]
- POSTGRES_PASSWORD=[some pass]
- POSTGRES_DB=[some db]
image: postgres:11.4-alpine
container_name: db
restart: always
networks:
- hasura-db
networks:
hasura-db: