Docker compose seems to be automatically setting links in hosts files when I start things up:
root#47f648654cc9:/opt/mio# cat /etc/hosts
172.17.1.54 47f648654cc9
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.1.54 mio-events-service.bridge
172.17.1.52 mongodb.bridge
172.17.1.54 mio-events-service
172.17.1.52 mongodb
172.17.1.53 consul
172.17.1.53 consul.bridge
172.17.1.55 registrator
172.17.1.55 registrator.bridge
172.17.1.56 kv-registration
172.17.1.56 kv-registration.bridge
172.17.1.57 rabbitmq
172.17.1.57 rabbitmq.bridge
i.e. I don't have to supply "links:" elements in docker-compose.yml to get links between containers. This seems to be contradictory to the documentation. Any ideas why it's doing this?
Thanks
Nick
UPDATE
Here's my compose file
consul:
image: registry.systems.com/consuld
container_name: consul
ports:
- "8400:8400"
- "8500:8500"
- "53:8600/udp"
environment:
- IP=192.168.99.100
hostname: dockermachine
registrator:
image: gliderlabs/registrator:latest
container_name: registrator
command: consul://consul:8500
volumes:
- "/var/run/docker.sock:/tmp/docker.sock"
links:
- consul
kv-registration:
image: registry.systems.com/nativ/consul-local-kv-registration
container_name: kv-registration
links:
- consul
rabbitmq:
image: rabbitmq
container_name: rabbitmq
ports:
- 4369:4369
- 5672:5672
- 15672:15672
- 25672:25672
mongodb:
image: mongo
container_name: mongodb
ports:
- 27017:27017
- 28017:28017
mio-events-service:
image: registry.systems.com/mio-events-service:1.0.190
container_name: mio-events-service
ports:
- 18113:18113
extra_hosts:
- "dockermachine:192.168.99.100"
I think you're using --x-networking which replaces links, and adds these host entries for every container on the same network.
See http://docs.docker.com/compose/networking/
Related
So, I am running my mongod replica set instances on docker. I am trying to establish the connection from my c++ application. Here is my docker-compose file.
version: '3'
services:
api:
hostname: api
build: .
ports:
- 8000:8000
- 8001:8001
- 8500:8500
depends_on:
- mongo1
- mongo2
- mongo3
mongo1:
hostname: mongo1
container_name: mongo1
image: mongo:4.0-xenial
expose:
- 27017
ports:
- 27011:27017
restart: always
entrypoint: ['/usr/bin/mongod', '--bind_ip_all', '--replSet', 'rs0']
mongo2:
hostname: mongo2
container_name: mongo2
image: mongo:4.0-xenial
expose:
- 27017
ports:
- 27012:27017
restart: always
entrypoint: ['/usr/bin/mongod', '--bind_ip_all', '--replSet', 'rs0']
mongo3:
hostname: mongo3
container_name: mongo3
image: mongo:4.0-xenial
expose:
- 27017
ports:
- 27013:27017
restart: always
entrypoint: ['/usr/bin/mongod', '--bind_ip_all', '--replSet', 'rs0']
I could establish the connection using host ip_address, but as the ip_address changes from network to network, it's not a feasible soultion if each developer needs to change ip_address everytime. Below is the connection string that's worked for me.
auto mongocxx_uri = "mongodb://192.168.18.4:27011,192.168.18.4:27012,192.168.18.4:27013/?replicaSet=rs0";
Below is the desired connection string or something which can be configured once or automatically.
auto mongocxx_uri = "mongodb://localhost:27011,localhost:27012,localhost:27013/?replicaSet=rs0";
But I am looking something with localhost, I can connect to individual instance using localhost but not for replica set.
I have tried to add alias for host names(mongo1, mongo2, mongo3) in /etc/hosts file but
that didn't work for me.
I assumed that docker uses /etc/mongod.conf file and tried change the bind_ip like below but that also did not work for me.
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0
Docker version 19.03.13
MongoDB server version: 4.0.21
Mongocxx 3.6.0 version
Below are the /etc/hosts file on my laptop
127.0.0.1 localhost
127.0.1.1 xxxxxxx
127.0.0.1 mongo1, mongo2, mongo3
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Below is the output from docker ps command
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fea5883e73ae mongo:4.0-xenial "/usr/bin/mongod --b…" 43 minutes ago Up 43 minutes 0.0.0.0:27013->27017/tcp mongo3
5a3846f44dfd mongo:4.0-xenial "/usr/bin/mongod --b…" 43 minutes ago Up 43 minutes 0.0.0.0:27011->27017/tcp mongo1
450f21091acc mongo:4.0-xenial "/usr/bin/mongod --b…" 43 minutes ago Up 43 minutes 0.0.0.0:27012->27017/tcp mongo2
Can someone let me know what am I missing?
Thank you.
I have been trying to dockerize my spring boot application which depends on redis, kafka and mongodb.
Following is the docker-compose.yml:
version: '3.3'
services:
my-service:
image: my-service
build:
context: ../../
dockerfile: Dockerfile
restart: always
container_name: my-service
environment:
KAFKA_CONFLUENT_BOOTSTRAP_SERVERS: kafka:9092
MONGO_HOSTS: mongodb:27017
REDIS_HOST: redis
REDIS_PORT: 6379
volumes:
- /private/var/log/my-service/:/var/log/my-service/
ports:
- 8080:8090
- 1053:1053
depends_on:
- redis
- kafka
- mongodb
portainer:
image: portainer/portainer
command: -H unix:///var/run/docker.sock
restart: always
container_name: portainer
ports:
- 9000:9000
- 9001:8000
volumes:
- /var/run/docker.sock:/var/run/docker.sock
redis:
image: redis
container_name: redis
restart: always
ports:
- 6379:6379
zookeeper:
image: wurstmeister/zookeeper
ports:
- 2181:2181
container_name: zookeeper
kafka:
image: wurstmeister/kafka
ports:
- 9092:9092
container_name: kafka
environment:
KAFKA_CREATE_TOPICS: "cms.entity.change:1:1" # topic:partition:replicas
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_PORT: 9092
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- "zookeeper"
mongodb:
image: mongo:latest
container_name: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME:
MONGO_INITDB_ROOT_PASSWORD:
ports:
- 27017:27017
volumes:
- ./data/db:/data/db
The issue is that this starts up mongo as a STANDALONE instance. So the APIs in my service that persist data are failing as mongo needs to start as a REPLICA_SET.
How can I edit my docker-compose file to start mongo as a REPLICA_SET?
I had the same issue and ended up on this stackoverflow post.
We had a requirement of using official mongoDB docker image (https://hub.docker.com/_/mongo ) and couldn't use bitnami as suggested in Vahid's answer.
This answer isn't exactly what's needed by the question asked and coming in 6 months too late; but it should give directions to someone who need to use the mongoDb standalone replicaset throw away instance for integration testing purpose. If you need to use it in PROD then you'll have to provide environment variables for volumes and auth as per Vahid's answer.
version: '3.7'
services:
mongodb:
image: mongo:latest
container_name: myservice-mongodb
networks:
- myServiceNetwork
expose:
- 27017
command: --replSet singleNodeReplSet
mongodb-replicaset:
container_name: mongodb-replicaset-helper
depends_on:
- mongodb
networks:
- myServiceNetwork
image: mongo:latest
command: bash -c "sleep 5 && mongo --host myservice-mongodb --port 27017 --eval \"rs.initiate()\" && sleep 2 && mongo --host myservice-mongodb --port 27017 --eval \"rs.status()\" && sleep infinity"
my-service:
depends_on:
- mongodb-replicaset
image: myserviceimage
container_name: myservicecontainer
networks:
- myServiceNetwork
environment:
myservice__Database__ConnectionString: mongodb://myservice-mongodb:27017/?connect=direct&replicaSet=singleNodeReplSet&readPreference=primary
myservice__Database__Name: myserviceDb
networks:
myServiceNetwork:
driver: bridge
NOTE: Please look at the way how connection string is passed as env variable to the service depending on mongo replicaset instance. You'd have to ensure that the name used in setting up the mongodb replicaset (in my case singleNodeReplicaSet) is passed on to the service depending on it.
Edited:
my previous answer was far wrong so I changed it. I managed to make it work using 'bitnami/mongodb:4.0'. Not sure if that would help you or not, but maybe it gives you some idea. They have a docker-compose file ready for replicaset mode.
version: '3'
services:
mdb-primary:
image: 'bitnami/mongodb:4.0'
environment:
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-primary
mdb-secondary:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=secondary
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-secondary
mdb-arbiter:
image: 'bitnami/mongodb:4.0'
depends_on:
- mdb-primary
environment:
- MONGODB_PRIMARY_HOST=mdb-primary
- MONGODB_REPLICA_SET_MODE=arbiter
- MONGODB_PRIMARY_ROOT_PASSWORD=somepassword
- MONGODB_REPLICA_SET_KEY=replicasetkey
- MONGODB_ADVERTISED_HOSTNAME=mdb-arbiter
mongo-cli:
image: 'bitnami/mongodb:latest'
don't forget to add volumes and map it to /bitnami on the primary node
the last container, mongo-cli is for testing purposes. So you can connect to the replicaset using the cli, there is an argument about that here if you like to read about it.
$ docker-compose exec mongo-cli bash
$ mongo "mongodb://mdb-primary:27017/test?replicaSet=replicaset"
I have some docker containers such as php, nginx, etc. A also have postgres locally because I have learned that dabatase inside docker container is a bad practise. But I can't connect to local postgres from docker container.
At this moment I have done next
In postgresql.conf I changed listen_addresses
listen_addresses = '*'
In pg_hba.conf I have added next line
host all all 0.0.0.0/0 md5
The I executed next command for iptables
iptables -I INPUT -p tcp -m tcp -s 0.0.0.0 --dport 5432 -j ACCEPT
Then I restarted postgres.
My database configuration
DB_CONNECTION=pgsql
DB_HOST=my_server_ip_address
DB_PORT=5432
DB_DATABASE=mydbname
DB_USERNAME=mydbuser
DB_PASSWORD=mydbpasswd
But i still can't connect to posgresql. At the same moment I can connect to postgres via psql or phpstorm
My docker-compose.yml
version: '3.7'
networks:
backend-network:
driver: bridge
frontend-network:
driver: bridge
services:
&app-service app: &app-service-template
container_name: k4fntr_app
build:
context: ./docker/php-fpm
args:
UID: ${UID?Use your user ID}
GID: ${GID?Use your group ID}
USER: ${USER?Use your user name}
user: "${UID}:${GID}"
hostname: *app-service
volumes:
- /etc/passwd/:/etc/passwd:ro
- /etc/group/:/etc/group:ro
- ./:/var/www/k4fntr
environment:
APP_ENV: "${APP_ENV}"
CONTAINER_ROLE: app
FPM_PORT: &php-fpm-port 9000
FPM_USER: "${UID:-1000}"
FPM_GROUP: "${GID:-1000}"
networks:
- backend-network
&queue-service queue:
<<: *app-service-template
container_name: k4fntr_queue
restart: always
hostname: *queue-service
depends_on:
- app
environment:
CONTAINER_ROLE: queue
&schedule-service schedule:
<<: *app-service-template
container_name: k4fntr_schedule
restart: always
hostname: *schedule-service
depends_on:
- app
environment:
CONTAINER_ROLE: scheduler
&sportlevel-listener sportlevel_listener:
<<: *app-service-template
container_name: k4fntr_sl_listener
restart: always
hostname: *sportlevel-listener
ports:
- "${SPORTLEVEL_LISTEN_PORT}:${SPORTLEVEL_LISTEN_PORT}"
depends_on:
- app
environment:
CONTAINER_ROLE: sl_listener
&php-fpm-service php-fpm:
<<: *app-service-template
container_name: k4fntr_php-fpm
user: 'root:root'
restart: always
hostname: *php-fpm-service
ports: [*php-fpm-port]
entrypoint: /fpm-entrypoint.sh
command: php-fpm --nodaemonize
networks:
- backend-network
- frontend-network
echo-server:
container_name: k4fntr_echo
image: oanhnn/laravel-echo-server
volumes:
- ./:/app
environment:
GENERATE_CONFIG: "false"
depends_on:
- app
ports:
- "6001:6001"
networks:
- backend-network
- frontend-network
nginx:
container_name: k4fntr_nginx
image: nginx
volumes:
- ./docker/nginx/config:/etc/nginx/conf.d
- ./:/var/www/k4fntr
depends_on:
- *php-fpm-service
ports:
- "${NGINX_LISTEN_PORT}:80"
networks:
- frontend-network
redis:
container_name: k4fntr_redis
image: redis
restart: always
command: redis-server
volumes:
- ./docker/redis/config/redis.conf:/usr/local/etc/redis/redis.conf
- ./docker/redis/redis-data:/data:rw
ports:
- "16379:6379"
networks:
- backend-network
I'm trying to access to my mongo database on docker with adminmongo.
Here's my docker-compose.yml
version: '3'
services:
mongo:
image: mongo
volumes:
- ~/data:/data/db
restart: always
expose:
- 6016
adminmongo:
image: mrvautin/adminmongo
expose:
- 1234
links:
- mongo:mongo
When i do a docker-compose up everything works fine, adminmongo also return me this : adminmongo_1_544d9a6f954c | adminMongo listening on host: http://localhost:1234
But when i go to localhost:1234 my navigator is telling me this page doesn't exist.
Here's what a docker ps return me :
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9c27d4a89254 mrvautin/adminmongo "/bin/sh -c 'node ap…" 38 seconds ago Up 33 seconds 1234/tcp iris_adminmongo_1_544d9a6f954c
2a7496a8c56a mongo "docker-entrypoint.s…" 40 minutes ago Up 38 seconds 6016/tcp, 27017/tcp iris_mongo_1_7f00356a3adc
I've found 2 issues here:
1st: Exposing a port is not enough. expose is just documentation, you need to publish (bind) a port to the host to be reachable. This is how it's done:
ports:
- 1234:1234
2nd: you have to configure adminmongo to listen to 0.0.0.0 because by default it starts listening on 127.0.0.1 and this makes it accessible only inside the container itself. From the documentation page you've included in your question, the Configuration section states that this can be done by passing an environment variable:
All above parameters are usable through the environment which makes it very handy to when using adminMongo as a docker container! just run docker run -e HOST=yourchoice -e PORT=1234 ...
Since you are using docker-compose, this is done by the following:
environment:
- HOST=0.0.0.0
Working example:
version: '3'
services:
mongo:
image: mongo
volumes:
- ~/data:/data/db
restart: always
expose:
- 6016
adminmongo:
image: mrvautin/adminmongo
ports:
- 1234:1234
environment:
- HOST=0.0.0.0
Example of docker-compose works :
version: '3'
services:
server:
container_name: docker_api_web_container
image: docker_api_web
build: .
volumes:
- ./src:/usr/src/node-app/src
- ./package.json:/usr/src/node-app/package.json
environment:
- ENV=DEVELOPMENT
- PORT=4010
ports:
- '9000:4010'
depends_on:
- 'mongo'
mongo:
container_name: docker_mongo_container
image: 'mongo'
ports:
- '27017:27017'
adminmongo:
container_name: docker_adminmongo_container
image: mrvautin/adminmongo
links: ['mongo:mongo']
environment:
- HOST=0.0.0.0
ports:
- '1234:1234'
You have to expose your service to the outside world like this:
version: '3'
services:
mongo:
image: mongo
volumes:
- ~/data:/data/db
restart: always
adminmongo:
image: mrvautin/adminmongo
ports:
- 1234:1234
Now you can access your adminmongo by http://localhost:1234.
And you don't have to use links here.Since compose creates a network and joins all services in the compose files. You can access other containers with their service names.
My docker-compose file looks like this:
version: '2'
services:
explore:
image: explore
build:
context: ./Explore
dockerfile: VsDockerfile
environment:
- "ElasticUrl=http://localhost:9200"
- "RabbitMq/Host=localhost"
- "RabbitMq/Username=guest"
- "RabbitMq/Password=guest"
networks:
- localnet
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
container_name: elasticsearch
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ports:
- 9200:9200
volumes:
- ./esdata:/usr/share/elasticsearch/data
networks:
- localnet
rabbit:
image: rabbitmq:3.6.7-management
hostname: rabbit
ports:
- 15672:15672
- 5672:5672
networks:
- localnet
networks:
localnet:
external:
name: localnet
If I type http://localhost:15672 in the browser, I get the rabbitmq interface, but if I tries to connect from my Explore project like this:
public SqlToRabbitProcessor(SqlToRabbitRepository sqlToRabbitRepository)
{
_sqlToRabbitRepository = sqlToRabbitRepository;
var factory = new ConnectionFactory
{
HostName = Environment.GetEnvironmentVariable("RabbitMq/Host"),
UserName = Environment.GetEnvironmentVariable("RabbitMq/Username"),
Password = Environment.GetEnvironmentVariable("RabbitMq/Password")
};
var rabbit = factory.CreateConnection();
channel = rabbit.CreateModel();
}
Then it breaks in the line
var rabbit = factory.CreateConnection();
with the error saying
ExtendedSocketException: Connection refused 127.0.0.1:5672
System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
ConnectFailureException: Connection failed
RabbitMQ.Client.EndpointResolverExtensions.SelectOne(IEndpointResolver resolver, Func selector)
BrokerUnreachableException: None of the specified endpoints were reachable
RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, string clientProvidedName)
As my comment under the question suggested, it's because the "localhost" defined in the web application part is it's containers localhost, and not the docker host..
just needed to change
- "ElasticUrl=http://localhost:9200"
- "RabbitMq/Host=localhost"
to
- "ElasticUrl=http://elasticsearch:9200"
- "RabbitMq/Host=rabbit"
I had the same issue with docker-compose.
I solved it by with hostname:
rabbit:
hostname: rabbit
command: sh -c "rabbitmq-plugins enable rabbitmq_management; rabbitmq-server"
image: rabbitmq
environment:
RABBITMQ_DEFAULT_USER: admin
RABBITMQ_DEFAULT_PASS: admin
ports:
- 5672:5672
- 15672:15672
Follow instructions on this post.
Just to benefit people that stumble upon this question. The --link feature is now considered legacy and is a prime candidate to be deprecated by docker.
The easiest way is to use
depends_on:
In order to do this, its recommended to first create a network like so"
docker network create <network_name>
Then use docker-compose up to spawn services that bind with each other. Look at the example below where I've bound my spring-boot app to rabbit-mq. You can clone my repo from here
version: "3.1"
services:
rabbitmq-container:
image: rabbitmq:3.5.3-management
hostname: rabbitmq-container
ports:
- 5673:5673
- 5672:5672
- 15672:15672
networks:
- resolute
resolute-container:
build: .
ports:
- 8080:8080
environment:
- spring_rabbitmq_host=rabbitmq-container
- spring_rabbitmq_port=5672
- spring_rabbitmq_username=guest
- spring_rabbitmq_password=guest
- resolute_rabbitmq_publishQueueName=resolute-run-request
- resolute_rabbitmq_exchange=resolute
depends_on:
- rabbitmq-container
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- resolute
networks:
resolute:
external:
name: resolute
See how I've created a network called resolute and bound the apps to the same network. I've also given my rabbitmq-container a hostname. This is because docker now prepends the container name and that makes it difficult to bind services by name.