docker-compose v3 services on several networks - docker-compose

I use docker-compose v3 file to deploy services on docker swarm mode cluster.
My services are elasticsearch and kibana. I want that kibana was accessible from outside, and that elasticsearch could be accessed by kibana and was not visible and accessible from outside. In order to reach this kind of behavior, I created 2 overlay networks called 'external' and 'elk_only'. I put elasticseach on 'elk_only' network and I placed kibana under 'elk_only' and 'external' networks. And the things do not work. When I go to localhost:5601 (kibana's port), I get a message: 'localhost refused to connect'.
The command I use to deploy services is
docker stack deploy --compose-file=elastic-compose.yml elkstack
The content of elastic-compose.yml file:
version: "3"
services:
elasticsearch:
image: elasticsearch:5.1
expose:
- 9200
networks:
- elk_only
deploy:
restart_policy:
condition: on-failure
kibana:
image: kibana:5.1
ports:
- 5601:5601
volumes:
- ./kibana/kibana.yml:/etc/kibana/kibana.yml
depends_on:
- elasticsearch
networks:
- external
- elk_only
deploy:
restart_policy:
condition: on-failure
networks:
elk_only:
driver: overlay
external:
driver: overlay
The content of kibana.yml is
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://elkstack_elasticsearch:9200"
Could you help me to solve this problem and understand what's going wrong? Any help would be appreciated!

Related

Traefik with multiple docker-compose.yml files

Hello traefik friends.
I just started to look into traefik. All tutorials show how to run one docker-compose.yml file with traefik togather with other containers. I most often have many separate docker-compose.yml files and very much would like to use them with traefik.
so here is my code for traefik container:
version: "3.3"
services:
traefik:
image: "traefik:v2.5"
container_name: "traefik"
command:
#- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.websecure.address=:443"
- "--certificatesresolvers.myresolver.acme.tlschallenge=true"
#- "--certificatesresolvers.myresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--certificatesresolvers.myresolver.acme.email=xxxxxxxxx#gmail.com"
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
ports:
- "443:443"
- "8080:8080"
networks:
- "traefik"
- "external"
volumes:
- "./letsencrypt:/letsencrypt"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
and the other exemplary docker-compose I would like to use with traefik:
version: '3.1'
services:
php:
image: php:7.4-apache
ports:
- 8081:80
volumes:
- ./php/www:/var/www/html/
labels:
- "traefik.enable=true"
- "traefik.http.routers.php.rule=host(`php.xxxxxx.com`)"
- "traefik.http.routers.php.entrypoints=websecure"
- "traefik.http.routers.php.tls.certresolver=myresolver"
unfortunately that doesnt seem to work (when I concat theese to files into one big docker-compose.yml file - it works fine. Could you point me in the right direction?
Traefik needs to be part of the networks for all the services it connects to. For me it works when I set network: host for Traefik. (And then you have to remove ports part.)
I do wonder how safe that is, I can't seem to access the admin interface from another machine, so that's good.
Each docker-compose.yml by default create its own network. So traefik from the traefik network can't access PHP server from some other "php-default" network.
see Compose Networking docs
We have to add the PHP server to the traefik network:
php/docker-compose.yml:
services:
php:
image: php:7.4-apache
# we need to tell the traefik what port is the container listening to
expose:
- 80
volumes:
- ./php/www:/var/www/html/
labels:
- "traefik.enable=true"
- "traefik.http.routers.php.rule=host(`php.xxxxxx.com`)"
- "traefik.http.routers.php.entrypoints=websecure"
- "traefik.http.routers.php.tls.certresolver=myresolver"
networks:
default:
name: traefik
external: true
OR if you want to have other networks
services:
php:
...
networks:
- traefik
...
networks:
traefik:
external: true
Note there is not defined port property, instead there is expose. The port exposes ports on the host, the expose act as mere documentation (see this Q) but the traefik read it.
And because of that, I think that in your traefik/docker-compose.yml the external network is unnecessary.

Traefik 2 Gateway Timeout

So I have the following docker-compose.yml
version: "3.7"
services:
roundclinic-mysql:
image: mysql:5.7
networks:
- spring-boot-mysql-network
environment:
- MYSQL_DATABASE=
- MYSQL_USER=
- MYSQL_PASSWORD=
- MYSQL_ROOT_PASSWORD=
volumes:
- ./mysqldata:/var/lib/mysql:rw,delegated
ports:
- "3306:3306"
web-service:
image: roundclinic/roundclinic:latest
networks:
- spring-boot-mysql-network
- traefik-network
depends_on:
- roundclinic-mysql
ports:
- 8080:8080
environment:
- "SPRING_PROFILES_ACTIVE=dev"
links:
- roundclinic-mysql
labels:
- "--providers.docker.network=traefik_default"
- "traefik.enable=true"
- "traefik.http.routers.roundclinic.rule=Host(`api-dev.roundclinic.app`)"
- "traefik.http.routers.roundclinic.entrypoints=web"
- "traefik.http.services.cal.loadbalancer.server.port=8080"
traefik:
image: "traefik:v2.2"
container_name: "traefik"
command:
- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "traefik.docker.network=traefik-network"
ports:
- "80:80"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
networks:
traefik-network:
driver: bridge
external: true
spring-boot-mysql-network:
driver: bridge
volumes:
my-db:
Spring boot starts up fine and can connect to mysql.
When I connect to http://api-dev.roundclinic.app:8080/../ I can hit my application just fine
When I connect to http://api-dev.roundclinic.app/../ I get a gateway timeout. I can see in the traefik logs that it's forwarding the request to what seems to be the correct IP and port, but nothing hits the actual application. I'm not sure what's going on here. Any help?
When accessing port 8080 you are bypassing Traefik and directly access your application, correct?
Generally speaking the Traefik labels look good. Entrypoint, Port and Host are defined, router and service port are present. These are usually all the requirements for Docker-based setups.
One thing that I noticed is that the traefik container uses "traefik.docker.network=traefik-network", but your web app uses:
"--providers.docker.network=traefik_default".
I am not sure if traefik_default is something that traefik provides but that mismatch in network names might be the issue.
I can't test if that is the problem but that would be the first thing to check.
One way would be to simplify your config but just always using the networks key from docker compose instead of mixing it with labels and arguments.

traefik v2.2 help using only docker-compose router service entrypoint

Started learning about docker, traefik for playing in home.
Aim: Put everything all together in docker-compose.yml and .env files, understand basics, comment accordingly.
Want to get dashboard from traefik.test.local/dashboard rather test.local:8080, similarly api should be accessed from traefik.test.local/api. So that don't have to think about port numbers.
added lines to /etc/hosts
127.0.0.1 test.local
127.0.0.1 traefik.test.local
docker-compose.yml
version: "3.7"
services:
traefik:
# The official v2 Traefik docker image
image: traefik:v2.2
# Lets name the container
container_name: traefik
command:
# Enables the web UI
- "--api.insecure=true"
# Tells Traefik to listen to docker
- "--providers.docker"
ports:
# The HTTP port
- "80:80"
# The Web UI (enabled by --api.insecure=true)
- "8080:8080"
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
#labels:
#- "traefik.http.routers.router.rule=Host(`traefik.test.local/dashboard`)"
#- "traefik.http.routers.router.rule=Host(`traefik.test.local/api`)"
restart:
always
Not able to understand how to connect from router to services. Also correct me if I am wrong anywhere. Thank you.
PS: OS: kde-neon
you can achieve this using the following definition, you need to add labels for the routers and service and not only the router
proxy:
image: traefik:v2.1
command:
- '--providers.docker=true'
- '--entryPoints.web.address=:80'
- '--entryPoints.metrics.address=:8082'
- '--providers.providersThrottleDuration=2s'
- '--providers.docker.watch=true'
- '--providers.docker.swarmMode=true'
- '--providers.docker.swarmModeRefreshSeconds=15s'
- '--providers.docker.exposedbydefault=false'
- '--providers.docker.defaultRule=Host("traefik.lvh.me")'
- '--accessLog.bufferingSize=0'
- '--api=true'
- '--api.dashboard=true'
- '--api.insecure=true'
- '--ping.entryPoint=web'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock:ro'
ports:
- '80:80'
- '8080:8080'
restart:
always
deploy:
labels:
- traefik.enable=true
- traefik.docker.network=monitoring
- traefik.http.services.traefik-dashboard.loadbalancer.server.port=8080
- traefik.http.routers.traefik-dashboard.rule=Host(`dashboard.traefik.lvh.me`)
- traefik.http.routers.traefik-dashboard.service=traefik-dashboard
- traefik.http.routers.traefik-dashboard.entrypoints=web
- traefik.http.services.traefik-api.loadbalancer.server.port=80
- traefik.http.routers.traefik-api.rule=Host(`api.traefik.lvh.me`)
- traefik.http.routers.traefik-api.service=traefik-api
- traefik.http.routers.traefik-api.entrypoints=web
logging:
driver: json-file
options:
'max-size': '10m'
'max-file': '5'
also if you use lvh.me domain you not need to edit /etc/hosts

How can I call Kibana using zuul

I have multiple services say Node-Service running on docker. Along with that, I have EFK Stack also running as a Docker Container.
The Node-Services are getting register on Eureka and getting called from the external network using Api-Gateway (Zuul).
My question is
How can I call Kibana (Service among EFK) using zuul routes?
Is it possible to register Kibana on Eureka ??
Here Is my docker-compose to run EFK
version: '2'
services:
elasticsearch:
image: elasticsearch
expose:
- 9200
ports:
- "9200:9200"
fluentd:
build: ./fluentd
volumes:
- ./fluentd/conf:/fluentd/etc
links:
- "elasticsearch"
ports:
- "24224:24224"
- "24224:24224/udp"
kibana:
image: kibana
links:
- "elasticsearch"
ports:
- "9201:5601"
Below is the snippt for docker-ps
Please Suggest the solution

How to name/alias a network in Docker swarm Stack

I would like to give an alias to my network in docker-swarm stack file.
Currently, the stack file looks like
version: '3'
networks:
mybridge:
services:
web:
restart: always
build: ./web
image: shivanand3939/web
expose:
- "8000"
volumes:
- ./output:/usr/src/app/static
command: /usr/local/bin/gunicorn -w 2 -b :8000 --access-logfile - classifierv2RestEndPoint_ridge_NB:create_app()
networks:
mybridge:
aliases:
- web
deploy:
replicas: 2
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
nginx:
restart: always
build: ./nginx/
image: shivanand3939/nginx
ports:
- "80:80"
volumes:
- /www/static
networks:
- mybridge
deploy:
replicas: 1
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
viz:
image: dockersamples/visualizer
ports:
- 8080:8080/tcp
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints: [node.role == manager]
when I run this file
docker stack deploy -c docker-stack.yml classifierbot
I get the following output
Creating network classifierbot_default
Creating network classifierbot_mybridge
Creating service classifierbot_viz
Creating service classifierbot_web
Creating service classifierbot_nginx
The network name is changed to classifierbot_mybridge from mybridge. Thus the 2 services nginx and web are not able to communicate among themselves.
So, my question is how can I give an alias to my network can also be referred as mybridge
Edit:
One way is add
networks:
mybridge:
external:
name: mybridge
and create mybridge outside the stack file but it defeats the purpose right? as I am unable to do everything in a single stack file
I am assuming by "alias" a network, you mean "give a service an alias" within a network. Because the alias of your network is given successfully. In your case, "mybridge".
Giving a service a network alias (what we usually think of as domain name) You are normally supposed to be able to do this in your services: block
nginx:
networks:
mybridge:
aliases: # https://github.com/moby/moby/issues/38066
- nginx
This is due to a bug that has not yet been resolved. There are tickets open for this:
https://github.com/moby/moby/issues/38066
https://github.com/moby/moby/issues/30313
For now, I use docker service to do this until it gets supported by the multi-service variant, docker stack up|deploy.
--hostname $SERVICE_NAME
--name $SERVICE_NAME