I would like to give an alias to my network in docker-swarm stack file.
Currently, the stack file looks like
version: '3'
networks:
mybridge:
services:
web:
restart: always
build: ./web
image: shivanand3939/web
expose:
- "8000"
volumes:
- ./output:/usr/src/app/static
command: /usr/local/bin/gunicorn -w 2 -b :8000 --access-logfile - classifierv2RestEndPoint_ridge_NB:create_app()
networks:
mybridge:
aliases:
- web
deploy:
replicas: 2
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
nginx:
restart: always
build: ./nginx/
image: shivanand3939/nginx
ports:
- "80:80"
volumes:
- /www/static
networks:
- mybridge
deploy:
replicas: 1
update_config:
parallelism: 2
delay: 10s
restart_policy:
condition: on-failure
viz:
image: dockersamples/visualizer
ports:
- 8080:8080/tcp
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints: [node.role == manager]
when I run this file
docker stack deploy -c docker-stack.yml classifierbot
I get the following output
Creating network classifierbot_default
Creating network classifierbot_mybridge
Creating service classifierbot_viz
Creating service classifierbot_web
Creating service classifierbot_nginx
The network name is changed to classifierbot_mybridge from mybridge. Thus the 2 services nginx and web are not able to communicate among themselves.
So, my question is how can I give an alias to my network can also be referred as mybridge
Edit:
One way is add
networks:
mybridge:
external:
name: mybridge
and create mybridge outside the stack file but it defeats the purpose right? as I am unable to do everything in a single stack file
I am assuming by "alias" a network, you mean "give a service an alias" within a network. Because the alias of your network is given successfully. In your case, "mybridge".
Giving a service a network alias (what we usually think of as domain name) You are normally supposed to be able to do this in your services: block
nginx:
networks:
mybridge:
aliases: # https://github.com/moby/moby/issues/38066
- nginx
This is due to a bug that has not yet been resolved. There are tickets open for this:
https://github.com/moby/moby/issues/38066
https://github.com/moby/moby/issues/30313
For now, I use docker service to do this until it gets supported by the multi-service variant, docker stack up|deploy.
--hostname $SERVICE_NAME
--name $SERVICE_NAME
Related
When trying to deploy my app on Docker swarm I have two services: NGINX to serve static files and app to compile some static files. To run static files compilation I'm using entrypoint in Compose file.
docker-compose.yml:
version: "3.9"
services:
nginx:
image: nginx
healthcheck:
test: curl --fail -s http://localhost:80/lib/tether/examples/viewport/index.html || exit 1
interval: 1m
timeout: 5s
retries: 3
volumes:
- /www:/usr/share/nginx/html/
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.role == manager
ports:
- "8000:80"
depends_on:
- client
client:
image: my-client-image:latest
restart: "no"
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.role == manager
volumes:
- /www:/app/www
entrypoint: /entrypoint.sh
entrypoint.sh
./node_modules/.bin/gulp compilescss
I tried adding restart: "no" in my service, but service is restarted on entrypoint completion anyway
Docker 23.0.0 is now out. As such you have two options:
stack files now support swarm jobs. Swarm understands that these run to completion. i.e. mode: replicated-job.
Docker Compose V3 Reference makes it clear that "restart:" applies to compose and "deploy.restart_policy.condition: on-failure" is the equivalent swarm statement.
I'm a beginner and Im a bit confused about how traefik works...
I want to use the app freqtrade (trading bot) as a docker service and replicate it with different type of configuration, if you have 5min you can go check this guy I want to do the same thing...
But I don't understant why I can't see my app running with traefik :
What I did :
Configure my domain to my server like that :
server config
And on this machine I create a docker swarm and the treafik service with this tutorial and then, my docker compose file look like that :
```
version: '3.3'
services:
traefik:
# Use the latest v2.2.x Traefik image available
image: traefik:v2.2
ports:
# Listen on port 80, default for HTTP, necessary to redirect to HTTPS
- 80:80
# Listen on port 443, default for HTTPS
- 443:443
networks:
- traefik-public
deploy:
placement:
constraints:
# Make the traefik service run only on the node with this label
# as the node with it has the volume for the certificates
- node.labels.traefik-public.traefik-public-certificates == true
labels:
# Enable Traefik for this service, to make it available in the public network
- traefik.enable=true
# Use the traefik-public network (declared below)
- traefik.docker.network=traefik-public
# Use the custom label "traefik.constraint-label=traefik-public"
# This public Traefik will only use services with this label
# That way you can add other internal Traefik instances per stack if needed
- traefik.constraint-label=traefik-public
# admin-auth middleware with HTTP Basic auth
# Using the environment variables USERNAME and HASHED_PASSWORD
- traefik.http.middlewares.admin-auth.basicauth.users=${USERNAME?Variable not set}:${HASHED_PASSWORD?Variable not set}
# https-redirect middleware to redirect HTTP to HTTPS
# It can be re-used by other stacks in other Docker Compose files
- traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
- traefik.http.middlewares.https-redirect.redirectscheme.permanent=true
# traefik-http set up only to use the middleware to redirect to https
# Uses the environment variable DOMAIN
- traefik.http.routers.traefik-public-http.rule=Host(`${DOMAIN?Variable not set}`)
- traefik.http.routers.traefik-public-http.entrypoints=http
- traefik.http.routers.traefik-public-http.middlewares=https-redirect
# traefik-https the actual router using HTTPS
# Uses the environment variable DOMAIN
- traefik.http.routers.traefik-public-https.rule=Host(`${DOMAIN?Variable not set}`)
- traefik.http.routers.traefik-public-https.entrypoints=https
- traefik.http.routers.traefik-public-https.tls=true
# Use the special Traefik service api#internal with the web UI/Dashboard
- traefik.http.routers.traefik-public-https.service=api#internal
# Use the "le" (Let's Encrypt) resolver created below
- traefik.http.routers.traefik-public-https.tls.certresolver=le
# Enable HTTP Basic auth, using the middleware created above
- traefik.http.routers.traefik-public-https.middlewares=admin-auth
# Define the port inside of the Docker service to use
- traefik.http.services.traefik-public.loadbalancer.server.port=8080
volumes:
# Add Docker as a mounted volume, so that Traefik can read the labels of other services
- /var/run/docker.sock:/var/run/docker.sock:ro
# Mount the volume to store the certificates
- traefik-public-certificates:/certificates
command:
# Enable Docker in Traefik, so that it reads labels from Docker services
- --providers.docker
# Add a constraint to only use services with the label "traefik.constraint-label=traefik-public"
- --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
# Do not expose all Docker services, only the ones explicitly exposed
- --providers.docker.exposedbydefault=false
# Enable Docker Swarm mode
- --providers.docker.swarmmode
# Create an entrypoint "http" listening on port 80
- --entrypoints.http.address=:80
# Create an entrypoint "https" listening on port 443
- --entrypoints.https.address=:443
# Create the certificate resolver "le" for Let's Encrypt, uses the environment variable EMAIL
- --certificatesresolvers.le.acme.email=${EMAIL?Variable not set}
# Store the Let's Encrypt certificates in the mounted volume
- --certificatesresolvers.le.acme.storage=/certificates/acme.json
# Use the TLS Challenge for Let's Encrypt
- --certificatesresolvers.le.acme.tlschallenge=true
# Enable the access log, with HTTP requests
- --accesslog
# Enable the Traefik log, for configurations and errors
- --log
# Enable the Dashboard and API
- --api
volumes:
# Create a volume to store the certificates, there is a constraint to make sure
# Traefik is always deployed to the same Docker node with the same volume containing
# the HTTPS certificates
traefik-public-certificates:
networks:
traefik-public:
driver: overlay
attachable: true
```
And deploy it :
docker stack deploy -c traefik.yml traefik
After that traefik works fine. Why I can't see the port 8080 in my entrypoint ? is it important for others services ?
Entrypoint traefik
I try to disable the firewall in configuration of the server and also do ufw allow 8080 but nothing change...
I create my a application like I create traefik service with this docker-compose file :
---
version: '3'
networks:
traefik_traefik-public:
external: true
services:
freqtrade:
image: freqtradeorg/freqtrade:stable
# image: freqtradeorg/freqtrade:develop
# Use plotting image
# image: freqtradeorg/freqtrade:develop_plot
# Build step - only needed when additional dependencies are needed
# build:
# context: .
# dockerfile: "./docker/Dockerfile.custom"
restart: unless-stopped
container_name: freqtrade
volumes:
- "./user_data:/freqtrade/user_data"
# Expose api on port 8080 (localhost only)
# Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
# before enabling this.
networks:
- traefik_traefik-public
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.role == manager
restart_policy:
condition: on-failure
delay: 5s
command: >
trade
--logfile /freqtrade/user_data/logs/freqtrade.log
--db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
--config /freqtrade/user_data/config.json
--strategy SampleStrategy
labels:
- traefik.http.routers.bot001.tls=true'
- traefik.http.routers.bot001.rule=Host(`bot001.bots.lordgoliath.com`)'
- traefik.http.services.bot001.loadbalancer.server.port=8080'
and this is a part of the configuation file of the bot (to access to the UI)
"api_server": {
"enabled": true,
"enable_openapi": true,
"listen_ip_address": "0.0.0.0",
"listen_port": 8080,
"verbosity": "info",
"jwt_secret_key": "somethingrandom",
"CORS_origins": ["https://bots.lordgoliath.com"],
"username": "api",
"password": "api"
},
then :
docker stack deploy -c docker-compose.yml freqtrade
So I have that :
goliath#localhost:~/freqtrade_test/user_data$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
nkvpjjztjibg freqtrade_freqtrade replicated 1/1 freqtradeorg/freqtrade:stable
6qryu28ute9i traefik_traefik replicated 1/1 traefik:v2.2 *:80->80/tcp, *:443->443/tcp
I see the bot running with the command docker service logs freqtrade_freqtrade but
when I try to go on my domain to see it have only the Traefik dashboard and can't see anything else running.
traefik http
traefik https
how I can see my app freqtrade running ? how can I access to the bot UI via my domain ?
Thanks !
Sorry for my bad English I hope this is clear enough to understand my problem
UPDATE
docker service inspect --pretty freqtrade_freqtrade
ID: o6bpaso69i9n6etybtj09xsqi
Name: ft1_freqtrade
Labels:
com.docker.stack.image=freqtradeorg/freqtrade:stable
com.docker.stack.namespace=ft1
Service Mode: Replicated
Replicas: 1
Placement:
Constraints: [node.role == manager]
UpdateConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Update order: stop-first
RollbackConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Rollback order: stop-first
ContainerSpec:
Image: freqtradeorg/freqtrade:stable#sha256:3b2f2acb5b9cfedaa7b07cf56af01d1a750bce4c3054bdbaf40ac27935c984eb
Args: trade --logfile /freqtrade/user_data/logs/freqtrade.log --db-url sqlite:////freqtrade/user_data/tradesv3.sqlite --config /freqtrade/user_data/config.json --strategy SampleStrategy
Mounts:
Target: /freqtrade/user_data
Source: /home/goliath/freqtrade_test/user_data
ReadOnly: false
Type: bind
Resources:
Networks: traefik_traefik-public
Endpoint Mode: vip
UPDATE NEW docker-compose.yml
---
version: '3'
networks:
traefik_traefik-public:
external: true
services:
freqtrade:
image: freqtradeorg/freqtrade:stable
# image: freqtradeorg/freqtrade:develop
# Use plotting image
# image: freqtradeorg/freqtrade:develop_plot
# Build step - only needed when additional dependencies are needed
# build:
# context: .
# dockerfile: "./docker/Dockerfile.custom"
restart: unless-stopped
container_name: freqtrade
volumes:
- "./user_data:/freqtrade/user_data"
# Expose api on port 8080 (localhost only)
# Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
# before enabling this.
networks:
- traefik_traefik-public
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.role == manager
restart_policy:
condition: on-failure
delay: 5s
labels:
- 'traefik.enabled=true'
- 'traefik.http.routers.bot001.tls=true'
- 'traefik.http.routers.bot001.rule=Host(`bot001.bots.lordgoliath.com`)'
- 'traefik.http.services.bot001.loadbalancer.server.port=8080'
command: >
trade
--logfile /freqtrade/user_data/logs/freqtrade.log
--db-url sqlite:////freqtrade/user_data/tradesv3.sqlite
--config /freqtrade/user_data/config.json
--strategy SampleStrategy
UPDATE docker network ls
goliath#localhost:~/freqtrade_test$ docker network ls
NETWORK ID NAME DRIVER SCOPE
003e00401b5d bridge bridge local
9f3d9a222928 docker_gwbridge bridge local
09a33afad0c9 host host local
r4u268yenm5u ingress overlay swarm
bed40e4a5c62 none null local
qo9w45gitke5 traefik_traefik-public overlay swarm
This is the minimal config you need to integrate in order to see the traefik dashboard on localhost:8080
version: "3.9"
services:
traefik:
image: traefik:latest
command: |
--api.insecure=true
ports:
- 8080:8080
Then, your minimal configuration to get traefik to route example.com to itself:
version: "3.9"
networks:
public:
attachable: true
name: traefik
services:
traefik:
image: traefik:latest
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
command: |
--api.insecure=true
--providers.docker.exposedbydefault=false
--providers.docker.swarmmode
--providers.docker.network=traefik
ports:
- 80:80
networks:
- public
deploy:
labels:
traefik.enable: "true"
traefik.http.routers.traefik.rule: Host(`example.com`)
traefik.http.services.traefik.loadbalancer.server.port: 8080
Now, minimal https support - using Traefik self signed certs to start with. Note that we configure tls on the https entrypoint, which means traefik implicitly creates http and https variants for each router.
version: "3.9"
networks:
public:
attachable: true
name: traefik
services:
traefik:
image: traefik:latest
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
command: |
--api.insecure=true
--providers.docker.exposedbydefault=false
--providers.docker.swarmmode
--providers.docker.network=traefik
--entrypoints.http.address=:80
--entrypoints.https.address=:443
--entrypoints.https.http.tls=true
deploy:
placement:
constraints:
- node.role == manager
ports:
# - 8080:8080
- 80:80
- 443:443
networks:
- public
deploy:
labels:
traefik.enable: "true"
traefik.http.routers.traefik.rule: Host(`example.com`)
traefik.http.services.traefik.loadbalancer.server.port: 8080
At this point, gluing in your le config should be simple.
Your freqtrade stack compose would need to be this. If this is a single node swarm, just omit the placement constraints, but when the swarm is large enough to have workers, then tasks that don't need to be on managers should explicitly be kept on workers.
Traefik needs to talk to the swarm api over the docker socket, which is on manager nodes only, which is why it must be node.role==manager.
version: "3.9"
networks:
traefik:
external: true
services:
freqtrade:
image: freqtradeorg/freqtrade:stable
command: ...
volumes: ...
networks:
- traefik
deploy:
placement:
constraints:
- node.role == worker
restart_policy:
max_attempts: 5
labels:
traefik.enabled: "true"
traefik.http.routers.bot001.rule: Host(`bot001.bots.lordgoliath.com`)
traefik.http.services.bot001.loadbalancer.server.port: 8080
I made 2 yml and when i run docker-compose -f postgresql.yml up its starts ok
and then when i run docker-compose -f postgresql2.yml up first exist code 0.
Is it even possible to run same image twice?
My main purpose to run same web app source twice with different db on the same server pc.
1 web app source 2 instances with self db each on one server(maybe its clearer definition).
Maybe there is better approach and I do and think everything in wrong way.
# This configuration is intended for development purpose, it's **your** responsibility to harden it for production
version: '3.8'
services:
freshhipster-postgresql:
image: postgres:13.1
environment:
- POSTGRES_USER=FreshHipster
- POSTGRES_PASSWORD=
- POSTGRES_HOST_AUTH_METHOD=trust
# If you want to expose these ports outside your dev PC,
# remove the "127.0.0.1:" prefix
ports:
- 5432:5432
and this no big difference
postgresql2.yml
# This configuration is intended for development purpose, it's **your** responsibility to harden it for production
version: '3.8'
services:
freshhipster-postgresql:
image: postgres:13.1
container_name: postgres2
volumes:
- pgdata:/var/lib/postgresql/data_vol2/
environment:
- POSTGRES_USER=FreshHipster
- POSTGRES_PASSWORD=
- POSTGRES_HOST_AUTH_METHOD=trust
# If you want to expose these ports outside your dev PC,
# remove the "127.0.0.1:" prefix
ports:
- 5433:5432
volumes:
pgdata:
external: true
Just use another service name freshhipster-postgresql2 on postgresql2.yml
version: '3.8'
services:
freshhipster-postgresql2:
image: postgres:13.1
container_name: postgres2
volumes:
- pgdata:/var/lib/postgresql/data_vol2/
environment:
- POSTGRES_USER=FreshHipster
- POSTGRES_PASSWORD=
- POSTGRES_HOST_AUTH_METHOD=trust
# If you want to expose these ports outside your dev PC,
# remove the "127.0.0.1:" prefix
ports:
- 5433:5432
volumes:
pgdata:
external: true
This is my Docker compose/stack file. When I deploy on a single node, everything works fine, but when I deploy on multiple nodes I get the following error:
invalid mount config for type bind bind source path does not exist
version: '3'
services:
shinyproxy:
build: /etc/shinyproxy
deploy:
replicas: 3
user: root:root
hostname: shinyproxy
image: shinyproxy-example
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 5000:5000
networks:
- proxynetwork
mysql:
image: mysql
deploy:
replicas: 3
volumes:
- /mysqldata:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root_password
MYSQL_DATABASE: keycloak
MYSQL_USER: keycloak
MYSQL_PASSWORD: password
networks:
- proxynetwork
keycloak:
deploy:
replicas: 3
image: jboss/keycloak
volumes:
- /etc/letsencrypt/live/ds-gym.de/fullchain.pem:/etc/x509/https/tls.crt
- /etc/letsencrypt/live/ds-gym.de/privkey.pem:/etc/x509/https/tls.key
#- /theme/govuk-social-providers/:/opt/jboss/keycloak/themes/govuk-social-providers/
environment:
- PROXY_ADDRESS_FORWARDING=true
- KEYCLOAK_USER=myadmin
- KEYCLOAK_PASSWORD=mypassword
ports:
- 8443:8443
networks:
- proxynetwork
networks:
proxynetwork:
external: true
I understand that the volumes path is expected on every other node too, but I think this is a very bad practice and my other 2 nodes are anyway just workers. How can I solve that problem? Hopefully there is a solution which allows me to keep the volumes, since I use the same file for docker-compose build to build my images.
Can someone help me?
Thank you :-)
If it is possible you could restrict this service to the node that has the required host path's using placement constraints. However I'm guessing that that's not an option in this use case.
Host mounted volumes should really not be used in a swarm deployment as it would cause redundant data in the filesystems between the nodes. (All files need to be present on all nodes).
One solution would be to implement NFS volumes:
volumes:
example:
driver_opts:
type: "nfs"
o: "addr=<NFS_SERVER_IP>,nolock,soft,rw"
device: ":/docker/path/to/configs"
This solution requires you to host a NFS-Server though. Also keep in mind that this approach is fine for configs but should not be used for file systems that need to provide high performance access.
Regarding your question about keeping your docker-compose file the same across environments: While it is technically possible to do so, most modern projects consist of a base compose file as well as an environment specific override for volumes,networks,images etc.
In a swarm your services will be deployed randomly on your available nodes.
I suppose your "to be mounted directory" is on the manager node, so deploy the wanted service on the manager node like so.
deploy:
placement:
constraints:
- node.role == manager
I use docker-compose v3 file to deploy services on docker swarm mode cluster.
My services are elasticsearch and kibana. I want that kibana was accessible from outside, and that elasticsearch could be accessed by kibana and was not visible and accessible from outside. In order to reach this kind of behavior, I created 2 overlay networks called 'external' and 'elk_only'. I put elasticseach on 'elk_only' network and I placed kibana under 'elk_only' and 'external' networks. And the things do not work. When I go to localhost:5601 (kibana's port), I get a message: 'localhost refused to connect'.
The command I use to deploy services is
docker stack deploy --compose-file=elastic-compose.yml elkstack
The content of elastic-compose.yml file:
version: "3"
services:
elasticsearch:
image: elasticsearch:5.1
expose:
- 9200
networks:
- elk_only
deploy:
restart_policy:
condition: on-failure
kibana:
image: kibana:5.1
ports:
- 5601:5601
volumes:
- ./kibana/kibana.yml:/etc/kibana/kibana.yml
depends_on:
- elasticsearch
networks:
- external
- elk_only
deploy:
restart_policy:
condition: on-failure
networks:
elk_only:
driver: overlay
external:
driver: overlay
The content of kibana.yml is
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.url: "http://elkstack_elasticsearch:9200"
Could you help me to solve this problem and understand what's going wrong? Any help would be appreciated!