I run AirFlow on local machine with docker-compose:
version: '2'
services:
postgresql:
image: bitnami/postgresql:10
volumes:
- postgresql_data:/bitnami/postgresql
environment:
- POSTGRESQL_DATABASE=bitnami_airflow
- POSTGRESQL_USERNAME=bn_airflow
- POSTGRESQL_PASSWORD=bitnami1
- ALLOW_EMPTY_PASSWORD=yes
redis:
image: bitnami/redis:5.0
volumes:
- redis_data:/bitnami
environment:
- ALLOW_EMPTY_PASSWORD=yes
airflow-scheduler:
image: bitnami/airflow-scheduler:1
environment:
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
- AIRFLOW_DATABASE_NAME=bitnami_airflow
- AIRFLOW_DATABASE_USERNAME=bn_airflow
- AIRFLOW_DATABASE_PASSWORD=bitnami1
- AIRFLOW_EXECUTOR=CeleryExecutor
- AIRFLOW_LOAD_EXAMPLES=no
volumes:
- airflow_scheduler_data:/bitnami
- ./airflow/dags:/opt/bitnami/airflow/dags
- ./airflow/plugins:/opt/bitnami/airflow/plugins
airflow-worker:
image: bitnami/airflow-worker:1
environment:
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
- AIRFLOW_EXECUTOR=CeleryExecutor
- AIRFLOW_DATABASE_NAME=bitnami_airflow
- AIRFLOW_DATABASE_USERNAME=bn_airflow
- AIRFLOW_DATABASE_PASSWORD=bitnami1
- AIRFLOW_LOAD_EXAMPLES=no
volumes:
- airflow_worker_data:/bitnami
- ./airflow/dags:/opt/bitnami/airflow/dags
- ./airflow/plugins:/opt/bitnami/airflow/plugins
airflow:
image: bitnami/airflow:1
environment:
- AIRFLOW_FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
- AIRFLOW_DATABASE_NAME=bitnami_airflow
- AIRFLOW_DATABASE_USERNAME=bn_airflow
- AIRFLOW_DATABASE_PASSWORD=bitnami1
- AIRFLOW_USERNAME=user
- AIRFLOW_PASSWORD=password
- AIRFLOW_EXECUTOR=CeleryExecutor
- AIRFLOW_LOAD_EXAMPLES=yes
ports:
- '8080:8080'
volumes:
- ./airflow/dags:/opt/bitnami/airflow/dags
- ./airflow/plugins:/opt/bitnami/airflow/plugins
volumes:
airflow_scheduler_data:
driver: local
airflow_worker_data:
driver: local
airflow_data:
driver: local
postgresql_data:
driver: local
redis_data:
driver: local
But when I sing in the UI interface, I see
"The scheduler does not appear to be running.
The DAGs list may not update, and new tasks will not be scheduled."
Why?
I use official docker images and there is no problem with this.
And another problem - while I don't switch AIRFLOW_LOAD_EXAMPLES=yes or no and restart docker-compose I don't see the updated DAGs' list. (
When I have got for AirFlow 1 docker-compose by Puckel, everything worked: https://github.com/puckel/docker-airflow/blob/master/README.md
Related
I have VM with docker containers in a cloud.
It have 2 containers: wireguard and redmine.
I have LDAP-authorization in redmine.
LDAP-server locates in private LAN (behind NAT), and I have VPN via wireguard to this LAN.
I need add route in Redmine-container so that redmine has access to a private LAN via Wireguard-container.
Now I make it by hand after containers start I write docker-compose exec redmine ip route add 192.168.42.0/23 via 172.20.0.50
Could you advice me, how implement it to my pipeline?
P.S. redmine-container already has entrypoint and cmd directives in Dockerfile.
version: '3.9'
services:
wireguard:
image: linuxserver/wireguard
cap_add:
- NET_ADMIN
- SYS_MODULE
volumes:
- ./wireguard-config:/config
- /lib/modules:/lib/modules
networks:
default:
ipv4_address: 172.20.0.50
sysctls:
- net.ipv4.conf.all.src_valid_mark=1 # for clients mode
restart: unless-stopped
postgres:
image: postgres:14.2-alpine
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
- 'POSTGRES_PASSWORD=MySUperSecret'
- 'POSTGRES_DB=redmine'
redmine:
image: redmine:5.0.1-alpine
cap_add:
- NET_ADMIN
volumes:
- redmine-files:/usr/src/redmine/files
- ./redmine-plugins:/usr/src/redmine/plugins
- ./configuration.yml:/usr/src/redmine/config/configuration.yml
ports:
- 80:3000
depends_on:
- postgres
environment:
- 'REDMINE_DB_POSTGRES=postgres'
- 'REDMINE_DB_DATABASE=redmine'
- 'REDMINE_DB_PASSWORD=MySUperSecret'
- 'REDMINE_PLUGINS_MIGRATE=true'
restart: unless-stopped
networks:
default:
ipam:
config:
- subnet: 172.20.0.0/24
volumes:
postgres-data:
redmine-files:
I solve my problem:
services:
wireguard:
image: linuxserver/wireguard
cap_add:
- NET_ADMIN
- SYS_MODULE
ports:
- 3000:3000
environment:
- TZ=Europe/Moscow
volumes:
- ./wireguard-config:/config
- /lib/modules:/lib/modules
sysctls:
- net.ipv4.conf.all.src_valid_mark=1 # for clients mode
restart: unless-stopped
postgres:
image: postgres:14.2-alpine
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
- 'POSTGRES_PASSWORD=MySUperSecret'
- 'POSTGRES_DB=redmine'
redmine:
image: redmine:5.0.2-alpine
network_mode: service:wireguard
volumes:
- redmine-files:/usr/src/redmine/files
- ./redmine-plugins:/usr/src/redmine/plugins
- ./configuration.yml:/usr/src/redmine/config/configuration.yml
# ports:
# - 80:3000
depends_on:
- postgres
environment:
- 'REDMINE_DB_POSTGRES=postgres'
- 'REDMINE_DB_DATABASE=redmine'
- 'REDMINE_DB_PASSWORD=MySUperSecret'
- 'REDMINE_PLUGINS_MIGRATE=true'
restart: unless-stopped
volumes:
postgres-data:
redmine-files:
I have currently a problem with mongoDB and Docker:
I have this error when I compose up:
ERROR: Named volume "mongo:/data/db:rw" is used in service "mongo" but no declaration was found in the volumes section.
In fact, i realized that i have a volume instead a container but i can't access to the data or manipulate them with the mongo shell.
I have to delete an index from one mongodb collection in the volume.
How i can do that knowing that i have no container but i have a volume instead.
version: '3.1'
services:
proxy:
image: jwilder/nginx-proxy:alpine
restart: always
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- $CERT_DIR:/etc/nginx/certs
app:
image: registry.eksae.fr/talent:${VERSION:-latest}
environment:
- NEXT_PUBLIC_APP_URL
- NEXT_PUBLIC_SENTRY_DSN=${SENTRY_DSN}
- NEXT_SERVER_MONGODB_URI
- NEXT_SERVER_MONGODB_USER
- NEXT_SERVER_MONGODB_PASSWORD
- NEXT_SERVER_MONGODB_DBNAME
- NEXT_SERVER_MONGODB_DEBUG
- NEXT_SERVER_MAILER_HOST
- NEXT_SERVER_MAILER_PORT
- NEXT_SERVER_MAILER_USER
- NEXT_SERVER_MAILER_PASS
- NEXT_SERVER_MAILER_SECURE
- NEXT_SERVER_REDIS_HOST
- NEXT_SERVER_TOKEN_EXPIRE_DAYS
- SENTRY_DSN
- SENTRY_RELEASE
- SENTRY_ENVIRONMENT
- SENTRY_ORG
- SENTRY_PROJECT
- SENTRY_LOG_LEVEL
- VIRTUAL_PORT=3000
- VIRTUAL_HOST
- CERT_NAME
extra_hosts:
- s0003113:10.100.173.107
- s0003112:10.100.173.106
- s0003110:10.100.173.105
restart: always
redis:
image: redis
restart: always
volumes:
- redis:/data
volumes:
redis:
driver: local
I use this docker-compose which runs MongoDB+ mongo-express Web interface:
https://gist.github.com/adamelliotfields/cd49f056deab05250876286d7657dc4b
How can I run docker(s) with mongodb cluster and mongo-express as web interface?
A have attached my docker-compose file: I have got it from bitnami and added mongo-express. But web interface does not work.
That is:
version: '3.1'
services: mongodb-sharded:
image: docker.io/bitnami/mongodb-sharded:4.4
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-sharded
- MONGODB_SHARDING_MODE=mongos
- MONGODB_CFG_PRIMARY_HOST=mongodb-cfg
- MONGODB_CFG_REPLICA_SET_NAME=cfgreplicaset
- MONGODB_REPLICA_SET_KEY=replicasetkey123
- MONGODB_ROOT_PASSWORD=example
ports:
- "27017:27017"
mongodb-shard0:
image: docker.io/bitnami/mongodb-sharded:4.4
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-shard0
- MONGODB_SHARDING_MODE=shardsvr
- MONGODB_MONGOS_HOST=mongodb-sharded
- MONGODB_ROOT_PASSWORD=example
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_REPLICA_SET_KEY=replicasetkey123
- MONGODB_REPLICA_SET_NAME=shard0
volumes:
- 'shard0_data:/bitnami'
mongodb-cfg:
image: docker.io/bitnami/mongodb-sharded:4.4
environment:
- MONGODB_ADVERTISED_HOSTNAME=mongodb-cfg
- MONGODB_SHARDING_MODE=configsvr
- MONGODB_ROOT_PASSWORD=example
- MONGODB_REPLICA_SET_MODE=primary
- MONGODB_REPLICA_SET_KEY=replicasetkey123
- MONGODB_REPLICA_SET_NAME=cfgreplicaset
volumes:
- 'cfg_data:/bitnami'
mongo-express:
image: mongo-express
restart: always
ports:
- 8081:8081
# - 27017:27017
environment:
ME_CONFIG_MONGODB_ADMINUSERNAME: root
ME_CONFIG_MONGODB_ADMINPASSWORD: example
XX_CONFIG_MONGODB_URL: mongodb://root:example#mongodb-sharded:27017/
ME_CONFIG_MONGODB_URL: mongodb://root:example#mongodb-cfg:27017/
# ongodb://[username:password#]host1[:port1][,...hostN[:portN]][/[defaultauthdb][?options]]
volumes: shard0_data:
driver: local cfg_data:
driver: local
I'm using a .yml to launch an Odoo instance & its postgresql through a traefik and it works perfectly fine. It assigns the domain and subdomain with a Let's Encrypt certificate, perfect.
My issue now is that I wish to launch more of these but I haven't succeeded in doing so.
I've tried:
Changing the ports: in the odoo section
Removing the ports: section from the odoo section
Editing the ports: in the traefik section
The .yml:
version: "3"
services:
odoo:
image: odoo:12.0
depends_on:
- db
restart: unless-stopped
networks:
- internal
ports:
- "8069:8069"
- "8072:8072"
environment:
- HOST=db
- USER=${ODOO_USER}
- PASSWORD=${ODOO_PASS}
volumes:
- ./odoo/odoo-web-data:/var/lib/odoo
- ./odoo/config:/etc/odoo
- ./odoo/addons:/mnt/extra-addons
- ./odoo/logs:/var/log/odoo
labels:
- 'traefik.http.routers.odoo.rule=Host(`${ODOO_TRAEFIK_URL}`)'
- 'traefik.http.routers.odoo.entrypoints=websecure'
- 'traefik.http.routers.odoo.tls.certresolver=odoo'
- 'traefik.port=8069'
- "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
- "traefik.http.routers.http-catchall.entrypoints=web"
- "traefik.http.routers.http-catchall.middlewares=redirect-to-https#docker"
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
db:
image: postgres:10
restart: unless-stopped
networks:
- internal
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=${ODOO_USER}
- POSTGRES_PASSWORD=${ODOO_PASS}
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- ./pgdata:/var/lib/postgresql/data/pgdata
traefik:
image: traefik:v2.0
networks:
- internal
- web
ports:
# The HTTP port
- "80:80"
- "443:443"
# The Web UI (enabled by --api.insecure=true)
- "8080:8080"
volumes:
- "./traefik/letsencrypt:/letsencrypt"
- "./traefik/traefik.yml:/etc/traefik.yml"
- "/var/run/docker.sock:/var/run/docker.sock"
command:
- "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker"
- "--providers.docker.defaultRule=Host(`{{ trimPrefix `/` .Name }}.${TRAEFIK_DEFAULT_DOMAIN}`)"
- "--entryPoints.web.address=:80"
- "--entryPoints.websecure.address=:443"
- "--certificatesResolvers.odoo.acme.httpchallenge=true"
- "--certificatesresolvers.odoo.acme.httpchallenge.entrypoint=web"
- "--certificatesresolvers.odoo.acme.email=${ACME_EMAIL}"
- "--certificatesresolvers.odoo.acme.storage=/letsencrypt/acme.json"
networks:
internal:
web:
external: true
Assuming I already have a Traefik + Odoo + Postgres already running, how could I go about launching more instances of these together?
My 1st goal is to write a docker-compose.yml file with the following:
1 docker for the MariaDB server
1 docker for the PrestaShop-1.7 server
1 docker for the PHPMyAdmin server
Can you please help me get it working correctly ?
Then, my 2nd goal is to set passwords and disallow the "no password".
Kind regards,
Arnaud.
I'm using the bitnami's dockers so I've started the following script:
version: "3"
networks:
prestashop-network:
driver: bridge
services:
mariadb:
image: 'bitnami/mariadb:10.3'
environment:
- MARIADB_USER=bn_prestashop
- MARIADB_DATABASE=bitnami_prestashop
- ALLOW_EMPTY_PASSWORD=yes
networks:
- prestashop-network
volumes:
- 'mariadb_data:/bitnami'
ports:
- 3307:3306
phpmyadmin:
image: bitnami/phpmyadmin:latest
volumes:
- 'phpmyadmin_data:/bitnami'
depends_on:
- mariadb
ports:
- 81:80
environment:
- PHPMYADMIN_ALLOW_NO_PASSWORD=true
networks:
- prestashop-network
prestashop_1.7:
image: 'bitnami/prestashop:1.7'
volumes:
- 'prestashop_data:/bitnami'
- ./docker/prestashop/custom-php.ini:/usr/local/etc/php/conf.d/custom.ini
- ./docker/prestashop/phpinfo.php:/var/www/html/phpinfo.php
depends_on:
- mariadb
ports:
- 8085:80
- 8086:443
environment:
- PRESTASHOP_FIRST_NAME=Toto
- PRESTASHOP_LAST_NAME=FAMILLE
- PRESTASHOP_PASSWORD=bitnami1
- PRESTASHOP_EMAIL=user#example.com
- PRESTASHOP_HOST=localhost
- PRESTASHOP_COUNTRY=fr
- PRESTASHOP_LANGUAGE=fr
- MARIADB_HOST=mariadb
- MARIADB_PORT_NUMBER=3306
- PRESTASHOP_DATABASE_USER=bn_prestashop
- PRESTASHOP_DATABASE_NAME=bitnami_prestashop
- PRESTASHOP_DATABASE_PASSWORD=bitnami1
- ALLOW_EMPTY_PASSWORD=yes
- MARIADB_ROOT_USER=root
- MARIADB_ROOT_PASSWORD=
- MYSQL_CLIENT_CREATE_DATABASE_NAME=bitnami_prestashop
- MYSQL_CLIENT_CREATE_DATABASE_USER=bn_prestashop
- SMTP_HOST=smtp.gmail.com
- SMTP_PORT=587
- SMTP_PROTOCOL=tls
- SMTP_USER=your_email#gmail.com
- SMTP_PASSWORD=your_password
networks:
- prestashop-network
volumes:
mariadb_data:
driver: local
prestashop_data:
driver: local
phpmyadmin_data:
driver: local
For information, I use Mac OS X Mojave with the following docker tools version:
$ docker-compose version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018
When I launch with the following command:
docker-compose up
Then the different images are downloaded and started.
When I try to access the PhpMyAdmin instance using http://localhost:81 I can reach the PhpMyAdmin instance correctly using root and no password.
I get two major problems:
I see the 'prestashop' database is created but empty
When I try to access the PrestaShop instance using http://localhost:8085 I get an error 500
When tying your docker-compose file I got this errors:
mariadb_1 | 2019-08-15 9:28:47 13 [Warning] Access denied for user 'bn_prestashop'#'192.168.48.4' (using password: YES)
prestashop_1.7_1 | mysql-c ERROR [canConnect] Connection with 'bn_prestashop' user is unsuccessful
You need to set up the user password in the mariadb container too.
This docker-compose file worked for me, may be you can build up from here.
version: '2'
services:
mariadb:
image: 'bitnami/mariadb:10.1'
environment:
- MARIADB_USER=bn_prestashop
- MARIADB_DATABASE=bitnami_prestashop
- MARIADB_PASSWORD=my_passwd
- ALLOW_EMPTY_PASSWORD=yes
volumes:
- 'mariadb_data:/bitnami'
prestashop:
image: 'bitnami/prestashop:1.7'
environment:
- MARIADB_HOST=mariadb
- MARIADB_PORT_NUMBER=3306
- PRESTASHOP_DATABASE_USER=bn_prestashop
- PRESTASHOP_DATABASE_NAME=bitnami_prestashop
- PRESTASHOP_DATABASE_PASSWORD=my_passwd
- ALLOW_EMPTY_PASSWORD=yes
- PRESTASHOP_FIRST_NAME=Toto
- PRESTASHOP_LAST_NAME=FAMILLE
- PRESTASHOP_PASSWORD=bitnami1
- PRESTASHOP_EMAIL=user#example.com
- PRESTASHOP_HOST=localhost
- PRESTASHOP_COUNTRY=fr
- PRESTASHOP_LANGUAGE=fr
- SMTP_HOST=smtp.gmail.com
- SMTP_PORT=587
- SMTP_PROTOCOL=tls
- SMTP_USER=your_email#gmail.com
- SMTP_PASSWORD=your_password
ports:
- '80:80'
- '443:443'
volumes:
- 'prestashop_data:/bitnami'
depends_on:
- mariadb
phpmyadmin:
image: 'bitnami/phpmyadmin:4'
ports:
- '8080:80'
- '8443:443'
depends_on:
- mariadb
volumes:
- 'phpmyadmin_data:/bitnami'
volumes:
mariadb_data:
driver: local
prestashop_data:
driver: local
phpmyadmin_data:
driver: local