I'm trying to deploy my service on a vm and I have this part in my docker-compose.yaml:
certbot:
container_name: certbot
image: certbot/certbot:v1.9.0
logging:
driver: "json-file"
options:
max-size: 500m
volumes:
- {vm_path}/certbot/certbot-etc:/etc/letsencrypt
- {vm_path}/certbot/certbot-var:/var/lib/letsencrypt
- {vm_path}/nginx/certbot-web-root:/var/www/certbot/html
- /etc/localtime:/etc/localtime:ro
- /etc/timezone:/etc/timezone:ro
command:
- certonly
- --webroot
- -w /var/www/certbot/html
- --email={email}
- --agree-tos
- --no-eff-email
- --dry-run
- -d {domain}
but when I run docker-compose up, I get this error from certbot:
certbot | /var/www/certbot/html does not exist or is not a directory
also the directory {vm_path}/nginx/certbot-web-root:/var/www/certbot/html exists on my vm and is absolute.
------UPDATE:
My VM's OS is Ubuntu 18.04.
I checked docker-compose config and there is no problem in it.
Also the result of run ls -l /var/www/certbot:
certbot | total 4
certbot | drwxr-xr-x 2 1001 1001 4096 Nov 4 10:54 html
and also the /var/www/certbot/html/ exists and is empty.
I've found the answer. The problem is the command section syntax, it should be:
command:
- certonly
- --webroot
- -w
- /var/www/certbot/html
- --email={email}
- --agree-tos
- --no-eff-email
- --dry-run
- -d
- {domain}
Related
Our travis builds have started failing and I can't figure out why. Our app runs in docker-compose and then we run cypress to against it. This used to work perfectly. Now the host port for the web server is just unresponsive. I've removed cypress and am just trying to run curl http://localhost:3001 and it just hangs. Here's the travis.yml. Any suggestions would be highly appreciated. I have tried fiddling for several hours with the docker versions, distros, localhost vs 127.0.0.1, etc to no avail. All of this works fine locally on my workstation.
language: node_js
node_js:
- "12.19.0"
env:
- DOCKER_COMPOSE_VERSION=1.25.4
services:
- docker
sudo: required
# Supposedly this is needed for Cypress to work in Ubuntu 16
# https://github.com/cypress-io/cypress-example-kitchensink/blob/master/basic/.travis.yml
addons:
apt:
packages:
- libgconf-2-4
before_install:
# upgrade docker compose https://docs.travis-ci.com/user/docker/#using-docker-compose
- sudo rm /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/${DOCKER_COMPOSE_VERSION}/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
# upgrade docker itself https://docs.travis-ci.com/user/docker/#installing-a-newer-docker-version
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
- sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
- sudo apt-get update
- sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
# Put the .env file in place
- cp .env.template .env
install:
# Install node modules (for jest and wait-on) and start up the docker containers
- cd next
- npm ci
- cd ..
- cd e2e
- npm ci
- cd ..
script:
- docker --version
- docker-compose --version
- docker-compose up --build -d
# Run unit tests
# - cd next
# - npm run test
# Run e2e tests
# - cd ../e2e
# - npx cypress verify
# - CYPRESS_FAIL_FAST=true npx wait-on http://localhost:3001 --timeout 100000 && npx cypress run --config video=false,pageLoadTimeout=100000,screenshotOnRunFailure=false
- sleep 30
- curl http://127.0.0.1:3001 --max-time 30
- docker-compose logs db
- docker-compose logs express
- docker-compose logs next
post_script:
- docker-compose down
The logs look like this:
The command "docker-compose up --build -d" exited with 0.
30.01s$ sleep 30
The command "sleep 30" exited with 0.
93.02s$ curl http://127.0.0.1:3001 --max-time 30
curl: (28) Operation timed out after 30001 milliseconds with 0 bytes received
The command "curl http://127.0.0.1:3001 --max-time 30" exited with 28.
The docker compose logs show nothing suspicious. It's as if the network wasn't set up correctly and docker is not aware of any requests.
Here is the docker-compose.yml in case it's useful:
version: '3.7'
services:
db:
image: mg-postgres
build: ./postgres
ports:
- '5433:5432'
environment:
POSTGRES_HOST_AUTH_METHOD: 'trust'
adminer:
image: adminer
depends_on:
- db
ports:
- '8080:8080'
express:
image: mg-server
build: ./express
restart: always
depends_on:
- db
env_file:
- .env
environment:
DEBUG: express:*
volumes:
- type: bind
source: ./express
target: /app
- /app/node_modules
ports:
- '3000:3000'
next:
image: mg-next
build: ./next
depends_on:
- db
- express
env_file:
- .env
volumes:
- type: bind
source: ./next
target: /app
- /app/node_modules
ports:
- '3001:3001'
command: ['npm', 'run', 'dev']
I'm working with docker image that uses Python3.6 as its base. All of the sudden it started crashing (exiting right after start up). So I bashed into the container and found out that
it crashes because connection to containerized postgres database fails all of the sudden.
The only error output I managed to get is forward host lookup failed: Unknown host which isn't telling me much.
entrypoint.sh:
echo "Waiting for postgres..."
while ! nc -z users-db 5432; do
sleep 0.1
done
echo "PostgreSQL started"
python manage.py run -h 0.0.0.0
error output:
Waiting for postgres...
users-db: forward host lookup failed: Unknown host
users-db: forward host lookup failed: Unknown host
users-db: forward host lookup failed: Unknown host
...
...
Dockerfile:
FROM python:3.6.9-slim
LABEL maintainer="abc"
RUN apt-get update && \
apt-get install -y netcat && \
apt-get clean
WORKDIR /usr/src/app
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh
RUN chmod +x /usr/src/app/entrypoint.sh
COPY . /usr/src/app
CMD ["/usr/src/app/entrypoint.sh"]
What strikes me about this is that it worked wonderfully until now and without me making any changes to the database container the connection failed.
What can I do to troubleshoot this ? If you need to review any files just ask and I'll share it here.
docker ps:
72d344cc61bf tdd_nginx "nginx -g 'daemon of…" 25 minutes ago Restarting (1) 55 seconds ago tdd_nginx_1
8ee2f8082e69 tdd_client "npm start" 26 minutes ago Up 25 minutes 0.0.0.0:3007->3000/tcp tdd_client_1
1ccfc3ca5600 tdd_users-db "docker-entrypoint.s…" 26 minutes ago Up 26 minutes 0.0.0.0:5435->5432/tcp tdd_users-db_1
--> 62af29277b78 tdd_users "/bin/bash -s" 22 minutes ago Exited (130) 2 minutes ago # <-- keeps crashing
docker-compose file:
version: '3.7'
services:
users:
build:
context: ./services/users
dockerfile: Dockerfile
volumes:
- './services/users:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- DATABASE_URL=postgres://postgres:postgres#users-db:5432/users_dev
- DATABASE_TEST_URL=postgres://postgres:postgres#users-db:5432/users_test
- SECRET_KEY=bart_simpson
depends_on:
- users-db
client:
build:
context: ./services/client
dockerfile: Dockerfile
volumes:
- './services/client:/usr/src/app'
- '/usr/src/app/node_modules'
ports:
- 3007:3000
environment:
- NODE_ENV=development
- REACT_APP_USERS_SERVICE_URL=${REACT_APP_USERS_SERVICE_URL}
depends_on:
- users
nginx:
build:
context: ./services/nginx
dockerfile: Dockerfile
restart: always
ports:
- 80:80
depends_on:
- users
- client
users-db:
build:
context: './services/users/project/db'
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
Problem was that I've got pyjwt library installed for generating web tokens and I used pyjwt.encode() instead of jwt.encode() in my code. That made a major difference to the functionality of the connection between containers. Still don't know why though. Containers are now running again. If somebody will vote to close this topic I'll understand as nobody would've guessed this.
I'm using the certbot/certbot container as in:
docker-compose run -d --rm --entrypoint 'certbot certonly --webroot -w /var/www/certbot --staging --email example#domain.se -d example.com --rsa-key-size 4096 --agree-tos --force-renewal ; sleep 3600' certbot
on the following compose file:
version: '3.5'
services:
nginx:
image: nginx:1.15-alpine
restart: unless-stopped
volumes:
- "~/dev/docker/projects/common/volumes/letsencrypt/nginx:/etc/nginx/conf.d"
- "~/dev/docker/projects/common/volumes/letsencrypt/certbot/conf:/etc/letsencrypt"
- "~/dev/docker/projects/common/volumes/letsencrypt/certbot/www:/var/www/certbot"
- "~/dev/docker/projects/common/volumes/letsencrypt/nginx:/var/www/nginx"
- "~/dev/docker/projects/common/volumes/logs:/var/log/nginx"
ports:
- "80:80"
- "443:443"
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
certbot:
image: certbot/certbot
restart: unless-stopped
volumes:
- "~/dev/docker/projects/common/volumes/letsencrypt/certbot/conf:/etc/letsencrypt"
- "~/dev/docker/projects/common/volumes/letsencrypt/certbot/www:/var/www/certbot"
- "~/dev/docker/projects/common/volumes/logs:/var/log/letsencrypt"
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
But it ignores the sleep command and the container goes away.
Whereas running the following:
docker-compose run -d --rm --entrypoint 'sleep 3600' certbot
keeps the container up and running.
I would like to keep the container up and running after the certbot failed.
You could move "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'" into dedicated script for example start.sh.
Mount it with docker-compose volumes
volumes:
- "./start.sh:/start.sh
entrypoint: /start.sh
I'm trying to mount my postgres.conf and pg_hba.conf using docker-compose and having difficulty understanding why it work when run using docker-cli and doesn't with docker-compose
The following docker-compose causes the image to crash with error:
/usr/local/bin/docker-entrypoint.sh: line 176: /config_file=/etc/postgresql/postgres.conf: No such file or directory
docker-compose.yml
services:
postgres-master:
image: postgres:11.4
container_name: postgres-master
volumes:
- ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
- /home/agilob/dockers/pg/data:/var/lib/postgresql/data:rw
- $PWD/pg:/etc/postgresql:rw
- /etc/localtime:/etc/localtime:ro
hostname: 'primary'
environment:
- PGHOST=/tmp
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
- MAX_CONNECTIONS=10
- MAX_WAL_SENDERS=5
- PG_MODE=primary
- PUID=1000
- PGID=1000
ports:
- "5432:5432"
command: 'config_file=/etc/postgresql/postgres.conf hba_file=/etc/postgresql/pg_hba.conf'
This command works fine:
docker run -d --name some-postgres -v "$PWD/postgres.conf":/etc/postgresql/postgresql.conf postgres -c 'config_file=/etc/postgresql/postgresql.conf'
also when I remove command: section and run the same docker-compose:
$ docker-compose -f postgres-compose.yml up -d
Recreating postgres-master ... done
$ docker exec -it postgres-master bash
root#primary:/# cd /etc/postgresql
root#primary:/etc/postgresql# ls
pg_hba.conf postgres.conf
The files are present in /etc/postgres.
Files in $PWD/pg are present:
$ ls pg
pg_hba.conf postgres.conf
The following works fine:
command: postgres -c config_file='/etc/postgresql/postgres.conf' -c 'hba_file=/etc/postgresql/pg_hba.conf'
After I launch my docker-compose up command, everything starts up and I run prisma deploy which also works fine, yet my application still returns the above error. I have been trying to find a solution to this for days, and there is nothing helpful online, and the few similar questions have been closed or ignored. I would appreciate getting help with this issue.
Here is my docker-compose.yml file:
version: '3'
services:
prisma:
env_file:
- .env
image: prismagraphql/prisma:1.34
restart: always
ports:
- "4466:4466"
environment:
PRISMA_CONFIG: |
port: 4466
databases:
default:
connector: mongo
uri: ${MONGODB_URI}
host: host.docker.internal
JWT_SECRET: ${JWT_SECRET}
mongo:
env_file:
- .env
image: mongo:3.6
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGODB_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGODB_PASSWORD}
ports:
- "27017:27017"
volumes:
- mongo:/var/lib/mongo
web:
env_file:
- .env
build: .
volumes:
- .:/usr/app/
- /usr/app/node_modules
ports:
- "4000:4000"
environment:
DATABASE_URL: ${MONGODB_URI}
volumes:
mongo:
My Dockerfile:
FROM node:8.16.0-alpine
WORKDIR /usr/app
COPY package.json .
RUN npm install --quiet
COPY . .
ENV DOCKERIZE_VERSION v0.6.0
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
&& tar -C /usr/local/bin -xzvf dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
&& rm dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz
CMD dockerize -wait tcp://mongo:27017 -wait tcp://prisma:4466 -timeout 60m npm start
My prisma.yml:
endpoint: http://localhost:4466
datamodel:
- db/types.prisma
- db/enums.prisma
databaseType: document
generate:
- generator: javascript-client
output: ./generated/prisma-client/
My prisma deploy command works, and it generates the mongo database, but when I try to query my application at localhost:4000, it looks like this and returns this error:
request to http://localhost:4466/ failed, reason: connect ECONNREFUSED 127.0.0.1:4466
But when I navigate to localhost:4466/_admin, the database is all set up fine and shows the three tables that should be there.
I checked if anything is running localhost:4466 by issuing this command: lsof -i :4466, and I can see that docker started up correctly.
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 5923 sguduguntla 24u IPv4 0x89ea943c9b98ff09 0t0 TCP *:4466 (LISTEN)
com.docke 5923 sguduguntla 25u IPv6 0x89ea943c87111549 0t0 TCP localhost:4466 (LISTEN)
When I run docker ps, you can also see the following output with the three images:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fa70fae37f10 prismagraphql/prisma:1.34 "/bin/sh -c /app/sta…" 35 minutes ago Up 31 minutes 0.0.0.0:4466->4466/tcp decal-board-graphql-server_prisma_1
d64b9f6dcd29 decal-board-graphql-server_web "docker-entrypoint.s…" 35 minutes ago Up 31 minutes 0.0.0.0:4000->4000/tcp decal-board-graphql-server_web_1
6f7dda5e58a0 mongo:3.6 "docker-entrypoint.s…" 35 minutes ago Up 31 minutes 0.0.0.0:27017->27017/tcp decal-board-graphql-server_mongo_1