I'm trying to mount my postgres.conf and pg_hba.conf using docker-compose and having difficulty understanding why it work when run using docker-cli and doesn't with docker-compose
The following docker-compose causes the image to crash with error:
/usr/local/bin/docker-entrypoint.sh: line 176: /config_file=/etc/postgresql/postgres.conf: No such file or directory
docker-compose.yml
services:
postgres-master:
image: postgres:11.4
container_name: postgres-master
volumes:
- ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
- /home/agilob/dockers/pg/data:/var/lib/postgresql/data:rw
- $PWD/pg:/etc/postgresql:rw
- /etc/localtime:/etc/localtime:ro
hostname: 'primary'
environment:
- PGHOST=/tmp
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
- MAX_CONNECTIONS=10
- MAX_WAL_SENDERS=5
- PG_MODE=primary
- PUID=1000
- PGID=1000
ports:
- "5432:5432"
command: 'config_file=/etc/postgresql/postgres.conf hba_file=/etc/postgresql/pg_hba.conf'
This command works fine:
docker run -d --name some-postgres -v "$PWD/postgres.conf":/etc/postgresql/postgresql.conf postgres -c 'config_file=/etc/postgresql/postgresql.conf'
also when I remove command: section and run the same docker-compose:
$ docker-compose -f postgres-compose.yml up -d
Recreating postgres-master ... done
$ docker exec -it postgres-master bash
root#primary:/# cd /etc/postgresql
root#primary:/etc/postgresql# ls
pg_hba.conf postgres.conf
The files are present in /etc/postgres.
Files in $PWD/pg are present:
$ ls pg
pg_hba.conf postgres.conf
The following works fine:
command: postgres -c config_file='/etc/postgresql/postgres.conf' -c 'hba_file=/etc/postgresql/pg_hba.conf'
Related
I am spinning up a Postgres database in Docker-compose and it worked great the first time. I wanted to test some things so I ran
docker-compose down
And after my changes. I ran docker-compose up. This time, I keep getting a message saying
PostgreSQL Database directory appears to contain a database; Skipping initialization
My docker-compose is as follows:
postgres_db:
image: postgres:14.1
restart: unless-stopped
environment:
- POSTGRES_PASSWORD=${BP_ADMIN_PASSWORD}
volumes:
- ./admin/${BP_ENV:-dev}/database:/docker-entrypoint-initdb.d
- ./data/postgres:/var/lib/postgresql/data
command: postgres -c logging_collector=on -c log_rotation_age=1d -c log_directory=/mnt/log -c
ports:
- "5432:5432"
I ran docker-compose down --volumes to try to get rid of everything, but it did not work.
Then, I deleted the entire ./data/postgres folder on my local drive but it still failed. After, I commented out - ./data/postgres:/var/lib/postgresql/data but it still did not work.
How do I get rid of the existing database?
Running docker-compose down --volumes will only delete Docker volumes. You're not using volumes; you're using bind-mounts, in which you are mounting a host directory inside your container:
volumes:
- ./admin/${BP_ENV:-dev}/database:/docker-entrypoint-initdb.d
- ./data/postgres:/var/lib/postgresql/data
The way you delete that data is by using rm, as in:
rm -rf ./data/postgres/*
If you wanted to use a Docker volume for the database directory, that would look like:
version: "3"
services:
postgres_db:
image: postgres:14.1
restart: unless-stopped
environment:
- POSTGRES_PASSWORD=${BP_ADMIN_PASSWORD}
volumes:
- ./admin/${BP_ENV:-dev}/database:/docker-entrypoint-initdb.d
- pgdata:/var/lib/postgresql/data
command: postgres -c logging_collector=on -c log_rotation_age=1d -c log_directory=/mnt/log -c
ports:
- "5432:5432"
volumes:
pgdata:
In this case, running docker-compose down -v (or --volumes, if you
prefer) would delete the database volumes.
It's been few days I am blocked on this problem with my project, it's working on localhost but not on gitlabCI.
I would like to build a test database on the postgres docker image in gitlabCI but it doesn't work, I have try a lot of things and lose a lot of hours before ask this there :'(.
below my docker-compose.yml file :
version: "3"
services:
nginx:
image: nginx:latest
container_name: nginx
depends_on:
- postgres
- monapp
volumes:
- ./nginx-conf:/etc/nginx/conf.d
- ./util/certificates/certs:/etc/nginx/certs/localhost.crt
- ./util/certificates/private:/etc/nginx/certs/localhost.key
ports:
- 81:80
- 444:443
networks:
- monreseau
monapp:
image: monimage
container_name: monapp
depends_on:
- postgres
ports:
- "3000:3000"
networks:
- monreseau
command: "npm run local"
postgres:
image: postgres:9.6
container_name: postgres
environment:
POSTGRES_USER: postgres
POSTGRES_HOST: postgres
POSTGRES_PASSWORD: postgres
volumes:
- ./pgDatas:/var/lib/postgresql/data/
- ./db_dumps:/home/dumps/
ports:
- "5432:5432"
networks:
- monreseau
networks:
monreseau:
and below my gitlab-ci.yml file:
stages:
# - build
- test
image:
name: docker/compose:latest
services:
- docker:dind
before_script:
- docker version
- docker-compose version
variables:
DOCKER_HOST: tcp://docker:2375/
# build:
# stage: build
# script:
# - docker build -t monimage .
# - docker-compose up -d
test:
stage: test
script :
- docker build -t monimage .
- docker-compose up -d
- docker ps
- docker exec -i postgres psql -U postgres -h postgres -f /home/dumps/test/dump_test_001 -c \\q
- exit
- docker exec -i monapp ./node_modules/.bin/env-cmd -f ./env/.env.builded-test npx jasmine spec/auth_queries.spec.js
- exit
this is the content of docker ps log on gitlabCI server :
docker ps on gitlab-CI
I thought to put postgres on host would work, but no I always have in gitlab-ci terminal:
psql: could not connect to server: Connection refused
Is the server running on host "postgres" (172.19.0.2) and accepting
TCP/IP connections on port 5432?
I also tried to put docker on host but error :
psql: could not translate host name "docker" to address: Name or service not known
little precision : it is working on localhost of my computer when i am doing make builded-test
bellow my makefile:
builded-test:
docker build -t monimage .
docker-compose up -d
docker ps
docker exec -i postgres psql -U postgres -h postgres -f /home/dumps/test/dump_test_001 -c \\q
exit
docker exec -i monapp ./node_modules/.bin/env-cmd -f ./env/.env.builded-test npx jasmine spec/auth_queries.spec.js
exit
docker-compose down
I want to make work postgres image in my docker-compose on gitlab CI to execute my tests help me please :) thanks by advance
UPDATE
Now it working in gitlab-runner but still not on gitlab when I push, I update the files like following
I added :
variables:
POSTGRES_DB: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ""
POSTGRES_HOST_AUTH_METHOD: trust
and changed
test:
stage: test
script :
- docker build -t monimage .
- docker-compose up -d
- docker ps
- docker exec postgres psql -U postgres **-h postgres** -f /home/dumps/test/dump_test_001
- docker exec monapp ./node_modules/.bin/env-cmd -f ./env/.env.builded-test npx jasmine spec/auth_queries.spec.js
in the .gitlab-ci.yml
but still don't work when I push it to gitlab, it give me :
sql: could not connect to server: Connection refused
Is the server running on host "postgres" (172.19.0.2) and accepting
TCP/IP connections on port 5432?
any ideas ? :)
Maybe you need to wait for PostgreSQL service to be up and running.
Can you add a 10 seconds delay before trying the psql stuff? Something like:
- sleep 10
If it works, then you can use a more specific solution to wait for PostgreSQL to be initialized, like Docker wait for postgresql to be running
I am using Postgres as a service in my docker-compose file. I want logging to log file to be enabled when I do docker-compose up. One way to enable logging is by editing postgres.conf file but it's not useful in this case. One other way is to do something like this
docker run --name postgresql -itd --restart always sameersbn/postgresql:10-2 -c logging_collector=on
but this isn't useful too cause I am not starting it from an image but as a docker-compose service. Any idea how I can start the docker-compose up with logging enabled in Postgres???
Here is the docker-compose to run the command -c in compose
version: '3.6'
services:
postgresql:
image: postgres:11.5
container_name: platops_postgres
volumes: ['platops-data:/var/lib/postgresql/data/', 'postgress-logs:/var/log/postgresql/']
command: ["postgres", "-c", "logging_collector=on", "-c", "log_directory=/logs", "-c", "log_filename=postgresql.log", "-c", "log_statement=all"]
environment:
- POSTGRES_USER=postgresql
- POSTGRES_PASSWORD=postgresql
ports: ['5432:5432']
volumes:
platops-data: {}
# uncomment and set the path of the folder to maintain persistancy
# data-postgresql:
# driver: local
# driver_opts:
# o: bind
# type: none
# device: /path/of/db/postgres/data/
postgress-logs: {}
# uncomment and set the path of the folder to maintain persistancy
# data-postgresql:
# driver: local
# driver_opts:
# o: bind
# type: none
# device: /path/of/db/postgres/logs/
For more information, you can check with the containers/postgress
Just like you command with docker run:
docker run --name postgresql -itd --restart always sameersbn/postgresql:10-2 -c logging_collector=on
that you add the -c logging_collector=on arguments for the ENTRYPOINT ["/sbin/entrypoint.sh"] to enable logging. (Dockerfile).
In docker-compose.yml file, use command: like this:
version: "3.7"
services:
database:
image: sameersbn/postgresql:10-2
command: "-c logging_collector=on"
# ......
When Postgresql contaienr run, it will run command: /sbin/entrypoint.sh -c logging_collector=on.
I keep getting
User "postgres" has no password assigned.
updated
.env
POSTGRES_PORT=5432
POSTGRES_DB=demo_db2
POSTGRES_USER=postgres
POSTGRES_PASSWORD=password
Even though the postgres password is set.
I'm trying to use the same variables from the following command
docker run --name demo4 -e POSTGRES_PASSWORD=password -d postgres
Could this be an issue with volumes ? im very confused.
I ran this command as well
docker run -it --rm --name demo4 -p 5432:5432 -e POSTGRES_PASSWORD=password -e POSTGRES_USER=postgress postgres:9.4
docker-compose.yml
# docker-compose.yml
version: "3"
services:
app:
build: .
depends_on:
- database
ports:
- 8000:8000
environment:
- POSTGRES_HOST=database
database:
image: postgres:9.6.8-alpine
volumes:
- pgdata:/var/lib/postgresql/pgdata
ports:
- 8002:5432
react_client:
build:
context: ./client
dockerfile: Dockerfile
image: react_client
working_dir: /home/node/app/client
volumes:
- ./:/home/node/app
ports:
- 8001:8001
env_file:
- ./client/.env
volumes:
pgdata:
You are missing the inclusion of the .env file...
Docker composer:
database:
environment:
- ENV_VAR=VALUE
or
database:
env_file:
- .env
Plain Docker:
docker run options --env ENV_VAR=VALUE ...
or
docker run options --env-file .env ...`
I am using docker compose to combine 2 images (tomcat with my app and database - postgres).
My compose file looks like this :
version: '3'
services:
tomcat:
build: ./tomcat-img
ports:
- "8080:8080"
depends_on:
- "db"
db:
build: ./db-img
volumes:
- db-data:/var/lib/postgres/data
ports:
- "5433:5432"
volumes:
db-data:
and here is dockerfile for database image:
FROM postgres:9.5-alpine
ENV POSTGRES_DB mydb
ENV POSTGRES_USER xxxx
ENV POSTGRES_PASSWORD xxxx
COPY init-db.sql /docker-entrypoint-initdb.d/
EXPOSE 5432
CMD ["postgres"]
Next I started my containers with docker-compose cli docker-compose -f docker-compose.yml up
and run psql tool with:
docker exec -it container_id psql -d xxxx -U xxxx
and insert new record. After that I check if there really is:
select * from my_table;
After that I tried stopped docker compose and remove containers with:
docker-compose -f docker-compose.yml down
and start it again
docker-compose -f docker-compose.yml up
when I run again psql tool of db container and select data in my_table, there is no previous inserted record ... Can you help me to fix it please? I need init my db with init-db.sql just once and next using that persist storage. Thanks for answers.
In my dockerized Postgresql with a data volume I am binding to /var/lib/postgresql and not to /var/lib/postgres/data. Try changing your compose file to
volumes:
- db-data:/var/lib/postgresql