I want to start a Postgres database using docker-compose and pass some configurations using the PGOPTIONS environment variable. My docker-compose.yml looks like this:
version: '2'
services:
db:
image: postgres
environment:
- POSTGRES_USER=dbuser
- POSTGRES_PASSWORD=dbpw
- PGOPTIONS='-c track_activities=on -c track_counts=on -c track_io_timing=on'
When I start this service using docker-compose up db I get:
db_1 | FATAL: parameter "track_io_timing" requires a Boolean value
db_1 | done
db_1 | server started
db_1 | FATAL: parameter "track_io_timing" requires a Boolean value
db_1 | psql: FATAL: parameter "track_io_timing" requires a Boolean value
It does not seem to be related to the tracking_io_timing parameter, because when I remove that one from the PGOPTIONS variable I get the same error, but for the track_counts variable. So I suspect there is something wrong with the way I declared PGOPTIONS.
Same result here when using your provided Docker-Compose file. I have tried to use the alternative syntax for defining environment variables and I do not get the errors anymore:
version: '2'
services:
db:
image: postgres
environment:
POSTGRES_USER: dbuser
POSTGRES_PASSWORD: dbpw
PGOPTIONS: "-c track_activities=on -c track_counts=on -c track_io_timing=on"
Related
I'm trying to spin up a postgresl database using docker compose on my mac running macOS 12.4.
My docker-compose.yml file is
version: '3.8'
services:
db:
image: postgres:latest
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: gymbro
ports:
- 5438:5432
volumes:
- ./pgdata:/var/lib/postgresql/data
Running docker-compose up results in the container exiting with status 1. Here is the relevant output
Attaching to docker-compose-gymbro-db-1
docker-compose-gymbro-db-1 | The files belonging to this database system will be owned by user "postgres".
docker-compose-gymbro-db-1 | This user must also own the server process.
docker-compose-gymbro-db-1 |
docker-compose-gymbro-db-1 | The database cluster will be initialized with locale "en_US.utf8".
docker-compose-gymbro-db-1 | The default database encoding has accordingly been set to "UTF8".
docker-compose-gymbro-db-1 | The default text search configuration will be set to "english".
docker-compose-gymbro-db-1 |
docker-compose-gymbro-db-1 | Data page checksums are disabled.
docker-compose-gymbro-db-1 |
docker-compose-gymbro-db-1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
docker-compose-gymbro-db-1 | creating subdirectories ... ok
docker-compose-gymbro-db-1 | selecting dynamic shared memory implementation ... posix
docker-compose-gymbro-db-1 | selecting default max_connections ... 100
docker-compose-gymbro-db-1 | selecting default shared_buffers ... 128MB
docker-compose-gymbro-db-1 | selecting default time zone ... Etc/UTC
docker-compose-gymbro-db-1 | creating configuration files ... ok
docker-compose-gymbro-db-1 | running bootstrap script ... ok
docker-compose-gymbro-db-1 | performing post-bootstrap initialization ... 2022-12-24 17:42:20.791 UTC [40] FATAL: data directory "/var/lib/postgresql/data" has invalid permissions
docker-compose-gymbro-db-1 | 2022-12-24 17:42:20.791 UTC [40] DETAIL: Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
docker-compose-gymbro-db-1 | child process exited with exit code 1
docker-compose-gymbro-db-1 | initdb: removing contents of data directory "/var/lib/postgresql/data"
docker-compose-gymbro-db-1 exited with code 1
Seems like there is a permissions issue,
docker-compose-gymbro-db-1 | performing post-bootstrap initialization ... 2022-12-24 17:42:20.791 UTC [40] FATAL: data directory "/var/lib/postgresql/data" has invalid permissions
docker-compose-gymbro-db-1 | 2022-12-24 17:42:20.791 UTC [40] DETAIL: Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
I can spin up the db using a docker volume, but I'm just curious as to why this is happening and if there is a possible solution.
On Linux here, so this might not work on Mac. However, why not avoid using the local volum?
Like:
version: '3'
services:
db:
image: postgres:13.2
env_file:
- .env
volumes:
- data:/var/lib/postgresql/data
volumes:
data: {}
This is a solution I always use and solved my issues once and for all when I used to have it.
However, there is a more interesting discussion on this here that you can go through: https://forums.docker.com/t/data-directory-var-lib-postgresql-data-pgdata-has-wrong-ownership/17963/22
I have seen some items I honestly won't recommend like changing the PG_DATA, but if you're comfortable with it, have a go.
Let me know what you think :)
I am trying to use services in gitlab ci, namely, postgres. Anyway, postgres service doesn't seem to be running, although I just copied what is there in Gitlab CI docs. In logs, after service logs that it started, some psql command (I don't know where it came from) gives an name resolution error. If I am doing something wrong here, what is the way to run postgres service in gitlab-ci?
Below are .gitlab-ci.yml file and logs:
.gitlab-ci.yml
image: ubuntu
services:
- name: postgres:12.2-alpine
alias: postgres
variables:
POSTGRES_DB: badr
POSTGRES_USER: badr
POSTGRES_PASSWORD: badr
PGHOST: postgres
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- test
test db:
stage: test
before_script:
- until (echo > /dev/tcp/postgres/5432) >/dev/null 2>&1;do >&2 echo "service not ready...sleeping";sleep 5;done
script:
- echo "connected to...$PGHOST"
- sleep 10
logs
Running with gitlab-runner 13.2.0-rc2 (45f2b4ec)
on docker-auto-scale fa6cab46
Preparing the "docker+machine" executor
00:55
Using Docker executor with image ubuntu ...
Starting service postgres:12.2-alpine ...
Pulling docker image postgres:12.2-alpine ...
Using docker image sha256:ae192c4d3adaebbbf2f023e1e50eaadfabccb6b08c855ac13d6ce2232381a58a for postgres:12.2-alpine ...
WARNING: Service postgres:12.2-alpine is already created. Ignoring.
Waiting for services to be up and running...
*** WARNING: Service runner-fa6cab46-project-14794655-concurrent-0-f52b350b86ad38db-postgres-0 probably didn't start properly.
Health check error:
service "runner-fa6cab46-project-14794655-concurrent-0-f52b350b86ad38db-postgres-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2020-07-31T09:15:00.377204181Z ********************************************************************************
2020-07-31T09:15:00.377254629Z WARNING: POSTGRES_HOST_AUTH_METHOD has been set to "trust". This will allow
2020-07-31T09:15:00.377259167Z anyone with access to the Postgres port to access your database without
2020-07-31T09:15:00.377262471Z a password, even if POSTGRES_PASSWORD is set. See PostgreSQL
2020-07-31T09:15:00.377265670Z documentation about "trust":
2020-07-31T09:15:00.377269181Z https://www.postgresql.org/docs/current/auth-trust.html
2020-07-31T09:15:00.377272282Z In Docker's default configuration, this is effectively any other
2020-07-31T09:15:00.377276152Z container on the same system.
2020-07-31T09:15:00.377295876Z
2020-07-31T09:15:00.377299453Z It is not recommended to use POSTGRES_HOST_AUTH_METHOD=trust. Replace
2020-07-31T09:15:00.377302412Z it with "-e POSTGRES_PASSWORD=password" instead to set a password in
2020-07-31T09:15:00.377305641Z "docker run".
2020-07-31T09:15:00.377308656Z ********************************************************************************
2020-07-31T09:15:00.404620899Z The files belonging to this database system will be owned by user "postgres".
2020-07-31T09:15:00.406021814Z This user must also own the server process.
2020-07-31T09:15:00.406074886Z
2020-07-31T09:15:00.406083517Z The database cluster will be initialized with locale "en_US.utf8".
2020-07-31T09:15:00.406087263Z The default database encoding has accordingly been set to "UTF8".
2020-07-31T09:15:00.406090884Z The default text search configuration will be set to "english".
2020-07-31T09:15:00.406094281Z
2020-07-31T09:15:00.406097490Z Data page checksums are disabled.
2020-07-31T09:15:00.406101511Z
2020-07-31T09:15:00.406197662Z fixing permissions on existing directory /var/lib/postgresql/data ... ok
2020-07-31T09:15:00.406858429Z creating subdirectories ... ok
2020-07-31T09:15:00.407274720Z selecting dynamic shared memory implementation ... posix
2020-07-31T09:15:00.428414929Z selecting default max_connections ... 100
2020-07-31T09:15:00.506801199Z selecting default shared_buffers ... 128MB
2020-07-31T09:15:00.689382376Z selecting default time zone ... UTC
2020-07-31T09:15:00.695744690Z creating configuration files ... ok
2020-07-31T09:15:01.009439741Z running bootstrap script ... ok
2020-07-31T09:15:01.355673765Z sh: locale: not found
2020-07-31T09:15:01.355836607Z 2020-07-31 09:15:01.355 UTC [30] WARNING: no usable system locales were found
2020-07-31T09:15:01.784080826Z performing post-bootstrap initialization ... ok
2020-07-31T09:15:02.416545146Z syncing data to disk ... ok
2020-07-31T09:15:02.416652656Z
2020-07-31T09:15:02.416854775Z initdb: warning: enabling "trust" authentication for local connections
2020-07-31T09:15:02.416911707Z You can change this by editing pg_hba.conf or using the option -A, or
2020-07-31T09:15:02.416917642Z --auth-local and --auth-host, the next time you run initdb.
2020-07-31T09:15:02.416962149Z
2020-07-31T09:15:02.416967325Z Success. You can now start the database server using:
2020-07-31T09:15:02.416970415Z
2020-07-31T09:15:02.416990907Z pg_ctl -D /var/lib/postgresql/data -l logfile start
2020-07-31T09:15:02.416995097Z
2020-07-31T09:15:02.440378884Z waiting for server to start....2020-07-31 09:15:02.440 UTC [35] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit
2020-07-31T09:15:02.442773414Z 2020-07-31 09:15:02.442 UTC [35] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-07-31T09:15:02.461804500Z 2020-07-31 09:15:02.461 UTC [36] LOG: database system was shut down at 2020-07-31 09:15:01 UTC
2020-07-31T09:15:02.465323529Z 2020-07-31 09:15:02.465 UTC [35] LOG: database system is ready to accept connections
2020-07-31T09:15:02.524643142Z done
2020-07-31T09:15:02.524766601Z server started
2020-07-31T09:15:02.537508874Z psql: error: could not connect to server: could not translate host name "postgres" to address: Name does not resolve
*********
Pulling docker image ubuntu ...
Using docker image sha256:1e4467b07108685c38297025797890f0492c4ec509212e2e4b4822d367fe6bc8 for ubuntu ...
Preparing environment
00:02
Getting source from Git repository
00:01
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/badrmoh/cicdtest/.git/
Created fresh repository.
Checking out 604433de as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ until (echo > /dev/tcp/postgres/5432) >/dev/null 2>&1;do >&2 echo "service not ready...sleeping";sleep 5;done
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
gitlab-runner
The problem was in PGHOST variable. It seems it is used internally by postgres container that is why it fails to start.
The solution is to set PGHOST in script directive:
image: ubuntu
services:
- name: postgres:9
alias: postgres
variables:
POSTGRES_DB: badr
POSTGRES_USER: badr
POSTGRES_PASSWORD: badr
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- test
test db:
stage: test
before_script:
- export PGHOST=postgres
- until (echo > /dev/tcp/$PGHOST/5432) >/dev/null 2>&1;do >&2 echo "service $PGHOST not ready...sleeping";sleep 5;done
script:
- echo "connected to...$PGHOST"
- sleep 10
Note: You can't use variables directive within jobs in this case since it seems to be populated before even starting services itself.
On Docker Desktop (Windows 10), i'm unable to start postgres container with docker-compose
version: '3'
services:
postgres:
image: postgres:latest
environment:
- POSTGRES_PASSWORD='somepassword'
- POSTGRES_HOST_AUTH_METHOD=trust
Getting error that password is not set:
$ docker-compose up --build
Starting complex_postgres_1 ... done
Attaching to complex_postgres_1
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
complex_postgres_1 exited with code 1
However, on Linux (CentOS 8) all works fine, even without last line - POSTGRES_HOST_AUTH_METHOD=trust
Has anyone experienced same issue on Windows 10 ?
Docker desktop is on latest version as well as docker-compose command
The single quote in your password is also going as a part of password
version: '3'
services:
postgres:
image: postgres:latest
environment:
- POSTGRES_PASSWORD=somepassword
- POSTGRES_HOST_AUTH_METHOD=trust
Since yesterday, I try to launch my docker project on Windows (Bootcamp MacBook Pro), and I have just one issue left : PostgreSQL image.
Error message :
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting default max_connections ... 20
postgres_1 | selecting default shared_buffers ... 400kB
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | creating configuration files ... ok
postgres_1 | 2019-01-22 16:57:37.016 UTC [79] FATAL: data directory "/var/lib/postgresql/data" has wrong ownership
postgres_1 | 2019-01-22 16:57:37.016 UTC [79] HINT: The server must be started by the user that owns the data directory.
postgres_1 | child process exited with exit code 1
postgres_1 | initdb: removing contents of data directory "/var/lib/postgresql/data"
postgres_1 | running bootstrap script ... kibati-docker_postgres_1 exited with code 1
I've searched everywhere, tried everything, and still have this issue…
What I tried is :
Create docker volume with docker volume create --name postgres -d local and use it in my docker-compose.yml
docker-compose.yml
version: '2'
services:
postgres:
image: postgres:latest
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
networks:
internal_ip:
ipv4_address: 192.168.1.2
volumes:
postgres:
external: true
Add volume directly in docker compose
volume sample
volumes:
postgres:
driver: local
Using relative or absolute Windows paths
Declare multiple volumes for differents PostgreSQL folders (conf, data…)
I tried to docker compose down, restart computer, remove images, nothing change, same error.
I already checked Shared Drive box in Docker Settings.
Sources of what I've tried :
https://glennsarti.github.io/blog/puppet-in-docker/
https://forums.docker.com/t/trying-to-get-postgres-to-work-on-persistent-windows-mount-two-issues/12456/5
https://forums.docker.com/t/data-directory-var-lib-postgresql-data-pgdata-has-wrong-ownership/17963/28
https://github.com/docker-library/postgres/issues/435
http://www.lukaszewczak.com/2016/09/run-postgresql-using-docker-with.html
https://gdevops.gitlab.io/tuto_docker/tutoriels/postgresql/postgresql.html
https://devask.cz/questions/48645804/mount-postgres-data-to-windows-host-directory
Is anyone as a working solution ? I will continue to make it work, and update post if I found Something.
Thanks in advance.
I've finally figured out what went wrong when I tried to use a volume for PostgreSQL data.
I had no idea that we used a docker-compose.override.yml, which declare a volume with a Windows path.
So here is a working solution to have PostgreSQL on Docker for Windows, with persisted data :
version: '2'
services:
postgres:
image: postgres:11.5
ports:
- 5432:5432
volumes:
- pgdata:/var/lib/postgresql/data
- pgconf:/etc/postgresql
- pglog:/var/log/postgresql
volumes:
pgdata:
driver: local
pgconf:
driver: local
pglog:
driver: local
(no additional command required)
container_name: postgresql
image: postgres:latest
environment:
POSTGRES_PASSWORD: postgres
# volumes:
# - "./database:/var/lib/postgresql/data/"
ports:
- "5432:5432"
Comment the volumes
This work around did the trick and worked on windows, i have to bring those two lines back on mac but that's the easiest way to work for now until some real solution get pushed by docker windows or posgres docker
I was having issues connecting to my PG database in my compose file. I finally eliminated all other factors and made this bare-bones compose file:
version: '2.1'
services:
database:
image: postgres:9.5.6
environment:
POSTGRES_PASSWORD: secretpass
ports:
- 5432:5432
I then started it up with this command:
docker-compose --file docker-compose.pg.yml up --build
and it seemed to start ok:
Starting app_database_1 ...
Starting app_database_1 ... done
Attaching to app_database_1
database_1 | LOG: database system was shut down at 2017-09-22 18:48:55 UTC
database_1 | LOG: MultiXact member wraparound protections are now enabled
database_1 | LOG: database system is ready to accept connections
database_1 | LOG: autovacuum launcher started
Then from another shell, tried to connect with this command:
psql postgresql://postgres:secretpass#0.0.0.0
And got this output on my first shell where the compose was run:
database_1 | FATAL: password authentication failed for user "postgres"
database_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
I have also tried to connect from another container running
Rails defined in the compose file and get the same error. I always end up with an authentication error. What am I missing?