I am using docker-compose.yml to create multiple running containers but failing to start Postgres docker server, with following logs and yes I have searched many related SO posts, but they didn't helped me out.
Creating network "complex_default" with the default driver
Creating complex_server_1 ... done
Creating complex_redis_1 ... done
Creating complex_postgres_1 ... done
Attaching to complex_postgres_1, complex_redis_1, complex_server_1
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
complex_postgres_1 exited with code 1
below is my docker-compose configuration:
version: '3'
services:
postgres:
image: 'postgres:11-alpine'
redis:
image: 'redis:latest'
server:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
as well as package.json inside server directory is following:
{
"dependencies": {
"body-parser": "^1.19.0",
"cors": "^2.8.4",
"express": "^4.16.3",
"nodemon": "^2.0.4",
"pg": "7.4.3",
"redis": "^2.8.0"
},
"scripts": {
"dev": "nodemon",
"start": "node index.js"
}
}
also for better consideration, I have attached my hands-on project structure:
A year ago it were actually working fine, Does anyone have any idea, what's going wrong here inside my docker-compose file now.
A year ago it were actually working fine, Does anyone have any idea, what's going wrong here inside my docker-compose file now.
Seems like you pulled the fresh image, where in the new image you should specify Postgres user password. You can look into Dockerhub, the image is update one month ago
postgress-11-alpine
db:
image: postgres:11-alpine
restart: always
environment:
POSTGRES_PASSWORD: example
As the error message is self expalinatory
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
or POSTGRES_HOST_AUTH_METHOD=trust use this which is not recommended.
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is *not* recommended.
POSTGRES_PASSWORD
This environment variable is required for you to use the PostgreSQL image. It must not be empty or undefined. This environment variable sets the superuser password for PostgreSQL. The default superuser is defined by the POSTGRES_USER environment variable.
Environment Variables
#Adiii yes, you are nearly right, so I have to explicitly mentioned the environment also for the postgres image but with no db parent tag.
So here, I am explicitly mentioning the docker-compose.yaml config to help others for better understanding, also now I am using recent stable postgres image version 12-alpine, currently latest is postgres:12.3
version: '3'
services:
postgres:
image: 'postgres:12-alpine'
environment:
POSTGRES_PASSWORD: postgres_password
redis:
image: 'redis:latest'
server:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
and so after docker-compose up the creating and running logs were as following:
PS E:\docker\complex> docker-compose up
Creating network "complex_default" with the default driver
Creating complex_postgres_1 ... done
Creating complex_redis_1 ... done
Creating complex_server_1 ... done
Attaching to complex_redis_1, complex_postgres_1, complex_server_1
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default time zone ... UTC
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # Redis version=6.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
postgres_1 | creating configuration files ... ok
redis_1 | 1:M 05 Aug 2020 14:24:48.693 * Running mode=standalone, port=6379.
redis_1 | 1:M 05 Aug 2020 14:24:48.693 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 05 Aug 2020 14:24:48.694 # Server initialized
redis_1 | 1:M 05 Aug 2020 14:24:48.694 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 05 Aug 2020 14:24:48.694 * Ready to accept connections
postgres_1 | running bootstrap script ... ok
server_1 |
server_1 | > # dev /app
server_1 | > nodemon
server_1 |
postgres_1 | performing post-bootstrap initialization ... sh: locale: not found
postgres_1 | 2020-08-05 14:24:50.153 UTC [29] WARNING: no usable system locales were found
server_1 | [nodemon] 2.0.4
server_1 | [nodemon] to restart at any time, enter `rs`
server_1 | [nodemon] watching path(s): *.*
server_1 | [nodemon] watching extensions: js,mjs,json
server_1 | [nodemon] starting `node index.js`
postgres_1 | ok
server_1 | Listening
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | initdb: warning: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2020-08-05 14:24:51.634 UTC [34] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-08-05 14:24:51.700 UTC [34] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-08-05 14:24:51.981 UTC [35] LOG: database system was shut down at 2020-08-05 14:24:50 UTC
postgres_1 | 2020-08-05 14:24:52.040 UTC [34] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | waiting for server to shut down....2020-08-05 14:24:52.121 UTC [34] LOG: received fast shutdown request
postgres_1 | 2020-08-05 14:24:52.186 UTC [34] LOG: aborting any active transactions
postgres_1 | 2020-08-05 14:24:52.188 UTC [34] LOG: background worker "logical replication launcher" (PID 41) exited with exit code 1
postgres_1 | 2020-08-05 14:24:52.188 UTC [36] LOG: shutting down
postgres_1 | 2020-08-05 14:24:52.669 UTC [34] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-08-05 14:24:52.954 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-08-05 14:24:53.136 UTC [43] LOG: database system was shut down at 2020-08-05 14:24:52 UTC
postgres_1 | 2020-08-05 14:24:53.194 UTC [1] LOG: database system is ready to accept connections
Hope this would help manyone.
Adding on ArifMustafa's answer, This worked for me.
postgres:
image: 'postgres:12-alpine'
environment:
POSTGRES_PASSWORD: mypassword
expose:
- 5432
volumes:
- postgres_data:/var/lib/postgres/data/
volumes:
postgres_data:
In my case there was an error about the POSTGRES_PASSWORD (in docker-compose.yml) and the DATABASE settings for PASSWORD in the project_level settings.py file.
After providing a password in docker-compose.yml
db: # For the PostgreSQL database
image: postgres:11
environment:
- POSTGRES_PASSWORD=example
It is necessary to provide the same password to enable access to the POSTGRES database between the two containers (in my case the web application and the database it depended on). In the project_level settings.py:
# Database
# https://docs.djangoproject.com/en/2.2/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'example',
'HOST': 'db',
'PORT': 5432
}
}
To resolve the error when using the command,
docker pull postgres
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
or POSTGRES_HOST_AUTH_METHOD=trust use this which is not recommended.
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all connections without a password. This is not recommended.
Here comes the solution for this:
$ docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
The default postgres user and database are created in the entrypoint with initdb.
The postgres database is a default database meant for use by users, utilities and third party applications.
postgresql.org/docs
Hello for some reason my postgres keeps restarting basically this is my docker compose:
version: "3.7"
services:
db:
image: postgres:12
restart: always
container_name: "db"
ports:
- "${DB_PORT}:5432"
volumes:
- ./pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${DB_USER}
POSTGRES_PASSWORD: ${DB_PASS}
POSTGRES_DB: ${DB_NAME}
api:
image: server_emasa
container_name: api
restart: always
depends_on:
- db
ports:
- "${SERVER_PORT}:${SERVER_PORT}"
and for some reason my postgres is restarting I tried using the docker compose logs to check and got this:
db | PostgreSQL Database directory appears to contain a database; Skipping initialization
db |
db | 2020-03-26 05:37:18.475 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db | 2020-03-26 05:37:18.475 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db | 2020-03-26 05:37:18.475 UTC [1] LOG: listening on IPv6 address "::", port 5432
db | 2020-03-26 05:37:18.558 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db | 2020-03-26 05:37:18.625 UTC [1] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.646 UTC [26] LOG: database system was interrupted; last known up at 2020-03-23 23:46:31 UTC
db | 2020-03-26 05:37:18.879 UTC [26] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.879 UTC [26] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.879 UTC [26] FATAL: could not open directory "pg_replslot": No such file or directory
db | 2020-03-26 05:37:18.880 UTC [1] LOG: startup process (PID 26) exited with exit code 1
db | 2020-03-26 05:37:18.880 UTC [1] LOG: aborting startup due to startup process failure
db | 2020-03-26 05:37:18.881 UTC [1] LOG: database system is shut down
I also had a similar problem after restarting my postgresql container:
LOG: could not open directory "pg_tblspc/memory/...": Not a directory
The reason of the error is probably because of data corruption in database.
You can solve the issue by deleting contents of pg_tblspc/ directory (make sure to backup all of them) or you can also try by recovering the latest backup of your database (if there's one).
The error refers to postgresql not being able to find critical files, but is not attempting to create them because your pgdata directory contains some files.
Try not to bind your pgdata directory to the docker container, just remove the volumes section in your docker-compose file
first remove associated docker volume and data folder then:
POSTGRES_DB: ${DB_NAME} will not work, use like this:
environment:
- 'POSTGRES_DB=med_db'
- 'POSTGRES_USER:postgres'
- 'POSTGRES_PASSWORD:pass'
Since yesterday, I try to launch my docker project on Windows (Bootcamp MacBook Pro), and I have just one issue left : PostgreSQL image.
Error message :
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting default max_connections ... 20
postgres_1 | selecting default shared_buffers ... 400kB
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | creating configuration files ... ok
postgres_1 | 2019-01-22 16:57:37.016 UTC [79] FATAL: data directory "/var/lib/postgresql/data" has wrong ownership
postgres_1 | 2019-01-22 16:57:37.016 UTC [79] HINT: The server must be started by the user that owns the data directory.
postgres_1 | child process exited with exit code 1
postgres_1 | initdb: removing contents of data directory "/var/lib/postgresql/data"
postgres_1 | running bootstrap script ... kibati-docker_postgres_1 exited with code 1
I've searched everywhere, tried everything, and still have this issue…
What I tried is :
Create docker volume with docker volume create --name postgres -d local and use it in my docker-compose.yml
docker-compose.yml
version: '2'
services:
postgres:
image: postgres:latest
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
networks:
internal_ip:
ipv4_address: 192.168.1.2
volumes:
postgres:
external: true
Add volume directly in docker compose
volume sample
volumes:
postgres:
driver: local
Using relative or absolute Windows paths
Declare multiple volumes for differents PostgreSQL folders (conf, data…)
I tried to docker compose down, restart computer, remove images, nothing change, same error.
I already checked Shared Drive box in Docker Settings.
Sources of what I've tried :
https://glennsarti.github.io/blog/puppet-in-docker/
https://forums.docker.com/t/trying-to-get-postgres-to-work-on-persistent-windows-mount-two-issues/12456/5
https://forums.docker.com/t/data-directory-var-lib-postgresql-data-pgdata-has-wrong-ownership/17963/28
https://github.com/docker-library/postgres/issues/435
http://www.lukaszewczak.com/2016/09/run-postgresql-using-docker-with.html
https://gdevops.gitlab.io/tuto_docker/tutoriels/postgresql/postgresql.html
https://devask.cz/questions/48645804/mount-postgres-data-to-windows-host-directory
Is anyone as a working solution ? I will continue to make it work, and update post if I found Something.
Thanks in advance.
I've finally figured out what went wrong when I tried to use a volume for PostgreSQL data.
I had no idea that we used a docker-compose.override.yml, which declare a volume with a Windows path.
So here is a working solution to have PostgreSQL on Docker for Windows, with persisted data :
version: '2'
services:
postgres:
image: postgres:11.5
ports:
- 5432:5432
volumes:
- pgdata:/var/lib/postgresql/data
- pgconf:/etc/postgresql
- pglog:/var/log/postgresql
volumes:
pgdata:
driver: local
pgconf:
driver: local
pglog:
driver: local
(no additional command required)
container_name: postgresql
image: postgres:latest
environment:
POSTGRES_PASSWORD: postgres
# volumes:
# - "./database:/var/lib/postgresql/data/"
ports:
- "5432:5432"
Comment the volumes
This work around did the trick and worked on windows, i have to bring those two lines back on mac but that's the easiest way to work for now until some real solution get pushed by docker windows or posgres docker
I was having issues connecting to my PG database in my compose file. I finally eliminated all other factors and made this bare-bones compose file:
version: '2.1'
services:
database:
image: postgres:9.5.6
environment:
POSTGRES_PASSWORD: secretpass
ports:
- 5432:5432
I then started it up with this command:
docker-compose --file docker-compose.pg.yml up --build
and it seemed to start ok:
Starting app_database_1 ...
Starting app_database_1 ... done
Attaching to app_database_1
database_1 | LOG: database system was shut down at 2017-09-22 18:48:55 UTC
database_1 | LOG: MultiXact member wraparound protections are now enabled
database_1 | LOG: database system is ready to accept connections
database_1 | LOG: autovacuum launcher started
Then from another shell, tried to connect with this command:
psql postgresql://postgres:secretpass#0.0.0.0
And got this output on my first shell where the compose was run:
database_1 | FATAL: password authentication failed for user "postgres"
database_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
I have also tried to connect from another container running
Rails defined in the compose file and get the same error. I always end up with an authentication error. What am I missing?
When using the https://github.com/kiasaki/docker-alpine-postgres image in docker compose, the scripts inside /docker-entrypoint.initdb.d are not being executed as I view from the logs:
db_1 | LOG: database system was shut down at 2016-09-05 18:16:02 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
The thing is, if I build the Dockerfile standalone and run the container, it will execute the sql files inside the folder, like you can see from the logs:
waiting for server to start....LOG: database system was shut down at 2016-09-05 18:28:18 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/syna-setup.sql
CREATE TABLE
INSERT 0 1
CREATE TABLE
waiting for server to shut down...LOG: received fast shutdown request
.LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
done
server stopped
LOG: database system was shut down at 2016-09-05 18:28:21 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
The docker-compose.yml:
version: '2'
services:
db:
build: ./db
environment:
- POSTGRES_PASSWORD=testpass
restart: always
app:
build: ./app
ports:
- "5000:5000"
volumes:
- .:/app/src
depends_on:
- db
environment:
- DB_SERVER=postgres://postgres:testpass#db:5432/postgres
What am I doing wrong here?
UPDATED as he use his own Dockerfile, not from the github link.
Could you proide the docker run command so that if I build the Dockerfile standalone and run the container, it will execute the sql files inside the folder. You should mount some directory to the container so it can find the SQL files.
I read the Dockerfile at https://github.com/kiasaki/docker-alpine-postgres/blob/master/Dockerfile and asked myself where are the SQL files? It just make the directory without copy SQL files
mkdir /docker-entrypoint-initdb.d
Then, in the docker-entrypoint.sh, it check if there's any SQL file to execute
for f in /docker-entrypoint-initdb.d/*; do
case "$f" in
*.sh) echo "$0: running $f"; . "$f" ;;
*.sql) echo "$0: running $f"; psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < "$f" && echo ;;
*) echo "$0: ignoring $f" ;;
esac
echo
done
I think we should mount a SQL directory, like
version: '2'
services:
db:
build: ./db
environment:
- POSTGRES_PASSWORD=testpass
restart: always
entrypoint:
- docker-entrypoint.sh
volumes:
- ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
- ./docker-entrypoint.sh:/docker-entrypoint.sh