Docker compose and postgres official image environment variables - postgresql

Actually I'm using the following docker-compose.yml file
version: '3.3'
services:
postgres:
container_name: postgres
image: postgres:latest
restart: always
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
PGDATA: /var/lib/postgresql/data/pgdata
ports:
- "5432:5432"
volumes:
- ./data/postgres/pgdata:/var/lib/postgresql/data/pgdata
I use also this .env file in the same directory of the docker-compose.yml file:
POSTGRES_USER=dbadm
POSTGRES_PASSWORD=dbpwd
POSTGRES_DB=db
Then I run a bash shell into container this way:
docker exec -ti postgres bash
And after this invoke the command:
psql -h postgres -U dbadm db
And I get the error:
psql: FATAL: password authentication failed for user "dbadm"
The strange fact is that if use the default image parameters:
psql -h postgres -U admin database
And insert the default password "password", it logs me in, and seems it's ignoring the environment variables.
What am I missing?
Additional logs from docker-compose up logs:
postgres | 2017-10-15 09:19:15.502 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2017-10-15 09:19:15.502 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres | 2017-10-15 09:19:15.505 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2017-10-15 09:19:15.524 UTC [22] LOG: database system was shut down at 2017-10-15 09:02:21 UTC
postgres | 2017-10-15 09:19:15.530 UTC [1] LOG: database system is ready to accept connections
Cannot see any "RUN" line about user, database and password setup.

according to https://hub.docker.com/_/postgres documentation.
Warning: the Docker specific variables will only have an effect if you start the container with a data directory that is empty; any pre-existing database will be left untouched on container startup.

I have created a docker-compose yml and .env file with the details you've provided and everything works fine as you can see from the pictures below:
I think your problem lies when you are passing the -h parameter.
Inside the container will always be localhost however, outside you will have to pass:
hostname: postgres
inside your docker-compose file so it will have the postgres hostname

Related

Can't connect to Docker Hosted Postgres

My attempt to start a PostgreSQL Docker container fails when doing docker-compose. I can connect to it if I do docker run instead. I don't know what the difference is. My intent with the compose is to have it create a database usable for development and that I can connect to with user postgres.
docker-compose.yml:
version: '3.7'
services:
my_app:
build:
dockerfile: ./dockerfile-my-app
context: .
command: tail -f /dev/null
depends_on:
- db
env_file: ./.env.development
ports:
- "8080:8080"
volumes:
- /c/Users/woodsman/linux-mint-woodsman/home/dev:/home/dev
db:
image: postgres:latest
env_file: ./.env.development
environment:
- POSTGRES_USER= postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=helloworld
- PGDATA=/var/lib/postgresql/data/helloworld2/
ports:
- "5432:5432"
restart: "no"
volumes:
- /c/Users/woodsman/linux-mint-woodsman/dbdata:/var/lib/postgresql/data/pgdata
volumes:
dbdata:
Attempting to connect to jdbc:postgresql://localhost:5432/helloworld using user postgres and password postgres. Please pardon any security concerns you may perceive. This will be tightened for development, and done much more securely when deployed to production.
The reported error as reported by SquirrelSQL is:
hello-world: FATAL: password authentication failed for user "postgres"
class org.postgresql.util.PSQLException: FATAL: password authentication failed for user "postgres"
The image hashcode is:
9dbc24674f25eb449df11179ed3717c47348fb3aa985ae14b3936d54c2c09dde
>docker logs c4c2
2022-05-16 05:28:50.560 UTC [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-05-16 05:28:50.560 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2022-05-16 05:28:50.560 UTC [1] LOG: listening on IPv6 address "::", port 5432
2022-05-16 05:28:50.564 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-05-16 05:28:50.571 UTC [63] LOG: database system was shut down at 2022-05-16 05:28:50 UTC
2022-05-16 05:28:50.579 UTC [1] LOG: database system is ready to accept connections
2022-05-16 05:29:17.184 UTC [70] FATAL: password authentication failed for user "postgres"
2022-05-16 05:29:17.184 UTC [70] DETAIL: Role "postgres" does not exist.
Connection matched pg_hba.conf line 100: "host all all all scram-sha-256"
2022-05-16 05:36:41.983 UTC [78] FATAL: password authentication failed for user "postgres"
2022-05-16 05:36:41.983 UTC [78] DETAIL: Role "postgres" does not exist.
Connection matched pg_hba.conf line 100: "host all all all scram-sha-256"
I hope for a Docker based answer. If there's some PostgreSQL command I need to do (or not do), please tell me how I can pass that through Docker.
Thanks,
Woodsman
Your main error is:
Role "postgres" does not exist.
It seems because whitespace here in docker-compose.yml:
POSTGRES_USER= postgres
In first start postgres was initialized, so when You change "POSTGRES_USER" in variable, role will not create. You can delete you volume and start docker-compose again.

docker compose psql: error: FATAL: role "postgres" does not exist

I faced a problem when I try to use psql command with my docker-compose file on my local Ubuntu machine:
psql: error: FATAL: role "postgres" does not exist
I tried to use others solution like removing docker image, volume. psql -U postgres doesn't work for me either.
I try to use first docker-compose up, then docker exec -it database bash
There's my docker-compose file
services:
db:
container_name: postgres
image: postgres:13.3-alpine
restart: always
user: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=root
ports:
- "5432:5432"
volumes:
- ./data/db:/var/lib/postgresql/data
Maybe this string tells something?
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
OUTPUT:
Attaching to postgres
postgres |
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres |
postgres | 2021-08-02 17:29:10.426 UTC [1] LOG: starting PostgreSQL 13.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20210424) 10.3.1 20210424, 64-bit
postgres | 2021-08-02 17:29:10.426 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2021-08-02 17:29:10.426 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres | 2021-08-02 17:29:10.429 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2021-08-02 17:29:10.433 UTC [12] LOG: database system was shut down at 2021-08-02 17:22:17 UTC
postgres | 2021-08-02 17:29:10.438 UTC [1] LOG: database system is ready to accept connections
postgres | 2021-08-02 17:37:53.452 UTC [33] FATAL: role "postgres" does not exist
postgres | 2021-08-02 17:37:56.958 UTC [35] FATAL: role "user" does not exist
postgres | 2021-08-02 17:41:54.294 UTC [45] FATAL: role "postgres" does not exist```
First, you've set POSTGRES_USER to root, so you're going to have a root user instead of postgres user.
Second, if a database already exists, it doesn't matter what you set for POSTGRES_USER and POSTGRES_PASSWORD -- postgres will use whatever is in the database.
So you can either:
Delete the database (rm -rf data/db) and start over, or
Edit your pg_hba.conf so that you don't need a password
docker-compose exec web rake db:migrate db:create - I have above error: role "..." does not exist.
When I try:
docker-compose exec web rake db:drop
docker-compose exec web rake db:setup
it's working.

How to connect to postgres created with docker-compose from outside host

I think this problem should be very easy to solve, but I am pulling my hair out. I have the following docker-compose:
version: "3"
services:
pg-db:
image: postgres:11
environment:
POSTGRES_PASSWORD: "postgres"
POSTGRES_USER: "postgres"
POSTGRES_DB: "postgres"
ports:
- 5432:5432
#network_mode: host
Then I run docker-compose up and it starts the container
pg-db_1_78e7ec4a95e6 | 2020-02-21 13:53:53.928 UTC [1] LOG: database system is ready to accept connections
I can connect to it with docker exec
docker exec -it docker_pg-db-compose_1_78e7ec4a95e6 psql -h pg-db -U postgres postgres
But I can't manage to connect to it by 'naked' psql:
psql postgresql://postgres:postgres#localhost:5432/postgres
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
psql postgresql://postgres:postgres#pg-dg:5432/postgres
psql: could not translate host name "pg-db" to address: Name or service not known
I've tried with network_mode but it doesn't help.
After starting docker-compose, you need to get the docker container ip by doing this:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
Then you can do postgresql://postgres:postgres#<container idp>:5432/postgres you should be able to connect to it
I'm using a modified version of the docker-compose file provided in the Postgres DockerHub page.
version: '3.1'
services:
db:
image: postgres:11
restart: always
ports:
- 5432:5432
environment:
POSTGRES_PASSWORD: postgres
When I run docker-compose up I get a very similar "ready" message:
db_1 | 2020-02-21 14:34:17.826 UTC [1] LOG: database system is ready to accept connections
At this point, I can connect to Postgres using psql.
$ psql -h localhost -U postgres
Password for user postgres:
psql (12.2, server 11.7 (Debian 11.7-1.pgdg90+1))
Type "help" for help.
postgres=#
I'm also able to connect using the connection string that you provided, and postico:
The differences are subtle, I'm not sure if the restart policy makes any difference here, but give it a try.

postgres with docker compose gives FATAL: role "root" does not exist error

I'm trying to create a simple demo with postgres on a local windows machine with docker desktop.
This is my yaml docker compose file named img.yaml:
version: '3.6'
services:
postgres-demo:
image: postgres:11.5-alpine
container_name: postgres-demo
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=Welcome
- POSTGRES_DB=conference_app
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
ports:
- 5432:5432
volumes:
- .:/var/lib/my_data
restart: always
I'm running it using the command:
docker-compose -f img.yaml up
And get the following output:
Starting postgres-demo ... done
Attaching to postgres-demo
postgres-demo | 2020-02-12 17:07:46.487 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres-demo | 2020-02-12 17:07:46.487 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres-demo | 2020-02-12 17:07:46.508 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres-demo | 2020-02-12 17:07:46.543 UTC [18] LOG: database system was shut down at 2020-02-12 17:07:10 UTC
postgres-demo | 2020-02-12 17:07:46.556 UTC [1] LOG: database system is ready to accept connections
And then, opening bash into the container with the command:
docker exec -it d47056217a97 bash
I want to watch the databases in container so I run in the bash the command:
psql \dt
And get the error:
psql: FATAL: role "root" does not exist.
Trying to create the database using the command: psql> create database conference_app; gives the error: psql: FATAL: role "conference_app" does not exist.
I'm puzzled. What am I doing wrong? Is my yaml missing something?
If you don’t specify the PGUSER environment variable, then psql will assume you want to use the current OS user as your database user name. In this case, you are using root as your OS user, and you will attempt to log in as root, but that user doesn’t exist in the database.
You’ll need to either call psql -U postgres, or su - Postgres first
See also the postgresql documentation
UPDATE: Someone suggested changing PGUSER to POSTGRES_USER -- this is actually incorrect. Postgres looks for PGUSER in the environment, but if you're using Docker, you'll tell Docker the correct user by using POSTGRES_USER, which gets assigned to PGUSER -- see the entrypoint source code
I specified user: postgres for the service in docker-compose file. Then deleted the existing container to re-spin it with the next docker-compose up execution. Container came up with the user "postgres", so you just need psql -l from there onwards(don't need -U flag)
This was my docker-compose file
version: '3.1'
services:
db:
image: postgres:12.6-alpine
restart: always
container_name: postgres12_6
user: postgres
environment:
- "POSTGRES_PASSWORD=postgres"
- "ES_JAVA_OPTS=-Xms1024m -Xmx3072m"
networks:
- esnet
adminer:
image: adminer
restart: always
ports:
- 8080:8080
networks:
esnet:
The container assumes that you are trying to connect to db with root user(current user) since you dont specify the user and database name on docker exec.
Following should work:
docker exec -it <container_id/container_name> psql -U <user_name> <database_name>
docker exec -it d47056217a97 psql -U postgres conference_app
in my case, I was using windows 10, I tried everything I could but still not works.
finally, I removed all the related docker image, docker container, local host folder, and restart windows.
and everything goes well

Docker-Compose + postgres: database and user creation not working (works without compose)

The docs for the postgres Docker image explain that you can make the image create a user and database on creation, using environment variables.
I can't seem to make that work using docker-compose:
# docker-compose.yml
services:
postgresql:
image: postgres:alpine
environment:
POSTGRES_DB: iotplatform
POSTGRES_USER: iotplatform
POSTGRES_PASSWORD: iotplatform
and here's what I run:
docker-compose up -d --force-recreate postgresql
docker-compose exec postgresql psql -U iotplatform
# psql: FATAL: role "iotplatform" does not exist
When I run docker-compose exec postgresql env, I see the environment variables as configured.
The logs don't say anything particular:
Attaching to iot-container-tracker_postgresql_1
postgresql_1 | 2018-12-17 17:35:05.754 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgresql_1 | 2018-12-17 17:35:05.754 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgresql_1 | 2018-12-17 17:35:05.757 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgresql_1 | 2018-12-17 17:35:05.770 UTC [21] LOG: database system was shut down at 2018-12-17 17:35:03 UTC
postgresql_1 | 2018-12-17 17:35:05.772 UTC [1] LOG: database system is ready to accept connections
postgresql_1 | 2018-12-17 17:35:22.639 UTC [34] FATAL: role "iotplatform" does not exist
EDIT: tried without docker-compose and it works.
docker run --name some-postgres -e POSTGRES_PASSWORD=iotplatform -e POSTGRES_DB=iotplatform -e POSTGRES_USER=iotplatform -d postgres:alpine
docker exec -it some-postgres psql -U iotplatform
iotplatform=#
What am I missing?
After running down and up it works. Apparently --force-recreate wasn't a hard enough reset.
Thanks #StéphaneJeandeaux for your help.