Docker runs PostgreSQL in "trust" mode - postgresql

I am currently learning docker and trying to run a docker container with the PostgreSQL database. I managed that once, and everything seemed to work fine. After some time, I tried to run another docker container with almost identical settings, however, it didn't go as expected. My problem is that now, whenever I try to run PostgreSQL container, initdb initializes the database in "trust" mode and accepts any connections without the password.
So far, I've tried running the command from the console:
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -p 32000:5432 -d postgres:14.5-alpine
As well as running the docker-compose.yaml:
services:
db:
container_name: Test_container
image: postgres:14.5-alpine
restart: unless-stopped
ports:
- "32000:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: mysecretpassword
Additionally, I tried ordering tags differently, different images, and different values, cleaning docker: removing all containers, images, and volumes, and even reinstalling docker, however, whenever I inspect logs of a newly created container, I get:
sh: locale: not found
2022-08-16 09:35:50.709 UTC [30] WARNING: no usable system locales were found
performing post-bootstrap initialization ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok
One of my assumptions was that docker, for some reason, doesn't see the password I am specifying and thus starts the database in "trust" mode, however, if I add
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test_db
to the docker-compose.yaml, test_db database is being created.
I'd appreciate any suggestions on how to make docker run PostgreSQL containers not in a "trust" mode as it should by default if the password is specified.
Juan González pointed out:
Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container.
So, according to the docs, I updated my docker-compose.yaml file:
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test_db
POSTGRES_HOST_AUTH_METHOD: scram-sha-256
POSTGRES_INITDB_ARGS: --auth-host=scram-sha-256
and once again tried swapping order and\or removing POSTGRES_INITDB_ARGS, but database still runs in "trust" mode.

As stated in Postgres' DockerHub documentation:
Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container.
However, if you don't want trust mode even in local connections, you can set the POSTGRES_HOST_AUTH_METHOD environment variable to override this behavior. More info at the documentation mentioned above.

As #jjanes pointed out in the comment to my question, the solution is to add POSTGRES_INITDB_ARGS: --auth=scram-sha-256 which would set both local and host types of connections.

Related

Connecting to docker PostgreSQL from PhpStorm gives password authentication failed

The problem
I am trying to connect to PostgreSQL from PhpStorm, but it returns the following error:
[28P01] FATAL: password authentication failed for user "app"
The situation
I have the following .env file setup:
POSTGRES_DB=app
POSTGRES_USER=app
POSTGRES_PASSWORD=password
POSTGRES_VERSION=15
And the following in docker-compose.yml:
version: '3'
services:
database:
image: postgres:${POSTGRES_VERSION}-alpine
container_name: database
env_file:
- .env
volumes:
- db-data:/var/lib/postgresql/data:rw
ports:
- '5432'
volumes:
db-data:
When running docker-compose up -d that does create a container & volume successfully.
So then I enter the following into my PhpStorm:
But then the error pops up, entering the password again doesn't fix anything.
I am running this on Ubuntu 22.04.1 LTS
What I've tried
I've rebuilt the container several times with different user & password combinations (making sure to use docker-compose down -v to get rid of the volume), all with the same result.
I've tried changing the password by executing docker exec -it database psql -U app and then running ALTER ROLE app WITH PASSWORD 'password', but this did not change anything.
I also saw online that it might have to do something with authentication of the user being setup as ident, but I cannot find a way to change this in the docker-compose.yml file.
The question
How could I set this up so I can connect my PhpStorm to the PostgreSQL database properly?
In my case I was running Windows 10 with WSL2 (Ubuntu). I had installed Postgres in the Ubuntu instance as part of setting up an app. I'd removed the app but the Postgres server was still running. When I attempted to connect to the Docker Postgres instance using localhost:5432 I was instead connecting to the WSL2 Postgres instance.
In my case, since I was no longer using Postgres in WSL2 I removed it and this resolved the issue. You could also stop it or use the host name/IP as mentioned by #jjanes

User not created in postgres docker image

I have a docker image that's not accepting credentials for a user that is defined in the yaml docker-compose file. When I go to the docker console for the container and check users it only lists postgres. Not sure what I am missing - here's the yaml file:
version: '3.8'
services:
db:
container_name: drewreport_container
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: mpassword
POSTGRES_USER: thedrewreport
POSTGRES_DB: thedrewreportdb
ports:
- "5432:5432"
volumes:
- thedrewreportdata:/var/lib/postgresql/data/
volumes:
thedrewreportdata:
Any ideas?
I can't reproduce your problem. Running docker-compose up, I see:
Creating network "docker_default" with the default driver
Creating volume "docker_thedrewreportdata" with default driver
Creating docker_client_1 ...
Creating docker_db_1 ...
Creating docker_client_1 ... done
Creating docker_db_1 ... done
Attaching to docker_db_1, docker_client_1
[...]
db_1 | 2021-07-19 23:03:39.676 UTC [1] LOG: database system is ready to accept connections
If I then connect with psql, I can authenticate using the username
and password you've defined in your docker-compose.yml:
# psql -h localhost -U thedrewreport thedrewreportdb
Password for user thedrewreport:
psql (13.3 (Debian 13.3-1.pgdg100+1))
Type "help" for help.
thedrewreportdb=# \du
List of roles
Role name | Attributes | Member of
---------------+------------------------------------------------------------+-----------
thedrewreport | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
thedrewreportdb=#
Note that any volumes specified in your docker-compose.yml will
persist between a docker-compose down and a docker-compose up, so
if you ever brought your stack up with different credentials, those
will never be replaced unless your explicitly destroy the volume by
running docker-compose down -v.
You can tell that docker-compose is re-using a volume if you don't
see a message like this when you run docker-compose up:
Creating volume "docker_thedrewreportdata" with default driver
If existing, the existing DB data you mount to /var/lib/postgresql/data/ will take precedence over the environment variables to initialize it.
You have 2 options:
Update the existing DB data to add your user / password / database. To do so you can use docker compose exec db bash and then connect using psql command to make your changes.
Delete or move your existing thedrewreportdata local volume, for instance updating it to ./thedrewreportdata_postgres:/var/lib/postgresql/data/
Once done, you can use docker compose exec db psql --username thedrewreport --dbname thedrewreportdb to doublecheck you can connect with your credentials to the updated DB.

Initialize PostgreSQL Container with docker-entrypoint-initdb.d script

I am trying to create a PostgreSQL 11.5 docker container. In doing so, I want to run a SQL script that creates the necessary users, tables, etc. However, whenever the container starts I see the following error:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
****************************************************
WARNING: No password has been set for the database.
This will allow anyone with access to the
Postgres port to access your database. In
Docker's default configuration, this is
effectively any other container on the same
system.
Use "-e POSTGRES_PASSWORD=password" to set
it in "docker run".
****************************************************
waiting for server to start....2019-09-16 17:16:26.568 UTC [42] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-09-16 17:16:26.677 UTC [43] LOG: database system was shut down at 2019-09-16 17:16:25 UTC
2019-09-16 17:16:26.691 UTC [42] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
/docker-entrypoint-initdb.d/init.sql: Permission denied
My Dockerfile looks like this:
FROM postgres:11.5
ADD ./scripts/init.sql /docker-entrypoint-initdb.d/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 5432
CMD ["postgres"]
And, my init.sql file looks like this:
CREATE USER mydb WITH PASSWORD 'password';
CREATE DATABASE mydb;
GRANT ALL PRIVILEGES ON DATABASE mydb TO mydb;
You'll notice neither of them does anything terribly complicated. However, I'm still getting the permission denied error. I've connected to the running container and confirmed that the init.sql file is in place on the filesystem. Any idea what I could be doing wrong here?
So from this Dockerfile I assume the user is postgress.
Try with this Dockerfile
FROM postgres:11.5
USER postgres
RUN whoami
ADD ./scripts/init.sql /docker-entrypoint-initdb.d/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 5432
CMD ["postgres"]
update:
Seems like the file not owned by Postgres user.
Try to set permission
ADD ./scripts/init.sql /docker-entrypoint-initdb.d/
RUN chown postgres:postgres /docker-entrypoint-initdb.d/init.sql
Initialize Postgres container with Data
Create a docker-compose.yml
version: '2'
services:
postgress-postgresql:
image: postgres:11.3
volumes:
# - ~/volumes/jhipster/postgress/postgresql/:/var/lib/postgresql/data/
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
environment:
- POSTGRES_USER=postgress
- POSTGRES_PASSWORD=
ports:
- 5432:5432
Create a init.sql with the script
CREATE USER platops WITH PASSWORD 'platops';
CREATE DATABASE platopsdb;
GRANT ALL PRIVILEGES ON DATABASE platopsdb TO platops;
RUN with docker-compose up -d
I had the same issue, mounted a .sh file via docker volumes. I checked permissions via ls -lah, in my case it was just -rw-r-----.
Using chmod 644 filename solved my issue.
The underlaying problem in our case was that the sql script was stored on a ntfs partition mounted with ntfs-3g which by default has got permissions' functionality disabled (https://superuser.com/questions/451475/chmod-doesnt-work). Running it on a normal ext4 partition solved the problem.
Disclaimer: I am aware it is not an answer to the question but it might shed some light why for some people it works and for others it does not.
Inside our team, it's working perfectly for the guy who's username is "admin" (literally). It does not work for me nor for out deployment server. Where our usernames are different.
Using "sudo" did not have any impact. It did not break his it did not fix ours.
My machine and his are MacOS. The server is Ubuntu.

password authentication failed for user "postgres" with docker-compose up on EC2

On EC2 linux server create by docker-machine, when I launch docker postgres:10.6 by docker-compose up, I have these loop errors :
FATAL: password authentication failed for user "postgres"
DETAIL: Password does not match for user "postgres".
Connection matched pg_hba.conf line 95: "host all all all md5"
I don't have these errors if I start container manually
=> docker run -e POSTGRES_PASSWORD=myPassword postgres:10.6
I don't have these errors in my local docker.
My docker-compose :
db:
container_name: postgres
image: postgres:10.6
restart: always
environment:
POSTGRES_PASSWORD: myPassword
ports:
- "5432:5432"
Does anyone know how to fix this?
It might be because the volume (or bind mount directory) being already initialized after your first start. The postgres user, and database creation only happens on the first start (ie, /var/lib/postgresql/data must not already contain database files).
Try to run:
docker-compose rm -fv postgres to delete any containers or volumes (in particular).
docker-compose up -d to start your container.
Sorry for that I have the answer to my question, it's not a bug I just have something that tries to connect permanently to postgres (through port 5432 which is open) ...
After search, I think it's an attempt at hacking because incoming connections never come from the same IP
connection received: host=45.120.120.149.81 port=47118
connection received: host=210.4.125.252 port=44774
connection received: host=82.223.55.254 port=36320
etc....

How to recreate Docker container?

I'm new to docker and I'm using docker compose. For some reason my postgres container is now broken
I'm trying this command docker-compose up --no-deps --build db
And it's returning me this:
MacBook-Pro-de-Javier:goxo.api javier$ docker-compose up --no-deps --build db
Recreating testapi_db_1
Attaching to testapi_db_1
db_1 | LOG: database system was shut down at 2017-04-20 17:19:05 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
Whenever I try to connect (with the same connection arguemnts than before) I get this:
^[[Adb_1 | FATAL: database "test" does not exist
This is part of my docker-compose.yml
version: "3"
services:
db:
image: postgres
ports:
- "3700:5432"
environment:
POSTGRES_HOST: "127.0.0.1"
POSTGRES_DB: "test"
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres1"
tmpfs:
- /tmp
- /var/run/postgresql
volumes:
- db:/var/lib/postgresql/data
- ./config/postgres-initdb.sh:/docker-entrypoint-initdb.d/initdb.sh
Any ideas on how can I recreate the docker image to be how it was before? It was working as it was created the first time
Thanks
EDIT 1: If I run docker-compose build && docker-compose up
Terminal throws this:
db uses an image, skipping
EDIT 2: This command does not create database again neither:
docker-compose up --force-recreate --abort-on-container-exit --build db
have you tried to rebuild your single postgres container?
docker build -t <postgrescontainer>
or with docker-compose:
docker-compose up --build
to recreate the images and not use the old 'used' ones.
You can have a look at the images on your system with
docker images
which should show your image, and then
docker history --no-trunc your_image
should show the commands used for the creation of the image
This my be insufficient, as when you see something like
ADD * /opt
you do not know exactly which files were copied, and what thoses files contained
There is also dockerfile-from-image
https://github.com/CenturyLinkLabs/dockerfile-from-image
which seems to have a bug recently (I do not know if it is fixed)