I'm trying to use the official postgres docker files (with the aim of extending them) but if I run them as a non-root local user, they simply refuse to start.
i.e. If I follow the basic instructions from https://hub.docker.com/_/postgres and run:
docker run -e POSTGRES_PASSWORD=mysecretpassword postgres
then I get:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2023-02-18 09:55:58.427 UTC [48] LOG: starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-02-18 09:55:58.452 UTC [48] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-02-18 09:55:58.525 UTC [51] LOG: database system was shut down at 2023-02-18 09:55:49 UTC
2023-02-18 09:55:58.550 UTC [48] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
2023-02-18 09:55:58.603 UTC [48] LOG: received fast shutdown request
waiting for server to shut down....2023-02-18 09:55:58.625 UTC [48] LOG: aborting any active transactions
2023-02-18 09:55:58.626 UTC [48] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
2023-02-18 09:55:58.626 UTC [49] LOG: shutting down
2023-02-18 09:55:58.650 UTC [49] LOG: checkpoint starting: shutdown immediate
2023-02-18 09:55:58.835 UTC [49] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.047 s, sync=0.023 s, total=0.209 s; sync files=2, longest=0.012 s, average=0.012 s; distance=0 kB, estimate=0 kB
2023-02-18 09:55:58.840 UTC [48] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2023-02-18 09:55:58.967 UTC [1] LOG: starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-02-18 09:55:58.968 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-02-18 09:55:58.969 UTC [1] LOG: could not create IPv6 socket for address "::": Address family not supported by protocol
2023-02-18 09:55:59.017 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-02-18 09:55:59.063 UTC [62] LOG: database system was shut down at 2023-02-18 09:55:58 UTC
2023-02-18 09:55:59.091 UTC [1] LOG: database system is ready to accept connections
and everything is happy.
However, if I following the instructions under "Arbitrary --user Notes"
and run:
docker run -it --rm --user "$(id -u):$(id -g)" -v /etc/passwd:/etc/passwd:ro -e POSTGRES_PASSWORD=mysecretpassword postgres
(or without the it or the rm or with just the user and not the group - makes no difference)
then I get:
chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted
chmod: changing permissions of '/var/run/postgresql': Operation not permitted
The files belonging to this database system will be owned by user "richard".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... initdb: error: could not change permissions of directory "/var/lib/postgresql/data": Operation not permitted
What am I missing / doing wrong?
Note that Switching Between Root and Non-Root Users in Docker is very out of date and that the answer given as a comment under How to create a postgres container with a non-root user? is simply what I'm trying to do here.
Running unprivileged, the official postgres image needs to run from a fixed UID - matching their image, you can't just re-use your own uid and /etc/passwd.
Try this:
docker run -it --rm --user "999:999" -e POSTGRES_PASSWORD=mysecretpassword postgres
Is there any way we could stop replication without logging into psql shell.
Disk-full situation lead to some corruption in PG files and keep on restarting.
2023-02-06 08:17:54 UTC [1] LOG: starting PostgreSQL 13.7 (Ubuntu 13.7-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, 64-bit
2023-02-06 08:17:54 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-02-06 08:17:54 UTC [1] LOG: listening on IPv6 address "::", port 5432
2023-02-06 08:17:54 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-02-06 08:17:54 UTC [8] LOG: database system was shut down at 2023-02-06 08:17:45 UTC
2023-02-06 08:17:54 UTC [8] PANIC: could not open file "pg_replslot/slot_name/state": No such file or directory
2023-02-06 08:17:55 UTC [1] LOG: startup process (PID 8) was terminated by signal 6: Aborted
2023-02-06 08:17:55 UTC [1] LOG: aborting startup due to startup process failure
2023-02-06 08:17:55 UTC [1] LOG: database system is shut down
Tried removing pg_replslot/slot_name which lead to "password auth failure" and After resetting DB password(via pg_hba.conf) DB is not showing up !
Is there any proper way to recover in this state? /pg/main files and pgdata directories seem to be available except this slot information.
Done below steps:
I'm using PSQL docker container.
disk used for PG got full. Cleaned up some log files and docker system prune was used to remove unused images which freed some space. But lead to this issue.
Multiple times, we have seen similar issue in Dev environments, Disk full leading to some corrupted files (unable to read/ No such file or directory) kind of errors.
Tried removing pg_replslot/slot_name directory and it allowed me to start PSQL container.(previously is was keep on restarting container)
Reset password by using trust in auth column in pg_hbda.conf.
Now \l in psql shell showing only postgres DB and default DB's. Not showing our custom DB.
We have main DB in a separate tablespace and is not showing up in the list.
_ MOST importantly, Standby is also having SAME errors ! Probably someone messed it?
I am trying to run postgreSQL via docker-compose and I am getting the issue that user/password is not created when I started the service.
version: "3"
services:
db:
image: postgres:latest
container_name: postgres
#volumes:
#- postgres-data:/var/lib/postgresql/data
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=postgrespassword
- POSTGRES_USER=postgres
- POSTGRES_DB=random_db_name
restart: always
I have this block of code in my docker-compose.yml and I run the following command:
docker-compose up -d (this allow me to start the service in background)
and when I check the logs I got:
docker logs -f 0e1731f95396
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2021-04-27 16:20:44.592 UTC [49] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-04-27 16:20:44.594 UTC [49] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-04-27 16:20:44.603 UTC [50] LOG: database system was shut down at 2021-04-27 16:20:44 UTC
2021-04-27 16:20:44.609 UTC [49] LOG: database system is ready to accept connections
done
server started
CREATE DATABASE
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
waiting for server to shut down...2021-04-27 16:20:44.889 UTC [49] LOG: received fast shutdown request
.2021-04-27 16:20:44.891 UTC [49] LOG: aborting any active transactions
2021-04-27 16:20:44.892 UTC [49] LOG: background worker "logical replication launcher" (PID 56) exited with exit code 1
2021-04-27 16:20:44.892 UTC [51] LOG: shutting down
2021-04-27 16:20:44.907 UTC [49] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2021-04-27 16:20:45.018 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-04-27 16:20:45.019 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-04-27 16:20:45.019 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-04-27 16:20:45.023 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-04-27 16:20:45.029 UTC [77] LOG: database system was shut down at 2021-04-27 16:20:44 UTC
2021-04-27 16:20:45.034 UTC [1] LOG: database system is ready to accept connections
But when I try to connect to this database locally I get the message: "FATAL: role "postgres" does not exist"
Do you have any input about how to solve this problem?
I already made a few attempts after reading a few comments from different places but I got always the same problem.
I was expecting to run locally postgreSQL and setup already a user/password and a Database with that name in the docker-compose
I am trying to set up a postgres container to start and run initializing the creation of a table. I've succeeded with the straight image from docker but now that I am trying to extend the image a little to create tables when it's produced and I can't get it running. Based off what I've read here How to create User/Database in script for Docker Postgres, this is what I have:
Dockerfile:
FROM library/postgres
COPY init.sql /docker-entrypoint-initdb.d/
init.sql:
CREATE TABLE incident_disposition (
incident_disposition_code VARCHAR,
incident_disposition_code_description VARCHAR
);
From what I understand, FROM library . . . pulls the postgres image from docker hub and the COPY pushes my init.sql script into the entry point so there is no need for a big dockerfile correct?
I then build the image no issue:
Build
docker build -t my_postgres_image .
But when I run I get the issues:
Run
docker run --name testing my_postgres_image --publish 8000:8080 --detach -e POSTGRES_PASSWORD=postgres -d postgres
Errors from logs
Error: Database is uninitialized and superuser password is not specified.
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is *not* recommended.
See PostgreSQL documentation about "trust":
https://www.postgresql.org/docs/current/auth-trust.html
Attempt from comment:
docker container logs testing
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2020-03-26 14:06:51.064 UTC [46] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-03-26 14:06:51.072 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-03-26 14:06:51.108 UTC [47] LOG: database system was shut down at 2020-03-26 14:06:50 UTC
2020-03-26 14:06:51.119 UTC [46] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
CREATE TABLE
2020-03-26 14:06:51.231 UTC [46] LOG: received fast shutdown request
waiting for server to shut down....2020-03-26 14:06:51.232 UTC [46] LOG: aborting any active transactions
2020-03-26 14:06:51.233 UTC [46] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
2020-03-26 14:06:51.234 UTC [48] LOG: shutting down
2020-03-26 14:06:51.290 UTC [46] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2020-03-26 14:06:51.345 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-03-26 14:06:51.345 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-03-26 14:06:51.345 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-03-26 14:06:51.361 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-03-26 14:06:51.387 UTC [64] LOG: database system was shut down at 2020-03-26 14:06:51 UTC
2020-03-26 14:06:51.398 UTC [1] LOG: database system is ready to accept connections
2020-03-26 14:07:27.715 UTC [72] ERROR: relation "incident_disposition" does not exist at character 15
2020-03-26 14:07:27.715 UTC [72] STATEMENT: select * from incident_disposition;
In addition to comments
Due to recent docker image's updates postgres images do not allow to connect to DB without a password from anywhere. So you need to specify username/password
docker run -p 8000:8080 -e POSTGRES_PASSWORD=postgres --name testing -d my_postgres_image
Or if you still don't want to use password, you can just set POSTGRES_HOST_AUTH_METHOD=trust environment variable:
docker run -p 8000:8080 -e POSTGRES_HOST_AUTH_METHOD=trust --name testing -d my_postgres_image
It is a typical Initialization scripts issue.
You can file the full explaination in postgresql docker page. https://hub.docker.com/_/postgres
Here is the brief intro:
1. One common problem is that if one of your /docker-entrypoint-initdb.d scripts fails (which will cause the entrypoint script to exit) and your orchestrator restarts the container with the already initialized data directory, it will not continue on with your scripts.
note:
in your case, you may need clean the historical docker containers(stopped) by
step 1: docker ps |grep
step 2: docker rm -f -v
Or if you are using docker-compose, the historical orchestrator could be easily removed by docker-compose down -v.
I use postgres:10 (https://hub.docker.com/_/postgres/) image for DB.
It is deployed in docker swarm cluster.
After running DB replica I got database system is shut down in DB's log.
2018-05-11 10:26:53.073 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432,
2018-05-11 10:26:53.073 UTC [1] LOG: listening on IPv6 address "::", port 5432,
2018-05-11 10:26:53.077 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432",
2018-05-11 10:26:53.092 UTC [20] LOG: database system was shut down at 2018-05-11 10:26:20 UTC,
2018-05-11 10:26:53.100 UTC [1] LOG: database system is ready to accept connections,
The files belonging to this database system will be owned by user "postgres".,
This user must also own the server process.,
,
The database cluster will be initialized with locale "en_US.utf8".,
The default database encoding has accordingly been set to "UTF8".,
The default text search configuration will be set to "english".,
,
Data page checksums are disabled.,
,
fixing permissions on existing directory /var/lib/postgresql/data ... ok,
creating subdirectories ... ok,
selecting default max_connections ... 100,
selecting default shared_buffers ... 128MB,
selecting dynamic shared memory implementation ... posix,
creating configuration files ... ok,
running bootstrap script ... ok,
performing post-bootstrap initialization ... ok,
,
WARNING: enabling "trust" authentication for local connections,
You can change this by editing pg_hba.conf or using the option -A, or,
--auth-local and --auth-host, the next time you run initdb.,
syncing data to disk ... ok,
,
Success. You can now start the database server using:,
,
pg_ctl -D /var/lib/postgresql/data -l logfile start,
,
waiting for server to start....2018-05-11 09:39:21.129 UTC [37] LOG: listening on IPv4 address "127.0.0.1", port 5432,
2018-05-11 09:39:21.130 UTC [37] LOG: could not bind IPv6 address "::1": Cannot assign requested address,
2018-05-11 09:39:21.130 UTC [37] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.,
2018-05-11 09:39:21.133 UTC [37] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432",
2018-05-11 09:39:21.147 UTC [38] LOG: database system was shut down at 2018-05-11 09:39:20 UTC,
2018-05-11 09:39:21.152 UTC [37] LOG: database system is ready to accept connections,
done,
server started,
CREATE DATABASE,
,
CREATE ROLE,
,
,
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*,
,
2018-05-11 09:39:21.595 UTC [37] LOG: received fast shutdown request,
waiting for server to shut down....2018-05-11 09:39:21.596 UTC [37] LOG: aborting any active transactions,
2018-05-11 09:39:21.598 UTC [37] LOG: worker process: logical replication launcher (PID 44) exited with exit code 1,
2018-05-11 09:39:21.599 UTC [39] LOG: shutting down,
2018-05-11 09:39:21.613 UTC [37] LOG: database system is shut down,
done,
server stopped,
,
PostgreSQL init process complete; ready for start up.,
,
2018-05-11 09:39:21.706 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432,
2018-05-11 09:39:21.706 UTC [1] LOG: listening on IPv6 address "::", port 5432,
2018-05-11 09:39:21.709 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432",
2018-05-11 09:39:21.724 UTC [64] LOG: database system was shut down at 2018-05-11 09:39:21 UTC,
2018-05-11 09:39:21.729 UTC [1] LOG: database system is ready to accept connections,
2018-05-11 10:26:20.444 UTC [1] LOG: received smart shutdown request,
2018-05-11 10:26:20.449 UTC [1] LOG: worker process: logical replication launcher (PID 70) exited with exit code 1,
2018-05-11 10:26:20.449 UTC [65] LOG: shutting down,
2018-05-11 10:26:20.460 UTC [1] LOG: database system is shut down,
Image :
FROM postgres:10
COPY healthcheck /usr/local/bin/
RUN chmod +x /usr/local/bin/healthcheck
HEALTHCHECK --interval=30s --timeout=30s --retries=3 \
CMD healthcheck
Snippet from docker-compose :
db_jackrabbit:
build: ./images/pgsql_jackrabbit
container_name: db_jackrabbit
environment:
- POSTGRES_DB=${JACK_POSTGRES_DB}
- POSTGRES_USER=${JACK_POSTGRES_USER}
- POSTGRES_PASSWORD=${JACK_POSTGRES_PASSWORD}
volumes:
- pgsql_jackrabbit_local:/var/lib/postgresql/data
ports:
- ${PORT_DB_JACKRABBIT}:5432
healthcheck:
#!/bin/bash
set -eo pipefail
host="$(hostname -i || echo '127.0.0.1')"
user="${POSTGRES_USER:-postgres}"
db="${POSTGRES_DB:-$POSTGRES_USER}"
export PGPASSWORD="${POSTGRES_PASSWORD:-}"
args=(
# force postgres to not use the local unix socket (test "external" connectibility)
--host "$host"
--username "$user"
--dbname "$db"
--quiet --no-align --tuples-only
)
if select="$(echo 'SELECT 1' | psql "${args[#]}")" && [ "$select" = '1' ]; then
exit 0
fi
exit 1
But DB still alive. It is shutdown periodically and accept connections again (
What it the problem?
Thanks in advance!
Ok, so I solved my issue. This issue helped me.
It seems like Postgres initialization process does stop the initialization process once done, and it is another process which follows up and accepts connexion.
Hence I had:
postgres:
deploy:
restart_policy:
condition: on-failure
window: 15m
and apparently docker received a end-of-process status code, so it stopped without going to the next process so never accepting connexions.
My interpretation may be incorrect, but at least if you face the issue, try removing restart_policy key to see if it fixes it.
I haven't tried to restore healthcheck yet, as it may also have undesired side effects.