Docker Compose postgres db does not start - postgresql

Hello for some reason my postgres keeps restarting basically this is my docker compose:
version: "3.7"
services:
db:
image: postgres:12
restart: always
container_name: "db"
ports:
- "${DB_PORT}:5432"
volumes:
- ./pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${DB_USER}
POSTGRES_PASSWORD: ${DB_PASS}
POSTGRES_DB: ${DB_NAME}
api:
image: server_emasa
container_name: api
restart: always
depends_on:
- db
ports:
- "${SERVER_PORT}:${SERVER_PORT}"
and for some reason my postgres is restarting I tried using the docker compose logs to check and got this:
db | PostgreSQL Database directory appears to contain a database; Skipping initialization
db |
db | 2020-03-26 05:37:18.475 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db | 2020-03-26 05:37:18.475 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db | 2020-03-26 05:37:18.475 UTC [1] LOG: listening on IPv6 address "::", port 5432
db | 2020-03-26 05:37:18.558 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db | 2020-03-26 05:37:18.625 UTC [1] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.646 UTC [26] LOG: database system was interrupted; last known up at 2020-03-23 23:46:31 UTC
db | 2020-03-26 05:37:18.879 UTC [26] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.879 UTC [26] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.879 UTC [26] FATAL: could not open directory "pg_replslot": No such file or directory
db | 2020-03-26 05:37:18.880 UTC [1] LOG: startup process (PID 26) exited with exit code 1
db | 2020-03-26 05:37:18.880 UTC [1] LOG: aborting startup due to startup process failure
db | 2020-03-26 05:37:18.881 UTC [1] LOG: database system is shut down

I also had a similar problem after restarting my postgresql container:
LOG: could not open directory "pg_tblspc/memory/...": Not a directory
The reason of the error is probably because of data corruption in database.
You can solve the issue by deleting contents of pg_tblspc/ directory (make sure to backup all of them) or you can also try by recovering the latest backup of your database (if there's one).

The error refers to postgresql not being able to find critical files, but is not attempting to create them because your pgdata directory contains some files.
Try not to bind your pgdata directory to the docker container, just remove the volumes section in your docker-compose file

first remove associated docker volume and data folder then:
POSTGRES_DB: ${DB_NAME} will not work, use like this:
environment:
- 'POSTGRES_DB=med_db'
- 'POSTGRES_USER:postgres'
- 'POSTGRES_PASSWORD:pass'

Related

Postgres in docker refuse to accept login from docker host

I have already googled this and searched in stack-overflow.
I have this simple docker-compose setup
version: '3.8'
services:
db:
image: postgres:15-bullseye
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=combiner
ports:
- '5432:5432'
volumes:
- db_data:/var/lib/postgresql/data
volumes:
db_data:
driver: local
It starts up just fine and I can see the log in the container.
PostgreSQL Database directory appears to contain a database; Skipping initialization
2022-11-06T12:49:06.168906100Z
2022-11-06T12:49:06.198469600Z 2022-11-06 12:49:06.198 UTC [1] LOG: starting PostgreSQL 15.0 (Debian 15.0-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-11-06T12:49:06.198585700Z 2022-11-06 12:49:06.198 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2022-11-06T12:49:06.198631000Z 2022-11-06 12:49:06.198 UTC [1] LOG: listening on IPv6 address "::", port 5432
2022-11-06T12:49:06.203216100Z 2022-11-06 12:49:06.203 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-11-06T12:49:06.209822300Z 2022-11-06 12:49:06.209 UTC [27] LOG: database system was shut down at 2022-11-06 12:49:05 UTC
2022-11-06T12:49:06.216866300Z 2022-11-06 12:49:06.216 UTC [1] LOG: database system is ready to accept connections
But I cannot connect to the database from outside the container.
I try to use Intellij's database tool and it get a password authentication error.
I am of course running docker-compose on the same computer that I run Intellij on and of course I am giving the correct pasword and user. I should not have to edit the pg_hba.conf file in the container to allow access from outside the container.
Any idea what is going wrong here?

Unable to connect to postgres using docker compose

I have a very simple docker-compose.yml file which currently is only to set up a posgresql database, but I can't connect to it once it is running. Instead I get this error:
2022-10-10 11:19:41.820 UTC [43] FATAL: lock file "postmaster.pid" already exists
2022-10-10 11:19:41.820 UTC [43] HINT: Is another postmaster (PID 1) running in data directory "/var/lib/postgresql/data"?
The docker-compose.yml is:
version: "3.9"
services:
db:
image: "postgres"
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- '5432:5432'
volumes:
- /pgdata:/var/lib/postgresql/data
When I try to connect to the running container I use:
docker exec -it -u postgres 31f122d6a413 postgres
The logs from the container are:
Recreating test-docker-proj ...
WARNING: Service "db" is using volume "/var/lib/postgresql/data" from the previous container. Host mapping "/pgdata" has no effect.
Recreating test-docker-proj ... done
Attaching to test-docker-proj
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2022-10-10 11:05:02.349 UTC [1] LOG: starting PostgreSQL 14.5 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
db_1 | 2022-10-10 11:05:02.349 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2022-10-10 11:05:02.349 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2022-10-10 11:05:02.352 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2022-10-10 11:05:02.356 UTC [22] LOG: database system was shut down at 2022-10-10 10:47:01 UTC
db_1 | 2022-10-10 11:05:02.361 UTC [1] LOG: database system is ready to accept connections
It seems that postgres/docker is persisting user login details from previous containers or something? I am at a bit of a loss anyway, so any guidance would be appreciated.
I am using:
Mac Monterey M1
docker-compose version 1.29.2
Probably worth adding that I can connect to the container using /bin/sh rather than postgres

postgresql running in docker compose doesn't create role

I am trying to run postgreSQL via docker-compose and I am getting the issue that user/password is not created when I started the service.
version: "3"
services:
db:
image: postgres:latest
container_name: postgres
#volumes:
#- postgres-data:/var/lib/postgresql/data
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=postgrespassword
- POSTGRES_USER=postgres
- POSTGRES_DB=random_db_name
restart: always
I have this block of code in my docker-compose.yml and I run the following command:
docker-compose up -d (this allow me to start the service in background)
and when I check the logs I got:
docker logs -f 0e1731f95396
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2021-04-27 16:20:44.592 UTC [49] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-04-27 16:20:44.594 UTC [49] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-04-27 16:20:44.603 UTC [50] LOG: database system was shut down at 2021-04-27 16:20:44 UTC
2021-04-27 16:20:44.609 UTC [49] LOG: database system is ready to accept connections
done
server started
CREATE DATABASE
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
waiting for server to shut down...2021-04-27 16:20:44.889 UTC [49] LOG: received fast shutdown request
.2021-04-27 16:20:44.891 UTC [49] LOG: aborting any active transactions
2021-04-27 16:20:44.892 UTC [49] LOG: background worker "logical replication launcher" (PID 56) exited with exit code 1
2021-04-27 16:20:44.892 UTC [51] LOG: shutting down
2021-04-27 16:20:44.907 UTC [49] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2021-04-27 16:20:45.018 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-04-27 16:20:45.019 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-04-27 16:20:45.019 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-04-27 16:20:45.023 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-04-27 16:20:45.029 UTC [77] LOG: database system was shut down at 2021-04-27 16:20:44 UTC
2021-04-27 16:20:45.034 UTC [1] LOG: database system is ready to accept connections
But when I try to connect to this database locally I get the message: "FATAL: role "postgres" does not exist"
Do you have any input about how to solve this problem?
I already made a few attempts after reading a few comments from different places but I got always the same problem.
I was expecting to run locally postgreSQL and setup already a user/password and a Database with that name in the docker-compose

Cannot connect to port 8080 adminer (Postgres+Adminer docker stack)

Problem
Following the postgres docker official page:
https://hub.docker.com/_/postgres
I've created "stack.yml"
and it's contain:
# Use postgres/example user/password credentials
version: '3.1'
services:
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: example
adminer:
image: adminer
restart: always
ports:
- 8080:8080
And then running command:
$ docker stack deploy -c stack.yml postgres
But after it is finished, I cannot open http://localhost:8080
it keep "Waiting for ..."
I followed everything from the docs, and keep retrying but keep failing, any help would be very appreciated?
update:
using docker-compose is working, but I'm still curious why running with docker stack ... won't work
Additional details
Software version:
Pop!_OS 20.04 LTS
Docker version 19.03.12, build 48a66213fe
Here is $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f0fa7f4ce6ef postgres:latest "docker-entrypoint.s…" 23 minutes ago Up 23 minutes 5432/tcp postgres_db.1.pq28sm95br3hhr92gvxpsrgwd
4a8b54019f7d adminer:latest "entrypoint.sh docke…" 23 minutes ago Up 23 minutes 8080/tcp postgres_adminer.1.kya7f232pjc4975ubj9ywa13x
Here is $ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
j7mxwf0rpi7g postgres_adminer replicated 1/1 adminer:latest *:8080->8080/tcp
we8izke0tb34 postgres_db replicated 1/1 postgres:latest
Here is docker logs for postgres:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2020-09-08 06:21:18.981 UTC [46] LOG: starting PostgreSQL 12.4 (Debian 12.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-09-08 06:21:18.982 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-09-08 06:21:18.994 UTC [47] LOG: database system was shut down at 2020-09-08 06:21:18 UTC
2020-09-08 06:21:18.997 UTC [46] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
2020-09-08 06:21:19.074 UTC [46] LOG: received fast shutdown request
waiting for server to shut down....2020-09-08 06:21:19.075 UTC [46] LOG: aborting any active transactions
2020-09-08 06:21:19.076 UTC [46] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
2020-09-08 06:21:19.077 UTC [48] LOG: shutting down
2020-09-08 06:21:19.090 UTC [46] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2020-09-08 06:21:19.186 UTC [1] LOG: starting PostgreSQL 12.4 (Debian 12.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-09-08 06:21:19.186 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-09-08 06:21:19.186 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-09-08 06:21:19.188 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-09-08 06:21:19.216 UTC [55] LOG: database system was shut down at 2020-09-08 06:21:19 UTC
2020-09-08 06:21:19.219 UTC [1] LOG: database system is ready to accept connections
Here is docker logs for adminer:
[Tue Sep 8 06:21:01 2020] PHP 7.4.10 Development Server (http://[::]:8080) started
Use an IP address instead of hostname

docker alpine postgres in compose not executing docker-entrypoint-initdb.d scripts

When using the https://github.com/kiasaki/docker-alpine-postgres image in docker compose, the scripts inside /docker-entrypoint.initdb.d are not being executed as I view from the logs:
db_1 | LOG: database system was shut down at 2016-09-05 18:16:02 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
The thing is, if I build the Dockerfile standalone and run the container, it will execute the sql files inside the folder, like you can see from the logs:
waiting for server to start....LOG: database system was shut down at 2016-09-05 18:28:18 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/syna-setup.sql
CREATE TABLE
INSERT 0 1
CREATE TABLE
waiting for server to shut down...LOG: received fast shutdown request
.LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
done
server stopped
LOG: database system was shut down at 2016-09-05 18:28:21 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
The docker-compose.yml:
version: '2'
services:
db:
build: ./db
environment:
- POSTGRES_PASSWORD=testpass
restart: always
app:
build: ./app
ports:
- "5000:5000"
volumes:
- .:/app/src
depends_on:
- db
environment:
- DB_SERVER=postgres://postgres:testpass#db:5432/postgres
What am I doing wrong here?
UPDATED as he use his own Dockerfile, not from the github link.
Could you proide the docker run command so that if I build the Dockerfile standalone and run the container, it will execute the sql files inside the folder. You should mount some directory to the container so it can find the SQL files.
I read the Dockerfile at https://github.com/kiasaki/docker-alpine-postgres/blob/master/Dockerfile and asked myself where are the SQL files? It just make the directory without copy SQL files
mkdir /docker-entrypoint-initdb.d
Then, in the docker-entrypoint.sh, it check if there's any SQL file to execute
for f in /docker-entrypoint-initdb.d/*; do
case "$f" in
*.sh) echo "$0: running $f"; . "$f" ;;
*.sql) echo "$0: running $f"; psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < "$f" && echo ;;
*) echo "$0: ignoring $f" ;;
esac
echo
done
I think we should mount a SQL directory, like
version: '2'
services:
db:
build: ./db
environment:
- POSTGRES_PASSWORD=testpass
restart: always
entrypoint:
- docker-entrypoint.sh
volumes:
- ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
- ./docker-entrypoint.sh:/docker-entrypoint.sh