docker alpine postgres in compose not executing docker-entrypoint-initdb.d scripts - postgresql

When using the https://github.com/kiasaki/docker-alpine-postgres image in docker compose, the scripts inside /docker-entrypoint.initdb.d are not being executed as I view from the logs:
db_1 | LOG: database system was shut down at 2016-09-05 18:16:02 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
The thing is, if I build the Dockerfile standalone and run the container, it will execute the sql files inside the folder, like you can see from the logs:
waiting for server to start....LOG: database system was shut down at 2016-09-05 18:28:18 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/syna-setup.sql
CREATE TABLE
INSERT 0 1
CREATE TABLE
waiting for server to shut down...LOG: received fast shutdown request
.LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
done
server stopped
LOG: database system was shut down at 2016-09-05 18:28:21 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
LOG: received fast shutdown request
LOG: aborting any active transactions
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
The docker-compose.yml:
version: '2'
services:
db:
build: ./db
environment:
- POSTGRES_PASSWORD=testpass
restart: always
app:
build: ./app
ports:
- "5000:5000"
volumes:
- .:/app/src
depends_on:
- db
environment:
- DB_SERVER=postgres://postgres:testpass#db:5432/postgres
What am I doing wrong here?

UPDATED as he use his own Dockerfile, not from the github link.
Could you proide the docker run command so that if I build the Dockerfile standalone and run the container, it will execute the sql files inside the folder. You should mount some directory to the container so it can find the SQL files.
I read the Dockerfile at https://github.com/kiasaki/docker-alpine-postgres/blob/master/Dockerfile and asked myself where are the SQL files? It just make the directory without copy SQL files
mkdir /docker-entrypoint-initdb.d
Then, in the docker-entrypoint.sh, it check if there's any SQL file to execute
for f in /docker-entrypoint-initdb.d/*; do
case "$f" in
*.sh) echo "$0: running $f"; . "$f" ;;
*.sql) echo "$0: running $f"; psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < "$f" && echo ;;
*) echo "$0: ignoring $f" ;;
esac
echo
done
I think we should mount a SQL directory, like
version: '2'
services:
db:
build: ./db
environment:
- POSTGRES_PASSWORD=testpass
restart: always
entrypoint:
- docker-entrypoint.sh
volumes:
- ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
- ./docker-entrypoint.sh:/docker-entrypoint.sh

Related

Unable to connect to postgres using docker compose

I have a very simple docker-compose.yml file which currently is only to set up a posgresql database, but I can't connect to it once it is running. Instead I get this error:
2022-10-10 11:19:41.820 UTC [43] FATAL: lock file "postmaster.pid" already exists
2022-10-10 11:19:41.820 UTC [43] HINT: Is another postmaster (PID 1) running in data directory "/var/lib/postgresql/data"?
The docker-compose.yml is:
version: "3.9"
services:
db:
image: "postgres"
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- '5432:5432'
volumes:
- /pgdata:/var/lib/postgresql/data
When I try to connect to the running container I use:
docker exec -it -u postgres 31f122d6a413 postgres
The logs from the container are:
Recreating test-docker-proj ...
WARNING: Service "db" is using volume "/var/lib/postgresql/data" from the previous container. Host mapping "/pgdata" has no effect.
Recreating test-docker-proj ... done
Attaching to test-docker-proj
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2022-10-10 11:05:02.349 UTC [1] LOG: starting PostgreSQL 14.5 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
db_1 | 2022-10-10 11:05:02.349 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2022-10-10 11:05:02.349 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2022-10-10 11:05:02.352 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2022-10-10 11:05:02.356 UTC [22] LOG: database system was shut down at 2022-10-10 10:47:01 UTC
db_1 | 2022-10-10 11:05:02.361 UTC [1] LOG: database system is ready to accept connections
It seems that postgres/docker is persisting user login details from previous containers or something? I am at a bit of a loss anyway, so any guidance would be appreciated.
I am using:
Mac Monterey M1
docker-compose version 1.29.2
Probably worth adding that I can connect to the container using /bin/sh rather than postgres

Running postgres container getting superuser password error?

I am trying to set up a postgres container to start and run initializing the creation of a table. I've succeeded with the straight image from docker but now that I am trying to extend the image a little to create tables when it's produced and I can't get it running. Based off what I've read here How to create User/Database in script for Docker Postgres, this is what I have:
Dockerfile:
FROM library/postgres
COPY init.sql /docker-entrypoint-initdb.d/
init.sql:
CREATE TABLE incident_disposition (
incident_disposition_code VARCHAR,
incident_disposition_code_description VARCHAR
);
From what I understand, FROM library . . . pulls the postgres image from docker hub and the COPY pushes my init.sql script into the entry point so there is no need for a big dockerfile correct?
I then build the image no issue:
Build
docker build -t my_postgres_image .
But when I run I get the issues:
Run
docker run --name testing my_postgres_image --publish 8000:8080 --detach -e POSTGRES_PASSWORD=postgres -d postgres
Errors from logs
Error: Database is uninitialized and superuser password is not specified.
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is *not* recommended.
See PostgreSQL documentation about "trust":
https://www.postgresql.org/docs/current/auth-trust.html
Attempt from comment:
docker container logs testing
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2020-03-26 14:06:51.064 UTC [46] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-03-26 14:06:51.072 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-03-26 14:06:51.108 UTC [47] LOG: database system was shut down at 2020-03-26 14:06:50 UTC
2020-03-26 14:06:51.119 UTC [46] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
CREATE TABLE
2020-03-26 14:06:51.231 UTC [46] LOG: received fast shutdown request
waiting for server to shut down....2020-03-26 14:06:51.232 UTC [46] LOG: aborting any active transactions
2020-03-26 14:06:51.233 UTC [46] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
2020-03-26 14:06:51.234 UTC [48] LOG: shutting down
2020-03-26 14:06:51.290 UTC [46] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2020-03-26 14:06:51.345 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-03-26 14:06:51.345 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-03-26 14:06:51.345 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-03-26 14:06:51.361 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-03-26 14:06:51.387 UTC [64] LOG: database system was shut down at 2020-03-26 14:06:51 UTC
2020-03-26 14:06:51.398 UTC [1] LOG: database system is ready to accept connections
2020-03-26 14:07:27.715 UTC [72] ERROR: relation "incident_disposition" does not exist at character 15
2020-03-26 14:07:27.715 UTC [72] STATEMENT: select * from incident_disposition;
In addition to comments
Due to recent docker image's updates postgres images do not allow to connect to DB without a password from anywhere. So you need to specify username/password
docker run -p 8000:8080 -e POSTGRES_PASSWORD=postgres --name testing -d my_postgres_image
Or if you still don't want to use password, you can just set POSTGRES_HOST_AUTH_METHOD=trust environment variable:
docker run -p 8000:8080 -e POSTGRES_HOST_AUTH_METHOD=trust --name testing -d my_postgres_image
It is a typical Initialization scripts issue.
You can file the full explaination in postgresql docker page. https://hub.docker.com/_/postgres
Here is the brief intro:
1. One common problem is that if one of your /docker-entrypoint-initdb.d scripts fails (which will cause the entrypoint script to exit) and your orchestrator restarts the container with the already initialized data directory, it will not continue on with your scripts.
note:
in your case, you may need clean the historical docker containers(stopped) by
step 1: docker ps |grep
step 2: docker rm -f -v
Or if you are using docker-compose, the historical orchestrator could be easily removed by docker-compose down -v.

Docker Compose postgres db does not start

Hello for some reason my postgres keeps restarting basically this is my docker compose:
version: "3.7"
services:
db:
image: postgres:12
restart: always
container_name: "db"
ports:
- "${DB_PORT}:5432"
volumes:
- ./pgdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: ${DB_USER}
POSTGRES_PASSWORD: ${DB_PASS}
POSTGRES_DB: ${DB_NAME}
api:
image: server_emasa
container_name: api
restart: always
depends_on:
- db
ports:
- "${SERVER_PORT}:${SERVER_PORT}"
and for some reason my postgres is restarting I tried using the docker compose logs to check and got this:
db | PostgreSQL Database directory appears to contain a database; Skipping initialization
db |
db | 2020-03-26 05:37:18.475 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db | 2020-03-26 05:37:18.475 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db | 2020-03-26 05:37:18.475 UTC [1] LOG: listening on IPv6 address "::", port 5432
db | 2020-03-26 05:37:18.558 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db | 2020-03-26 05:37:18.625 UTC [1] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.646 UTC [26] LOG: database system was interrupted; last known up at 2020-03-23 23:46:31 UTC
db | 2020-03-26 05:37:18.879 UTC [26] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.879 UTC [26] LOG: could not open directory "pg_tblspc": No such file or directory
db | 2020-03-26 05:37:18.879 UTC [26] FATAL: could not open directory "pg_replslot": No such file or directory
db | 2020-03-26 05:37:18.880 UTC [1] LOG: startup process (PID 26) exited with exit code 1
db | 2020-03-26 05:37:18.880 UTC [1] LOG: aborting startup due to startup process failure
db | 2020-03-26 05:37:18.881 UTC [1] LOG: database system is shut down
I also had a similar problem after restarting my postgresql container:
LOG: could not open directory "pg_tblspc/memory/...": Not a directory
The reason of the error is probably because of data corruption in database.
You can solve the issue by deleting contents of pg_tblspc/ directory (make sure to backup all of them) or you can also try by recovering the latest backup of your database (if there's one).
The error refers to postgresql not being able to find critical files, but is not attempting to create them because your pgdata directory contains some files.
Try not to bind your pgdata directory to the docker container, just remove the volumes section in your docker-compose file
first remove associated docker volume and data folder then:
POSTGRES_DB: ${DB_NAME} will not work, use like this:
environment:
- 'POSTGRES_DB=med_db'
- 'POSTGRES_USER:postgres'
- 'POSTGRES_PASSWORD:pass'

FATAL: password authentication failed for user "postgres" using simple docker

I was having issues connecting to my PG database in my compose file. I finally eliminated all other factors and made this bare-bones compose file:
version: '2.1'
services:
database:
image: postgres:9.5.6
environment:
POSTGRES_PASSWORD: secretpass
ports:
- 5432:5432
I then started it up with this command:
docker-compose --file docker-compose.pg.yml up --build
and it seemed to start ok:
Starting app_database_1 ...
Starting app_database_1 ... done
Attaching to app_database_1
database_1 | LOG: database system was shut down at 2017-09-22 18:48:55 UTC
database_1 | LOG: MultiXact member wraparound protections are now enabled
database_1 | LOG: database system is ready to accept connections
database_1 | LOG: autovacuum launcher started
Then from another shell, tried to connect with this command:
psql postgresql://postgres:secretpass#0.0.0.0
And got this output on my first shell where the compose was run:
database_1 | FATAL: password authentication failed for user "postgres"
database_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
I have also tried to connect from another container running
Rails defined in the compose file and get the same error. I always end up with an authentication error. What am I missing?

How to install and run postgress using docker-compose.yml version "1"

1: docker-compose.yml
postgres96:
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- ./Postgres/data:/var/lib/postgresql/data
env:
POSTGRES_PASSWORD: admin#123
POSTGRES_USER: postgres
2) $ docker-compose up &
postgres96_1 | LOG: database system was not properly shut down; automatic recovery in progress
postgres96_1 | LOG: invalid record length at 0/1570D50: wanted 24, got 0
postgres96_1 | LOG: redo is not required
postgres96_1 | LOG: MultiXact member wraparound protections are now enabled
postgres96_1 | LOG: database system is ready to accept connections
postgres96_1 | LOG: autovacuum launcher started
But while testing from pg-admin-iv on windows it show that 'user postgress has no password',
So is that flow from YML file is right, i want simply up the postgress and put data out side docker-container, so How to achieve this?
I would use named volumes for that instead of hosted mapped volumes:
postgres96:
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- data:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: admin#123
POSTGRES_USER: postgres
Probably you have a permission issue in your folder.
You data is saved in a volume outside the container. Then check with:
docker volume ls
Also use docker-compose as:
docker-compose up -d
Instead of &
And
docker-compose logs -f
For further information this is the last part of my log:
postgres96_1 | PostgreSQL init process complete; ready for start up.
postgres96_1 |
postgres96_1 | LOG: database system was shut down at 2016-11-14 21:05:51 UTC
postgres96_1 | LOG: MultiXact member wraparound protections are now enabled
postgres96_1 | LOG: database system is ready to accept connections
postgres96_1 | LOG: autovacuum launcher started
postgres96_1 | LOG: incomplete startup packet
Regards