I'm having issues with postgres for 2 docker deployments on the same host. I need to deploy the same app with docker compose on the same host. I have one docker-compose file and two .env for each instance. The app is built with Ruby and Hanami web framework.
When deploying the first instance (production) it runs fine. With the second instance (staging) I get this error during docker-compose setup:
PostgreSQL Database directory appears to contain a database; Skipping initialization
.
.
.
2022-07-30 16:13:48.978 UTC [33] FATAL: password authentication failed for user "some_user"
postgres_1 | 2022-07-30 16:13:48.978 UTC [33] DETAIL: Role "some_user" does not exist.
I also see the port being used is 5432, which is the one used for production.
For staging, I've specified the other_user and a different port as you can see in the file below.
Here are the files in question with which I wanted to separate these DB instances:
docker-compose
version: '3'
services:
postgres:
image: postgres
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${POSTGRES_DB}
ports:
- ${POSTGRES_PORT}:${POSTGRES_PORT}
volumes:
- postgres:/var/lib/postgresql/data
web:
build: .
command: >
bash -c "bundle exec hanami db migrate
&& bundle exec rake initial_settings:add_default_language
&& bundle exec rake initial_settings:add_session_validity
&& bundle exec rake import_user:create
&& bundle exec rake super_admin:create
&& bundle exec hanami assets precompile
&& bundle exec hanami server
&& cp -r apps/wordrocket/assets/webfonts public/webfonts
&& cp -r apps/wordrocket/assets/webfonts public/assets/webfonts
&& cp -r apps/wordrocket/assets/images/sort*.png public/assets
&& cp -r apps/wordrocket/assets/images/sort*.png public
&& cp -r apps/wordrocket/assets/images/ui-icons*.png public/assets/wordrocket
&& mkdir public/assets/images
&& cp -r apps/wordrocket/assets/images/sort*.png public/assets/images"
volumes:
- ./hanami_log/hanami_app.log:/usr/src/app/hanami_log/hanami_app.log
links:
- postgres
depends_on:
- postgres
nginx:
image: nginx:alpine
restart: unless-stopped
tty: true
ports:
- "${NGINX_PORT}:80"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./nginx_log:/var/log/nginx
depends_on:
- web
volumes:
postgres:
web:
nginx:
Production .env
POSTGRES_USER=some_user
POSTGRES_PASSWORD=some_password
POSTGRES_DB=app_production
NGINX_PORT=7080
COMPOSE_PROJECT_NAME=myAPP
POSTGRES_PORT=5432
Staging .env.staging
POSTGRES_USER=other_user
POSTGRES_PASSWORD=other_password
POSTGRES_DB=app_staging
NGINX_PORT=7081
COMPOSE_PROJECT_NAME=myAPP_staging
POSTGRES_PORT=5433
Makefile
production:
docker-compose -f docker-compose.yml --env-file .env up
staging:
docker-compose -f docker-compose.yml --env-file .env.staging up
I'm guessing I don't separate the postgres instances/dbs correctly, but I'm not sure. Also, is this a Docker or host problem?
+++ EDIT +++
I've made a separate docker-compose file for staging in order to differentiate between volumes.
production is the same:
volumes:
- postgres:/var/lib/postgresql/data
.
.
.
volumes:
postgres:
web:
nginx:
staging:
volumes:
- postgres_staging:/var/lib/postgresql/data
.
.
.
volumes:
postgres_staging:
web:
nginx:
This is the startup log up to the first error:
nginx_1 | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx_1 | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx_1 | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default time zone ... Etc/UTC
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
postgres_1 | performing post-bootstrap initialization ... ok
nginx_1 | 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx_1 | /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
nginx_1 | /docker-entrypoint.sh: Configuration complete; ready for start up
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | initdb: warning: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2022-07-31 10:03:12.510 UTC [47] LOG: starting PostgreSQL 14.4 (Debian 14.4-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
postgres_1 | 2022-07-31 10:03:12.513 UTC [47] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2022-07-31 10:03:12.524 UTC [48] LOG: database system was shut down at 2022-07-31 10:03:12 UTC
postgres_1 | 2022-07-31 10:03:12.529 UTC [47] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | waiting for server to shut down...2022-07-31 10:03:12.745 UTC [47] LOG: received fast shutdown request
postgres_1 | .2022-07-31 10:03:12.748 UTC [47] LOG: aborting any active transactions
postgres_1 | 2022-07-31 10:03:12.749 UTC [47] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
postgres_1 | 2022-07-31 10:03:12.749 UTC [49] LOG: shutting down
postgres_1 | 2022-07-31 10:03:12.771 UTC [47] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2022-07-31 10:03:12.865 UTC [1] LOG: starting PostgreSQL 14.4 (Debian 14.4-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
postgres_1 | 2022-07-31 10:03:12.865 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2022-07-31 10:03:12.865 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2022-07-31 10:03:12.871 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2022-07-31 10:03:12.879 UTC [61] LOG: database system was shut down at 2022-07-31 10:03:12 UTC
postgres_1 | 2022-07-31 10:03:12.885 UTC [1] LOG: database system is ready to accept connections
postgres_1 | 2022-07-31 10:03:13.014 UTC [68] FATAL: password authentication failed for user "some_user"
Now this is the only instance being started. The production is not running even and it still tries to authenticate the production user.
Is it ignoring the .env.staging file?
+++ END EDIT +++
Best, Seba
I am running very basic docker-compose with following docker-compose.yml and getting following error.
docker-compose.yml
services:
redis:
image: redis
db:
image: postgres:9.4
environment:
- POSTGRES_USER=udcoker
- POSTGRES_PASSWORD=udcoker
- POSTGRES_DB=docker
vote:
image: voting-app
ports:
- 5000:80
depends_on:
- redis
worker:
image: worker-app
depends_on:
- redis
- db
environment:
- POSTGRES_USER=udcoker
- POSTGRES_PASSWORD=udcoker
- POSTGRES_DB=docker
result:
image: result-app
ports:
- 5001:80
depends_on:
- db
environment:
- POSTGRES_USER=udcoker
- POSTGRES_PASSWORD=udcoker
- POSTGRES_DB=docker
Error generated,
db_1 | vacuuming database template1 ... ok
db_1 | copying template1 to template0 ... ok
db_1 | copying template1 to postgres ... ok
db_1 | syncing data to disk ...
db_1 | WARNING: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
db_1 | ok
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | postgres -D /var/lib/postgresql/data
db_1 | or
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
db_1 | waiting for server to start....LOG: database system was shut down at 2021-03-03 12:45:13 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
worker_1 | Waiting for db
result_1 | Waiting for db
db_1 | done
db_1 | server started
db_1 | CREATE DATABASE
db_1 |
db_1 |
db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1 |
db_1 | LOG: received fast shutdown request
db_1 | LOG: aborting any active transactions
db_1 | LOG: autovacuum launcher shutting down
db_1 | waiting for server to shut down....LOG: shutting down
db_1 | LOG: database system is shut down
worker_1 | Waiting for db
result_1 | Waiting for db
db_1 | done
db_1 | server stopped
db_1 |
db_1 | PostgreSQL init process complete; ready for start up.
db_1 |
db_1 | LOG: database system was shut down at 2021-03-03 12:45:15 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
db_1 | FATAL: password authentication failed for user "postgres"
db_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all all md5"
db_1 | FATAL: password authentication failed for user "postgres"
db_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all all md5"
I checked on various stackoverflow existing questions but none of them are rectifying actual issue. Some suggesting to run postgres docker and bash it to update the password but its a workaround. Anyone has any concrete solution to this?
I figured it out finally and would be good to share what mistake I had done.
I changed the database configuration by providing environment variable in the docker-compose.yml file in db service as mentioned in the question already,
environment:
- POSTGRES_USER=udcoker
- POSTGRES_PASSWORD=udcoker
- POSTGRES_DB=docker
But the same needs to be updated in all the other custom services (open source github) which are trying to connect to this depende-on postgres database with same user/password/database which was not done.
However the error given by docker-compose is not explaining the situation clearly.
In All, I updated the applications with above username/password/database in connection property and the containers are up.
I am using docker-compose.yml to create multiple running containers but failing to start Postgres docker server, with following logs and yes I have searched many related SO posts, but they didn't helped me out.
Creating network "complex_default" with the default driver
Creating complex_server_1 ... done
Creating complex_redis_1 ... done
Creating complex_postgres_1 ... done
Attaching to complex_postgres_1, complex_redis_1, complex_server_1
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
complex_postgres_1 exited with code 1
below is my docker-compose configuration:
version: '3'
services:
postgres:
image: 'postgres:11-alpine'
redis:
image: 'redis:latest'
server:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
as well as package.json inside server directory is following:
{
"dependencies": {
"body-parser": "^1.19.0",
"cors": "^2.8.4",
"express": "^4.16.3",
"nodemon": "^2.0.4",
"pg": "7.4.3",
"redis": "^2.8.0"
},
"scripts": {
"dev": "nodemon",
"start": "node index.js"
}
}
also for better consideration, I have attached my hands-on project structure:
A year ago it were actually working fine, Does anyone have any idea, what's going wrong here inside my docker-compose file now.
A year ago it were actually working fine, Does anyone have any idea, what's going wrong here inside my docker-compose file now.
Seems like you pulled the fresh image, where in the new image you should specify Postgres user password. You can look into Dockerhub, the image is update one month ago
postgress-11-alpine
db:
image: postgres:11-alpine
restart: always
environment:
POSTGRES_PASSWORD: example
As the error message is self expalinatory
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
or POSTGRES_HOST_AUTH_METHOD=trust use this which is not recommended.
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is *not* recommended.
POSTGRES_PASSWORD
This environment variable is required for you to use the PostgreSQL image. It must not be empty or undefined. This environment variable sets the superuser password for PostgreSQL. The default superuser is defined by the POSTGRES_USER environment variable.
Environment Variables
#Adiii yes, you are nearly right, so I have to explicitly mentioned the environment also for the postgres image but with no db parent tag.
So here, I am explicitly mentioning the docker-compose.yaml config to help others for better understanding, also now I am using recent stable postgres image version 12-alpine, currently latest is postgres:12.3
version: '3'
services:
postgres:
image: 'postgres:12-alpine'
environment:
POSTGRES_PASSWORD: postgres_password
redis:
image: 'redis:latest'
server:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
and so after docker-compose up the creating and running logs were as following:
PS E:\docker\complex> docker-compose up
Creating network "complex_default" with the default driver
Creating complex_postgres_1 ... done
Creating complex_redis_1 ... done
Creating complex_server_1 ... done
Attaching to complex_redis_1, complex_postgres_1, complex_server_1
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default time zone ... UTC
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # Redis version=6.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
postgres_1 | creating configuration files ... ok
redis_1 | 1:M 05 Aug 2020 14:24:48.693 * Running mode=standalone, port=6379.
redis_1 | 1:M 05 Aug 2020 14:24:48.693 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 05 Aug 2020 14:24:48.694 # Server initialized
redis_1 | 1:M 05 Aug 2020 14:24:48.694 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 05 Aug 2020 14:24:48.694 * Ready to accept connections
postgres_1 | running bootstrap script ... ok
server_1 |
server_1 | > # dev /app
server_1 | > nodemon
server_1 |
postgres_1 | performing post-bootstrap initialization ... sh: locale: not found
postgres_1 | 2020-08-05 14:24:50.153 UTC [29] WARNING: no usable system locales were found
server_1 | [nodemon] 2.0.4
server_1 | [nodemon] to restart at any time, enter `rs`
server_1 | [nodemon] watching path(s): *.*
server_1 | [nodemon] watching extensions: js,mjs,json
server_1 | [nodemon] starting `node index.js`
postgres_1 | ok
server_1 | Listening
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | initdb: warning: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2020-08-05 14:24:51.634 UTC [34] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-08-05 14:24:51.700 UTC [34] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-08-05 14:24:51.981 UTC [35] LOG: database system was shut down at 2020-08-05 14:24:50 UTC
postgres_1 | 2020-08-05 14:24:52.040 UTC [34] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | waiting for server to shut down....2020-08-05 14:24:52.121 UTC [34] LOG: received fast shutdown request
postgres_1 | 2020-08-05 14:24:52.186 UTC [34] LOG: aborting any active transactions
postgres_1 | 2020-08-05 14:24:52.188 UTC [34] LOG: background worker "logical replication launcher" (PID 41) exited with exit code 1
postgres_1 | 2020-08-05 14:24:52.188 UTC [36] LOG: shutting down
postgres_1 | 2020-08-05 14:24:52.669 UTC [34] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-08-05 14:24:52.954 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-08-05 14:24:53.136 UTC [43] LOG: database system was shut down at 2020-08-05 14:24:52 UTC
postgres_1 | 2020-08-05 14:24:53.194 UTC [1] LOG: database system is ready to accept connections
Hope this would help manyone.
Adding on ArifMustafa's answer, This worked for me.
postgres:
image: 'postgres:12-alpine'
environment:
POSTGRES_PASSWORD: mypassword
expose:
- 5432
volumes:
- postgres_data:/var/lib/postgres/data/
volumes:
postgres_data:
In my case there was an error about the POSTGRES_PASSWORD (in docker-compose.yml) and the DATABASE settings for PASSWORD in the project_level settings.py file.
After providing a password in docker-compose.yml
db: # For the PostgreSQL database
image: postgres:11
environment:
- POSTGRES_PASSWORD=example
It is necessary to provide the same password to enable access to the POSTGRES database between the two containers (in my case the web application and the database it depended on). In the project_level settings.py:
# Database
# https://docs.djangoproject.com/en/2.2/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'example',
'HOST': 'db',
'PORT': 5432
}
}
To resolve the error when using the command,
docker pull postgres
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
or POSTGRES_HOST_AUTH_METHOD=trust use this which is not recommended.
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all connections without a password. This is not recommended.
Here comes the solution for this:
$ docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
The default postgres user and database are created in the entrypoint with initdb.
The postgres database is a default database meant for use by users, utilities and third party applications.
postgresql.org/docs
I'm starting postgresql 12.3 image via dokcer-compose
I extended the image to execute simple SQL (create schema, table) on start.
From the logs I can see that the SQL commands are executed but when I log to the DB the changes are missing.
My Dockerfile:
FROM postgres
ENV INITDB_DIR /docker-entrypoint-initdb.d
ENV POSTGRES_PASSWORD=...
ENV POSTGRES_DB mydb
COPY initdb.d ${INITDB_DIR}
My docker-compose.yml:
version: "3.7"
services:
postgres:
image: postgres
build: ./postgres
restart: on-failure
ports:
- "5432:5432"
networks:
- postgres
environment:
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:...}
- POSTGRES_DB=mydb
networks:
postgres:
driver: bridge
create_schema.sql (located in ./postgres/initdb.d)
create schema "foo";
create table "foo"."message"(
"id" BIGINT NOT NULL,
"value" TEXT
);
alter table "foo"."message" add constraint "foo_message_pk" primary key("id");
And finally the logs produced when executing: docker-compose up --build postgres
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default time zone ... Etc/UTC
postgres_1 | creating configuration files ... ok
postgres_1 | running bootstrap script ... ok
postgres_1 | performing post-bootstrap initialization ... ok
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | initdb: warning: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2020-05-27 07:34:00.098 UTC [47] LOG: starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2020-05-27 07:34:00.112 UTC [47] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-05-27 07:34:00.143 UTC [48] LOG: database system was shut down at 2020-05-27 07:33:59 UTC
postgres_1 | 2020-05-27 07:34:00.151 UTC [47] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 | CREATE DATABASE
postgres_1 |
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/create_schema.sql
postgres_1 | CREATE SCHEMA
postgres_1 | CREATE TABLE
postgres_1 | ALTER TABLE
postgres_1 |
postgres_1 |
postgres_1 | waiting for server to shut down....2020-05-27 07:34:00.588 UTC [47] LOG: received fast shutdown request
postgres_1 | 2020-05-27 07:34:00.593 UTC [47] LOG: aborting any active transactions
postgres_1 | 2020-05-27 07:34:00.595 UTC [47] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
postgres_1 | 2020-05-27 07:34:00.595 UTC [49] LOG: shutting down
postgres_1 | 2020-05-27 07:34:00.643 UTC [47] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-05-27 07:34:00.708 UTC [1] LOG: starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2020-05-27 07:34:00.709 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-05-27 07:34:00.709 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-05-27 07:34:00.719 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-05-27 07:34:00.744 UTC [74] LOG: database system was shut down at 2020-05-27 07:34:00 UTC
postgres_1 | 2020-05-27 07:34:00.753 UTC [1] LOG: database system is ready to accept connections
When I connect to the database only the default database (postgres) is present. i also tried the above without creating mydb database but still the schema or table were not present.
In one folder I have 4 files: base.py, Dockerfile, docker-compose.yml and wait-for-it.sh.
base.py:
import psycopg2
print("JJJJJJJJ")
conn = psycopg2.connect("dbname='base123' user='postgres' host='db' password='pw1234'")
cur = conn.cursor()
cur.execute("CREATE TABLE test(id serial PRIMARY KEY, num int);")
Dockerfile:
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get -y install python-pip
RUN apt-get update
RUN pip install --upgrade pip
RUN pip install psycopg2-binary
COPY base.py base.py
COPY wait-for-it.sh wait-for-it.sh
RUN chmod +x /wait-for-it.sh
CMD ["python","base.py"]
docker-compose.yml
version: '3'
services:
db:
image: 'postgres:latest'
expose:
- "5432"
ports:
- "5559:5432"
environment:
POSTGRES_PASSWORD: pw1234
POSTGRES_DB: base123
aprrka:
build: .
depends_on:
- db
command: ["./wait-for-it.sh", "db:5432", "--", "python", "base.py"]
wait-fot-it.sh:
link to code in github
After docker-compose up I have this OUTPUT:
...
Creating postgres_db_1 ... done
Creating postgres_aprrka_1 ... done
Attaching to postgres_db_1, postgres_aprrka_1
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1 | creating subdirectories ... ok
db_1 | selecting default max_connections ... 100
db_1 | selecting default shared_buffers ... 128MB
db_1 | selecting dynamic shared memory implementation ... posix
aprrka_1 | wait-for-it.sh: waiting 15 seconds for db:5432
db_1 | creating configuration files ... ok
db_1 | running bootstrap script ... ok
db_1 | performing post-bootstrap initialization ... ok
db_1 | syncing data to disk ... ok
db_1 |
db_1 | WARNING: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
db_1 | waiting for server to start....2018-08-10 09:59:13.529 UTC [38] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2018-08-10 09:59:13.582 UTC [39] LOG: database system was shut down at 2018-08-10 09:59:12 UTC
db_1 | 2018-08-10 09:59:13.599 UTC [38] LOG: database system is ready to accept connections
db_1 | done
db_1 | server started
db_1 | CREATE DATABASE
db_1 |
db_1 | ALTER ROLE
db_1 |
db_1 |
db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1 |
db_1 | 2018-08-10 09:59:14.585 UTC [38] LOG: received fast shutdown request
db_1 | waiting for server to shut down....2018-08-10 09:59:14.593 UTC [38] LOG: aborting any active transactions
db_1 | 2018-08-10 09:59:14.613 UTC [38] LOG: worker process: logical replication launcher (PID 45) exited with exit code 1
db_1 | 2018-08-10 09:59:14.613 UTC [40] LOG: shutting down
db_1 | 2018-08-10 09:59:14.660 UTC [38] LOG: database system is shut down
db_1 | done
db_1 | server stopped
db_1 |
db_1 | PostgreSQL init process complete; ready for start up.
db_1 |
db_1 | 2018-08-10 09:59:14.726 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2018-08-10 09:59:14.726 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2018-08-10 09:59:14.729 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2018-08-10 09:59:14.806 UTC [65] LOG: database system was shut down at 2018-08-10 09:59:14 UTC
db_1 | 2018-08-10 09:59:14.829 UTC [1] LOG: database system is ready to accept connections
db_1 | 2018-08-10 09:59:15.217 UTC [72] LOG: incomplete startup packet
aprrka_1 | wait-for-it.sh: db:5432 is available after 5 seconds
aprrka_1 | JJJJJJJJ
postgres_aprrka_1 exited with code 0
When I type command: psql -U postgres -h localhost -p 5559 in another terminal I can connect to base1234, but when type `\dt' i have no any relations, so my py file don't create table in postgres database. I think that no connection exist with postgres db. Please, help me
You need to close the cursor and commit all pending transactions to the database for the changes to be effective.
Add the following 2 lines at the end of your base.py:
cur.close()
conn.commit()