I am running the docker project on docker toolbox windows 7 sp1 and the project does not give any error but still due to postgres not working the whole project in not functional.
The code of Docker compose file is :
version: '3'
services:
postgres:
image: 'postgres:latest'
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_DB: "db"
POSTGRES_PASSWORD: postgres_password
POSTGRES_HOST_AUTH_METHOD: "trust"
DATABASE_URL: postgresql://postgres:p3wrd#postgres:5432/postgres
deploy:
restart_policy:
condition: on-failure
window: 15m
redis:
image: 'redis:latest'
nginx:
restart: always
build:
dockerfile: Dockerfile.dev
context: ./nginx
ports:
- '3050:80'
api:
depends_on:
- "postgres"
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- ./server/copy:/usr/src/app/data
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
client:
depends_on:
- "postgres"
build:
dockerfile: Dockerfile.dev
context: ./client
volumes:
- ./client/copy:/usr/src/app/data
- /usr/src/app/node_modules
worker:
build:
dockerfile: Dockerfile.dev
context: ./worker
volumes:
- ./worker/copy:/usr/src/app/data
- /usr/src/app/node_modules
depends_on:
- "postgres"
But when i run the project i get this :
redis_1 | 1:C 29 May 2020 05:07:37.909 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 29 May 2020 05:07:37.910 # Redis version=6.0.3, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 29 May 2020 05:07:37.911 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 29 May 2020 05:07:37.922 * Running mode=standalone, port=6379.
redis_1 | 1:M 29 May 2020 05:07:37.928 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 29 May 2020 05:07:37.929 # Server initialized
redis_1 | 1:M 29 May 2020 05:07:37.929 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 29 May 2020 05:07:37.933 * Loading RDB produced by version 6.0.3
redis_1 | 1:M 29 May 2020 05:07:37.934 * RDB age 8 seconds
redis_1 | 1:M 29 May 2020 05:07:37.934 * RDB memory usage when created 0.81 Mb
redis_1 | 1:M 29 May 2020 05:07:37.934 * DB loaded from disk: 0.001 seconds
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2020-05-29 05:07:38.927 UTC [1] LOG: starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2020-05-29 05:07:38.928 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-05-29 05:07:38.929 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-05-29 05:07:38.933 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-05-29 05:07:38.993 UTC [24] LOG: database system was shut down at 2020-05-29 05:07:29 UTC
api_1 |
api_1 | > # dev /usr/src/app
api_1 | > nodemon
api_1 |
api_1 | [nodemon] 1.18.3
api_1 | [nodemon] to restart at any time, enter `rs`
api_1 | [nodemon] watching: *.*
With or without data volumes the same error comes and due to which the project is not running. Please Help
I would assume that once the API starts, it attempts to connect to the Postgres. If yes, this is a typical error that many Docker developers experience where the Postgres DB is not yet ready to accept connections and apps are trying to connect to it.
You can solve the problem by trying either of the following approaches:
Make your API layer wait for a certain amount of time (Enough for the Postgres DB to boot up)
Thread.Sleep(60); # should be enough so that Postgres DB can start
Implement a retry mechanism that will wait for let's say 10 seconds everytime the connection fails to establish.
If this doesn't work, I would recommend for you to check whether there is a Postgres DB installed outside of the container that owns the port you were trying to access.
Along with Allan Chua's answer, Please mention a startup dependency on Postgres service in your docker-compose file.
depends_on:
- postgres
Add this to your api service.
Related
I am an error while creating a Pgadmin docker container.
I am trying to create a default server and add database connection on creation of docker container.
Please check below logs from pgadmin container.
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2023-01-26 16:43:57.062 UTC [36] LOG: starting PostgreSQL 12.13 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
2023-01-26 16:43:57.064 UTC [36] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-01-26 16:43:57.076 UTC [37] LOG: database system was shut down at 2023-01-26 16:43:56 UTC
2023-01-26 16:43:57.083 UTC [36] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
2023-01-26 16:43:57.171 UTC [36] LOG: received fast shutdown request
waiting for server to shut down....2023-01-26 16:43:57.173 UTC [36] LOG: aborting any active transactions
2023-01-26 16:43:57.176 UTC [36] LOG: background worker "logical replication launcher" (PID 43) exited with exit code 1
2023-01-26 16:43:57.179 UTC [38] LOG: shutting down
2023-01-26 16:43:57.198 UTC [36] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
Docker-compose file:
postgres:
# image: postgres:14.1-alpine
build:
context: ./
dockerfile: ./Docker/Postgres/Dockerfile
container_name: postgres
image: postgres
restart: always
ports:
- 5432:5432
networks:
- 'default'
environment:
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres"
expose:
- 5432
pgadmin4:
build:
context: .
dockerfile: ./Docker/PgAdmin/Dockerfile
container_name: pgadmin
image: pgadmin
depends_on:
- postgres
environment:
- PGADMIN_DEFAULT_EMAIL=test#test.com
- PGADMIN_DEFAULT_PASSWORD=test
volumes:
- ./Docker/PgAdmin/servers.json:/pgadmin4/servers.json
networks:
- 'default'
ports:
- 9050:80
restart: unless-stopped
healthcheck:
test: wget localhost/misc/ping -q -O - > /dev/null 2>&1 || exit 1
interval: 5s
timeout: 10s
retries: 5
I am using postgres:12-alpine3.16 and dpage/pgadmin4 images.
Can someone please help me to resolve the issue?
Faced a problem with connection of postgresql to keycloak. In keycloak I tried to use 5433 and 5432 port in "KEYCLOAK_DATABASE_PORT", but none of them is not working
docker compose
version: "3.8"
volumes:
postgres_data:
driver: local
services:
postgres:
container_name: postgres.beltweet.com
image: library/postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: beltweet
ports:
- "5433:5432"
restart: unless-stopped
keycloak:
image: docker.io/bitnami/keycloak:latest
container_name: keycloak.beltweet.com
environment:
KEYCLOAK_ADMIN_USER: admin
KEYCLOAK_ADMIN_PASSWORD: 260
KEYCLOAK_MANAGEMENT_PASSWORD: 260
KEYCLOAK_DATABASE_PORT: 5432
KEYCLOAK_DATABASE_HOST: postgres
KEYCLOAK_DATABASE_NAME: beltweet
KEYCLOAK_CREATE_ADMIN_USER: 'true'
KEYCLOAK_DATABASE_USER: postgres
KEYCLOAK_DATABASE_PASSWORD: postgres
KEYCLOAK_HTTP_PORT: 3033
KEYCLOAK_HTTPS_PORT: 3034
KEYCLOAK_JGROUPS_DISCOVERY_PROTOCOL: JDBC_PING
KEYCLOAK_CACHE_OWNERS_COUNT: 3
KEYCLOAK_AUTH_CACHE_OWNERS_COUNT: 3
depends_on:
postgres:
condition: service_started
links:
- "postgres:postgres"
error
Starting postgres.beltweet.com ... done
Recreating keycloak.beltweet.com ... done
Attaching to postgres.beltweet.com, keycloak.beltweet.com
postgres.beltweet.com |
postgres.beltweet.com | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres.beltweet.com |
postgres.beltweet.com | 2022-03-17 14:33:35.516 UTC [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
keycloak.beltweet.com | keycloak 14:33:35.99
keycloak.beltweet.com | keycloak 14:33:35.99 Welcome to the Bitnami keycloak container
postgres.beltweet.com | 2022-03-17 14:33:35.516 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres.beltweet.com | 2022-03-17 14:33:35.516 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres.beltweet.com | 2022-03-17 14:33:35.517 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres.beltweet.com | 2022-03-17 14:33:35.520 UTC [26] LOG: database system was shut down at 2022-03-17 14:33:30 UTC
postgres.beltweet.com | 2022-03-17 14:33:35.523 UTC [1] LOG: database system is ready to accept connections
keycloak.beltweet.com | keycloak 14:33:35.99 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-keycloak
keycloak.beltweet.com | keycloak 14:33:36.00 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-keycloak/issues
keycloak.beltweet.com | keycloak 14:33:36.00
keycloak.beltweet.com | keycloak 14:33:36.00 INFO ==> ** Starting keycloak setup **
keycloak.beltweet.com | keycloak 14:33:36.01 INFO ==> Validating settings in KEYCLOAK_* env vars...
keycloak.beltweet.com | keycloak 14:33:36.02 INFO ==> Trying to connect to PostgreSQL server postgres...
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
I do not understand, why you let postgres expose port 5433 and configure keycloak to connect on 5432.
I took your docker-compose example and fixed the port assignment. Additionally I exposed port 8080 for keycloak server:
version: "3.8"
volumes:
postgres_data:
driver: local
services:
postgres:
container_name: postgres.beltweet.com
image: library/postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: beltweet
ports:
- "5432:5432"
restart: unless-stopped
keycloak:
image: docker.io/bitnami/keycloak:latest
container_name: keycloak.beltweet.com
ports:
- "8080:8080"
environment:
KEYCLOAK_ADMIN_USER: admin
KEYCLOAK_ADMIN_PASSWORD: 260
KEYCLOAK_MANAGEMENT_PASSWORD: 260
KEYCLOAK_DATABASE_PORT: 5432
KEYCLOAK_DATABASE_HOST: postgres
KEYCLOAK_DATABASE_NAME: beltweet
KEYCLOAK_CREATE_ADMIN_USER: 'true'
KEYCLOAK_DATABASE_USER: postgres
KEYCLOAK_DATABASE_PASSWORD: postgres
KEYCLOAK_HTTP_PORT: 3033
KEYCLOAK_HTTPS_PORT: 3034
KEYCLOAK_JGROUPS_DISCOVERY_PROTOCOL: JDBC_PING
KEYCLOAK_CACHE_OWNERS_COUNT: 3
KEYCLOAK_AUTH_CACHE_OWNERS_COUNT: 3
depends_on:
postgres:
condition: service_started
links:
- "postgres:postgres"
It works:
keycloak.beltweet.com | keycloak 12:47:54.29 INFO ==> Trying to connect to PostgreSQL server postgres...
keycloak.beltweet.com | keycloak 12:47:54.29 INFO ==> Found PostgreSQL server listening at postgres:5432
keycloak.beltweet.com | keycloak 12:47:54.30 INFO ==> Configuring database settings
keycloak.beltweet.com | keycloak 12:48:05.96 INFO ==> Configuring jgroups settings
keycloak.beltweet.com | keycloak 12:48:10.89 INFO ==> Configuring cache count
keycloak.beltweet.com | keycloak 12:48:15.94 INFO ==> Configuring authentication cache count
keycloak.beltweet.com | keycloak 12:48:22.87 INFO ==> Configuring log level
keycloak.beltweet.com | keycloak 12:48:27.71 INFO ==> Configuring proxy address forwarding
keycloak.beltweet.com | keycloak 12:48:32.68 INFO ==> Configuring node identifier
keycloak.beltweet.com |
keycloak.beltweet.com | keycloak 12:48:37.33 INFO ==> ** keycloak setup finished! **
keycloak.beltweet.com | keycloak 12:48:37.35 INFO ==> ** Starting keycloak **
I'm trying to set up a simple database system using a PostgreSQL Docker container and an Adminer container as a backend. Note this is all running on a RaspberryPi 4.
docker-compose.yml:
services:
postgres:
image: postgres
restart: on-failure
ports:
- 5432:5432
env_file:
- ./.auth_file
volumes:
- db:/var/lib/postgresql/data
adminer:
image: adminer
restart: on-failure
ports:
- 8070:8080
env_file:
- ./.auth_file
depends_on:
- postgres
volumes:
db:
I then run it using docker-compose up:
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2021-04-01 13:25:25.919 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on arm-unknown-linux-gnueabihf, compiled by gcc (Debian 8.3.0-6) 8.3.0, 32-bit
postgres_1 | 2021-04-01 13:25:25.919 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2021-04-01 13:25:25.919 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2021-04-01 13:25:26.028 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2021-04-01 13:25:26.090 UTC [27] LOG: database system was shut down at 2021-04-01 13:20:41 UTC
postgres_1 | 2021-04-01 13:25:26.116 UTC [1] LOG: database system is ready to accept connections
adminer_1 | [Sun Jun 14 00:31:20 2071] PHP 7.4.16 Development Server (http://[::]:8080) started
adminer_1 | [Sun Jun 14 00:30:16 2071] [::ffff:192.168.16.1]:50886 Accepted
The last line shows that I tried to access localhost:8070 in the Browser right on the Pi. The page just loads forever and Adminer never returns the page.
Could this have to do with some firewall issues?
Thanks for any help!
Found the solution. This has to do with the Docker Container being unable to fetch the current time from the host, due to some version issues surrounding Docker, Alpine (the base image for Adminer) and libseccomp. This is also evident by the fact that Adminer shows entirely different timestamps than the Postgres-Container in my original post.
The full explanation can be found here: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0#time64_requirements
After following the steps linked above, the docker-compose.yml should look something like this:
adminer:
image: adminer
...
security_opt:
- seccomp=config/compose/default.json # required to avoid clock_gettime() errors
Replace config/compose/default.json with the relative path to the modified default.json.
I want to see if docker container with PostgreSQL is ready using Netcat utility.
My entrypoint.sh script seems to be unable to spot the DB up and running.
When I login into docker and run NC in verbose mode I get
DNS fwd/rev mismatch: db != telegram_messages_db_1.telegram_messages_default
What am I doing wrong?
My setup
entrypoint.sh
echo "Waiting for postgres..."
while ! nc -z db 5432; do
sleep 0.1
done
echo "PostgreSQL started"
docker-compose.yaml
version: '3.7'
services:
messages:
build:
context: .
dockerfile: Dockerfile
entrypoint: ['/usr/src/app/entrypoint.sh'] # new
volumes:
- .:/usr/src/app
ports:
- 5001:5000
environment:
.....
depends_on:
- db
db:
build:
context: ./backend/db
dockerfile: Dockerfile
expose:
- 5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
docker-compose output
Attaching to telegram_messages_db_1, telegram_messages_messages_1
messages_1 | Waiting for postgres...
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2020-03-29 23:55:22.103 UTC [1] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit
db_1 | 2020-03-29 23:55:22.103 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2020-03-29 23:55:22.104 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2020-03-29 23:55:22.115 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2020-03-29 23:55:22.170 UTC [20] LOG: database system was shut down at 2020-03-29 23:46:56 UTC
db_1 | 2020-03-29 23:55:22.190 UTC [1] LOG: database system is ready to accept connections
Also...
If I run nc -z -v telegram_messages_db_1.telegram_messages_default 5432
I do get a nice response:
telegram_messages_db_1.telegram_messages_default [172.19.0.2] 5432 (postgresql) open
Removing all containers, rebooting and building from scratch solved the issue.
I will leave it here for posterity.
I recently had to restart my machine, thus shut down my docker container and server connections. After restart PGAdmin4 no longer shows docker volume data, however my application is still able to access the docker volume data. I'm able to log in to my db as user postgres, though no tables show in the db. PGAdmin4 on 5050:5050, Postgres on 5432:5432, app on 5000:5000. I'm able to log into the db via PGAdmin while the container is paused, which wasn't the case before, finding it hard to understand why.
I've got a docker-compose file like so:
version: "3.7"
services:
postgres:
container_name: postgres
restart: always
image: postgres:12.2
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${POSTGRES_DB}
volumes:
- flask-app-db:/var/lib/postgresql/data
ports:
- 5432:5432
networks:
- pgnetwork
pgadmin4:
container_name: pgadmin4
restart: always
image: dpage/pgadmin4
environment:
- PGADMIN_DEFAULT_EMAIL=${PGADMIN_DEFAULT_EMAIL}
- PGADMIN_DEFAULT_PASSWORD=${PGADMIN_DEFAULT_PASSWORD}
- SERVER_PORT=${SERVER_PORT}
volumes:
- pga4:/var/lib/pgadmin
ports:
- 5050:5050
networks:
- pgnetwork
dplio:
container_name: dplio
restart: always
build:
context: .
dockerfile: Dockerfile
environment:
# MY_ENV_VARS
ports:
- 5000:5000
volumes:
- .:/dplio
networks:
- pgnetwork
depends_on:
- postgres
networks:
pgnetwork:
driver: bridge
volumes:
flask-app-db:
name: flask-app-db
pga4:
name: pga4
My terminal output when the containers are run:
Starting postgres ... done
Starting pgadmin4 ... done
Starting dplio ... done
Attaching to pgadmin4, postgres, dplio
postgres |
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres |
postgres | 2020-03-17 11:18:55.599 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres | 2020-03-17 11:18:55.599 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2020-03-17 11:18:55.600 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres | 2020-03-17 11:18:55.604 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2020-03-17 11:18:55.619 UTC [26] LOG: database system was shut down at 2020-03-17 10:41:43 UTC
postgres | 2020-03-17 11:18:55.623 UTC [1] LOG: database system is ready to accept connections
pgadmin4 | [2020-03-17 11:18:56 +0000] [1] [INFO] Starting gunicorn 19.9.0
pgadmin4 | [2020-03-17 11:18:56 +0000] [1] [INFO] Listening at: http://[::]:80 (1)
pgadmin4 | [2020-03-17 11:18:56 +0000] [1] [INFO] Using worker: threads
pgadmin4 | [2020-03-17 11:18:56 +0000] [80] [INFO] Booting worker with pid: 80
dplio | * Serving Flask app "app" (lazy loading)
dplio | * Environment: docker
dplio | * Debug mode: on
dplio | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
dplio | * Restarting with stat
dplio | * Debugger is active!
dplio | * Debugger PIN: 209-285-043
I thought to try a pg_restore from the docker volume to host but that doesn't seem to make sense as a potential solution, I think it's a connection issue between the docker container and my host machine, not a data loss problem.
I set PGAdmin4 port to 5050 in the browser, though it worked fine without that before I restarted.
I tried visiting http://localhost:5050/?key=xxxxxxx http://0.0.0.0:5050/?key=xxxxxxx and http://1270.0.0.1:5050/?key=xxxxxxx, the key taken from ~/.pgAdmin4.XXXXXXXXXXXXXXXXXX.addr same issue.
I've tried removing my docker app image and rebuilding it, too.
Can anyone let me know what I'm missing? A step by step for shutting down server/container/network connections and restarting would be super helpful!
If you need more info let me know.
Thanks!