Docker Postgres PGAdmin4 MacOS - pgadmin4 does not show docker Postgres volume data after restart - postgresql

I recently had to restart my machine, thus shut down my docker container and server connections. After restart PGAdmin4 no longer shows docker volume data, however my application is still able to access the docker volume data. I'm able to log in to my db as user postgres, though no tables show in the db. PGAdmin4 on 5050:5050, Postgres on 5432:5432, app on 5000:5000. I'm able to log into the db via PGAdmin while the container is paused, which wasn't the case before, finding it hard to understand why.
I've got a docker-compose file like so:
version: "3.7"
services:
postgres:
container_name: postgres
restart: always
image: postgres:12.2
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${POSTGRES_DB}
volumes:
- flask-app-db:/var/lib/postgresql/data
ports:
- 5432:5432
networks:
- pgnetwork
pgadmin4:
container_name: pgadmin4
restart: always
image: dpage/pgadmin4
environment:
- PGADMIN_DEFAULT_EMAIL=${PGADMIN_DEFAULT_EMAIL}
- PGADMIN_DEFAULT_PASSWORD=${PGADMIN_DEFAULT_PASSWORD}
- SERVER_PORT=${SERVER_PORT}
volumes:
- pga4:/var/lib/pgadmin
ports:
- 5050:5050
networks:
- pgnetwork
dplio:
container_name: dplio
restart: always
build:
context: .
dockerfile: Dockerfile
environment:
# MY_ENV_VARS
ports:
- 5000:5000
volumes:
- .:/dplio
networks:
- pgnetwork
depends_on:
- postgres
networks:
pgnetwork:
driver: bridge
volumes:
flask-app-db:
name: flask-app-db
pga4:
name: pga4
My terminal output when the containers are run:
Starting postgres ... done
Starting pgadmin4 ... done
Starting dplio ... done
Attaching to pgadmin4, postgres, dplio
postgres |
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres |
postgres | 2020-03-17 11:18:55.599 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres | 2020-03-17 11:18:55.599 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2020-03-17 11:18:55.600 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres | 2020-03-17 11:18:55.604 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2020-03-17 11:18:55.619 UTC [26] LOG: database system was shut down at 2020-03-17 10:41:43 UTC
postgres | 2020-03-17 11:18:55.623 UTC [1] LOG: database system is ready to accept connections
pgadmin4 | [2020-03-17 11:18:56 +0000] [1] [INFO] Starting gunicorn 19.9.0
pgadmin4 | [2020-03-17 11:18:56 +0000] [1] [INFO] Listening at: http://[::]:80 (1)
pgadmin4 | [2020-03-17 11:18:56 +0000] [1] [INFO] Using worker: threads
pgadmin4 | [2020-03-17 11:18:56 +0000] [80] [INFO] Booting worker with pid: 80
dplio | * Serving Flask app "app" (lazy loading)
dplio | * Environment: docker
dplio | * Debug mode: on
dplio | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
dplio | * Restarting with stat
dplio | * Debugger is active!
dplio | * Debugger PIN: 209-285-043
I thought to try a pg_restore from the docker volume to host but that doesn't seem to make sense as a potential solution, I think it's a connection issue between the docker container and my host machine, not a data loss problem.
I set PGAdmin4 port to 5050 in the browser, though it worked fine without that before I restarted.
I tried visiting http://localhost:5050/?key=xxxxxxx http://0.0.0.0:5050/?key=xxxxxxx and http://1270.0.0.1:5050/?key=xxxxxxx, the key taken from ~/.pgAdmin4.XXXXXXXXXXXXXXXXXX.addr same issue.
I've tried removing my docker app image and rebuilding it, too.
Can anyone let me know what I'm missing? A step by step for shutting down server/container/network connections and restarting would be super helpful!
If you need more info let me know.
Thanks!

Related

Error timeout reached before the port went into state "inuse"

Faced a problem with connection of postgresql to keycloak. In keycloak I tried to use 5433 and 5432 port in "KEYCLOAK_DATABASE_PORT", but none of them is not working
docker compose
version: "3.8"
volumes:
postgres_data:
driver: local
services:
postgres:
container_name: postgres.beltweet.com
image: library/postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: beltweet
ports:
- "5433:5432"
restart: unless-stopped
keycloak:
image: docker.io/bitnami/keycloak:latest
container_name: keycloak.beltweet.com
environment:
KEYCLOAK_ADMIN_USER: admin
KEYCLOAK_ADMIN_PASSWORD: 260
KEYCLOAK_MANAGEMENT_PASSWORD: 260
KEYCLOAK_DATABASE_PORT: 5432
KEYCLOAK_DATABASE_HOST: postgres
KEYCLOAK_DATABASE_NAME: beltweet
KEYCLOAK_CREATE_ADMIN_USER: 'true'
KEYCLOAK_DATABASE_USER: postgres
KEYCLOAK_DATABASE_PASSWORD: postgres
KEYCLOAK_HTTP_PORT: 3033
KEYCLOAK_HTTPS_PORT: 3034
KEYCLOAK_JGROUPS_DISCOVERY_PROTOCOL: JDBC_PING
KEYCLOAK_CACHE_OWNERS_COUNT: 3
KEYCLOAK_AUTH_CACHE_OWNERS_COUNT: 3
depends_on:
postgres:
condition: service_started
links:
- "postgres:postgres"
error
Starting postgres.beltweet.com ... done
Recreating keycloak.beltweet.com ... done
Attaching to postgres.beltweet.com, keycloak.beltweet.com
postgres.beltweet.com |
postgres.beltweet.com | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres.beltweet.com |
postgres.beltweet.com | 2022-03-17 14:33:35.516 UTC [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
keycloak.beltweet.com | keycloak 14:33:35.99
keycloak.beltweet.com | keycloak 14:33:35.99 Welcome to the Bitnami keycloak container
postgres.beltweet.com | 2022-03-17 14:33:35.516 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres.beltweet.com | 2022-03-17 14:33:35.516 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres.beltweet.com | 2022-03-17 14:33:35.517 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres.beltweet.com | 2022-03-17 14:33:35.520 UTC [26] LOG: database system was shut down at 2022-03-17 14:33:30 UTC
postgres.beltweet.com | 2022-03-17 14:33:35.523 UTC [1] LOG: database system is ready to accept connections
keycloak.beltweet.com | keycloak 14:33:35.99 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-keycloak
keycloak.beltweet.com | keycloak 14:33:36.00 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-keycloak/issues
keycloak.beltweet.com | keycloak 14:33:36.00
keycloak.beltweet.com | keycloak 14:33:36.00 INFO ==> ** Starting keycloak setup **
keycloak.beltweet.com | keycloak 14:33:36.01 INFO ==> Validating settings in KEYCLOAK_* env vars...
keycloak.beltweet.com | keycloak 14:33:36.02 INFO ==> Trying to connect to PostgreSQL server postgres...
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
keycloak.beltweet.com | timeout reached before the port went into state "inuse"
I do not understand, why you let postgres expose port 5433 and configure keycloak to connect on 5432.
I took your docker-compose example and fixed the port assignment. Additionally I exposed port 8080 for keycloak server:
version: "3.8"
volumes:
postgres_data:
driver: local
services:
postgres:
container_name: postgres.beltweet.com
image: library/postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: beltweet
ports:
- "5432:5432"
restart: unless-stopped
keycloak:
image: docker.io/bitnami/keycloak:latest
container_name: keycloak.beltweet.com
ports:
- "8080:8080"
environment:
KEYCLOAK_ADMIN_USER: admin
KEYCLOAK_ADMIN_PASSWORD: 260
KEYCLOAK_MANAGEMENT_PASSWORD: 260
KEYCLOAK_DATABASE_PORT: 5432
KEYCLOAK_DATABASE_HOST: postgres
KEYCLOAK_DATABASE_NAME: beltweet
KEYCLOAK_CREATE_ADMIN_USER: 'true'
KEYCLOAK_DATABASE_USER: postgres
KEYCLOAK_DATABASE_PASSWORD: postgres
KEYCLOAK_HTTP_PORT: 3033
KEYCLOAK_HTTPS_PORT: 3034
KEYCLOAK_JGROUPS_DISCOVERY_PROTOCOL: JDBC_PING
KEYCLOAK_CACHE_OWNERS_COUNT: 3
KEYCLOAK_AUTH_CACHE_OWNERS_COUNT: 3
depends_on:
postgres:
condition: service_started
links:
- "postgres:postgres"
It works:
keycloak.beltweet.com | keycloak 12:47:54.29 INFO ==> Trying to connect to PostgreSQL server postgres...
keycloak.beltweet.com | keycloak 12:47:54.29 INFO ==> Found PostgreSQL server listening at postgres:5432
keycloak.beltweet.com | keycloak 12:47:54.30 INFO ==> Configuring database settings
keycloak.beltweet.com | keycloak 12:48:05.96 INFO ==> Configuring jgroups settings
keycloak.beltweet.com | keycloak 12:48:10.89 INFO ==> Configuring cache count
keycloak.beltweet.com | keycloak 12:48:15.94 INFO ==> Configuring authentication cache count
keycloak.beltweet.com | keycloak 12:48:22.87 INFO ==> Configuring log level
keycloak.beltweet.com | keycloak 12:48:27.71 INFO ==> Configuring proxy address forwarding
keycloak.beltweet.com | keycloak 12:48:32.68 INFO ==> Configuring node identifier
keycloak.beltweet.com |
keycloak.beltweet.com | keycloak 12:48:37.33 INFO ==> ** keycloak setup finished! **
keycloak.beltweet.com | keycloak 12:48:37.35 INFO ==> ** Starting keycloak **

Docker: Adminer backend for PostgrSQL database times out

I'm trying to set up a simple database system using a PostgreSQL Docker container and an Adminer container as a backend. Note this is all running on a RaspberryPi 4.
docker-compose.yml:
services:
postgres:
image: postgres
restart: on-failure
ports:
- 5432:5432
env_file:
- ./.auth_file
volumes:
- db:/var/lib/postgresql/data
adminer:
image: adminer
restart: on-failure
ports:
- 8070:8080
env_file:
- ./.auth_file
depends_on:
- postgres
volumes:
db:
I then run it using docker-compose up:
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2021-04-01 13:25:25.919 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on arm-unknown-linux-gnueabihf, compiled by gcc (Debian 8.3.0-6) 8.3.0, 32-bit
postgres_1 | 2021-04-01 13:25:25.919 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2021-04-01 13:25:25.919 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2021-04-01 13:25:26.028 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2021-04-01 13:25:26.090 UTC [27] LOG: database system was shut down at 2021-04-01 13:20:41 UTC
postgres_1 | 2021-04-01 13:25:26.116 UTC [1] LOG: database system is ready to accept connections
adminer_1 | [Sun Jun 14 00:31:20 2071] PHP 7.4.16 Development Server (http://[::]:8080) started
adminer_1 | [Sun Jun 14 00:30:16 2071] [::ffff:192.168.16.1]:50886 Accepted
The last line shows that I tried to access localhost:8070 in the Browser right on the Pi. The page just loads forever and Adminer never returns the page.
Could this have to do with some firewall issues?
Thanks for any help!
Found the solution. This has to do with the Docker Container being unable to fetch the current time from the host, due to some version issues surrounding Docker, Alpine (the base image for Adminer) and libseccomp. This is also evident by the fact that Adminer shows entirely different timestamps than the Postgres-Container in my original post.
The full explanation can be found here: https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.13.0#time64_requirements
After following the steps linked above, the docker-compose.yml should look something like this:
adminer:
image: adminer
...
security_opt:
- seccomp=config/compose/default.json # required to avoid clock_gettime() errors
Replace config/compose/default.json with the relative path to the modified default.json.

Pgadmin container kills postgres when using docker-compose

I run postgres using docker-compose -f postgres.yml up. When I try to run pgadmin4 using docker-compose -f pgadmin4.yml up, postgres container is automatically killed.
I am new to docker and doesn't understand why is this issue happening?
postgres.yml
version: "3.7"
services:
postgres:
container_name: "postgres"
image: "postgres"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: ****
volumes:
- maple:/var/lib/postgresql/data
volumes:
maple:
name: maple
external: true
pgadmin.yml
version: "3.7"
services:
postgres:
container_name: "pgadmin4"
image: "dpage/pgadmin4"
ports:
- "5433:80"
environment:
PGADMIN_DEFAULT_EMAIL: ******
PGADMIN_DEFAULT_PASSWORD: *****
volumes:
- pgadmin4-vol:/var/lib/pgadmin
volumes:
pgadmin4-vol:
name: pgadmin4-vol
external: true
$ docker-compose -f psql.yml up
Starting postgres ... done
Attaching to postgres
postgres |
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres |
postgres | 2020-11-27 10:18:05.635 UTC 1 LOG: starting PostgreSQL 13.0 (Debian 13.0-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres | 2020-11-27 10:18:05.636 UTC 1 LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2020-11-27 10:18:05.636 UTC 1 LOG: listening on IPv6 address "::", port 5432
postgres | 2020-11-27 10:18:05.639 UTC 1 LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2020-11-27 10:18:05.643 UTC [24] LOG: database system was shut down at 2020-11-27 10:17:59 UTC
postgres | 2020-11-27 10:18:05.647 UTC 1 LOG: database system is ready to accept connections
postgres | 2020-11-27 10:18:11.089 UTC 1 LOG: received fast shutdown request
postgres | 2020-11-27 10:18:11.092 UTC 1 LOG: aborting any active transactions
postgres | 2020-11-27 10:18:11.099 UTC 1 LOG: background worker "logical replication launcher" (PID 30) exited with exit code 1
postgres | 2020-11-27 10:18:11.108 UTC [25] LOG: shutting down
postgres | 2020-11-27 10:18:11.127 UTC 1 LOG: database system is shut down
postgres exited with code 0
$ docker-compose -f pgadmin.yml up
Recreating postgres ... done
Attaching to pgadmin4
pgadmin4 | sudo: setrlimit(RLIMIT_CORE): Operation not permitted
pgadmin4 | [2020-11-27 10:18:13 +0000] 1 [INFO] Starting gunicorn 19.9.0
pgadmin4 | [2020-11-27 10:18:13 +0000] 1 [INFO] Listening at: http://[::]:80 (1)
pgadmin4 | [2020-11-27 10:18:13 +0000] 1 [INFO] Using worker: threads
pgadmin4 | /usr/local/lib/python3.8/os.py:1023: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
pgadmin4 | return io.open(fd, *args, **kwargs)
pgadmin4 | [2020-11-27 10:18:13 +0000] [87] [INFO] Booting worker with pid: 87
docker inpsect 69438bebc3f9 here
your services are both named postgres
so the pgadmin.yml recreates the service postgres, wich is your first database container.. as the logs are clearly showing
'Recreating postgres ... done'
take another name for the pgadmin service then postgres and it should work.
version: "3.7"
services:
pgadmin:
also docker-compose is for building stacks, if you would have done this in one compose file you would have seen the error from begin:
docker-compose.failing.yml:
version: "3.7"
services:
postgres:
container_name: "postgres"
image: "postgres"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: test
volumes:
- maple:/var/lib/postgresql/data
postgres:
container_name: "pgadmin4"
image: "dpage/pgadmin4"
ports:
- "5433:80"
environment:
PGADMIN_DEFAULT_EMAIL: test
PGADMIN_DEFAULT_PASSWORD: test
volumes:
- pgadmin4-vol:/var/lib/pgadmin
volumes:
maple:
name: maple
external: true
pgadmin4-vol:
name: pgadmin4-vol
external: true
docker-compose.working.yml:
version: "3.7"
services:
postgres:
container_name: "postgres"
image: "postgres"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: test
volumes:
- maple:/var/lib/postgresql/data
pgadmin:
container_name: "pgadmin4"
image: "dpage/pgadmin4"
ports:
- "5433:80"
environment:
PGADMIN_DEFAULT_EMAIL: test
PGADMIN_DEFAULT_PASSWORD: test
volumes:
- pgadmin4-vol:/var/lib/pgadmin
volumes:
maple:
name: maple
external: true
pgadmin4-vol:
name: pgadmin4-vol
external: true

Not able to connect to Postgres database from golang Docker container

I am trying to connect to a Postgres database in a separate container from another separate container that stores a Go server:
Pool, err = pgxpool.Connect(context.Background(),"postgres://postgres:postgres#postgres:5432/postgres")
After doing so, I receive the following error:
2020/09/25 13:40:08 failed to connect to `host=postgres user=postgres database=postgres`: hostname resolving error (lookup postgres on 192.168.65.1:53: no such host)
Here is the docker-compose.yml:
version: "3.8"
services:
postgres:
image: postgres:alpine
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- 5432:5432
restart: always
server:
build: .
depends_on:
- postgres
ports:
- "2302:2302"
- "80:80"
restart: always
I am successfully able to connect to the Postgres database from my OS. Here are the Postgres container initialization logs:
postgres_1 |
postgres_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1 |
postgres_1 | 2020-09-25 15:37:50.529 UTC [1] LOG: starting PostgreSQL 13.0 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-09-25 15:37:50.529 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-09-25 15:37:50.529 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-09-25 15:37:50.532 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-09-25 15:37:50.536 UTC [20] LOG: database system was shut down at 2020-09-25 15:37:49 UTC
postgres_1 | 2020-09-25 15:37:50.540 UTC [1] LOG: database system is ready to accept connections
Any clues would be helpful. Thank you in advance.
Try to share a network
version: "3.8"
services:
postgres:
image: postgres:alpine
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- 5432:5432
restart: always
networks: [ "go_develop" ]
server:
build: .
depends_on:
- postgres
ports:
- "2302:2302"
- "80:80"
restart: always
networks: [ "go_develop" ]
networks:
go_develop:
driver: bridge

How to use Netcat to check whether postgresql docker container is up

I want to see if docker container with PostgreSQL is ready using Netcat utility.
My entrypoint.sh script seems to be unable to spot the DB up and running.
When I login into docker and run NC in verbose mode I get
DNS fwd/rev mismatch: db != telegram_messages_db_1.telegram_messages_default
What am I doing wrong?
My setup
entrypoint.sh
echo "Waiting for postgres..."
while ! nc -z db 5432; do
sleep 0.1
done
echo "PostgreSQL started"
docker-compose.yaml
version: '3.7'
services:
messages:
build:
context: .
dockerfile: Dockerfile
entrypoint: ['/usr/src/app/entrypoint.sh'] # new
volumes:
- .:/usr/src/app
ports:
- 5001:5000
environment:
.....
depends_on:
- db
db:
build:
context: ./backend/db
dockerfile: Dockerfile
expose:
- 5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
docker-compose output
Attaching to telegram_messages_db_1, telegram_messages_messages_1
messages_1 | Waiting for postgres...
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2020-03-29 23:55:22.103 UTC [1] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit
db_1 | 2020-03-29 23:55:22.103 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2020-03-29 23:55:22.104 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2020-03-29 23:55:22.115 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2020-03-29 23:55:22.170 UTC [20] LOG: database system was shut down at 2020-03-29 23:46:56 UTC
db_1 | 2020-03-29 23:55:22.190 UTC [1] LOG: database system is ready to accept connections
Also...
If I run nc -z -v telegram_messages_db_1.telegram_messages_default 5432
I do get a nice response:
telegram_messages_db_1.telegram_messages_default [172.19.0.2] 5432 (postgresql) open
Removing all containers, rebooting and building from scratch solved the issue.
I will leave it here for posterity.