Using the postgres image in a test - postgresql

Given the following docker-compose.test.yml file:
version: '2'
services:
sut:
build: .
command: nosetests
environment:
- DJANGO_SETTINGS_MODULE=test.settings
links:
- db
db:
image: postgres
expose:
- 5432
Building the sut container
docker-compose -f docker-compose.test.yml build sut
Running the container:
thomas#linuxclientlobnek01:~/github/djlobnek$ docker-compose -f docker-compose.test.yml run sut /bin/bash
root#6694ec7148ac:/djlobnek# psql -h localhost -U postgres
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
**could not connect to server: Connection refused**
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?

It is possible that your application is attempting to connect to the database before the database is ready to accept connections. If this is the case, then implementing some sort of wait-for-the-database loop into the sut container can resolve the problem. I have in the past used something like:
while ! psql -h db -U postgres postgres -c 'select 1'; do
echo "waiting for database"
sleep 1
done
This is unnecessary if your application knows how to retry unsuccessful database connections.
Before you try this, it's a good idea to verify that the postgres container is actually functioning (e.g., by entering the sut container with docker exec and attempting to manually connect using psql).
Update
Trying to connect to localhost from inside the sut container won't work (postgres isn't run inside that container). You would need to use the hostname db, or you would need to the ip address of the db container. E.g:
postgres -h db -U postgres postgres
You could use localhost as the hostname if you docker exec into the db container itself.

Related

How to connect to a Postgres database running in local Docker container through locally-run psql command?

I'm running a docker container with the vanilla Postgres image on my local machine. I'd like to connect to the database from my local machine (i.e., not from "within the container". However, on trying to connect, I get an error.
Here's my docker-compose.yml file:
version: "3.8"
services:
db:
image: postgres
restart: always
ports:
- 5432:5432
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: mypassword
Here's how I start up:
docker-compose run db
Here's how I connect:
psql -h localhost -p 5432 -U postgres
This produces the error:
could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?
If I spin up the database without Docker Compose, the same connection command works as expected:
docker run --name mypg -p 5432:5432 -e POSTGRES_PASSWORD=password postgres
I could just go with the flow and use the command above. But this seems to be pointing to a flaw in how I think about Docker/Compose. For example, maybe Docker Compose's internal DNS resolver makes this approach fail.
Any ideas?
Version info:
psql --version
psql (PostgreSQL) 13.3
I have read through several SO posts, including these, but they don't address or fix the problem I'm seeing:
docker-compose: accessing postgres' shell (psql)
Can't connect to postgres when using docker-compose
Try docker-compose up db instead of run. Using run will run a one-off command against your container, whereas up will turn on the container and leave it running, so another application should be able to access it.
https://docs.docker.com/compose/faq/#whats-the-difference-between-up-run-and-start

SSH to docker container that refuse to start

My spring boot app is connecting to postgres and I want to set /etc/hosts like in add-postgres-port-hosts but my container refusing to run when I try to run it using docker run -d -p 8080:8080 springio/flyaway-postgres it fails to run in the background because it tries to find localhost inside the container -
Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
So how can I run docker exec [container] nc -v -z localhost 5432 in a container that I cannot run in detach mode?
Thank you.

Docker on MacOS, unable to bind Postgres to port 5432

Unsure if this is an issue with Docker for Mac, my Postgres installation (via Homebrew, or some other strange local issue going on.
Probably worth mentioning that I have never ran into this issue on Linux.
Here is a snippet from my docker-compose.yml file that uses the postgres image:
services:
postgres:
image: postgres
environment:
- POSTGRES_DB=some_db
- POSTGRES_USER=some_user
- POSTGRES_PASSWORD=supersecretpassword
ports:
- 5432:5432
After starting compose services docker-compose up, I can see my postgres container running (docker ps).
I can also connect to it directly via docker exec
docker exec -it <container_id> psql -d some_db -U some_user
Everything working as expected so far...
However, when I try to connect to my Docker postgres instance via my local psql client:
psql -h localhost -p 5432 -U some_user -d some_db
It is not connecting to the Docker instance of postgres and rather is trying to use my local postgres instance.
Now the strange part...
When I change the port binding in docker-compose.yml
from 5432:5432
to 5433:5432 (5433 can be any open port)
I am able to connect to the Docker postgres instance as expected via my local psql client:
psql -h localhost -p 5433 -U some_user -d some_db
# After being prompted for my password I'm in!
I don't mind binding to a different port, but I'm still so curious what's happening here!
Anyone know what is happening?
Am I just not able to bind to port 5432?
Is this a Docker for Mac thing?
Is there an issue in the way I installed Postgres?
Thanks!

Error when running psql command in /docker-entrypoint-initdb.d/db_init.sh (psql: could not connect to server: Connection refused)

I used to use a script (/docker-entrypoint-initdb.d/db_init.sh) to loop through database dumps in a folder copied into the postgres container and restoring them. It used to work nicely but recently it stopped working.
I get following error:
postgres_server_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/db_init.sh
postgres_server_1 | --> db_init.sh: Found /dumps/dataset_1.dump as dataset_1
postgres_server_1 | psql: could not connect to server: Connection refused
postgres_server_1 | Is the server running on host "localhost" (127.0.0.1) and accepting
postgres_server_1 | TCP/IP connections on port 5432?
The db_init.sh script loops through a folder containing database dumps and checks if the database exists already. If not it restores the dump.
/docker-entrypoint-initdb.d/db_init.sh content:
shopt -s nullglob
for i in /dumps/*.dump;
do
db_name=${i##*/}
db_name=${db_name%.dump}
echo "--> db_init.sh: Found $i as $db_name"
if psql -U postgres -h localhost -lqt | cut -d \| -f 1 | grep -qw ${db_name}; then
echo "--> db_init.sh: Database ${db_name} already exists."
else
echo "--> db_init.sh: Setting up database: ${db_name}"
createdb -U postgres -h localhost-T template0 ${db_name}
psql -U postgres -h localhost ${db_name} < ${i}
fi
done
echo "--> db_init.sh: Setup complete."
I am using docker-compose to start the postgres container (and some others).
The docker-compose.yml content:
version: '3'
services:
postgres_server:
image: postgres
volumes:
- /etc/localtime:/etc/localtime:ro
- ./data/dumps:/dumps:ro
- ./scripts/db_init.sh:/docker-entrypoint-initdb.d/db_init.sh
environment:
- TZ=Europe/Berlin
restart: always
volumes:
postgres_data:
driver: local
Now what I don't understand is why there seems to be a connection error usually associated with trying to connect to a postgres database from a different machine or container. But the script itself is running in the postgres container and a volume connects the directory containing the dumps into the container.
Running the psql command from within the container using docker exec -it container_name bash works fine and the dumps are there. Why do the psql commands work when executing them manually from within the container but not when executed via /docker-entrypoint-initdb.d/db_init.sh?
It looks like this commit has broken your script.
Explanation:
PostgreSQL may accept connections not only via TCP/IP, but also via UNIX socket. The -h localhost argument tells psql to use TCP connections rather than UNIX socket.
If you look into the current docker-entrypoint.sh version, you will see, that during the execution of scripts in /docker-entrypoint-initdb.d PostgreSQL listens only on the UNIX socket, and the startup log says:
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
This means that psql -h localhost will not connect to the database, as PostgreSQL does not listen on IP socket. You must use psql without -h localhost option to make it use UNIX socket instead of TCP connections.
But why running psql -h localhost manually works?
If you look into the docker-entrypoint.sh again, you will see that when all init scripts are executed, PostgreSQL is being stopped and then started again in normal (operational) mode, in which it listens both on UNIX and IP sockets:
LOG: listening on IPv4 address "0.0.0.0", port 5432
LOG: listening on IPv6 address "::", port 5432
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
So, when the startup process is complete, you may connect to PostgreSQL using TCP connections, thus, entering into the container and running psql -h localhost succeeds.

How to connect to postgres by psql (-h localhost -p 5432) when docker container starts

I have some docker-compose.yml file. In this file is defined db service (postgres:9.6):
db:
image: postgres:9.6
volumes:
- ./test/data/postgresql/:/var/lib/postgresql/data:delegated
- ./test/bootstrap/postgres:/docker-entrypoint-initdb.d:delegated
ports:
- 15432:5432
environment:
- POSTGRES_DB=test
- POSTGRES_USER=test
- PGDATA=/var/lib/postgresql/data/testdata
And some sh script located in ./test/bootstrap/postgres. In this script i try to connect to postgres by psql like that: psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -h localhost -p 5432 (I know that -h localhost -p 5432 is not necessary but i need it).
When my container is up error occured:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
But after this i restart (after first start this script not executed) my db container then inside in container manually run sh script and it executed successfully. It can connect to localhost and port 5432 even if it is first start.
If i connect in script like that psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" there is no connection error.
Can someone explain me this strange behavior? Why i can't connect to localhost:5432 when container is starting but if connect manually from container it connect successfully?
Thanks!
My understanding is that during the initialization of the container the server is listening on Unix-domain socket.
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
The scripts in the docker-entrypoint-init.d directory are executed while the server runs in this mode. You're trying to connect via TCP/IP by specifying -h localhost.
Try to remove that argument to connect to the server using Unix sockets.
try to add a sleep command in your script to give enougth time to your database to start correctly.
According to docs at hub.docker.com/_/postgres, to add a custom script to run when your container starts, create a shell script like the one below and create a custom Dockerfile where you'll copy the shell script into /docker-entrypoint-initdb.d/
Assuming you save the following script as initdatabase.sh in the same folder as your custom Dockerfile also defined below in example, when you build the image, you'll have created your custom image which runs the following sql statements defined below.
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER user;
CREATE DATABASE customdatabase;
GRANT ALL PRIVILEGES ON DATABASE bloodapp TO user;
// add other sql statements here
EOSQL
FROM mdillon/postgis:9.6-alpine
COPY initdatabase.sh /docker-entrypoint-initdb.d/