Error when running psql command in /docker-entrypoint-initdb.d/db_init.sh (psql: could not connect to server: Connection refused) - postgresql

I used to use a script (/docker-entrypoint-initdb.d/db_init.sh) to loop through database dumps in a folder copied into the postgres container and restoring them. It used to work nicely but recently it stopped working.
I get following error:
postgres_server_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/db_init.sh
postgres_server_1 | --> db_init.sh: Found /dumps/dataset_1.dump as dataset_1
postgres_server_1 | psql: could not connect to server: Connection refused
postgres_server_1 | Is the server running on host "localhost" (127.0.0.1) and accepting
postgres_server_1 | TCP/IP connections on port 5432?
The db_init.sh script loops through a folder containing database dumps and checks if the database exists already. If not it restores the dump.
/docker-entrypoint-initdb.d/db_init.sh content:
shopt -s nullglob
for i in /dumps/*.dump;
do
db_name=${i##*/}
db_name=${db_name%.dump}
echo "--> db_init.sh: Found $i as $db_name"
if psql -U postgres -h localhost -lqt | cut -d \| -f 1 | grep -qw ${db_name}; then
echo "--> db_init.sh: Database ${db_name} already exists."
else
echo "--> db_init.sh: Setting up database: ${db_name}"
createdb -U postgres -h localhost-T template0 ${db_name}
psql -U postgres -h localhost ${db_name} < ${i}
fi
done
echo "--> db_init.sh: Setup complete."
I am using docker-compose to start the postgres container (and some others).
The docker-compose.yml content:
version: '3'
services:
postgres_server:
image: postgres
volumes:
- /etc/localtime:/etc/localtime:ro
- ./data/dumps:/dumps:ro
- ./scripts/db_init.sh:/docker-entrypoint-initdb.d/db_init.sh
environment:
- TZ=Europe/Berlin
restart: always
volumes:
postgres_data:
driver: local
Now what I don't understand is why there seems to be a connection error usually associated with trying to connect to a postgres database from a different machine or container. But the script itself is running in the postgres container and a volume connects the directory containing the dumps into the container.
Running the psql command from within the container using docker exec -it container_name bash works fine and the dumps are there. Why do the psql commands work when executing them manually from within the container but not when executed via /docker-entrypoint-initdb.d/db_init.sh?

It looks like this commit has broken your script.
Explanation:
PostgreSQL may accept connections not only via TCP/IP, but also via UNIX socket. The -h localhost argument tells psql to use TCP connections rather than UNIX socket.
If you look into the current docker-entrypoint.sh version, you will see, that during the execution of scripts in /docker-entrypoint-initdb.d PostgreSQL listens only on the UNIX socket, and the startup log says:
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
This means that psql -h localhost will not connect to the database, as PostgreSQL does not listen on IP socket. You must use psql without -h localhost option to make it use UNIX socket instead of TCP connections.
But why running psql -h localhost manually works?
If you look into the docker-entrypoint.sh again, you will see that when all init scripts are executed, PostgreSQL is being stopped and then started again in normal (operational) mode, in which it listens both on UNIX and IP sockets:
LOG: listening on IPv4 address "0.0.0.0", port 5432
LOG: listening on IPv6 address "::", port 5432
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
So, when the startup process is complete, you may connect to PostgreSQL using TCP connections, thus, entering into the container and running psql -h localhost succeeds.

Related

Cannot access Postgres Docker container in AWS CodeBuild

I've got a simple Code Build project for a Django App and I want to start a Postgres database in a docker container for testing.
The commands are:
docker run --name django-test-db -d -p 5432:5432 -e POSTGRES_PASSWORD=djangotest -e POSTGRES_USER=configuration-service-master -e POSTGRES_DB=configuration-service postgres:12-alpine
docker exec django-test-db psql -U configuration-service-master -c \"\\dn\"
On my local machine everything works quite fine, but in AWS CodeBuild I'm receiving the error:
psql: error: could not connect to server: could not connect to server:
No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Try two things:
Connect with '-h' flag to specify host so the connection goes through TCP/IP instead of Unix Socket
psql -h localhost -p 5432
Enable privileged mode on Build project:
https://docs.aws.amazon.com/codebuild/latest/userguide/change-project.html#change-project-console

flyway unable to connect to postgres container within docker-entrypoint-initdb.d script

I'm trying to extend the postgres Docker image to potentially (via an environment variable flag) execute flyway DB migrations on DB init. My Dockerfile is here:
FROM postgres:9.6
# Install curl and java (for Flyway)
RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl openjdk-8-jre
# Install Flyway
ENV FLYWAY_VERSION 4.2.0
ENV FLYWAY_INSTALL_DIR /usr/src/flyway
ENV FLYWAY_CONF ${FLYWAY_INSTALL_DIR}/flyway-${FLYWAY_VERSION}/conf/flyway.conf
ENV FLYWAY_EXE ${FLYWAY_INSTALL_DIR}/flyway-${FLYWAY_VERSION}/flyway
RUN mkdir -p ${FLYWAY_INSTALL_DIR} && \
curl -L https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/${FLYWAY_VERSION}/flyway-commandline-${FLYWAY_VERSION}.tar.gz | \
tar -xzC ${FLYWAY_INSTALL_DIR} && \
chmod +x ${FLYWAY_EXE}
# Copy migration scripts
ENV MIGRATIONS_LOCATION /flyway/migrations
COPY migrations $MIGRATIONS_LOCATION
COPY init_db.sh /docker-entrypoint-initdb.d/init_db.sh
With my init_db.sh startup script:
#!/bin/bash
set -e
RUN_MIGRATIONS="${RUN_MIGRATIONS:-false}"
DB_URL="jdbc:postgresql://localhost:5432/$DB_NAME"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE DATABASE $DB_NAME;
EOSQL
if [ "$RUN_MIGRATIONS" == "true" ]; then
echo "running migrations ..."
${FLYWAY_EXE} -user=$POSTGRES_USER -password=$POSTGRES_PASSWORD -url=$DB_URL -locations="filesystem:$MIGRATIONS_LOCATION" migrate
fi
However, when running the container with RUN_MIGRATIONS=true, flyway fails to connect to postgres:
docker build . -t postgres-flyway && docker run -e DB_NAME=db -e RUN_MIGRATIONS=true -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres postgres-flyway
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....LOG: database system was shut down at 2018-08-06 02:19:32 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
done
server started
ALTER ROLE
/usr/local/bin/docker-entrypoint.sh: sourcing /docker-entrypoint-initdb.d/init_db.sh
CREATE DATABASE
running migrations ...
Flyway 4.2.0 by Boxfuse
ERROR:
Unable to obtain Jdbc connection from DataSource (jdbc:postgresql://localhost:5432/db) for user 'postgres': Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State : 08001
Error Code : 0
Message : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
The postgres image runs postgres on port 5432 (as usual) so I'm at a loss on why flyway is unable to connect to postgres over localhost:5432.
I also noticed that within this context, pg_isready states that postgres is accepting connections but when specifying the hostname as localhost or 127.0.0.1 it is unable to reach postgres either. That is, by inserting a few pg_isready commands in my init_db.sh script:
...
pg_isready
pg_isready -p 5432
pg_isready -h localhost -p 5432
...
I see the following log output on postgres init:
...
/var/run/postgresql:5432 - accepting connections
/var/run/postgresql:5432 - accepting connections
localhost:5432 - no response
...
I'm suspicious that I've reached a limitation of postgres' initialize context, but I would like to understand why postgres is unreachable over localhost/127.0.0.1:5432 at this point of initialization.
I had the same problem running flyway when creating a docker image for my database based on the postgres:10.5 image. I added the following to my entrypoint.sh before running flyway, to confirm that the problem I was seeing was caused by the docker-entrypoint.sh change #Nick Maraston posted in his answer:
echo "$(date) - waiting for database to start"
while ! pg_isready -h localhost -p 5432 -d $POSTGRES_DB
do
echo "$(date) - waiting for database to start"
sleep 10
done
The result was that the above code looped for ever. I then replaced it with the following code to restart the database listening for TCP/IP connections on localhost:
pg_ctl -D "$PGDATA" -m fast -w stop
pg_ctl -D "$PGDATA" \
-o "-c listen_addresses='localhost'" \
-w start
Rather than restarting the database like this, a cleaner solution would be to use the JDBC -socketFactory option explained here.
I discovered the problem while digging through the images entry point script. A recent change to the image restricts postgres to only listen for connections over a unix domain socket during internal initialization: https://github.com/docker-library/postgres/pull/440
It is true that postgres docker is listening to a unix socket, eg. /var/run/postgresql/.s.PGSQL.5432. But it is not necessary to force the server to switch its listening address. Postgres database URI allows a connection string to point to a unix socket.
Reference: Connect to a database over a unix socket using SQLAlchemy
The example provided:
export DATABASE_URL=postgres://user:password#/dbname?host=/path/to/unix/socket
I was able to omit the host, and decided to use this environment variable in my /docker-entrypoint-initdb.d/*.sh script instead. Note that no string follows the # symbol, nor is there a host query string here. You may need to explicitly define the host depending on your application.
Solution:
export DATABASE_URL="postgres://$POSTGRES_USER:$POSTGRES_PASSWORD#/$POSTGRES_DB"

How to connect to postgres by psql (-h localhost -p 5432) when docker container starts

I have some docker-compose.yml file. In this file is defined db service (postgres:9.6):
db:
image: postgres:9.6
volumes:
- ./test/data/postgresql/:/var/lib/postgresql/data:delegated
- ./test/bootstrap/postgres:/docker-entrypoint-initdb.d:delegated
ports:
- 15432:5432
environment:
- POSTGRES_DB=test
- POSTGRES_USER=test
- PGDATA=/var/lib/postgresql/data/testdata
And some sh script located in ./test/bootstrap/postgres. In this script i try to connect to postgres by psql like that: psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -h localhost -p 5432 (I know that -h localhost -p 5432 is not necessary but i need it).
When my container is up error occured:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
But after this i restart (after first start this script not executed) my db container then inside in container manually run sh script and it executed successfully. It can connect to localhost and port 5432 even if it is first start.
If i connect in script like that psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" there is no connection error.
Can someone explain me this strange behavior? Why i can't connect to localhost:5432 when container is starting but if connect manually from container it connect successfully?
Thanks!
My understanding is that during the initialization of the container the server is listening on Unix-domain socket.
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
The scripts in the docker-entrypoint-init.d directory are executed while the server runs in this mode. You're trying to connect via TCP/IP by specifying -h localhost.
Try to remove that argument to connect to the server using Unix sockets.
try to add a sleep command in your script to give enougth time to your database to start correctly.
According to docs at hub.docker.com/_/postgres, to add a custom script to run when your container starts, create a shell script like the one below and create a custom Dockerfile where you'll copy the shell script into /docker-entrypoint-initdb.d/
Assuming you save the following script as initdatabase.sh in the same folder as your custom Dockerfile also defined below in example, when you build the image, you'll have created your custom image which runs the following sql statements defined below.
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER user;
CREATE DATABASE customdatabase;
GRANT ALL PRIVILEGES ON DATABASE bloodapp TO user;
// add other sql statements here
EOSQL
FROM mdillon/postgis:9.6-alpine
COPY initdatabase.sh /docker-entrypoint-initdb.d/

Using the postgres image in a test

Given the following docker-compose.test.yml file:
version: '2'
services:
sut:
build: .
command: nosetests
environment:
- DJANGO_SETTINGS_MODULE=test.settings
links:
- db
db:
image: postgres
expose:
- 5432
Building the sut container
docker-compose -f docker-compose.test.yml build sut
Running the container:
thomas#linuxclientlobnek01:~/github/djlobnek$ docker-compose -f docker-compose.test.yml run sut /bin/bash
root#6694ec7148ac:/djlobnek# psql -h localhost -U postgres
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
**could not connect to server: Connection refused**
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
It is possible that your application is attempting to connect to the database before the database is ready to accept connections. If this is the case, then implementing some sort of wait-for-the-database loop into the sut container can resolve the problem. I have in the past used something like:
while ! psql -h db -U postgres postgres -c 'select 1'; do
echo "waiting for database"
sleep 1
done
This is unnecessary if your application knows how to retry unsuccessful database connections.
Before you try this, it's a good idea to verify that the postgres container is actually functioning (e.g., by entering the sut container with docker exec and attempting to manually connect using psql).
Update
Trying to connect to localhost from inside the sut container won't work (postgres isn't run inside that container). You would need to use the hostname db, or you would need to the ip address of the db container. E.g:
postgres -h db -U postgres postgres
You could use localhost as the hostname if you docker exec into the db container itself.

Dockerizing PostgreSQL - psql Connection refused

I'm playing around with Docker and I would like to Dockerize a Postgres container.
I'm following the official example but I can not connect to the image running using psql.
I created the Dockerfile with the content of the example. I builded an image from the Dockerfile and assigned it a name. Then I run the PostgreSQL server container (in the foreground).
~/test » docker run --rm -P --name pg_test eg_postgresql
2014-10-10 06:12:43 UTC LOG: database system was interrupted; last known up at 2014-10-10 06:12:29 UTC
2014-10-10 06:12:43 UTC LOG: database system was not properly shut down; automatic recovery in progress
2014-10-10 06:12:43 UTC LOG: redo starts at 0/1782F68
2014-10-10 06:12:43 UTC LOG: record with zero length at 0/1782FA8
2014-10-10 06:12:43 UTC LOG: redo done at 0/1782F68
2014-10-10 06:12:43 UTC LOG: last completed transaction was at log time 2014-10-10 06:12:29.2487+00
2014-10-10 06:12:43 UTC LOG: database system is ready to accept connections
2014-10-10 06:12:43 UTC LOG: autovacuum launcher started
Then I open another terminal to find out the port:
~/test » docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
aaedb0479139 eg_postgresql:latest "/usr/lib/postgresql 3 days ago Up 41 seconds 0.0.0.0:49154->5432/tcp pg_test
So I can use psql to connect to the instance. But I can't...
~/test » psql -h localhost -p 49154 -d docker -U docker --password
Password for user docker:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 49154?
could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 49154?
could not connect to server: Connection refused
Is the server running on host "localhost" (fe80::1) and accepting
TCP/IP connections on port 49154?
Any help is appreciated.
Solution for me was simply using host.docker.internal instead of localhost in your connection string.
https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/issues/225
If you add the --publish option to the docker run command
docker run --rm -P --publish 127.0.0.1:5432:5432 --name pg_test eg_postgresql
when you run the docker file, then the following will work (note the port is now 5432)
psql -h localhost -p 5432 -d docker -U docker --password
Running this on my mac worked for me:
$ boot2docker ip
The VM's Host only interface IP address is: 192.168.59.103
And then connect along the lines of:
$ psql -h 192.168.59.103 -p 49159 -d docker -U docker --password
It's less than ideal to have to do this all, but the instructions at https://docs.docker.com/installation/mac/ indicate it is the correct solution if you want to connect directly from you mac.
To get IP address of your running container you can:
docker inspect pg_test | grep IPAddress
and then use it instead of 'localhost'.
Where pg_test is container name received from
docker ps
Someone has answered to this question here
To sum up : It is not enough to publish the container's port because on OS x the docker containers are running on virtual machines.
To retrieve the "real" address of the virtual machine you should use the tool which is managing your docker daemon and containers.
I would recommand to use docker-machine
The logic is very similar to wf answer but instead of using boot2docker it uses docker-machine.
1-/ Retrieve the name of the virtual machine
docker-machine ls
2-/ Retrieve the IP Address of the virtual machine by using its name e.g, default
docker-machine ip default
3-/ Use this value wherever you need to specify an host value.