Docker PostgreSQL service: could not bind IPv6 socket: Cannot assign requested address - postgresql

Question

I am running a PostgreSQL service with Docker. For some reason, PostgreSQL wants to bind to IPV6 - although I haven't specified that anywhere (at least as far as I'm aware).
As a result of this, I am unable to connect to PG. Relevant details follow below:
Dockerfile
FROM postgres:9.6
RUN apt-get update \
&& apt-get -y install apt-utils \
&& apt-get -y install python3 \
&& apt-get -y install postgresql-plpython3-9.6
COPY sql /docker-entrypoint-initdb.d/
EXPOSE 5432
# Add VOLUMEs to allow backup of config, logs and databases
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
PostgreSQL log file contents
LOG: received fast shutdown request
LOG: aborting any active transactions
waiting for server to shut down....LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
LOG: could not bind IPv6 socket: Cannot assign requested address
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
LOG: database system was shut down at 2017-10-09 21:22:22 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
I run the container using the following command: docker run --name my_db_service_cntnr image_tag
When I run the following command: docker container port my_db_service_cntnr, I get nothing returned:
me#yourbox:~/path/to/pgdb$ docker container port my_db_service_cntnr
me#yourbox:~/path/to/pgdb$
I know PostgreSQL is running in the container:
me#yourbox:~/path/to/pgdb$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
824ffe17c5b9 df:pg "docker-entrypoint..." 16 hours ago Up 5 minutes 5432/tcp my_db_service_cntnr
me#yourbox:/path/to/pgdb$ docker container inspect my_db_service_cntnr | grep Address
"LinkLocalIPv6Address": "",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"GlobalIPv6Address": "",
"IPAddress": "172.17.0.2",
"MacAddress": "02:42:ac:11:00:02",
"IPAddress": "172.17.0.2",
"GlobalIPv6Address": "",
"MacAddress": "02:42:ac:11:00:02"
Yet when I attempt to connect to PostgreSQL (using default port of 5432), it fails to connect to the database:
Attempting to connect via psql
me#yourbox:~/path/to/pgdb$ psql -h 172.17.0.2 -U postgres -p 5432
psql: could not connect to server: Connection refused
Is the server running on host "172.17.0.2" and accepting
TCP/IP connections on port 5432?
me#yourbox:~/path/to/pgdb$
It seems port 5432 is not being listened to on my machine, despite me specifying that the PG image EXPOSE port 5432:
me#yourbox:~/path/to/pgdb$ sudo lsof -i -P | grep -i "listen"
lighttpd 1477 www-data 4u IPv4 22342 0t0 TCP *:80 (LISTEN)
dnsmasq 1645 nobody 5u IPv4 26954 0t0 TCP CEBERUS:53 (LISTEN)
master 2182 root 12u IPv4 28720 0t0 TCP localhost:25 (LISTEN)
master 2182 root 13u IPv6 28721 0t0 TCP ip6-localhost:25 (LISTEN)
rhythmbox 3149 me 17u IPv4 33925 0t0 TCP *:3689 (LISTEN)
rhythmbox 3149 me 18u IPv6 33926 0t0 TCP *:3689 (LISTEN)
cupsd 8432 root 10u IPv6 87004 0t0 TCP ip6-localhost:631 (LISTEN)
cupsd 8432 root 11u IPv4 87005 0t0 TCP localhost:631 (LISTEN)
What is causing this error, and how do I resolve it?

Answer 1

This isn't a complete answer, but should get you closer to an answer. It covers the dockerism's that are required for your debug steps in the question.
Run a postgres container
$ CID=$(docker run -d postgres)
$ echo $CID
48024dc71aa446...
Get the PID of the container
$ PID=$(docker inspect -f {{.State.Pid}} $CID)
$ echo $PID
7994
Get the logs from the container, check for errors.
$ docker logs $CID
Process list from the container
$ docker exec -ti $CID ps -ef
UID PID PPID C STIME TTY TIME CMD
postgres 1 0 0 23:19 ? 00:00:00 postgres
postgres 49 1 0 23:19 ? 00:00:00 postgres: checkpointer process
postgres 50 1 0 23:19 ? 00:00:00 postgres: writer process
postgres 51 1 0 23:19 ? 00:00:00 postgres: wal writer process
postgres 52 1 0 23:19 ? 00:00:00 postgres: autovacuum launcher pr
postgres 53 1 0 23:19 ? 00:00:00 postgres: stats collector proces
postgres 54 1 0 23:19 ? 00:00:00 postgres: bgworker: logical repl
root 66 0 0 23:26 ? 00:00:00 ps -ef
Run ss in the container looking for listening tcp processes (like lsof)
$ docker exec -ti $CID ss -lntp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:5432 *:*
LISTEN 0 128 :::5432 :::*
Outside the container won't report on ports in container namespaces
$ ss -lntp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:* users:(("sshd",pid=592,fd=3))
LISTEN 0 128 :::22 :::* users:(("sshd",pid=592,fd=4))
From the host, you can use nsenter to enter the containers namespace and run commands.
$ nsenter -t $PID -n ss -lntp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:5432 *:* users:(("postgres",pid=7994,fd=3))
LISTEN 0 128 :::5432 :::* users:(("postgres",pid=7994,fd=4))
$ nsenter -t $PID -n ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
52: eth0#if53: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.17.0.3/16 scope global eth0
valid_lft forever preferred_lft forever
Get the IP of the container from Docker
$ IP=$(docker inspect -f '{{.NetworkSettings.Networks.bridge.IPAddress}}' $CID)
$ echo $IP
172.17.0.3
Test the connection
$ psql -h $IP -U postgres -p 5432
Mapped ports
With a mapped port, the ports on the host changes slightly
$ CID=$(docker run -d -p 5432:5432 postgres)
$ echo $CID
020f72394fcd...
Now the container has a port configured
$ docker container port $CID
5432/tcp -> 0.0.0.0:5432
$ docker inspect -f {{.NetworkSettings.Ports}} $CID
map[5432/tcp:[{0.0.0.0 5432}]]
The port will also be listening on the host
$ ss -lntp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:* users:(("sshd",pid=592,fd=3))
LISTEN 0 128 :::22 :::* users:(("sshd",pid=592,fd=4))
LISTEN 0 128 :::5432 :::* users:(("docker-proxy",pid=8571,fd=4))
And you can connect to localhost or your hosts IP now
$ psql -h 127.0.0.1 -U postgres -p 5432