PostgreSQL and pgAdmin both running in Docker containers and connecting - postgresql

I am attempting to run both PostgreSQL and pgAdmin in Docker containers. The idea is that the PostgreSQL database should be accessible to any applications I have running on the host machine, and also to pgAdmin.
I am using this command to run PostgreSQL:
docker run -d -e POSTGRES_USER=username -e POSTGRES_PASSWORD=password --name postgres -p 5432:5432 postgres
And to run pgAdmin:
docker run -d -p 1111:1111 --name pgadmin -e "PGADMIN_LISTEN_PORT=1111" -e "PGADMIN_DEFAULT_EMAIL=admin#test.com" -e "PGADMIN_DEFAULT_PASSWORD=test" dpage/pgadmin4
If I go to localhost:1111, I can connect to pgAdmin and login. However, when I try to connect to my local PostgreSQL instance, it gets no response.
Therefore, I tried to run pgAdmin with access to the host internet using --net=host instead of -p 1111:1111:
docker run -d --net=host --name pgadmin -e "PGADMIN_LISTEN_PORT=1111" -e "PGADMIN_DEFAULT_EMAIL=admin#test.com" -e "PGADMIN_DEFAULT_PASSWORD=test" dpage/pgadmin4
Now, when I try to go to localhost:1111 to connect to pgAdmin, I get no response in my browser.
Docker Compose is a possible solution, as I could link the two containers together so they could access each other without having to worry about ports, but I also need pgAdmin to be able to access PostgreSQL instances on other machines, as well as my local one.
I feel like --net=host is broken in Docker. There's a whole thread here with a lot of confusion.
My setup:
Host: Windows 10
Docker: Docker Desktop Community v2.0.0.3 (31259)
Update
I have now tried using --link postgres on the pgAdmin container and it allows me to connect to my local instance of PostgreSQL but not non-local ones, the full command is:
docker run -d -p 1111:1111 --link postgres --name pgadmin -e "PGADMIN_LISTEN_PORT=1111" -e "PGADMIN_DEFAULT_EMAIL=admin#test.com" -e "PGADMIN_DEFAULT_PASSWORD=test" dpage/pgadmin4

The commands were not wrong, only the connection in pgAdmin, so the full list of commands are:
For PostgreSQL:
docker run -d -e POSTGRES_USER=username -e POSTGRES_PASSWORD=password --name postgres -p 5432:5432 postgres
And pgAdmin:
docker run -d -p 1111:1111 --name pgadmin -e "PGADMIN_LISTEN_PORT=1111" -e "PGADMIN_DEFAULT_EMAIL=admin#test.com" -e "PGADMIN_DEFAULT_PASSWORD=test" dpage/pgadmin4
Now the pgAdmin container won't connect to localhost but needs the IP of the PostgreSQL container. Run:
docker inspect postgres
Inspect result:
[
{
...
"NetworkSettings": {
...
"Networks": {
...
"IPAddress": "172.17.0.3",
...
}
}
}
]
We're only interested in the IPAddress from the inspect command. This is the IP which pgAdmin should connect to.
pgAdmin is also capable of access external IPs from your machine.

When you create docker containers it creates a bridge network. First find the network range for the bridge network. You can use ifconfig to find it. Let's say 172.17.0.5 is ip of pgAdmin, Then create a user 'root'#'172.17.0.5' for PostgreSQL and give database permissions for that user. Then you can connect to the database. Also check if port 3306 is accessible using telnet.

Related

psql command from official page on dockerhub

I am learning Docker. I have practiced a lot, including testing commands from the official Postgres page on dockerhub.
I ran this command:
docker run -it --rm --network some-network postgres psql -h some-postgres -U postgres
Could someone give a complete and concrete example to make this command work (i mean with a real existing container). I can't see how it could work.
docker run create a docker container
-it create a connection to said container (kinda like TTY) taking in what we write into interactive bash in the container
--rm delete the container when it exit
--network some-network assign some-network network to the container
postgres name of the image
psql -h some-postgres -U postgres connect to PostgreSQL at some-postgres address using postgres user.
Combine the entire command and flags: create a PostgreSQL container and the use the psql command from inside the container to connect to some-postgres using postgres user
For more flags and usage, you can learning from the doc here
Probably, in the Docker hub page is not perfectly clear but your command is used to connect to an already existing Postgres instance.
So, for example, you first create a container with the command:
docker run -it --rm --name postgresql -p 5432:5432 -e POSTGRES_USER=admin -e POSTGRES_PASSWORD=admin -d postgres:latest
then you can execute your command to connet to it
docker run -it --rm postgres psql -h <your_ip> -U postgres
If your container is running locally, you can get the ip from the bash command ip address
The network attibute is related to the container you first startup so you can decide to leave or remove from the command in relation to the container deploy.

No access to docker container's exposed port from host

when I start docker container like this:
sudo docker run -p5432:5432 -d -e POSTGRES_PASSWORD=test_pass -e POSTGRES_USER=test_user -e POSTGRES_DB=test_db --name postgres postgres:12
I can see it's started by command sudo docker ps. But when I try to connect to the container from host using
psql -Utest_user -p5432 -h localhost -d test_db
it just hangs for several minutes and then reports that wasn't able to connect.
But when I add --net host option like this:
sudo docker run --net host -p5432:5432 -d -e POSTGRES_PASSWORD=test_pass -e POSTGRES_USER=test_user -e POSTGRES_DB=test_db --name postgres postgres:12
everything starts working as expected, I can connect to the postgresql the same psql command.
The same happens to other containers which I run, not only created from postgres:12 image.
I can only make requests to them when I set --net host option.
But I need to expose different ports like for example 2000:5432 to run, for example, several postgres containers simultaneously.
What should I do to make it work? My machine is Ubuntu:20, in case if it matters, and docker is fresh new one installed by instruction from the official site yesterday.
You can't connect to database container because by default it only allows connections from the localhost ( local machines in the same network ).
When you start docker container it makes it's own network ( usually in 172.0.0.0/something ip range).
When you set the flag -net host, docker takes your host's ip address for it's own, and that's why you are able to connect to the database ( because then you are both on the same network ).
The solution is either use the -net host flag, or to edit the config file for the database container to allow external connections which is not recommended.

How to create Postgres backups from docker container

I have a Postgres 9.5.4 database running inside a docker container. I am using the official Postgres images and I have been trying to create backups from my database, but not luck so far. According docker documentation I was expecting the following command to store the dump.tar file in the /releases/ folder in the host machine (outside the temporal docker container)
docker run -i --rm --link my_postgres:postgres --volume /releases/:/tmp/ postgres:latest pg_dumpall > /tmp/dump.tar
But this throws the following error:
pg_dumpall: could not connect to database "template1": could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Any idea what could be wrong?
Actually I have found the solution
docker run -i --rm --link my_container:postgres -v /releases/:/tmp/ postgres:latest bash -c 'exec pg_dumpall -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres > /tmp/my_backup.tar'
Where Docker arguments:
-i For interactive processes, like a shell, bash, etc.
--rm For deleting the container once the backup is finished
--link For linking this temporal container to my_container, which contains the currently running Postgres database.
-v /releases/:/tmp/ Creates shared volume. The content inside the folder /tmp/ in the container (In this case my_backup.tar) will be automatically visible on folder /releases/ in the host machine.
And bash arguments:
pg_dumpall To export PostgreSQL cluster into a script file.
-h "$POSTGRES_PORT_5432_TCP_ADDR" Obtains the ip address out of the Postgres variable.
-p "$POSTGRES_PORT_5432_TCP_PORT" Obtains the Port out of the Postgres variable (These variables are defined only in the temporal container we just created, as result of the linkage with my_container)
-U The username for connecting to the database

can't run docker image of jhipster webapp

I have a jhipster monolithic web app with postgress database. I built a docker image using
./gradlew bootRepackage -Pprod buildDocker
Now when I try to run the image using docker run , it fails with following error.
Caused by: org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:247)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:65)
I tried few things like, but still get the same error:
docker create -v /var/lib/postgresql/data --name spring_app_data postgres:9.5.1
docker run --volumes-from spring_app_data --name spring_app_pg -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=password -d -P postgres:9.5.1
docker run -it --link spring_app_pg:postgres --rm postgres sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
docker run --name spring_app_container --link spring_app_pg:spring_app_pg -p 8080:8080 -d wmd_server_pg
Any suggestions on how to run the docker image for a webapp with PostgreSQL. BTW I get same kind of error when I use mongodb.
Going by your example commands your database won't be accessible as localhost from the app, it will be via the named container. Configure your apps database connection to use spring_app_pg:5432.
Also, don't use links. Use a user defined network, most likely a bridge is all you will need.
docker network create my_app
docker run --net=my_app --name=spring_app_pg <dbimage>
docker run --net=my_app --name=spring_app_container <appimage>
That should give you the same result as your linked setup.

Docker container for Postgres 9.1 not exposing port 5432 to host

I'm trying to use a Docker container to run a PostgreSQL server, and connect with it from my host machine.
My configuration is:
Host machine: Mac OS X 10.10.5
Docker 1.10.1
I've done this:
Step 1: create a volume for permanent postgres data
docker volume create --name postgres_data
Step 2: Start the postgres instance
UPDATE: As suggested in comments, I specified port mapping when running the container
docker run --name my_postgres_container -e POSTGRES_PASSWORD=my_password -v postgres_data:/var/lib/postgresql/data -p 5432:5432 -d postgres:9.1
Step 3: connect to Docker instance by doing this:
docker run -it --link my_postgres_container:postgres --rm postgres:9.1 sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
But I want to connect to that instance just by:
psql -h localhost -p 5432 -U postgres
Like if I had installed Postgres locally in my host machine.
The problem is port 5432 is not exposed. So, I can't connect with it:
sudo lsof -i -P | grep -i "listen" --> no port 5432 open
UPDATE 2: Even stranger. I've also done this:
Stop Docker. Then, run a normal PostgreSQL 9.4.4 instance in my host machine (no docker involved here, just postgres running in my Mac OS X host, listening on port 5432). Everything is normal:
sudo lsof -i -P | grep -i "postgres"
postgres 14100 jorge 6u IPv4 0x780274158cebef01 0t0 TCP localhost:5432 (LISTEN)
I can connect with my local postgres instance without any problem (look the output of the command: is the postgres compiled for Mac OS X, my host):
psql -h localhost -U postgres -c "select version()"
version
---------------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 9.4.4 on x86_64-apple-darwin14.3.0, compiled by Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn), 64-bit
(1 row)
Now the fun part. I start my Docker instance again, while the host PostgreSQL instance is running.
It starts! (and it shouldn't). I can even connect using docker run...
docker run -it --link my_postgres_instance:postgres --rm postgres:9.1 sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres'
If I run select version() now, it shows postgres running inside my docker instance at the same time postgres is running in my host, out of docker, using the same 5432 port. (Look at the ouput, is postgres compiled for Debian, the OS inside the postgres:9.1 container)
postgres=# select version();
version
------------------------------------------------------------------------------------------------
PostgreSQL 9.1.20 on x86_64-unknown-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit
(1 row)
Why?
Does it make sense? My final goal is to run a Django app in another Docker container and connect with my Postgres instance. How could I do that?
It's 2018 and I just had a similar problem. The solution for me seemed to be with the order of props to docker. e.g. this resulted in no port being exposed;
docker run -d --name posttest postgres:alpine -e POSTGRES_PASSWORD=fred -p 5432:5432
while this worked fine (image exposed port 5432 as expected);
docker run --name posttest -d -p 5432:5432 -e POSTGRES_PASSWORD=fred postgres:alpine
Run the postgre image with the correct Port Mapping using -p <host_port>:<container_port>:
docker run --same-options-as-step-one -d -p 5432:5432 postgres:9.1
Your docker host is a virtual machine, which has it's own IP adddres.
You can detect this IP address by entering the following command:
docker-machine ip
The answer will be something like 192.168.99.100
When you have mapped the ports using the -p 5432:5432 switch, you will be able to connect to postgres with any tool from your dev machine using the IP address mentioned.
I was able to connect using container IP or host IP, except localhost (127.0.0.1).
To get container id run
docker ps
Find required container id and run
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <container_id>
Port must be exposed.
Here is an example of docker-compose.yml which starts two containers postgres and adminer, which is database management tool you can use to connect to postgres:
version: '3'
services:
adminer:
image: adminer
restart: always
ports:
- 8080:8080
postgres:
image: postgres:11.4-alpine
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
I had a similar issue. My problem was simply 0.0.0.0 not mapping to localhost so I just had to add to psql
psql --host=0.0.0.0
This is presuming
docker port <container name>
outputs
5432/tcp -> 0.0.0.0:5432
Other answers work, but don't explain why they work.
Given the command:
psql -h localhost -p 5432:5432 -U postgres
localhost is actually a special value that tells psql to look for a unix socket connection, instead of going over TCP. We can't use unix sockets to connect to docker services.
Changing the command like so fixes it, by forcing TCP:
psql -h 127.0.0.1 -p 5432:5432 -U postgres
That will work as long as you docker run ... -p 5432:5432 .... Specifying the IP returned by docker-machine ip also forces TCP, so that also works.
I had a similar problem working in a VMWare virtual machine with Lubuntu. The VM had been paused and then was restarted. The PostgreSQL Docker container was correctly mapped and listening on localhost:5432, but I always got:
psql: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Restarting the VM solved the problem in my case.
Try this to install postgresql
docker run --name postgres -d -p 5432:5432 -e POSTGRES_PASSWORD=fred postgres:alpine
Or Change port host machine Heree (mac)
docker run --name postgres -d -p 5436:5432 -e POSTGRES_PASSWORD=fred postgres:alpine
Tip:
Install pgadmin4
docker run -p 5050:80 -e "PGADMIN_DEFAULT_EMAIL=name#example.com" -e "PGADMIN_DEFAULT_PASSWORD=admin" -d dpage/pgadmin4