Postgres in Docker port matching - postgresql

I need to run several equals Postgres containers on different ports via docker-compose. My problem is that I do not understand how to connect from my terminal to container's psql.
First of all, I should say that my local Postgres is off, so 5432 is free. Also I have listen_addresses = '*' in my containers.
Here is list of my attempts (to make it easier, I've posted examples for one Postgres container):
Run without port specifying
docker-compose.yml:
version: "3"
services:
postgres:
build:
context: ../../
dockerfile: db-load-test/sharding-benchmark/Dockerfile
environment:
- POSTGRES_PASSWORD=postgres
check port:
$ docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------
sharding-benchmark_postgres_1 docker-entrypoint.sh postgres Up 5432/tcp
try to connect:
$ psql -p 5432 -U postgres
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
$ psql -h localhost -p 5432 -U postgres
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
$ psql -h 0.0.0.0 -p 5432 -U postgres
psql: could not connect to server: Connection refused
Is the server running on host "0.0.0.0" and accepting
TCP/IP connections on port 5432?
Specify port and expose it in .yaml (I've changed 5432 to 5433 to make it more expressive)
$ docker-compose ps
Name Command State Ports
----------------------------------------------------------------------------------------------------------------------------
sharding-benchmark_postgres_1 docker-entrypoint.sh postgres Up 5432/tcp, 0.0.0.0:49162->5433/tcp,:::49162->5433/tcp
Similar attempts to connect was unsuccessful. Also it's unclear to my why Postgres still listen 5432:
postgres_1 | 2021-08-02 12:49:01.394 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2021-08-02 12:49:01.394 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2021-08-02 12:49:01.397 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
Use docker directly
docker run --rm -it -e POSTGRES_PASSWORD=postgres -p 5432:5432/tcp postgres:11
It works with psql -h localhost -p 5432 -U postgres, but if I use docker run with 5432:5433 it breaks, which is unclear to me.

In your docker-compose.yml file, you need to specify ports: for each of the database containers. In each of these the second number must be the normal port for the database (5432); the first number can be any number of your choosing that's not used by another container or host process.
version: '3'
services:
pg1:
image: postgres
# Use default PostgreSQL port 5432 on the host
ports: ['5432:5432']
pg2:
image: postgres
# First port must be different from pg1's above
# Second port must be exactly 5432
ports: ['5433:5432']
When you connect from the host, use a host name of localhost and the first published ports: number. (If you're using the older Docker Toolbox setup, use the docker-machine ip address of the Docker VM; if you're connecting from a different host, use the Docker host's DNS name or IP address.)
# connect to pg1
psql -h localhost -p 5432 -U postgres
# connect to pg2
psql -h localhost -p 5433 -U postgres
You do not need to expose: ports in the docker-compose.yml file; this doesn't really actually do anything. You shouldn't need to reconfigure the stock postgres image beyond its basic environment-variable settings. It doesn't make sense to connect to 0.0.0.0, a special address that means "everywhere" when you're setting up a network listener.
If you're connecting from a different container, none of this matters. Use the Compose service name of the other container (like pg2) and the standard port number (5432). ports: are completely ignored, and localhost means "this container". The two containers must be on the same Docker network; Compose will create a network named default and just using that without explicit networks: settings is fine.

Related

When running psql in a Docker container, how to do I reference my Postgres host in another Docker container?

I have the following two containers in my docker-compose.yml file
postgres:
image: postgres:10.5
ports:
- 5105:5432
...
web:
restart: always
build: ./web
ports: # to access the container from outside
- "8000:8000"
env_file: .env
command: /usr/local/bin/gunicorn directory.wsgi:application --reload -w 1 -b :8000
volumes:
- ./web/:/app
depends_on:
- postgres
When I'm logged in to my "web" container (an Ubuntu 18 container), I'd like to be able to login to the PostGres container. How do I do this? I tried this
root#0868cef9c65c:/my-app# PGPORT=5432 PGPASSWORD=password psql -h localhost -Uchicommons directory_data
psql: error: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
but this doesn't seem to be working.
In a Docker container, localhost refers to the container itself.
By default, Docker compose creates a docker bridge network and connects each container to it. From a container on the bridge network, you can reach other containers using their service names. So to reach the database container, you'd use postgres as the host name, like this
PGPORT=5432 PGPASSWORD=password psql -h postgres -Uchicommons directory_data
On the bridge network, you use the native ports. So it's port 5432 for Postgres. If you only need to access a container from other containers on the bridge network, you don't need to map the port to a host port. Mapping to a host port is only needed if you need to access the container from the host computer.

How to connect to postgres created with docker-compose from outside host

I think this problem should be very easy to solve, but I am pulling my hair out. I have the following docker-compose:
version: "3"
services:
pg-db:
image: postgres:11
environment:
POSTGRES_PASSWORD: "postgres"
POSTGRES_USER: "postgres"
POSTGRES_DB: "postgres"
ports:
- 5432:5432
#network_mode: host
Then I run docker-compose up and it starts the container
pg-db_1_78e7ec4a95e6 | 2020-02-21 13:53:53.928 UTC [1] LOG: database system is ready to accept connections
I can connect to it with docker exec
docker exec -it docker_pg-db-compose_1_78e7ec4a95e6 psql -h pg-db -U postgres postgres
But I can't manage to connect to it by 'naked' psql:
psql postgresql://postgres:postgres#localhost:5432/postgres
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
psql postgresql://postgres:postgres#pg-dg:5432/postgres
psql: could not translate host name "pg-db" to address: Name or service not known
I've tried with network_mode but it doesn't help.
After starting docker-compose, you need to get the docker container ip by doing this:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
Then you can do postgresql://postgres:postgres#<container idp>:5432/postgres you should be able to connect to it
I'm using a modified version of the docker-compose file provided in the Postgres DockerHub page.
version: '3.1'
services:
db:
image: postgres:11
restart: always
ports:
- 5432:5432
environment:
POSTGRES_PASSWORD: postgres
When I run docker-compose up I get a very similar "ready" message:
db_1 | 2020-02-21 14:34:17.826 UTC [1] LOG: database system is ready to accept connections
At this point, I can connect to Postgres using psql.
$ psql -h localhost -U postgres
Password for user postgres:
psql (12.2, server 11.7 (Debian 11.7-1.pgdg90+1))
Type "help" for help.
postgres=#
I'm also able to connect using the connection string that you provided, and postico:
The differences are subtle, I'm not sure if the restart policy makes any difference here, but give it a try.

docker: get container's external IP

Running several docker containers including postgres database remotely.
$docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------------
fiware-cygnus /cygnus-entrypoint.sh Up (healthy) 0.0.0.0:5050->5050/tcp, 0.0.0.0:5080->5080/tcp
fiware-elasticsearch /docker-entrypoint.sh elas ... Up 9200/tcp, 9300/tcp
fiware-grafana /run.sh Up 0.0.0.0:53153->3000/tcp
fiware-iotagent pm2-runtime bin/lwm2mAgent ... Up (healthy) 0.0.0.0:4041->4041/tcp, 5684/tcp, 0.0.0.0:5684->5684/udp
fiware-memcached docker-entrypoint.sh memca ... Up 11211/tcp
fiware-mongo docker-entrypoint.sh --bin ... Up 0.0.0.0:27017->27017/tcp
fiware-nginx nginx -g daemon off; Up 0.0.0.0:53152->53152/tcp, 80/tcp
fiware-orion /usr/bin/contextBroker -fg ... Up (healthy) 0.0.0.0:1026->1026/tcp
fiware-postgres docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp
fiware-wirecloud /docker-entrypoint.sh Up (healthy) 8000/tcp
I want to get the external IP address of the fiware-postgres container so I can connect via pgAdmin, instead of managing db via postgres client. It appears the IPAddress of fiware-postgres is only accessible internally.
$docker inspect fiware-postgres | grep "IPAddress"
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "172.19.0.3",
pgAdmin error:
Unable to connect to server:
could not connect to server: Operation timed out
Is the server running on host "172.19.0.3" and accepting
TCP/IP connections on port 5432?
Is there a way to get it's external IP, or at least some way of connecting via pgAdmin (remember docker container run on a remote, accessed via ssh).
EDIT
The remote host is accessible via ssh root#193.136.x.x:2222 so the postgres port 5432 cannot be reached.
pgAdmin settings(Connection Tab):
Host: 193.136.xx.xx
Port: 5432
Maintenance database: postgres
Username: postgres
Password: password
pgAdmin error:
Unable to connect to server:
could not connect to server: Operation timed out
Is the server running on host "193.136.x.x" and accepting
TCP/IP connections on port 5432?
postgres service definition (in docker-compose):
postgres:
restart: always
image: postgres:10
hostname: postgres
container_name: fiware-postgres
expose:
- "5432"
ports:
- "5432:5432"
networks:
- default
environment:
- "POSTGRES_PASSWORD=password"
- "POSTGRES_USER=postgres"
- "POSTGRES_DB=postgres"
volumes:
- ./postgres-data:/var/lib/postgresql/data
build:
context: .
shm_size: '2gb'
Assuming that you have exposed the ports of your container to the underlying host, then the application is accessible via the IP address of the underlying host.
The 172.19.0.3 IP address is an IP address that exists inside the Docker network on the host: it's not available externally. If your remote host has IP 1.2.3.4 and your application port has been exposed, e.g. in your compose file you have something like:
ports:
- "5432:5432"
Then your application should be accessible via 1.2.3.4:5432.
EDIT:
If you are connecting from your local machine to the remote server using pgAdmin, then there should be no need to ssh: you should find that the service is still available at 1.2.3.4:5432. However, if you have some form of firewall in the way (e.g. you're sshing in to an AWS server), then access to 5432 on the remote host may well be blocked. In that case, consider using port forwarding to route connections on your local server to the remote server, via the ssh connection.
ssh -L 5432:193.136.x.x:5432 193.136.x.x:2222
Now connect using pgAdmin to localhost:5432 or 127.0.0.1:5432.
Here's the solution that works for me:
ssh -p 2222 root#193.136.xx.xx -L 5432:localhost:5432
postgres server finally accessible via pgAdmin

Postgresql via Docker - postgres is not running automatically

The main problem is that i cannot run postgresql even on vm with the error:
root#a2c8a58d4e0e:/# psql -h localhost -U psqluser -W
Password for user psqluser:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
For that purpose i run this commands inside VM:
pg_createcluster 9.6 main --start
/etc/init.d/postgresql start
And then it works properly on VM. But that's manually.
I configured everything by official docker repo docs.
This is my docker compose file:
version: "3.3"
services:
postgresql:
build:
context: .
dockerfile: postgresql
container_name: Postgres
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_DB: 'psqldb'
POSTGRES_USER: 'psqluser'
POSTGRES_PASSWORD: 'temp123'
volumes:
- /home/VOLUMES/DB/Postgresql:/var/lib/postgresql
I did inheritance from original repo as i want to run postgresql service automatically. Otherwise it's not running.
postgresql file:
FROM postgres:9.6.11
RUN pg_createcluster 9.6 main --start
RUN /etc/init.d/postgresql start
It does not run Postgres as well. Only manually inside VM.
What's wrong?
Within docker compose, ports aren't exposed on the host by default. https://docs.docker.com/compose/compose-file/
ports is virtual within the docker compose network. If you want to expose them to the host machine, you can use the expose option instead of ports.
Alternatively, you can also run docker-compose run with the --service-ports flag which will automatically expose the ports to the host when running.
docker-compose run --service-ports postgresql (see doc)

Using Docker to launch web app, can't connect to Postgresql DB?

I received the following error when trying to write session data using Tomcat's PersistentManager to a Postgres DB running on my local machine:
SEVERE: A SQL exception occurred org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
The application itself runs in a docker container. For completeness sake, my current context.xml file is:
<?xml version='1.0' encoding='utf-8'?>
<Context>
<Manager className="org.apache.catalina.session.PersistentManager"
distributable="true" processExpiresFrequency="6" maxIdleBackup="0" debug="99" >
<Store className="org.apache.catalina.session.JDBCStore"
driverName="org.postgresql.Driver"
connectionURL="jdbc:postgresql://localhost/admin?stringtype=unspecified"
connectionName="admin" connectionPassword="admin"
sessionAppCol="app_name" sessionDataCol="session_data" sessionIdCol="session_id"
sessionLastAccessedCol="last_access" sessionMaxInactiveCol="max_inactive"
sessionTable="tomcat_sessions_tb" sessionValidCol="valid_session" />
</Manager>
</Context>
Pursuant to the suggestions here: Postgresql : Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections
I confirmed via a netstat -aln | grep LISTEN that Postgresql is running and listening on the correct ports:
tcp4 0 0 127.0.0.1.5432 *.* LISTEN
tcp6 0 0 ::1.5432 *.* LISTEN
and that my postgresql.conf (located in usr/local/var/postgres) has listen_addresses = localhost and port = 5432, which mirrors the host and port of my running server in Pgadmin3.
I suspect that the problem is that Docker runs in a VM, and thus the local information I have obtained may not be the whole story. Reading up on the available information online, it seems that I may require some sort of bridged networking.
However, I admit I am a novice in this area, and I'm unsure of how to set it up.
Why I can NOT connect to localhost:5432?
Cat your container's /etc/hosts
$ sudo docker exec -it [container] cat /etc/hosts
For docker networks is bridge by default, the localhost inside points to container itself(Docker default bridge network).
Then you don't have 5432 listening in your container:
$ sudo docker exec [container] nc -v -z localhost 5432
Solution 1. If you wanna hardcode the "localhost:5432" inside your config xml, the easiest way is creating your container with the option "--net=host":
$ sudo docker run --net=host -it ...
Solution 2. Change the localhost of your docker host ip inside the container
Get your docker host ip:
$ sudo docker inspect -f '{{ .NetworkSettings.Gateway }}'
192.168.5.1
Enter your container:
$ sudo docker exec -it [container] /bin/bash
Edit the file /etc/hosts to point the localhost to docker host ip:
$ sudo vim /etc/hosts
192.168.5.1 localhost
Solution 3. Modify your db config file to use an alias instead of localhost:
connectionURL="jdbc:postgresql://DB_ALIAS/admin?stringtype=unspecified"
Then add the DB_ALIAS to the container's hosts :
$ sudo docker run --add-host DB_ALIAS:192.168.5.1 -it [image] ...
If you are using docker-compose together with postgres image, than you can reuse service name as IP inside jdbc connection (here: app-db)
web:
build: ./web
ports:
- "8080:8080"
links:
- app-db
environment:
- MYAPP_JDBC_URL=jdbc:postgresql://app-db:5432/somedb
- MYAPP_JDBC_USER=someuser
- MYAPP_JDBC_PASS=pass
app-db:
image: postgres:9.6
environment:
- POSTGRES_USER=someuser
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=somedb
expose:
- 5432
volumes_from:
- app-db-data
app-db-data:
image: cogniteev/echo
command: echo 'Data Container for PostgreSQL'
volumes:
- /opt/postgresdata/:/var/lib/postgresql/data
The best decision!
jdbc:postgresql://host.docker.internal:5432/somedb
Don't thank.
I had to expose port with -p 5432:5432:
docker run --name postgres -e POSTGRES_PASSWORD=secret -d -p 5432:5432 postgres
I was getting the same error but this simple solution works perfect for me.
sudo docker run -d --net="host" -it <IMAGE>
Now I can run my app https://x.x.x.x:pppp/../.. and everything works fine. I hope this helps