Running several docker containers including postgres database remotely.
$docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------------
fiware-cygnus /cygnus-entrypoint.sh Up (healthy) 0.0.0.0:5050->5050/tcp, 0.0.0.0:5080->5080/tcp
fiware-elasticsearch /docker-entrypoint.sh elas ... Up 9200/tcp, 9300/tcp
fiware-grafana /run.sh Up 0.0.0.0:53153->3000/tcp
fiware-iotagent pm2-runtime bin/lwm2mAgent ... Up (healthy) 0.0.0.0:4041->4041/tcp, 5684/tcp, 0.0.0.0:5684->5684/udp
fiware-memcached docker-entrypoint.sh memca ... Up 11211/tcp
fiware-mongo docker-entrypoint.sh --bin ... Up 0.0.0.0:27017->27017/tcp
fiware-nginx nginx -g daemon off; Up 0.0.0.0:53152->53152/tcp, 80/tcp
fiware-orion /usr/bin/contextBroker -fg ... Up (healthy) 0.0.0.0:1026->1026/tcp
fiware-postgres docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp
fiware-wirecloud /docker-entrypoint.sh Up (healthy) 8000/tcp
I want to get the external IP address of the fiware-postgres container so I can connect via pgAdmin, instead of managing db via postgres client. It appears the IPAddress of fiware-postgres is only accessible internally.
$docker inspect fiware-postgres | grep "IPAddress"
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "172.19.0.3",
pgAdmin error:
Unable to connect to server:
could not connect to server: Operation timed out
Is the server running on host "172.19.0.3" and accepting
TCP/IP connections on port 5432?
Is there a way to get it's external IP, or at least some way of connecting via pgAdmin (remember docker container run on a remote, accessed via ssh).
EDIT
The remote host is accessible via ssh root#193.136.x.x:2222 so the postgres port 5432 cannot be reached.
pgAdmin settings(Connection Tab):
Host: 193.136.xx.xx
Port: 5432
Maintenance database: postgres
Username: postgres
Password: password
pgAdmin error:
Unable to connect to server:
could not connect to server: Operation timed out
Is the server running on host "193.136.x.x" and accepting
TCP/IP connections on port 5432?
postgres service definition (in docker-compose):
postgres:
restart: always
image: postgres:10
hostname: postgres
container_name: fiware-postgres
expose:
- "5432"
ports:
- "5432:5432"
networks:
- default
environment:
- "POSTGRES_PASSWORD=password"
- "POSTGRES_USER=postgres"
- "POSTGRES_DB=postgres"
volumes:
- ./postgres-data:/var/lib/postgresql/data
build:
context: .
shm_size: '2gb'
Assuming that you have exposed the ports of your container to the underlying host, then the application is accessible via the IP address of the underlying host.
The 172.19.0.3 IP address is an IP address that exists inside the Docker network on the host: it's not available externally. If your remote host has IP 1.2.3.4 and your application port has been exposed, e.g. in your compose file you have something like:
ports:
- "5432:5432"
Then your application should be accessible via 1.2.3.4:5432.
EDIT:
If you are connecting from your local machine to the remote server using pgAdmin, then there should be no need to ssh: you should find that the service is still available at 1.2.3.4:5432. However, if you have some form of firewall in the way (e.g. you're sshing in to an AWS server), then access to 5432 on the remote host may well be blocked. In that case, consider using port forwarding to route connections on your local server to the remote server, via the ssh connection.
ssh -L 5432:193.136.x.x:5432 193.136.x.x:2222
Now connect using pgAdmin to localhost:5432 or 127.0.0.1:5432.
Here's the solution that works for me:
ssh -p 2222 root#193.136.xx.xx -L 5432:localhost:5432
postgres server finally accessible via pgAdmin
Related
I have the following two containers in my docker-compose.yml file
postgres:
image: postgres:10.5
ports:
- 5105:5432
...
web:
restart: always
build: ./web
ports: # to access the container from outside
- "8000:8000"
env_file: .env
command: /usr/local/bin/gunicorn directory.wsgi:application --reload -w 1 -b :8000
volumes:
- ./web/:/app
depends_on:
- postgres
When I'm logged in to my "web" container (an Ubuntu 18 container), I'd like to be able to login to the PostGres container. How do I do this? I tried this
root#0868cef9c65c:/my-app# PGPORT=5432 PGPASSWORD=password psql -h localhost -Uchicommons directory_data
psql: error: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
but this doesn't seem to be working.
In a Docker container, localhost refers to the container itself.
By default, Docker compose creates a docker bridge network and connects each container to it. From a container on the bridge network, you can reach other containers using their service names. So to reach the database container, you'd use postgres as the host name, like this
PGPORT=5432 PGPASSWORD=password psql -h postgres -Uchicommons directory_data
On the bridge network, you use the native ports. So it's port 5432 for Postgres. If you only need to access a container from other containers on the bridge network, you don't need to map the port to a host port. Mapping to a host port is only needed if you need to access the container from the host computer.
I'm attempting to connect PGAdmin to a docker container and found this post (https://stackoverflow.com/a/57729412/11923025) very helpful in doing so. But I've tried testing using a port other than 5432 and am not having any luck.
For example, I tried using 5434 in my docker-compose file, and tried using that port in pgadmin but got the below error (This is the IP address found from using docker inspect)
This is what my docker-compose file looks like (I am using different ports for 'expose' and 'ports' on purpose, to try and narrow down which one will allow me to connect through PGAdmin, but am having no luck
database:
image: postgres:10.4-alpine
container_name: kafka-nodejs-example-database
environment:
POSTGRES_USER: "abcdef"
POSTGRES_PASSWORD: "abcdef"
expose:
- "5435"
ports:
- 8000:5434
pgadmin:
image: dpage/pgadmin4
ports:
- 5454:5454/tcp
environment:
- PGADMIN_DEFAULT_EMAIL=admin#mydomain.com
- PGADMIN_DEFAULT_PASSWORD=postgres
- PGADMIN_LISTEN_PORT=5454
Why is is that pgadmin has no issues with 5432 but when I ask it to use another port it throws this error?
I should note, the error in the screenshot above is from attempting to connect postgres container to a pgadmin container. I also tried connecting to the postgres container in my local application of pgadmin, and get a different timeout error below. I even get this same error for port 5432 when trying to connect using my local copy of pgadmin.
You exposed port 5434 of your container, but PostgreSQL itself is still configured to listen on port 5432. That is why you don't reach the database.
After running initdb and before starting PostgreSQL, configure the cluster, for example with
echo 'port = 5434' >> datadir/postgresql.auto.conf
But there should not be any need to start PostgreSQL on a different port. Just map port 5432 to 5434.
The PostgreSQL server listens on port 5432. Just changing things in Docker-level configuration doesn't change where the server process itself listens. That means:
The second number in ports: must be 5432. (The first number can be any number you want.)
Connections from other Docker containers must connect to port 5432. (ports: are ignored and aren't required.)
If you expose: a port, it must also be 5432. (This has no practical effect and duplicates EXPOSE 5432 in the Dockerfile.)
Since each container runs in an isolated network namespace, there's no particular reason to want to change this. You can run multiple database containers, and each will have its own separate container port 5432; they will not conflict.
The main problem is that i cannot run postgresql even on vm with the error:
root#a2c8a58d4e0e:/# psql -h localhost -U psqluser -W
Password for user psqluser:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
For that purpose i run this commands inside VM:
pg_createcluster 9.6 main --start
/etc/init.d/postgresql start
And then it works properly on VM. But that's manually.
I configured everything by official docker repo docs.
This is my docker compose file:
version: "3.3"
services:
postgresql:
build:
context: .
dockerfile: postgresql
container_name: Postgres
restart: always
ports:
- "5432:5432"
environment:
POSTGRES_DB: 'psqldb'
POSTGRES_USER: 'psqluser'
POSTGRES_PASSWORD: 'temp123'
volumes:
- /home/VOLUMES/DB/Postgresql:/var/lib/postgresql
I did inheritance from original repo as i want to run postgresql service automatically. Otherwise it's not running.
postgresql file:
FROM postgres:9.6.11
RUN pg_createcluster 9.6 main --start
RUN /etc/init.d/postgresql start
It does not run Postgres as well. Only manually inside VM.
What's wrong?
Within docker compose, ports aren't exposed on the host by default. https://docs.docker.com/compose/compose-file/
ports is virtual within the docker compose network. If you want to expose them to the host machine, you can use the expose option instead of ports.
Alternatively, you can also run docker-compose run with the --service-ports flag which will automatically expose the ports to the host when running.
docker-compose run --service-ports postgresql (see doc)
I'm using the following docker-compose.yml file:
version: "3.5"
services:
db:
image: postgres:latest
volumes:
- ./tmp/postgresql/:/var/run/postgresql/:rw
ports:
- 5432:5432
environment:
POSTGRES_DB: dev
POSTGRES_USER: username
POSTGRES_PASSWORD: pw
I can connect to the postgres instance from my host as I have 5432 port forwarded to the host. I wanted to try connecting via the socket, but I am hitting issues. I'm not sure if this is possible?
From my host I can use the following:
psql --host=/Users/jalbert/Projects/postgres-sockets/tmp/postgresql -U username -d dev`
psql: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/Users/jalbert/Projects/postgres-sockets/tmp/postgresql/.s.PGSQL.5432"?
Although, I do see the socket file present there. If I go within the docker container I can connect via the socket no problem using a similar command.
Am I missing something to allow the host to use the socket connection? Unfortunately, I am not too familiar with the socket technologies.
I run into the strange problem. I've created docker-compose file to build php + nginx + postgres services:
version: '2'
services:
db:
image: orchardup/postgresql
ports:
- "5433:5432"
environment:
LC_ALL: C.UTF-8
POSTGRESQL_USER: postgres
POSTGRESQL_DB: db
POSTGRESQL_PASS: postgres
php:
build: .docker/php-fpm
ports:
- "9002:9002"
volumes:
- .:/var/www/symfony:cached
- ./var/log/symfony:/var/www/symfony/var/log:cached
links:
- db
nginx:
build: .docker/nginx
ports:
- "8001:80"
links:
- php
volumes_from:
- php
volumes:
- ./var/log/nginx/:/var/log/nginx:cached
After that I created DB schema by running bin/console doctrine:schema:update --force . The tables and migrations created just fine. Seems like DB connection is ok. I checked this by connecting to db from my machine through psql with credentials from .env, the tables are there.
But when I go to the web page and trying to authorize, I get an error told me the connection is not ok:
Connection refused Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5433?"
I checked in both case I have dev environment - from the web page and from the console. I tried 5433 and 5432 ports with no success. I tried everything I could find for 3 hours.
This is the output from the postgres container:
# netstat -tlpn | grep 5432
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 12/postgres
tcp6 0 0 :::5432 :::* LISTEN 12/postgres
# grep listen /etc/postgresql/9.3/main/postgresql.conf
listen_addresses = '*' # what IP address(es) to listen on;
The only way for containers to talk to each other is through IPs. By linking multiple containers together through --link (or links in docker-compose), docker creates a secure tunnel between those two containers so that we don't need to expose any ports externally.
If you try to connect to your database from your local environment through a database client, you will be able to connect to it from 127.0.0.1:5433 as the port is exposed to your host through the docker-compose file. This is the reason why your schema update command succeeded.
Docker exposes connectivity information for the source container to
the recipient container in two ways:
Environment variables,
Updating the /etc/hosts file.
Ref: https://docs.docker.com/network/links/#communication-across-links
In order to connect to your database (which is running in the db container) from the php container, you will need to get the host of your db container through the environment variable DB_PORT_5432_TCP_ADDR (I might be wrong on this, but type env in your php container's terminal to verify. You will need to SSH into your php container).
Alternatively, you can use the second method, which is just db as the hostname instead of 127.0.0.1 since docker updated the /etc/hosts file in the php container to map your linked container's name to its IP, and in this case, the value mapped to the hostname db is the same as the value stored in the environment variable DB_PORT_5432_TCP_ADDR.