docker confusion about port - postgresql

I created a docker postgres container by using a compose file:
postgres:
image: postgres:12-alpine
volumes:
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
ports:
- "5432:5432"
init.sql:
CREATE USER test WITH PASSWORD '123456';
ALTER USER test WITH SUPERUSER;
CREATE DATABASE test;
I also created a pgAdmin4 container by running:
docker pull dpage/pgadmin4
docker run -p 5050:80 \
-e "PGADMIN_DEFAULT_EMAIL=user#domain.com" \
-e "PGADMIN_DEFAULT_PASSWORD=SuperSecret" \
-d dpage/pgadmin4
when setting up the connection in pgAdmin. I need to set up the server connection to 172.17.0.2 by looking at the postgres container IPAddress: docker container inspect 02e83f5f39d6 | grep IPAddress, using localhost as connection string will fail in pgAdmin4.
However, when I use python sqlalchemy, I could connect to localhost:5432.
import pandas as pd
from sqlalchemy import create_engine, MetaData
eng = create_engine('postgresql://test:123456#localhost:5432/test', echo=True)
meta = MetaData()
meta.reflect(eng, schema='public')
This is confusing. Why pgAdmin didn't recognize the port mapping? Is it because it is running from another container? What should be the correct set up then?

localhost is relative to the container / host.
For the pgAdmin container, localhost is itself. So you need to specify the IP to make pgAdmin point to the correct host.
Similarly for the python script running on your machine, localhost is the host machine (your PC). But since you specified the port mapping 5432:5432 the port on localhost:5432 is forwarded by docker the postgres containers port 5432. So if the script connects to localhost:5432 - it is forwarded to the container by docker.
There is nothing wrong for with this setup. Better would be have the pgAdmin container as part of the same compose file. With that you can utilize the name of the service as a DNS mapping to the container IP for easier management.

Related

No access to docker container's exposed port from host

when I start docker container like this:
sudo docker run -p5432:5432 -d -e POSTGRES_PASSWORD=test_pass -e POSTGRES_USER=test_user -e POSTGRES_DB=test_db --name postgres postgres:12
I can see it's started by command sudo docker ps. But when I try to connect to the container from host using
psql -Utest_user -p5432 -h localhost -d test_db
it just hangs for several minutes and then reports that wasn't able to connect.
But when I add --net host option like this:
sudo docker run --net host -p5432:5432 -d -e POSTGRES_PASSWORD=test_pass -e POSTGRES_USER=test_user -e POSTGRES_DB=test_db --name postgres postgres:12
everything starts working as expected, I can connect to the postgresql the same psql command.
The same happens to other containers which I run, not only created from postgres:12 image.
I can only make requests to them when I set --net host option.
But I need to expose different ports like for example 2000:5432 to run, for example, several postgres containers simultaneously.
What should I do to make it work? My machine is Ubuntu:20, in case if it matters, and docker is fresh new one installed by instruction from the official site yesterday.
You can't connect to database container because by default it only allows connections from the localhost ( local machines in the same network ).
When you start docker container it makes it's own network ( usually in 172.0.0.0/something ip range).
When you set the flag -net host, docker takes your host's ip address for it's own, and that's why you are able to connect to the database ( because then you are both on the same network ).
The solution is either use the -net host flag, or to edit the config file for the database container to allow external connections which is not recommended.

How to connect to containerized database with its IP?

I'm new to Docker. I successfully created a PostgreSQL container my-database and I am able to access it from SQLTools on my local machine with server address localhost and the port.
I got the containerized database's IP address from the following command:
docker container inspect my-database
But when I go back to SQLTools or the PHP web application (not containerized) and try to connect to my-database with the IP address I got above, it couldn't connect successfully.
What am I missing here?
FYI, I also created another container and was able to connect to my-database with the following way: Use the same network for both my-database and the second container.
It all depends on how you enable access to the database.
If your php service runs in the same machine, then localhost could work
If its on a different machine in the same network, then use the network IP assigned to that machine. If you have your php server in a totally different location, then you may want to use something like an nginx reverse proxy to your docker container.
So in your case you should get the ip:port where your db container runs and use that. Docker inspect shows the internal network ip which only helps other containers in the same virtual network connect to a container.
You never need the docker inspect IP address. You can only connect to it in two specific circumstances: if you're in a different container on the same Docker network, or if you're (a) not in a container at all, (b) on the same host, and (c) that host is running native Linux and not a different OS.
You've already identified the right answers. Say you start the container as
docker network create any-network-name
docker run \
--name database \
--net any-network-name \
-p 5432:5432 \
postgres
From another Docker container on the any-network-name network, you can use the database container name as a DNS name, and avoid the manual IP lookup; ignore any -p options and use the "normal" port for the service. From outside a container, you can use the host's DNS name or IP address (or, if it's the same host, localhost) and the first -p port number.
If you're running this in Docker Compose, it automatically creates a network for you and registers containers under their Compose service name; you do not need networks:, container_name:, or other similar settings.
version: '3.8'
services:
database:
image: postgres
ports: ['5432:5432']
application:
build: .
environment:
- PGHOST=postgres

How to connect to a Postgres database running in local Docker container through locally-run psql command?

I'm running a docker container with the vanilla Postgres image on my local machine. I'd like to connect to the database from my local machine (i.e., not from "within the container". However, on trying to connect, I get an error.
Here's my docker-compose.yml file:
version: "3.8"
services:
db:
image: postgres
restart: always
ports:
- 5432:5432
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: mypassword
Here's how I start up:
docker-compose run db
Here's how I connect:
psql -h localhost -p 5432 -U postgres
This produces the error:
could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?
If I spin up the database without Docker Compose, the same connection command works as expected:
docker run --name mypg -p 5432:5432 -e POSTGRES_PASSWORD=password postgres
I could just go with the flow and use the command above. But this seems to be pointing to a flaw in how I think about Docker/Compose. For example, maybe Docker Compose's internal DNS resolver makes this approach fail.
Any ideas?
Version info:
psql --version
psql (PostgreSQL) 13.3
I have read through several SO posts, including these, but they don't address or fix the problem I'm seeing:
docker-compose: accessing postgres' shell (psql)
Can't connect to postgres when using docker-compose
Try docker-compose up db instead of run. Using run will run a one-off command against your container, whereas up will turn on the container and leave it running, so another application should be able to access it.
https://docs.docker.com/compose/faq/#whats-the-difference-between-up-run-and-start

Populating a Dockerized PostgreSQL Database through a Dockerized Spring Boot Application

I have a Postgres server running inside Docker. Inside this Postgres server, I have a database named 'aa'.
I also have a Docker image of a Spring Boot Application. When this image is executed in Docker, database tables should be created inside database 'aa'.
In order to achieve this, I executed the following steps:
Run the Postgres Server inside Docker
docker run --name PostgresServer --e POSTGRES_PASSWORD=*** -d Postgres
Enter PostgresServer then Create database 'aa'
sudo docker exec -it PostgresServer psql -U postgres
CREATE DATABASE AA;
3. Run the Sprint Boot Docker Image (this is where the problem happens)
docker run -v /Users/juancesard.pineda/Desktop/brapi:/home/brapi/properties -d brapicoordinatorselby/brapi-java-server:v2
I checked the logs: It says database 'aa' does not exist wherein clearly it exists in the Postgres Server
org.postgresql.util.PSQLException: FATAL: database "aa" does not exist
Some additional info:
Docker listens at port 8080
Postgres server listens at port 5432
My application.properties file looks like this:
server.port = 8080
server.servlet.context-path=/Users/juancesard.pineda/Desktop/brapi/application.properties/germplasm
spring.datasource.url=jdbc:postgresql://host.docker.internal:5432/aa
spring.datasource.username=****
spring.datasource.password=****
spring.datasource.driver-class-name=org.postgresql.Driver
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.show-sql=false
spring.jpa.properties.hibernate.hbm2ddl.import_files=sql/crops.sql, sql/lists.sql, sql/locations.sql, sql/people.sql, sql/programs.sql, sql/trials.sql, sql/seasons.sql, sql/studies.sql, sql/breeding_methods.sql, sql/germplasm.sql, sql/attribute_defs.sql, sql/attribute_values.sql, sql/seed_lots.sql, sql/observation_units.sql, sql/crosses.sql, sql/pedigree.sql, sql/events.sql, sql/images.sql, sql/observation_variables.sql, sql/observations.sql, sql/samples.sql, sql/allele_calls.sql, sql/genome_maps.sql, sql/references.sql, sql/vendor.sql
spring.mvc.dispatch-options-request=true
Am I missing some config in my application.properties file?
Thank you in advance
The problem is in your application.properties file. The datasource url is pointing to host.docker.internal. This should only be used when connecting something from inside a docker container to something running outside docker on the host machine (documentation). You need a different solution because your Spring server and Postgres server are both running inside docker containers.
You need a Docker Network, specifically a bridge network, to connect your different containers. Docker Bridge Networks
Try something like this
Run this command to create a network in docker called “brapi_net”
$> docker network create --driver bridge network_name
Run the postgres container and link it to the network. Create your database schema as normal.
$> docker run --name PostgresServer --network=network_name --e POSTGRES_PASSWORD=*** -d Postgres
$> sudo docker exec -it PostgresServer psql -U postgres
$> CREATE DATABASE aa;
With a Docker network, container names act as server names. Modify your application.properties to reflect this
spring.datasource.url=jdbc:postgresql://PostgresServer:5432/aa
Run the server container and link it to the network.
$> docker run -v /Users/juancesard.pineda/Desktop/brapi:/home/brapi/properties --network=network_name -d brapicoordinatorselby/brapi-java-server:v2

Docker on MacOS, unable to bind Postgres to port 5432

Unsure if this is an issue with Docker for Mac, my Postgres installation (via Homebrew, or some other strange local issue going on.
Probably worth mentioning that I have never ran into this issue on Linux.
Here is a snippet from my docker-compose.yml file that uses the postgres image:
services:
postgres:
image: postgres
environment:
- POSTGRES_DB=some_db
- POSTGRES_USER=some_user
- POSTGRES_PASSWORD=supersecretpassword
ports:
- 5432:5432
After starting compose services docker-compose up, I can see my postgres container running (docker ps).
I can also connect to it directly via docker exec
docker exec -it <container_id> psql -d some_db -U some_user
Everything working as expected so far...
However, when I try to connect to my Docker postgres instance via my local psql client:
psql -h localhost -p 5432 -U some_user -d some_db
It is not connecting to the Docker instance of postgres and rather is trying to use my local postgres instance.
Now the strange part...
When I change the port binding in docker-compose.yml
from 5432:5432
to 5433:5432 (5433 can be any open port)
I am able to connect to the Docker postgres instance as expected via my local psql client:
psql -h localhost -p 5433 -U some_user -d some_db
# After being prompted for my password I'm in!
I don't mind binding to a different port, but I'm still so curious what's happening here!
Anyone know what is happening?
Am I just not able to bind to port 5432?
Is this a Docker for Mac thing?
Is there an issue in the way I installed Postgres?
Thanks!