how to interpret db volume entries in a docker compose file - postgresql

I'm in the process of setting up my first postgresql docker container, as part of a distributed application using docker compose, and am somewhat confused on the syntax of the compose file. Still pretty new to docker so forgive me if this is straight out of docker 101.
Here's what it looks like for my postgres container in docker-compose.yml:
version: '2'
services:
database:
image: postgres
container_name: database-container-name
environment:
- POSTGRES_PASSWORD=some_password_here
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- PGDATA=/var/lib/postgresql/data/db-files/
ports:
- 5433:5432
volumes:
- ./.db/data:/var/lib/postgresql/data:delegated
- ./.db/init:/docker-entrypoint-initdb.d
The volumes bit is what throws me for a loop. Can someone explain what's going on there, is the container mapping its /.db/data/ folder to my local /var/lib/postgresql/data folder? I've looked at some documentation but it's not sinking in.
Also, in the ports section above, what's the deal with 5433:5432? Does that mean my port 5433 maps to the docker container's port 5432? If so, does this mean if I connnect to psql or pgAdmin on port 5433 on my box it's silently mapping to the postgresql instance in the docker container?

The volumes bit is what throws me for a loop. Can someone explain what's going on there, is the container mapping its /.db/data/ folder to my local /var/lib/postgresql/data folder?
In the volumes section of your database service, you're creating a couple of bind mounts between your host and your container. The syntax of each volume entry (in this situation) is HOST_PATH:CONTAINER_PATH:OPTIONS. So when you see:
- ./.db/data:/var/lib/postgresql/data:delegated
You are mapping the local (to your docker-compose.yml) .db/data directory onto /var/lib/postgresql/data in the container. In other words, changes made in one directory will be visible in the other.
The delegated option is specific to MacOS; the docs say:
delegated: The container runtime’s view of the mount is authoritative. There may be delays before updates made in a container are visible on the host.
The volume section of your service corresponds to the -v option to docker run; you may find more interesting in the docker run documentation in addition to the bind-mount docs I linked above.
Also, in the ports section above, what's the deal with 5433:5432? Does that mean my port 5433 maps to the docker container's port 5432? If so, does this mean if I connnect to psql or pgAdmin on port 5433 on my box it's silently mapping to the postgresql instance in the docker container?
The ports section is for publishing ports on your host. The syntax is HOST_PORT:CONTAINER_PORT. So when you see:
- 5433:5432
This is mapping host port 5433 to container port 5432. In other words, you can connect to your postgres database by connecting to port 5433 on your host. This is probably in place to avoid a conflict if you already had a postgres instance running on your host, which would already be bound to port 5432.

Related

Error: P1001: Can't reach database server at `localhost`:`5432`

I'm having a problem when running the npx prisma migrate dev command. Docker desktop tells me that the database is running correctly on port 5432 but I still can't see the problem.
I tried to put connect_timeout=300 to the connection string, tried many versions of postgres and docker, but I can't get it to work.
I leave you the link of the repo and photos so you can see the detail of the code.
I would greatly appreciate your help, since I have been lost for a long time with this.
Repo: https://github.com/gabrielmcreynolds/prisma-vs-typeorm/tree/master/prisma-project
Docker-compose.yml
version: "3.1"
services:
postgres:
image: postgres
container_name: postgresprisma
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=santino2002
ports:
- "5432:5432"
volumes:
- postgres:/var/lib/postgresql/data
volumes:
postgres:
Error:
Error: P1001: Can't reach database server at localhost:5432
Please make sure your database server is running at localhost:5432.
Docker ps show this:
Looks like the application and the database are running on two separate containers. So, in this case, connecting to localhost:5432 from the application container will try to connect to 5432 port within that container and not in the docker host's localhost.
To connect to database from the application container, use postgres:5432 (If they are on the same network) or <dockerhost>:5432.
Your docker ps output is showing that your postgres container has no ports connected to your local network.
It should look something similiar to this on ports column.
0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
But yours is just 5432/tcp
You need to open ports for your postgres container.
Your docker-compose.yml file you posted in the question is correct. Probably you started postgres container with no ports first, then changed your docker-compose.yml file to have ports. So you just need to restart it now.
Use docker compose down && docker compose up --build -d to do that.

How to connect to containerized database with its IP?

I'm new to Docker. I successfully created a PostgreSQL container my-database and I am able to access it from SQLTools on my local machine with server address localhost and the port.
I got the containerized database's IP address from the following command:
docker container inspect my-database
But when I go back to SQLTools or the PHP web application (not containerized) and try to connect to my-database with the IP address I got above, it couldn't connect successfully.
What am I missing here?
FYI, I also created another container and was able to connect to my-database with the following way: Use the same network for both my-database and the second container.
It all depends on how you enable access to the database.
If your php service runs in the same machine, then localhost could work
If its on a different machine in the same network, then use the network IP assigned to that machine. If you have your php server in a totally different location, then you may want to use something like an nginx reverse proxy to your docker container.
So in your case you should get the ip:port where your db container runs and use that. Docker inspect shows the internal network ip which only helps other containers in the same virtual network connect to a container.
You never need the docker inspect IP address. You can only connect to it in two specific circumstances: if you're in a different container on the same Docker network, or if you're (a) not in a container at all, (b) on the same host, and (c) that host is running native Linux and not a different OS.
You've already identified the right answers. Say you start the container as
docker network create any-network-name
docker run \
--name database \
--net any-network-name \
-p 5432:5432 \
postgres
From another Docker container on the any-network-name network, you can use the database container name as a DNS name, and avoid the manual IP lookup; ignore any -p options and use the "normal" port for the service. From outside a container, you can use the host's DNS name or IP address (or, if it's the same host, localhost) and the first -p port number.
If you're running this in Docker Compose, it automatically creates a network for you and registers containers under their Compose service name; you do not need networks:, container_name:, or other similar settings.
version: '3.8'
services:
database:
image: postgres
ports: ['5432:5432']
application:
build: .
environment:
- PGHOST=postgres

Error creating postgres container with Docker

I'm initializing my studies with docker. When I creating the postgreSQL container with Docker, appear this error:
I've already reinstall docker and tried change the database port, but must be don't have success
The error indicates that the 5432 port is already in use, in the HOST. This means that this port is occupied by another application. This could be for example a Postgres instance that you have on your machine or maybe another container using up this port.
The port option is -p hostPort:ContainerPort, so to have access to the Postgres port of the container which is 5432 in the host at port 5000 try:
-p 5000:5432

Connect to Windows Postgres from Debian Docker container

I am running Postgres on a Windows 10 computer, and I want to connect to it from a Docker container. I've followed instructions from many sources and things should be working, but they're not.
Command line used to create Docker container:
docker run --rm -d --network=host --name mycontainer myimage
In postgresql.conf:
listen_addresses = '*'
In pg_hba.conf:
host all all 172.17.0.0/16 trust
In the bash shell of my container, I run:
psql -h 127.0.0.1
and I get the error:
psql: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432?
Needless to say, Postgres is definitely running on my computer and I am able to query it from local applications. What am I missing?
THIS WON'T WORK FOR DOCKER v18.03 AND ONWARDS
The answer is already there - From inside of a Docker container, how do I connect to the localhost of the machine?
This question is related to a mysql setup, but it should work for your case too.
FOR DOCKER v18.03 ONWARDS
Use host.docker.internal to refer to the host machine.
https://docs.docker.com/docker-for-windows/networking/#i-cannot-ping-my-containers
As you've discovered, --network-host doesn't work with Docker for Windows or Docker for Mac. It only works on Linux hosts.
One option for this scenario might be to host PostgreSql in a container, also. If you deploy them with a docker-compose file, you should be able to have two separate Docker containers (one for the database and one for your service) that are networked together. By default, docker-compose will expose containers to others in the same compose file using the container name as its DNS name.
You could also consider including the database in the same container as your service, but I think the docker-compose solution is better for several reasons:
It adheres to the best practice of each container having only a single process and single responsibility.
It means that you can easily change and re-deploy your service without having to recreate the database container.
Configure the connection inside your docker container with the real ip-address of your host or as workaround with a dns name

How do you register a Postgres container host on docker-compose up?

I have an app (Flask/Pony ORM) container that must register the host for a postgres db container on start up. How can I get the host's (postgres container) location?
On my local set up I have "localhost" but this does not work in the app container.
I tried "172.17.0.2" which was what the pg containers IP when I was not using docker-compose.
But both of these return an error.
Is there a way to specify a host location for a particular container in the docker-compose file?
You must link your two containers in your docker-compose file. Then you will be able to access to your database using the service name of the container.
Example
links:
- postgres:postgres
Link your docker-compose.yml if you need more help.