I am currently trying to use Docker for my new Django/Postgres project. I am working on a Mac and usually use Postico to quickly connect to my database.
I used to connect like here:
I used the official Docker documentation to setup docker-compose. I now have the issue, that I can't connect via Postico to the postgres db. It seems to me that the problem comes from the ports not being exposed.
version: '3'
services:
db:
image: postgres
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Just map the port to the host machine, add this to the db service in your Compose file:
ports:
- "5432:5432"
Also make sure to set the postgres password variable in the compose file like this
environment:
POSTGRES_PASSWORD: example
The default user is postgres, you can change it with the POSTGRES_USER variable.
You can read about the usage of the image with all options here: https://hub.docker.com/_/postgres/
By default Compose sets up a single network for your app.
Each container can be accessed by the name of the service in the compose file.
In your case you don't have to expose the port to the host machine for your web app to have access to it. You can simply use db as the hostname for postgres (and 5432 for the port) from any other service running on the same compose.
Actually a very similar example is provided in the docker compose documentation:
https://docs.docker.com/compose/networking/
Related
I am building a Flask application in Python. I'm using SQLAlchemy to connect to PostgreSQL.
In the flask application, I'm using this to connect SQLAlchemy to PostgreSQL
engine = create_engine('postgresql://postgres:[mypassword]#db:5432/employee-manager-db')
And this is my docker-compose.yml
version: '3.8'
services:
backend:
build:
context: .
dockerfile: Dockerfile
ports:
- 8000:8000
volumes:
- .:/app
links:
- db:db
depends_on:
- pgadmin
db:
image: postgres:14.5
restart: always
volumes:
- .dbdata:/var/lib/postgresql
hostname: postgres
environment:
POSTGRES_PASSWORD: [mypassword]
POSTGRES_DB: employee-manager-db
pgadmin:
image: 'dpage/pgadmin4'
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: [myemail]
PGADMIN_DEFAULT_PASSWORD: [mypassword]
ports:
- "5050:80"
depends_on:
- db
I can do "docker build -t employee-manager ." to build the image. However, when I do "docker run -p 5000:5000 employee-manager" to run the image, I get an error saying
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "db" to address: Try again
Does anybody know how to fix this? Thank you so much for your help
Your containers are on different networks and that is why they don't see each other.
When you run docker-compose up, docker-compose creates a separate network and puts all the services defined inside docker-compose.yml on that network. You can see that with docker network ls.
When you run a container with docker run, it is attached to the default bridge network, which is isolated from other networks.
There are several ways to fix this, but this one will serve you in many other scenarios:
Run docker container ls and identify the name or ID of the db container that was started with docker-compose
Then run your container with:
# ID_or_name from the previous point
docker run -p 5000:5000 --network container:<ID_or_name> employee-manager
This attached the new container to the same network as your database container.
Other ways include creating a network manually and defining that network as default in the docker-compose.yml. Then you can use docker run --network <network_name> ... to attach other containers to that network.
docker run doesn't read any of the information in the docker-compose.yml file, and it doesn't see things like the Docker network that Compose automatically creates.
In your case you already have the service fully-defined in the docker-compose.yml file, so you can use Compose commands to build and restart it
docker-compose build
docker-compose up -d # will delete and recreate changed containers
(If the name of the image is important to you – maybe you're pushing to a registry – you can specify image: alongside build:. links: are obsolete and you should remove them. I'd also avoid replacing the image's content with volumes:, since this misses any setup or modification that's done in the Dockerfile and it means you're running untested code if you ever deploy the image without the mount.)
I have my Postgres container running, built from this docker-compose file:
version: "3.9"
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
ports:
- "5432:5432"
environment:
- POSTGRES_DB=db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password.
It spins up fine & my other dockerized servers can connect to it. But, if I open up a CLI from outside the docker instance & try to connect with
psql postgres://postgres:password#localhost:5432/db
Or try to add a database connection in PyCharm, I get
psql: error: could not connect to server: FATAL: database "db" does not exist
as a response. What do I need to do to allow outside calls to the containerized database? I've tried adding "expose:5432" to the docker compose, but that didnt help.
answered my own question here - by stopping PgAdmin from running, which I guess was blocking the 5432 port, the containerized postgres service could be accessed by the CLI & PyCharm
I have a PostgreSQL container set up that I can successfully connect to with Adminer but I'm getting an authentication error when trying to connect via something like DBeaver using the same credentials.
I have tried exposing port 5432 in the Dockerfile and can see on Windows for docker the port being correctly binded. I'm guessing that because it is an authentication error that the issue isn't that the server can not be seen but with the username or password?
Docker Compose file and Dockerfile look like this.
version: "3.7"
services:
db:
build: ./postgresql
image: postgresql
container_name: postgresql
restart: always
environment:
- POSTGRES_DB=trac
- POSTGRES_USER=user
- POSTGRES_PASSWORD=1234
ports:
- 5432:5432
adminer:
image: adminer
restart: always
ports:
- 8080:8080
nginx:
build: ./nginx
image: nginx_db
container_name: nginx_db
restart: always
ports:
- "8004:8004"
- "8005:8005"
Dockerfile: (Dockerfile will later be used to copy ssl certs and keys)
FROM postgres:9.6
EXPOSE 5432
Wondering if there is something else I should be doing to enable this to work via some other utility?
Any help would be great.
Thanks in advance.
Update:
Tried accessing the database through the IP of the postgresql container 172.28.0.3 but the connection times out which suggests that PostgreSQL is correctly listening on 0.0.0.0:5432 and for some reason the user and password are not usable outside of Docker even from the host machine using localhost.
Check your pg_hba.conf file in the Postgres data folder.
The default configuration is that you can only login from localhost (which I assume Adminer is doing) but not from external IPs.
In order to allow access from all external addresses vi password authentication, add the following line to your pg_hba.conf:
host all all * md5
Then you can connect to your postgres DB running in the docker container from outside, given you expose the Port (5432)
Use the command docker container inspect ${container_number}, this will tell you which IPaddress:ports are exposed external to the container.
The command 'docker container ls' will help identify the 'container number'
After updating my default db_name, I also had to update the docker-compose myself by explicitly exposing the ports as the OP did
db:
image: postgres:13-alpine
volumes:
- dev-db-data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=devdb
- POSTGRES_USER=user
- POSTGRES_PASSWORD=1234
ports:
- 5432:5432
But the key here was restarting the server! DBeaver has connected to localhost:5432 :)
I have two containers that don't connect to each other:
1. I made an image postgres that get data from dump.sql
here is Dockerfile:
FROM postgres:11.1-alpine
COPY restore_db.sh /docker-entrypoint-initdb.d/
COPY db.sql /backup/
ENV PGDATA=/data
Then I created container with docker run --name db -p 5432:5432 db
4.I made a image with app. Dockerfile for app look like:
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY build/libs/ /app/
# Make port 80 available to the world outside this container
EXPOSE 8085
# Define environment variable
ENV NAME app
# Run app when the container launches
CMD java -jar /app/olympic-0.0.1-SNAPSHOT.jar
I made a container with run.
then i use docker-compose up with file that looks like:
version: '3'
services:
db:
image: db-data
container_name: postgres
ports:
- 5432:5432
volumes:
- ./pg_data:/data
environment:
POSTGRES_DB: innovation
POSTGRES_USER: postgres
PGDATA: /data
restart: always
web:
image: app
container_name: roc
environment:
POSTGRES_HOST: db
ports:
- 8085:8085
restart: always
links:
- db
```
here is property file:
```
spring.datasource.url=jdbc:postgresql://db:5432/innovation
spring.datasource.username=postgres
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
logging.level.org.hibernate.SQL=DEBUG
logging.level.root=INFO
spring.output.ansi.enabled=ALWAYS
logging.level.org.hibernate.type.descriptor.sql.BasicBinder=TRACE
spring.liquibase.change-log=classpath:liqubase/db.changelog-master.xml
spring.liquibase.url=jdbc:postgresql://db:5432/innovation
spring.liquibase.user=postgres
```
Thet are not able to be connected.
I always got an error:
org.postgresql.util.PSQLException: Connection to db:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
First of all I dont see any network defined in your Docker files for both containers
So I assume ther are on $project-default network.
docker network inspect $project-default
will give you list of all containers using default network.
Now coming to the containers, Let's assume DB is Container 1 (10.1.1.2) and Spring App is Container 2 (10.1.1.3).
You can get running containers IP by running
docker inspect containerName
You are exposing 5432 and 8085 port for db and Spring respectively
Inside Spring app container property file spring.datasource.url
localhost:5432 or db:5432 (not sure what is db hostname mapped to) is not accessable as DB is in different container.
You can try 10.1.1.2:5432
When you are exposing 5432 and 8085 port from Host machine you can access these port.
eg in Docker for Windows it would be 192.168.99.100:5432
but same cant be access from inside container.
spring.datasource.url=jdbc:postgresql://10.1.1.2:5432/innov should work assuming DB is up and running
I have configured a production postgres sql database.
If I need to do debugging work, I don't want to be interacting with the production database or else that will affect the user base. Instead, I need to create a local environment such that nothing will be changed in the production database during debugging.
I am using Postgres SQL 10 and PGAdmin 4
How can I achieve that?
Thanks.
You could set up a test environment with docker.
first a docker-compose.yml file:
version: "3"
services:
db:
image: postgres:10-alpine
volumes:
- ./local_path:/var/lib/postgresql/data
ports:
- "8000:5432"
expose:
- "5432"
admin:
image: dpage/pgadmin4
environment:
- PGADMIN_DEFAULT_EMAIL=admin#admin.com
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- "8080:80"
See the documentation for the docker postgres image on how to set environment variables to define user/password/db name. https://hub.docker.com/_/postgres/
I'm not too familiar with pgadmin but container has minimal setup options:
https://hub.docker.com/r/dpage/pgadmin4/
Then you start the containers with sudo docker-compose up.
The db container is publishing its port on 8000 on your host machine, so there should be no conflict with the postgres server running on the host.
To connect:
psql -h localhost -p 8000 -U postgres
The admin page should be available at port 8080 on your host machine.
When you connect the admin to the database in the UI, the hostname is db and the port is 5432
Now that you have a docker container set up, you might also consider using it for production also :)