Cannot connect to PostgreSQL when running docker compose [duplicate] - postgresql

I'm building an app running on NodeJS using postgresql.
I'm using SequelizeJS as ORM.
To avoid using real postgres daemon and having nodejs on my own device, i'm using containers with docker-compose.
when I run docker-compose up
it starts the pg database
database system is ready to accept connections
and the nodejs server.
but the server can't connect to database.
Error: connect ECONNREFUSED 127.0.01:5432
If I try to run the server without using containers (with real nodejs and postgresd on my machine) it works.
But I want it to work correctly with containers. I don't understand what i'm doing wrong.
here is the docker-compose.yml file
web:
image: node
command: npm start
ports:
- "8000:4242"
links:
- db
working_dir: /src
environment:
SEQ_DB: mydatabase
SEQ_USER: username
SEQ_PW: pgpassword
PORT: 4242
DATABASE_URL: postgres://username:pgpassword#127.0.0.1:5432/mydatabase
volumes:
- ./:/src
db:
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: pgpassword
Could someone help me please?
(someone who likes docker :) )

Your DATABASE_URL refers to 127.0.0.1, which is the loopback adapter (more here). This means "connect to myself".
When running both applications (without using Docker) on the same host, they are both addressable on the same adapter (also known as localhost).
When running both applications in containers they are not both on localhost as before. Instead you need to point the web container to the db container's IP address on the docker0 adapter - which docker-compose sets for you.
Change:
127.0.0.1 to CONTAINER_NAME (e.g. db)
Example:
DATABASE_URL: postgres://username:pgpassword#127.0.0.1:5432/mydatabase
to
DATABASE_URL: postgres://username:pgpassword#db:5432/mydatabase
This works thanks to Docker links: the web container has a file (/etc/hosts) with a db entry pointing to the IP that the db container is on. This is the first place a system (in this case, the container) will look when trying to resolve hostnames.

For further readers, if you're using Docker desktop for Mac use host.docker.internal instead of localhost or 127.0.0.1 as it's suggested in the doc. I came across same connection refused... problem. Backend api-service couldn't connect to postgres using localhost/127.0.0.1. Below is my docker-compose.yml and environment variables as a reference:
version: "2"
services:
api:
container_name: "be"
image: <image_name>:latest
ports:
- "8000:8000"
environment:
DB_HOST: host.docker.internal
DB_USER: <your_user>
DB_PASS: <your_pass>
networks:
- mynw
db:
container_name: "psql"
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_DB: <your_postgres_db_name>
POSTGRES_USER: <your_postgres_user>
POSTGRES_PASS: <your_postgres_pass>
volumes:
- ~/dbdata:/var/lib/postgresql/data
networks:
- mynw

If you send database vars separately. You can assign a database host.
DB_HOST=<POSTGRES_SERVICE_NAME> #in your case "db" from docker-compose file.

I had two containers one called postgresdb, and another call node
I changed my node queries.js from:
const pool = new Pool({
user: 'postgres',
host: 'localhost',
database: 'users',
password: 'password',
port: 5432,
})
To
const pool = new Pool({
user: 'postgres',
host: 'postgresdb',
database: 'users',
password: 'password',
port: 5432,
})
All I had to do was change the host to my container name ["postgresdb"] and that fixed this for me. I'm sure this can be done better but I just learned docker compose / node.js stuff in the last 2 days.

If none of the other solutions worked for you, consider manual wrapping of PgPool.connect() with retry upon having ECONNREFUSED:
const pgPool = new Pool(pgConfig);
const pgPoolWrapper = {
async connect() {
for (let nRetry = 1; ; nRetry++) {
try {
const client = await pgPool.connect();
if (nRetry > 1) {
console.info('Now successfully connected to Postgres');
}
return client;
} catch (e) {
if (e.toString().includes('ECONNREFUSED') && nRetry < 5) {
console.info('ECONNREFUSED connecting to Postgres, ' +
'maybe container is not ready yet, will retry ' + nRetry);
// Wait 1 second
await new Promise(resolve => setTimeout(resolve, 1000));
} else {
throw e;
}
}
}
}
};
(See this issue in node-postgres for tracking.)

As mentioned here.
Each container can now look up the hostname web or db and get back the appropriate container’s IP address. For example, web’s application code could connect to the URL postgres://db:5432 and start using the Postgres database.
It is important to note the distinction between HOST_PORT and CONTAINER_PORT. In the above example, for db, the HOST_PORT is 8001 and the container port is 5432 (postgres default). Networked service-to-service communication uses the CONTAINER_PORT. When HOST_PORT is defined, the service is accessible outside the swarm as well.
Within the web container, your connection string to db would look like postgres://db:5432, and from the host machine, the connection string would look like postgres://{DOCKER_IP}:8001.
So DATABASE_URL should be postgres://username:pgpassword#db:5432/mydatabase

I am here with a tiny modification about handle this.
As Andy say in him response.
"you need to point the web container to the db container's"
And taking in consideration the official documentation about docker-compose link's
"Links are not required to enable services to communicate - by default, any service can reach any other service at that service’s name."
Because of that, you can keep your docker_compose.yml in this way:
docker_compose.yml
version: "3"
services:
web:
image: node
command: npm start
ports:
- "8000:4242"
# links:
# - db
working_dir: /src
environment:
SEQ_DB: mydatabase
SEQ_USER: username
SEQ_PW: pgpassword
PORT: 4242
# DATABASE_URL: postgres://username:pgpassword#127.0.0.1:5432/mydatabase
DATABASE_URL: "postgres://username:pgpassword#db:5432/mydatabase"
volumes:
- ./:/src
db:
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: pgpassword
But it is a kinda cool way to be verbose while we are coding. So, your approach it is nice.

Related

i'm trying to build a docker image of my strapi backend connecting with postgres. The images builds but running the container fails and i get error [duplicate]

I'm building an app running on NodeJS using postgresql.
I'm using SequelizeJS as ORM.
To avoid using real postgres daemon and having nodejs on my own device, i'm using containers with docker-compose.
when I run docker-compose up
it starts the pg database
database system is ready to accept connections
and the nodejs server.
but the server can't connect to database.
Error: connect ECONNREFUSED 127.0.01:5432
If I try to run the server without using containers (with real nodejs and postgresd on my machine) it works.
But I want it to work correctly with containers. I don't understand what i'm doing wrong.
here is the docker-compose.yml file
web:
image: node
command: npm start
ports:
- "8000:4242"
links:
- db
working_dir: /src
environment:
SEQ_DB: mydatabase
SEQ_USER: username
SEQ_PW: pgpassword
PORT: 4242
DATABASE_URL: postgres://username:pgpassword#127.0.0.1:5432/mydatabase
volumes:
- ./:/src
db:
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: pgpassword
Could someone help me please?
(someone who likes docker :) )
Your DATABASE_URL refers to 127.0.0.1, which is the loopback adapter (more here). This means "connect to myself".
When running both applications (without using Docker) on the same host, they are both addressable on the same adapter (also known as localhost).
When running both applications in containers they are not both on localhost as before. Instead you need to point the web container to the db container's IP address on the docker0 adapter - which docker-compose sets for you.
Change:
127.0.0.1 to CONTAINER_NAME (e.g. db)
Example:
DATABASE_URL: postgres://username:pgpassword#127.0.0.1:5432/mydatabase
to
DATABASE_URL: postgres://username:pgpassword#db:5432/mydatabase
This works thanks to Docker links: the web container has a file (/etc/hosts) with a db entry pointing to the IP that the db container is on. This is the first place a system (in this case, the container) will look when trying to resolve hostnames.
For further readers, if you're using Docker desktop for Mac use host.docker.internal instead of localhost or 127.0.0.1 as it's suggested in the doc. I came across same connection refused... problem. Backend api-service couldn't connect to postgres using localhost/127.0.0.1. Below is my docker-compose.yml and environment variables as a reference:
version: "2"
services:
api:
container_name: "be"
image: <image_name>:latest
ports:
- "8000:8000"
environment:
DB_HOST: host.docker.internal
DB_USER: <your_user>
DB_PASS: <your_pass>
networks:
- mynw
db:
container_name: "psql"
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_DB: <your_postgres_db_name>
POSTGRES_USER: <your_postgres_user>
POSTGRES_PASS: <your_postgres_pass>
volumes:
- ~/dbdata:/var/lib/postgresql/data
networks:
- mynw
If you send database vars separately. You can assign a database host.
DB_HOST=<POSTGRES_SERVICE_NAME> #in your case "db" from docker-compose file.
I had two containers one called postgresdb, and another call node
I changed my node queries.js from:
const pool = new Pool({
user: 'postgres',
host: 'localhost',
database: 'users',
password: 'password',
port: 5432,
})
To
const pool = new Pool({
user: 'postgres',
host: 'postgresdb',
database: 'users',
password: 'password',
port: 5432,
})
All I had to do was change the host to my container name ["postgresdb"] and that fixed this for me. I'm sure this can be done better but I just learned docker compose / node.js stuff in the last 2 days.
If none of the other solutions worked for you, consider manual wrapping of PgPool.connect() with retry upon having ECONNREFUSED:
const pgPool = new Pool(pgConfig);
const pgPoolWrapper = {
async connect() {
for (let nRetry = 1; ; nRetry++) {
try {
const client = await pgPool.connect();
if (nRetry > 1) {
console.info('Now successfully connected to Postgres');
}
return client;
} catch (e) {
if (e.toString().includes('ECONNREFUSED') && nRetry < 5) {
console.info('ECONNREFUSED connecting to Postgres, ' +
'maybe container is not ready yet, will retry ' + nRetry);
// Wait 1 second
await new Promise(resolve => setTimeout(resolve, 1000));
} else {
throw e;
}
}
}
}
};
(See this issue in node-postgres for tracking.)
As mentioned here.
Each container can now look up the hostname web or db and get back the appropriate container’s IP address. For example, web’s application code could connect to the URL postgres://db:5432 and start using the Postgres database.
It is important to note the distinction between HOST_PORT and CONTAINER_PORT. In the above example, for db, the HOST_PORT is 8001 and the container port is 5432 (postgres default). Networked service-to-service communication uses the CONTAINER_PORT. When HOST_PORT is defined, the service is accessible outside the swarm as well.
Within the web container, your connection string to db would look like postgres://db:5432, and from the host machine, the connection string would look like postgres://{DOCKER_IP}:8001.
So DATABASE_URL should be postgres://username:pgpassword#db:5432/mydatabase
I am here with a tiny modification about handle this.
As Andy say in him response.
"you need to point the web container to the db container's"
And taking in consideration the official documentation about docker-compose link's
"Links are not required to enable services to communicate - by default, any service can reach any other service at that service’s name."
Because of that, you can keep your docker_compose.yml in this way:
docker_compose.yml
version: "3"
services:
web:
image: node
command: npm start
ports:
- "8000:4242"
# links:
# - db
working_dir: /src
environment:
SEQ_DB: mydatabase
SEQ_USER: username
SEQ_PW: pgpassword
PORT: 4242
# DATABASE_URL: postgres://username:pgpassword#127.0.0.1:5432/mydatabase
DATABASE_URL: "postgres://username:pgpassword#db:5432/mydatabase"
volumes:
- ./:/src
db:
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: pgpassword
But it is a kinda cool way to be verbose while we are coding. So, your approach it is nice.

Docker container getting connection refused from postgres container in docker-compose

I've been beating my head against this for a few days now and I'm finally asking for help after trying to find the solution myself from all over.
I have a docker-compose file that looks like this:
services:
db:
image: ...
container_name: db
ports:
- "8095:5432"
networks:
- mynetwork
springservice:
image: ...
container_name: springservice
depends_on:
- db
ports:
- "8090:8090"
networks:
- mynetwork
environment:
- SPRING_DATASOURCE_URL: jdbc:postgresql://db:8095/dbname
- SPRING_DATASOURCE_USER: user
- SPRING_DATASOURCE_PASSWORD: password
networks:
mynetwork:
driver: bridge
name: mynetwork
Postgres has to be put to another port because we've got 3 postgres containers in that compose, so each get their own port.
Postgres's listen_address is set to "*".
pg_hba is set with "host all 0.0.0.0/0 md5"
Both containers come up, but when I curl from the service container to http://db:8095/ , I get connection refused.
What am I missing here?
Your port mapping is meaningless inside the docker network. This is only a mapping to the host system. Inside the network, the container is always available on its native port.
- SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/dbname
Also note that you don't need to publish the port to access it from inside the network. Doing so for a database can impose security risks. If you can, you should not publish it. That way, it will be only accessible from inside the docker network.

Handling multiple Postgresql Instances on same port between Local & Docker container

I have an issue where my docker container is delivering an error message, database_1 | 2021-05-03 23:33:49.552 UTC [33] FATAL: role "myname" does not exist, for the postgres container I am running and I'm under the impression that it is possibly tied to the fact that its running on the same port as my postgres instance that runs locally on my computer as a background service. Not completely certain, but it seems strange as the role (or I assume username) is present when I connect to a database with my local instance running. Is there something that I can do to further debug? When I run a local node server for the application the credentials work without any issue.
Here is my docker-compose.yml setup:
version: "3.9"
services:
redis:
image: redis:alpine
database:
image: postgres
ports:
- "5432:5432"
environment:
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: ${DB_DATABASE}
volumes:
- nextjs_auth_template:/var/lib/postgresql/data/ # persist data even if container shuts down
app:
image: nextjs-auth-boilerplate
build: .
depends_on:
- redis
- database
ports:
- "3000:3000"
environment:
- REDIS_HOST=redis
- DB_HOSTNAME=database
volumes:
nextjs_auth_template:
Here is my .env file:
DB_USERNAME=myname
DB_PASSWORD=''
DB_DATABASE=nextjs_auth_template
DB_HOSTNAME=127.0.0.1
DB_URL=postgresql://myname#127.0.0.1/nextjs_auth_template
If you are running both database instances on the same port and the same host, the problem must be that there is something that tries to connect as that user. If you are running the instance where the user exists, everything works. If you are running the other instance, you'll receive the error.
You'll have to figure out from where the client connects. For that, add %h to log_line_prefix to get the client IP address logged (change the parameter in the postgresql.conf file and reload the server). If you get no IP address in the log, that means that the connection is from your local computer.

Connecting to Postgres Docker server - authentication failed

I have a PostgreSQL container set up that I can successfully connect to with Adminer but I'm getting an authentication error when trying to connect via something like DBeaver using the same credentials.
I have tried exposing port 5432 in the Dockerfile and can see on Windows for docker the port being correctly binded. I'm guessing that because it is an authentication error that the issue isn't that the server can not be seen but with the username or password?
Docker Compose file and Dockerfile look like this.
version: "3.7"
services:
db:
build: ./postgresql
image: postgresql
container_name: postgresql
restart: always
environment:
- POSTGRES_DB=trac
- POSTGRES_USER=user
- POSTGRES_PASSWORD=1234
ports:
- 5432:5432
adminer:
image: adminer
restart: always
ports:
- 8080:8080
nginx:
build: ./nginx
image: nginx_db
container_name: nginx_db
restart: always
ports:
- "8004:8004"
- "8005:8005"
Dockerfile: (Dockerfile will later be used to copy ssl certs and keys)
FROM postgres:9.6
EXPOSE 5432
Wondering if there is something else I should be doing to enable this to work via some other utility?
Any help would be great.
Thanks in advance.
Update:
Tried accessing the database through the IP of the postgresql container 172.28.0.3 but the connection times out which suggests that PostgreSQL is correctly listening on 0.0.0.0:5432 and for some reason the user and password are not usable outside of Docker even from the host machine using localhost.
Check your pg_hba.conf file in the Postgres data folder.
The default configuration is that you can only login from localhost (which I assume Adminer is doing) but not from external IPs.
In order to allow access from all external addresses vi password authentication, add the following line to your pg_hba.conf:
host all all * md5
Then you can connect to your postgres DB running in the docker container from outside, given you expose the Port (5432)
Use the command docker container inspect ${container_number}, this will tell you which IPaddress:ports are exposed external to the container.
The command 'docker container ls' will help identify the 'container number'
After updating my default db_name, I also had to update the docker-compose myself by explicitly exposing the ports as the OP did
db:
image: postgres:13-alpine
volumes:
- dev-db-data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=devdb
- POSTGRES_USER=user
- POSTGRES_PASSWORD=1234
ports:
- 5432:5432
But the key here was restarting the server! DBeaver has connected to localhost:5432 :)

Docker Compose + Postgres: Expose port

I am currently trying to use Docker for my new Django/Postgres project. I am working on a Mac and usually use Postico to quickly connect to my database.
I used to connect like here:
I used the official Docker documentation to setup docker-compose. I now have the issue, that I can't connect via Postico to the postgres db. It seems to me that the problem comes from the ports not being exposed.
version: '3'
services:
db:
image: postgres
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Just map the port to the host machine, add this to the db service in your Compose file:
ports:
- "5432:5432"
Also make sure to set the postgres password variable in the compose file like this
environment:
POSTGRES_PASSWORD: example
The default user is postgres, you can change it with the POSTGRES_USER variable.
You can read about the usage of the image with all options here: https://hub.docker.com/_/postgres/
By default Compose sets up a single network for your app.
Each container can be accessed by the name of the service in the compose file.
In your case you don't have to expose the port to the host machine for your web app to have access to it. You can simply use db as the hostname for postgres (and 5432 for the port) from any other service running on the same compose.
Actually a very similar example is provided in the docker compose documentation:
https://docs.docker.com/compose/networking/