I'm trying to dockerize a NodeJS / PostgreSQL app, but i can't run knex migrations, I'm getting the following error :
Error: connect ECONNREFUSED 172.18.0.2:15432
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1161:14)
Here is my docker-compose.yml :
version: "3"
services:
app:
build: .
depends_on:
- db
links:
- db
ports:
- "3000:3000"
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: users-microservice
DB_HOST: db
db:
image: postgres:10.4-alpine
expose:
- "5432"
ports:
- "15432:5432"
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: users-microservice
The Dockerfile for the 'app' service :
FROM node:10.1-alpine
EXPOSE 3000 9229 15432
COPY . /home/app
WORKDIR /home/app
RUN npm install
RUN npm install -g knex
CMD ./scripts/start.sh
and in the start.sh, the following command works :
until PGPASSWORD=$POSTGRES_PASSWORD psql -h "$DB_HOST" -U "$POSTGRES_USER" -c '\d'; do
>&2 echo "Postgres is unavailable - sleeping"
sleep 1
done
>&2 echo "Postgres is up - executing command"
So I can connect to postgres through the CLI, but knex can't, why is that ? Am I getting something wrong, I'm new to Docker ?
Best regards,
I assume you're trying to run a command inside the service app when you see this error message (but maybe you can specify)?
My guess then would be that you're trying to connect to db:15432 from within app. Note, that saying
ports:
- "15432:5432"
only makes sure that you can call the service from your host machine on port 15432. But if you want to call the service db from within app, you still have to use db:5432.
The answer from #the-bass saved me. I'm going to include my docker-compose.yml and knexfile.js files to hopefully make this clearer for people with this issue who find this post.
What I was doing wrong was that I was not setting the host environment variable correctly in my app container. Once I changed it to just use db (the name of my postgres container) and made sure to include the port in my knexfile, all worked.
knexfile.js
require("dotenv").config();
module.exports = {
development: {
client: "pg",
connection: {
host: process.env.DB_HOST,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_DATABASE,
port: 5432,
},
pool: {
min: 2,
max: 10,
},
migrations: {
directory: "./data/migrations",
},
},
production: {
client: "pg",
connection: {
host: process.env.DB_HOST,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD,
database: process.env.DB_DATABASE,
port: 5432,
},
pool: {
min: 2,
max: 10,
},
migrations: {
directory: "./data/migrations",
},
},
};
docker-compose.yml
version: "3.7"
services:
db:
image: postgres:12.2
restart: always
ports:
- 5432:5432
volumes:
- my-postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_USER=me
- POSTGRES_PASSWORD=password
- POSTGRES_DB=my-data
node-app:
image: node:12.16.2
command: sh -c "npm install && npm install -g knex && knex migrate:latest && npm start"
ports:
- 3001:3001
working_dir: /home/node/app
volumes:
- ./:/home/node/app
environment:
- NODE_ENV=development
- DB_HOST=db
- DB_USER=HG
- DB_PASSWORD=password
- DB_DATABASE=my-data
depends_on:
- db
links:
- db
volumes:
my-postgres-data:
Related
I am getting this issue on
`docker-compose up`
This is my configuration files:-
Dockerfile:
FROM python:3.9
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "uvicorn", "app.main:app","--host","0.0.0.0","--port","8000 " ]
docker-compose.yaml:
version: '3'
services:
api:
build: .
depends_on:
- postgres
ports:
- 8000:8000
volumes:
- ./:/usr/src/app:ro
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
# env_file:
# - ./.env
environment:
- DATABASE_HOST_NAME=postgres
- DATABASE_PORT=${DATABASE_PORT}
- DATABASE_NAME=${DATABASE_NAME}
- DATABASE_USERNAME=${DATABASE_USERNAME}
- DATABASE_PASSWORD=${DATABASE_PASSWORD}
- SECRET_KEY=${SECRET_KEY}
- ALGORITHM=${ALGORITHM}
- ACCESS_TOKEN_EXPIRE_MINUTES=${ACCESS_TOKEN_EXPIRE_MINUTES}
postgres:
image: postgres
environment:
- POSTGRES_USER=${DATABASE_USERNAME}
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=${DATABASE_NAME}
volumes:
- postgres-db:/var/lib/postgresql/data
volumes:
postgres-db:
When i ran docker-compose build and docker-compose up -d these two commands ran succesfully but when i hit the login endpoint, it's giving the error
fastapi-api-1 | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "postgres" (192.168.112.2), port 5432 failed: FATAL: password authentication failed for user "root"
My DATABASE_USERNAME is root
I don't know why I am not able to fetch data from my postgreSQL database. I always got the same error when I do :
docker logs web I got that :
django.db.utils.OperationalError: could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432? could not connect to server: Address not available
I tried a lot of things but without effects...
Here is my docker-compose :
version: '3.7'
services:
web:
container_name: web
build:
context: .
dockerfile: Dockerfile
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/usr/src/web/
ports:
- 8000:8000
- 3000:3000
- 5432:5432
stdin_open: true
depends_on:
- db
db:
container_name: db
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=mydb
- POSTGRES_HOST=localhost
volumes:
postgres_data:
And this is the dockerfile :
# pull official base image
FROM python:3.8.3-alpine
# set work directory
WORKDIR /usr/src/web
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update \
&& apk add postgresql-dev gcc python3-dev musl-dev
RUN apk add zlib-dev jpeg-dev gcc musl-dev
# install nodejs
RUN apk add --update nodejs nodejs-npm
# copy project
ADD . .
# install dependencies
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
in my settings.py I have this for the database :
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'mydb',
'USER': 'admin',
'PASSWORD': 'pass',
'HOST': 'localhost',
'PORT': '5432',
}
}
Could you help me please ?
I have absolutely no ideas how to solve that, I tried a lot of things but without any effects...
For instance, I tried to change in settings this parameter :
'HOST': 'localhost' to 'HOST':'db'
But I got that errors when I type that docker logs web :
django.db.utils.OperationalError: FATAL: password authentication failed for user "admin"
Thank you very much
have you tried to EXPOSE the port 5432 on the database container?
something like
db:
container_name: db
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_HOST_AUTH_METHOD=trust
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=mydb
- POSTGRES_HOST=localhost
expose:
- "5432"
or inside the Dockerfile EXPOSE 5432/tcp
here is a link where you can check out the difference between ports: and expose:
What is the difference between docker-compose ports vs expose
I've got the following docker-compose file and it serves up the application on port 80 fine.
version: '3'
services:
backend:
build: ./Django-Backend
command: gunicorn testing.wsgi:application --bind 0.0.0.0:8000 --log-level debug
expose:
- "8000"
volumes:
- static:/code/backend/static
env_file:
- ./.env.prod
nginx:
build: ./nginx
ports:
- 80:80
volumes:
- static:/static
depends_on:
- backend
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
volumes:
static:
postgres_data:
Once in there I can log into the admin and add an extra user which gets saved to the database as I can reload the page and the user is still there. Once I stop the backend docker container however that user is gone. Given Postgres is operating in a different container and I'm not bringing it down I'm unsure how stopping the backend container and restarting it is causing the data not to be available.
Thanks in advance.
EDIT:
I'm bringing up the docker container with the following command.
docker-compose -f docker-compose.prod.yml up -d
I'm bringing down the container by just using docker desktop and stopping the container.
I'm running DJANGO 3 for the backend and I've also tried adding a superuser in the terminal when the container is running:
# python manage.py createsuperuser
Username (leave blank to use 'root'): mikey
Email address:
Password:
Password (again):
This password is too common.
Bypass password validation and create user anyway? [y/N]: y
Superuser created successfully.
Which works and the user appears while the container is running. However, once again when I shut the container down via docker desktop and then restart it that user that was just created is gone.
FURTHER EDIT:
settings.py using dotenv "from dotenv import load_dotenv"
DATABASES = {
"default": {
"ENGINE": os.getenv("SQL_ENGINE"),
"NAME": os.getenv("SQL_DATABASE"),
"USER": os.getenv("SQL_USER"),
"PASSWORD": os.getenv("SQL_PASSWORD"),
"HOST": os.getenv("SQL_HOST"),
"PORT": os.getenv("SQL_PORT"),
}
}
with the .env.prod file having the following values:
DEBUG=0
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
SQL_ENGINE=django.db.backends.postgresql
SQL_DATABASE=postgres
SQL_USER=postgres
SQL_PASSWORD=postgres
SQL_HOST=db
SQL_PORT=5432
SOLUTION:
Read the comments to see the diagnosis by other legends but updated docker-compose file looks like this. Note the "depends_on" block.
version: '3'
services:
backend:
build: ./Django-Backend
command: gunicorn testing.wsgi:application --bind 0.0.0.0:8000 --log-level debug
expose:
- "8000"
volumes:
- static:/code/backend/static
env_file:
- ./.env.prod
depends_on:
- db
nginx:
build: ./nginx
ports:
- 80:80
volumes:
- static:/static
depends_on:
- backend
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
expose:
- "5432"
volumes:
static:
postgres_data:
FINAL EDIT:
Added the following code to my entrypoint.sh file to ensure Postgres is ready to accept connections by the backend container.
if [ "$DATABASE" = "postgres" ]
then
echo "Waiting for postgres..."
while ! nc -z $SQL_HOST $SQL_PORT; do
sleep 0.1
done
echo "PostgreSQL started"
fi
I'm trying to get a setup going with a webservice that consumes a postgres database. Should be simple to setup, but I'm getting errors. So, first thing I want to make sure is that the database I set up is actually there and running.
To test this I substitute the "consumer" or "client" for an alpine interactive shell like so:
version: '3'
services:
db:
image: postgres:10.1-alpine
container_name: db
expose:
- 5432
volumes:
- "dbdata:/var/lib/postgresql/data"
environment:
- POSTGES_USER=user
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=db
web:
image: alpine:latest
stdin_open: true
tty: true
entrypoint: /bin/sh
depends_on:
- db
volumes:
dbdata:
Then I run the following command to get into the interactive shell:
docker-compose run web
and the following command to get in the database:
apk --update add postgresql-client && rm -rf /var/cache/apk/*
psql -h db -U user db
I get a plain denial from postgresql:
psql: FATAL: password authentication failed for user "user"
Same error message for each combo of username/password/databasename I try. Not much helpful.
What am I doing wrong here?
You have a typo in your docker-compose file. You mispelled POSTGRES here:
POSTGES_USER=user
That means the user user isn't being created. If I correct that typo, so that I have:
version: '3'
services:
db:
image: postgres:10.1-alpine
expose:
- 5432
volumes:
- "dbdata:/var/lib/postgresql/data"
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=pass
- POSTGRES_DB=db
web:
image: alpine:latest
stdin_open: true
tty: true
entrypoint: /bin/sh
depends_on:
- db
volumes:
dbdata:
Start the environment:
docker-compose up -d
Attach to the web contained and install the postgresql client:
$ docker attach project_web_1
/ # apk add --update postgresql-client
Then I can connect without a problem:
/ # psql -h db -U user db
Password for user user:
psql (11.2, server 10.1)
Type "help" for help.
db=#
I am successfully able to connect the servicebot service to the postgresql running within the docker container but I want to connect the servicebot to the postgresql running in instance ie not inside docker container.
I have installed the postgresql successfully. I have set the environment variables related to postgrsql in the docker-compose.yml as bellow. How can I make the docker-compose.yml connect to the
docker-compose.yml
version: '2'
services:
servicebot:
image: servicebot/servicebot
environment:
POSTGRES_DB_PORT : "5432"
POSTGRES_DB_HOST : "localhost"
POSTGRES_DB_USER : "servicebot_user"
POSTGRES_DB_PASSWORD : "servicebot_pass"
POSTGRES_DB_NAME : "servicebot_user"
PORT : "3000"
volumes:
- upload-data:/usr/src/app/uploads
- environment-file:/usr/src/app/env
- db-data:/var/lib/postgresql/data
# links:
# - db
ports:
- "80:3000"
- "443:3001"
command: ["sh", "-c", "node /usr/src/app/bin/wait-for-it.js db 5432 && npm run-script start"]
volumes:
upload-data:
environment-file:
db-data:
Previously I had a service named db for postgresql and connected to it with links as you can see, I have commented that out now.
I am very new to postgresql and not able to figure out the right way. I have tried few ways but nothing came to my success.
Tried:
Adding extra_hosts to the ip if the instance
Adding host.docker.internal instead of localhost
Error Logs:
On docker logs servicename It does not show anything. The service
stops after 29 30 seconds.
Your problem is POSTGRES_DB_HOST pointing to "localhost", as "localhost" will be the container running your current service.
If you want to connect to a postgre running in your host (localhost) I think you can use this special value host.docker.internal.
Just a heads up that if you're running Docker on a MAC (macOS), you need to use: docker.for.mac.host.internal instead.
I think you should add more information about postgresql to docker-compose.yml file as following:
version: '2'
services:
db:
image: postgres:13
environment:
- POSTGRES_PASSWORD=servicebot_pass
- POSTGRES_USER=servicebot_user
- POSTGRES_DB=postgres
restart: always
volumes:
- ./postgresql:/var/lib/postgresql/data
servicebot:
image: servicebot/servicebot
depends_on:
- db
environment:
POSTGRES_DB_PORT : "5432"
POSTGRES_DB_HOST : "localhost"
POSTGRES_DB_USER : "servicebot_user"
POSTGRES_DB_PASSWORD : "servicebot_pass"
POSTGRES_DB_NAME : "servicebot_user"
PORT : "3000"
volumes:
- upload-data:/usr/src/app/uploads
- environment-file:/usr/src/app/env
- db-data:/var/lib/postgresql/data
# links:
# - db
ports:
- "80:3000"
- "443:3001"
command: ["sh", "-c", "node /usr/src/app/bin/wait-for-it.js db 5432 && npm run-script start"]
volumes:
upload-data:
environment-file:
db-data: