Docker compose: unable to connect to Postgres [closed] - postgresql

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 2 years ago.
Improve this question
I have Go as my API and PostgreSQL as my database.
I can run my backend using the docker container when executed in development environment. However, when I run my dockerfile and docker-compose. The database is not connecting to postgres.
Dockerfile
FROM golang:alpine
RUN mkdir /backend
ADD . /backend/
WORKDIR /backend
COPY go.mod .
COPY go.sum .
COPY .env .
RUN go mod download
RUN go build -o main .
EXPOSE 3002
CMD ["./main"]
docker-compose.yml
version: '3.3'
services:
postgres:
image: "postgres"
ports:
- "5432:5432"
env_file:
- .env
volumes:
- ./postgres/:/var/lib/postgresql/data/
restart: always
networks:
- "backend.network"
nginx:
image: nginx:latest
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
depends_on:
- backend
- postgres
ports:
- "8080:8080"
links:
- 'postgres'
- 'backend'
networks:
- "backend.network"
backend:
build: "."
ports:
- "3002:3002"
depends_on:
- postgres
links:
- postgres
restart: "always"
networks:
- "backend.network"
networks:
backend.network:
.env
DB_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_USER=postgres
POSTGRES_PASSWORD=pass
POSTGRES_NAME=db
Go connection
DB, err = gorm.Open("postgres", fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable", config.Config("DB_HOST"), port, config.Config("POSTGRES_USER"), config.Config("POSTGRES_PASSWORD"), config.Config("POSTGRES_NAME")))
if err != nil {
panic("failed to connect database")
}
Error
failed to connect databasedial tcp 127.0.0.1:5432: connect: connection refused
exit status 1
I really don't know what's wrong with my dockerfile.

As #blami pointed out, the DB_HOST should be "postgres". Also, you are passing the .env file to the postgres image but not to the backend one (it may be using default values). If it keeps failing please comment the error you get.
Finally I would recommend to use a builder image to compile your server and then copy it to a smaller image using alpine (if you want to go a step further you can build the final image from scratch), making the container size as lightweight as we can, just like this:
FROM golang:1.15-alpine as builder
COPY . /backend
WORKDIR /backend
RUN go build -o main -ldflags="-s -w" . # If you are building the image below from scratch specify the CGO_ENABLED=0 env var
-----
FROM alpine:3.12.1
COPY --from=builder /backend/main /usr/bin/
EXPOSE 3002
ENTRYPOINT ["/usr/bin/main"]
Dockerfile notes based on best practices article:
Try to specify images version whenever possible as the lastest one may introduce new bugs to your app. However, it's not strictly required.
Prefer COPY to ADD when the resource we are trying to copy is not an url.
If the purpose of the container is purely running a server it's recommended to use ENTRYPOINT instead of CMD, which will ignore any parameters passed when running the container (if you want to use "exec" command then CMD is preferred).
Go binary note: -ldflags="-s -w" is often used to reduce the binary size by stripping the debugging information (more info here).

as #blami said:
your backend is trying to connect to "localhost" as each of your containers will have its own "localhost".
youc can use your machine host ip, or your database container ip. you can find it using docker inspect <container_name>

Related

Can't connect with docker-compose to Postgres database

I'm trying to build a docker-compose file that will spin up my EF Core web api project, connecting to my Postgres database.
I'm having a hard time getting the EF project connecting to the database.
This is what I currently have for my docker-compose.yml:
version: '3.8'
services:
web:
container_name: 'mybackendcontainer'
image: 'myuser/mybackend:0.0.6'
build:
context: .
dockerfile: backend.dockerfile
ports:
- 8080:80
depends_on:
- postgres
networks:
- mybackend-network
postgres:
container_name: 'postgres'
image: 'postgres:latest'
environment:
- POSTGRES_USER=username
- POSTGRES_PASSWORD=MySuperSecurePassword!
- POSTGRES_DB=MyDatabase
networks:
- mybackend-network
expose:
- 5432
volumes:
- ./db-data/:/var/lib/postgresql/data/
pgadmin:
image: dpage/pgadmin4
ports:
- 15433:80
env_file:
- .env
depends_on:
- postgres
networks:
- mybackend-network
volumes:
- ./pgadmin-data/:/var/lib/pgadmin/
networks:
mybackend-network:
driver: bridge
And my web project docker file looks like this:
# Get base DSK Image from Microsoft
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /app
# Copy the CSPROJ file and restore any dependencies (via NUGET)
COPY *.csproj ./
RUN dotnet restore
# Copy the project files and build our release
COPY . ./
RUN dotnet publish -c Release -o out
# Generate runtime image - do not include the whole SDK to save image space
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
EXPOSE 80
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "MyBackend.dll"]
And my connection string looks like this:
User ID =bootcampdb;Password=MySuperSecurePassword!;Server=postgres;Port=5432;Database=MyDatabase; Integrated Security=true;Pooling=true;
Currently I have two problems:
I'm getting Npgsql.PostgresException (0x80004005): 57P03: the database system is starting up when I do docker-compose -up. I tried to add the healthcheck to my postgress db but that did not work. When I go to my Docker desktop app, and start my backend again, that message goes away and I get my second problem...
Secondly after the DB started it's saying: FATAL: password authentication failed for user "username". It looks like it's not creating my user for the database. I even changed not to use .env files but have the value in my docker-compose file, but its still not working. I've tried to do docker-compose down -v to ensure my volumes gets deleted.
Sorry these might be silly questions, I'm still new to containerization and trying to get this to work.
Any help will be appreciated!
Problem 1: Having depends_on only means that docker-compose will wait until your postgres container is started before it starts the web container. The postgres container needs some time to get ready to accept connections and if you attempt to connect before it's ready, you get the error you're seeing. You need to code your backend in a way that it'll wait until Postgres is ready by retrying the connection with a delay.
Problem 2: Postgres only creates the user and database if no database already exists. You probably have an existing database in ./db-data/ on the host. Try deleting ./db-data/ and Postgres should create the user and database using the environment variables you've set.

How to make sure docker-compose will not remove my volume with postgres data

I am running a simple django webapp with docker-compose. I define both a web service and a db service in a docker-compose.yml file:
version: "3.8"
services:
db:
image: postgres
volumes:
- postgres_data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- "8000:8000"
env_file:
- ./.env.dev
depends_on:
- db
volumes:
postgres_data:
I start the service by running:
docker-compose up -d
I can load some data in there with a custom django command that I wrote for my app. Everything is running fine (with data) on localhost:8000.
However, when I run
docker-compose down
(so without -v) and then again
docker-compose up -d
the database is empty again. The volume was not persisted. From what I read in the docker-compose docs and also in several posts here at SO, persisting the volume and reusing it when you start a new container should be the default behavior (which, if I understand it correctly, you can disable by using the --renew-anon-volumes flag).
However in my case, the volume is not persisted. Or maybe it is, but my data is gone.
By doing docker volume ls I can see that my volume (I'll use the name my_volume here) still exists after the docker-compose down command. However, the CreatedAt value has been changed. This makes me think it's a different volume with the same name, and my data is already gone, but I don't know how to confirm that.
This SO answer suggests to mount the volume on /var/lib/postgresql instead of /var/lib/postgresql/data. However, I've seen other resources (like this one) where the opposite is suggested. I've tried both, but neither option works.
Thanks for any advice.
It turns out that the Dockerfile of my app was using an entrypoint in which the following command was executed: python manage.py flush which clears all data in the database. As this gets executed every time the app container starts, it clears all data. It had nothing to do with docker-compose.

docker-compose succeed but server does not response when request

I have built a RESTful API web service using Flask framework, Redis as main database, MongoDB as a backup store and Celery as task queue to store data into MongoDB in background
Then I dockerize my application using docker-compose. Here is my docker-compose.yml:
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/app
redis:
image: "redis:alpine"
ports:
- "6379:6379"
mongo:
image: "mongo:3.6.5"
ports:
- "27017:27017"
environment:
MONGO_INITDB_DATABASE: syncapp
Here is my Dockerfile:
# base image
FROM python:3.5-alpine
MAINTAINER xhoix <145giakhang#gmail.com>
# copy just the requirements.txt first to leverage Docker cache
# install all dependencies for Python app
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
# install dependencies in requirements.txt
RUN pip install -r requirements.txt
# copy all content to work directory /app
COPY . /app
# specify the port number the container should expose
EXPOSE 5000
# run the application
CMD ["python", "/app/app.py"]
After run command docker-compose up, the app server, Redis and Mongo server just run well. But when I use Postman or curl to call the API, for example http://127.0.0.1:5000/sync/api/v1.0/users, which should return JSON format of all users, but the result is Could not get any response: There was an error connecting to http://127.0.0.1:5000/sync/api/v1.0/users.
I have no idea why this happens.
Thanks for any help and suggestion!
I found the cause of the issue:
After an hour debug, it turns out that I only need to change the app host to 0.0.0.0. Maybe when mapping port, docker default will be 0.0.0.0, since when I run command docker-compose ps, the PORTS column of each container has format 0.0.0.0:<port> -> <port>. I don't know this is the cause of the issue or not, but I did it and the problem is solved
If operating system Linux then use :
ifconfig -a
If operating system Windows then use :
ipconfig /all
Then check the interface like docker or something with virtualization, and use the ipv4 or inet
Or Just use the docker command:
docker network inspect bridge
Then use the gateway ip on IPAM

docker-compose mongodb phoenix, [error] failed to connect: ** (Mongo.Error) tcp connect: connection refused - :econnrefused

Hi I am getting this error when I try to run docker-compose up on my yml file.
This is my docker-compose.yml file
version: '3.6'
services:
phoenix:
# tell docker-compose which Dockerfile it needs to build
build:
context: .
dockerfile: Dockerfile.development
# map the port of phoenix to the local dev port
ports:
- 4000:4000
# mount the code folder inside the running container for easy development
volumes:
- . .
# make sure we start mongodb when we start this service
depends_on:
- db
db:
image: mongo:latest
volumes:
- ./data/db:/data/db
ports:
- 27017:27017
This is my Dockerfile:
# base image elixer to start with
FROM elixir:1.6
# install hex package manager
RUN mix local.hex --force
RUN mix local.rebar --force
# install the latest phoenix
RUN mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez --force
# create app folder
COPY . .
WORKDIR ./
# install dependencies
RUN mix deps.get
# run phoenix in *dev* mode on port 4000
CMD mix phx.server
Is this a problem with my dev.exs setup or something to do with the compatibility of docker and phoenix / docker and mongodb?
https://docs.docker.com/compose/compose-file/#depends_on explicitly says:
There are several things to be aware of when using depends_on:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started. If you need to wait for a service to be ready,
and advises you to implement the logic to wait for mongodb to spinup and be ready to accept connections by yourself: https://docs.docker.com/compose/startup-order/
In your case it could be something like:
CMD wait-for-db.sh && mix phx.server
where wait-for-db.sh can be as simple as
#!/bin/bash
until nc -z localhost 27017; do echo "waiting for db"; sleep 1; done
for which you need nc and wait-for-db.sh installed in the container.
There are plenty of other alternative tools to test if db container is listening on the target port.
UPDATE:
The network connection between containers is described at https://docs.docker.com/compose/networking/:
When you run docker-compose up, the following happens:
A network called myapp_default is created, where myapp is name of the directory where docker-compose.yml is stored.
A container is created using phoenix’s configuration. It joins the network myapp_default under the name phoenix.
A container is created using db’s configuration. It joins the network myapp_default under the name db.
Each container can now look up the hostname phoenix or db and get back the appropriate container’s IP address. For example, phoenix’s application code could connect to the URL mongodb://db:27017 and start using the Mongodb database.
It was an issue with my dev environment not connecting to the mongodb url specified in docker-compose. Instead of localhost, it should be db as named in my docker-compose.yml file
For clarity to dev env:
modify config/dev.exs to (replace with correct vars)
username: System.get_env("PGUSER"),
password: System.get_env("PGPASSWORD"),
database: System.get_env("PGDATABASE"),
hostname: System.get_env("PGHOST"),
port: System.get_env("PGPORT"),
create a dot env file on the root folder of your project (replace with relevant vars to the db service used)
PGUSER=some_user
PGPASSWORD=some_password
PGDATABASE=some_database
PGPORT=5432
PGHOST=db
Note that we have added port.
Host can be localhost but should be mongodb or db or even url when working on a docker-compose or server or k8s.
will update answer for prod config...

HapiJS web app inside docker container not responding

I utilize HapiJS via docker compose 2+
.env
NODE_VIEWS_PATH=../
NODE_PUBLIC_PATH=../
MONGODB_URI=mongodb://127.0.0.1:27017/mahrio
WEB_DOMAIN=http://127.0.0.1:6085
deep down somewhere I am setting the HapiJS stuff via these .env files, but for Docker I understand I need to do some changes.. no problem I made a docker specific version
docker.env
NODE_VIEWS_PATH=../
NODE_PUBLIC_PATH=../
MONGODB_URI=mongodb://mongo:27017/mahrio
WEB_DOMAIN=http://0.0.0.0:6085
I've tried 0.0.0.0 and 127.0.0.1 , neither work
I can see everything seems to work however when I goto localhost:6085 I get no response.
127.0.0.1 didn’t send any data.
Dockerfile
FROM node:carbon
# Create app directory
RUN mkdir -p /usr/src/mahrio
WORKDIR /usr/src/mahrio
COPY package*.json /usr/src/mahrio
RUN npm install
# If you are building your code for production
# RUN npm install --only=production
COPY . /usr/src/mahrio
EXPOSE 6085
CMD ["npm", "start"]
docker-compose.yml
version: "2"
services:
app:
build: .
container_name: mahrio
depends_on:
- mongo
env_file:
- docker.env
ports:
- "6085:6085"
restart: always
mongo:
container_name: mongo
image: mongo
volumes:
- ./tmp:/data/db
ports:
- "27017:27017"
any ideas ? No errors from nodejs are coming, everything looks A-OKAY at the console and I know it works outside docker just fine.
Edit: added the docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
25e7a4c3f350 mahriomedium_app "npm start" 24 hours ago Up About a minute 0.0.0.0:6085->6085/tcp mahrio
c8d691777aa0 mongo "docker-entrypoint..." 3 days ago Up About a minute 0.0.0.0:27017->27017/tcp mongo
docker logs
> mahrio-medium#0.0.1 start /usr/src/mahrio
> node server/index.js
Running Development!
MongoDB Config...
Server running at: http://127.0.0.1:6085
MongoDB connected!
db connection opened
it turned out WEB_DOMAIN was the wrong env var
the right var is NODE_URI to set to 0.0.0.0
all works now