Docker compose read connection reset by peer error on pipeline - postgresql

when running a docker compose in a pipeline I'm getting this error when the tests on the pipleine are making use of mycontainer's API.
panic: Get "http://localhost:8080/api/admin/folder": read tcp 127.0.0.1:60066->127.0.0.1:8080: read: connection reset by peer [recovered]
panic: Get "http://localhost:8080/api/admin/folder": read tcp 127.0.0.1:60066->127.0.0.1:8080: read: connection reset by peer
This is my docker copose file:
version: "3"
volumes:
postgres_vol:
driver: local
driver_opts:
o: size=1000m
device: tmpfs
type: tmpfs
networks:
mynetwork:
driver: bridge
services:
postgres:
image: postgres:14
container_name: postgres
restart: always
environment:
- POSTGRES_USER=xxx
- POSTGRES_PASSWORD=xxx
- POSTGRES_DB=newdatabase
volumes:
#- ./postgres-init-db.sql:/docker-entrypoint-initdb.d/postgres-init-db.sql
- "postgres_vol:/var/lib/postgresql/data"
ports:
- 5432:5432
networks:
- mynetwork
mycontainer:
image: myprivaterepo/mycontainer-image:1.0.0
container_name: mycontainer
restart: always
environment:
- DATABASE_HOST=postgres
- DATABASE_PORT=5432
- DATABASE_NAME=newdatabase
- DATABASE_USERNAME=xxx
- DATABASE_PASSWORD=xxx
- DATABASE_SSL=false
depends_on:
- postgres
ports:
- 8080:8080
networks:
- mynetwork
mycontainer is listening on port 8080 and locally everything works fine.
However, when I run the pipeline which is initiating this docker compose is where I'm getting the error.
Basically, I'm running some tests in the pipeline that make use of mycontainer API (http://localhost:8080/api/admin/folder).
If I run the docker compose locally and I reproduce the steps followed on my pipeline to make use of the API everything is working fine. I can comunicate locally with both containers through localhost.
Also, I tried using healthchecks on the containers and 127.0.0.1:8080:8080 on mycontainer & 127.0.0.1:5432:5432 in postgres (including 0.0.0.0:8080:8080 & 0.0.0.0:5342:5432 just in case).
Any idea about that?

I was able to reproduce your error in a pipeline.
Make sure that you are not catching anything (e.g the code that's interacting with your container's API).
You did not mention anything related to your pipeline but just in case, delete the catching on your pipeline.

Related

Access Postgres database remotely that's hosted on Azure in docker container with webapi

I am new to Azure cloud services so excuse me if this is a dumb question.
I have a docker-compose file with a .Net core webapi and postgres database. I have it running on Azure as a web-app and its working (I can see when I query the API that there's data in the database). However I would like to get access to the database remotely so that I can inspect and see the data in the database via pgAdmin or something similar.
I did bind a port to my pgAdmin site in my docker-compose but it does not seem like that port is open. I've read somewhere that only port 80 and 443 can be exposed from Azure web-apps when using multi-image containers. (This docker-compose works locally 100% and I can access the pgAdmin site and see the database with all its tables).
So my question is, how do I run my web-api with my postgres database on azure and have visibility to my database?
Docker-compose file:
version: '3.8'
services:
web:
container_name: 'bootcampapi'
image: 'myimage'
build:
context: .
dockerfile: backend.dockerfile
restart: always
ports:
- 8080:80
depends_on:
postgres:
condition: service_healthy
networks:
- bootcampbackend-network
postgres:
container_name: 'postgres'
restart: always
image: 'postgres:latest'
healthcheck:
test: ["CMD-SHELL", "pg_isready"]
interval: 10s
timeout: 5s
retries: 5
environment:
- POSTGRES_USER=myusername
- POSTGRES_PASSWORD=mypassword
- POSTGRES_DB=database-name
- PGDATA=database-data
networks:
- bootcampbackend-network
ports:
- 5432:5432
volumes:
- database-data:/var/lib/postgresql/data/
pgadmin:
image: dpage/pgadmin4
ports:
- 15433:80
env_file:
- .env
depends_on:
- postgres
networks:
- bootcampbackend-network
volumes:
- database-other:/var/lib/pgadmin/
networks:
bootcampbackend-network:
driver: bridge
As you have found, App Service only listens on one port. One solution around that is to use a reverse proxy like Nginx to route the traffic to both your containers.
BTW, build, depends_on and networks are unsupported. See doc

Docker-compose can't connect to Docker postgres container

My Postgres DB is running in a Docker container. When container is started, it says it's ready to listen on 5432.
My application container is set to depend on it.
container_name: my_postgres_db
image: library/postgres:latest
network_mode: bridge
expose:
- 5432
ports:
- 5432:5432
environment:
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=admin
- POSTGRES_DB=localdb
restart: always
The config for app:
container_name: my_test_app
depends_on:
- db
build:
context: ./
dockerfile: Dockerfile
image: my_test_app
ports:
- 8080:8080
Based on solutions to the similar questions, I changed the localhost in the DB URL to:
spring.datasource.url=jdbc:postgresql://db:5432/localdb
It causes another error = "Unknown host exception". Even if I manage to build app this way, it still doesn't work. Logs say,
Connection to localhost:5432 refused
What else am I missing?
Why is it still listening to localhost:5432 when I obviously changed it to db:5432 and gradlew clean/build it?
just change the network_mode in postgres and app to host
network_mode: host
note that this will ignore the expose option and will use the host network an containers network

Connecting two a database in a another container with docker-compose

I'm trying to set up a docker-compose where one container has the database and the other one has the application. To my best knowledge, I need to connect two containers with a network.
version: "3"
services:
psqldb:
image: "postgres"
container_name: "psqldb"
environment:
- POSTGRES_USER=usr
- POSTGRES_PASSWORD=pwd
- POSTGRES_DB=mydb
ports:
- "5432:5432"
expose:
- "5432"
volumes:
- $HOME/docker/volumes/postgres/:/var/lib/postgresql/data
networks:
- backend
sbapp:
image: "dsb"
container_name: "dsb-cont"
ports:
- "8080:8080"
expose:
- "8080"
depends_on:
- psqldb
networks:
- backend
networks:
backend:
I also tried it with setting network driver to bridge (which didn't change the end result)
networks:
backend:
driver: bridge
I'm using the following url to connect to the database
spring.datasource.url=jdbc:postgresql://psqldb:5432/mydb
I also tried to expose the port and connect using localhost
spring.datasource.url=jdbc:postgresql://localhost:5432/mydb
I also tried to use port 5433 instead of 5432.
No matter what combination I used, I got the following error
Connection to psqldb:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
in my app container. However, my database container remains up and I can connect to it fine from host with the url
spring.datasource.url=jdbc:postgresql://localhost:5432/mydb
I can also connect to the database from host if I remove psqldb container entirely from docker-compose.yml (and use the latter connection url).
If it makes any difference, I'm using Spring
Boot for application with the Dockerfile
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG JAR_FILE
COPY ${JAR_FILE} app.jar
EXPOSE 8080
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom", "-jar", "app.jar"]
and I'm building the image with the command
docker build --build-arg JAR_FILE=target/*.jar -t dsb .
What am I missing in the two container setup?
The issue I had was that docker depends_on only starts the containers in the defined order but doesn't wait for them to be actually ready.
https://docs.docker.com/compose/startup-order/

How to access postgres-docker container other docker container without ip address

How to access postgres-docker container other docker container without ip address?
I want to store data in postgres by using myweb. in jar given host like localhost:5432/db..
Here my compose file:
version: "3"
services:
myweb:
build: ./myweb
container_name: app
ports:
- "8080:8080"
- "9090:9090"
networks:
- front-tier
- back-tier
depends_on:
- "postgresdb"
postgresdb:
build: ./mydb
image: ppk:postgres9.5
volumes:
- dbdata:/var/lib/postgresql
ports:
- "5432:5432"
networks:
- back-tier
volumes:
dbdata: {}
networks:
front-tier:
back-tier:
Instead of localhost:5432/db.. use postgresdb:5432/db.. connection string.
By default the container has the same hostname as the service name.
Here is my minimal working example, which is connecting a java client (boxfuse/flyway) with postgres server. The most important part is the heath check, which is delaying the start of the myweb container to the time when postgres is ready to accept connections.
Note that this can be directly executed by docker-compose up, it dosen't have any other dependencies. Both the images are from docker hub.
version: '2.1'
services:
myweb:
image: boxfuse/flyway
command: -url=jdbc:postgresql://postgresdb/postgres -user=postgres -password=123 info
depends_on:
postgresdb:
condition: service_healthy
postgresdb:
image: postgres
environment:
- POSTGRES_PASSWORD=123
healthcheck:
test: "pg_isready -q -U postgres"
That is the Docker Networking problem. The solution is to use postgresdb:5432/db in place of localhost:5432/db because the two service is in the same network named back-tier and docker deamon will use name service like a DNS name to make communication between the two container. I think that my solution will help you so.

docker link resolves to localhost

I'm stuck on a very strange docker problem that I've not encountered before. What I want to do is to use docker-compose to make my application available from the internet. It's currently running on a instance on DigitalOcean and I'm currently working with the following docker-compose.yml:
version: '2.2'
services:
mongodb:
image: mongo:3.4
volumes:
- ./mongo:/data/db
ports:
- "27017"
mongoadmin: # web UI for mongo
image: mongo-express
ports:
- "8081:8081"
links:
- "mongodb:mongo"
environment:
- ME_CONFIG_OPTIONS_EDITORTHEME=ambiance
- ME_CONFIG_BASICAUTH_USERNAME=user
- ME_CONFIG_BASICAUTH_PASSWORD=pass
app:
image: project/name:0.0.1
volumes:
- ./project:/usr/src/app
working_dir: /usr/src/app
links:
- "mongodb:mongodb"
environment:
- NODE_ENV=production
command: ["npm", "start"]
ports:
- "3000:3000"
Mongoadmin connects properly and is able to connect to the database, while the database itself cannot be connected to from outside the host.
The problem is that the app won't connect to the right address. It is a express server using mongoose to connect to the database. Before connecting I'm logging the url it will connect to. In my config.js I've listed mongodb://mongodb/project, but this is resolved to localhost thus resulting in MongoError: failed to connect to server [localhost:27017] on first connect. The name of the container is resolved, but not to the proper address.
I've tried to connect to the IP (in the 172.18.0.0 range) that docker addressed to the container, but that also resolved to localhost. I've looked into /etc/hosts but this does not show anything related to this. Furthermore, I'm baffled because the mongo-express container is able to connect.
I've tried changing the name of the container, thinking it might be block for some reason due to previous runs or something like that, but this did not resolve the issue
I've tried both explicit links and implicit using dockers internal DNS resolve, but both did not work.
When binding port 27017 to localhost it is able to connect, but because of security and easy configuration via environment variables, I rather have the mongodb instance not bound to localhost.
I've also tried to run this on my local machine and that works as expected, being that both mongoadmin and app are able to connect to the mongodb container. My localmachine runs Docker version 1.12.6, build 78d1802, while the VPS runs on Docker version 17.06.2-ce, build cec0b72, thus a newer version.
Could this be a newly introduced bug? Or am I missing something else? Any help would be appreciated.
Your docker-compose file seems not have linked the app and mongodb container.
You have this:
app:
image: project/name:0.0.1
volumes:
- ./project:/usr/src/app
working_dir: /usr/src/app
environment:
- NODE_ENV=production
command: ["npm", "start"]
ports:
- "3000:3000"
While I think it should be this:
app:
image: project/name:0.0.1
volumes:
- ./project:/usr/src/app
working_dir: /usr/src/app
links:
- "mongodb:mongodb"
environment:
- NODE_ENV=production
command: ["npm", "start"]
ports:
- "3000:3000"