This is my docker-compose file that runs when I do docker-compose up -d on mac. I am now trying this on windows, with docker-toolbox (as docker desktop isn't supported on my windows). I run my application on http://localhost:1337 and then the application needs to talk to inside this container. Works totally fine on mac.
version: '3.4'
services:
# Add a redis instance to which our app can connect. Quite simple.
redis-dev:
image: redis:5.0.5-alpine
ports:
- 6379:6379
# Add a postgres instance as our primary data store
postgres-dev:
image: postgres:11.5-alpine
environment:
- POSTGRES_DB=the-masjid-app
ports:
- 5432:5432
volumes:
# Here we specify that docker should keep postgres data,
# so the next time we start docker-compose,
# our data is intact.
- the-masjid-app-pgdata-dev:/var/lib/postgresql/data
# Add a postgres instance as our primary data store
postgres-test:
image: postgres:11.5-alpine
environment:
- POSTGRES_DB=the-masjid-app
ports:
- 5433:5432
# Here we can configure settings for the default network
networks:
default:
# Here we can configure settings for the postgres data volume where our data is kept.
volumes:
the-masjid-app-pgdata-dev:
Doing the same thing in Windows is giving me:
Error: Redis connection to localhost:6379 failed - connect
ECONNREFUSED 127.0.0.1:6379 at TCPConnectWrap.afterConnect [as
oncomplete] (net.js:1141:16)
Any ideas on how to fix?
Related
when running a docker compose in a pipeline I'm getting this error when the tests on the pipleine are making use of mycontainer's API.
panic: Get "http://localhost:8080/api/admin/folder": read tcp 127.0.0.1:60066->127.0.0.1:8080: read: connection reset by peer [recovered]
panic: Get "http://localhost:8080/api/admin/folder": read tcp 127.0.0.1:60066->127.0.0.1:8080: read: connection reset by peer
This is my docker copose file:
version: "3"
volumes:
postgres_vol:
driver: local
driver_opts:
o: size=1000m
device: tmpfs
type: tmpfs
networks:
mynetwork:
driver: bridge
services:
postgres:
image: postgres:14
container_name: postgres
restart: always
environment:
- POSTGRES_USER=xxx
- POSTGRES_PASSWORD=xxx
- POSTGRES_DB=newdatabase
volumes:
#- ./postgres-init-db.sql:/docker-entrypoint-initdb.d/postgres-init-db.sql
- "postgres_vol:/var/lib/postgresql/data"
ports:
- 5432:5432
networks:
- mynetwork
mycontainer:
image: myprivaterepo/mycontainer-image:1.0.0
container_name: mycontainer
restart: always
environment:
- DATABASE_HOST=postgres
- DATABASE_PORT=5432
- DATABASE_NAME=newdatabase
- DATABASE_USERNAME=xxx
- DATABASE_PASSWORD=xxx
- DATABASE_SSL=false
depends_on:
- postgres
ports:
- 8080:8080
networks:
- mynetwork
mycontainer is listening on port 8080 and locally everything works fine.
However, when I run the pipeline which is initiating this docker compose is where I'm getting the error.
Basically, I'm running some tests in the pipeline that make use of mycontainer API (http://localhost:8080/api/admin/folder).
If I run the docker compose locally and I reproduce the steps followed on my pipeline to make use of the API everything is working fine. I can comunicate locally with both containers through localhost.
Also, I tried using healthchecks on the containers and 127.0.0.1:8080:8080 on mycontainer & 127.0.0.1:5432:5432 in postgres (including 0.0.0.0:8080:8080 & 0.0.0.0:5342:5432 just in case).
Any idea about that?
I was able to reproduce your error in a pipeline.
Make sure that you are not catching anything (e.g the code that's interacting with your container's API).
You did not mention anything related to your pipeline but just in case, delete the catching on your pipeline.
This error is ONLY occurring on one of my 4 devices, and I am trying to debug it. This device is a Macbook pro with an Intel processor.
The database container (db service) spins up but doesn't create the database.
version: "3.7"
services:
db:
networks:
new:
aliases:
- database
restart: always
container_name: db
image: postgres:latest
ports:
- 5433:5432
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=user
- POSTGRES_DB=core
# - PGDATA=/tmp
volumes:
- ./pgdata:/var/lib/postgresql/data
migrate:
image: migrate/migrate
depends_on:
- db
networks:
- new
volumes:
- ./db/migrations:/migrations
command: ["-path", "/migrations", "-database", "postgres://user:password#database:5432/core?sslmode=disable", "up"]
links:
- db
web:
networks:
- new
build: .
ports:
- "8080:8080"
volumes:
- .:/server
links:
- db
depends_on:
- db
- redis
environment:
PORT: 8080
CONNECTION_STRING_DEV: db://user:password#db:5433/db
DSN: "db://user:password#db:5433/core"
redis:
networks:
- new
image: "redis"
ports:
- "6379:6379"
networks:
new:
The container stops at 2022-01-19 15:37:02.916 UTC [49] LOG: database system is ready to accept connections and never actually executes "CREATE DATABASE"
Because the database isn't created, my connected Go API isn't functioning properly. The docker-compose should be creating the database "core", spinning up the redis instance, and then spinning up the web service. Afterwards, I typically pull up the migrate container which makes my database migrations. All of my other devices (macOS, windows, and linux), function properly and bring up the database when docker-compose up web is run.
here is warning from postgresql docker image page:
Warning: scripts in /docker-entrypoint-initdb.d are only run if you start the container with a data directory that is empty; any pre-existing database will be left untouched on container startup. One common problem is that if one of your /docker-entrypoint-initdb.d scripts fails (which will cause the entrypoint script to exit) and your orchestrator restarts the container with the already initialized data directory, it will not continue on with your scripts.
so one of the reason that your host you are using has something in ./pgdata
also you they have pretty detailed documentation on how you can extend image or run something on startup - you can actually clean up everything on first startup.
https://hub.docker.com/_/postgres
First I have built an image using Dockerfile:
FROM openjdk:8-jdk-alpine
ARG JAR_FILE=target/*-SNAPSHOT.jar
ADD ${JAR_FILE} app.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/app.jar"]
as I have two docker-compose files one for production:
version: "3"
services:
app:
image: "demo:latest"
container_name: demo-production-api
restart: always
depends_on:
- "productiondb"
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://productiondb:5432/testdb
- SPRING_DATASOURCE_HIKARI_JDBC_URL=jdbc:postgresql://productiondb:5432/testdb
- SPRING_DATASOURCE_USER=tester
- SPRING_DATASOURCE_PASSWORD=test
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
ports:
- "8440:8443"
productiondb:
image: "postgres:latest"
container_name: productiondb
ports:
- "5430:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres-db-production:/usr/local/var/postgres
volumes:
postgres-db-production:
and one for develop:
version: "3"
services:
app:
image: "demo:latest"
container_name: demo-develop-api
restart: always
depends_on:
- "developdb"
environment:
- SPRING_DATASOURCE_URL=jdbc:postgresql://developdb:5432/testdb
- SPRING_DATASOURCE_HIKARI_JDBC_URL=jdbc:postgresql://developdb:5432/testdb
- SPRING_DATASOURCE_USER=tester
- SPRING_DATASOURCE_PASSWORD=test
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
ports:
- "8441:8443"
developdb:
image: "postgres:latest"
container_name: developdb
ports:
- "5431:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres-db-develop:/usr/local/var/postgres
volumes:
postgres-db-develop:
I build both images using:
docker-compose -p demo-production-api -f docker-compose.yml up -d && docker-compose -p demo-develop-api -f docker-compose-develop.yml up -d
Now I was able to build both environments demo-develop-api and demo-production-api as well, the Spring Boot application from demo-develop-api docker image runs using the command:
docker run -it demo-develop-api
The application runs but I keep getting this error:
Caused by: java.net.UnknownHostException: productiondb
The above error happened after changing the database host in the application.properties file from localhost to productiondb first I was getting the following:
org.postgresql.util.PSQLException: Connection to localhost:5432
refused. Check that the hostname and port are correct and that the
postmaster is accepting TCP/IP connections.
Why this issue occurring or what is the cause?
How to solve this kind of issue?
As far as I see it, the issue might be that you have binded port 5430 and 5431 to 5432 and you might be having the port set to 5432 in your application.resources file. Your application should be trying to connect to the database by using either port 5430 or 5431 for production and development respectively. Please check and try this. So, make a port change in the application.resources file.
So after a long time of debugging and trials, hopefully, this is is going to save people hours, it turned out that actually, the Spring Boot application inside the container was restarting runs and crashes without any errors, which made me more confused why it is not listening or opening a port. I even doubt it that it could be a firewall or something. So basically I just tried to get a shell from the container by doing:
docker exec -it <container id or image> sh
Note: Since I am using the image openjdk:8-jdk-alpine don't do below you will not get a shell:
docker exec -it <container id or image> bash
Then I tried to get a list of open ports by doing:
netstat -tulpn | grep ":8443"
The port 8443 was not listed, I thought it could be a problem with the java program not being running, tried to execute the spring boot which executed but without any errors and the shell itself was exiting which made me more confused.
Until I have found out that container was restartig because of Spring Boot was crashing. So I enabled verbose mode by adding the below properties to application.properties then rebuild the image again:
logging.level.org.springframework.web=DEBUG
logging.level.org.hibernate=DEBUG
So I retried the last above steps where I get a shell and execute the app.jar and it turned out that the database testdb did not exist.
UPDATE: So to sum up here how I modifed my project, I created two Spring Boot Profiles for my case one for develop application-develop.properties and one for production application-production.properties:
So inside the application-develop.properties I have it mapped to a develop postgres container host and port:
spring.datasource.url=jdbc:postgresql://developdb:5432/testdb
spring.datasource.hikari.jdbc-url=jdbc:postgresql://developdb:5432/testdb
spring.datasource.username=tester
spring.jpa.generate-ddl=true
spring.datasource.password=test
spring.jpa.database-platform=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL9Dialect
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
server.port=8443
And for application-production.properties:
spring.datasource.url=jdbc:postgresql://productiondb:5432/testdb
spring.datasource.hikari.jdbc-url=jdbc:postgresql://productiondb:5432/testdb
spring.datasource.username=tester
spring.jpa.generate-ddl=true
spring.datasource.password=test
spring.jpa.database-platform=postgres
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL9Dialect
spring.jpa.hibernate.ddl-auto=create-drop
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
server.port=8443
And in the docker-compose file for develop I just define the Spring Boot profile environment variable to:
environment:
- SPRING_PROFILES_ACTIVE=develop
And for production docker-compose file I define it as below:
environment:
- SPRING_PROFILES_ACTIVE=production
I'm stuck on a very strange docker problem that I've not encountered before. What I want to do is to use docker-compose to make my application available from the internet. It's currently running on a instance on DigitalOcean and I'm currently working with the following docker-compose.yml:
version: '2.2'
services:
mongodb:
image: mongo:3.4
volumes:
- ./mongo:/data/db
ports:
- "27017"
mongoadmin: # web UI for mongo
image: mongo-express
ports:
- "8081:8081"
links:
- "mongodb:mongo"
environment:
- ME_CONFIG_OPTIONS_EDITORTHEME=ambiance
- ME_CONFIG_BASICAUTH_USERNAME=user
- ME_CONFIG_BASICAUTH_PASSWORD=pass
app:
image: project/name:0.0.1
volumes:
- ./project:/usr/src/app
working_dir: /usr/src/app
links:
- "mongodb:mongodb"
environment:
- NODE_ENV=production
command: ["npm", "start"]
ports:
- "3000:3000"
Mongoadmin connects properly and is able to connect to the database, while the database itself cannot be connected to from outside the host.
The problem is that the app won't connect to the right address. It is a express server using mongoose to connect to the database. Before connecting I'm logging the url it will connect to. In my config.js I've listed mongodb://mongodb/project, but this is resolved to localhost thus resulting in MongoError: failed to connect to server [localhost:27017] on first connect. The name of the container is resolved, but not to the proper address.
I've tried to connect to the IP (in the 172.18.0.0 range) that docker addressed to the container, but that also resolved to localhost. I've looked into /etc/hosts but this does not show anything related to this. Furthermore, I'm baffled because the mongo-express container is able to connect.
I've tried changing the name of the container, thinking it might be block for some reason due to previous runs or something like that, but this did not resolve the issue
I've tried both explicit links and implicit using dockers internal DNS resolve, but both did not work.
When binding port 27017 to localhost it is able to connect, but because of security and easy configuration via environment variables, I rather have the mongodb instance not bound to localhost.
I've also tried to run this on my local machine and that works as expected, being that both mongoadmin and app are able to connect to the mongodb container. My localmachine runs Docker version 1.12.6, build 78d1802, while the VPS runs on Docker version 17.06.2-ce, build cec0b72, thus a newer version.
Could this be a newly introduced bug? Or am I missing something else? Any help would be appreciated.
Your docker-compose file seems not have linked the app and mongodb container.
You have this:
app:
image: project/name:0.0.1
volumes:
- ./project:/usr/src/app
working_dir: /usr/src/app
environment:
- NODE_ENV=production
command: ["npm", "start"]
ports:
- "3000:3000"
While I think it should be this:
app:
image: project/name:0.0.1
volumes:
- ./project:/usr/src/app
working_dir: /usr/src/app
links:
- "mongodb:mongodb"
environment:
- NODE_ENV=production
command: ["npm", "start"]
ports:
- "3000:3000"
I have a multiple container application, that is using the postgres image in docker-compose.yml file. Postgres container has volume on host machine for persistent storage.
When I run docker-compose up at first time all is fine, postgres creates db files in my host folder.
After it I need to shut down application temporarily with docker-compose down if I'll change code of web container.
When I run docker-compose up second time, postgres overwriting all db files, but I need that data not changes. How can I solve this issue?
My docker-compose.yml
version: '2'
services:
web:
build: ./web
command: python3 main.py
volumes:
- ./web:/app
ports:
- "80:80"
depends_on:
- db
- redis
links:
- db:db
- redis:redis
db:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD:0000
volumes:
- ./pgdb:/var/lib/postgresql/data
redis:
image: redis
ports:
- "6379:6379"
command: redis-server --appendonly yes
volumes:
- ./redisdb:/data
I solve this problem. It occurs probably because I changed permissions for pgdb directory with host root user. By default I couldn't open pgdb in host machine because owner is postgres user. I could be wrong but after I stopped to change the resolutions the problem was gone.