How to fix "Error: getaddrinfo ENOTFOUND redis" on docker? - docker-compose

I am using Redis using NestJS and I see following error. I am going through different articles like here and looks like I am following the same but still getting this error.
Steps:
I used docker compose up command
Made sure that host in redis.module.ts is same as service name in docker-compose.yml which is redis.
What am I missing here?
Error:
Error: getaddrinfo ENOTFOUND redis
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:71:26)
Code:
redis.module.ts
import { CacheModule, Module } from '#nestjs/common';
import { ConfigModule, ConfigService } from '#nestjs/config';
import { RedisService } from './redis.service';
import * as redisStore from 'cache-manager-redis-store';
import { envVariables } from '../env.variables';
#Module({
imports: [
CacheModule.registerAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: async (configService: ConfigService) => ({
store: redisStore,
host: process.env.REDIS_HOST,
port: configService.get('REDIS_PORT'),
ttl: configService.get('CACHE_TTL'),
max: configService.get('MAX_ITEM_IN_CACHE'),
}),
}),
],
providers: [RedisService],
exports: [RedisService],
})
export class RedisModule {}
.env
#REDIS
REDIS_HOST=redis
docker-compose.yml
version: "3.8"
services:
partnersusers:
image: partnersusers
build:
context: .
dockerfile: ./Dockerfile
environment:
- RUN_ENV=dev
- NODE_ENV=development
ports:
- "4000:4000"
networks:
- default
redis:
image: 'redis:alpine'
ports:
- "6379:4000"
networks:
default:
driver: bridge
Error in Docker:

I'm not an expert, but I notice a couple of things on your docker-compose.yml file.
First your redis service is missing the network assignation:
networks:
- default
Without this, redis-commander won't be able to find it as it's not on the same network.
Second redis by default runs on port: 6379 if you want it to run on port 4000 I believe you will need to specify an env var for it.
Or here maybe you just confused the order of the port matching which should've been: 4000:6379 (host_port:container_port).
this is my working docker-compose.yml for reference:
---
version: '3.8'
services:
...
redis:
image: redis
container_name: redis
hostname: redis
environment:
- ALLOW_EMPTY_PASSWORD=yes
ports:
- '6379:6379'
networks:
- my-net
redis-commander:
depends_on:
- redis
container_name: redis-commander
hostname: redis-commander
image: rediscommander/redis-commander:latest
restart: always
environment:
- REDIS_HOSTS=local:redis:6379 # note: this has to be the port the redis container exposes.
ports:
- "8081:8081"
networks:
- my-net
networks:
my-net:
Hope this helps :)

Related

Multiple Mongo Container

I wrote the following docker-compose file:
version: '3.7'
services:
mongo-db-showcase-db:
image: mongo:latest
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: '${DB_USER}'
MONGO_INITDB_ROOT_PASSWORD: '${DB_PASSWORD}'
ports:
- '27018:27017'
frontend-showcase:
build:
context: showcase
dockerfile: Dockerfile-frontend-showcase
image: showcase-frontend-showcase-image:latest
ports:
- '80:80'
backend-showcase:
build:
context: showcase
dockerfile: Dockerfile-backend-showcase
image: showcase-backend-showcase-image:latest
environment:
DATABASE_URL: 'mongodb://${DB_USER}:${DB_PASSWORD}#mongo-db-showcase-db:27018/'
ports:
- '3000:3000'
links:
- mongo-db-showcase-db
mongo-db-admin-manager-db:
image: mongo:latest
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: '${DB_USER}'
MONGO_INITDB_ROOT_PASSWORD: '${DB_PASSWORD}'
ports:
- '27017:27017'
frontend-manager:
build:
context: manager-app
dockerfile: Dockerfile-frontend-manager
image: manager-frontend-manager-image:latest
ports:
- '8080:80'
backend-manager:
build:
context: manager-app
dockerfile: Dockerfile-backend-manager
image: manager-backend-manager-image:latest
environment:
DATABASE_URL: 'mongodb://${DB_USER}:${DB_PASSWORD}#mongo-db-admin-manager-db:27017/'
ports:
- '8000:8000'
links:
- mongo-db-admin-manager-db
Here I have two mongo db container, one reachable to port 27017 the other should be reachable to port 27018. The service connecting to the db into port 27017 connect fine, the other doesn't connect. The error I get is:
MongooseServerSelectionError: connect ECONNREFUSED 172.20.0.6:27018
at NativeConnection.Connection.openUri (/app/node_modules/mongoose/lib/connection.js:807:32)
at /app/node_modules/mongoose/lib/index.js:342:10
at /app/node_modules/mongoose/lib/helpers/promiseOrCallback.js:32:5
at new Promise (<anonymous>)
at promiseOrCallback (/app/node_modules/mongoose/lib/helpers/promiseOrCallback.js:31:10)
at Mongoose._promiseOrCallback (/app/node_modules/mongoose/lib/index.js:1176:10)
at Mongoose.connect (/app/node_modules/mongoose/lib/index.js:341:20)
at Object.<anonymous> (/app/backend/app.js:26:10)
at Module._compile (internal/modules/cjs/loader.js:999:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) {
reason: TopologyDescription {
type: 'Unknown',
servers: Map { 'mongo-db-showcase-db:27018' => [ServerDescription] },
stale: false,
compatible: true,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
logicalSessionTimeoutMinutes: undefined
}
}
172.20.0.6 is the address of the container.
What could be the problem? Any suggestion is really appreciated.
Thanks in advance to anyone that would want to answer.
Your connection string port should be 27017 because all your container running inside a internal network, 27018 is the port you forwarded and can only connect from outside.
DATABASE_URL: 'mongodb://${DB_USER}:${DB_PASSWORD}#mongo-db-showcase-db:27017/'

Broken DAG: [/usr/local/airflow/dags/my_dag.py] No module named 'airflow.operators.subdag'

I'm running airflow inside docker container and getting airflow image (puckel/docker-airflow:latest) from docker hub. I can access Airflow UI through localhost:8080 but without executing the DAG and the error mentioned in the subject above. I'm even writing pip command to install apache-airflow in my Dockerfile. Here is how my Dockerfile, docker-compose.yml and dag.py looks like:
Dockerfile:
FROM puckel/docker-airflow:latest
RUN pip install requests
RUN pip install pandas
RUN pip install 'apache-airflow'
docker-compose.yml:
version: '3.7'
services:
redis:
image: redis:5.0.5
environment:
REDIS_HOST: redis
REDIS_PORT: 6379
ports:
- 6379:6379
postgres:
image: postgres:9.6
environment:
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=airflow
- POSTGRES_DB=airflow
- PGDATA=/var/lib/postgresql/data/pgdata
volumes:
- ./pgdata:/var/lib/postgresql/data/pgdata
logging:
options:
max-size: 10m
max-file: "3"
webserver:
build: ./dockerfiles
restart: always
depends_on:
- postgres
- redis
environment:
- LOAD_EX=n
- FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
- EXECUTOR=Celery
logging:
options:
max-size: 10m
max-file: "3"
volumes:
- ./dags:/usr/local/airflow/dags
- ./config/airflow.cfg:/usr/local/airflow/airflow.cfg
ports:
- "8080:8080"
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3
flower:
build: ./dockerfiles
restart: always
depends_on:
- redis
environment:
- EXECUTOR=Celery
ports:
- "5555:5555"
command: flower
scheduler:
build: ./dockerfiles
restart: always
depends_on:
- webserver
volumes:
- ./dags:/usr/local/airflow/dags
environment:
- LOAD_EX=n
- FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
- EXECUTOR=Celery
command: scheduler
worker:
build: ./dockerfiles
restart: always
depends_on:
- scheduler
volumes:
- ./dags:/usr/local/airflow/dags
environment:
- FERNET_KEY=46BKJoQYlPPOexq0OhDZnIlNepKFf87WFwLbfzqDDho=
- EXECUTOR=Celery
command: worker
dag.py:
from airflow import DAG
from airflow.operators.subdag import SubDagOperator
from airflow.operators.python import PythonOperator, BranchPythonOperator
from airflow.operators.bash import BashOperator
from datetime import datetime
from random import randint
def _choosing_best_model(ti):
accuracies = ti.xcom_pull(task_ids=[
'training_model_A',
'training_model_B',
'training_model_C'
])
if max(accuracies) > 8:
return 'accurate'
return 'inaccurate'
def _training_model(model):
return randint(1, 10)
with DAG("test",
start_date=datetime(2021, 1 ,1),
schedule_interval='#daily',
catchup=False) as dag:
training_model_tasks = [
PythonOperator(
task_id=f"training_model_{model_id}",
python_callable=_training_model,
op_kwargs={
"model": model_id
}
) for model_id in ['A', 'B', 'C']
]
choosing_best_model = BranchPythonOperator(
task_id="choosing_best_model",
python_callable=_choosing_best_model
)
accurate = BashOperator(
task_id="accurate",
bash_command="echo 'accurate'"
)
inaccurate = BashOperator(
task_id="inaccurate",
bash_command=" echo 'inaccurate'"
)
training_model_tasks >> choosing_best_model >> [accurate, inaccurate]
Am I missing anything here? Please let me know if you can. Thanks :)

How to get Dapr Service to Service Invocation to work when running under docker-compose?

I am receiving the following error when trying to call a service using Dapr SDK.
System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:3500)
---> System.Net.Sockets.SocketException (111): Connection refused
Here is my docker-compose settings of the service I am trying to call:
quest-service:
image: ${DOCKER_REGISTRY-gamification}/quest-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
build:
context: .
dockerfile: Services/LW.Gamification.QuestService/Dockerfile
ports:
- "5110:80"
- "50010:50001"
quest-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Quest-Service",
"-app-port", "80",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- quest-service
network_mode: "service:quest-service"
And the settings for the caller:
player-service:
image: ${DOCKER_REGISTRY-gamification}/player-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
build:
context: .
dockerfile: Services/LW.Gamificaiton.PlayerService/Dockerfile
ports:
- "5109:80"
- "50009:50001"
player-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Player-Service",
"-app-port", "80",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- player-service
network_mode: "service:player-service"
And here is the code that is failing to work:
// demo service to service call
var httpClient = DaprClient.CreateInvokeHttpClient("Quest-Service");
var requestUri = $"api/v1/Quest";
var result = await httpClient.GetFromJsonAsync<IEnumerable<string>>(requestUri);
Note: Messaging is working fine. :-)
I am new to Dapr so I must be doing something silly wrong, maybe something to do with ports.. I just don't know!
From following this question :Dapr Client Docker Compose Issue
I managed to get this partly working using the following docker-compose config:
services:
placement:
image: "daprio/dapr"
command: ["./placement", "-port", "50000", "-log-level", "debug"]
ports:
- "50000:50000"
quest-service:
image: ${DOCKER_REGISTRY-gamification}/quest-service:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
- DAPR_GRPC_PORT=50010
build:
context: .
dockerfile: Services/LW.Gamification.QuestService/Dockerfile
ports:
- "5110:80"
- "50010:50010"
depends_on:
- placement
- rabbitmq
- redis
- seq
- zipkin
quest-service-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Quest-Service",
"-app-port", "80",
"-placement-host-address", "placement:50000",
"-dapr-grpc-port", "50010",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- quest-service
network_mode: "service:quest-service"
generatetraffic:
image: ${DOCKER_REGISTRY-gamification}/generatetraffic:${TAG:-latest}
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SeqServerUrl=http://seq
- DAPR_GRPC_PORT=50017
build:
context: .
dockerfile: Services/LW.Gamification.GenerateTraffic/Dockerfile
ports:
- "5117:80"
- "50017:50017"
depends_on:
- placement
- rabbitmq
- redis
- seq
- zipkin
generatetraffic-dapr:
image: "daprio/daprd:latest"
command: ["./daprd",
"-app-id", "Generate-Traffic",
"-app-port", "80",
"-placement-host-address", "placement:50000",
"-dapr-grpc-port", "50017",
"-components-path", "/Components",
"-config", "/Configuration/config.yaml"
]
volumes:
- "./Dapr/Components/:/Components"
- "./Dapr/Configuration/:/Configuration"
depends_on:
- generatetraffic
network_mode: "service:generatetraffic"
However I still have issues with some of the documented APIs not working!.
var httpClient = DaprClient.CreateInvokeHttpClient("Quest-Service");
var requestUri = $"api/v1/Quest";
var result = await httpClient.GetAsync(requestUri);
Still fails?

SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:5432 when using docker to use sequelize

I'm getting the following error when dockerizing a node postgres database using sequelize as an orm backend
Unhandled rejection SequelizeConnectionRefusedError: connect
ECONNREFUSED 127.0.0.1:5432 app_1 | at
connection.connect.err
(/home/app/node_modules/sequelize/lib/dialects/postgres/connection-manager.js:170:24)
These lines of code seems to be the culprit, docker should not be connecting these credentials as this is for my local.
if (process.env.NODE_ENV === "production") {
var sequelize = new Sequelize(process.env.DATABASE_URL);
} else {
// docker is looking at these credentials..... when it should not
var sequelize = new Sequelize("elifullstack", "eli", "", {
host: "127.0.0.1",
dialect: "postgres",
pool: {
max: 100,
min: 0,
idle: 200000,
// #note https://github.com/sequelize/sequelize/issues/8133#issuecomment-359993057
acquire: 1000000,
},
});
}
docker-compose.yml
# docker-compose.yml
version: "3"
services:
app:
build: ./server
depends_on:
- database
ports:
- 5000:5000
environment:
# database refers to the database server at the bottom called "database"
- PSQL_HOST=database
- PSQL_USER=postgres
- PORT=5000
- PSQL_NAME=elitypescript
command: npm run server
client:
build: ./client
image: react_client
links:
- app
working_dir: /home/node/app/client
volumes:
- ./:/home/node/app
ports:
- 3001:3001
command: npm run start
env_file:
- ./client/.env
database:
image: postgres:9.6.8-alpine
volumes:
- database:/var/lib/postgresql/data
ports:
- 3030:5432
volumes:
database:
./server/dockerFile
FROM node:10.6.0
COPY . /home/app
WORKDIR /home/app
COPY package.json ./
RUN npm install
EXPOSE 5000
I looked at other similar questions like the following, but it ultimately did not help solve the issue.
Docker - SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:3306
SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:3306
I solved it...
What i did was change this
host: "127.0.0.1",
to this
let sequelize;
if (process.env.NODE_ENV === "production") {
sequelize = new Sequelize(process.env.DATABASE_URL);
} else {
sequelize = new Sequelize(
process.env.POSTGRES_DB || "elitypescript",
process.env.POSTGRES_USER || "eli",
"",
{
host: process.env.PSQL_HOST || "localhost",
dialect: "postgres",
pool: {
max: 100,
min: 0,
idle: 200000,
// #note https://github.com/sequelize/sequelize/issues/8133#issuecomment-359993057
acquire: 1000000,
},
}
);
}
that way the host would be set to docker environment variable like this
PSQL_HOST: database
and that connects to
database:
image: postgres:9.6.8-alpine
volumes:
- database:/var/lib/postgresql/data
ports:
- 3030:5432
Edit
# docker-compose.yml
version: "3"
services:
app:
build: ./server
depends_on:
- database
ports:
- 5000:5000
environment:
PSQL_HOST: database
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_DB: ${POSTGRES_DB:-elitypescript}
command: npm run server
client:
build: ./client
image: react_client
links:
- app
working_dir: /home/node/app/client
volumes:
- ./:/home/node/app
ports:
- 3001:3001
command: npm run start
env_file:
- ./client/.env
database:
image: postgres:9.6.8-alpine
volumes:
- database:/var/lib/postgresql/data
ports:
- 3030:5432
volumes:
database:

mailcow + jwilder reverse proxy

I try to set up my own mailserver, Mailcow was recommended.
DNS-provider:
Cloudflare with
CNAME mail.examle.com => examle.com, proxied
Because it is proxies, I cannot use normal ports like mentioned in the docs. Therefore I have to setup some forwarding...
Router:
Fritzbox with port forwadring
2052 => 25
2053 => 465
8080 => 587
2082 => 143
2083 => 993
2086 => 110
2087 => 995
8880 => 4190
Docker:
I use jwilders reverse proxy and it's LE-companion, which works well with everything else I have hosted so far.
${DOCKERDIR}/docker-compose-js.yml
version: '3'
services:
proxy:
build: ./reverse_proxy
container_name: proxy
restart: always
ports:
- 80:80
- 443:443
volumes:
- ${DOCKERDIR}/reverse_proxy/certs:/etc/nginx/certs:ro
- ${DOCKERDIR}/reverse_proxy/vhost.d:/etc/nginx/vhost.d
- ${DOCKERDIR}/reverse_proxy/html:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
environment:
- PUID=33
- PGID=33
labels:
com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: ""
networks:
- proxy-tier
depends_on:
- le
le:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: le
volumes:
- ${DOCKERDIR}/reverse_proxy/certs:/etc/nginx/certs:rw
- ${DOCKERDIR}/reverse_proxy/vhost.d:/etc/nginx/vhost.d
- ${DOCKERDIR}/reverse_proxy/html:/usr/share/nginx/html
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- PUID=33
- PGID=33
- DEFAULT_EMAIL=*****
- NGINX_PROXY_CONTAINER=proxy
networks:
- proxy-tier
networks:
proxy-tier:
Then there is a (slightly) modified file for mailcow, just mentioning the changes
%{DOCKERDIR}/mailcow/docker-compose.yml
nginx-mailcow:
...
# ports:
# - "${HTTPS_BIND:-0.0.0.0}:${HTTPS_PORT:-443}:${HTTPS_PORT:-443}"
# - "${HTTP_BIND:-0.0.0.0}:${HTTP_PORT:-80}:${HTTP_PORT:-80}"
...
There seems to be no way to remove those ports from it's original docker-compose.yml despite it not being recommended.
For all other changes I got
${DOCKERDIR}/mailcow/docker-compose-override.yml
version: '2.1'
services:
nginx-mailcow:
networks:
proxy-tier:
environment:
- VIRTUAL_HOST=${MAILCOW_HOSTNAME},${ADDITIONAL_SAN}
- VIRTUAL_PORT=8080
- VIRTUAL_PROTO=http
- LETSENCRYPT_HOST=${MAILCOW_HOSTNAME},${ADDITIONAL_SAN}
volumes:
- ${DOCKERDIR}/reverse_proxy/certs/${MAILCOW_HOSTNAME}:/etc/ssl/mail/
- ${DOCKERDIR}/reverse_proxy/certs/dhparam.pem:/etc/ssl/mail/dhparams.pem:ro
ports:
dovecot-mailcow:
volumes:
- ${DOCKERDIR}/reverse_proxy/certs/${MAILCOW_HOSTNAME}:/etc/ssl/mail/
- ${DOCKERDIR}/reverse_proxy/certs/dhparam.pem:/etc/ssl/mail/dhparams.pem:ro
postfix-mailcow:
volumes:
- ${DOCKERDIR}/reverse_proxy/certs/${MAILCOW_HOSTNAME}:/etc/ssl/mail/
- ${DOCKERDIR}/reverse_proxy/certs/dhparam.pem:/etc/ssl/mail/dhparams.pem:ro
networks:
proxy-tier:
And finally the mailcow.conf (changes only)
${DOCKERDIR}/mailcow/mailcow.conf
MAILCOW_HOSTNAME=mail.example.com
HTTP_PORT=8080
#HTTP_BIND=0.0.0.0
HTTP_BIND=proxy
HTTPS_PORT=8443
#HTTPS_BIND=0.0.0.0
HTTPS_BIND=proxy
SKIP_LETS_ENCRYPT=y
When I try to connect to mail.example.com I get Error 526 Invalid SSL certificate.
Could someone pls show me where my config is wrong and how to change it so I get mailcow working?