I'm running Postgres image in Docker on an M1 Mac with mapped ports "5432:5432". My app can connect to the DB from the host machine by calling localhost:5432. I'm now trying to run the app within Docker and I'm puzzled by the behavior I see.
This command works:
docker run --name api --add-host host.docker.internal:host-gateway -e DB_HOST=host.docker.internal -p 8000:8000
But when I try to replicate the same by putting the api within the docker-compose like this, it doesn't work:
services:
postgres:
image: postgres:14.2
ports:
- "5432:5432"
networks:
- my-network
api:
image: api
environment:
DB_HOST: host.docker.internal
extra_hosts:
- "host.docker.internal:host-gateway"
Connecting to the DB fails:
failed to connect to host=host.docker.internal user=postgres database=postgres: failed to receive message (unexpected EOF)
I've also tried to put the api container on the same my-network network as postgres, and changing the DB host to be the DB container:
api:
image: api
environment:
DB_HOST: postgres
networks:
- my-network
but that gives me a different error:
failed to connect to host=postgres user=postgres database=postgres: dial error (dial tcp 192.168.192.2:5432: connect: connection refused)
The DB is listening at IPv4 address "0.0.0.0", port 5432 and IPv6 address "::", port 5432. Why would the docker run command work but the other two not work?
As David figured out the issue in the comments, I would like to suggest wait-for-it for such an issue, Instant of waiting a bit then start manually again.
wait-for-it.sh usually located at entrypoint.sh like this
#!/bin/sh
# Wait fot the cassandra db to be ready
./wait-for.sh cassandra:9042 --timeout=0
--timeout=0 , will keep on waiting till cassandra is up and running.
And can be used directly in docker-compose.yaml like the following example :
version: "2"
services:
web:
build: .
ports:
- "80:8000"
depends_on:
- "cassandra"
command: ["./wait-for-it.sh", "cassandra:9042"]
db:
image: cassandra
For more information, You can check Control startup and shutdown order in Compose
Wait-for-it github
I want to use Typesense with Mongodb. To do so I first need to enable replicaset.
From what I read here and there, to achieve it with docker compose, it must be done in two steps, like this :
version: '3.8'
services:
mongodb:
image: mongo
container_name: 'mongodb'
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: example
MONGO_INITDB_DATABASE: testDb
volumes:
- ./mongodb-config/mongod.test.conf:/etc/mongod.conf:ro
- ./mongodb-config/keyfile:/test/keyfile
ports:
- '27017:27017'
expose:
- '27017'
entrypoint:
- bash
- -c
- |
chmod 400 /test/keyfile
chown 999:999 /test/keyfile
exec docker-entrypoint.sh $$#
command: mongod --replSet testRs --keyFile /test/keyfile --bind_ip localhost;
mongosetup:
image: mongo
depends_on:
- mongodb
restart: 'no'
entrypoint:
[
'bash',
'-c',
"sleep 10 && mongo admin --host mongodb:27017 -u root -p example --verbose --eval 'rs.initiate()'",
]
log from mongosetup :
MongoDB shell version v5.0.9
connecting to: mongodb://mongodb:27017/admin?compressors=disabled&gssapiServiceName=mongodb
{"t":{"$date":"2022-07-20T14:58:50.437Z"},"s":"D1", "c":"NETWORK", "id":20109, "ctx":"js","msg":"Creating new connection","attr":{"hostAndPort":"mongodb:27017"}}
{"t":{"$date":"2022-07-20T14:58:50.440Z"},"s":"D1", "c":"-", "id":23074, "ctx":"js","msg":"User assertion","attr":{"error":"InternalError: couldn't connect to server mongodb:27017, connection attempt failed: SocketException: Error connecting to mongodb:27017 (192.168.0.2:27017) :: caused by :: Connection refused","file":"src/mongo/scripting/mozjs/mongo.cpp","line":745}}
Error: couldn't connect to server mongodb:27017, connection attempt failed: SocketException: Error connecting to mongodb:27017 (192.168.0.2:27017) :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:372:17
#(connect):2:6
{"t":{"$date":"2022-07-20T14:58:50.441Z"},"s":"D1", "c":"-", "id":23074, "ctx":"js","msg":"User assertion","attr":{"error":"Location12513: connect failed","file":"src/mongo/shell/shell_utils.cpp","line":537}}
{"t":{"$date":"2022-07-20T14:58:50.441Z"},"s":"I", "c":"QUERY", "id":22787, "ctx":"js","msg":"MozJS GC heap stats","attr":{"phase":"prologue","total":4799817,"limit":0}}
{"t":{"$date":"2022-07-20T14:58:50.444Z"},"s":"I", "c":"QUERY", "id":22787, "ctx":"js","msg":"MozJS GC heap stats","attr":{"phase":"epilogue","total":153,"limit":0}}
{"t":{"$date":"2022-07-20T14:58:50.444Z"},"s":"D1", "c":"-", "id":23074, "ctx":"main","msg":"User assertion","attr":{"error":"Location12513: connect failed","file":"src/mongo/scripting/mozjs/proxyscope.cpp","line":310}}
exception: connect failed
exiting with code 1
There must be something wrong with my way to try to connect, because I can do that manually without any problem :
docker exec -it 0188651cfb4b5789e832226094dae48fb8efd92fc318ce7b0286312f9a3d81de bash
root#0188651cfb4b:/# mongosh -u root -p example
test> rs.initiate()
{
info2: 'no configuration specified. Using a default configuration for the set',
me: 'localhost:27017',
ok: 1
}
testRs [direct: other] test>
I must be missing something, and since I'm a beginner with docker & mongo, I'm stuck :(
Thanks for your help
--bind_ip localhost in the command of mongodb container tells mongo to listen on local interface only. Try to connect to it from mongosetup manually and you will face the same error.
Use --bind_ip_all option instead.
I'm running an app with mongo db with docker.
Here is the mongodb part of the docker-compose file.
mongodb:
restart: always
image: mongo:3.6.3
container_name: "mongodb"
environment:
- MONGO_DATA_DIR=/data/db
- MONGO_LOG_DIR=/dev/nul
- MONGO_INITDB_ROOT_USERNAME=MasterUser
- MONGO_INITDB_ROOT_PASSWORD=MasterPassword
ports:
- 27017:27017
when i start the apps with docker-compose i see that mongodb is not detecting the root username and password therefore i can't login to mongodb and create user for my applications.
Strange thing is that, when i create a compose file only for mongodb it runs and uses the credentials i gave (MasterUser/MasterPassword)
i can also see that master user password and username are set in ENV in the container
docker exec -it mongodb env | grep -i master
MONGO_INITDB_ROOT_USERNAME=MasterUser
MONGO_INITDB_ROOT_PASSWORD=MasterPassword
however they dont work to login to db
docker exec -it mongodb mongo -u MasterUser -p MasterPassword --authenticationDatabase admin
MongoDB shell version v3.6.3
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.3
2020-07-18T19:27:22.378+0000 E QUERY [thread1] Error: Authentication failed. :
#(auth):6:1
#(auth):1:2
exception: login failed
Trying to use the mgo.v2 package to connect to the mongodb server. I have started the server using:
mongod --auth
I am able to connect to the server using the terminal using:
$ mongo -u "username" -p "password" --authenticationDatabase "db"
But when I use:
mgo.Dial("mongodb://usernamer:password#127.0.0.1:27017/dbname")
It gives me an error saying {"error":"no reachable servers"}.
My docker-compose.yml file is as below
version: "2"
services:
todo:
build:
context: .
dockerfile: todo/Dockerfile
restart: always
volumes:
- .:/go/src/prac
container_name: todo
ports:
- 8800:8081
mongodb:
command: mongod --auth
container_name: mongodb
image: mongo:latest
ports:
- 27017:27017
The problem appears to be that you are trying to connect to 127.0.0.1. MongoDB is not in the same container, so this won't work.
mgo.Dial("mongodb://usernamer:password#127.0.0.1:27017/dbname")
You should instead be connecting to the MongoDB container you defined by using the name you chose.
mgo.Dial("mongodb://usernamer:password#mongodb:27017/dbname")
Docker Compose creates a network for your containers in which they can access each other using the names you have defined as hostnames. Note that you don't need to define ports for containers to reach each other; these are only needed to reach containers from outside Docker.
I have a simple program written in golang which connect to a mongodb instance thus:
func main() {
session, err := mgo.Dial("localhost:27017")
if err != nil {
panic(err)
}
defer session.Close()
session.SetMode(mgo.Monotonic, true)
ensureIndex(session)
}
I use docker compose to spin up two containers one to spin up my GOLANG API and the other to spin up mongo:
version: "3.3"
services:
api:
image: golang:latest
volumes:
- .:/go
working_dir: /go
environment:
- GOPATH=/go
command:
go run /go/src/main.go
ports:
- "8082:8000"
depends_on:
- db
networks:
- api-net
db:
build: .
expose:
- '27017'
container_name: 'mongo'
ports:
- "27017:27017"
networks:
- api-net
networks:
api-net:
driver: bridge
I use a Dockerfile in order to spin up mongo with a conf file that sets the instance to bind to all IPV4 and IPV6 networks:
FROM mongo:latest
COPY mongod.conf /etc/mongod.conf
CMD mongod --config /etc/mongod.conf##
and this is the mongod.conf file:
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0
When I run docker-compose.up this is what I get:
Building db
Step 1/3 : FROM mongo:latest
---> a0f922b3f0a1
Step 2/3 : COPY mongod.conf /etc/mongod.conf
---> Using cache
---> f270e718c11e
Step 3/3 : CMD mongod --config /etc/mongod.conf
---> Running in 89ffc2495a2a
Removing intermediate container 89ffc2495a2a
---> fe2677d53122
Successfully built fe2677d53122
Successfully tagged carsupermarket_db:latest
WARNING: Image for service db was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating mongo ... done
Creating carsupermarket_api_1 ... done
Attaching to mongo, carsupermarket_api_1
api_1 | panic: no reachable servers
api_1 |
api_1 | goroutine 1 [running]:
api_1 | main.main()
api_1 | /go/src/main.go:38 +0x291
api_1 | exit status 2
I've trawled Google and stackoverflow and the only thing I could manage to find which is vaguely related to my issue is:
mongod --bind_ip using docker-compose version 2
however, my docker-compose.yml file as it stands 'Should' on paper work.
Can someone please point me in the correct direction as to why my GOLANG code cannot find my mongodb instance.
I think that when you call mgo.Dial("localhost:27017") the localhost part refers to the localhost within the golang container, and not the host the the containers are running on.
Docker will resolved container names as host names, change the dial to mgo.Dial("mongo:27017") and it should work.
From the docker container networking bridge docs
User-defined bridges provide automatic DNS resolution between containers.
Containers on the default bridge network can only access each other by
IP addresses, unless you use the --link option, which is considered
legacy. On a user-defined bridge network, containers can resolve each
other by name or alias.
https://docs.docker.com/network/bridge/
version: "2"
services:
api:
image: golang:latest
volumes:
- .:/go
# Work dir should be defined in dockerfile
#working_dir: /go
environment:
- GOPATH=/go
command:
go run /go/src/main.go
ports:
- "8082:8000"
depends_on:
- db
db:
build: .
container_name: 'mongo'
# You should define some volumes here if you want the data to persist.
You should be able to connect from the golang container to the mongo container using inter container communication using the hostname: 'mongo'