I'm working on a very basic (I thought) starter program in Go using MongoDB and Docker. Trying to get a handle on these before we start using them at work.
I've got my MongoDB running in a docker container, just using my local host, using the official Docker image. This is running fine, I can connect to it through MongoDB Compass and modify the DB.
My next task was to build a separate Docker container that is able to read and write to the DB. I'm using MongoDB-Go-Driver (https://godoc.org/github.com/mongodb/mongo-go-driver/mongo) for this as mgo is no longer kept up.
This is my code, I'm just following the numerous tutorials online to make a simple connection and then ping the DB to ensure connectivity.
client, err := mongo.Connect("mongodb://localhost:27017")
if err != nil {
log.Fatal("error ", err)
}
// Check the connection
err = client.Ping(context.TODO(), nil)
if err != nil {
log.Fatal("error2 ", err)
}
fmt.Println("Connected to MongoDB!")
It always fails on doing any operation on the DB (Find, FindOne, Ping, etc.) with error2 server selection timeout
This is my docker-compose file I'm running.
version: "3"
services:
datastore:
image: mongo
ports:
- "27017:27017"
networks:
- maccaptionNet
volumes:
- .:/go/src/maccaption_microservice/dbdata
jobservice:
image: jobservicemaccaption:1.0
networks:
- maccaptionNet
depends_on:
- "datastore"
networks:
maccaptionNet:
driver: bridge
I'm brand new to MongoDB and after hours of research haven't made any progress on this.
I've read through https://docs.mongodb.com/manual/core/read-preference-mechanics/
https://docs.mongodb.com/manual/replication/
Can anyone point me in the right direction for this? I haven't been able to find a lot on this specific issue.
Thanks!
When you running the service and mongodb in docker you can't use localhost since the service is in a different container than mongodb, and from docker point of view it's under a different ip address.
You can connect with the service name you specify in docker-compose datastore
mongo.Connect("mongodb://datastore:27017")
Edit:
from: https://docs.docker.com/compose/networking/
By default Compose sets up a single network for your app. Each
container for a service joins the default network and is both
reachable by other containers on that network, and discoverable by
them at a hostname identical to the container name
Meaning that if you run multiple containers via compose, you can access one container from the other by the container name,
Basically when docker-compose starts, it sets up the network, and each container in the compose joins the network under its container name. For a container's point if view, localhost is just the container itself, while he can search for other container's name and get back the container’s IP address.
Assuming that the docker is running on your localhost, you can set the name in etc/hosts file like this:
127.0.0.1 datastore
(if not just replace 127.0.0.1 with the docker ip)
And in the app you will connect with mongodb://datastore:27017
So you will be able to run the service both in the docker and from outside, if you'll decide to run only the db in docker
docker-compose start datastore
If you are connecting to one docker from another (like it is written in your docker-compose file, and using bridge network mode, you have to change your localhost to the hostname, like datastore
client, err := mongo.Connect("mongodb://datastore:27017")
When your go script uses localhost, it expects the database to located in the same docker
I think my answer might be unrelated but still, I was getting the same error and it was because my IP address was not listed in the IP whitelist tab in MongoDB atlas, so make sure you have your IP address there before trying to connect.
I had the same problem but found another way to address this issue. You can just pass network parameter while running docker image and this way docker points to correct localhost.
docker run --network="host" ....
Source for this solution
Somehow i've fix this problem in a different way: by changing ports from "27018:27017" to "27017:27017".
IDK why this helps. Maybe if Mongo sees not default port it thinks there are cluster of Mongo's nodes.
I got this problem when I tired to connect to
mongodb v4.0.10
with
pymongo==4.0.2 not worked
pymongo==3.12.3 worked
Check your packages
mongodb v5.0.2 works with pymongo==4.0.2
Related
On a virtual machine I have 2 docker containers running with the names <postgres> and <system> that run on the network with name <network>. I can't change options in these containers. I have created a flask application that connects to a database and outputs the required information. To connect from local computer I use
conn = psycopg2.connect(
database="db", user='user1', password='user1_passwd'
host='<VM_ip>', port='<db_port>',
sslmode='require',
sslcert='./user1.crt',
sslkey='./user1.key')
and it worked great.
But, when I run my application on the same VM and specify
conn = psycopg2.connect(
database="db", user='user1', password='user1_passwd'
host='<postgres>.<network>', port='<db_port>',
sslmode='require',
sslcert='./user1.crt',
sslkey='./user1.key')
I get an error:
psycopg2.OperationalError: could not parse network address \"<postgres>.<network>\": Name or service not known.
Local connections are allowed in pg_hba, the problem is in connecting from the new container in VM.
Here are the settings of my new container:
version: '3'
services:
app:
container_name: app
restart: always
build: ./app
ports:
- "5000:5000"
command: gunicorn -w 1 -b 0.0.0.0:8000 wsgi:server
I tried to make the same connection as from the local computer, specifying the VM_ip, but that didn't help either.
I also tried to specify the <postgres> container ip instead of its name in the host=, but this also caused an error.
Do you know what could be the problem?
You need to create a network first which you will use to communicate between containers. You can do that by:
docker network create <example> #---> you can name it whatever you want
Then you need to connect both containers with the network that you made.
docker run -d --net example --name <postgres_container> <postgres_image>
docker run -d --net example --name <flask_container> <flask_image>
You can read more about the docker network in its documentation here:
https://docs.docker.com/network/
from what I can see you might be using the docker-compose file for the deployment of the services, you can add one more layer above the service layer for the network where you can define the network that is supposed to be used by the services that are deployed. The network that is defined needs also be mentioned in the service definition this lets the Internal DNS engine that docker-compose creates in the background discover all the services in the network with the help of the service name.
A Bridge network may be a good driver to be used here.
You can use the following links for a better understanding of networks in docker-compose.
https://docs.docker.com/compose/compose-file/compose-file-v3/#network
https://docs.docker.com/compose/compose-file/compose-file-v3/#networks
I've followed the instructions in https://tsmx.net/docker-local-mongodb/ but I still get the following error:
**panic: unable to connect to MongoDB (local): no reachable servers
**
I even tried the following but still get the same error:
_ = pflag.String("mongodb-addr", "127.0.0.1:27017", "MongoDB connection address")
My connection code is as follows:
dbAddr := d.cfg.GetString("mongodb-addr")
session, err := mgo.Dial(dbAddr)
And my docker run command is as follows:
docker run image_name
I'm using macOS Monterey. Any help would be greatly appreciated. Thanks.
If the application and the MongoDB are on the same docker network, then use the docker name to connect to the MongoDB container.
If the MongoDB is running in the server where the application is running in docker container, then use the IP of the server to communicate to the MongoDB. 127.0.0.1 from within the container will try to find the MongoDB within the same Docker as the application.
if you run mongo like this :
mongo:
image: mongo
restart: always
volumes:
- ./mongo-data:/data/db
env_file: .env
ports:
- 27017:27017
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_USERNAME}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PASSWORD}
then you can connect from Go like this :
var cred options.Credential
cred.Username = MongoUsername
cred.Password = MongoPassword
clientOption := options.Client().ApplyURI(mongodb://mongodb:27017).SetAuth(cred)
I was facing the same issue and this command did the trick for me, was mentioned here.
Docker provides a host network which lets containers share your host’s networking stack. This approach means localhost inside a container resolves to the physical host, instead of the container itself.
docker run -d --network=host my-container:latest
hope it help someone.
I am using a simple go CRUD api that uses MongoDB and moving it to docker containers. I cannot connect to the MongoDB for some reason. After researching I cannot find a solution.
I have tried:
Exposing/publishing ports, so I can use the container name instead of 'mongodb:localhost:27017/' when trying to connect to the client.
I have also removed any network config in my compose.yml file so that there is no network confusion.
This is my compose.yml file:
version: '3.4'
services:
mongodb:
image: mongo:4.0.4
restart: always
ports:
- 27017:27017
mongo_todo:
build: ./mongo_todo
ports:
- 3000:3000
depends_on:
- mongodb
go_todo:
build: ./go_todo
ports:
- 80:80
depends_on:
- mongo_todo
This is my mongo_todo Dockerfile:
FROM golang:1.14
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
EXPOSE 80
And this is how I am trying to connect to the client:
// Set client options
clientOptions := options.Client().ApplyURI("mongodb://mongodb:27017")
// Connect to MongoDB
Client, _ = mongo.Connect(context.TODO(), clientOptions)
// Check the connection
err := Client.Ping(context.TODO(), nil)
if err != nil {
log.Fatal(err)
}
The program is logging a Fatal error when trying to ping the client to check the connection.
Log output:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2e0 pc=0x9a798c]
goroutine 1 [running]:
go.mongodb.org/mongo-driver/mongo.(*Client).Ping(0x0, 0xd61e20, 0xc000028030, 0x0, 0x1, 0x0)
/go/src/go.mongodb.org/mongo-driver/mongo/client.go:229 +0x21c
main.main()
/go/src/app/main.go:30 +0x16f
I understand that it is some kind of networking issue but I have no idea why it is not working and I have a feeling that there's something simple that I am missing.
Please ask me for more information if needed and thanks in advance for any help.
Edit: I can ping the hostname mongodb from my go_todo container so not sure if it is a network issue.
Yeah, this is confusing a bit (read: annoying) part of using docker-compose.
The service called mongodb is not what the container name is that you set your ApplyURI to. docker-compose prefixes a "project name" to the container name.
Run a docker ps and look for the column 'NAMES' (should be the last one). Use the name of your mongodb instance for the URI host, and it should connect.
I am so stupid and clearly a docker noob.
I thought that docker-compose up rebuilt my container images but it doesn't and so my changes were not being used.
Thanks you for your comments though I appreciate it!
You should debug the code and check which object is Nil. Maybe you are calling method on Nil object.
I will suggest you to explicitly create a Docker network and attach your MongoDB docker instance and your application to it. Then access your MongoDB using container name. I also faced the connection issue and I resolved that using Docker network.
As #CenterOrbit suggested, name your containers explicitly otherwise docker will give it a random name, and you won't be able to connect to MongoDB.
I am working on golang project, recently I read about docker and try to use docker with my app. I am using mongoDB for database.
Now problem is that, I am creating Dockerfile to install all packages and compile and run the go project.
I am running mongo data as locally, if I am running go program without docker it gives me output, but if I am using docker for same project (just installing dependencies with this and running project), it compile successfully but not gives any output, having error::
CreateSession: no reachable servers
my Dockerfile::
# Start from a Debian image with the latest version of Go installed
# and a workspace (GOPATH) configured at /go.
FROM golang
WORKDIR $GOPATH/src/myapp
# Copy the local package files to the container's workspace.
ADD . /go/src/myapp
#Install dependencies
RUN go get ./...
# Build the installation command inside the container.
RUN go install myapp
# Run the outyet command by default when the container starts.
ENTRYPOINT /go/bin/myapp
# Document that the service listens on port 8080.
EXPOSE 8080
EXPOSE 27017
When you run your application inside Docker, it's running in a virtual environment; It's just like another computer but everything is virtual, including the network.
To connect your container to the host, Docker gives it an special ip address and give this ip an url with the value host.docker.internal.
So, assuming that mongo is running with binding on every interface on the host machine, from the container it could be reached with the connection string:
mongodb://host.docker.internal:21017/database
Simplifying, Just use host.docker.internal as your mongodb hostname.
In your golang project, how do you specify connection to mongodb? localhost:27017?
If you are using localhost in your code, your docker container will be the localhost and since you don't have mongodb in the same container, you'll get the error.
If you are starting your docker with command line docker run ... add --network="host". If you are using docker-compose, add network_mode: "host"
Ideally you would setup mongodo in it's own container and connect them from your docker-compose.yml -- but that's not what you are asking for. So, I won't go into that.
In future questions, please include relevant Dockerfile, docker-compose.yml to the extent possible. It will help us give more specific answer.
I'm new to learning how to use goLang to build microservices. I had a whole project up and running locally, but when I tried deploying it I ran into a problem. The session I was working with (mgo.Dial("localhost")) was no longer working. When I put this into a docker image, it failed to connect to the local host, which makes sense, since the docker image builds it over a new OS (alpine in my case). I was wondering what I should do to get it to connect.
To be clear, when I was researching this, most people wanted to connect to a mongoDB session that is a docker container, I want to connect to a mongoDB session from within a docker container. Also once I'm ready for deployment I'll be using StatefulSet with kubernetes if that changes anything.
For example, this is what I want my program to be like:
sess, err := mgo.Dial("localhost") //or whatever
if err != nil {
fmt.Println("failed to connect")
else {
fmt.Println("connected")
What I tried doing:
Dockerfile:
FROM alpine:3.6
COPY /build/app /bin/
EXPOSE 8080
ENTRYPOINT ["/bin/app"]
In terminal:
docker build -t hell:4 .
docker run -d -p 8080:8080 hell:4
And as you can expect, it says not connected. Also the port mapping is for the rest of the project, not this part.
Thanks for your help!
I think you should not try to connect to the MongoDB server running on your machine. Think about deploying the whole application lateron you want a MongoDB server running together with your service on some cloud or server.
That problem could be solved by setting up an additional container and link it to your Go Web App. Docker compose can handle this. Just place a docker-compose.yml file in the directory you are executing your docker build in.
version: '3'
services:
myapp:
build: .
image: hell:4
ports:
- 8080:8080
links:
- mongodb
depends_on:
- mongodb
mongodb:
image: mongo:latest
ports:
- "27017:27017"
environment:
- MONGODB_USER="user"
- MONGODB_PASS="pass"
Something like this should do it (not tested). You have two services: One for your app that gets build according to your Dockerfile in the directory in which you currently are. Additionally it links to a service called mongodb defined below. The mongodb service is accessible via the service name mongodb.
If your mongoDB server is running in your host machine, replace localhost by you host IP.