MongoDB Cannot run tests when building Docker image - Server selection error: server selection timeout - mongodb

I'm running a mongo image in a container with this config:
version: '3'
services:
mongodb:
image: mongo
ports:
- '27017:27017'
environment:
- MONGO_INITDB_ROOT_USERNAME=user
- MONGO_INITDB_ROOT_PASSWORD=password
- MONGO_INITDB_DATABASE=microservices
volumes:
- ./docker-entrypoint-initdb.d/mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js:ro
And I have started to build microservices with golang and I want to start dockerizing those as well to speed up the development process. However when I try to run this Dockerfile below it panics at the run test command with the error:
database url: mongodb://user:password#127.0.0.1:27017/microservices
server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: 127.0.0.1:27017, Type: Unknown, Last error: connection() error occurred during connection handshake: dial tcp 127.0.0.1:27017: connect: connection refused }, ] }
panic: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: 127.0.0.1:27017, Type: Unknown, Last error: connection() error occurred during connection handshake: dial tcp 127.0.0.1:27017: connect: connection refused }, ] }
The db connection works since when I try to run the test from the go files, they pass. There only seems to be any problem when I try to run them from the container when building it up.
FROM golang:1.18 as build
WORKDIR /go/src/app
COPY . .
RUN go mod download
RUN go vet -v /go/src/app/...
RUN go test -v /go/src/app/...
RUN CGO_ENABLED=0 go build -o /go/bin/app/authsvc /go/src/app/authentication/main.go
FROM gcr.io/distroless/static-debian11
COPY --from=build /go/bin/app/authsvc /
COPY --from=build /go/src/app/authentication/.env /
CMD ["/authsvc"]

Change database url
try mongodb
database url: mongodb://user:password#mongodb:27017/microservices

So first of all you must check your default network ip with
docker network ls
Result
After get network id and use this command :
docker inspect <your network id>
Inside result schema get your Gateway ip
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "111.11.0.0/11",
"Gateway": "111.11.0.1"
}
]
}
and use that ip
database url: mongodb://user:password# < your Gateway ip > :27017/microservices

Related

Unable to import json into mongo db atlas

I'm attempting to import json into an atlas db.
The attempt uses this code:
mongoimport --uri "mongodb://usr:pass#myurlname.uia8q.mongodb.net/paintura?retryWrites=true&w=majority" --collection products --file pathtojson
all parameters in the above are correct via the connect docs.
exact error is:
error connecting to host: could not connect to server: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: myurlname.uia8q.mongodb.net:27017, Type: Unknown, Last error: connection() error occured during connection handshake: dial tcp: lookup paintura.uia8q.mongodb.net: no such host }, ] }
For the user I have atlas admin selected so I assume it should have full privelege.
I also have cleared my ip in the ip list.
Any ideas what I'm missing?

MongoNetworkError when connecting from different container

PHP dev new to NodeJS and I am struggling to get my NodeJS container to connect to my MongoDB container. As far I can see I have all the correct NPMs installed in my Docker file and the docker-compose is correct. Please note that I have not added the containers to the same network or but in the link to the db service into the nodejs container, although I did try this and got pretty much the same result.
Unsure why I am getting the error below when I bash into the nodejs container and run node app.js
Error
[nodemon] clean exit - waiting for changes before restart
[nodemon] restarting due to changes...
[nodemon] starting `node app.js`
(node:92) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
Server is listening on port 3000
Could not connect to the database. Exiting now... { MongoNetworkError: failed to connect to server [localhost:27017] on first connect [MongoNetworkError: connect ECONNREFUSED localhost:27017]
at Pool.<anonymous> (/usr/src/app/node_modules/mongodb/lib/core/topologies/server.js:431:11)
at Pool.emit (events.js:193:13)
at createConnection (/usr/src/app/node_modules/mongodb/lib/core/connection/pool.js:559:14)
at connect (/usr/src/app/node_modules/mongodb/lib/core/connection/pool.js:973:11)
at makeConnection (/usr/src/app/node_modules/mongodb/lib/core/connection/connect.js:39:11)
at callback (/usr/src/app/node_modules/mongodb/lib/core/connection/connect.js:261:5)
at Socket.err (/usr/src/app/node_modules/mongodb/lib/core/connection/connect.js:286:7)
at Object.onceWrapper (events.js:281:20)
at Socket.emit (events.js:193:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at processTicksAndRejections (internal/process/task_queues.js:81:17)
name: 'MongoNetworkError',
errorLabels: [ 'TransientTransactionError' ],
[Symbol(mongoErrorContextSymbol)]: {} }
docker-compose.yml
version: '3.5' # We use version 3.5 syntax
services: # Here we define our service(s)
frontend:
container_name: angular
build: ./angular_app
volumes:
- ./angular_app:/usr/src/app
ports:
- 4200:4200
command: >
bash -c "npm install && ng serve --host 0.0.0.0 --port 4200"
depends_on:
- api
# NodeJS/Express service for API
api:
image: nodeexpress
build:
context: ./node_server
dockerfile: Dockerfile
volumes:
- ./node_server:/usr/src/app
- /usr/src/app/node_modules
ports:
- 3000:3000
links:
- mongoservice
depends_on:
- mongoservice
# Mongo database service
mongoservice:
image: mongo
container_name: mongocontainer
restart: always
environment:
MONGO_INITDB_ROOT_USERNAME: ${DB_MONGO_ROOTUSER}
MONGO_INITDB_ROOT_PASSWORD: ${DB_MONGO_ROOTPWD}
ports:
- ${DB_MONGO_EXTERNAL_PORT}:${DB_MONGO_INTERNAL_PORT}
volumes:
- ${DB_MONGO_VOLUME1}
volumes:
data:
external: true
networks:
default:
driver: bridge
Dockerfile (for api service - nodejs express)
FROM node:11-alpine
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY . .
RUN npm install
#RUN npm install mysql
RUN npm install mongodb --save
#RUN npm install --save body-parser express mysql2 sequelize helmet cors
RUN npm install --save body-parser express mongoose helmet cors
RUN npm install --save nocache
RUN npm install nodemon --save
EXPOSE 4300
#CMD ["npm", "run", "start"]
CMD [ "npm", "run", "start.dev" ]
app.js
const express = require('express');
const bodyParser = require('body-parser');
// create express app
const app = express();
// parse requests of content-type - application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: true }))
// parse requests of content-type - application/json
app.use(bodyParser.json());
// Configuring the database
const dbConfig = require('./config/database.config');
const mongoose = require('mongoose');
mongoose.Promise = global.Promise;
// Connecting to the database
// Connection string variants attempted
// mongodb://root:secret#0.0.0.0:27017/angapp2
// mongodb://root:secret#127.0.0.1:27017/angapp2
// mongodb://root:secret#mongoservice:27017/angapp2
mongoose.connect('mongodb://root:secret#localhost:27017/angapp2', {
useNewUrlParser: true
}).then(() => {
console.log("Successfully connected to the database");
}).catch(err => {
console.log('Could not connect to the database. Exiting now...', err);
process.exit();
});
// define a simple route
app.get('/', (req, res) => {
res.json({"message": "Welcome to EasyNotes application. Take notes quickly. Organize and keep track of all your notes."});
});
// Require Notes routes
require('./routes/note.routes.js')(app);
// listen for requests
app.listen(3000, () => {
console.log("Server is listening on port 3000");
});
What I've tried:
Attempted the various connection string variants in terms of the host name, i.e. localhost, 127.0.0.1, 0.0.0.0, mongoservice
Also ran docker inspect <container-id> on the mongo service and got the internal IP address of the container and tried that in the connection string
Added RUN npm install mongodb --save to node servers Dockerfile
Managed to connect Robo 3D GUI to the Mongo container without issue
Bashed into Mongo service and managed to log into the DB and run some statements as a test that the service was working fine.
Maybe its just me being blind but, it seems that you are trying to connect to your Database on Port 27017 but in your Docker-Compose File you set the Port of the Database to 8081. Try Matching the ports.

using mongdb in docker with Go

i tried to run MongoDB in docker-compose and connect my Go server with MongoDB in docker seem not to work for me, thought i already use mongodb:MyContanerName:27017
when i run my go server and then i tired to request on my client, the error something like this
2019/09/08 14:10:37 Unexpected error: rpc error: code = Internal desc = internal error: server selection error: server selection timeout
current topology: Type: Unknown
Servers:
Addr: mongo:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection(mongo:27017[-125]) connection is closed
i already did docker-compose up -d for my docker, and it was running on my docker ,
the full code is trigger here golang and mongodb

Mup error with docker and mongodb while deploying to Digital Ocean

I am trying to deploy a Meteor app using mup, and it is giving me all the time this error:
[xxx.xxx.xxx.xxx] x Start Mongo: FAILED
------------------------------------STDERR------------------------------------
Error response from daemon: Container a1617b2aaaa3fd4aeb8e1241ec90ae32b4a88b0df9c95e5d73de608b68788ef0 is not running
docker: Error response from daemon: driver failed programming external connectivity on endpoint mongodb (2f5db292f7022a39dfe3ff5fdf5ed7de38ddc7bb6f787625cd0fb45a5fa10cd6): Error starting userland proxy: listen tcp 127.0.0.1:27017: bind: address already in use.
------------------------------------STDOUT------------------------------------
3.2.15: Pulling from library/mongo
Digest: sha256:ef3277c7221e8512a1657ad90dfa2ad13ae2e35aacce6cd7defabbbdcf67ca76
Status: Image is up to date for mongo:3.2.15
mongodb
mongodb
Running mongo:3.2.15
97a28fe9d2dca6130ffe6622e662a230ecea34214920673c574a4d3e57fafb3c
------------------------------------------------------------------------------
I have checked and I can connect through ssh to the server. I am using one of the One-click droplets from Digital Ocean because it came with Mongodb installed already. The version of Mongodb on the server is 3.4 and in my app I am using 3.2.15, I do not know if I should downgrade or upgrade either one or the other, or if even if the problem is there.
My mup.js file looks like this:
module.exports = { servers: {
one: {
host: 'xxx.xxx.xxx.xxx',
username: 'root',
pem: '~/.ssh/id_rsa'
// or neither for authenticate from ssh-agent
}
},
meteor: {
name: 'myApp',
path: '../myApp/',
servers: {
one: {},
},
buildOptions: {
serverOnly: true,
},
env: {
// If you are using ssl, it needs to start with https://
ROOT_URL: 'http://app.com',
"PORT": 80,
MONGO_URL: 'mongodb://localhost/local',
//MONGO_OPLOG_URL: 'mongodb://mongodb/local',
},
deployCheckWaitTime: 60,
docker: {
// change to 'abernix/meteord:base' if your app is using Meteor 1.4 - 1.5
image: 'abernix/meteord:base',
},
// Show progress bar while uploading bundle to server
// You might need to disable it on CI servers
enableUploadProgressBar: true
},
mongo: {
version: '3.2.15',
servers: {
one: {}
}
},
};
I am going crazy with this, can someone help me please?
Try creating a one click droplet on Digital Ocean with Ubuntu instead of mongo as mup sets up mongo for you already. I believe Meteor UP assumes it's doing the deploy from fresh settings.

Graylog2 mongo profiler plugin won't connect to mongo instance, error: "Exception opening socket" (everything dockerized)

I am trying to play with graylog's mongo profiler plugin using docker to run everything. But I can't get any profiling logs into graylog.
When I start the mongo input from the graylog UI it eventually times out with an error:
Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:37017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}].
This is my setup based on following the graylog dockerhub installation and the mongo profiler plugin guide and modifying bits:
(1) I bring up graylog, mongo and elastic using a docker-compose file:
version: '2'
services:
some-mongo:
image: "mongo:3"
some-elasticsearch:
image: "elasticsearch:2"
command: "elasticsearch -Des.cluster.name='graylog'"
graylog:
image: graylog2/server:2.2.1-1
environment:
GRAYLOG_PASSWORD_SECRET: somepasswordpepper
GRAYLOG_ROOT_PASSWORD_SHA2: 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
GRAYLOG_WEB_ENDPOINT_URI: http://127.0.0.1:9000/api
links:
- some-mongo:mongo
- some-elasticsearch:elasticsearch
ports:
- "9000:9000"
- "514:514/udp"
- "12202:12202"
- "37017:37017"
That has worked fine so far and I've been able to send in syslog udp messages and gelf http messages.
(2) I created a separate mongo docker container with ports mapped because I worry that if I use 27017, that graylog might look in its own internal mongodb container:
docker run -d -p 37017:27017 mongo:2.4
I start a mongo session and enable profiling for a "graylog" database:
$ mongo --port 37017
> use graylog
> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 100, "ok" : 1 }
> db.foo.insert({_id:1})
// Check that profiling data is being written to system.profile:
> db.system.profile.find().limit(1).sort( { ts : -1 } ).pretty()
{
"op" : "query",
"ns" : "graylog.foo",
"query" : {
},
"ntoreturn" : 0,
"ntoskip" : 0,
....
"allUsers" : [ ],
"user" : ""
}
So it seems like the mongod instance is running and profiling is working.
(3) I download the plugin jar and docker cp it into the plugins dir inside the graylog docker container. Something like:
docker cp graylog-plugin-mongodb-profiler-2.0.1.jar e89a2decda37:/usr/share/graylog/plugin
Then restart graylog.
I can see that the file is there:
$ docker exec -it e89a2decda37 /bin/sh
# ls /usr/share/graylog/plugin
graylog-plugin-anonymous-usage-statistics-2.2.1.jar graylog-plugin-map-widget-2.2.1.jar
graylog-plugin-beats-2.2.1.jar graylog-plugin-mongodb-profiler-2.0.1.jar
graylog-plugin-collector-2.2.1.jar graylog-plugin-pipeline-processor-2.2.1.jar
graylog-plugin-enterprise-integration-2.2.1.jar
So that part seemed to work fine and I can see an entry "Mongo profiler input" in the list of input types in the graylog UI.
(4) I create a "Mongo profiler input" input with:
hostname: localhost
port: 37017
database: graylog
(5) After I click save, the input tries to start then eventually fails like above. Restarting graylog or trying to restart the input results in the same failures.
I have tried step (2) with different versions of mongo in case there was some driver incompatibility but they all fail with same error. I've tried docker images:
mongo:3
mongo:2.6
mongo:2.4
Thanks in advance!
Like Thilo suggested above, the hostname for the graylog input shouldn't be "localhost" as that will point graylog to the docker container hosting it.
So I found the ip of the host machine using:
docker exec -it [CONTAINER ID] /bin/sh
/sbin/ip route|awk '/default/ { print $3 }'
and rewired the input and Bob's your Uncle!