Graylog2 mongo profiler plugin won't connect to mongo instance, error: "Exception opening socket" (everything dockerized) - mongodb

Question

I am trying to play with graylog's mongo profiler plugin using docker to run everything. But I can't get any profiling logs into graylog.
When I start the mongo input from the graylog UI it eventually times out with an error:
Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:37017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}].
This is my setup based on following the graylog dockerhub installation and the mongo profiler plugin guide and modifying bits:
(1) I bring up graylog, mongo and elastic using a docker-compose file:
version: '2'
services:
some-mongo:
image: "mongo:3"
some-elasticsearch:
image: "elasticsearch:2"
command: "elasticsearch -Des.cluster.name='graylog'"
graylog:
image: graylog2/server:2.2.1-1
environment:
GRAYLOG_PASSWORD_SECRET: somepasswordpepper
GRAYLOG_ROOT_PASSWORD_SHA2: 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
GRAYLOG_WEB_ENDPOINT_URI: http://127.0.0.1:9000/api
links:
- some-mongo:mongo
- some-elasticsearch:elasticsearch
ports:
- "9000:9000"
- "514:514/udp"
- "12202:12202"
- "37017:37017"
That has worked fine so far and I've been able to send in syslog udp messages and gelf http messages.
(2) I created a separate mongo docker container with ports mapped because I worry that if I use 27017, that graylog might look in its own internal mongodb container:
docker run -d -p 37017:27017 mongo:2.4
I start a mongo session and enable profiling for a "graylog" database:
$ mongo --port 37017
> use graylog
> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 100, "ok" : 1 }
> db.foo.insert({_id:1})
// Check that profiling data is being written to system.profile:
> db.system.profile.find().limit(1).sort( { ts : -1 } ).pretty()
{
"op" : "query",
"ns" : "graylog.foo",
"query" : {
},
"ntoreturn" : 0,
"ntoskip" : 0,
....
"allUsers" : [ ],
"user" : ""
}
So it seems like the mongod instance is running and profiling is working.
(3) I download the plugin jar and docker cp it into the plugins dir inside the graylog docker container. Something like:
docker cp graylog-plugin-mongodb-profiler-2.0.1.jar e89a2decda37:/usr/share/graylog/plugin
Then restart graylog.
I can see that the file is there:
$ docker exec -it e89a2decda37 /bin/sh
# ls /usr/share/graylog/plugin
graylog-plugin-anonymous-usage-statistics-2.2.1.jar graylog-plugin-map-widget-2.2.1.jar
graylog-plugin-beats-2.2.1.jar graylog-plugin-mongodb-profiler-2.0.1.jar
graylog-plugin-collector-2.2.1.jar graylog-plugin-pipeline-processor-2.2.1.jar
graylog-plugin-enterprise-integration-2.2.1.jar
So that part seemed to work fine and I can see an entry "Mongo profiler input" in the list of input types in the graylog UI.
(4) I create a "Mongo profiler input" input with:
hostname: localhost
port: 37017
database: graylog
(5) After I click save, the input tries to start then eventually fails like above. Restarting graylog or trying to restart the input results in the same failures.
I have tried step (2) with different versions of mongo in case there was some driver incompatibility but they all fail with same error. I've tried docker images:
mongo:3
mongo:2.6
mongo:2.4
Thanks in advance!

Answer 1

Like Thilo suggested above, the hostname for the graylog input shouldn't be "localhost" as that will point graylog to the docker container hosting it.
So I found the ip of the host machine using:
docker exec -it [CONTAINER ID] /bin/sh
/sbin/ip route|awk '/default/ { print $3 }'
and rewired the input and Bob's your Uncle!