Can a Mongo-Express container connect to MongoDB with TLS? - mongodb

I have a mongo container, started with the requireTLS TLS mode, and a mongo-express container. Mongo-express does not seem to manage to connect to mongo using TLS.
My docker-compose.yml:
version: '3.1'
services:
mongodb1:
image : "mongo:4.2"
container_name : "mongodb-001"
ports:
- '27017:27017'
environment:
MONGO_INITDB_ROOT_USERNAME : "admin"
MONGO_INITDB_ROOT_PASSWORD : "adminpasswd"
volumes:
- "./mongo-data:/data/db"
- "./etc_mongod.conf:/etc/mongod.conf"
- "./certificates:/etc/certificates:ro"
command:
- "--tlsMode"
- "preferTLS"
- "--tlsDisabledProtocols"
- "none"
- "--tlsCertificateKeyFile"
- "/etc/certificates/certificateKey.pem"
- "--tlsCAFile"
- "/etc/certificates/CA.crt"
- "--tlsAllowConnectionsWithoutCertificates"
mongo-express:
image : "mongo-express:latest"
container_name : "mongo-express-001"
ports:
- '8081:8081'
depends_on:
- mongodb1
volumes:
- "./certificates/CA.crt:/etc/certificates/CA.crt:ro"
environment:
ME_CONFIG_MONGODB_SERVER: "mongodb-001"
ME_CONFIG_MONGODB_PORT: "27017"
ME_CONFIG_MONGODB_ENABLE_ADMIN: "false"
ME_CONFIG_MONGODB_AUTH_DATABASE: "admin"
ME_CONFIG_MONGODB_AUTH_USERNAME: "admin"
ME_CONFIG_MONGODB_AUTH_PASSWORD: "adminpasswd"
ME_CONFIG_MONGODB_ADMINUSERNAME: "admin"
ME_CONFIG_MONGODB_ADMINPASSWORD: "adminpasswd"
ME_CONFIG_SITE_SSL_ENABLED: "true"
ME_CONFIG_MONGODB_CA_FILE: "/etc/certificates/CA.crt"
...and the error message I get:
mongodb-001 | 2020-10-09T14:16:13.299+0000 I NETWORK [listener] connection accepted from 172.31.0.3:44774 #2 (1 connection now open)
mongodb-001 | 2020-10-09T14:16:13.305+0000 I NETWORK [conn2] Error receiving request from client: SSLHandshakeFailed: The server is configured to only allow SSL connections. Ending connection from 172.31.0.3:44774 (connection id: 2)
mongodb-001 | 2020-10-09T14:16:13.305+0000 I NETWORK [conn2] end connection 172.31.0.3:44774 (0 connections now open)
mongo-express-001 |
mongo-express-001 | /node_modules/mongodb/lib/server.js:265
mongo-express-001 | process.nextTick(function() { throw err; })
mongo-express-001 | ^
mongo-express-001 | Error [MongoError]: connection 0 to mongodb-001:27017 closed
mongo-express-001 | at Function.MongoError.create (/node_modules/mongodb-core/lib/error.js:29:11)
mongo-express-001 | at Socket.<anonymous> (/node_modules/mongodb-core/lib/connection/connection.js:200:22)
mongo-express-001 | at Object.onceWrapper (events.js:422:26)
mongo-express-001 | at Socket.emit (events.js:315:20)
mongo-express-001 | at TCP.<anonymous> (net.js:674:12)
mongo-express-001 exited with code 1
Note that:
I can connect to MongoDB using a mongo shell with the same parameters I pass to mongo-express:
mongo "mongodb://admin:adminpasswd#mongodb-001:27017/admin?authSource=admin" --tls --tlsCAFile certificates/CA.crt
If I start MongoDB in preferTLS mode, the mongo-express connection works

tl;dr
Create a new config.js file with the following code
module.exports = {
mongodb: {
connectionOptions: {
ssl: true,
}
}
};
and mount that file in your docker compose file at /node_modules/mongo-express/config.js
Explanation
It appears to be an issue with their config.default.js file. In it, they have this
module.exports = {
mongodb: {
// if a connection string options such as server/port/etc are ignored
connectionString: mongo.connectionString || getConnectionStringFromEnvVariables(),
connectionOptions: {
// ssl: connect to the server using secure SSL
ssl: process.env.ME_CONFIG_MONGODB_SSL || mongo.ssl,
// sslValidate: validate mongod server certificate against CA
sslValidate: process.env.ME_CONFIG_MONGODB_SSLVALIDATE || true,
// sslCA: array of valid CA certificates
sslCA: sslCAFromEnv ? [sslCAFromEnv] : [],
// autoReconnect: automatically reconnect if connection is lost
autoReconnect: true,
// poolSize: size of connection pool (number of connections to use)
poolSize: 4,
}
}
You'll notice the existence of an env var that's not listed (with reason) in their documentation. ME_CONFIG_MONGODB_SSL
This is required to enable tls support. Setting the env var yourself however does nothing but throw an error and break express due to it not being cast to a Boolean. So it just reads as a string 'true' or 'false'.
This code is fixed in their npm package code, but they haven't updated their docker image since late 2021. So the only "fix" I've found for this is to create a new config.js file with the following code
module.exports = {
mongodb: {
connectionOptions: {
ssl: true,
sslValidate: true,
}
}
};
Then mount this file at /node_modules/mongo-express/config.js in your docker-compose file. It'll read these and overwrite the defaults.
Note: I added the sslValidate key:value pair as well due to the fact that it suffers from the same lack of type casting. So if you omit the ME_CONFIG_MONGODB_SSLVALIDATE env var entirely, it'll be set as true, but if you include the env var as either true or false, it'll just simply break (undefined behaviour).

Related

Cannot create a mongo database with docker

I'm having trouble creating a mongo database using the docker-compose command. Docker desktop tells me that everything is up and running including the db, but all I get is the standard 'admin, config, local' not the db I want to create. Here's my docker-compose.yaml
version: '3'
services:
app:
build: ./
entrypoint: ./.docker/entrypoint.sh
ports:
- 3000:3000
volumes:
- .:/home/node/app
depends_on:
- db
db:
image: mongo:4.4.4
restart: always
volumes:
- ./.docker/dbdata:/data/db
- ./.docker/mongo:/docker-entrypoint-initdb.d
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=root
- MONGO_INITDB_DATABASE=nest
mongo-express:
image: mongo-express
restart: always
ports:
- 8081:8081
environment:
- ME_CONFIG_MONGODB_SERVER=db
- ME_CONFIG_MONGODB_AUTH_USERNAME=root
- ME_CONFIG_MONGODB_AUTH_PASSWORD=root
- ME_CONFIG_MONGODB_ADMINUSERNAME=root
- ME_CONFIG_MONGODB_ADMINPASSWORD=root
depends_on:
- db
my init.js inside .docker/mongo
db.routes.insertMany([
{
_id: "1",
title: "Primeiro",
startPosition: {lat: -15.82594, lng: -47.92923},
endPosition: {lat: -15.82942, lng: -47.92765},
},
{
_id: "2",
title: "Segundo",
startPosition: {lat: -15.82449, lng: -47.92756},
endPosition: {lat: -15.82776, lng: -47.92621},
},
{
_id: "3",
title: "Terceiro",
startPosition: {lat: -15.82331, lng: -47.92588},
endPosition: {lat: -15.82758, lng: -47.92532},
}
]);
and my dockerfile
FROM node:14.18.1-alpine
RUN apk add --no-cache bash
RUN npm install -g #nestjs/cli
USER node
WORKDIR /home/node/app
and this is the 'error' log I get from docker when I run the nest container with mongodb, nest app and mongo express(there is actually a lot more but SO keeps thinking that it is spam for some reason.
about to fork child process, waiting until server is ready for connections.
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
Error saving history file: FileOpenFailed Unable to open() file /home/mongodb/.dbshell: No such file or directory
{"t":{"$date":"2022-06-01T19:39:15.542+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn2","msg":"Connection ended","attr":{"remote":"127.0.0.1:39304","connectionId":2,"connectionCount":0}}
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.js
{"t":{"$date":"2022-06-01T19:39:15.683+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"127.0.0.1:39310","connectionId":3,"connectionCount":1}}
{"t":{"$date":"2022-06-01T19:39:15.684+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn3","msg":"client metadata","attr":{"remote":"127.0.0.1:39310","client":"conn3","doc":{"application":{"name":"MongoDB Shell"},"driver":{"name":"MongoDB Internal Client","version":"4.4.4"},"os":{"type":"Linux","name":"Ubuntu","architecture":"x86_64","version":"18.04"}}}}
{"t":{"$date":"2022-06-01T19:39:15.701+00:00"},"s":"I", "c":"STORAGE", "id":20320, "ctx":"conn3","msg":"createCollection","attr":{"namespace":"nest.routes","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"f689868e-af6d-4ec6-b555-dcf520f24788"}},"options":{}}}
{"t":{"$date":"2022-06-01T19:39:15.761+00:00"},"s":"I", "c":"INDEX", "id":20345, "ctx":"conn3","msg":"Index build: done building","attr":{"buildUUID":null,"namespace":"nest.routes","index":"_id_","commitTimestamp":{"$timestamp":{"t":0,"i":0}}}}
uncaught exception: ReferenceError: colection is not defined :
#/docker-entrypoint-initdb.d/init.js:23:1
failed to load: /docker-entrypoint-initdb.d/init.js
exiting with code -3
this is what running docker-compose ps shows
NAME COMMAND SERVICE STATUS PORTS
nest-api-app-1 "./.docker/entrypoin…" app running 0.0.0.0:3000->3000/tcp
nest-api-db-1 "docker-entrypoint.s…" db running 27017/tcp
nest-api-mongo-express-1 "tini -- /docker-ent…" mongo-express running 0.0.0.0:8081->8081/tcp
this what my docker desktop shows
The MongoDB container only creates a database if no database already exists. You probably already have one, which is why a new database isn't created and your initialization script isn't run.
Delete the contents of ./.docker/dbdata on the host. Then start the containers with docker-compose and Mongo should create your database for you.

Can't connection DB use ktor application

I try start ktor application with postgresql database. For it i used docker compose. My docker-compose.yml.
version: '3.0'
services:
ktor-sample:
build: ./
command: ./ktor-sample
ports:
- "8080:8080"
depends_on:
- db
db:
restart: unless-stopped
image: postgres:9.6.10-alpine
volumes:
- ./test:/var/lib/postgresql/data
ports:
- "5433:5433"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD:
POSTGRES_DB: AutoHistory
I every one receive error:
WARN Exposed - Transaction attempt #2 failed: Connection to localhost:5433 refused.
Check that the hostname and port are correct and that the postmaster is accepting
TCP/IP connections.. Statement(s): null
ktor-sample_1 | org.postgresql.util.PSQLException: Connection to localhost:5433
refused. Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections.
This database is created and can connection to it.
As for me I do next I create an object for initializing of db
it can look like this
object DbFactory {
fun init() {
val pool = hikari()
val db = Database.connect(pool)
transaction(db) {
SchemaUtils.create(Creators, Visitors, Places, Roles, UserRoles, Users)
}
}
private fun hikari(): HikariDataSource {
val hikariConfig = HikariConfig("db.properties")
return HikariDataSource(hikariConfig)
}
}
and in Application.kt, on the first place initialize your db
fun main() {
embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
DbFactory.init()
...
the file which is describing your data should live in root and be called db.properties look something like
jdbcUrl=jdbc:postgresql://localhost:5433/AutoHistory
dataSource.driverClass = org.postresql.Driver
dataSource.driver=postgresql
dataSource.database= AutoHistory
dataSource.user= postgres
dataSource.password=
I have no such rows in docker-compose.yml
ktor-sample:
build: ./
command: ./ktor-sample
ports:
- "8080:8080"
depends_on:
- db
and do start my app manually but anyway I think it should help you

Cannot configure a Mongo replicaSet from docker init script

I am trying to set up a 2 node replicaSet in docker for local development only. Single node already works fine, but there are keyfile issues when trying to add a member as part of the docker init script (NB I see the keyfile is set correctly from the logs). The same command works fine from a shell though, not via the init script.
Basically, the current config has worked fine for one node, but adding another gives the following error:
mongo_1 | {"t":{"$date":"2021-07-21T16:33:19.583+00:00"},"s":"W", "c":"REPL", "id":23724, "ctx":"ReplCoord-0","msg":"Got error response on heartbeat request","attr":{"hbStatus":{"code":13,"codeName":"Unauthorized","errmsg":"command replSetHeartbeat requires authentication"},"requestTarget":"mongo-secondary:27017","hbResp":{"ok":1.0}}}
mongo_1 | {"t":{"$date":"2021-07-21T16:33:19.583+00:00"},"s":"E", "c":"REPL", "id":21426, "ctx":"conn2","msg":"replSetInitiate failed","attr":{"error":{"code":74,"codeName":"NodeNotFound","errmsg":"replSetInitiate quorum check failed because not all proposed set members responded affirmatively: mongo-secondary:27017 failed with command replSetHeartbeat requires authentication"}}}
If I remove mongo-secondary from the set, then after startup use a shell to mongo to load the exact same config, everything works fine (they keyfile is used and the set is made with both members).
Currently my config is:
# docker-compose.yml
mongo: &MONGO
image: mongo:4.4
restart: unless-stopped
volumes:
- mongo_data:/data/db
- ./scripts/docker/mongo/001_mongo_init.js:/docker-entrypoint-initdb.d/001_mongo_init.js:ro
- ./scripts/docker/mongo/mongo-entrypoint.sh:/mongo-entrypoint
- ./conf/mongodb/mongod-config.yml:/etc/mongod.yml
entrypoint: sh /mongo-entrypoint
ports:
- 27017:27017
env_file:
- ./env/mongo.env
command: --auth --config /etc/mongod.yml
extra_hosts:
- mongo:127.0.0.1
mongo-secondary:
<<: *MONGO
volumes:
- mongo_secondary_data:/data/db
- ./scripts/docker/mongo/mongo-entrypoint.sh:/mongo-entrypoint
- ./conf/mongodb/mongod-config.yml:/etc/mongod.yml
ports:
- 27018:27017
extra_hosts:
- mongo-secondary:127.0.0.1
# mongo-entrypoint.sh
#!/bin/sh
set -eu
# Create the keyfile used for mongo replicaSet auth.
keyfile=/home/keyfile
echo "Creating replicaSet keyfile..."
echo "keyfile" > ${keyfile}
chmod 0400 $keyfile
chown mongodb $keyfile
echo "Created replicaSet keyfile."
# original entrypoint
exec docker-entrypoint.sh "$#"
// 001_mongo_init.js
function getEnv(envVar) {
const ret = run('sh', '-c', `printenv ${envVar} > /tmp/${envVar}.txt`);
if (ret !== 0) throw Error(`Value "${envVar}" is not present in the environment.`);
return cat(`/tmp/${envVar}.txt`).trim(); // NB cat leaves a \n at the end of text
}
// create replicaset
const rsconf = {
_id: getEnv('MONGODB_REPLICA_SET'),
members: [
{
_id: 0,
host: 'mongo:27017',
},
{
_id: 1,
host: 'mongo-secondary:27017',
priority: 0, // prevent from becoming master
},
],
};
rs.initiate(rsconf);
rs.conf();
// further code to create users etc.
# mongod-config.yml
---
security:
keyFile: /home/keyfile
replication:
replSetName: rs0
enableMajorityReadConcern: true

Failed to connect to a mongodb docker container from mongoose on host

I'm trying to connect to mongodb running in docker from the app running on host using mongoose but it failed.
I can't use the port 27017 for the new mongodb container because it is used by other container. So I followed the guide here for setting it up using the compose.
Below are the snippets:
docker-compose.yml
version: '3'
services:
db:
image: mongo:latest
restart: always
ports:
- '8081:8081'
environment:
MONGO_INITDB_ROOT_USERNAME: root1
MONGO_INITDB_ROOT_PASSWORD: password1
But when I do docker-ps, port 27017 still there but I'm not sure if that causes an issue.
PORTS
0.0.0.0:8081->8081/tcp, 27017/tcp
Then I created a new user in admin database.
use admin
db.createUser(
{
user: "admin1",
pwd: "password2",
roles: [ { role: "userAdminAnyDatabase", db: "admin" }, "readWriteAnyDatabase" ]
}
)
server.js
const connectOption = {
useNewUrlParser: true,
user: 'admin1',
pass: 'password2',
authSource: 'admin',
}
const mongoURL = 'mongodb://localhost:8081/app1';
mongoose.connect(mongoURL, connectOption)
.then(() => console.log('MongoDB Connected'))
.catch(error => console.log(error));
And the error I received is
{
MongoNetworkError: failed to connect to server [localhost:8081] on first connect [MongoNetworkError: write EPIPE]
...
...
...
name: 'MongoNetworkError',
errorLabels: [ 'TransientTransactionError' ],
[Symbol(mongoErrorContextSymbol)]: {}
}
Assuming you're running nodejs application as a docker-compose service, in db service remove ports section (including - '8081:8081'
line). In server.js, change const mongoURL = 'mongodb://localhost:8081/app1'; to const mongoURL = 'mongodb://db:27017/app1';.
If you want to access the db from host machine, change ports 8081:8081 to <give-a-port-number>:27017.

Graylog container cannot connect to MongoDB container

I have some troubles setting up Graylog2 under docker. Everything works until I try using authentication. All I get is the following error repeated forever.
Trying both root and graylog user (in both graylog and admin db) gives the same result.
The log from mongodb says both users are created during setup. But graylog says it does not find any graylog user in database graylog. Same with user root.
I'm new to MongoDB and have no idea how authentication works. But from what I understand authentication (similar to --auth parameter) is activated when providing user/pw for root account (https://github.com/docker-library/mongo/pull/145).
Is it possible that Graylog ses a different authentication mechanism than MongoDB is excpecting? See line #158 in the pasted log
Error message as root user
mongodb_1 | 2017-04-16T13:27:52.486+0000 I NETWORK [thread1] connection accepted from 172.18.0.4:46566 #12 (1 connection now open)
mongodb_1 | 2017-04-16T13:27:52.495+0000 I NETWORK [conn12] received client metadata from 172.18.0.4:46566 conn12: { driver: { name: "mongo-java-driver", version: "unknown" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.4.0-72-generic" }, platform: "Java/Oracle Corporation/1.8.0_72-internal-b15" }
mongodb_1 | 2017-04-16T13:27:52.525+0000 I ACCESS [conn12] SCRAM-SHA-1 authentication failed for root on graylog from client 172.18.0.4:46566 ; UserNotFound: Could not find user root#graylog
mongodb_1 | 2017-04-16T13:27:52.543+0000 I - [conn12] end connection 172.18.0.4:46566 (1 connection now open)
Error message as graylog user (Full log on pastebin)
mongodb_1 | 2017-04-16T15:47:48.404+0000 I NETWORK [thread1] connection accepted from 172.18.0.4:41602 #7 (1 connection now open)
mongodb_1 | 2017-04-16T15:47:48.410+0000 I NETWORK [conn7] received client metadata from 172.18.0.4:41602 conn7: { driver: { name: "mongo-java-driver", version: "unknown" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "4.4.0-72-generic" }, platform: "Java/Oracle Corporation/1.8.0_72-internal-b15" }
mongodb_1 | 2017-04-16T15:47:48.418+0000 I ACCESS [conn7] SCRAM-SHA-1 authentication failed for graylog on graylog from client 172.18.0.4:41602 ; UserNotFound: Could not find user graylog#graylog
mongodb_1 | 2017-04-16T15:47:48.423+0000 I - [conn7] end connection 172.18.0.4:41602 (1 connection now open)
This is my ./docker-composer.yml
version: '2'
services:
mongodb:
build: ./mongodb
volumes:
- /docker/mongodb/data:/data/db
elasticsearch:
image: "elasticsearch:2"
command: "elasticsearch -Des.cluster.name='graylog'"
volumes:
- /docker/elasticsearch/data:/usr/share/elasticsearch/data
graylog:
image: graylog2/server
volumes:
- /docker/graylog/journal:/usr/share/graylog/data/journal
- /docker/graylog/config:/usr/share/graylog/data/config
environment:
#GRAYLOG_MONGODB_URI: mongodb://root:drUqGGCMh#mongodb:27017/graylog
GRAYLOG_MONGODB_URI: mongodb://graylog:vWGzncmBe9#mongodb:27017/graylog
depends_on:
- mongodb
- elasticsearch
ports:
- "9000:9000"
./mongodb/Dockerfile
FROM mongo:3
ENV MONGO_INITDB_ROOT_USERNAME: root
ENV MONGO_INITDB_ROOT_PASSWORD: drUqGGCMh
ADD grayloguser.js /docker-entrypoint-initdb.d/grayloguser.js
./mogodb/grayloguser.js
db.getSiblingDB('graylog');
db.createUser(
{
user: "graylog",
pwd: "vWGzncmBe9",
roles: [
{ role: "dbOwner", db: "graylog" }
]
}
);
Your MongoDB script is incorrect.
Either assign the return value of db.getSiblingDB('graylog') to a variable and use that for createUser(), or keep using use graylog instead:
graylog = db.getSiblingDB('graylog');
graylog.createUser(
{
user: "graylog",
pwd: "vWGzncmBe9",
roles: [
{ role: "dbOwner", db: "graylog" }
]
}
);
In other words, just stick to the MongoDB documentation: https://docs.mongodb.com/manual/tutorial/create-users/#username-password-authentication