Start MongoDb Test Container with credentials - mongodb

I am able to start the mongo image, insert and read data just fine using the snippet below. Similar to the redis example on testcontainers.org.
private static final int MONGO_PORT = 27017;
#ClassRule
public static MongoDBContainer mongo = new MongoDBContainer("mongo:3.2.4")
.withExposedPorts(MONGO_PORT);
By default mongo doesn't have credentials but I'm looking for a way to set the credentials so that my app's MongoClient can get user/password from system properties and connect properly. I've tried adding the root user/password with the below but that didn't set the credentials properly.
#ClassRule
public static MongoDBContainer mongo = new MongoDBContainer("mongo:3.2.4")
.withExposedPorts(MONGO_PORT)
.withEnv("MONGO_INITDB_ROOT_USERNAME", "admin")
.withEnv("MONGO_INITDB_ROOT_PASSWORD", "admin");
My question is: How can I start the test container with a username and password to allow my app to connect to it during my integration test using wiremock.

Checking the docs you can have a GenericContainer rather than specifically a MongoDbContainer (not certain this makes much difference given it looks largely like I've got the same as what you've already tried)...
I've then run:
private static final int MONGO_PORT = 27017;
/**
* https://hub.docker.com/_/mongo shows:
*
* $ docker run -d --network some-network --name some-mongo \
* -e MONGO_INITDB_ROOT_USERNAME=mongoadmin \
* -e MONGO_INITDB_ROOT_PASSWORD=secret \
* mongo
*/
public static GenericContainer mongo = new GenericContainer(DockerImageName.parse("mongo:4.4.1"));
public static void main(String[] args) {
List<String> list = new ArrayList<>();
list.add("MONGO_INITDB_ROOT_USERNAME=mongoadministrator");
list.add("MONGO_INITDB_ROOT_PASSWORD=secret");
list.add("MONGO_INITDB_DATABASE=db");
mongo.setEnv(list);
mongo.withExposedPorts(MONGO_PORT);
mongo.start();
}
the logs from the container show:
docker logs [container_id]:
...
Successfully added user: {
"user" : "mongoadministrator", <<<<<
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
...
I can login successfully inside the container with my new administrative user:
> docker exec -it 3ae15f01074c bash
Error: No such container: 3ae15f01074c
> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
755e214f23d6 mongo:4.4.1 "docker-entrypoint.s…" 2 seconds ago Up 2 seconds 0.0.0.0:32803->27017/tcp elegant_keldysh
cdb4f55930f4 testcontainers/ryuk:0.3.0 "/app" 3 seconds ago Up 3 seconds 0.0.0.0:32802->8080/tcp testcontainers-ryuk-ef84751e-bfd4-41eb-b381-1c1206186eda
> docker exec -it 755e214f23d6 bash
root#755e214f23d6:/# mongo admin -u mongoadministrator
MongoDB shell version v4.4.1
Enter password: <<<<<<<<<<<<<<<< BAD PASSWORD ENTERED HERE
connecting to: mongodb://127.0.0.1:27017/admin?compressors=disabled&gssapiServiceName=mongodb
Error: Authentication failed. :
connect#src/mongo/shell/mongo.js:374:17
#(connect):2:6
exception: connect failed
exiting with code 1
root#755e214f23d6:/# mongo admin -u mongoadministrator
MongoDB shell version v4.4.1
Enter password: <<<<<<<< GOOD PASSWORD secret ENTERED HERE
connecting to: mongodb://127.0.0.1:27017/admin?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("63279398-d9c6-491d-9bd9-6b619dc4a99d") }
MongoDB server version: 4.4.1
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
https://docs.mongodb.com/
Questions? Try the MongoDB Developer Community Forums
https://community.mongodb.com
---
The server generated these startup warnings when booting:
2020-10-24T22:06:24.914+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
---
---
Enable MongoDBs free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
>

In short, you can find working example of test with MongoDB container here.
To provide, more details: you can configure authentication to MongoDB test container by using GenericContainer and setting environment with following properties MONGO_INITDB_ROOT_USERNAME, MONGO_INITDB_ROOT_PASSWORD:
#ClassRule
public static final GenericContainer<?> MONGODB = new GenericContainer<>(DockerImageName.parse(MONGO_IMAGE))
.withEnv("MONGO_INITDB_ROOT_USERNAME", USERNAME)
.withEnv("MONGO_INITDB_ROOT_PASSWORD", PASSWORD)
.withEnv("MONGO_INITDB_DATABASE", TEST_DATABASE)
.withExposedPorts(MONGO_PORT);
Then you should use MongoClient with corresponding MongoCredential. For example, below is the test, that writes and reads document to/from MongoDB container.
#Test
public void shouldWriteAndReadMongoDocument() {
ServerAddress serverAddress = new ServerAddress(MONGODB.getHost(), MONGODB.getMappedPort(MONGO_PORT));
MongoCredential credential = MongoCredential.createCredential(USERNAME, AUTH_SOURCE_DB, PASSWORD.toCharArray());
MongoClientOptions options = MongoClientOptions.builder().build();
MongoClient mongoClient = new MongoClient(serverAddress, credential, options);
MongoDatabase database = mongoClient.getDatabase(TEST_DATABASE);
MongoCollection<Document> collection = database.getCollection(TEST_COLLECTION);
Document expected = new Document("name", "foo").append("foo", 1).append("bar", "string");
collection.insertOne(expected);
Document actual = collection.find(new Document("name", "foo")).first();
assertThat(actual).isEqualTo(expected);
}
Notes:
To find more details on environment variables, you check documentation here (specifically, Environment Variables section)
To find more details on authenticated connection to MongoDB, you check documentation here

Related

Flyway not able to find role with postgres docker

I am trying to run my first flyway example using docker postgres image but getting the following error:
INFO: Flyway Community Edition 6.4.2 by Redgate
Exception in thread "main" org.flywaydb.core.internal.exception.FlywaySqlException:
Unable to obtain connection from database (jdbc:postgresql://localhost/flyway-service) for user 'flyway-service': FATAL: role "flyway-service" does not exist
-------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State : 28000
Error Code : 0
Message : FATAL: role "flyway-service" does not exist
at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:65)
at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:80)
I looked up into the docker container and can see that the user role flyway-service is created as part of the docker-compose execution:
$ docker exec -it flywayexample_postgres_1 bash
root#b2037e382112:/# psql -U flyway-service;
psql (12.2 (Debian 12.2-2.pgdg100+1))
Type "help" for help.
flyway-service=# \du;
List of roles
Role name | Attributes | Member of
----------------+------------------------------------------------------------+-----------
flyway-service | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
flyway-service=#
Main class is:
public static void main( String[] args ) {
var flyway = Flyway.configure().schemas("flyway_test_schema")
.dataSource("jdbc:postgresql://localhost/flyway-service", "flyway-service",
"password")
.load()
.migrate();
System.out.println( "Flyway example's hello world!" );
}
}
The migration called src/main/resources/db/migration/V1__Create_person_table.sql:
create table PERSON (
ID int not null,
NAME varchar(100) not null
);
Docker-compose yml file:
version: "3.8"
services:
postgres:
image: postgres:12.2
ports: ["5432:5432"]
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=flyway-service
I am running this code on MAC OSX. I assume, I am missing something obvious here, but not sure what! Any pointers would be appreciated.
Finally managed to figure out the issue with the help of a friend! The problem was not with the attached code but with a postgres daemon process running on the same port 5432 by an old Postgres installation.
I found the complete uninstallation procedure here. After removing the additional daemon process got only one port listening.
a$ lsof -n -i4TCP:5432
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 654 root 50u IPv6 0x7ae1b5f8fbcf1cb 0t0 TCP *:postgresql (LISTEN)

what is client property of knexfile.js

In knex documentation of configuration of knexfile.js for PostgreSQL, they have a property called client, which looks this way:
...
client: 'pg'
...
However, going through some other projects that utilize PostgreSQL I noticed that they have a different value there, which looks this way:
...
client: 'postgresql'
...
Does this string correspond to the name of some sort of command line tool that is being used with the project or I misunderstand something?
Postgresql is based on a server-client model as described in 'Architectural Fundamentals'
psql is the standard cli client of postgres as mentioned here in the docs.
A client may as well be a GUI such as pg-admin, or a node-package such as 'pg' - here's a list.
The client parameter is required and determines which client adapter will be used with the library.
You should also read the docs of 'Server Setup and Operation'
To initialize the library you can do the following (in this case on localhost):
var knex = require('knex')({
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'your_database_user',
password : 'your_database_password',
database : 'myapp_test'
}
})
The standard user of the client deamon ist 'postgres' - which you can use of course, but its highly advisable to create a new user as stated in the docs and/or apply a password to the standard user 'postgres'.
On Debian stretch i.E.:
# su - postgres
$ psql -d template1 -c "ALTER USER postgres WITH PASSWORD 'SecretPasswordHere';"
Make sure you delete the command line history so nobody can read out your pwd:
rm ~/.psql_history
Now you can add a new user (i.E. foobar) on the system and for postgres
# adduser foobar
and
# su - postgres
$ createuser --pwprompt --interactive foobar
Lets look at the following setup:
module.exports = {
development: {
client: 'xyz',
connection: { user: 'foobar', database: 'my_app' }
},
production: { client: 'abc', connection: process.env.DATABASE_URL }
};
This basically tells us the following:
In dev - use the client xyz to connect to postgresqls database my_app with the user foobar (in this case without pwd)
In prod - retrieve the globalenv the url of the db-server is set to and connect via the client abc
Here's an example how node's pg-client package opens a connection pool:
const pool = new Pool({
user: 'foobar',
host: 'someUrl',
database: 'someDataBaseName',
password: 'somePWD',
port: 5432,
})
If you could clarify or elaborate your setup or what you like to achieve a little more i could give you some more detailed info - but i hope that helped anyways..

Authenticating with MongoDB from JavaScript file

I'm trying to write a MongoDB script as part of automating customer data deletion in accordance with GDPR (current process is manual), but I'm having problems authenticating from the script. Authentication works perfectly from the command line, but not from a script.
I'm not certain which mechanism I should use, but since there are only two that don't require external files (which I don't use in the command line), I figured I'd try both.
When running from the command line, this is the command I use:
$ mongo -u [adminusr] -p '[adminpwd]' --authenticationDatabase admin
This script:
var host = '127.0.0.1';
var port = '27017';
var user = 'admin';
var pwd = 'secrets';
var authDB = 'admin';
try {
var conn = new Mongo('mongodb://' + host + ':' + port + '/' + authDB);
conn.auth({
user: user,
pwd: pwd,
mechanism: 'SCRAM-SHA-1',
digest: false
});
// deletion commands...
} catch (e) {
print(e);
}
gives this error:
$ mongo erasure.js
MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.10
Error: Authentication failed.
Changing the auth part to this:
conn.auth({
user: user,
pwd: pwd,
mechanism: 'MONGODB-CR',
db: authDB,
digest: false
});
gives me this error:
$ mongo erasure.js
MongoDB shell version v3.4.2
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.10
Error: auth failed
The official documentation on writing MongoDB scripts and authentication is also very scarce on how to do this.
How can I get the auth to work?
Thanks.

MongoDB db.serverStatus() gives error when running using tunnel that is targeted to api.cloudfoundry.com

Following is the console session...
C:\Users\xxx>vmc tunnel myMongoDB
Getting tunnel connection info: OK
Service connection info:
username : uuuu
password : pppp
name : db
url : mongodb://uuuu:pppp#172.30.xx.xx:25200/db
Starting tunnel to myMongoDB on port 10000.
1: none
2: mongo
3: mongodump
4: mongorestore
Which client would you like to start?: 2
Launching 'mongo --host localhost --port 10000 -u uuuu -p pppp db'
MongoDB shell version: 2.0.6
connecting to: localhost:10000/db
> db.serverStatus()
{ "errmsg" : "need to login", "ok" : 0 }
>
Which credentials should I use to login (assuming should use db.auth) to get rid of the error "{ "errmsg" : "need to login", "ok" : 0 }".
When I run the same in micro CF on my machine it works ok and gives me the expected output.
P.S. I'm trying this to get to know the current connections on my application, written in node.js. Trying to debug some issues with connections to the DB. If there is any other alternative that I can use please suggest that as well.
this should work! Not sure why your tunnel isn't connecting, my immediate suggestion is to try another instance of MongoDB and see if the same error occurs.
If you are trying to inspect the bound services on you node.js app you should be able to inspect them in the VCAP_SERVICES environment variable. For example;
var http = require('http');
var util = require('util');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(util.inspect(process.env.VCAP_SERVICES));
res.write("\n\n************\n\n");
res.end(util.inspect(req.headers));
}).listen(3000);
This code is currently running at http://node-headers.cloudfoundry.com/ to serve as an example. However, mongodb connections for node.js applications should automatically configure to the bound service. If this does not work for you, then please do let me know.

nodejs chat example does not work

I've come across a node chat example on github, When I try to run it, I see the following error:
Error connecting to mongo perhaps it isn't running ?
I've installed mongo 0.9.2, nodejs 5.2 pre, npm 3.0 and other dependencies. The example can be found here: https://github.com/gregstewart/chat.io
I can not determine whether if the example not really works or I didn't run it right. Please help.
Did you install and start mongo-db on your system? This error is mostly because of a missing mongo instance running on the local machine.
Check out the follwing code excerpts from chat.io.
main.js:
/**
* Configure the user provider (mongodB connection for user data storage)
*/
var userProvider = new UserProvider('localhost', 27017);
Creates a new UserProvider object using host and port for database (localhost:27017, mongo-db default).
UserProvider.js:
UserProvider = function(host, port) {
this.db = new mongo.Db('node-mongo-chat', new Server(host, port, {auto_reconnect: true}, {}));
this.db.addListener('error', function(error) {
console.log('Error connecting to mongo -- perhaps it isn\'t running?');
});
this.db.open(function() {
});
};
Opening the connection to the server, printing out an error on failure (the error you mentioned above).
Consider reading up on the mongo-db docs concerning installation and setup here