How do I get the full uri including username and password with the mongodbatlas provider in terraform - mongodb

When I try to output the mongodb uri with Terraform and the mongodb atlas provider, I can't get the full uri with username and password. For example, when I do something like:
terraform {
required_version = "~> 0.14.7"
required_providers {
mongodbatlas = {
source = "mongodb/mongodbatlas"
version = "0.8.2"
}
}
}
provider "mongodbatlas" {
public_key = var.mongodbatlas_public_key
private_key = var.mongodbatlas_private_key
}
data "mongodbatlas_cluster" "db" {
project_id = var.mongodbatlas_project_id
name = format("some-db-name-%s", var.env)
}
output "db_url" {
value = data.mongodbatlas_cluster.db.connection_strings[0].address_srv
}
I always get a uri of the form: mongodb+srv://some-db-name-staging.xjcol.mongodb.net
Adding that as an environment variable to my web app in order to connect to db does not work as it needs to authenticate with a username and password. Manually adding the username and password to that string as in mongodb+srv://[username]:[password]#some-db-name-staging.xjcol.mongodb.net works and the app can connect to the db fine.

While I get what you're trying to achieve, I suspect you're mixing things here. Let me explain:
MongoDB allows you to create database users that are able to authenticate using password. Those can be created using mongodbatlas_database_user resource.
You can create your cluster (or source cluster information) using both the resource or data source the way you're trying to achieve it.
However, cluster creation is independent of database and database user creation, meaning that what you're getting from Terraform is just a generic connection string from Mongo where not even Mongo knows which user/database you want to connect to.
I suggest you to compose your own connection string and pass it along to your application using a post-provisioning script, either using your Terraform outputs of cluster and database user, or simply composing it by yourself if you already know the info upfront.
In case you're using AWS, MongoDB Atlas supports connection strings using IAM Users and IAM Roles. This is a much better, safer approach than dealing with passwords and all the extra burden managing passwords implies. If this sounds like something you'd like to explore, do let me know.

My solution was to use the string replace function with mongodbatlas_database_user resource:
replace(mongodbatlas_advanced_cluster.mongodb_cluster.connection_strings[0].standard_srv, "mongodb+srv://", "mongodb+srv://${mongodbatlas_database_user.userspace_db_user.username}:${coalesce(nonsensitive(mongodbatlas_database_user.userspace_db_user.password), "null")}#")

Related

Hashicorp Vault Kerberos Auth with VaultSharp

I'm having great difficulty getting Kerberos Auth working with Vault using VaultSharp.
I don't have control over Vault server but I've been informed that it is configured and ready to use.
I'm using .NET running in IIS and I want to make use of the service account that IIS is running under so that I don't need to store additional secrets or user/passwords.
Here is the code I'm using and the error:
public string GetSecretWithKerberosAuthUsingVaultSharp(string keyName, string vaultBaseAddress, string vaultResourcePath, string mountPoint)
{
IAuthMethodInfo authMethod = new KerberosAuthMethodInfo(); // uses network credential by default.
var vaultClientSettings = new VaultClientSettings(vaultBaseAddress, authMethod);
IVaultClient vaultClient = new VaultClient(vaultClientSettings);
var result = vaultClient.V1.Secrets.KeyValue.V2.ReadSecretAsync(vaultResourcePath, mountPoint: mountPoint).Result;
//Above line gives this error message:
//{"request_id":"a85dfbb3-b283-3513-7cd3-01ad757eed1b","lease_id":"","renewable":false,"lease_duration":0,"data":null,"wrap_info":null,"warnings":["Unauthorised.\n\n"],"auth":null}
var resultData = result.Data;
string secret = resultData.Data[keyName].ToString();
return secret;
}
I have managed to get it working using token auth as well as through the CLI but that is not quite what I want.
authMethod.Credentials.UserName/Domain both are empty strings.
Don't know if they are supposed to be populated in this case or not but documentation states that it "uses network credentials by default"
Any help appreciated.
Is your web application running in integrated Windows Auth mode, with anonymous auth disabled?
If no, please make it work in that mode for your web app to have the Windows Integrated Auth context so that web calls from VaultSharp to Vault API can have the security context.
If yes, then can you please try a couple of things?
var kerberosAuthInfo = new KerberosAuthMethodInfo(CredentialCache.DefaultCredentials);
If the above doesn't work, then can you try explicit credentials.
var kerberosAuthInfo = new KerberosAuthMethodInfo(new NetworkCredential(userName, password, domain));
Ideally, the web app context should carry the integrated windows context so that you don't need to provide explicit credentials, but it might be worth trying to ensure that it works first and then we can backtrack as to why the context is not being passed.

Connecting to access controlled MongoDB replicaSet from Python Eve

I have an Eve application that I want to connect to a mongodb replicaset. I was able to do thisby setting the MONGO_URI parameter in settings.py to mongodb://host/dbName?replicaSet=replicaSetname
however, now that I have enabled access control on the replica set, I can't seem to pass the user credentials in.
I am using a uri of the form:
mongodb://user:password#host/dbName?replicaSet=replicaSetName&authSource=admin
Most infuriatingly of all, when I serve this URI directly to the database, it connects and logs in without issue. But if I try to access something through eve I get the following error:
pymongo.errors.OperationFailure: Authentication failed.
I have tried several variations, including settings the MONGO_USERNAME and MONGO_PASSWORD parameters in eve. The only possible issue I can think of is that the user I created for eve has limited permissions (readWrite on one database).
After much effort I managed to figure out how to do it on eve, it involved a few settings that were not mentioned in the eve documentation:
MONGO_HOST = 'mypod-1.mongo,mypod-2.mongo,mypod-3.mongo;
MONGO_USERNAME = 'user'
MONGO_PASSWORD = 'pass'
MONGO_DBNAME = 'mydb'
MONGO_AUTH_SOURCE = 'admin'
MONGO_REPLICA_SET = 'mySet'
And also, because this tripped me up, don't have MONGO_URI set to anything in your settings, not even None, as eve will attempt to use the URI over any other setting if it finds that variable declared.

Newbie help - how to connect to AWS Redshift cluster (currently using Aginity)

(I'm afraid I'm probably about to reveal myself as completely unfit for the task at hand!)
I'm trying to setup a Redshift cluster and database to help manage data for a class/group project.
I have a dc2.large cluster running with either default options, or what looked like the most generic in the couple of place I was forced to make entries.
I have downloaded Aginity (Win64) as it is described as being specialized for Redshift. That said, I can't find any instructions for connecting using it. The connection dialog requests the follwoing:
Server: using the endpoint for my cluster (less :57xx at the end).
UserID: the Master username for the database defined for the cluster.
Password: to match the UserID
SSL Mode (Disable, Allow, Prefer, Require): trying various options
Database: as named in cluster setup
Port: as defined in cluster setup
I can't get it to connect ("failed to establish connection") and don't know if I'm entering something wrong in Aginity or if I haven't set up my cluster properly.
Message: Failed to establish a connection to 'abc1234-smtm.crone7m2jcwv.us-east-1.redshift.amazonaws.com'.
Type : Npgsql.NpgsqlException
Source : Npgsql
Trace : at Npgsql.NpgsqlClosedState.Open(NpgsqlConnector context, Int32 timeout)
at Npgsql.NpgsqlConnector.Open()
at Npgsql.NpgsqlConnection.Open()
at Aginity.MPP.Common.BaseDataProvider.get_Connection()
at Aginity.MPP.Common.BaseDataProvider.CreateCommand(String commandText, CommandType commandType, IDataParameter[] commandParams)
at Aginity.MPP.Common.BaseDataProvider.ExecuteReader(String commandText, CommandType commandType, IDataParameter[] commandParams)
--- Inner Exception: ---
......
It seems there is not enough information going into Aginity to authorize connection to my cluster - no account credential are supplied. For UserID, am I meant to enter the ID of a valid user? Can I use the root account? What would the ID look like? I have setup a User with FullAccess to S3 and Redshift, then entered the UserID in this format
arn:aws:iam::600123456789:user/john
along with the matching password, but that hasn't worked either.
The only training/tutorial I have been able to find/do on this is the Intro AWS direct you to, at https://qwiklabs.com/focuses/2366, which uses a web-based client that I can't find outside of the tutorial (pgweb).
Any advice what I am doing wrong, and how to do it right?
Well, I think I got it working - I haven't had a chance to see if I can actually create table yet, but it seems to be connected. I had to allow inbound traffic from outside the VPC, as per the above snapshot.
I'm guessing there's a better way than opening it up to all IP addresses, but I don't know the users' (fellow team members) IPs, and aren't they all subject to change depending on the device they're using to connect?
How does one go about getting inside the VPC to connect that way, presumably more securely?

Authentication DB setting not working using mongoDB URI configuration

I am triyng to connect pyeve with a MongoDB Atlas replica set (https://cloud.mongodb.com/). I've connected successfully DB management tools from the same host, to make sure the deployment is working OK.
One particularity is that using Atlas, all users must authenticate against auth database, I cannot put my users in the application database, so I need to set authSource in MONGO_URI.
Now, when defining the MONGO_URI for the replica set, in settings.py, like this:
MONGO_URI = mongodb://<USER>:<PASS>#my-shard-00-00-tlati.mongodb.net:27017,my-shard-00-01-tlati.mongodb.net:27017,my-shard-00-02-tlati.mongodb.net:27017/<MY_DB>?ssl=true&replicaSet=my-shard-0&authSource=admin
The authSource=admin parameter seems to be ignored, (I've checked debugging pymongo's auth and the authentication source used is None).
MONGO_AUTH_SOURCE could be used to set the authorization database, but it has no effect since MONGO_URI is used in preference of the other configuration variables, according to eve's documentation.
Is this an issue or am I doing it wrong?
Found out that the problem was that I was using version 0.4.1 for flask-pymongo. Updating it to version 0.5.1 fixed the problem.

Hosting the database separately for Meteor apps

It seems to be a common and safer practice to host the database separately from Meteor apps. That is to say, have an EC2 instance for your Meteor app, and an EC2 instance for your MongoDB, and make them talk to one another.
From what I understand, people do this because it's more secure, and it allows them to deploy newer versions of their app without touching the database.
I'd like to do this with Amazon EC2 alone, as opposed to using another 3rd party service, like Compose.io.
How can I host a Meteor app and its database separately on two EC2 instances, and have them communicate with one another?
It is common practice, and people mostly do it because it offers you the ability to scale them both independently.
As to the how, you'll want to obviously configure each of your Amazon EC2 instances, installing meteor on one, and MongoDB on the other. You'll also need to configure your VPC (Amazon Virtual Private Cloud) so that your MongoDB instance accepts incoming connections on whatever port you specify (default is 27017), so that your Meteor Application can connect.
After that it's just a matter of telling your meteor app where to go to get the database connection. The most secure way of doing this will be to set a couple Environment Variables, named MONGODBSERVER and MONGODBPORT, DBUSER, DBPASSWORD, etc.
You'll then want to set some variables in your server Meteor code, using something like:
Meteor.startup(function() {
var DbUser = process.env.DBUSER;
var DbPassword = process.env.DBPASSWORD;
var MongoDBServer = process.env.MONGODBSERVER;
var MongoDBPort = process.env.MONGODBPORT;
});
And if you're using the native MongoDB Driver, connecting becomes trivial:
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect('mongodb://DbUser:DbPassword#MongoDBServer:MongoDBPort/databasename', function(err, db) {
...
});
Then it's just a matter of constructing your Mongo models using something like:
Temperatures = new Mongo.Collection('temperatures');
Temperatures._ensureIndex({temp: 1, time: 1});
And then taking action on those models in regard to the database:
Temperatures.insert({temp: ftemp, time: Math.floor(Date.now() / 1000)});
I'll also mention that http://modulus.io is a really decent Meteor hosting solution. I'd recommend them, unless you are stuck on using Amazon EC2 instances, which is fine, but more complicated for a simple application.
You need to set an Environment Variable for Mongo where it is hosted
MONGO_URL
mongodb://:#hostingproviderurl:port/xxx?autoReconnect=true&connectTimeoutMS=60000
the correct mongodb:// url string would be provided by the mongodb hosting provider.