We are going to deploy MongoDB using Chef. We have found a handful of cookbooks that take care of installing MongoDB on a server. However, none of the cookbooks we've found handle MongoDB user setup.
We would preferably store the auth info for MongoDB users in encrypted data bags.
Does a cookbook that handles this exist?
I don't believe there are any public cookbooks that handle that currently. You would probably want to build on the following two cookbooks:
https://github.com/edelight/chef-mongodb
Already has a library that connects to the Mongo server. You'd probably want to use it as an example for how to interact with mongo within a chef library
https://github.com/opscode-cookbooks/database
The master recipe is designed for creating databases and users from a databag (adapting for encryption would be trivial)
There are good library providers for interacting with several types of databases. You would just use the connection examples from the first cookbook to reimplement those providers for mongo.
Related
As a part of my thesis project, I have been given a MongoDB dump of size 240GB which is on my external hard drive. I'll have to use this data to run my python scripts for a short duration. However, since my dataset is huge and I cannot mongoimport on my local mongodb server (since I don't have enough internal memory), my professor gave me a $100 google cloud platform coupon so I can use the google cloud computing resources.
So far I have researched that I can do it this way:
Create a compute engine in GCP and install mongodb on remote engine. Transfer the MongoDB dump to remote instance and run the scripts to get the output.
This method works well but I'm looking for a method to create a remote database server in GCP so I that I can run my scripts locally, which is something like one of the following.
Creating a remote mongodb server on GCP so that I can establish a remote mongo connection to run my scripts locally.
Transferring the mongodb dump to google's datastore so then I can use the datastore API to remotely connect and run my scripts locally.
I have given a thought of using MongoDB atlas but because of the size of the data, I will be billed hugely and I cannot use my GCP coupon.
Any help or suggestions on how of either of the two methods can be implemented is appreciated.
There is 2 parts to your question
First, you can create a compute engine VM with MongoDB installed and load your backup on it. Then, open the right firewall rules for allowing the connexion from your local environment to the Google Compute Engine VM. The connexion will be performed with a simple login/password.
You can use a static IP on your VM. By the way, in case of reboot on the VM you will keep the same IP (and it will be easier for your local connexion).
Second, BE CAREFUL to datastore. It's a good product, serverless NoSQL database, document oriented, but it's absolutely not the MongoDB equivalent. You can't perform aggregate, you are limited in search capabilities,... It's designed for specific use case (I don't know yours, but don't think that is the MongoDB equivalent!).
Anyway, if you use Datastore, you will have to use a service account or to install Google Cloud SDK on your local environment to be authenticated and to be able to request Datastore API. No login/password in this case.
Situation:
Have created today a new Compose for MongoDB Service instance in Bluemix
Need:
I have to access this MongoDB DIRECTLY with tools (eg. Mongo Managemant Studio Pro, mongo.exe, etc.) for bulkloading, testing, ad-hoc data fix, etc.
Problem:
I have not found any docs, samples nor a CLEAR statement that
a) gives me some confirmation that THIS is possible
b) gives me COMPLETE information (not just some technical fragments that might have worked year ago) how to do it.
Maybe I am looking to the wrong places or do not know the right people. However I am stuck on this, and before quitting Bluemix MongoDB maybe somebody has a copy/past solution or handson step by step manual.
Any help welcome. Thanks!
Connecting to MongoDB service in Bluemix from an application is possible. For this answer I have used the application "Robo3T" and here are the steps:
Access your MongoDB Service on you Bluemix account. Usually under
"Cloud Foundry Services"
Open section "Manage", from "Connection Settings" copy from "HTTPS" the connection address and port. In this example "sl-eu-lon-2-portal.5.dblayer.com" and "20651"
In Robo3T create a new connection with the connection address from previous step
In tab Authentication configure database name, username and password
. The credentials are found as in step 1
From "Connection Settings" copy the SSL Certificate into a text file and save locally.
In Robo3T Add the certificate to the connection in the "SSL" tab
Test the connection and save the settings
Answer
YES, Bluemix hosted Compose for MongoDB instances can be connected from the mongo Shell and some updated DB Managment tools.
However, you have to make sure, that in case you are running the newest DB versions, that your tools (shell and DB management GUIs) comply with the newest DB features such as encryption etc.
Origin of the Problem
My problem was due to older and therefore incompatible versions of the mongo shell and DB-managment tools running against the newest MongoDB versions with their specialities on encription and multiple servers to be handled in the URI.
At least two DB managment tools are not compatible with the newest DB version and will take their time to get fixed. The problem is, that both will not tell you about this. They just do not not connect. No logs on either side. Period.
So my advise here: look for tool providers who express dedicated compliance with the specific version of your DB.
Advise to the Bluemix Team
It might not take much time to provide some sample connection strings for the most common tools like the mongo shell, MongoBooster, etc. to take the hassle and guesswork out of interpreting the Environment variables and figuring out what is needed for specific connection strings and what is not.
For instance MongoDB Atlas hosting provides for every cluster readymade connection strings for many tools you can just copy/past and done!
Connecting to Atlas took me 5 Minutes. For Bluemix I have lost hours! Not because it is complex, but because the documentation and the generated Info is somehow incomplete and messy - at least for the ones who do not connection strings for their living!
We are working on Sitecore deployment in Azure.
Sitecore Experience Platform 8.0 rev. 160115
MongoDB - 3.0.4
We installed MongoDB, and we can connect to localhost using Robomongo. We can only see “Analytics” database/collections.
Our connection strings setup are:
Connectionstring.config
But the other 3 databases and collections are not created.
Tracking.live
Tracking.history
Tracking.contact
In Sitecore.Analytics.config file – the setting “Analytics.Enabled” is set to true.
Sitecore.Analytics.config
In log we found some references to xDB cloud initialization failed issues, therefore we disabled it.
Are we missing any configurations? Any help or suggestions are appreciated.
Thank you
Keep in mind that MongoDB is schemaless. Of course, in a production environment you would probably have to create these databases manually - to ensure that access rights are assigned correctly. But in a development environment, any database can be created on the fly.
The only reason the analytics database was created for you is because Sitecore creates indexes for the Interactions collection. Otherwise, you wouldn't see this database until xDB wrote some data into it. Same goes for any MongoDB collection - those won't appear until there's either data being written or an index created.
The other three databases will be created once the aggregation/processing logic is executed. I.e. when your instance starts to actually collect and process visit data.
As a conclusion, don't worry about these databases missing (for now). Just verify that xDB functionality is working properly.
I've been following a tutorial with express, node and mongo.
I have in a config file on the server side:
production:{
db:'mongodb://MYUSERNAME:MYPASSWORD#ds033307.mongolab.com:33307/dbname',
rootPath:rootPath,
port:process.env.PORT||80
}
so, i have my username and password in clear text in a server side javascript file. should i be worried about this? if yes, where else can I put it?
Thanks.
Edit: I went back and had a look at mongolab and heroku (where my site is hosted) docs.
Where I found: "The MongoLab add-on contributes one config variable to your Heroku environment: MONGOLAB_URI", and so I was able to put the MONGOLAB_URI env var into my config and move the password out of the source code.
With regards to the same datacenter, am I right to assume heroku would not be hosting my mongolab database in their datacenter, but would instead be calling out to a cloud service mongo database? Not much I can do then, is there, if I want to stick with mongolab and heroku?
I know this question is old but according to Heroku's docs they currently use 2 datacenters (https://devcenter.heroku.com/articles/regions#data-center-locations).
Their US server is 'amazon-web-services::us-east-1' and their EU alternative is 'amazon-web-services::eu-west-1'.
Both of these data centers are available when launching mongo instances on Mongolab so you can choose for both your app and your db to be on the same datacenter giving much improved security.
I think you should always be concerned about storing passwords in source code files. Generally you would be much better off keeping it in a configuration file that is managed separately. This gives you the flexibility to use the same code with a different configuration file to point to development or qa databases.
Of bigger concern perhaps - are you hosting your application in the same datacenter that MongoLab is hosting your database? If not, that user name and password, along with your data, will traverse the internet in the clear.
MongoLab does not currently support SSL (other than for their RestAPI) so even they recommend being in the same data center:
Do you support SSL?
Not yet but it is on our roadmap to be available in Summer 2014. In
the meantime, we highly recommend that you run your application and
database in the same datacenter. If you have a Dedicated plan, we also
highly recommend that you configure custom firewall rules for your
database(s).
Rest API:
Each MongoLab account comes with a REST API that can be used to access
the databases, collections and documents belonging to that account.
The API exposes most the operations you would find in the MongoDB
driver, but offers them as a RESTful interface over HTTPS.
I would definitely read MongoLab's security page fairly closely:
https://docs.mongodb.com/manual/security/
How can I acces MongoDB service running on Appfog from my local? I want to use it with gui application. Is it possible?
I am assuming that you are using an AppFog MongoDB add-on and that the GUI tool you want to use is running locally on your machine, is that correct?
It's not clear to me what GUI application you're trying to use, but chances are it accepts a MongoDB URI of the form "mongodb://[user]:[pass]#[host]:[port]/[dbname]" You can use this info to connect through the db shell, too. Or if your tool doesn't take a URI, you can cut&paste specific values from the URI according to the format.
If you are using an AppFog MongoDB addon, then you can get the value for your URI from the Env Variables section of your app page at the AppFog portal. You can also get that value in your code, per something like https://docs.appfog.com/add-ons/mongolab.
You should also keep in mind that MongoDB services often have their own management UI that you can use. If you go to the Addons section of your application page on AppFog, scroll to your provider and click "Manage" you can access their tools as well.
Good luck, and let me know if I have misunderstood your question.
Sincerely,
Eric