I need to "kerberize" our Greenplum cluster. One of the aspects of this is that I should kerberize the interface between the GP master and its Segment Hosts. I have been unable to determine if this is supported or not.
I have seen the parameters in the posgresql.conf file (krb_server_keyfile and krb_srvname) and have tried to set these, but it does not seem to work (Greenplum still works, it just does not appear the connection is kerberized).
I did this with hadoop and it was pretty straight forward, but, again, cannot figure out how to do it in GP or if it is even possible. Any ideas?
Thanks
Refer to Greenplum HD Manager 1.2 Installation and User Guide for instructions on how to deploy Kerberos. The document is related to Hadoop, but should serve for a general Greenplum install.
So... the answer, as near as I can tell is this:
First, for clarification, there are two places where I am required to "kerberize" GP. The first in master/slave connectivity. This turned out the be easy enough after I learned this communication is ssh based. I just switched the rsa/dsa generated passwordless authorization with Kerberos SSH. I am not sure this is really any more or less secure, but a requirement none-the-less. The second is locking down the administrative/jdbc access. This should easy, after all GP is based upon Postgres, I have have secured Postgres with Kerberos in the past. Unfortuntaly, GP is based upon Postgres 8.2. This was before GSS support for Kerberos was added to Postgres, and I cannot get this to work. I am not positive that it can. Maybe GP will upgrade to 8.4 (at a minimum) soon and I can try that.
Related
Has anyone managed to get Debezium to work over AWS RDS Postgres with rds.force_ssl turned on in the parameter group?
The connector seems to work for a bit, and then we begin to receive errors like Database connection failed when writing to copy and Exception thrown while calling task.commit().
I have scoured the web searching for this issue, and I see many people encountered it, and many Jira issues opened about it.
The response generally is "Check your network configuration" or "Disable your SSL".
I just can't get it to work for some reason, and obviously disabling encryption in transit is not possible in production use cases (at least in ours).
I would appreciate any kind of help or insight into how to solve this!
Situation:
Have created today a new Compose for MongoDB Service instance in Bluemix
Need:
I have to access this MongoDB DIRECTLY with tools (eg. Mongo Managemant Studio Pro, mongo.exe, etc.) for bulkloading, testing, ad-hoc data fix, etc.
Problem:
I have not found any docs, samples nor a CLEAR statement that
a) gives me some confirmation that THIS is possible
b) gives me COMPLETE information (not just some technical fragments that might have worked year ago) how to do it.
Maybe I am looking to the wrong places or do not know the right people. However I am stuck on this, and before quitting Bluemix MongoDB maybe somebody has a copy/past solution or handson step by step manual.
Any help welcome. Thanks!
Connecting to MongoDB service in Bluemix from an application is possible. For this answer I have used the application "Robo3T" and here are the steps:
Access your MongoDB Service on you Bluemix account. Usually under
"Cloud Foundry Services"
Open section "Manage", from "Connection Settings" copy from "HTTPS" the connection address and port. In this example "sl-eu-lon-2-portal.5.dblayer.com" and "20651"
In Robo3T create a new connection with the connection address from previous step
In tab Authentication configure database name, username and password
. The credentials are found as in step 1
From "Connection Settings" copy the SSL Certificate into a text file and save locally.
In Robo3T Add the certificate to the connection in the "SSL" tab
Test the connection and save the settings
Answer
YES, Bluemix hosted Compose for MongoDB instances can be connected from the mongo Shell and some updated DB Managment tools.
However, you have to make sure, that in case you are running the newest DB versions, that your tools (shell and DB management GUIs) comply with the newest DB features such as encryption etc.
Origin of the Problem
My problem was due to older and therefore incompatible versions of the mongo shell and DB-managment tools running against the newest MongoDB versions with their specialities on encription and multiple servers to be handled in the URI.
At least two DB managment tools are not compatible with the newest DB version and will take their time to get fixed. The problem is, that both will not tell you about this. They just do not not connect. No logs on either side. Period.
So my advise here: look for tool providers who express dedicated compliance with the specific version of your DB.
Advise to the Bluemix Team
It might not take much time to provide some sample connection strings for the most common tools like the mongo shell, MongoBooster, etc. to take the hassle and guesswork out of interpreting the Environment variables and figuring out what is needed for specific connection strings and what is not.
For instance MongoDB Atlas hosting provides for every cluster readymade connection strings for many tools you can just copy/past and done!
Connecting to Atlas took me 5 Minutes. For Bluemix I have lost hours! Not because it is complex, but because the documentation and the generated Info is somehow incomplete and messy - at least for the ones who do not connection strings for their living!
I'm using the datanucleus mongodb maven plugin and "access platform" for connecting my java app to mongodb using JPA.
I've followed the instructions on http://docs.mongodb.org/manual/tutorial/deploy-replica-set/
on a ubuntu VM, added db1.mongo, db2.mongo and db3.mongo into the hosts file on both the guest vm and the host (Mac OS X).
I got a simple java app connecting to the servers, (as described in http://www.datanucleus.org/products/accessplatform_3_0/mongodb/support.html).
When I connect the app to the primary (connection url: mongodb:db1.mongo:27017/ops?replicaSet=rs0) everything works just fine, but when I add the other two mongodb's to the connection url, so it becomes mongodb:db1.mongo:27017/ops?replicaSet=rs0,db2.mongo:27018,db3.mongo:27019 I get the exception:
com.mongodb.MongoException: can't find a master
at com.mongodb.DBTCPConnector.checkMaster(DBTCPConnector.java:503)
at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:236)
at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:216)
...
I've searched for this error, but the ones I have found concerns use of localhost/127.0.0.1. I tried to mitigate that by running mongodb on a separate VM and thus a non-local IP as well as adding the names to the hosts file.
The primary goal with trying mongodb is to achieve availability so replication and being able to failover is extremely important. Transactions and consistency between nodes in case of failure is not a problem, neither are we concerned about loosing an update or two once in a while so mongodb looks like a good alternative using JPA (I'm utterly fed up with mysql :-)
Thanks in advance for your help!
I used multiple MongoDB servers when I originally wrote that support and worked back then. Not got time now, but you can look at the DataNucleus code that parses your datastore connection URL and converts it into MongoDB java API calls. Should strip the servers apart and then call "new Mongo(serverAddrs);". If its passing it in correctly (debugger?), then the problem is possibly Mongo-specific, as opposed to what DataNucleus does for you.
Also make sure you're using v3.1.2 (or later) of datanucleus-mongodb
I think you've misformatted your MongoDB URI. Instead of this:
mongodb:db1.mongo:27017/ops?replicaSet=rs0,db2.mongo:27018,db3.mongo:27019
Do this:
mongodb:db1.mongo:27017,db2.mongo:27018,db3.mongo:27019/ops?replicaSet=rs0
Am working on a project using python/flask/mongodb, and needed to figure out the hosting/infrastructure.
Prefer to use VPS'es instead of services like Heroku.
Which in your opinion is the better option ?
a) Linode app/web server + mongoHQ
b) Linode app/web server + mongodb & replicas setup on a few Linodes
c) AWS EC2 with MongoHQ (sicne mongohq is hosted on EC2 as well)
d) Any other advisable suggestions?
I am experienced in Linux/DB in general, hence more concerned about the maintenance efforts, distracting me from delivering actual application features.
Thanks.
This is a hard question to answer as there's no correct answer to this.
This all depends on your traffic, application uptime needs and ultimately your manpower and infrastructure.
If you're a one person startup and with limited Mongo DB knowledge (ie setting up replicasets, sharding etc) then go with Mongo HQ. I have done some test instances with Heroku as the app layer and they work well enough.
As you grow it might be a good idea to hire a dedicated mongo person and/or sign up for paid 10gen support which also gives you access to their monitoring console and other goodies to help management.
My only suggestion would be to watch your queries... When generating search result pages, specify only the fields you want and need from the db... Your throughout and latency aye generally a bigger cost for queries from remote systems than execution time for a query.
Also see if MongoLab or MongoHQ have in site or hosting from a priority connection to your vps site of choice.
Having managed db hosting can be a really nice thing.
I am currently trying to setup a redirect on write for an installation of OpenLdap 2.2.
I have two instances running. One is configured to be read-only (only read access, database specified as read-only) and has redirect configured to point to the second instance. The second instance is configured to allow for the desired write permissions.
When I attempt a modify on the first instance it fails as expected but does not send back the referral. Am I missing a piece of the configuration? Am I even on the right path? Any guidance would be greatly appreciated. Thanks.
In the database section of you slapd.conf do you add the redirection like this ? :
updateref "ldap://master-host:port/"
So, it turns out the best way to do this is to go ahead and set up replication using slurpd and point all requests at the slave instance. Unfortunately you can't set up the master and slave on the same host (for obvious reasons, but still), so I had to spin up a second VM to get this going.
Honestly, if I was not trying to replicate a redirect problem it wouldn't be worth it, but I have to duplicate a production issue.
For more information on slapd and specifically slurpd, the OpenLDAP documentation is actually crazy helpful: slurpd config for OpenLDAP 2.2