I have an appication deployed on AWS EKS that uses an RDS PostgreSQL database. I have downloaded the intermediate and root certifcates, and added them to a trust store, as described in this post: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html
However I cannot connect via SSL with sslmode=verify-full and I think it's because I do not have a copy of the certificate generated when RDS creates the DB instance and installs the certificate on the instance, as described here: https://aws.amazon.com/premiumsupport/knowledge-center/rds-connect-ssl-connection/
The certificate generated when the database is provisioned has the hostname of the server as the Common Name, and I think this is used to veryify the host when a client connects.
Does anyone know where I can download this certiifacte or if I have misunderstood how to do this, tell me what it is I am doing wrong?
Thanks
You need to do multiple things:
Download the ca certs from https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem.
Import this cert into the ca-certs of the JDK/JRE in your docker image by using this command keytool -importcert -alias aws-certs -trustcacerts -file /path/to/global-bundle.pem -storepass changeit -cacerts -noprompt
Note: You might have to run this command as root/sudo depending on the permissions on the cacerts file in the JDK_HOME/lib/security folder.
Make changes to your postgres jdbc url as decribed here; basically adding sslmode=verify-full
Related
I am trying to create and use a local kafka keystore.
I have created a trust store at repository_name/kafka/truststore.jks
I have config
props["ssl.truststore.location"] = "classpath:kafka/truststore.jks"
props["ssl.truststore.password"] = "password"
but when i boot up the app I get the kafka error
java.nio.file.NoSuchFileException: classpath:kafka/truststore.jks at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)...
This is a problem with the pathing I have used as I tried the absolute path and got no error. This however is not a solution as it is a shared repo, the path must be from the repo not absolute. I did try some alternate pathing such as kafka/truststore.jks which did not work.
The command used to create the keystore was
keytool -keystore truststore.jks -genkey -alias ts
I have set up a SonarQube and configured SSL certificates to make the URL always HTTPS using CertBot. As of now, the PostgreSQL database has a public IP and below are the values changed in sonar.properties file:
sonar.jdbc.username=weakusername
sonar.jdbc.password=strongpassword
sonar.web.host=127.0.0.1
sonar.jdbc.url=jdbc:postgresql://xx.xxx.xxx.xxx/sonarqube
sonar.search.javaOpts=-Xms512m -Xmx512m
# Change max limits
sysctl -w vm.max_map_count=262144
I am using Cloud SQL PostGres as the database. I would like to allow Only SSL Only Connections to the database and here the way how to do it, generate client certificate, etc.
After setting "Allow only SSL Connections" to true I understand there is a way to connect to the database using the client certificate described here.
Below is the command to start the psql client:
psql "sslmode=verify-ca sslrootcert=server-ca.pem \
sslcert=client-cert.pem sslkey=client-key.pem \
hostaddr=[INSTANCE_IP] \
user=postgres dbname=[DB_NAME]"
However, SonarQube is not able to connect to the Database (Not sure how to tell SonarQube to use the client certificates).
What changes are required in the configuration file to make SonarQube use appropriate client certificate and connect to the database using SSL?
You should add the following to the URL:
jdbc:postgresql://xx.xxx.xxx.xxx/sonarqube?ssl=true&sslmode=verify-ca&sslrootcert=/path/to/server-ca.pem&sslkey=/path/to/client-key.pem&sslcert=/path/to/client-cert.pem
See the documentation for the available SSL connection parameters and SSL client configuration.
Convert client key format from PEM to PK8:
openssl pkcs8 -topk8 -inform PEM -outform DER -in client-key.pem -out client-key.pk8 -nocrypt
Be sure to update the value for the sslkey query param in sonar.jdbc.url with the new path/filename.
How to make connection from mongo-spark connector to mongodb when only TLS/ssl enabled for mongo DB ?
How to pass the uri and collection name in read config to make connection with TLS/ssl enabled mongodb instance?
Thanks in advance ?
To make the ssl connection from Spark to the Mongo server you will need to trust the Mongo certificate, or the CA (certificate authority) that has signed that certificate. This is the most important part, and the trickiest one for me to figure it out.
Spark is a Java application, so it get the certificates from a jks trustStore. you will need to import the Mongo certificate (only the public part) into a trustStore to make it available for spark. To do so:
Get the Mongo certificate: Ask the DBA or the sysadmin who has setup the mongo to provide the certificate to you. Other aproach is to get it with openssl:
$ openssl s_client -connect mongodb:27017
CONNECTED(00000003)
depth=0 C = ES, ST = Madrid, L = Madrid, O = HOME, OU = HOME, CN=mongodb mongo.hostname.local
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/C=ES/ST=Madrid/L=Madrid/O=COMPANY/OU=AREA/CN=mongo.hostname.local
i:/C=ES/ST=Madrid/L=Madrid/O=COMPANY/OU=AREA/CN=mongo.hostname.localIssuing CA
---
Server certificate
-----BEGIN CERTIFICATE-----
[..... A bunch of base64 text....]
-----END CERTIFICATE-----
Get the part from the -----BEGIN CERTIFICATE----- to -----BEGIN CERTIFICATE----- and save it in a .cert file
Import it into a trustStore
$ keytool -import -file /path/to/your/mongodb.crt -alias mongodb -keystore /path/to/your/trustStore.jks
Enter keystore password: 123456
...
...
Trust this certificate? [no]: yes
Certificate was added to keystore
Make sure the keystore is accesible from all your spark cluster nodes.
Now, you have your server certificate imported. If you need mutual TLS you will need to provide a valid client certificate. This certificate, and the certificate private key, should be in a jks keyStore (it could be in the same trustStore file you have stored the Mongo server certificate because it uses the same format). If are not going to use mutual TLS you don't need to do this, but you have to check that the MongoDB instance is able to accept connections without client certificates. This is with the flag sslAllowConnectionsWithoutCertificates
The next step is specifying in the connection URI that you want to use TLS. This is fairly simple, just add the ?ssl=true to your connection string. So the connection URI will be something like this
mongodb://user:pw#host:port/db.collection?ssl=true
Now you can summit your job. When summiting the job we also need to specify the location of our trustStore, and the libraries for the mongo connector:
/spark/bin/spark-submit \
--master spark://spark-master:7077 \
--packages org.mongodb.spark:mongo-spark-connector_2.11:2.2.0 \
--conf spark.executor.extraJavaOptions="-Djavax.net.ssl.trustStore=/path/to/your/trustStore.jks -Djavax.net.ssl.trustStorePassword=yourPassword" \
--conf spark.driver.extraJavaOptions="-Djavax.net.ssl.trustStore=/path/to/your/trustStore.jks -Djavax.net.ssl.trustStorePassword=yourPassword" \
/yourJob.jar
We use the extraJavaOptions for the driver and the executor to pass these parameters. If you are using mutual TLS, include the following extra java options:
-Djavax.net.ssl.keyStore=/path/to/your/keyStore.jks
-Djavax.net.ssl.keyStorePassword=yourPassword
The /path/to/your/keyStore.jks is where you have stored your client certificates.
If the spark connector library is not already installed, you may run into trouble. The spark process will go to maven to download the library, but it will not be able to verify the maven certificates because we have specified another keyStore with just our certificate. One workaround is to import our certificate directly into the default keystore located at $JAVA_HOME/jre/lib/security/cacerts. The default password is changeit. Remember to do this in every worker node too.
I hope it helps!
Sources:
https://github.com/brunocfnba/spark-mongo-ssl
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_spark-component-guide/content/spark-encryption.html
https://community.hortonworks.com/articles/147113/how-to-configure-your-spark-application-to-use-mon.html
https://mapr.com/support/s/article/Unable-to-find-valid-certification-path-to-requested-target-error-while-accessing?language=en_US
Am trying to configure glassfish-4 for SSL but it seems i can't get it to work. By default glassfish uses port 8181 for SSL which of course works very well from my localhost prompting the browser to add the certificate in its truststore but from remote this fails with the exception -- SSL received a record that exceeded the maximum permissible length-- on mozilla firefox. I also tried generating my own key/pair using java's keytool in the config folder of the domain1, I then changed certificate nickname on glassfish to the alias of the new keystore but this fails to work instead returns the default certificate from alias s1as even after restarting the server. Below is the command i used to generate the keystore file.
keytool -genkey -alias MY_ALIAS_NAME -validity 1000 -keyalg RSA -keysize 1024 -keystore /opt/......./domain1/config/keystore.jks
The first and last names being the IP address of the remote when generating it. Please help, thanks.
I am trying to connect to RDS running on AWS (Amazon Web Services) using SSL. I saw limited info in PosgreSQL pgAdmin III docs about fields on SSL tab.
RDS instances are setup to accept SSL connections by default.
I've downloaded the public key from Amazon and converted it from a .pem to a .crt file using openSSL. On the SSL tab in pgAdmin III I entered path to converted key file "Server Root Certificate File" field.
I can connect to instance without issue but there is no indication that the data is being transferred over SSL. AWS does not set their RDS instances to use SSL exclusivly so I may be connected without using SSL and not know it.
Does pgAdmin III show any indication when it's connected using SSL (like a lock icon)?
Can anyone provide additional info that describes the fields (SSL dropdown, Client Cert File, Client Key) on the SSL tab in pgAdmin III?
Thanks.
I have not used SSL with PGAdmin on AWS, but I have on a server, and I can tell you that you know when you are connected to a server via PGAdmin, I'm not sure how there is ambiguity there, can you see the databases, tables?
The quoted post below might help you with connecting to a server via SSL.
On the client, we need three files. For Windows, these files must be
in %appdata%\postgresql\ directory. For Linux ~/.postgresql/
directory. root.crt (trusted root certificate) postgresql.crt (client
certificate) postgresql.key (private key)
Generate the the needed files on the server machine, and then copy
them to the client. We'll generate the needed files in the /tmp/
directory.
First create the private key postgresql.key for the client machine,
and remove the passphrase.
openssl genrsa -des3 -out /tmp/postgresql.key 1024
openssl rsa -in /tmp/postgresql.key -out /tmp/postgresql.key
Then create the certificate postgresql.crt. It must be signed by our
trusted root (which is using the private key file on the server
machine). Also, the certificate common name (CN) must be set to the
database user name we'll connect as.
openssl req -new -key /tmp/postgresql.key -out /tmp/postgresql.csr -subj '/C=CA/ST=British Columbia/L=Comox/O=TheBrain.ca/CN=www-data'
openssl x509 -req -in /tmp/postgresql.csr -CA root.crt -CAkey server.key -out /tmp/postgresql.crt -CAcreateserial
Copy the three files we created from the server /tmp/ directory to the
client machine.
Copy the trusted root certificate root.crt from the server machine to
the client machine (for Windows pgadmin %appdata%\postgresql\ or for
Linux pgadmin ~/.postgresql/). Change the file permission of
postgresql.key to restrict access to just you (probably not needed on
Windows as the restricted access is already inherited). Remove the
files from the server /tmp/ directory.
From: http://www.howtoforge.com/postgresql-ssl-certificates
First, login as your postgresql admin user then run the following to install sslinfo on RDS:
create extension sslinfo;
To verify if you're connected via ssl simply run the following query in your session:
select ssl_is_used();
If it returns true (t), then you're connected via SSL.