I was trying to connect to the Neo4j HTTPS server through a the neo4j-rest-client using the snakeoil.key and snakeoil.cert files generated after installing Neo4j from debian sources. However, it seems like those files are not PEM formatted, and behind scenes, neo4j-rest-client is only able to handle PEM formatted files because of httplib2. So, then I created my own ssl.crt and ssl.key PEM files, but this time looks like Neo4j doesn't like this format. Am I right? Is there any way to have Neo4j Server running in SSL mode with PEM files?
AFAIK, a PEM file should look like
-----BEGIN CERTIFICATE-----
blahblahblah
-----END CERTIFICATE-----
No, they are not PEM files, but you can create a PEM file from the certificate file (snakeoil.crt). See here for instructions:
http://www.digicert.com/ssl-support/pem-ssl-creation.htm
Related
I got a .pem file for mongoDB authentication purpose.
The Windows machine hostname TEST001 (I read it by below method) is embedded in the file so that only that machine has the right to communicate with mongoDB.
When I read the subjectAltName in the given .pem file:
C:\Prj\Certs>openssl x509 -noout -ext subjectAltName -in test_mongodb.pem
X509v3 Subject Alternative Name:
DNS:companyABC, DNS:companyABC.TEST.INTER, DNS:TEST001, DNS:TEST001.TEST.INTER
Now, I have another machine hostname is TEST002 and would want it authenticated to the mongoDB as well.
Is that possible to create new .pem based on the given .pem file, by just alter the hostname (subjectAltName)?
I just want to minimize the effort to create completely new .pem file.
How to make connection from mongo-spark connector to mongodb when only TLS/ssl enabled for mongo DB ?
How to pass the uri and collection name in read config to make connection with TLS/ssl enabled mongodb instance?
Thanks in advance ?
To make the ssl connection from Spark to the Mongo server you will need to trust the Mongo certificate, or the CA (certificate authority) that has signed that certificate. This is the most important part, and the trickiest one for me to figure it out.
Spark is a Java application, so it get the certificates from a jks trustStore. you will need to import the Mongo certificate (only the public part) into a trustStore to make it available for spark. To do so:
Get the Mongo certificate: Ask the DBA or the sysadmin who has setup the mongo to provide the certificate to you. Other aproach is to get it with openssl:
$ openssl s_client -connect mongodb:27017
CONNECTED(00000003)
depth=0 C = ES, ST = Madrid, L = Madrid, O = HOME, OU = HOME, CN=mongodb mongo.hostname.local
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/C=ES/ST=Madrid/L=Madrid/O=COMPANY/OU=AREA/CN=mongo.hostname.local
i:/C=ES/ST=Madrid/L=Madrid/O=COMPANY/OU=AREA/CN=mongo.hostname.localIssuing CA
---
Server certificate
-----BEGIN CERTIFICATE-----
[..... A bunch of base64 text....]
-----END CERTIFICATE-----
Get the part from the -----BEGIN CERTIFICATE----- to -----BEGIN CERTIFICATE----- and save it in a .cert file
Import it into a trustStore
$ keytool -import -file /path/to/your/mongodb.crt -alias mongodb -keystore /path/to/your/trustStore.jks
Enter keystore password: 123456
...
...
Trust this certificate? [no]: yes
Certificate was added to keystore
Make sure the keystore is accesible from all your spark cluster nodes.
Now, you have your server certificate imported. If you need mutual TLS you will need to provide a valid client certificate. This certificate, and the certificate private key, should be in a jks keyStore (it could be in the same trustStore file you have stored the Mongo server certificate because it uses the same format). If are not going to use mutual TLS you don't need to do this, but you have to check that the MongoDB instance is able to accept connections without client certificates. This is with the flag sslAllowConnectionsWithoutCertificates
The next step is specifying in the connection URI that you want to use TLS. This is fairly simple, just add the ?ssl=true to your connection string. So the connection URI will be something like this
mongodb://user:pw#host:port/db.collection?ssl=true
Now you can summit your job. When summiting the job we also need to specify the location of our trustStore, and the libraries for the mongo connector:
/spark/bin/spark-submit \
--master spark://spark-master:7077 \
--packages org.mongodb.spark:mongo-spark-connector_2.11:2.2.0 \
--conf spark.executor.extraJavaOptions="-Djavax.net.ssl.trustStore=/path/to/your/trustStore.jks -Djavax.net.ssl.trustStorePassword=yourPassword" \
--conf spark.driver.extraJavaOptions="-Djavax.net.ssl.trustStore=/path/to/your/trustStore.jks -Djavax.net.ssl.trustStorePassword=yourPassword" \
/yourJob.jar
We use the extraJavaOptions for the driver and the executor to pass these parameters. If you are using mutual TLS, include the following extra java options:
-Djavax.net.ssl.keyStore=/path/to/your/keyStore.jks
-Djavax.net.ssl.keyStorePassword=yourPassword
The /path/to/your/keyStore.jks is where you have stored your client certificates.
If the spark connector library is not already installed, you may run into trouble. The spark process will go to maven to download the library, but it will not be able to verify the maven certificates because we have specified another keyStore with just our certificate. One workaround is to import our certificate directly into the default keystore located at $JAVA_HOME/jre/lib/security/cacerts. The default password is changeit. Remember to do this in every worker node too.
I hope it helps!
Sources:
https://github.com/brunocfnba/spark-mongo-ssl
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_spark-component-guide/content/spark-encryption.html
https://community.hortonworks.com/articles/147113/how-to-configure-your-spark-application-to-use-mon.html
https://mapr.com/support/s/article/Unable-to-find-valid-certification-path-to-requested-target-error-while-accessing?language=en_US
I have created CSR using openssl and I want it to be signed by the Microsoft CA using command line with template as webServer.
This is the way to generate. Got it! .. On a Microsoft CA server machine issue below command:
"certreq.exe -submit -attrib "CertificateTemplate:WebServer" certifcatesigningrequest.csr "
haproxy does not start anymore, it shows the error
bind <ip>:443' : unable to load SSL private key from PEM file ...
We did not change anything on the certificates or configuration. Since the last start we only made normal updates to the system.
To find the error, I generated a completely new certificate (self signed) but the error still exists.
This is the structure of the PEM file:
-----BEGIN CERTIFICATE-----
MIIDXjCCAkY...
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEpgIBAAKC....
-----END RSA PRIVATE KEY-----
I also tried to convert the private key with
openssl pkcs8 -topk8 -inform pem -in server.key -outform pem -nocrypt -out server_new.key
but haproxy still shows the same error.
I'm trying for hours now but I can not find the reason. Please help! Thank you!
Update:
The problem has something to do with file access. The PEM file was stored at /data/ssl/domainname/domainname.pem. File rights are ok. When I move the PEM file to /etc/haproxy then everything is ok.
The order in which the cert and key files appear in the pem is important. Use the following to create the pem file.
cat example.com.crt example.com.key > example.com.pem
The problem I was running into on CentOS was SELinux was getting in the way. To test if SELinux is the problem execute the following as root: setenforce 0, then try restarting the haproxy. If it works, there is an SELinux problem. (You can re-enable SELinux now and try to fix the underlying problem with the command setenforce 1).
Since I have the certificates in the folder /etc/haproxy/certificates, the following command worked to get the right permissions on the files restorecon -v -R /etc/haproxy (depending on your OS and SELinux config this may or may not work).
For me the problem was caused by this line in combined PEM file:
-----END CERTIFICATE----------BEGIN RSA PRIVATE KEY-----
After I split it I could start HaProxy and load it OK:
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
For the latest version of letsencrypt certbot,fullchain.pem and privkey.pem files will be generated for you in /etc/letsencrypt/live/example.com folder. They need to be combined in order to HAProxy to read it properly.
cat fullchain.pem privkey.pem > example.com.pem
In HAProxy configuraion /etc/haproxy/haproxy.cfg
bind *:443 ssl crt /etc/letsencrypt/live/example.com/example.com.pem
I also encountered this error. You might want to try to remove the passphrase from the private key before you begin ripping your hair out. It solved the problem for me. I think HAProxy is supposed to ask you for the password on restart, but it didn't in my case using 'sudo /etc/init.d/haproxy restart
To remove the password, try
'openssl rsa -in [PRIVATE_KEY_FILE] -out nopassphrase.key'
Is passphrase necesssary? There's a discussion in the link below.
https://security.stackexchange.com/questions/70495/ssl-certificate-is-passphrase-necessary-and-how-does-apache-know-it
Did you append your certificate's private key to the end of the file?
HAProxy requires a "full chain" - certificate, intermediate authority (if you have one), and then private key. E.g.:
cat cert.pem cert.key > /haproxy/certs/fullchain.pem
The problem for me was a strange character at the beginning of the key.
This character did not show up when I cated the file because the character was <feff> otherwise known as the UTF-8 BOM (Byte Order Mark). It only showed up when I opened the file in vim.
I wouldn't expect this to be very common, but hopefully it saves someone some headache.
Just for information, in my case I had space character in front of "-----BEGIN RSA PRIVATE KEY-----" sequence and that broke the pem file.
I'd like to add, for people which join here and have the same issue, that you have to keep your intermediate certificates in the chain as well...
So if you have a chain with some layers, don't only take the rootca but also the intermediate certificates into your pem file
SElinux was the problem for me as well. HAProxy reported it could not read the file due to permissions even though the permissions matched other pem files in the folder. Our process is automated which is likely why SELinux is involved. The solution that seems to work for me so far (leaving SELinux running) is:
#!/bin/sh
if [ "$2" == "add" ]; then
sudo touch /etc/haproxy/ssl/$1
sudo cat $1 > /etc/haproxy/ssl/$1
sudo chmod 644 /etc/haproxy/ssl/$1
fi
if [ "$2" == "delete" ]; then
sudo rm /etc/haproxy/ssl/$1
fi
echo "performed $2 on $1";
I am trying to connect to RDS running on AWS (Amazon Web Services) using SSL. I saw limited info in PosgreSQL pgAdmin III docs about fields on SSL tab.
RDS instances are setup to accept SSL connections by default.
I've downloaded the public key from Amazon and converted it from a .pem to a .crt file using openSSL. On the SSL tab in pgAdmin III I entered path to converted key file "Server Root Certificate File" field.
I can connect to instance without issue but there is no indication that the data is being transferred over SSL. AWS does not set their RDS instances to use SSL exclusivly so I may be connected without using SSL and not know it.
Does pgAdmin III show any indication when it's connected using SSL (like a lock icon)?
Can anyone provide additional info that describes the fields (SSL dropdown, Client Cert File, Client Key) on the SSL tab in pgAdmin III?
Thanks.
I have not used SSL with PGAdmin on AWS, but I have on a server, and I can tell you that you know when you are connected to a server via PGAdmin, I'm not sure how there is ambiguity there, can you see the databases, tables?
The quoted post below might help you with connecting to a server via SSL.
On the client, we need three files. For Windows, these files must be
in %appdata%\postgresql\ directory. For Linux ~/.postgresql/
directory. root.crt (trusted root certificate) postgresql.crt (client
certificate) postgresql.key (private key)
Generate the the needed files on the server machine, and then copy
them to the client. We'll generate the needed files in the /tmp/
directory.
First create the private key postgresql.key for the client machine,
and remove the passphrase.
openssl genrsa -des3 -out /tmp/postgresql.key 1024
openssl rsa -in /tmp/postgresql.key -out /tmp/postgresql.key
Then create the certificate postgresql.crt. It must be signed by our
trusted root (which is using the private key file on the server
machine). Also, the certificate common name (CN) must be set to the
database user name we'll connect as.
openssl req -new -key /tmp/postgresql.key -out /tmp/postgresql.csr -subj '/C=CA/ST=British Columbia/L=Comox/O=TheBrain.ca/CN=www-data'
openssl x509 -req -in /tmp/postgresql.csr -CA root.crt -CAkey server.key -out /tmp/postgresql.crt -CAcreateserial
Copy the three files we created from the server /tmp/ directory to the
client machine.
Copy the trusted root certificate root.crt from the server machine to
the client machine (for Windows pgadmin %appdata%\postgresql\ or for
Linux pgadmin ~/.postgresql/). Change the file permission of
postgresql.key to restrict access to just you (probably not needed on
Windows as the restricted access is already inherited). Remove the
files from the server /tmp/ directory.
From: http://www.howtoforge.com/postgresql-ssl-certificates
First, login as your postgresql admin user then run the following to install sslinfo on RDS:
create extension sslinfo;
To verify if you're connected via ssl simply run the following query in your session:
select ssl_is_used();
If it returns true (t), then you're connected via SSL.