Golang connect to Postgres using SSL certificate - postgresql

First of all, question is languate-agnostic. I'm trying to write a simple application that connects to PostgreSQL using SSL.
I created certificates using scripts:
# Create CA private key
openssl genrsa -des3 -out root.key 4096
#Remove a passphrase
openssl rsa -in root.key -out root.key
# Create a root Certificate Authority (CA)
openssl \
req -new -x509 \
-days 365 \
-subj "/CN=localhost" \
-key root.key \
-out root.crt
# Create server key
openssl genrsa -des3 -out server.key 4096
#Remove a passphrase
openssl rsa -in server.key -out server.key
# Create a root certificate signing request
openssl \
req -new \
-key server.key \
-subj "/CN=localhost" \
-text \
-out server.csr
# Create server certificate
openssl \
x509 -req \
-in server.csr \
-text \
-days 365 \
-CA root.crt \
-CAkey root.key \
-CAcreateserial \
-out server.crt
I created a database using:
init.sql
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
CREATE TABLE TESTING_DATA (
ID SERIAL PRIMARY KEY,
UUID UUID UNIQUE NOT NULL DEFAULT uuid_generate_v4(),
NAME TEXT NOT NULL,
INFO NUMERIC(3, 2)
);
INSERT INTO TESTING_DATA (NAME, INFO)
VALUES
('Notebook', 1),
('Office supplies', 2),
('Pencil', 2),
('Office supplies', 1),
('Eraser', 1),
('Coffee', 1),
('Cookies', 2),
('Puzzles', 5)
;
postgresql.conf
ssl = on
ssl_ca_file = '/etc/postgres/security/root.crt'
ssl_cert_file = '/etc/postgres/security/server.crt'
ssl_key_file = '/etc/postgres/security/server.key'
password_encryption = scram-sha-256
pg_hba.conf
local all all md5
host all all 127.0.0.1/32 md5
hostssl all all 0.0.0.0/0 cert clientcert=1
Dockerfile
FROM postgres:12-alpine
ENV POSTGRES_USER=pguser
ENV POSTGRES_PASSWORD=pgpassword
ENV POSTGRES_DB=securitylearning
COPY pg_hba.conf postgresql.conf /etc/postgresql/config/
COPY --chown=postgres:postgres root.crt server.crt server.key /etc/postgres/security/
COPY init.sql /docker-entrypoint-initdb.d/
EXPOSE 5432
CMD ["postgres", "-c", "config_file=/etc/postgresql/config/postgresql.conf", "-c", "hba_file=/etc/postgresql/config/pg_hba.conf"]
I launched a container, I ensured that from the container itself I can connect to database and select something from the table.
I created a simple program:
server.go
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
)
func main() {
connection := fmt.Sprint(
" host=localhost",
" port=5432",
" user=pguser",
" dbname=securitylearning",
" sslmode=verify-full",
" sslrootcert=root.crt",
" sslkey=client.key",
" sslcert=client.crt",
)
db, err := sql.Open("postgres", connection)
defer db.Close()
if err != nil {
panic(err)
}
err = db.Ping()
if err != nil {
panic(err)
}
row := db.QueryRow("SELECT * FROM TESTING_DATA")
fmt.Println(row)
}
I tried to:
place files root.crt, server.crt, server.key next to the compiled binary and add to the connection string in go file sslrootcert, sslcert, sslkey respectively
place same files, but with names root.crt, postgresql.crt, postgresql.key in ~/.postgresql/ directory, because pq uses them by default.
For now, it's not working. I randomly get one of those two errors:
read: connection reset by peer
or
EOF
Could you please help? What am I missing here? Or could you point me to some resources? Thanks in advance.
Update 1
Thanks to suggestion in comments, I created client key and certificate, using
# Create client key
openssl genrsa -out client.key 4096
#Remove a passphrase
openssl rsa -in client.key -out client.key
# Create client certificate signing request
openssl \
req -new \
-key client.key \
-subj "/CN=172.17.0.2" \
-out client.csr
# Create client certificate
openssl \
x509 -req \
-in client.csr \
-CA root.crt \
-CAkey root.key \
-CAcreateserial \
-days 365 \
-text \
-out client.crt
I'm using 172.17.0.2 in CN, because it's host IP from docker container's perspective.
I've tried both:
using following keys in connection string from program
" sslrootcert=root.crt",
" sslkey=client.key",
" sslcert=client.crt",
copying root.crt, client.key, client.srt to ~/.postgresql/, trying psql with
psql "host=localhost port=5432 user=pguser dbname=securitylearning sslmode=verify-full sslrootcert=root.crt sslkey=client.key sslcert=client.crt"
or without password.
Both ways still fail to connect. In psql case I get error
psql: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.

Thanks to suggestions in comments I managed to solve it.
First of all, as suggested, I stepped back and tried to proceed with smaller steps. Such as, securely connect with psql from host.
Mistake 1
I forgot to add the following property to postgresql.conf
listen_addresses = '*'
The documentation says:
If the list is empty, the server does not listen on any IP interface at all, in which case only Unix-domain sockets can be used to connect to it.
Mistake 2
I fell into a little misconception with certificates and their common names (CN). The following points should be applied to scripts that create certificates. In short:
CN for CA can be anything as long as it is different from the server's
CN. See this question and answer for details
CN for server must be IP/hostname by which we will call server from client (here it's localhost. But if the database would be located at
cooldatabase.com <- this would be server's CN)
CN for client must be username by which we will connect (here, it's
pguser)
When I fixed these two issues - I managed to connect via both psql and go program! Also, the default postgresql.conf is very informative!

Related

What command do I use to create a password-protected x509 certificate?

I created an x509 certificate using openssl. I created the certificate using this openssl command:
req -new -x509 -key privkey.der -out cert.pfx
where "privkey.der" is the private key I created. I want to make this certificate password protected. Is there a command that would allow me to do so?
Thanks!

Differences between generated x509 certificates in kubernetes v1.10.11 and v1.11.5

I have been creating an amazon AMI with kubernetes installed on it to use as a worker node in EKS, I install the kubelet binary from the amazon-eks s3 bucket.
After upgrading from k8s version 1.10.11 to 1.11.5 I noticed a difference in the x509 certificate that is generated when installing kubelet.
If I jump onto one of the worker nodes with 1.10.11 installed and run this command openssl s_client -connect localhost:10250 2>/dev/null | openssl x509 -noout -text I get the following output for X509v3 Subject Alternative Name:
DNS:ip-<my-ip>.eu-central-1.compute.internal, DNS:ip-<my-ip>, IP Address:<my-ip>
whereas, if I run the same command on a worker node with 1.11.5 installed I get the following output for X509v3 Subject Alternative Name:
DNS:ip-<my-ip>
The only change between the two nodes is the version of kubernetes installed.
Am I missing anything that is now required as of version 1.11.x to set the additional Subject Alternative Names as seemed to be previously done in v1.10.x? I require the IP address to be set in the certificate in the format IP Address:<my-ip> which I was getting for free in version 1.10.
FYI I am running kubelet with the following args:
ExecStart=/usr/bin/kubelet \
--address=0.0.0.0 \
--authentication-token-webhook \
--authorization-mode=Webhook \
--allow-privileged=true \
--cloud-provider=aws \
--cluster-dns=DNS_CLUSTER_IP \
--cluster-domain=cluster.local \
--cni-bin-dir=/opt/cni/bin \
--cni-conf-dir=/etc/cni/net.d \
--container-runtime=docker \
--max-pods=MAX_PODS \
--node-ip=INTERNAL_IP \
--network-plugin=cni \
--pod-infra-container-image=602401143452.dkr.ecr.REGION.amazonaws.com/eks/pause-amd64:3.1 \
--cgroup-driver=cgroupfs \
--register-node=true \
--kubeconfig=/var/lib/kubelet/kubeconfig \
--feature-gates=RotateKubeletServerCertificate=true \
--anonymous-auth=false \
--client-ca-file=CLIENT_CA_FILE \
--node-labels=env=NODE_LABEL
As far as handling the certificates there are not Kubernetes specific differences between 1.10.11 and 1.11.5. It might be related to specific EKS AMI for the nodes that you are using (make sure they are matching)
If not you can manually create the certificates for the kubelet using the same CA as the one in your Kubernetes master. For example:
easyrsa
./easyrsa --subject-alt-name="IP:${MASTER_IP},"\
"IP:-<my-ip>,"\
"DNS:ip-<my-ip>.eu-central-1.compute.internal,"\
"DNS:ip-<my-ip>,"\
--days=10000 \
build-server-full server nopass
openssl
Config (csr.conf):
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
C = <country>
ST = <state>
L = <city>
O = <organization>
OU = <organization unit>
CN = <my-ip>
[ req_ext ]
subjectAltName = #alt_names
[ alt_names ]
DNS.1 = ip-<my-ip>.eu-central-1.compute.internal
DNS.2 = ip-<my-ip>
IP.1 = <my-ip>
[ v3_ext ]
authorityKeyIdentifier=keyid,issuer:always
basicConstraints=CA:FALSE
keyUsage=keyEncipherment,dataEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName=#alt_names
Create CSR:
$ openssl req -new -key server.key -out server.csr -config csr.conf
Create certificate:
$ openssl x509 -req -in server.csr -CA cluster-ca.crt -CAkey cluster-ca.key \
-CAcreateserial -out server.crt -days 10000 \
-extensions v3_ext -extfile csr.conf
cfssl
In a similar fashion you can use cfssl, described here.

Cannot establish SSL/TLS connection between Kong 0.10.x and datastore postgresql-9.6

I am using Kong 10.x with datastore postgresql 9.6.
I want to establish ssl connection between kong and it's datastore postgresql.
But I get the following errors:
Postgresql Error:
LOG: could not accept SSL connection: tlsv1 alert unknown ca
Error from kong:
/usr/local/share/lua/5.1/kong/cmd/migrations.lua:34: [postgres error] could not retrieve current migrations: [postgres error] connection refused
Below are my Kong and Postgresql Configurations:
Kong:
# Kong configuration file
# DATASTORE
database = postgres
pg_host = 10.0.1.191
pg_port = 5432
pg_user = kong
pg_password = kong
pg_database = kong
pg_ssl = on
pg_ssl_verify = on
# DEVELOPMENT & MISCELLANEOUS
lua_ssl_trusted_certificate = /opt/postgres_ssl/postgresql.crt # Absolute path to the certificate
Postgresql:
pg_hba.conf:
hostssl all all 10.0.1.191/32 md5 clientcert=1
postgresql.conf:
listen_addresses = '10.0.1.191'
ssl_cert_file = 'server.crt'
ssl_key_file = 'server.key'
ssl_ca_file = 'root.crt'
Certificate Generation Procedure:
openssl genrsa -passout pass:mypass -des3 -out server.key 1024
openssl rsa -passin pass:mypass -in server.key -out server.key
chmod 400 server.key
openssl req -new -key server.key -days 3650 -out server.crt -x509 -subj '/C=IN/ST=Maharastra/L=Mumbai/O=Development/CN=10.0.1.191'
cp server.crt root.crt
openssl genrsa -passout pass:iotadmin -des3 -out postgresql.key 1024
openssl rsa -in postgresql.key -out postgresql.key -passin pass:mypass
openssl req -new -key postgresql.key -days 3650 -out postgresql.csr -subj '/C=IN/ST=Maharastra/L=Mumbai/O=Development/CN=kong'
openssl x509 -req -in postgresql.csr -CA root.crt -CAkey server.key -out postgresql.crt -CAcreateserial
The error message says that Kong doesn't trust the CA which signed the certificate of the database. This doesn't surprise much, because it only knows the latter, but not the certificate of the CA.
Try using the root certificate for your lua_ssl_trusted_certificate config entry and it should work:
lua_ssl_trusted_certificate = /path/to/your/root.crt

Running dev env with mongo + ssl

Background
I'm trying to run mongo locally in the same way that production will run, with full ssl verification enabled. Mongo is complaining about the certs being self-signed, but I'm specifying a ca.crt file, that I think should be treated as a root cert to validate against. If that's reasonable, then I think either my mongo config, or the cert generation is not correct.
SSL keys/certs/pem
To create the ssl stuff I'm running the following
#!/bin/sh
# Generate self signed root CA cert
openssl req -nodes -x509 -newkey rsa:2048 -keyout ca.key -out ca.crt -subj "/emailAddress=dev#gmail.com"
# Generate server cert to be signed
openssl req -nodes -newkey rsa:2048 -keyout server.key -out server.csr -subj "/emailAddress=dev#gmail.com"
# Sign the server cert
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt
# Create server PEM file
cat server.key server.crt > server.pem
# Generate client cert to be signed
openssl req -nodes -newkey rsa:2048 -keyout client.key -out client.csr -subj "/emailAddress=dev#gmail.com"
# Sign the client cert
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAserial ca.srl -out client.crt
# Create client PEM file
cat client.key client.crt > client.pem
Mongo DB config
The mongo config I'm then running with (inside docker), is the following. (Where /data/mongo is the location generated to above).
net:
port: 27017
ssl:
mode: requireSSL
CAFile: /data/mongo/ca.crt
PEMKeyFile: /data/mongo/server.pem
allowInvalidHostnames: true
setParameter:
enableLocalhostAuthBypass: true
and running via
mongo --config config/location
Connecting to mongo
I then try to connect to the server using the mongo command line as follows.
mongo --ssl --sslPEMKeyFile /data/mongo/client.pem --sslCAFile /data/mongo/ca.crt
And get the following output
MongoDB shell version: 3.2.14
connecting to: test
2017-07-19T20:12:31.456+0000 I NETWORK [initandlisten] connection accepted from 127.0.0.1:60516 #1 (1 connection now open)
2017-07-19T20:12:31.461+0000 E NETWORK [conn1] SSL peer certificate validation failed: self signed certificate
2017-07-19T20:12:31.461+0000 I NETWORK [conn1] end connection 127.0.0.1:60516 (0 connections now open)
2017-07-19T20:12:31.461+0000 E NETWORK [thread1] SSL peer certificate validation failed: self signed certificate
2017-07-19T20:12:31.461+0000 E QUERY [thread1] Error: socket exception [CONNECT_ERROR] for SSL peer certificate validation failed: self signed certificate :
connect#src/mongo/shell/mongo.js:229:14
#(connect):1:6
exception: connect failed
Got it! Basically it needed more data in the subject line, or CN needed to be ROOTCA for the CA. Anyone that could comment on why would be appreciated.
#!/bin/sh
prefix="/C=CN/ST=GD/L=city/O=company"
# Generate self signed root CA cert
openssl req -nodes -x509 -newkey rsa:2048 -keyout ca.key -out ca.crt -subj "${prefix}/CN=ROOTCA"
# Generate server cert to be signed
openssl req -nodes -newkey rsa:2048 -keyout server.key -out server.csr -subj "${prefix}/CN=127.0.0.1"
# Sign the server cert
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt
# Create server PEM file
cat server.key server.crt > server.pem
# Generate client cert to be signed
openssl req -nodes -newkey rsa:2048 -keyout client.key -out client.csr -subj "${prefix}/CN=127.0.0.1"
# Sign the client cert
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAserial ca.srl -out client.crt
# Create client PEM file
cat client.key client.crt > client.pem
Some related resources if anyone is having similar troubles
Answer was found/taken from
https://www.mongodb.com/blog/post/secure-mongodb-with-x-509-authentication
https://raw.githubusercontent.com/tjworks/mongoscripts/master/x509/setup-x509.sh
An stack exchange ticket almost identical to mine can also be found at
https://dba.stackexchange.com/questions/151251/mongodb-error-self-signed-certificate-in-certificate-chain?newreg=20bca440682842c085a8764dd7c91e96

Certificate invalid

I am trying this for the past week. I dont know where I am going wrong. I want to setup a MDM server that dont have static IP. I have a DNS resolvable name for the server. I have the identity.key and identity.csr for which I created the MDM vendor certificate. I wrote a following .bat file to generate other certificates
echo 1. Creating Certificate Authority (CA)
echo For 'Common Name' enter something like 'MDM Test CA
openssl req -new -x509 -extensions v3_ca -keyout cakey.key -out cacert.crt -days 365
echo 2. Creating the Web Server private key and certificate request
echo For 'Common Name' enter your server's DNS Name
openssl genrsa 2048 > server.key
openssl req -new -key server.key -out server.csr
echo 3. Signing the server key with the CA. You'll the CA passphrase from step 1.
openssl x509 -req -days 365 -in server.csr -CA cacert.crt -CAkey cakey.key -CAcreateserial -out server.crt -extfile .\server.cnf -extensions ssl_server
echo 4. Signing the identity key with the CA. You'll the CA passphrase from step 1.
echo Give it a passphrase. You'll need to include that in the IPCU profile
openssl x509 -req -days 365 -in identity.csr -CA cacert.crt -CAkey cakey.key -CAcreateserial -out identity.crt
openssl pkcs12 -export -out identity.p12 -inkey identity.key -in identity.crt -certfile cacert.crt
I used the identity.p12 file and created a encoded plist file for which push notification certificate is created. I also create a MDM profile in IPCU with the identity.p12 file. When I try to install the MDM profile, the mobile console says "Certificate in the server is invalid" and in the server it says SSL handshake failed and the server gets hanged.
What might be the issue?
In server.cnf -> subjectAltName I gave a parameter named DNS with DNS resolvable name and removed the IP. Now it is working fine.