pgx tls connection throws client cert invalid error for valid cert - postgresql

Question

I'm trying to use pgx to make a TLS connection to a postgres 10 db.
My connection string is similar to: "host='my-host.com' port='5432' dbname='my-db' user='my-db-user' sslmode='verify-full' sslcert='/path/to/db_user.crt' sslkey='/path/to/db_user.key' sslrootcert='/path/to/ca_roots.pem'"
When I run this directly with psql on the command-line, it works, so the cert and key files must be valid. db_user.crt and db_user.key are both PEM files. (the command-line also works with sslmode='verify-full', so the rootcert should also be ok)
But when I initialize a pgx pool with that connection string, it fails with:
FATAL: connection requires a valid client certificate (SQLSTATE 28000)
Is go expecting something other than PEM? Or is there a different way ssl cert and key pair is supposed to be initialized with pgx?
Code
import (
"context"
"fmt"
"os"
"github.com/jackc/pgx/v4"
"github.com/jackc/pgx/v4/pgxpool"
)
type mockLogger struct{}
func (ml *mockLogger) Log(ctx context.Context, level pgx.LogLevel, msg string, data map[string]interface{}) {
fmt.Printf("[%s] %s : %+v\n", level.String(), msg, data)
}
func connect() error {
connStr := "host='my-host.com' port='5432' dbname='my-db' user='my-db-user' sslmode='verify-full' sslcert='/path/to/db_user.crt' sslkey='/path/to/db_user.key' sslrootcert='/path/to/ca_roots.pem'"
poolCfg, err := pgxpool.ParseConfig(connStr)
if err != nil {
return err
}
poolCfg.ConnConfig.Logger = &mockLogger{}
poolCfg.ConnConfig.LogLevel = pgx.LogLevelTrace
fmt.Printf("using connection string: \"%s\"\n", poolCfg.ConnString())
connPool, err := pgxpool.ConnectConfig(context.TODO(), poolCfg)
if err != nil {
return err
}
connPool.Close()
return nil
}
func main() {
if err := connect(); err != nil {
fmt.Printf("%+v\n", err)
os.Exit(1)
}
}
Output from calling connect():
using connection string: "host='my-host.com' port='5432' dbname='my-db' user='my-db-user' sslmode='require' sslcert='/path/to/db_user.crt' sslkey='/path/to/db_user.key' sslrootcert='/path/to/ca_roots.pem'"
[info] Dialing PostgreSQL server : map[host:my-host.com]
[error] connect failed : map[err:failed to connect to `host=my-host.com user=my-db-user database=my-db`: server error (FATAL: connection requires a valid client certificate (SQLSTATE 28000))]
failed to connect to `host=my-host.com user=my-db-user database=my-db`: server error (FATAL: connection requires a valid client certificate (SQLSTATE 28000))

Answer 1

Summary
Turns out for go, the cert pointed to by sslcert needed to contain the full client cert chain.
When /path/to/db_user.crt contained the client cert followed by client cert chain, the pgx connection worked.
Whereas the psql command worked in both cases:
when sslcert was just the leaf client cert without the chain
when sslcert contained client cert + chain
Not sure why psql was fine without the full chain, but it works now.
Details
Under-the-hood, pgx uses the pgconn module to create the connection. That, in turn, is just calling tls.X509KeyPair on the contents of the sslcert and sslkey files.
pgconn/config.go:
func configTLS(settings map[string]string, thisHost string, parseConfigOptions ParseConfigOptions) ([]*tls.Config, error) {
[...]
sslcert := settings["sslcert"]
sslkey := settings["sslkey"]
[...]
if sslcert != "" && sslkey != "" {
[...]
certfile, err := ioutil.ReadFile(sslcert)
if err != nil {
return nil, fmt.Errorf("unable to read cert: %w", err)
}
cert, err := tls.X509KeyPair(certfile, pemKey)
if err != nil {
return nil, fmt.Errorf("unable to load cert: %w", err)
}
tlsConfig.Certificates = []tls.Certificate{cert}