We have a Golang app built with Gorm that connects with PostgreSQL 14 (hosted in the same machine) with an SSL mode of verify-full. The certificates were issued by ZeroSSL, and generated with the help of Acme.sh. In the PostgreSQL configuration, we have set the following certificate-related variables:
ssl = on
ssl_cert_file = '/etc/ssl/certs/our-certificate.pem'
ssl_key_file = '/etc/ssl/private/our-certificate.key'
The Golang app and the PostgreSQL server are hosted in an Ubuntu 20.04 machine.
Our Golang app depends on a bunch of environment variables for, among others, database passwords and usernames. When we pass those variables to our app directly, i.e.:
DB_HOST=dbhost DB_NAME=dbname DB_USERNAME=username DB_PASSWORD=123456789 [...] ./go-app
Everything works well.
On the other hand, we also tried exporting those variables first. Exporting the variables is done by putting the variables inside an env file first, then doing the following:
set -a
source env-file.env
set +a
The environment variables are exported now at this point, and we can echo the variables from the terminal.
Unfortunately, when our Golang app reaches the point where it would attempt to connect with the PostgreSQL server, we would get the following error message:
error="failed to connect to `host=dbhost user=username database=dbname`: failed SASL auth (FATAL: password authentication failed for user \"username\" (SQLSTATE 28P01))"
I have already logged the contents of the exported database-related environment variables to the console, and they were of the same values as the variables directly passed to the process.
Why does this happen? It's a weird behaviour for me, but I believe I am simply lacking some necessary information.
Related
I'm starting to use Postgres as a database in my application, and I need to configure an agent that serves as a build server as part of the build we are running unit tests.
I need to configure the agent and I'm having trouble as the agent name is different for each machine, and there is a configuration in Postgres that needs to be applied when using windows authentication in the pg_hba.conf and pg_indent.conf.
In the pg_indent.conf I need to use a system user, and I have one, but in the following syntax user#comutername:
# MAPNAME SYSTEM-USERNAME PG-USERNAME
MapForSSPI user#computername postgres
The issue is when the agent is starting (we have a dynamic pool) the name of the computer is different, and I want to avoid using the computer name, and use something like user#localhost or the equivalent in Postgres.
How I can achieve that?
You can use wildcards in pg_ident.conf:
MapForSSPI /^user# postgres
That will match everything that starts with user#. If the “system user name” starts with a /, the rest of the string will be treated as a regular expression.
Please promise me that you will not allow your application to connect as a superuser, that is an unnecessary risk.
I have 2 systems. system A and system B and both are DB2 servers. I want to be able to access system B database from system A. Both have a database called TESTDB. I am trying to run the following command to create a server.
CREATE WRAPPER "drdawrapper"
LIBRARY 'libdb2drda.so'
OPTIONS (DB2_FENCED 'Y'
);
db2 "CREATE SERVER "PRD_SERVER_SSL_FLEX" TYPE DB2/UDB VERSION '11' WRAPPER "drdawrapper" AUTHORIZATION "xyz" PASSWORD "xyz" OPTIONS (DB2_CONCAT_NULL_NULL 'Y',DB2_VARCHAR_BLANKPADDED_COMPARISON 'Y',DBNAME 'TESTDB',HOST '169.62.253.230',NO_EMPTY_STRING 'N',PORT '50001',SECURITY 'SSL',STRING_UNITS 'S');"
But I keep getting:
DB21034E The command was processed as an SQL statement because it was not a
valid Command Line Processor command. During SQL processing it returned:
SQL1101N Remote database "TESTDB" on node "<unknown>" could not be accessed
with the specified authorization id and password. SQLSTATE=08004
Node directory:
db2 list node directory
Node Directory
Number of entries in the directory = 1
Node 1 entry:
Node name = TESTNODE
Comment =
Directory entry type = LOCAL
Protocol = TCPIP
Hostname = 123.21.23.12
Service name = 50001
The credentials are correct. I am not sure what node is it looking for. Any pointers?
Your question is more about configuration than programming.
As you appear to be encrypting the federated connection it can be wise to first verify that the encrypted connection works at the command-line, separately from federation. This irons out a lot of the detail and is easier to troubleshoot. After you get that working, you can then begin on encrypting the federated connection.
Please follow the detailed instructions here (choose the correct Db2-version):
You have to know in advance which kind of SSL/TLS trust verification you want (i.e. either single cert (client trusts the server - simplest and easiest), or multiple certs (both sides trust the other - more setup, arguably more secure), because this determines the configuration.
Ensure both of your Db2 instances and databases are properly configured for SSL.
Catalog the remote-node locally with security SSL (db2 catalog tcpip node ... remote ... server ...security ssl)
Catalog the remote-database locally on the new node name (db2 catalog database ... at node ...) followed by db2 terminate .
Verify a command-line connect to the remote database using the federated credentials, using the configured db2dsdriver.cfg if using SSLSERVERCERTIFICATE method, or using the keystore/stash configuration ( db2 connect to remotedb user ... using ... ). Use the same userid/password that you will use later in the create server command.
Once that command-line connect works, you can proceed with the encrypted federation link, via db2 create wrapper... and db2 create server....
There's no need to use quotes around the wrapper name, just let it fold, otherwise quotes are just annoying redundant noise, although it is not a mistake.
Inside the script for create server command options instead of AUTHORIZATION "xyz" PASSWORD "xyz" use AUTHORIZATION \"xyz\" PASSWORD \"xyz\" (i.e. escape the quotes).
For one-sided trust, use SSL_SERVERCERTIFICATE in the create server options clause and ensure the value is accurate (fully qualified path to the remote-db2instance-certificate-file), and that the file/directory permissions are valid.
For mutual trusts, use both SSL_KEYSTORE and SSL_KEYSTASH keywords with correct values, in the create server options clause (having previously ensured your keystores are properly populated, as verified by a command-line connect above).
You may also want to consider create user mapping depending on the requirements.
Finally you can create your nicknames, and test out the federated link by querying those nicknames.
I am trying to to install Kylo from scratch. I am accessing postgresql from a remote machine. I am able to access postgresql on the remote machine from my machine. But I can't put up the kylo service.
Here is my kylo_service.log file
Please ensure that the following properties are set correctly in your kylo-services/conf/application.properties file:
spring.datasource.username=<REPLACE_ME_WITH_USERNAME>
spring.datasource.password=<REPLACE_ME_WITH_PASSWORD>
hive.datasource.username=<REPLACE_ME_WITH_USERNAME>
hive.metastore.datasource.username=<REPLACE_ME_WITH_USERNAME>
hive.metastore.datasource.password=<REPLACE_ME_WITH_PASSWORD>
nifi.service.mysql.database_user=<REPLACE_ME_WITH_USERNAME>
nifi.service.mysql.password=<REPLACE_ME_WITH_PASSWORD>
nifi.service.kylo_mysql.database_user=<REPLACE_ME_WITH_USERNAME>
nifi.service.kylo_mysql.password=<REPLACE_ME_WITH_PASSWORD>
#Note: The value for this property is the password for the dladmin user.
nifi.service.kylo_metadata_service.rest_client_password=<REPLACE_ME_WITH_PASSWORD>
modeshape.datasource.username=${spring.datasource.username}
modeshape.datasource.password=${spring.datasource.password}
metadata.datasource.username=${spring.datasource.username}
metadata.datasource.password=${spring.datasource.password}
http://kylo.readthedocs.io/en/latest/installation/AdditionalConfiguration.html
We have a Windows EC2 instance on which we are running a custom command line application (C# console app using NpgSQL) to connect to a PostgreSQL RDS instance. Based on the instructions here:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.SSL
we created a new DB parameter group with rds.force_ssl set to 1 and rebooted our RDS instance. We also downloaded and imported to Windows the pem file referenced on the page.
I was able to connect to the RDS instance from my Windows EC2 instance via pgAdmin by specifying SSL mode as Verify-Full. Our command-line application reads connection strings from a file and they look like this now that I've added the sslmode parameter:
Server=OurInstanceAddress;Port=5432;SearchPath='$user,public,topology';Database=OurDatabase;User Id=username;Password=mypassword;sslmode=verify-full;
Using this connection string failed with the error referenced at the bottom of the page:
FATAL: no pg_hba.conf entry for host "host.ip", user "someuser", database "postgres", SSL off
I tried adding the sslrootcert parameter, but I'm not sure if I'm dealing with it properly. I tried using the example (sslrootcert=rds-ssl-ca-cert.pem) and I tried using the name of the pem that I downloaded. I feel like there is something about the path information that I'm giving to the sslrootcert parameter that isn't right, especially in a Windows environment. I've tried using the name, I've tried using the following paths:
- sslrootcert=C:\keys\rds-combined-ca-bundle.pem - single backslash
- sslrootcert=C:\\\keys\\\rds-combined-ca-bundle.pem - double backslash
- sslrootcert=C:/keys/rds-combined-ca-bundle.pem - Linux style backslash
All of these produced the same error mentioned above.
Any insight would be appreciated.
I solved it using the environment variables instead for specifiying cert paths in connection url
-DPGSSLROOTCERT=/certs/root.crt
-DPGSSLKEY=/certs/amazon-postgresql.key
-PGSSLCERT=/certs/amazon-postgresql.crt
Although I'm in cygwin. There are some hints in the documentation when using windows here https://www.postgresql.org/docs/9.0/static/libpq-ssl.html
I'm attempting to connect to a remote server using a foreign data wrapper to transfer some data between databases but keey getting 'SSLMode Value "require" Invalid When SSL Support Is Not Compiled In'. These are the snippets I'm running:
CREATE SERVER remote FOREIGN DATA WRAPPER postgres_fdw
OPTIONS(dbname '...', host '...', port '...', sslmode 'require');
CREATE USER MAPPING FOR USER SERVER remote
OPTIONS (user '...', password '...');
CREATE FOREIGN TABLE remote.accounts (
...
);
SELECT * FROM remote.accounts;
DETAIL: sslmode value "require" invalid when SSL support is not compiled in
I've tried connecting into psql using "sslmode=require" - but that doesn't seem to help. I've also included sslmode as a flag when creating the server. The postgres instance I'm connecting to is on Heroku. The queries run if I connect to a different database on a remote server - but fail from my local database. How can I fix the SSL support is not compiled in?
The local PostgreSQL instance uses the local libpq library to connect to the remote instance.
If, at compile time, that library has been configured with --with-openssl (an option to pass to the top-level ./configure script of postgres source tree), then it gets dynamically linked to the openssl library and the call sites to encrypt a session are added.
If on the other hand, it was not configured with this option, the libpq produced doesn't have any SSL capability at all, and it's impossible with that library to connect to remote servers that require SSL.
According to the error message, you are in the second case.
The only way out of this problem is to replace your installed libpq library by another one that is compiled with SSL support, which also mean that you have OpenSSL installed on your system. Either compile it yourself or install it through packages depending on your preferences. In general pre-compiled versions of PostgreSQL tend to have SSL support enabled these days.
Even if libpq sources are inside the server's source, it's possible to compile them separately and use just that library independently of the rest.
There's a question on that: How to download, compile & install ONLY the libpq source on a server that DOES NOT have PostgreSQL installed