I'm having troubles to connect an existing heroku database to Google Datastudio. I'm trying to add the connection and I get the following:
Access denied, please check your username and password.
Now, I'm 100% sure that I'm correct on those credentials and the problem comes from somewhere else.
I've tried with different setup, either a free or a paid PSQL instance, nothing works.
I've also setup a dummy account on elephantsql and the connection worked the first time without any issue.
Do you have any idea of the cause of that problem?
Edit:
Just found https://www.en.advertisercommunity.com/t5/Data-Studio/Heroku-Postgres-lt-gt-Google-Data-Studio/m-p/1031729 which is not helpful at the time of writing this post.
Since the February 6, 2018 update, Google DataStudio allows SSL connections with PostgreSQL, which is necessary to connect to a database created via Heroku.
To enable SSL you need to provide client key+cert and server cert, which can be accomplished by taking the following steps:
Generate a self-signed cert + key with openssl for client key + certificate:
openssl req \
-newkey rsa:2048 -nodes -keyout client.key \
-x509 -days 365 -out client.crt
Use the postgres_get_server_cert.py script to get the self-signed server cert from heroku psql:
https://raw.githubusercontent.com/thusoy/postgres-mitm/master/postgres_get_server_cert.py
The problem is that Heroku Postgres requires an SSL connection which doesn't seem possible with Data Studio at the moment. Hopefully Google will add that option soon.
Make sure to run the openssl command on one line to generate the client.key and client.crt in one command. It took me a couple of tries of downloading the certificates (unable to reach host error), but this finally got me connected to Heroku Postgres with GDS.
I thought I would mention that I have used this for quite awhile, but every time my database undergoes maintenance it breaks and I have to manually reconnect the certificates. I developed a better approach - connect the data to Google BigQuery and do your blends there, and then use the BigQuery Community Connector. The charts are more performant this way AND you can now use query parameters on blended data.
Of course, DataStudio won't connect directly to Heroku Postgres for the same reason, so I use a service called Fivetran to grab the raw data and send it to Google BigQuery. There is a cost to this, of course, but for some projects it may be worth it. At some point I will move my database off of Heroku to either AWS or Google itself to allow a direct connection, but that is a larger project.
Related
I'm supporting two plants that are running a labview application, that is basically a black box and I can't modify, which is using a postgresql 9.2 database to which it connects using SSL. All is running on Debian 7.2. The company that built the system no longer exists.
Host:Port: 10.10.101.10:5432 (SSL require)
DB-Name: PPCDB (User: postgres)
postgres-Meldung:
SSL error: certificate verify failed
I have root, I've tried disabling SSL in postgres which works partially allowing the labview application to start, however none of the software clients work which is obviously a problem.
Can anyone guide me on how to renew the certificates so that the application will be able to connect to the database? Again, the application I can not modify, I can only stop it and start it. Is it even possible?
With the recent update from AWS regarding updating the SSL Certificate for RDS:
In order to update your certificates to rds-ca-2019, the database clients which use SSL will need to be updated first, to do so download the new certificate from https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem
I also tried the intermediate certificates.
We use RDS PostgreSQL db. I tried to connect to PostgreSQL using pgadmin with the new certificate.
I have specified the postgresql username, password, SSH tunnel along with the new pem certificate.
I am getting error "No password or public key available!"
I am able to connect using the existing pem file. Why am I not able to connect with the new pem file to the postgresql DB?
It may be too early to use the new certificate. I included rds-ca-2019-root.pem in my codebase and once deployed updated my database connection string to use it, but the app was unable to connect.
However, I was successful when I put rds-combined-ca-bundle.pem in my codebase that I got from here as mentioned here and referenced that in my database connection URL.
N.B. I am using mysql on a rails app from heroku, but the SSL challenges should be the same
I am going through the exact same process as the answered question found here (Connect a Google CloudSQL Postgres database to Data Studio), but I am not as advanced a user as most. I am encountering a similar problem to what was mentioned in the above question, and then some:
I have created a client certificate and downloaded the
client-cert.pem, client-key.pem and the server-ca.pem files to my
local machine.
I received the same error when attempting to link
Data Studio to our Google Cloud SQL Postgres Database as a data
source. " Can't reach the host. Please double check your connection parameters. Learn more about database connectors here. "
I tried running the following command " openssl pkcs8 -topk8 -inform PEM -outform DER -in client-key.pem -out client-key.pkcs8 -nocrypt " to change the format of the
client key in our Cloud Shell Project, but I received this error "
pkcs8: Cannot open input file client-key.pem, No such file or
directory ".
I would assume that I should not be running this command from the Cloud Shell Project terminal then. Would anybody know where I should be running this command instead? I can provide any extra material if needed. Thank you in advance - much appreciated.
It sounds like the file doesn't exist location where you are running the command. Did you upload the client-key.pem into Cloud Shell?
With questions like this, it looks like the only way to prevent man-in-the-middle attacks is to use sslmode=verify-full. This is also explained in the Postgresql docs.
When I use the DATABASE_URL provided by Heroku in my app, is there a guarantee that there won't be a MITM attack? (Or asked another way, does Heroku do something behind the scenes to guarantee that Heroku <-> Heroku-Postgres is secure?)
No, Heroku Postgres is not secure against a MitM. The highest risk is running pg:psql from the command line, since it's fairly easy to get between your laptop and Heroku by using a WiFi Pineapple or similar tools. Getting between your Dynos and the database is much harder. This is possible since Heroku hasn't created a CA to sign their database certificates, thus there's no trust root to use when you're connecting.
I did a writeup of actually performing such a MitM against Heroku Postgres here, please file a support ticket with Heroku saying you'd like to see them create a CA and issue a trusted root we can use to authenticate connections if you care about this.
Backstory:
I've gotten jdbc to connect to postgres using a client cert. In java I set the user in the properties, and the driver looks it up in the keystore and sends it along. All was good.
But I just found out that I won't be getting certs with a CN of pg-user. The certs I'll be getting will have a CN of pg-user.XYZ.foo.com & pg-user.ABC.foo.com. This looks like a job for username maps. Hey they even have regexp, it'll be perfect.
I got unix user root logging in to postgres as pg-user using a username map and local ident authentication using psql -d db -U pg-user. But in that case postgres knows BOTH that the user is root, AND is trying to log in as pg-user.
Problem:
What I can't figure out is how to tell the postgres jdbc driver to grab the cert from the keystore with a CN of pg-user.XYZ.foo.com, but present to postgres as user pg-user. It appears to be the single argument of user that controls both. Does anyone know how to do this?
This page includes a list of the connection options, but it doesn't seem to offer a way to split the user names. The closest I'm seeing is the option to write my own sslfactory, and I'm really hoping to avoid that...
Thanks to #harmic's comment I was able to solve this.
Starting with to following three files:
pg-user.pem which has a CN of pg-user.XYZ.foo.com
pg-user-chain.pem which contains the chain certs
pg-user.private_key which contains the private key for the cert
Then I created the pkcs12 file like this:
cat pg-user.pem pg-user-chain.pem > cert-and-chain.pem
openssl pkcs12 -export -out ssl_cert.p12 -in cert-and-chain.pem -name pg-user -inkey private_key.pem -passout pass:{password here}
After that I declared the user property for the connection to be pg-user, and it worked. In order to test, I altered the regexp in pg_ident to not match, then I could no longer log in, I changed it back and I could.