I have a RDS PSQL14 database on eu-central-1 and would like to connect this to Quicksight as a new data source.
However, I always get the following error:
sourceErrorCode: GENERIC_SQL_EXCEPTION
sourceErrorMessage: The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.
I have no clue how could I edit the pg_hba.conf file. Though I know that the VPC in which the RDS is in would allow the connection.
Most likely this is something with sha256 vs md5 authentication or so I've read in a couple of posts, but I don't know for sure. Please someone educate me :)
This was a huge time waster.
tldr: psql 13 and up uses scram-sha-256 but older version only support md5.
The reason behind this is that AWS Quicksight is using PostgreSQL JDBC driver 42.2.1
If you try to connect this with any of the newer psql versions it will fail due to a change made to the password authentication method used in the more recent versions of PostgreSQL (scram-sha-256). However, the 42.2.x driver only supports connecting via md5 passwords.
How to solve?
1. Downgrade:
If you downgrade to version 12.9 or below the problem should sort itself out.
2. Change Auth to md5:
!! it will affect all users and connections !!
Create a new Parameter group.
Once a new Parameter group is created > Search for password_encryption > Edit parameters
Select md5 (change the "rds.accepted_password_auth_method" parameter to allow md5 + scram which would allow you to create a user with md5 for QuickSight to connect with).
Save changes
Then you can modify your database to use the DB parameter group created in step 1.
Create a new user to be used in the QuickSight authentication with the RDS instance with the necessary permissions.
3. New Quicksight user with session-level md5:
Verify current password_encryption value:
show password_encryption;
Set the session variable of the parameter to 'md5:
set password_encryption = 'md5';
Create a user and assign it the necessary credentials
create user (username) with password '(password)';
grant connect on database (database) to (username);
Use the user to connect from QuickSight and it should be able to connect successfully using the "md5" encryption and not "scram-sha-256".
Related
I'm working on a project with the kosmtik tool together with osm2pgsql to be able to generate data for OPENS STREET MAP directly from my postgres database, and I'm having this connection problem when I make requests to the database, they are being accessed by the file localconfig.js and together with project.mml
Has anyone had a similar problem or worked with this tool to be able to use it in OSM ??
From PostgreSQL authentication method 10 not supported, you can understand that your client cannot authenticate using scram-sha-256, and from POSTGRES_HOST_AUTH_METHOD in https://hub.docker.com/_/postgres, you get that scram-sha-256 is the default auth method when using postgres 14 and later.
So, you can either update the postgresql client you use, or downgrade postgres to v13. You could also choose to disable passwords or use md5 auth method on postgres.
It does looks like kosmtik uses mapnik, which has Unable to connect to Postgresql 14 with scram-sha-256 password hashing · Issue #4283 · mapnik/mapnik.
I connect to my PostgreSQL databases (which run on AWS EC2 instances) using certificate authentication (and not passwords). An example of a psql command I would use to connect to one of my databases is:
psql "host=<AWS EC2 instance> user=<db user> sslcert=<path to .crt> sslkey=<path to .key> sslrootcert=<path to .crt> sslmode=require dbname=<db name>"
I would like to be able to connect Tableau Desktop to one of my databases. The standard PostgreSQL connector does not allow me to specify which SSL certs to use (but it does have a Require SSL checkbox - which I think is for encryption not authentication).
So I have tried to use the ODBC connector. I make the following entries:
Connect Using
Driver: PostgreSQL Unicode
Connection Attributes
Server: <AWS EC2 instance>
Port: 5432
Database: <db name>
Username: <db user>
String Extras: sslcert=<path to .crt>; sslkey=<path to .key>; sslrootcert=<path to .crt>; sslmode=require
By much trial and error I think the String Extras require a semicolon as a delimiter, but I still cannot connect. I get the error message:
An error occurred while communicating with Other Databases (ODBC).
Unable to connect to the server. Check that the server is running and that you have access privileges to the requested database.
FATAL: connection requires a valid client certificate
Generic ODBC requires additional configuration. The driver and DSN (data source name) must be installed and configured to match the connection.
Unable to connect to the server "<AWS EC2 instance>" using the driver "PostgreSQL Unicode". Check that the server is running and that you have access privileges to the requested database.
I would be really grateful to hear from anyone who has connected Tableau Desktop to PostgreSQL using certificate authentication and could tell me what I’m doing wrong. Cheers!
I managed to fix this myself, and just in case there's someone out there who might be vaguely interested I'll go through the salient features:
1. Created a DSN (Data Store Name)
This made testing much easier rather than constantly retyping details into dialog boxes.
Rather than create manually I used this: http://www.odbcmanager.net/
On Mac OS I had to run it as sudo from the terminal or it wouldn't create anything.
I created a User DSN, which on my Mac have details stored in /Users/<user name>/.odbc.ini
It picked up the PostgreSQL Unicode driver that I'd previously installed. On Mac OS you might need to do brew install brew install psqlodbc and/or brew install unixodbc
Most of the configuration was specified by adding key/value pairs e.g. DBNAME <db name>
Set SSLMODE verify-ca (changed from require to make behaviour specific - psql assumes verify-ca/verify-full if SSL certs are supplied)
The cert paths are set with key Pqopt, value sslcert=<path to .crt> sslkey=<path to .key> sslrootcert=<path to .crt>. Must be lower case and only spaces between entries!
Here's what it added to my user's odbc.ini file:
[<DSN name>]
Driver = <path to driver, this was mine /usr/local/lib/psqlodbcw.so>
Description = <description>
SSLMODE = verify-ca
HOST = <host>
DBNAME = <database>
PORT = 5432
UID = <db user>
Pqopt = sslcert=<path to .crt> sslkey=<path to .key> sslrootcert=<path to .crt>
2. Tableau Desktop Connection
Select Other Database (ODBC) connector.
Select the DSN you created previously (it should be automatically picked up), it will automatically populate host, port, db and user fields, then click Sign In.
Once signed in behaviour was slightly different to the dedicated PostgreSQL connector. Tables did not appear on left-hand side initially. But they are accessible - they can all be displayed by clicking on the 'contains' radio button and searching with a blank name.
Hope this helps. Let me know if you've got any questions.
Changing the password is easily done through the console. Is there any way to change the master username after creation on RDS for PostgreSQL? If so, how?
You can't change username. You can check the following links that describe how to change master password and if Amazon adds the ability to change username you will find there:
Try to find at AWS CLI for RDS:
modify-db-instance --db-instance-identifier <value> --master-user-password (string)
--master-user-password (string)
The new password for the DB instance master user. Can be any printable
ASCII character except "/", """, or "#".
Changing this parameter does not result in an outage and the change is
asynchronously applied as soon as possible. Between the time of the
request and the completion of the request, the MasterUserPassword
element exists in the PendingModifiedValues element of the operation
response. Default: Uses existing setting
Constraints: Must be 8 to 41 alphanumeric characters (MySQL, MariaDB,
and Amazon Aurora), 8 to 30 alphanumeric characters (Oracle), or 8 to
128 alphanumeric characters (SQL Server).
The Amazon RDS Command Line Interface (CLI) has been deprecated. Instead, use the AWS CLI for RDS.
Via the AWS Management Console, choose the instance you need to reset the password for, click ‘Modify’ then choose a new master password.
If
you don’t want to use the AWS Console, you can use the
rds-modify-db-instance command (as per Amazon’s documentation for RDS)
to reset it directly, given the AWS command line tools:
rds-modify-db-instance instance-name --master-user-password
examplepassword
No. As of April 2019 one cannot reset the 'master username'.
You cannot do it directly. However you can use the database migration service from AWS:
https://aws.amazon.com/dms/
Essentially you define the current database instance as your source and the new database with the correct username as your target of the migration.
This way you migrate the data from one to another database instance. As such you can change all properties including the username.
This approach has some drawbacks:
You need to configure the migration. Which takes a bit of time.
The data is migrated. This may lead unexpected behavior since not everything is eventually migrated (e.g. views etc.)
It depends how you setup everything you may experience a downtime.
Though this may not be ideal for every use-case, I did find a workaround that allows for changing the username of the master user of an AWS RDS DB.
I am using PgAdmin4 with PostgreSQL 14 at the time of writing this answer.
Login with the master user you want to change the name of
Create a new user with the following privileges and membership
Privileges and Membership
Can login - yes
Superuser - no (not possible with a managed AWS RDS DB instance, if you need complete superuser access DO NOT use a managed AWS RDS DB)
Create roles - yes
Create databases - yes
Inherit rights from the parent roles - yes
Can initiate streaming replication and backups - no (again, not possible directly without superuser permission)
Be sure to note the password used, as you will need to access this new account at least 1 time to complete the name change
Register a server with the credentials created in step 2. Disconnect from the server but do NOT remove it! Connect to the new server created
Expand Login/Group Roles and click on the master user whom you are changing the name
Click the edit icon, edit the name, and save.
Right click the server with the master username, select Properties
Update the name under the General tab if desired
Update the username under the Connection tab to whatever you changed the master username above
Save and reconnect to the server with the master user
You have successfully updated the master user's name on a managed AWS RDS DB instance, proud of you!
As #tdubs's answer states, it is possible to change the master username for a Postgres DB instance in AWS RDS. Whether it is advisable – probably not.
Here are the SQL commands you need to issue:
Create a temporary user with the CREATEROLE privilege (while being logged in with the old master user)
CREATE ROLE temp_master PASSWORD '<temporary password>' LOGIN CREATEROLE;
Now connect to the database with the temp_master user
ALTER ROLE "<old_master_username>" RENAME TO "<new_master_username>";
-- NOTICE: MD5 password cleared because of role rename
ALTER ROLE "<new_master_username>" PASSWORD '<new password>';
Now connect to the database with the <new_master_username> user in order to clean up the temporary role
DROP ROLE temp_master;
And you're done!
Warning
AWS RDS does not know that the master username has been changed, so it will keep displaying the old one and assumes that is still the master username.
This means that if you use the AWS CLI or website to update the master password, it will have no effect.
And when connecting to the database with psql you'll see:
WARNING: role "<old_master_username>" does not exist
I am working on an open source application that has PostgreSQL as its default DBMS. Now when I install it on my system, its configuration is so that PostgreSQL also gets installed with it.
My problem is with getting access to the installed PostgreSQL database. The database that gets created during installation is named iviewdb.
I read at many forums that the default superuser is postgres, but when I try to get access to the database using this username through a command prompt, it prompts me for password that I don't have.
I wanted to know from where in the PostgreSQL installation directory
the default username and password with the port number to access the database is stored. I have even tried changing the pg_hba.conf file, but that creates a problem with the application and it won't start then.
How can I find the password for this database? I am working in a Windows environment.
The password isn't stored in an easily recoverable manner, and if you change the password, the application may not be able to access the database properly. You may find the password in the application settings or documentation, though.
If you decide to risk changing the postgres user's password, stop the application and PostgreSQL service, and then edit pg_hba.conf. Add (or change if it already exists) a line (if it doesn't exist, add it before any other "host...." lines):
host all all 127.0.0.1/32 trust
And restart the PostgreSQL service. That should give you access from localhost, where you could change the postgres user's password, or add yourself another user with the permissions you want. Then set the pg_hba.conf file back the way it was and restart.
I've encountered this similar problem, and I noticed that the default being set for PostgreSQL upon installation in my case is as follows:
username = postgres
password = ' '
I am trying to setup PostgreSQL and allow only certain Windows users to access the data from the database. Setting up Windows Authentication is Quite easy with MS SQL, but I can't figure out how to set it up in PostgreSQL.
I have gone through the documentation at http://www.postgresql.org/docs/current/static/auth-methods.html
and edited the pg_hba file. But after doing so, the PostgreSQL service fails to start.
Is the Postgresql server running on Windows as well as the clients then you might test with this to see if this works:
host all all 0.0.0.0/0 sspi
Magnus Hagander, a Postgresql developer, elaborates on this:
"All users connecting from the local machine, your domain, or a trusted domain will be automatically authenticated using the SSPI configured authentication (you can enable/disable things like NTLMv2 or LM using Group Policy - it's a Windows configuration, not a PostgreSQL one). You still need to create the login role in PostgreSQL, but that's it. Note that the domain is not verified at all, only the username. So the user Administrator in your primary and a trusted domain will be considered the same user if they try to connect to PostgreSQL. Note that this method is not compatible with Unix clients."
If you mix Unix-Windows then you have to resort to kerberos using GSSAPI which means you have to do some configuration. This article on deploying Pg in Windows environments may perhaps lead you in the right path.
If anyone else encouters this like I did so starting from 9.5 you wil need to add an optional parameter both to the ipv4 and ipv6 in order for this to work
include_realm=0
so the whole thing will look like
host all your_username 127.0.0.1/32 sspi include_realm=0