MD5 Hashed passwords: where does it get hashed? - postgresql

I am connecting to a local PostgreSQL database with an md5 hashed password.
It works but I want to understand what is happening under the hood.
Does pq hash the password before it is sent over the network? How would it know whether to hash it or leave it as plain text? The server (in pg_hba.conf) is the one who specifies the authentication method of how the password is sent over the connection.
Is there a handshake that goes on between pq and psql before the connection string with the password is sent over?
user := "foo_user"
password := "test"
dbname := "bar"
connectionString := fmt.Sprintf(
"user=%s password=%s dbname=%s",
user,
password,
dbname)
db, err := sql.Open("postgres", connectionString)
The user was created with a password via:
ALTER USER foo_user WITH PASSWORD 'test';
And verified that the password is being stored as a hash:
postgres=# SELECT rolname, rolpassword from pg_authid;
rolname | rolpassword
-------------------+-------------------------------------
postgres |
pg_signal_backend |
foo_user | md51083525553eab8f4090ada980d2b86e7
(3 rows)
And pg_hba.conf is completely unmodified:
# maintenance (custom daily cronjobs, replication, and similar tasks).
#
# Database administrative login by Unix domain socket
local all postgres peer
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all peer
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5

There are two places where password hashing takes place, and they should not be confused.
When a password is set for a role, it is concatenated with the user name and then hashed. This is the password stored in pg_authid, and that is the actual password used by PostgreSQL. PostgreSQL does not use the password you entered, but a hashed version of it, to keep attackers form stealing your password and then trying it outside of PostgreSQL (in case the same password is used in several places).
So to break into a database, you do not actually need to know the clear text password, the “actual” hashed password is enough.
There are two places where the hashing can take place:
On the server side, if you use
CREATE/ALTER ROLE ... PASSWORD 'mypassword';
This is not so good, because the password is sent over the wire in clear text and it can show up in the database log.
On the client side, if you use
CREATE/ALTER ROLE ... PASSWORD 'hashedpassword';
This is better, and it is what psql uses internally if you use the \password command.
During session authentication, if the authentication method specified for the database and user demands it. The server will then respond to the connection request with a AuthenticationMD5Password message (see the documentation).
The client then hashes the clear text password to obtain the actual password, then hashes it again with a random “salt” provided by the server.
The server hashes the password from pg_authid in the same fashion and compares the result.

Related

pgbouncer and scram-sha-256 setup

I was able to get SCRAM-SHA-256 authentication to work with pgpool, but I haven't been able to find a good example how to set this up in pgbouncer. I'm trying to use auth_query. In postgres, the user that pgbouncer will connect as has the password encrypted and stored in SCRAM-SHA-256. But I don't know how to create the entries in userlist.txt. This is supposed to be the format:
SCRAM-SHA-256$<iterations>:<salt>$<storedkey>:<serverkey>
What exactly are the storedkey and serverkey and how do I generate those? Which tools can I use to create this? In pgpool, I can use pg_enc but I don't see anything for pgbouncer.
You don't construct the SCRAM hashed password yourself, you get it by querying the pg_authid table in the PostgreSQL database:
SELECT rolpassword
FROM pg_authid
WHERE rolname = 'pgbouncer';
However, as the documentation says:
The passwords or secrets stored in the authentication file serve two purposes. First, they are used to verify the passwords of incoming client connections, if a password-based authentication method is configured. Second, they are used as the passwords for outgoing connections to the backend server, if the backend server requires password-based authentication (unless the password is specified directly in the database's connection string). The latter works if the password is stored in plain text or MD5-hashed. SCRAM secrets can only be used for logging into a server if the client authentication also uses SCRAM, the PgBouncer database definition does not specify a user name, and the SCRAM secrets are identical in PgBouncer and the PostgreSQL server (same salt and iterations, not merely the same password). This is due to an inherent security property of SCRAM: The stored SCRAM secret cannot by itself be used for deriving login credentials.
So if that user is used as auth_user, you cannot use a SCRAM hashed password for that user, but you have to use the clear text password.

Why Is Knex.js Failing on Password Authentication When My Database Has No Password?

I'm trying to connect to my database using Knex like so:
const client = knex({
client: 'postgresql',
connection: {
host: '127.0.0.1',
user: 'me',
database: 'my_db'
}
});
client('some_table').then(console.log);
When I created the database (using createdb my_db at the command line) I set no password. At the command line I can run psql -d my_db and it works just fine.
However, when I try to use Knex, I get an error:
Unhandled rejection error: password authentication failed for user "me"
This happens whether I set a null password, an empty string ('') password, or leave the field off of the configuration entirely.
Can anyone explain why Knex insists on failing password authentication ... when there is no password (and when I can connect without one at the command line just fine)?
PostgreSQL does not permit login based on the presence/absence of a password. Rather, all login authentication is handled via pg_hba.conf. In your particular case--creating the me user without a password (or using null or '', as you would describe it)--the absence of the password doesn't necessarily allow your user to log in. In fact, setting no password will not allow that user to log in unless pg_hba.conf was set to some password-less setting (i.e., peer or trust).
If you want password-less login for the me user (I assume for testing/development purposes, as having password-less login is not good security practice in production), you could simply set trust-level authentication in pg_hba.conf:
#conn_origin database user ip_mask auth_method
host all me 0.0.0.0/0 trust
The more secure method of implementing password-less authentication would be to use a .pgpass file or set the PGPASSWORD environment variable. Seeing that you are trying to use knex.js, you may be better off with tweaking pg_hba.conf as above. Again, I don't know what your intentions are, but please be safe out there ;)
Disclosure: I work for EnterpriseDB (EDB)

postgres uses a database password or a user password

I imported a postgres database in my local postgres server.
I had to connect to the database (to allows django to retrive data) using the file called setup.local.
There is required to specify: DB_HOST=localhost, DB_NAME, DB_USER, DB_PASSWORD.
DB_HOST is localhost without any doubt. The DB_name is the one I choose importing (psql imported_db < downloaded_DB)
DB_USER is my_name (or I can change the owner ALTER DATABASE imported_db OWNER TO other_name).
The wire thing, for me, is that I have to use the user (either the my_name or other_name) password and not the database password (even if the variable name is DB_PASSWORD).
So the question:
does a psql database have a password or just the roles/users have ones and use them to access the database?
Andrea
Passwords are set for USER and ROLE only. A user may access multiple databases, according to the GRANTs for the ROLE.
See also:
https://www.postgresql.org/docs/10/static/ddl-priv.html
https://www.postgresql.org/docs/10/static/client-authentication.html
https://www.postgresql.org/docs/10/static/user-manag.html
DB_HOST=localhost is a key here. Look into the pg_hba.conf you will find ident against localhost connections most probably.
https://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-IDENT
When ident is specified for a local (non-TCP/IP) connection, peer
authentication (see Section 20.3.6) will be used instead.
https://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-PEER
The peer authentication method works by obtaining the client's
operating system user name from the kernel and using it as the allowed
database user name (with optional user name mapping). This method is
only supported on local connections.

Postgres: 'role does not exist' using dblink, but it does

So this is super strange. I'm trying to do a simple select using dblink as such:
SELECT * FROM dblink('dbname=my_db_name, user=my_user,
password=password, hostaddr=127.0.0.1', 'SELECT action, object,
created_at, id FROM my_table') AS de(ACTION VARCHAR, OBJECT VARCHAR,
created_at TIMESTAMP, id INT)
And I immediately am getting an error message of:
PG::SqlclientUnableToEstablishSqlconnection: ERROR: could not
establish connection DETAIL: FATAL: role "my_user," does not exist
But if I connect to psql locally and print out the list of users using \du you'll see it's listed:
List of roles
Role name | Attributes | Member of
---------------+------------------------------------------------+-----------
MyName | Superuser, Create role, Create DB, Replication | {}
my_user | Create DB | {}
I'm really at a loss as to how to fix this, googling hasn't helped me much at all either. Any thoughts as to why it's giving me this error message?
When I connect using my superuser account I don't need to specify a password and the dblink runs fine, so I'm quite confused. Here is by pg_hba.conf:
local all all trust
local all all md5
host all all 192.168.33.1/24 trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 trust
host all all ::1/128 md5
Thanks in advance!
Try removing the commas from the connection string.
SELECT * FROM dblink('dbname=my_db_name user=my_user password=password hostaddr=127.0.0.1', 'SELECT action, object, created_at, id FROM my_table') AS de(ACTION VARCHAR, OBJECT VARCHAR, created_at TIMESTAMP, id INT)
This should fix the issue where it tries to authenticate as my_user,.
Secondly, your pg_hba configuration is an issue. Because you have the 'trust' method first, it will be used. Either try placing the 'md5' entries before the 'trust' entries, or remove the 'trust' entries. (Set a password for your own postgres account first, so that you can still authenticate once passwords are required.)
The reason for the second issue is that if 'trust' is enabled for the hostname you're connected from, it will authenticate the user without requiring a password, and since non-superusers cannot connect without a password, it will fail to connect.
Finally, the user running the query that calls dblink must be authenticated using a password. So if you're using 'trust' authentication to connect to the database, then running dblink, you will also get an error. To fix this, change to md5 authentication and connect using a password.
You're using commas in the connection string. The error message says:
FATAL: role "my_user," does not exist
Which indicates the exact error in this case. This is not the way do do it. You should only space separate the items in the connection string like:
SELECT * FROM dblink('dbname=my_db_name user=my_user password=password hostaddr=127.0.0.1', 'SELECT action, object, created_at, id FROM my_table') AS de(ACTION VARCHAR, OBJECT VARCHAR, created_at TIMESTAMP, id INT)

PostgreSQL - Password authentication fail after adding group roles

I'm pretty new to PostreSQL. I've spent all morning trying to get user logins working properly, and I'm terribly frustrated by now!
So, I have a PostGIS database, version 9.2, as part of the OpenGeo suite of software. I could access the database with the postgres user, but want to make a group role and user with access to a database so that it can create tables and update/select/delete etc. in that database.
I can create a user, that works, and I can login with that user. I can create a group role and assign privileges to the role. I can then add the user to the group role, and then can NO LONGER LOGIN!
It even got so that when I added the user postgres to the group, that user can't login. I remove the group but the inability to login persists.
Now, I have played with pg_hba.conf a lot. And I can now login as postgres, but only if 'trust' is enabled, and I can't login with any software, such as PGAdminIII.
It would be great to get some advice as to what is going wrong, and to enable authenticated logins again.
pg_hba:
Code:
# Database administrative login by Unix domain socket
local all postgres trust
local all gisadmin trust
# TYPE DATABASE USER ADDRESS METHOD
local all opengeo md5
local all opengeo md5
# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 md5
host all gisadmin localhost trust
# IPv6 local connections:
host all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local replication postgres peer
#host replication postgres 127.0.0.1/32 md5
#host replication postgres ::1/128 md5
Just as a final note, I would like to be able to login via a SSH tunnel. I could do that originally with the 'postgres' user, but now that I added and removed that user from a group, I can't.
Edit: the error messages...
If I login via SSH tunnel in PGAdmin with the correct password, I get the error "FATAL: password authentication failed for the user ..."
If I try to login via SSH tunnel in PGAdmin without a password, while the trust option is set, I get the error: "error connecting to the server: fe_sendauth: no password supplied".
The first error still happens when logging in locally, just via SSH and psql, but the second one goes away and I can log in.
output of \du+:
output of \dg+
I think this behaviour may be related with pgAdminIII, because i'm having similar issues and it seems every time i connect to my db as postgres using pgAIII and look at the definition tab in user properties, the check box for the "expiring date" is checked and either 1/1/1970 or 31/12/1969 are set as expiring date. The solution proposed by Daniel works, so it's obvious the problem is the expiration of the password.
Seems that this bug was corrected in pgAdmin 1.16.2 as you can see in the changelog:
http://www.pgadmin.org/development/changelog.php
Cheers
In the \du+ output, the Password valid until 1970-01-01 00:00:00+00... look quite suspicious. Strictly speaking, the passwords for gisadmin, postgisrw and postgres are no longer valid so that might explain why password-based authentication methods fail for these accounts.
You may try ALTER USER username valid until 'infinity' on these accounts and see if that solves the problem.
Also when connecting to PG through a SSH tunnel, be aware that the pg_hba.conf entries starting with host are ignored. These entries are only considered for connections to Unix domain sockets. Sometimes users confuse that with connections from localhost to itself.