I am trying to follow the Debezium tutorial for Postgres and set up a replication user. I am creating the replication user as follows:
CREATE ROLE replication_role WITH REPLICATION LOGIN;
CREATE USER debezium WITH PASSWORD 'my-secret-pw';
GRANT replication_role TO debezium;
CREATE ROLE replication_group WITH LOGIN;
GRANT replication_group TO postgres;
GRANT replication_group TO debezium;
ALTER TABLE person OWNER TO replication_group;
and it sets up the replication user as follows:
List of roles
Role name | Attributes | Member of
-------------------+------------------------------------------------------------+--------------------------------------
debezium | | {replication_role,replication_group}
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {replication_group}
replication_group | | {}
replication_role | Replication | {}
When I try to start the Postgres source connector, I get the error as follows:
io.debezium.DebeziumException: Creation of replication slot failed
...
Caused by: org.postgresql.util.PSQLException: FATAL: must be superuser or replication role to start walsender
How do I fix the permissions issue so that I can start replicating from Postgres?
The replication attribute is not inherited. That attribute must be set directly on the role which will use it. You say you are following the tutorial, but as far as I can tell you are not. You seem to be confusing the permissions/attributes needed to create publications, with the ones needed to stream the publications.
Related
initially I have
Role name | Attributes | Member of
------------+------------------------------------------------------------+-----------
hezhenghao | Create DB | {}
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
and I typed
postgres=# REASSIGN OWNED BY hezhenghao to postgres
postgres-# ;
REASSIGN OWNED
postgres=# REASSIGN OWNED BY hezhenghao to postgres; REASSIGN OWNED
postgres=# DROP OWNED BY hezhenghao;
DROP OWNED
postgres=# DROP USER hezhenghao;
DROP ROLE
Now there is only one user
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------------------+-----------
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
However when I type psql in terminal, I still got asked to Password for user hezhenghao:
and then I would end up with psql: FATAL: password authentication failed for user "hezhenghao"
I am new to postgres so I don't really understand what's going on here. Can someone help me with this?
If you don't specify a user with -U then psql will default to the username of the user currently logged in. In this case, it sounds like that user is hezhenghao. Use -U postgres to log in as the postgres user.
Since Google Cloud SQL for PostgreSQL doesn't give us a superuser (not even the postgres user), I can't see what queries other sessions are running from pg_stat_activity, nor can I terminate other sessions if needed.
For example:
postgres#testdb=> select pg_terminate_backend(1584);
ERROR: 42501: must be a member of the role whose process is being terminated or member of pg_signal_backend
LOCATION: pg_terminate_backend, misc.c:319
Time: 23.800 ms
Without true superuser access, how do we do these things in Cloud SQL PostgreSQL instances? Only the cloudsqladmin account is superuser and AFAIK I can't become that:
postgres#testdb=> \dg
List of roles
Role name | Attributes | Member of
-------------------+------------------------------------------------------------+---------------------
cloudsqladmin | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
cloudsqlagent | Create role, Create DB | {cloudsqlsuperuser}
cloudsqlreplica | Replication | {}
cloudsqlsuperuser | Create role, Create DB | {}
don | Create role, Create DB | {cloudsqlsuperuser}
postgres | Create role, Create DB | {cloudsqlsuperuser}
postgres#testdb=> set role cloudsqladmin;
ERROR: 42501: permission denied to set role "cloudsqladmin"
LOCATION: call_string_check_hook, guc.c:9803
Time: 25.293 ms
FWIW, you can terminate a session if you log in as that session's user. Users can terminate any of their sessions, standard PostgreSQL stuff.
postgres#postgres=> select pg_terminate_backend(23644);
ERROR: 42501: must be a member of the role whose process is being terminated or member of pg_signal_backend
LOCATION: pg_terminate_backend, misc.c:319
don#postgres=> select pg_terminate_backend(23644);
pg_terminate_backend
----------------------
t
(1 row)
pg_terminate_backend says:
pg_terminate_backend(pid int) - Terminate a backend. This is also allowed if the calling role is a member of the role whose backend is being terminated or the calling role has been granted pg_signal_backend, however only superusers can terminate superuser backends.
When you are a member of Google's cloudsqlsuperuser (default for gcloud sql users create) you can grant yourself:
GRANT pg_signal_backend TO myuser;
Then you can terminate any session except Superusers:
SELECT pg_terminate_backend(pid), * FROM pg_stat_activity
WHERE usename = 'rogue_user' AND pid <> pg_backend_pid();
pg_signal_backend was introduced in v9.6 - exactly the version on Google Cloud!
Alternative way is to be within other role to be able to terminate session being me:
GRANT other TO me;
It is not possible to give a superuser role to a PostgreSQL user in Cloud SQL. {1}
In any case you can consult information about the instance in the graphs and logs inside the Cloud SQL section in Cloud Console. {2}
{1}: https://cloud.google.com/sql/docs/postgres/users#other_postgresql_users
{2}: https://cloud.google.com/sql/docs/postgres/diagnose-issues
I just created a new postgres RDS instace on aws (through the dashboard), and I gave it a default user, lets call him "jack".
When I logged in to the instance, I saw my created user "jack", and that he had a role "rds_superuser" attached. (so I thought that I can do the same things that I used to do with superuser on a regular postgres server).
I checked the documentation, I saw that wasn't possible.
As logged in as the default user "stan", I created a new database user like "stan", and wanted to create a new databases with the owner being the user "stan", I couldn't?
I entered something like this:
CREATE DATABASE foobar WITH OWNER = stan;
But I got an error, saying something like:
ERROR: must be member of role "stan"
So, what I did was, made the role "stan", logged out as the default user "jack", logged into the RDS instance as "stan", and created that database with him as the owner.
Since I had three different users, I had to repeat that last step three times.
My question, is there a way, that I can make the default user "jack" that I created during RDS postgres creation, capable of creating new databases (like superuser on a regular postgres server installation) and giving the different owners like this:
CREATE DATABASE foobar WITH OWNER = stan;
Tnx,
Tom
you were supposed to grant stan to rds_superuser in order to do that. You did:
rds=> create user stan;
CREATE ROLE
rds=> CREATE DATABASE foobar WITH OWNER = stan;
ERROR: must be member of role "stan"
you should:
rds=> grant stan to su_rdsadm;
GRANT ROLE
rds=> CREATE DATABASE foobar WITH OWNER = stan;
CREATE DATABASE
I did it as rds superuser:
rds=> \du+ su_rdsadm
List of roles
Role name | Attributes | Member of | Description
-------------+-------------------------------+----------------------+-------------
su_rdsadm | Create role, Create DB +| {rds_superuser,stan} |
| Password valid until infinity | |
rds=> select current_user;
current_user
--------------
su_rdsadm
(1 row)
It's good to know this further. This limitation of rds_superuser for ownership/grants and so on will keep hitting you until you grant role whose objects you want to manipulate (or on which behalf you want to grant) to rds superuser.
In Google's SQL Cloud Postgres service, when I create a database via the Web Console for a PostgreSQL instance, it automatically sets the owner of the database's default "public" schema to be cloudsqladmin. It seems I cannot change the ownership:
mydb=> \dn
List of schemas
Name | Owner
--------+---------------
public | cloudsqladmin
(1 row)
mydb=> alter schema public owner to postgres;
ERROR: must be owner of schema public
mydb=> \du
List of roles
Role name | Attributes | Member of
-------------------+------------------------------------------------+---------------------
cloudsqladmin | Superuser, Create role, Create DB, Replication | {}
cloudsqlagent | Create role, Create DB | {cloudsqlsuperuser}
cloudsqlreplica | Replication | {}
cloudsqlsuperuser | Create role, Create DB | {}
pg_signal_backend | Cannot login | {}
postgres | Create role, Create DB | {cloudsqlsuperuser}
mynewuser | Create role, Create DB | {cloudsqlsuperuser}
I also created a "mynewuser" through the web console, and cannot remove the "mynewuser" from the "cloudsqlsuperuser" group:
mydb=> alter group cloudsqlsuperuser drop user mynewuser;
ERROR: "cloudsqlsuperuser" can't be altered
If I wanted to create a database with a public schema that only a new user has access to (and owns), should I be doing this outside of the Google web ui? It seems like any databases I create are owned by cloudsqladmin, and any users I create are those "cloudsqlsuperuser" members. If I wanted to restrict permissions for a user, should I create that user normally via psql and bypass the web ui altogether?
From my experience, you seem to have to bypass the web ui / cli tool entirely.
When you create a database through the cli tool:
gcloud sql databases create DBNAME --instance INSTANCE
It assigns ownership to cloudsqlsuperuser, same as through the gui from the sounds of it.
When I have created a user specifically through the CLI tool:
gcloud sql users create USER 'host' --instance INSTANCE
Those users get the same permissions as cloudsqlsuperuser. In this case, it is possible to alter the ownership of the database. I had success through psql command connecting as the user I wanted to own the database and running:
ALTER DATABASE database OWNER TO user;
However if the user was created via psql (not glcoud cli), then the permission are not the same and the above failed.
I'd be tempted to create your instance, set the 'postgres' users password through the tool, then psql into the instance from there and do everything you need via sql commands. I think the tool does some things very nicely (as does the UI), but its a pain later on.
If anyone knows better, I'd love to hear how you can work with the default gcloud user.
Basically what happens here is that a usual CREATE DATABASE statement seems to create a new database based on the template0 database. This database is owned by cloudsqladmin. A role only Google has access to. When the gcloud or web GUI is used, it executes the following query:
CREATE DATABASE mydb TEMPLATE template1;
For template1 the owner is set to cloudsqlsuperuser a role that gets assigned to the postgres user, and other users created through the GUI.
So if you would like to create a database using sql with the appropriate privileges, just execute the statement above, and your public schema will then be owned by the cloudsqlsuperuser, and can be altered using the default postgres user, or other users created through the web GUI.
Connect to the database mydb by owner user (for exaple, it is mynewuser).
If you want to change the public schema owner, first you should make the user postgres owner of your database mydb:
mydb=> ALTER DATABASE mydb OWNER TO postgres;
After that, you can change the public schema owner:
mydb=> ALTER SCHEMA public OWNER TO postgres;
Besides, to remove your mynewuser from the cloudsqlsuperuser group (role) use:
mydb=> REVOKE cloudsqlsuperuser FROM mynewuser;
Note: The default postgres user in Google Cloud Platform's (GCP) Cloud SQL (PostgreSQL) is not a superuser of the instance. Also, all users created from the GCP web UI have cloudsqlsuperuser role by default, and the following attributes (privileges): CREATEROLE, CREATEDB and LOGIN. They don't have the SUPERUSER or REPLICATION attributes.
I'm sure I'm missing something simple, but I've created the following:
postgres=# \du
List of roles
Role name | Attributes | Member of
-----------+-----------------------------------------+-----------
admin | No inheritance, Create DB, Cannot login | {}
postgres | Superuser, Create role, Create DB | {}
wade | | {admin}
(Note that Cannot login and No inheritance don't affect what's happening to wade, here. See the PostgreSQL documentation for role membership to understand why. —bignose)
However, when I try to create a db, I get:
bin wwilliam$ createdb -U wade test
Password:
createdb: database creation failed: ERROR: permission denied to create database
What am I missing?
An excerpt from the manual:
The INHERIT attribute governs inheritance of grantable privileges (that is, access privileges for database objects and role memberships). It does not apply to the special role attributes set by CREATE ROLE and ALTER ROLE. For example, being a member of a role with CREATEDB privilege does not immediately grant the ability to create databases, even if INHERIT is set; it would be necessary to become that role via SET ROLE before creating a database.
(Emphasis mine).
In documentation:
The role attributes LOGIN, SUPERUSER, CREATEDB, and CREATEROLE can be thought of as special privileges, but they are never inherited as ordinary privileges on database objects are. You must actually SET ROLE to a specific role having one of these attributes in order to make use of the attribute
So you must activate admin role using SET ROLE admin; before creating DB.