Install extension in PostgreSQL database via migration without superuser role? - postgresql

We are using migrations (via Sequelize, in JavaScript) to maintain changes to our database. I have a need to add a CREATE EXTENSION call but since I am running as the database creator, and not superuser, I get a permission denied to create extension.
Is there a way to modify security on a single database to allow a user to install an extension via a migration file? IOW, when I create the "naked" database and apply my permissions, can I set security up to allow CREATE EXTENSION and DROP EXTENSION for a specific user?

No, there is no setting for CREATE ROLE for granular CREATE/DROP EXTENSION control. I assume, it depends on the extension you use, but you based on the documentation:
Loading an extension requires the same privileges that would be
required to create its component objects. For most extensions this
means superuser or database owner privileges are needed. The user who
runs CREATE EXTENSION becomes the owner of the extension for purposes
of later privilege checks, as well as the owner of any objects created
by the extension's script.
I was running into the same issue with pg_trgm extension, but even setting role to database owner did not resolve the issue.

Could not find a way to do this inside a migration file.
The technique we ended up using was to have a subfolder in the migration folder, called postgres to hold anything that must be run as superuser. The subfolder contained the migrations and a config file for running things as a superuser. So, the command-line becomes:
$ sequelize db:migrate --migrations-path "migrations/postgres" --config "migrations/postgres"
$ sequelize db:migrate
The inconvenience of a more complex command-line was mitigated by the fact that the "special" migrations that required this isolation also made them more evident.

For anyone still looking, here is how to create an extension via a sequelize migration. It doesn't address the secondary question of user roles however. But this works for me.
'use strict';
module.exports = {
up: (queryInterface, Sequelize) => {
return queryInterface.sequelize.query('CREATE EXTENSION extensionName;');
},
down: (queryInterface, Sequelize) => {
return queryInterface.sequelize.query('DROP EXTENSION extensionName;');
}
};

Related

prisma db pull doesn't see a new table

I have existing schema for prisma.
I copied the table from another schema to be included in the Prisma schema.
But when I run prisma db pull new table doesn't appear in Prisma schema.
Why?
If you use supabase and running this command and it returns something like this 'The following models were commented out because we couldn't retrieve columns for them. Please check your privileges.' or something similar regarding privileges, the solution is to go to your SQL editor from supabase and put this command and execute it
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO postgres;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO postgres;
You're misinterpreting the function of the prisma db pull command.
From the docs
The db pull command connects to your database and adds Prisma models to your Prisma schema that reflect the current database schema.
Basically, it will make your Prisma schema match the existing database schema. What you want is the opposite here: Update the database schema to match the current Prisma schema.
You can do this in two ways:
If you want to update the database and keep the changes in your migration history, use the prisma migrate dev command. More Info
If you just want to update the database without creating a migration, use the prisma db push command. More info
More information is available in the docs explaining how you should choose between 1 and 2.
I had a similar issue once and a quick check confirmed for me that it was the lack of security permissions granted for prisma on new table in the database itself.
Try this:
Note the name of the database user that Prisma connects to the database with. You'll likely find this via your schema.prisma file, or perhaps via a DATABASE_URL config setting in the related .env file if you're using that with prisma.
Go into the database itself and ensure that database user which Prisma connects with has been granted sufficient security privileges to that new table. (note: what 'sufficient' is I cannot say since it depends on your own needs. At a guess, I'd say at least 'select' permission would be needed.)
Once you've ensure that user has sufficient privileges, try running a prisma db pull command once again.
For reference, another thing you could do is:
cross-check against one of the other tables that is already in your database that works correctly with prisma.
compare the security privileges of that old table with the security privileges of the new table and see if there are any differences.

How to allow create extension without superuser privilege?

I went through the question/answers cannot create extension without superuser role and other related, which tells that I cannot avoid that issue without being a superuser.
But I am working with an AWS server now, in which I am able to install extension tablefunc without being a superuser, but I cannot do the same on a new postgres server. I am attaching a screenshot here of the roles present in the AWS database, here you will see, role pgadmin is not a superuser (and also none of the role group it's attached to are superuser, though it doesn't matter anyhow), but I can create the extension with it.
I understand the other workarounds, but just need a clarification for this. How is it working in AWS and why can't I replicate the same on a new Postgres server.
If you have access to and can update the extension control file, do this:
echo 'superuser = false' >> /usr/share/postgresql/14/extension/pg_cron.control
That example is for pg_cron but the same approach should work for any extension.

Using Postgres PGCrypto encryption requires superuser to run view queries

Using: Postgres 9, CentOS 7,
Postgres Data directory not in default location but used RSync to make sure permissions were correct. And yes appropriate .config files were changed.
When I try to query a view containing an encrypted item as a NON superuser (Testuser), I get this error:
ERROR: must be superuser to read files CONTEXT: PL/pgSQL function
decrypt_data(bytea) line 13 at assignment
If I run that same query using POSTGRES superuser, the query completes fine.
This seems to be a file system read permission error when trying to read the Key files. Everything I see using encryption seem to not mention how to run without being superuser.
I have already run the following grants for Testuser:
GRANT ALL PRIVILEGES ON DATABASE xxx_db to Testuser;
GRANT SELECT ON ALL TABLES IN SCHEMA xxxxx TO Testuser;
GRANT ALL ON ALL TABLES IN SCHEMA xxxxx TO Testuser;
The test user can create tables, views, basically anything within that db.. just not read encryption keys.
The permissions on the keys are 775 right now, I even tried 777 without luck.
Any Ideas?
pgcrypto is a PostgreSQL extension described here:
https://www.postgresql.org/docs/current/static/pgcrypto.html
but it doesn't provide a decrypt_data(bytea) function.
This function seems to be custom code that happens to open a server-side file, with pg_read_file() or a similar method.
These methods are restricted to superusers to avoid normal users to read on the server's filesystem, no matter what are the Unix rights of the particular file they want to read.
You can verify this in the source of decrypt_data(bytea), which can be obtained with:
select pg_get_functiondef('decrypt_data(bytea)'::regprocedure);
or \df+ decrypt_data(bytea) from within psql.
I found the issue. I need to grant the user with function permissions.
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA xxxxx TO yyyyyyyyy;

Why can only a superuser CREATE EXTENSION hstore, but not on Heroku?

When I attempt to enable hstore on my database:
=> CREATE EXTENSION IF NOT EXISTS hstore;
ERROR: permission denied to create extension "hstore"
HINT: Must be superuser to create this extension.
My user is not a superuser, but is the owner of the database.
According to the CREATE EXTENSION docs:
Loading an extension requires the same privileges that would be required to create its component objects. For most extensions this means superuser or database owner privileges are needed. The user who runs CREATE EXTENSION becomes the owner of the extension for purposes of later privilege checks, as well as the owner of any objects created by the extension's script.
What is hstore doing that requires superuser privileges? Is it affecting parts of the cluster outside the database I'm adding it to?
Further confundity:
The DB user Heroku Postgres provides is not a superuser:
Heroku Postgres users are granted all non-superuser permissions on their database. These include SELECT, INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER, CREATE, CONNECT, TEMPORARY, EXECUTE, and USAGE.
However, that user is able to CREATE EXTENSION hstore:
To create any supported extension, open a session with heroku pg:psql and run the appropriate command:
$ heroku pg:psql
Pager usage is off.
psql (9.2.4)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.
ad27m1eao6kqb1=> CREATE EXTENSION hstore;
CREATE EXTENSION
ad27m1eao6kqb1=>
(For context, I'm attempting to set up a Dokku deployment, so the comparison to Heroku is especially important.)
The hstore extension creates functions that call code from an external dynamic object, which requires superuser privilege. That's why creating the hstore extension requires superuser privilege.
As for Heroku, it is my understanding that they are running with a special extension whitelisting module, which allows users to create certain extensions even though they are not superusers. I believe it is based on this code: https://github.com/dimitri/pgextwlist. You can try to install that code yourself if you want the same functionality in your databases.
ALTER USER myuser WITH SUPERUSER;
If you run this command from a superuser, this solves your CREATE EXTENSION issue. You may check your available users with \du to find a superuser.
This is not related to heroku.
This is how I solved this issue in ubuntu 18.04.
Provide postgres super user access.
sudo su postgres
Then I run:
psql -U postgres your_database_name -c 'create extension hstore;'
Now I can alter table your_database_name and add hstore type columns in it.
Connect to your database
psql -d your_database_name -U your_user_role
And
alter table your_table_name add your_column_name HSTORE;
Though there might be saveral different ways to do it, but I solve it in this way.
Hope this will help novice users like me.

Postgis installation: type "geometry" does not exist

I am trying to create table with Postgis. I do it by this page. But when I import postgis.sql file, I get a lot of errors:
ERROR: type "geometry" does not exist
Does anybody know how can I fix it?
I had the same problem, but it was fixed by running following code
CREATE EXTENSION postgis;
In detail,
open pgAdmin
select (click) your database
click "SQL" icon on the bar
run "CREATE EXTENSION postgis;" code
If the Postgis-Extension is loaded, then your SQL perhaps does not find the geometry-type because of missing search-path to the public schema.
Try
SET search_path = ..., public;
in the first line of your scsript. (replace ... with the other required search-paths)
You can do it from terminal:
psql mydatabasename -c "CREATE EXTENSION postgis";
To get psql to stop on the first error, use -v ON_ERROR_STOP=1 (which is off by default, which is why you see many errors). For example:
psql -U postgres -d postgis -v ON_ERROR_STOP=1 -f postgis.sql
The actual error is something like "could not load library X", which can vary on your situation. As a guess, try this command before installing the sql script:
ldconfig
(you might need to prefix with sudo depending on your system). This command updates the paths to all system libraries, such as GEOS.
This error may also occur if you try to use postgis types on another schema rather than public.
If you are creating you own schema, using postgis 2.3 or higher and encounter this error, do the following as stated here:
CREATE SCHEMA IF NOT EXISTS my_schema;
CREATE extension postgis;
UPDATE pg_extension
SET extrelocatable = TRUE
WHERE extname = 'postgis';
ALTER EXTENSION postgis
SET SCHEMA my_schema;
ALTER EXTENSION postgis
UPDATE TO "2.5.2next";
ALTER EXTENSION postgis
UPDATE TO "2.5.2";
SET search_path TO my_schema;
Then you can proceed to use postgis functinalities.
You must enable the extension on your database.
psql my_database -c "CREATE EXTENSION postgis;"
You also need to ensure that the user you are trying to use the postgis extension as, has access to the schema where postgis is setup (which in the tutorials I read is called 'postgis').
I just had this error, and it was solved because I had only given a new user access to the database. In the database I'd created, I ran:
grant all on schema postgis to USERNAME;
And this solved this error
run this query first:
"CREATE EXTENSION postgis"
The answers here may solve your problem, however if you already have postgis enabled on your DB, the issue may be that you are trying to restore a postgis table (with a geometry column) into a schema other than where your postgis extension is enabled. In pgAdmin you can click on the postgis extension and see which schema is specified. If you are trying to restore a table with geometry column into a different schema, you might get this error.
I resolved this by altering my postgis extension - however I'm not sure if that was necessarily the best way to do it. All I know is that it allowed me to restore the table.
First make sure you have (matching to pg version: psql -V) postgis installed:
sudo apt install postgis postgresql-9.6-postgis-2.3
Just before tables creation add:
db.engine.execute('create extension postgis')
db.create_all()
This has already been answered but I wanted to add a more thorough answer that explains why certain commands work, and in what circumstances to use them, and of course, how to figure out which circumstances you are in.
First, you need to check that PostGIS is actually installed on your box. When connected to postgres, such as via psql, run:
SELECT PostGIS_Full_Version();
If it's not installed, look up distro- and version-specific instructions for installing PostGIS and install it.
Assuming PostGIS is installed, the error is usually the result of not having "created" (this is an unfortunately misleading use of language, the effect is more like "enabling" the extension) the extension for the particular database. The way PostgreSQL is set up, by default new databases do not come with any extensions enabled, and you need to enable ("create") them per-database. In order to do this you need to run the following command.
It only needs to be run once:
CREATE EXTENSION postgis;
I think you need superuser privileges for the particular database in question, in order to run this command.
Assuming postgres is configured so that the permissions allow, you can execute this command from the command line by running the following command:
psql my_database -c "CREATE EXTENSION postgis;"
You may need to use the -U flag and specify a user.
In some cases, however, the extension may have already been created, and installed under a different schema than public, and the problem may be one of permissions. This can arise like in the situation #mozboz describes, if you create a new user but don't give it access to the schema. To detect this case, look for a separate schema in the database, with a table called spatial_ref_sys, as this is created when the extension is created.
In this case you may need to run, when connected to the database:
GRANT USAGE ON SCHEMA schema_name TO username;
In my experience, this situation is rare, and I have never found any reason to set things up this way. The schema_name is often, but not always postgis. By default if you run the first command here, it will create the extension under the public schema.
I think USAGE is usually sufficient for most cases, but you might want to grant more privileges if you want the user to be able to actually edit data; the only time this has ever come up for me was adding new projections to spatial_ref_system, but even this is rare as by default that table includes most commonly used projections.
Or...
cursor.execute('create extension postgis')
in your python program, using a current cursor from psycopg2.
My experience was I was trying to load the ism file to a different file than "public". I initialised the postgis extension on another schema other than public. So that didn't work. So finally I had to remove the extension, and than created the postgis extension in public. After that I could load the osm into my new schema
Osm always looks for the extension file in public, irregardless of where u intend to install the osm files in another schema
Verify the public search_path is not included for the user:
SELECT usename, useconfig from pg_user;
-- { search_path=pg_catalog, public }
SHOW SEARCH_PATH;
-- public, topology
Method #1: SET the search_path to public
SET search_path = "public";
CREATE TABLE IF NOT EXISTS sample_geom
(
geom_1 GEOMETRY,
geom_2 GEOMETRY(Polygon, 4326) NOT NULL
);
ALTER TABLE sample_geom
OWNER TO root;
Method #2: Use the qualified object name for the GEOMETRY object type (public.GEOMETRY)
CREATE TABLE IF NOT EXISTS sample_geom
(
geom_1 public.GEOMETRY,
geom_2 public.GEOMETRY(Polygon, 4326) NOT NULL
);
ALTER TABLE sample_geom
OWNER TO root;
Source: Demystifying Schemas & search_path through Examples
Using pgAdmin 4,you can sort this:
Click on the SQL query button (or go to Tools >
Query Tool).
Enter the following query into the query text field to load the PostGIS spatial extension and Click the Play button in the toolbar (or press F5) to “Execute the query.”
CREATE EXTENSION postgis;
Succesful feedback
Now confirm that PostGIS is installed by running a PostGIS function:
SELECT postgis_full_version();
Code
Feedback