Can't access postgres extention created in docker entrypoint file, until I manually created it in psql - postgresql

I'm trying to use the timescaledb extension, and so I'm running their official docker image.
In the last line of my docker entrypoint file, I run:
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
I verify that it's available for use with \dx in psql. As soon as I try to make use of the extension, I get:
No function matches the given name and argument types. You might need to add explicit type casts.
I find I have to add it manually by execing into psql and running CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
What's wrong with my entrypoint? Why do I have to manually create the extension after the container is built and running?
Edit: here's the full entrypoint script:
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER test_user PASSWORD 'password123';
ALTER USER test_user WITH SUPERUSER; --needed to create timescaledb extension
CREATE DATABASE testdb OWNER test_user;
GRANT ALL PRIVILEGES ON DATABASE testdb TO test_user;
CREATE DATABASE tsdb OWNER test_user;
GRANT ALL PRIVILEGES ON DATABASE tsdb TO test_user;
ALTER USER test_user CREATEDB;
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
EOSQL

Disclaimer: I don't know docker, but this seems only related to Postgres, not Docker itself
create extension will create the extension in database psql is currently connected to. Seeing the script, this is most probably the default database postgres that you connect to.
So the extension will be created in the postgres database, not in the testdb database.
You have two options on how to change that:
1. Use the template1 database
Anything created in the template1 database will automatically be created in every database created afterwards. So if you connect to the template database and run the create extension before creating the test database, the extension will automatically be available:
psql -v ON_ERROR_STOP=1 --dbname=template1 --username "$POSTGRES_USER" <<-EOSQL
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
CREATE USER test_user PASSWORD 'password123';
ALTER USER test_user WITH SUPERUSER; --needed to create timescaledb extension
CREATE DATABASE testdb OWNER test_user;
GRANT ALL PRIVILEGES ON DATABASE testdb TO test_user;
CREATE DATABASE tsdb OWNER test_user;
GRANT ALL PRIVILEGES ON DATABASE tsdb TO test_user;
ALTER USER test_user CREATEDB;
EOSQL
Note that the extension is created before anything else. The actual order isn't that important, the only thing that is important is that it's done before creating a new database.
2. Connect to the newly created database
Switch to the newly created database from within psql before you create the extension using the \connect command in psql
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER test_user PASSWORD 'password123';
ALTER USER test_user WITH SUPERUSER; --needed to create timescaledb extension
CREATE DATABASE testdb OWNER test_user;
GRANT ALL PRIVILEGES ON DATABASE testdb TO test_user;
CREATE DATABASE tsdb OWNER test_user;
GRANT ALL PRIVILEGES ON DATABASE tsdb TO test_user;
ALTER USER test_user CREATEDB;
\connect testdb
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
EOSQL
The main difference between the two methods is that with the first, the extension will be available automatically in all databases that are created in the future. Whereas with the second method it is only available in the testdb
Unrelated, but: the newly created user does not need the superuser privilege as the extension is created using the postgres user, not the newly created one.

To chime in with the prior answer, the TimescaleDB extension is applied per database, so if you run CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE; without first connecting to your desired database with \c yourdatabase it will apply the extension to the default database. See http://docs.timescale.com/v0.9/getting-started/setup for the step-by-step instructions to apply after install.

Related

Revoke CONNECT permission to new DB by default

this is happening on newly created Postgres 13 instance on GCP's Cloud SQL.
I would like to disallow users to CONNECT to newly created databases. So I am modifying template database:
REVOKE ALL ON DATABASE template1 FROM public;
Then, I create new Database:
CREATE DATABASE mydb TEMPLATE template1;
Then I am checking CONNECT permissions for existing user:
SELECT datname
FROM pg_database
WHERE has_database_privilege('someuser', datname, 'CONNECT')
datname
---------------
cloudsqladmin
postgres
template0
mydb
So as you can see someuser has CONNECT permission to mydb.
Why is this happening? How can I prevent users to connect to newly created databases?
The permissions on a database are not copied from the template database during CREATE DATABASE. Rather, all new databases have the same default permissions: CONNECT and TEMP for PUBLIC, and all permissions for the owner.
There is no way to change that default, short of modifying the PostgreSQL code.

postgres main role is not postgres, how can I set it up to postgress

In terminal, when I write psql it login to "Coyr". I want "postgres" to be the main user and have all the attributes. How can I accomplish that?
I guess you ran initdb as user coyr without the -U option, and now you want to rename the bootstrap superuser.
That is easy:
drop the user postgres you created
create a new superuser maxi
connect as maxi and run
ALTER ROLE coyr RENAME TO postgres;
connect as postgres and
DROP ROLE maxi;
By renaming coyr will have lost its password, so if you need one, you have to set it again.

Postgres restore missing privileges

I am running through some very basic tests to confirm a backup/restore process is working on a local environment.
the issue I am running into is that it doesn't look as though pg_dump/pg_restore/psql is restoring a database to the same state.
A sample of what I am doing below from start to finish.
CREATE DATABASE testdb WITH ENCODING='UTF8' CONNECTION LIMIT=-1;
CREATE TABLE a
(
a INT
);
INSERT INTO a(a)
SELECT 1 UNION ALL
SELECT 2;
SELECT * FROM a;
GRANT ALL PRIVILEGES ON DATABASE testdb TO testuser;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO testuser;
Then running the pg_dump
pg_dump -Fc -v --host=localhost --username=postgres --dbname=testdb -f C:\test\testdb.dump
creating a side by side restore for this example
CREATE DATABASE testdb_restore WITH ENCODING='UTF8' CONNECTION LIMIT=-1;
pg_restore -v --host=localhost --username=postgres --dbname=testdb_restore C:\test\testdb.dump
Now when I right click on testdb in pgadmin and click "Create Script" I get the following
-- Database: testdb
-- DROP DATABASE testdb;
CREATE DATABASE testdb
WITH
OWNER = postgres
ENCODING = 'UTF8'
LC_COLLATE = 'English_Australia.1252'
LC_CTYPE = 'English_Australia.1252'
TABLESPACE = pg_default
CONNECTION LIMIT = -1;
GRANT ALL ON DATABASE testdb TO postgres;
GRANT TEMPORARY, CONNECT ON DATABASE testdb TO PUBLIC;
GRANT ALL ON DATABASE testdb TO testuser;
When I click on the perform the same on testdb_restore, I get the following
-- Database: testdb_restore
-- DROP DATABASE testdb_restore;
CREATE DATABASE testdb_restore
WITH
OWNER = postgres
ENCODING = 'UTF8'
LC_COLLATE = 'English_Australia.1252'
LC_CTYPE = 'English_Australia.1252'
TABLESPACE = pg_default
CONNECTION LIMIT = -1;
As you can see I am missing the extra privileges from the original database.
I'm sure this is a very simple thing but I am currently lost on this one. I have also tried using methods and also pg_dump create database option added in and no difference.
Please note: I am extremely new to postgres and coming from a SQL Server background.
Roles and permissions are stored and managed per cluster, not per database, which is why pg_dump is not dumping them. You should use pg_dumpall if you are happy to have a dump of the whole cluster.
Alternatively, you can use pg_dumpall -r to dump roles only, and then pg_dump your database, and apply both scripts.
Unfortunately the privileges for a database are not included in a pg_dump. You'd have to use pg_dumpall for that, but that dumps all databases.
I know this is annoying. It is a bug of long standing that nobody has fixed yet.

how to get pg_dump to include create user command

I need to create a dump of all the commands below.
test=> create user rdstest login password 'rdstest';
CREATE ROLE
test=> grant connect on database test to rdstest;
GRANT
test=> create user devadmin login password 'devtest';
CREATE ROLE
test=> grant connect on database test to devadmin;
GRANT
test=> grant rdstest to devadmin with admin option;
GRANT ROLE
test=> grant rdstest to devadmin;
GRANT ROLE
test=> create schema authorization rdstest;
CREATE SCHEMA
when i tried to create it using pg_dump
as pg_dump -U devadmin -h ****xxx.rds.amazonaws.com test > Outfile.sql
I can only see the schema related commands
CREATE SCHEMA rdstest;
ALTER SCHEMA rdstest OWNER TO rdstest;
How to get pg_dump to include all the commands:create user command,grant connect on database test to rdstest etc.
pg_dump can not do that, because pg_dump only dumps a single database and that information is not part of one database, but stored globally in the Postgres "cluster".
You need to use pg_dumpall for that, using the --globals-only option:
pg_dumpall --globals-only --file=globals.sql

Schema created in Docker by postgres user does not exist

I am building a Docker container for my PostgresSQL 9.4.5 database and I'd like to create the table structures when the container starts. In the script included in my /docker-entrypoint-initdb.d/ directory is this series of commands:
#!/usr/bin/env bash
export PGUSER=postgres
echo "***CREATING DATABASE, USERS, AND ROLES***"
psql <<-EOSQL
CREATE USER vpager WITH PASSWORD '<pass>';
CREATE DATABASE vpager;
GRANT ALL PRIVILEGES ON DATABASE vpager TO vpager;
REVOKE ALL ON DATABASE vpager FROM PUBLIC;
CREATE ROLE standarduser;
CREATE USER vpageruser WITH PASSWORD '<pass>';
\connect vpager
CREATE SCHEMA ticketing;
GRANT ALL PRIVILEGES ON SCHEMA ticketing TO vpager;
GRANT CONNECT ON DATABASE vpager TO standarduser;
GRANT SELECT, UPDATE, INSERT, DELETE ON ALL TABLES IN SCHEMA ticketing TO standarduser;
GRANT standarduser TO vpageruser;
EOSQL
echo "***CREATING TABLE STRUCTURES***"
psql -U vpager vpager < /docker-entrypoint-initdb.d/create-vpager-tables.sql
The SQL script:
CREATE TABLE ticketing.merchant
(
merchant_id SERIAL PRIMARY KEY,
now_serving INT
);
CREATE TABLE ticketing.ticket
(
ticket_id SERIAL,
merchant_id INT REFERENCES ticketing.merchant(merchant_id),
create_ts TIMESTAMP,
PRIMARY KEY (ticket_id, merchant_id)
);
As you can see, the idea is to create a non-privileged user for doing CRUD operations. The problem is that when the create-vpager-tables.sql command runs, it can't see the schema that postgres created:
ERROR: schema "ticketing" does not exist
STATEMENT: CREATE TABLE ticketing.merchant
(
merchant_id SERIAL PRIMARY KEY,
now_serving INT
);
ERROR: schema "ticketing" does not exist
ERROR: schema "ticketing" does not exist
Yet, if I add a CREATE SCHEMA statement to the top of create-vpager-tables.sql, it says it already exists:
ERROR: schema "ticketing" already exists
How do I resolve these errors? I tried setting my search path at the top of the SQL file, but the same thing happened; it's like a "Schrodinger" schema.
I figured it out. I put the create-vpager-tables.sql in the /docker-entrypoint-initdb.d directory, and apparently the image also tries to source SQL files there. Moving it to /tmp (and granting USAGE on the schema to standarduser) fixed the problem.