How to create database template on azure postgresql service - postgresql

While using the database service for postgresql om azure, it looks like it is not possible to create a custom template database.
What I want to achieve, is that a regular account, csn create new databases with a specific extension enabled.
The creation can be delegated, but the enabling of the extension fails in my test for all but the initial database admin account.

I've just tried the following in Azure Database for PostgreSQL and it worked (please bear with me as I walk you through my steps in Azure).
In Azure CLI (bash) followed steps in Quick Start for Azure PostgreSQL to create a new resource group > create a PG server in it > create a new DB (originaldb) on that PG server. All worked just fine.
Then I enabled a earthdistance and cube (pre-req for earthdistance) extensions for orginaldb (the DB I created in step 1).
Then I used CLI to create another DB (dbclone) using orginaldb as a template:
CREATE DATABASE dbclone TEMPLATE originaldb;
It worked just fine and cube and earthdistance extensions are enabled in dbclone DB.
Now on to trying it with another PG user: I created a PG user (user1) and granted this user DB creation privilege.
Then I logged on to my server as user1 and created another DB from CLI using the same command:
CREATE DATABASE dbclone2 TEMPLATE originaldb;
It worked too and I see that cube and earthdistance extensions are enabled in dbclone2 database.
Is that what you're trying to do? Are you hitting errors following the same steps or you're trying to do something different?

Related

Error restoring db in Azure PostgreSQL Flexible server: 'extension "azure" is not allow-listed'

I'm using pgAdmin to backup/restore a database from one Azure PostgreSQL Flexible server to another. The source server was created last year, the destination server is new.
The restore process fails early with the error:
ERROR: extension "azure" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL
I came across this post https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/introducing-ability-to-allow-list-extensions-in-postgresql/ba-p/3219124 announcing recent changes to PostgreSQL Flexible Server. If I'm reading this correctly, my new database server is affected by this change and I need to allow specific extensions under the "azure.extensions" server parameter.
In the backup file I can see:
CREATE EXTENSION IF NOT EXISTS azure WITH SCHEMA public;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp" WITH SCHEMA public;
And in Azure Portal I can see "UUID-OSSP" under the new "azure.extensions" server parameter, though there's nothing called just "azure". I enabled UUID-OSSP but the restore process still fails with the same error.
What am I missing here?
It's suggestable to install TimescaleDB that is having the package of supportive extension.
To learn abot TimescaleDB. (https://docs.timescale.com/timescaledb/latest/)
Change the Postgres's parameters for "shared_preload_libraries"
After creating extension
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
Follow the entire procedure from the below link.
https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions

Copy data from Postgres DB (GCP Project A) to another Postgres DB (GCP Project B)

I would be happy to get your help / feedback re data load.
Goal:
Load source data from a Postgres database, which is located in GCP project A to another Postgres database, which is located in GCP project B.
Challenge:
Get a connection (I have an IAM account with sufficient rights to run a COPY TO / COPY FROM command) to the Postgres DB in GCP Project A and copy the table either to a CSV or create a dump that can be used in order to be inserted to another Postgres DB in GCP Project B.
How do I connect to the database (e.g. if I create a key, where shall I store the json keyfile and would that approach even be feasible?) with this IAM email account?
Other ways I've researched were to use psycopg2 (thus I could use the function cursor.copy_expert (which doesn’t need any superuser right or Postgres user credentials and copy the data), but I didn’t succeed in connecting to the database with psycopg2 due to challenges with cloud proxy.
Another idea was to use pg_dump or gcloud sql export csv.
I would be curious if some of you were facing a similar challenge and how did you solve it and what might be the best way/practice
You can have a try out database migration service. You can set up a continuous migration configuration and use Cloud SQL for PostgreSQL.
Hello after a lot of searching I've come to these solutions:
If you have continuous copy, you need to use the database migration service, check this documentation.
If you have one shot copy:
you can restore your instance, see the bottom page of this documentation
you can create a bucket and backup your instance on it, then import it from the other project

Installation of pg_cron on Azure Flexible PostgeSQL

I am trying to install pg-cron extension for Azure PostgreSQL Flexible server.
According to documentation found here:
https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#postgres-13-extensions
pg_cron is available extension, but when I am trying to install it:
create schema cron_pg;
CREATE EXTENSION pg_cron SCHEMA cron_pg;
What I get is:
SQL Error [0A000]: ERROR: extension "pg_cron" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL
Hint: to see the full allow list of extensions, please run: "show azure.extensions;"
When executing:
show azure.extensions;
pg_cron is missing:
address_standardizer,address_standardizer_data_us,amcheck,bloom,btree_gin,btree_gist,citext,cube,dblink,dict_int,dict_xsyn,earthdistance,fuzzystrmatch,hstore,intagg,intarray,isn,lo,ltree,pageinspect,pg_buffercache,pg_freespacemap,pg_partman,pg_prewarm,pg_stat_statements,pg_trgm,pg_visibility,pgaudit,pgcrypto,pgrowlocks,pglogical,pgstattuple,plpgsql,postgis,postgis_sfcgal,postgis_tiger_geocoder,postgis_topology,postgres_fdw,sslinfo,tablefunc,tsm_system_rows,tsm_system_time,unaccent,uuid-ossp,lo,postgis_raster
What am I doing wrong?
You can tell pg_cron to run jobs in another database by updating the database column job in the jobs table.
For example:
UPDATE cron.job SET database = 'wordpress' WHERE jobname = 'wordpress-job';
Pretty late but this issue showed up when I was searching for same problem but with pg_trgm extension. After some looking around eventually realised you just need to update the database settings.
Go to Database in Azure Portal, then to Server parameters and search for azure.extensions parameter. You can then click on the list and enable/disable desired extensions (PG_CRON is available), the server will restart on save and then you will be able to enable the extensions in database.
Seems that the pg_cron extension is already enabled, by default, in the default 'postgres' database.
The reason why I was not seeing this is because I am not using the default 'postgres' database. I have created my own DB which I was connected to.
This actually does not resolve my problem, because I can't execute jobs from pg_cron across databases...

GCP Cloud SQL Terraform Postgres extension

Is there an official way of installing an extension on a GCP Postgress Cloud SQL instance via Terraform?
Closest I've found is this unofficial Postgres resource but it's not immediately clear how to link the two. This issue on their tracker sort of shows how, but far from a step by step guide.
if it makes any difference, I'm trying to provision a Postgres Cloud SQL instance with PostGIS.
Thanks.
Terraform is a deployment tool, to create all your infrastructure. To install an extension on Postgres, you need a installation tool, because you need to connect to the database and to run a command.
It's the same thing if you want to create a user in the database and you want to grant some privileges on it.
In summary, you can't achieve that with Terraform. I recommend you to have an installation tool, such as Ansible to perform this action.
An alternative is to create, with Terraform, a micro VM with a startup script that connect the database, run the command and destroy itself at the end.

How to grants pg_read_server_files OR alternative on AWS RDS PostgreSQL

I have a python script to modify a PostgreSQL database with a csv file. The script, the csv and the database used to be on the same private server. I'd like to migrate on AWS so I do some experiments. My first attempt is to run the script on my server and modify the new PostgreSQL database created with RDS. So everything seems to work except that I don't have the pg_read_server_files role on my "main" database user. As far as I understand this is normal, but I don't know what I should do to achieve my mission.
Any ideas ?
Thanks for reading