I'm using pgAdmin to backup/restore a database from one Azure PostgreSQL Flexible server to another. The source server was created last year, the destination server is new.
The restore process fails early with the error:
ERROR: extension "azure" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL
I came across this post https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/introducing-ability-to-allow-list-extensions-in-postgresql/ba-p/3219124 announcing recent changes to PostgreSQL Flexible Server. If I'm reading this correctly, my new database server is affected by this change and I need to allow specific extensions under the "azure.extensions" server parameter.
In the backup file I can see:
CREATE EXTENSION IF NOT EXISTS azure WITH SCHEMA public;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp" WITH SCHEMA public;
And in Azure Portal I can see "UUID-OSSP" under the new "azure.extensions" server parameter, though there's nothing called just "azure". I enabled UUID-OSSP but the restore process still fails with the same error.
What am I missing here?
It's suggestable to install TimescaleDB that is having the package of supportive extension.
To learn abot TimescaleDB. (https://docs.timescale.com/timescaledb/latest/)
Change the Postgres's parameters for "shared_preload_libraries"
After creating extension
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
Follow the entire procedure from the below link.
https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions
I would be happy to get your help / feedback re data load.
Goal:
Load source data from a Postgres database, which is located in GCP project A to another Postgres database, which is located in GCP project B.
Challenge:
Get a connection (I have an IAM account with sufficient rights to run a COPY TO / COPY FROM command) to the Postgres DB in GCP Project A and copy the table either to a CSV or create a dump that can be used in order to be inserted to another Postgres DB in GCP Project B.
How do I connect to the database (e.g. if I create a key, where shall I store the json keyfile and would that approach even be feasible?) with this IAM email account?
Other ways I've researched were to use psycopg2 (thus I could use the function cursor.copy_expert (which doesn’t need any superuser right or Postgres user credentials and copy the data), but I didn’t succeed in connecting to the database with psycopg2 due to challenges with cloud proxy.
Another idea was to use pg_dump or gcloud sql export csv.
I would be curious if some of you were facing a similar challenge and how did you solve it and what might be the best way/practice
You can have a try out database migration service. You can set up a continuous migration configuration and use Cloud SQL for PostgreSQL.
Hello after a lot of searching I've come to these solutions:
If you have continuous copy, you need to use the database migration service, check this documentation.
If you have one shot copy:
you can restore your instance, see the bottom page of this documentation
you can create a bucket and backup your instance on it, then import it from the other project
I am trying to install pg-cron extension for Azure PostgreSQL Flexible server.
According to documentation found here:
https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#postgres-13-extensions
pg_cron is available extension, but when I am trying to install it:
create schema cron_pg;
CREATE EXTENSION pg_cron SCHEMA cron_pg;
What I get is:
SQL Error [0A000]: ERROR: extension "pg_cron" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL
Hint: to see the full allow list of extensions, please run: "show azure.extensions;"
When executing:
show azure.extensions;
pg_cron is missing:
address_standardizer,address_standardizer_data_us,amcheck,bloom,btree_gin,btree_gist,citext,cube,dblink,dict_int,dict_xsyn,earthdistance,fuzzystrmatch,hstore,intagg,intarray,isn,lo,ltree,pageinspect,pg_buffercache,pg_freespacemap,pg_partman,pg_prewarm,pg_stat_statements,pg_trgm,pg_visibility,pgaudit,pgcrypto,pgrowlocks,pglogical,pgstattuple,plpgsql,postgis,postgis_sfcgal,postgis_tiger_geocoder,postgis_topology,postgres_fdw,sslinfo,tablefunc,tsm_system_rows,tsm_system_time,unaccent,uuid-ossp,lo,postgis_raster
What am I doing wrong?
You can tell pg_cron to run jobs in another database by updating the database column job in the jobs table.
For example:
UPDATE cron.job SET database = 'wordpress' WHERE jobname = 'wordpress-job';
Pretty late but this issue showed up when I was searching for same problem but with pg_trgm extension. After some looking around eventually realised you just need to update the database settings.
Go to Database in Azure Portal, then to Server parameters and search for azure.extensions parameter. You can then click on the list and enable/disable desired extensions (PG_CRON is available), the server will restart on save and then you will be able to enable the extensions in database.
Seems that the pg_cron extension is already enabled, by default, in the default 'postgres' database.
The reason why I was not seeing this is because I am not using the default 'postgres' database. I have created my own DB which I was connected to.
This actually does not resolve my problem, because I can't execute jobs from pg_cron across databases...
Is there an official way of installing an extension on a GCP Postgress Cloud SQL instance via Terraform?
Closest I've found is this unofficial Postgres resource but it's not immediately clear how to link the two. This issue on their tracker sort of shows how, but far from a step by step guide.
if it makes any difference, I'm trying to provision a Postgres Cloud SQL instance with PostGIS.
Thanks.
Terraform is a deployment tool, to create all your infrastructure. To install an extension on Postgres, you need a installation tool, because you need to connect to the database and to run a command.
It's the same thing if you want to create a user in the database and you want to grant some privileges on it.
In summary, you can't achieve that with Terraform. I recommend you to have an installation tool, such as Ansible to perform this action.
An alternative is to create, with Terraform, a micro VM with a startup script that connect the database, run the command and destroy itself at the end.
I have a python script to modify a PostgreSQL database with a csv file. The script, the csv and the database used to be on the same private server. I'd like to migrate on AWS so I do some experiments. My first attempt is to run the script on my server and modify the new PostgreSQL database created with RDS. So everything seems to work except that I don't have the pg_read_server_files role on my "main" database user. As far as I understand this is normal, but I don't know what I should do to achieve my mission.
Any ideas ?
Thanks for reading