I have a Postgres database instance in GCP running on version 9.6.
I want to upgrade the Postgres version to a newer version, and I use GCPs "Database migration" for that purpose.
I have 2 databases in the instance and they fill around 800 GB in total.
The problem is that the migration gets in stuck. There are no errors in the migration log.
Copy of monitoring
In short, my question is:
How can I check in what phase is the migration and what is the issue?
Thanks.
Related
I'm trying to connect to an RDS cluster in AWS that's an Aurora PostgreSQL database. It's a brand-new database that I created along with the instances that I have Jira deployed to. However, when I try to connect to the instance from the Jira configuration screen I get this error
You have specified a database that is not empty, please specify an empty database.
I haven't touched this database at all, why is it giving me this error? I have one read and one write database in my cluster and the "hostname" is the endpoint for my write database, which is what the docs say. Could this be an issue with the Jira version I'm using?
This is the download link I'm using in my user-data script to install Jira. I'm also using PostgreSQL version 12.11
https://www.atlassian.com/software/jira/downloads/binary/atlassian-servicedesk-4.19.1-x64.bin
I switched to a different PostgreSQL version and now it's working.
PostgreSQL version 12.11 was giving me the error and switching to version 13.7 works as expected.
Can we take base backup from postgresql old version db to new version db.
For Example I have two DB servers. One server is having old version Postgres 9.x and another one is running Postgres 13.
If I want to take base backup from postgres 9.x for Postgres 13, can we do that directly using the pg_basebackup command in Postgres 13 DB server.
Is there any other way to complete the above task in Postgres.
I have a plan below to complete the above task if the base backup does not work for lower to higher versions. But it is a slow process.
DB A has a lower version and DB B has a higher version.
So to take base backup, updating the A DB version same as B DB is the only option.
Once the DB A updated successfully, Then start the base backup from A DB with updated Postgres version to the B DB.
We can refer to the below site for the DB update process.
https://www.migops.com/blog/upgrading-postgresql-9-6-to-postgresql-13/
The DB update process will take time to complete. In that time the write process will not work on the primary DB. All the write process will be affected because of this. If any issue happens in the update process, almost everything will be a mess.
I will suggest the old version DB to the new version db base backup process to complete this activity.
Do you guys have any suggestions?
You are mixing up physical file-system level backup (pg_basebackup) and logical backup, also known as dump or export (pg_dump). You can only use the latter for upgrading. A pg_basebackup can only be restored to the same major PostgreSQL version.
I'm trying to upgrade an RDS database cluster engine from Aurora PostgreSQL 9.6.19 before its end of life, I made copy and tried to upgrade to 9.6.21 and 10.16 but everytime the same problem happens:
Database cluster is in a state that cannot be upgraded: Postgres cluster is in a state where pg_upgrade can not be completed successfully.
The status of the database is Available so maybe it refers to something else but I don't know what and how to fix it, I've tried looking for answers to no avail.
Has anyone fixed this?
The pg_upgrade_internal log file will usually contain details on any failures/errors.
You can take a look on these logs using the command line:
aws rds describe-db-log-files --db-instance-identifier my-db-instance
Or via console, or RDS API.
For more information take a look on these links: Upgrading the PostgreSQL DB engine for Amazon RDS, Viewing and listing database log files
Currently I am trying to upgrade my postgres 9.1 to 10 using logical replication. As 9.1 does not support native logical_replication, I tried slony and made a replica successfully.
P.S. The above replica I created is using a sample dump from an year ago which is only of 800mb.
Now I have few questions.
How can I validate whether the replica has all the data replicated successfully. Few suggested to put the master on maintenance mode(few downtime) and do a last N items comparison with both the database on all the tables.
I tried with 800mb. Will there be any problem when I go and try with 100+ GB?
Please share your personal experience here. I have been trying to document what are the things that could go wrong so I can always try to anticipate the next course of action.
You may use the Data Validator that is shipped with trial version of EDB Postgres Replication server for validating the data between old and new PostgreSQL databases.
You may read the details of Data Validator at Data Validator Document
To download the EDB Replication Server please follow this link: EDB Replication Server
Disclosure: I work for EnterpriseDB (EDB)
According to:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_UpgradeDBInstance.Upgrading.html
Manually Upgrading the Engine Version
To perform a major version upgrade of a DB cluster, you can restore a
snapshot of the DB cluster and specify a higher major engine version.
For information about restoring a DB cluster, see Restoring from a DB
Cluster Snapshot.
Yet I don't see anywhere the option of choosing a database engine at restore.
Is the documentation incorrect?
Am I missing something?
This is now supported by Aurora Postgres. There is no need to restore a snapshot into a new cluster anymore. The procedure is explained in the documentation.
As of 2020/05/22:
"Currently, you can do a major version upgrade from Aurora PostgreSQL version 9.6.11 or 9.6.12 database clusters to Aurora PostgreSQL version 10.11."
I also encountered the same issue when attempting to upgrade Aurora Postgres from 9.x to 10.x. The GUI and aws-cli does not allow you to specify a higher version number when restoring a snapshot.
AWS support confirms this is NOT supported at this time, this from my chat session with the tech:
I should have recollected earlier. Currently there isn't an in-place upgrade path from Aurora 9.x to version 10.
This is something that is frequently requested for and the engineering team is working on but we don't have an ETA as to when this will be fixed.
AWS did offer two options:
Use pg_dump and pg_restore to extract and load the data into an upgrade target cluster.
Use AWS Data Migration Services to migrate data. DMS has the advantage of minimal downtime but it comes with many limitations you would have to evaluate. For example some data types, like JSON and timestamps with time zones are 'partially migrated'.
Links to AWS DMS documentation using PostgreSQL as a source and/or target of DMS:
Postgres as DMS Source: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.Homogeneous.DMS
Postgres as DMS Target:
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html
The tech I chatted with favored pg_dump/pg_restore because it's more native, but it involve more downtime.