I'm trying to upgrade an RDS database cluster engine from Aurora PostgreSQL 9.6.19 before its end of life, I made copy and tried to upgrade to 9.6.21 and 10.16 but everytime the same problem happens:
Database cluster is in a state that cannot be upgraded: Postgres cluster is in a state where pg_upgrade can not be completed successfully.
The status of the database is Available so maybe it refers to something else but I don't know what and how to fix it, I've tried looking for answers to no avail.
Has anyone fixed this?
The pg_upgrade_internal log file will usually contain details on any failures/errors.
You can take a look on these logs using the command line:
aws rds describe-db-log-files --db-instance-identifier my-db-instance
Or via console, or RDS API.
For more information take a look on these links: Upgrading the PostgreSQL DB engine for Amazon RDS, Viewing and listing database log files
Related
I'm trying to connect to an RDS cluster in AWS that's an Aurora PostgreSQL database. It's a brand-new database that I created along with the instances that I have Jira deployed to. However, when I try to connect to the instance from the Jira configuration screen I get this error
You have specified a database that is not empty, please specify an empty database.
I haven't touched this database at all, why is it giving me this error? I have one read and one write database in my cluster and the "hostname" is the endpoint for my write database, which is what the docs say. Could this be an issue with the Jira version I'm using?
This is the download link I'm using in my user-data script to install Jira. I'm also using PostgreSQL version 12.11
https://www.atlassian.com/software/jira/downloads/binary/atlassian-servicedesk-4.19.1-x64.bin
I switched to a different PostgreSQL version and now it's working.
PostgreSQL version 12.11 was giving me the error and switching to version 13.7 works as expected.
We are migrating data from a couple of Aurora instances to our own single Aurora instance. All of them run Postgres. I realized that some of our tasks are configured to use PostgreSQL while others are configured to use Amazon Aurora PostgreSQL. Now we have not had any visible problems using either of them (with source being an Aurora instance). But I am curious on whether this can create any future problems. Does anyone have an answer on what the differences are, since I couldn't find any documentation on this.
According to:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_UpgradeDBInstance.Upgrading.html
Manually Upgrading the Engine Version
To perform a major version upgrade of a DB cluster, you can restore a
snapshot of the DB cluster and specify a higher major engine version.
For information about restoring a DB cluster, see Restoring from a DB
Cluster Snapshot.
Yet I don't see anywhere the option of choosing a database engine at restore.
Is the documentation incorrect?
Am I missing something?
This is now supported by Aurora Postgres. There is no need to restore a snapshot into a new cluster anymore. The procedure is explained in the documentation.
As of 2020/05/22:
"Currently, you can do a major version upgrade from Aurora PostgreSQL version 9.6.11 or 9.6.12 database clusters to Aurora PostgreSQL version 10.11."
I also encountered the same issue when attempting to upgrade Aurora Postgres from 9.x to 10.x. The GUI and aws-cli does not allow you to specify a higher version number when restoring a snapshot.
AWS support confirms this is NOT supported at this time, this from my chat session with the tech:
I should have recollected earlier. Currently there isn't an in-place upgrade path from Aurora 9.x to version 10.
This is something that is frequently requested for and the engineering team is working on but we don't have an ETA as to when this will be fixed.
AWS did offer two options:
Use pg_dump and pg_restore to extract and load the data into an upgrade target cluster.
Use AWS Data Migration Services to migrate data. DMS has the advantage of minimal downtime but it comes with many limitations you would have to evaluate. For example some data types, like JSON and timestamps with time zones are 'partially migrated'.
Links to AWS DMS documentation using PostgreSQL as a source and/or target of DMS:
Postgres as DMS Source: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.Homogeneous.DMS
Postgres as DMS Target:
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html
The tech I chatted with favored pg_dump/pg_restore because it's more native, but it involve more downtime.
We have a four-node Hadoop cluster with HDP 2.4 and Kerberos installed in it. As this is our production cluster, We wanted to have HA for all the services including the PostgreSQL database which is used by Hive, Ambari, and Oozie for storing the metadata. However, the version of our postgreSQL, which is 8.4.2 doesn't support the inbuilt feature(stream replication) of Postgres.
So, we have decided to upgrade PostgreSQL to a version(9.3) ambari supports.
I followed this link to upgrade the Postgres. Everything went well. Expect that, we are getting the following error when restarting ambari server.
Ambari Server running with administrator privileges.
Running initdb: This may take upto a minute.
Data directory is not empty!
[FAILED]
Could someone help?
Thanks.
Your server want's to initilize the Database. I guess your Server does not see the Ambari DB. Use ambari-server setup zu restore the database Connection. Than the sever should start perfectly.
I found the fix for the issue here.
I am in the process of configuring the backup gem (http://backup.github.io/backup/v4/) to run on my EC2 instance, copy the PostgreSQL database in RDS, and store the backup in a new S3 bucket.
The backup gem runs the pg_dump command, however AWS doesn't allow for the same version of Postgres to be installed on both EC2 and RDS, resulting in the following error:
pg_dump: server version: 9.4.7; pg_dump version: 9.2.13
pg_dump: aborting because of server version mismatch
This is because the EC2 instance has version:
$ pg_dump --version
pg_dump (PostgreSQL) 9.2.13
And the RDS instance has version:
9.4.7-R1 (with the only other version option of 9.5.2-R1)
On EC2, running yum list postgres* only offers Available Packages up to PostgreSQL 9.3.
So it seems like I am unable to either downgrade RDS or upgrade EC2 to a matching version.
Here is my Backup gem model config if it helps: https://gist.github.com/anonymous/35f6f9e81846f53693fb03662c2192ad
Before too many people start reminding me that RDS has built-in backups, I am aware. My use-case: instead of only having full database fallbacks, I would also like the ability to roll back individual users' data to different time periods without affecting the whole database. I planned on keeping these manual backups and eventually writing a script to pull previous user specific data from them.
My friend recommended another option: If a user wants to roll back, I could spin up a new RDS from the automated snapshots, clone my EC2 instance, connect them to each other, collect the user specific data from that snapshot, and then merge those changes back into the main EC2 instance.
Set up PostgreSQL’s YUM repository on your EC2 instance:
https://yum.postgresql.org/
and install a newer PostgreSQL client version.