According to:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_UpgradeDBInstance.Upgrading.html
Manually Upgrading the Engine Version
To perform a major version upgrade of a DB cluster, you can restore a
snapshot of the DB cluster and specify a higher major engine version.
For information about restoring a DB cluster, see Restoring from a DB
Cluster Snapshot.
Yet I don't see anywhere the option of choosing a database engine at restore.
Is the documentation incorrect?
Am I missing something?
This is now supported by Aurora Postgres. There is no need to restore a snapshot into a new cluster anymore. The procedure is explained in the documentation.
As of 2020/05/22:
"Currently, you can do a major version upgrade from Aurora PostgreSQL version 9.6.11 or 9.6.12 database clusters to Aurora PostgreSQL version 10.11."
I also encountered the same issue when attempting to upgrade Aurora Postgres from 9.x to 10.x. The GUI and aws-cli does not allow you to specify a higher version number when restoring a snapshot.
AWS support confirms this is NOT supported at this time, this from my chat session with the tech:
I should have recollected earlier. Currently there isn't an in-place upgrade path from Aurora 9.x to version 10.
This is something that is frequently requested for and the engineering team is working on but we don't have an ETA as to when this will be fixed.
AWS did offer two options:
Use pg_dump and pg_restore to extract and load the data into an upgrade target cluster.
Use AWS Data Migration Services to migrate data. DMS has the advantage of minimal downtime but it comes with many limitations you would have to evaluate. For example some data types, like JSON and timestamps with time zones are 'partially migrated'.
Links to AWS DMS documentation using PostgreSQL as a source and/or target of DMS:
Postgres as DMS Source: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.Homogeneous.DMS
Postgres as DMS Target:
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.PostgreSQL.html
The tech I chatted with favored pg_dump/pg_restore because it's more native, but it involve more downtime.
Related
I'm trying to upgrade an RDS database cluster engine from Aurora PostgreSQL 9.6.19 before its end of life, I made copy and tried to upgrade to 9.6.21 and 10.16 but everytime the same problem happens:
Database cluster is in a state that cannot be upgraded: Postgres cluster is in a state where pg_upgrade can not be completed successfully.
The status of the database is Available so maybe it refers to something else but I don't know what and how to fix it, I've tried looking for answers to no avail.
Has anyone fixed this?
The pg_upgrade_internal log file will usually contain details on any failures/errors.
You can take a look on these logs using the command line:
aws rds describe-db-log-files --db-instance-identifier my-db-instance
Or via console, or RDS API.
For more information take a look on these links: Upgrading the PostgreSQL DB engine for Amazon RDS, Viewing and listing database log files
We are migrating data from a couple of Aurora instances to our own single Aurora instance. All of them run Postgres. I realized that some of our tasks are configured to use PostgreSQL while others are configured to use Amazon Aurora PostgreSQL. Now we have not had any visible problems using either of them (with source being an Aurora instance). But I am curious on whether this can create any future problems. Does anyone have an answer on what the differences are, since I couldn't find any documentation on this.
We want to migrate an external postgresql database into amazon RDS however for some time we need to keep both of them working and in sync. I have found ways of doing it but only with RDS being the master and not the Slave. Is there a good and viable solution which could help us?
There is AWS service Database Migration Service which can be used to migrate external databases to Amazon AWS RDS.
https://docs.aws.amazon.com/dms/latest/userguide/Welcome.html
It supports postgresql 9.4 and higher.
There are different task types and you need to use Ongoing Replication to keep them sync.
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Task.CDC.html
To setup that task you need to enable Logical Replication for source database
I have a requirement of replicate data from AWS RDS Postgres(9.6) Database to On-Premise Postgres(9.5) Database. I have found stuff about replication from On-premise to On-premise. But How can we implement it for AWS RDS to On-premise?
I do this using Bucardo.
Check-out this: https://bucardo.org/Bucardo/
With Bucardo you can replicate RDS postgres instance to a slave postgres present somewhere, only configuring slave, so without the needs to configure RDS stuff.
Also you can do this with zero downtime.
Anyway I am not sure this will work using different versions of Postgresql. You should use same version if possible. I tested it with 9.4.x and it is working.
UPDATE
I can confirm that this is working also using different version of Postgres, for example I was able to replicate with these versions:
AWS RDS postgresql 9.4.x
On-premise postgresql 9.6.x
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.FeatureSupport.LogicalReplication
Beginning with PostgreSQL version 9.4, PostgreSQL supports the
streaming of WAL changes using logical replication slots. Amazon RDS
supports logical replication for a PostgreSQL DB instance version
9.4.9 and higher and 9.5.4 and higher. Using logical replication, you can set up logical replication slots on your instance and stream
database changes through these slots to a client like pg_recvlogical.
Logical slots are created at the database level and support
replication connections to a single database.
mind possible problems eg https://dba.stackexchange.com/questions/173267/aws-rds-postgres-logical-replication
can I migrate from AWS RDS to standalone Postgres instance using AWS DMS?
I RTFM, but It does not state anywhere clearly If I can or not. In theory migration should be the same - create supporting scheme in RDS and move on. But have anyone done it?
Well, from AWS DMS manual:
AWS Database Migration Service (AWS DMS) can migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL or MySQL to Amazon Aurora. The source or target database must be on an AWS service.
In your case the source is on an AWS service, and if by "standalone" you mean a PostgreSQL instance on EC2 machine, then your target is as well. So, based on that, then answer should be "yes".
Yes, that is possible. We have moved data from RDS Postgres to Postgres Installed on EC2 Instance using DMS and works great so far. Make sure to create endpoints for the source/target and create a replication instance and you are good to go