AWS PostgreSQL `Cannot find version X for aurora` - postgresql

Similar to Error creating DB Instance: InvalidParameterCombination: Cannot find version 5.6.10a for aurora-mysql but for Postgres, which has a different cause.
* error creating RDS Cluster (mycompany-pulumi-aurora-cluster) Instance: InvalidParameterCombination: Cannot find version 12.6 for aurora
status code: 400, request id: someid
But 12.6 is valid for Aurora PostgreSQL in that region:
$ aws rds describe-db-engine-versions --engine aurora-postgresql --query '*[].[EngineVersion]' --output text --region us-east-2
9.6.3
9.6.6
9.6.8
9.6.9
9.6.11
9.6.12
9.6.16
9.6.17
9.6.18
9.6.19
9.6.21
10.4
10.5
10.6
10.7
10.11
10.12
10.13
10.14
10.14
10.16
11.4
11.6
11.7
11.8
11.9
11.11
12.4
12.6

There's two simultaneous causes of this error:
Aurora versions are restricted based on the instance type
You need to specify the engineVersion for each cluster instance (as well as the cluster itself).
Resulting code (I'm using Pulumi, but the same logic applies for Terraform, raw CloudFormation etc).
const rdsCluster = new aws.rds.Cluster("default", {
clusterIdentifier: config.database.clusterName,
engine: "aurora-postgresql",
// Restricted based on https://docs.amazonaws.cn/en_us/AmazonRDS/latest/AuroraUserGuide/Concepts.DBInstanceClass.html#Concepts.DBInstanceClass.SupportAurora
engineVersion: "12.7",
databaseName: config.database.name,
masterUsername: config.database.username,
masterPassword: config.database.password,
dbSubnetGroupName: rdsSubnetGroup.name,
storageEncrypted: true,
});
// https://www.pulumi.com/docs/reference/pkg/aws/rds/clusterinstance/#engineversion_nodejs
const clusterInstances: Array<aws.rds.ClusterInstance> = [];
for (let range = 0; range < config.database.instanceCount; range++) {
clusterInstances.push(
new aws.rds.ClusterInstance(`clusterInstances-${range}`, {
identifier: `aurora-cluster-demo-${range}`,
clusterIdentifier: rdsCluster.id,
instanceClass: config.database.instanceType,
// Engine must be specified otherwise AWS will give a misleading error about DB versions not being available.
// https://github.com/hashicorp/terraform-provider-aws/issues/2419#issuecomment-347060274
engine: "aurora-postgresql",
engineVersion: "12.7",
})
);
}

Related

Postgis st_transform error: SQL Error [XX000]: ERROR: could not form projection (LWPROJ) from 'srid=7801' to 'srid=4326'

I have 4 servers with the same versions of OS (Ubuntu 20.04), Postgis and Postgre:
POSTGIS="3.2.1 5fae8e5" [EXTENSION] PGSQL="130" GEOS="3.10.1-CAPI-1.16.0" PROJ="6.3.1" LIBXML="2.9.10" LIBJSON="0.13.1" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)"
PostgreSQL 13.6 (Ubuntu 13.6-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit
The query below works on any server except one:
`
SELECT
st_asgeojson(st_transform("table-name".geom, 4326)) AS way
FROM "table-name"
`
The server with the issue responds this error:
SQL Error [XX000]: ERROR: could not form projection (LWPROJ) from 'srid=7801' to 'srid=4326'
I upgraded the postgis server to 3.3.2 but the result is the same.
Can you help me?

Postgis ST_TRANSFORM failing with ESPG 27200

I have a database that was using postgis 2.2.1 extension on postgresql 9.5 database on server A. But, recently I have upgraded this to 2.5.5 and copied this database to new server B which also has postgis 2.5.5. Result of Postgis_full_version() on A is:
POSTGIS="2.5.5" [EXTENSION] PGSQL="95" GEOS="3.7.1-CAPI-1.11.1 27a5e771" PROJ="Rel. 4.9.2, 08 September 2015" GDAL="GDAL 1.11.3, released 2015/09/16" LIBXML="2.9.3" LIBJSON="0.11.99" LIBPROTOBUF="1.2.1" RASTER
and same for server B is:
POSTGIS="2.5.5" [EXTENSION] PGSQL="95" GEOS="3.8.0-CAPI-1.13.1 " PROJ="Rel. 6.3.1, February 10th, 2020" GDAL="GDAL 3.0.4, released 2020/01/28" LIBXML="2.9.10" LIBJSON="0.13.1" LIBPROTOBUF="1.3.3" RASTER
On server B, the database is throwing error when using ESPG 27200 in ST_transform queries but working for other ESPG projections.
For example, when I am executing query:
select st_asText(st_transform(st_geomfromtext('POINT(172 -43)', 4326), 27200));
I am getting this error:
NOTICE: PostGIS was unable to transform the point because either no grid shift files were found, or the point does not lie within the range for which the grid shift is defined. Refer to the ST_Transform() section of the PostGIS manual for details on how to configure PostGIS to alter this behaviour.
ERROR: transform: couldn't project point (172 -43 0): failed to load datum shift file (-38)
SQL state: XX000
Difference between server A and server B config is A is running on GDAL 1.11
I am a newbie to postgresql and postgis, so I am not sure how to fix the problem.

Serverless-framework postgresql version upgrading problem

We use serverless-framework and Postgresql.
Type: AWS::RDS::DBInstance
Properties:
EngineVersion: 11.8
It was 11.8 before, we successfully upgraded it from 11.8 to 11.9 by changing the version in serverless.yml in EngineVersion.
But now we need to upgrade it to 11.10 and we have the following issue:
The error message:
Cannot upgrade postgres from 11.9 to 11.1 (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: e0d1923a-cf98-44ea-a9e4-bc3871e33bf6; Proxy: null).
So it looks like it tries to upgrade to 11.1 rather than 11.10.
The 11.10 value you're specifying for the EngineVersion property is processed as a number by the YAML engine since you're not using quotes, so the trailing zeros are removed and the resulting value is 11.1. You need to specify the value in quotes, like this: "11.10"
It seems like Serverless ignores '0' in Engine Version that you are trying to set up.
Could you try to escape version with quotes?
It will look like this:
Type: AWS::RDS::DBInstance
Properties:
EngineVersion: '11.10'
It should help to solve the problem and upgrade version successfully

How can I fix 'DB Clusters not supported for engine' when restoring a snapshot on AWS Aurora?

I'm using the AWS API to restore a cluster snapshot. My code is simple enough, and follows the restoreDBClusterFromSnapshot documentation fairly closely:
await rds.restoreDBClusterFromSnapshot({
DBClusterIdentifier: SNAPSHOT_NAME,
SnapshotIdentifier: `arn:aws:rds:eu-west-1:ACCOUNT_ID:cluster-snapshot:SNAPSHOT_NAME`,
Engine: "postgres",
EngineMode: 'provisioned',
EngineVersion: '9.6.12',
Tags: [
{
Key: 'Creator',
Value: USERNAME
}
]
}).promise()
However this call fails with
DB Clusters not supported for engine: postgres
I know that's not true - we're running a Postgres Cluster in production.
How can I restore the cluster snapshot?
The error is a little deceptive - it's not that clusters aren't available for postgres engine, is that postgres isn't a valid engine name.
The correct name for AWS postgres engine is aurora-postgresql. I wasn't able to find any mention of this in the AWS documentation however:
Running rds.describeDBClusters() on the existing cluster made through the Management Console shows the Engine as aurora-postgresql.
The TypeScript info for createDBInstance() mentions:
Valid Values: aurora (for MySQL 5.6-compatible Aurora) aurora-mysql (for MySQL 5.7-compatible Aurora) aurora-postgresql mariadb mysql oracle-ee oracle-se2 oracle-se1 oracle-se postgres sqlserver-ee sqlserver-se sqlserver-ex sqlserver-web
await rds.restoreDBClusterFromSnapshot({
DBClusterIdentifier: SNAPSHOT_NAME,
SnapshotIdentifier: `arn:aws:rds:eu-west-1:ID_NUMBER:cluster-snapshot:${SNAPSHOT_NAME}`,
Engine: "aurora-postgresql",
EngineMode: 'provisioned',
EngineVersion: '9.6.12',
Tags: [
{
Key: 'Creator',
Value: USERNAME
}
]
}).promise()

Heroku can't connect with Postgres DB/Knex/Express

I have an Express API deployed to Heroku, but when I attempt to run the migrations, it throws the following error:
heroku run knex migrate:latest Running knex migrate:latest on ⬢
bookmarks-node-api... up, run.9925 (Free) Using environment:
production Error: connect ECONNREFUSED 127.0.0.1:5432
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1117:14)
In my knexfile.js, I have:
production: {
client: 'postgresql',
connection: {
database: process.env.DATABASE_URL
},
pool: {
min: 2,
max: 10
},
migrations: {
directory: './database/migrations'
}
}
I also tried assigning the migrations directory to tableName: 'knex_migrations' which throws the error:
heroku run knex migrate:latest Running knex migrate:latest on ⬢
bookmarks-node-api... up, run.7739 (Free) Using environment:
production Error: ENOENT: no such file or directory, scandir
'/app/migrations'
Here is the config as set in Heroku:
-node-api git:(master) heroku pg:info
=== DATABASE_URL
Plan: Hobby-dev
Status: Available
Connections: 0/20
PG Version: 10.7
Created: 2019-02-21 12:58 UTC
Data Size: 7.6 MB
Tables: 0
Rows: 0/10000 (In compliance)
Fork/Follow: Unsupported
Rollback: Unsupported
I think the issue is that for some reason, it is looking at localhost for the database, as if the environment is being read as development though the trace shows Using environment: production.
When you provide an object as your connection you're providing individual parts of the connection information. Here, you're saying that the name your database is everything contained in process.env.DATABASE_URL:
connection: {
database: process.env.DATABASE_URL
},
Any keys you don't provide values for fall back to defaults. An example is the host key, which defaults to the local machine.
But the DATABASE_URL environment variable contains all of the information that you need to connect (host, port, user, password, and database name) in a single string. That whole value should be your connection setting:
connection: process.env.DATABASE_URL,
You should check to see if the Postgres add-on is setup as described in these docs since the DATABASE_URL is automatically set for you as stated here.