I am looking for a way to manage schema changes to my AWS Aurora Postgres instance.
My whole AWS stack is set up using a Cloudformation template which is used to automatically deploy the stack when a change is detected in the source control. The Cloudformation template is built, a change set is prepared and finally excecuted on the stack.
I was hoping that the table definition of my Aurora instance could go inside the Cloudformation template somehow, so the schema migrations could be a part of the change set. Is this possible?
Note, I have seen this recommendation: https://aws.amazon.com/blogs/opensource/rds-code-change-deployment/
For anything custom like that use a Custom Resource Lambda that you can include in your Cloud Formation stack. The Lambda will need a layer for your postgress driver and it needs to include the migration script in the Lambda.
See the answer at this link, you will get 3 different options how you can trigger the Lambda.
Is it possible to trigger a lambda on creation from CloudFormation template
Related
I am trying to do MongoDB on AWS following the AWS deployment guide. It is defaulted to boot up m5.large EC2s. However, I am only experimenting so I want to use a free tier EC2. When I add t2.micro to the allowed values and set it as default I get an error as pictured below.
Is there anyway I can get MongoDB running on AWS with 3 replications using the cloudformation method with free tier t2.micro instances.? If not, any better methods?
The MongoDB on AWS - Quick Start has multiple templates that are deployed.
I notice that the NodeInstanceType is used and defined in multiple templates, presumably with the values passed from the master template to the node templates. Therefore, your changes will probably need to be made on any template that defines the NodeInstanceType parameter. I recommend you check all of the templates for such references.
We already have a cluster and instance of Aurora PostgreSql in abc region. Now as part of disaster recovery strategy, we are trying to create a read replica in a xyz region.
I was able to create it manually by clicking on "Add Region" in AWS web console. As explained here.
As part of it, following as been created.
1. A global database to the existing cluster
2. Secondary region cluster
3. Secondary region instance.
Everything is fine. Now I have to implement this through cloud formation script.
My first question is, can we do this through Cloud formation script without losing data if primary cluster and instance already created ?
If possible, please share aws doc for cloud formation scripts.
Please see the other post on this subject: CloudFormation templates for Global Aurora Database
The API that is required for setting up the GlobalCluster is AWS::RDS::GlobalCluster and this is currently not listed in CloudFormation documentation.
I was able to do the same using Terraform and that is documented for PostgreSQL here: Getting Aurora PostgreSQL Global Database setup using Terraform
how to setup tables of aws aurora mysql using aws cloudformation or aws cdk?
in my setup i have a serverless app using lambda for various microservices. the datebase is a serverless aurora mysql database. to provision the aws infrastructure i will use aws CDK. i like to setup the database using some migration tools like liquibase or sequelize.
for the moment i am using a separat lambda function. the lambda function executes liquibase to setup db changes. but i have to execute the function separately after deployment with CDK is succeded.
an execution triggered after the execution of the cloudformation stack (cdk stack) would be optimal?! I would like to avoid a CI / CD stack via code pipeline.
does anyone has best practice to setup database at provision time?
Cloud watch rules
Cloud watch rules based on cloudformation events can be used to route events for processing lambda. Cloud watch rules can be a part of the CDK deployment description.
The triggered function can then execute liquibase, flyway, sequelize or something else to spinup or change db.
---- or ----
Cloudformation custom resource
AWS cloudformation custom ressource can execute a lambda function during cloudformation lifecycle.
The triggered function can then execute liquibase, flyway, sequelize or something else to spinup or change db.
I use Cloudformation custom resources for running database migrations and initial database setup scripts at deployment time.
This is the recommended way for running DB migrations for serverless applications if you don't want to rely on a CI/CD pipeline to do it for you.
Here's a well written blog post by Alex DeBrie about CF custom resources: https://www.alexdebrie.com/posts/cloudformation-custom-resources/
New to cloudformation. I am spawning PostgreSQL RDS instance using a aws cloudformation script. Is there a way to enable PostGIS (and other extensions) from aws cloudFormation script?
Working with PostGIS PostGIS is an extension to PostgreSQL for storing
and managing spatial information. If you are not familiar with
PostGIS, you can get a good general overview at PostGIS Introduction.
You need to perform a bit of setup before you can use the PostGIS
extension. The following list shows what you need to do; each step is
described in greater detail later in this section.
Connect to the DB instance using the master user name used to create the DB instance.
Load the PostGIS extensions.
Transfer ownership of the extensions to therds_superuser role.
Transfer ownership of the objects to the rds_superuser role.
Test the extensions.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html
I'm not sure but maybe you can create a lambda function and RDS with your cloudformation and then you can invoke your lambda to do above steps. You need to try.
best,
I think this can be done with AWSUtility::CloudFormation::CommandRunner.
Basically we can run bash command with this (https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/)
I don't think you will be able to achieve it by using cloudformation. Cloudformation is a provisioning tool not a configuration management tool.
I use CloudFormation's AWS::RDS::DBCluster resource to create my Aurora MySQL database cluster.
My question is, has anyone created stored procedures as well as events in Aurora MySQL via CloudFormation? Is that even possible?
Delivering these via CloudFormation would allow me to recreate the infrastructure without deploying the stored procedures and events separately.
There's no way to configure stored procedures and events with the AWS::RDS::DBCluster CloudFormation resource directly.
My suggestion would be to provision an AWS::EC2::Instance containing a UserData script that installs the mysql client, then executes the contents of a user-provided MySQL script creating events/stored-procedures on the newly-created DB instance.