I have created a serverless DB cluster using
aws rds create-db-cluster --db-cluster-identifier psql-test --engine aurora-postgresql --engine-version 10.12 --engine-mode serverless --scaling-configuration MinCapacity=2,MaxCapacity=4,SecondsUntilAutoPause=1000,AutoPause=true --enable-http-endpoint --master-username postgres --master-user-password password
Which parameter should be used to include an initial database like it is possible here?
You can find this information on the official API documentation or on the help information of the command
But you have to use --database-name nameOfYourDB
--database-name (string)
The name for your database of up to 64 alphanumeric characters. If
you do not provide a name, Amazon RDS doesn't create a database in
the DB cluster you are creating.
Related
Overall I'm trying to create a Datastream Connection to a Postgres database in Cloud SQL.
As I'm trying to configure it all through Terraform, I'm stuck on how I should create a Replication Slot. This guide explains how to do it through the Postgres Client and running SQL commands, but I thought there might be a way to do it in the Terraform configuration directly.
Example SQL that I would like to replicate in Terraform:
ALTER USER [CURRENT_USER] WITH REPLICATION;
CREATE PUBLICATION [PUBLICATION_NAME] FOR ALL TABLES;
SELECT PG_CREATE_LOGICAL_REPLICATION_SLOT('[REPLICATION_SLOT_NAME]', 'pgoutput');
If not, does anyone know how to run the Postgres SQL commands against the Cloud SQL database through Terraform?
I have setup the Datastream and Postgres connection for all other parts. I'm expecting that there is a Terraform setting I'm missing or a way to run Postgres commands against the Google Cloud SQL Postgres database.
Unfortunately, there is no terraform resource for specifying a replication slot on a google_sql_database_instance.
I am using RDS Postgresql db instance - v11.12 (not Aurora) and I cannot see option "Create Aurora read replica" as expected. Any config I need to update?
I checked and see the region us-east-1 have that Aurora matching version with Postgres 11.12. But not sure why I cannot see replica option.
Appreciate any help :)
This happened for me as well, I didn't see the obvious reason why I can't create an Aurora Read Replica, so I tried creating it via CLI and it worked:
To create cluster
aws rds create-db-cluster
--db-cluster-identifier <new cluster name>
--db-subnet-group-name <subnet: same as original db>
--vpc-security-group-ids <security group id: like sg-12345678>
--engine aurora-postgresql
--engine-version <same as original db>
--replication-source-identifier <arn of original db: like arn:aws:rds:eu-west-1:065414729932:db:my-database>
<if encrypted>
--storage-encrypted
--kms-key-id <kms key arn, like: arn:aws:kms:eu-west-1:065514443312:key/195dff9f-0ee3-1024-b61f-3e19e4523495>
and instance in it
aws rds create-db-instance
--db-cluster-identifier <cluster name>
--db-instance-class <desired instnace class>
--db-instance-identifier <id of new instance>
--engine aurora-postgresql
and you will get Aurora Read Replica cluster (which will be read-only unless you promote it)
Commands are taken from official guide "Migrating data to Amazon Aurora"
I need help related to the configuration of PostgREST with the PostgreSQL database dumped into Heroku or AWS.
What are the exact steps from the beginning?
can I migrate from AWS RDS to standalone Postgres instance using AWS DMS?
I RTFM, but It does not state anywhere clearly If I can or not. In theory migration should be the same - create supporting scheme in RDS and move on. But have anyone done it?
Well, from AWS DMS manual:
AWS Database Migration Service (AWS DMS) can migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL or MySQL to Amazon Aurora. The source or target database must be on an AWS service.
In your case the source is on an AWS service, and if by "standalone" you mean a PostgreSQL instance on EC2 machine, then your target is as well. So, based on that, then answer should be "yes".
Yes, that is possible. We have moved data from RDS Postgres to Postgres Installed on EC2 Instance using DMS and works great so far. Make sure to create endpoints for the source/target and create a replication instance and you are good to go
I am very new to AWS, and Postgresql.
I have created a Postgresql db (using rds on aws)
I have uploaded several documents to multiple s3 buckets
I have a EC2 (Amazon Linux 64 bit) running
I tried to use a data pipeline, but nothing seems to be available (template) for Postgres. I can't figure out how to connect to my RDS instance and import/export data from postgres.
I assumed that I could use EC2 to grab from my S3 bucket and import into Postgres in lieu of no data pipeline template being available. If it is possible I have no idea how.. Please advise if possible..
S3 -> RDS direct load is now possible for PostgreSQL Aurora and RDS PostgreSQL >= 11.1 as aws_s3 extension.
Amazon Aurora with PostgreSQL Compatibility Supports Data Import from Amazon S3
Amazon RDS for PostgreSQL Now Supports Data Import from Amazon S3
Parameters are similar to those of PostgreSQL COPY command
psql=> SELECT aws_s3.table_import_from_s3(
'table_name', '', '(format csv)',
'BUCKET_NAME', 'path/to/object', 'us-east-2'
);
Be warned that this feature does not work for older versions.
I wish AWS extends COPY command in RDS Postgresql as they did in Redshift. But for now they haven't and we have to do it by ourselves.
Install awscli on your EC2 box (it might have been installed by default)
Configure your awscli with credentials
Use aws s3 sync or aws s3 cp commmands to download from s3 to your local directory
Use psql command to \COPY the files into your RDS (requires \ to copy from client directory)
Example:
aws s3 cp s3://bucket/file.csv /mydirectory/file.csv
psql -h your_rds.amazonaws.com -U username -d dbname -c '\COPY table FROM ''file.csv'' CSV HEADER'
The prior answers have been superseded by more recent events at AWS.
There is now excellent support for S3-to-RDS-database loading via the Data Pipeline service (which can be used for many other data conversion tasks too, this is just one example).
This AWS article is for S3-to-RDS-MySQL. Should be very similar for RDS-Postgres.
http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-template-copys3tords.html
if you can launch the psql client and connect to RDS on EC2 instance, you should be able to use the following command:
\copy customer_orders from 'myfile.csv' with DELIMITER ','