We currently have a setup for staging in RDS and we need to replicate that RDS in our Dev Kubernetes environment. What will be the steps required for the same?
Related
We created a PostgreSQL production database in AWS RDS a few months back and now we have received a requirement from the application team to install a connection pooler - PgBouncer in it.
I couldn't see any proper document from the AWS team showing the steps to install a PgBouncer and configure it in the AWS RDS PostgreSQL DB instance.
Is it possible to install PgBouncer in the AWS RDS PostgreSQL DB instance? If yes, can someone help me and share the steps to install and configure PgBouncer in the AWS RDS PostgreSQL?
Thank you!
I have a Node JS application that has come into my situation. The app has a Postgres database that is currently hosted on Heroku that needs to be migrated to RDS. I have created a Postgres Aurora database cluster and have performed a restore using pg_restore. I am now trying to connect the Node JS application to the RDS database cluster and am having some trouble how to connect the app to the database in RDS.
Note, the RDS database cluster is private. The only access to the database is within the VPC. If there are any guides available to achieve this connectivity, that would greatly be appreciated.
I am new working with Airflow and Kubernetes. I am trying to use apache Airflow in Kubernetes.
To deploy it I used this chart: https://github.com/apache/airflow/tree/master/chart.
When I deploy it like in the link above a PostgreSQL database is created. When I explore the value.yml file of the chart I found this:
# Configuration for postgresql subchart
# Not recommended for production
postgresql:
enabled: true
postgresqlPassword: postgres
postgresqlUsername: postgres
I cannot find why is not recommended for production.
and also this:
data:
# If secret names are provided, use those secrets
metadataSecretName: ~
resultBackendSecretName: ~
# Otherwise pass connection values in
metadataConnection:
user: postgres
pass: postgres
host: ~
port: 5432
db: postgres
sslmode: disable
resultBackendConnection:
user: postgres
pass: postgres
host: ~
port: 5432
db: postgres
sslmode: disable
What is recommended for production? use my own PostgreSQL database outside Kubernetes? If it is correct, how can I use it instead this one? How I have to modify it to use my own postgresql?
The reason why it is not recommended for production is because the chart provides a very basic Postgres setup.
In container world containers are transient unlike processes in the VM world. So likelihood of database getting restarted or killed is high. So if we are running stateful components in K8s, someone needs to make sure that the Pod is always running with its configured storage backend.
The following tools help to run Postgres with High Availablity on K8s/containers and provides various other benefits:
Patroni
Stolon
We have used Stolon to run 80+ Postgres instances on Kubernetes in a microservices environment. These are for public facing products so services are heavily loaded as well.
Its very easy to setup a Stolon cluster once you understand its architecture. Apart from HA it also provides replication, standby clusters and CLI for cluster administration.
Also please consider this blog as well for making your decision. It brings in the perspective of how much Ops will be involved in different solutions.
Managing databases in Kubernetes its a pain and not recommended due to scaling, replicating, backups, among other common tasks are not as easy to do, what you should do is set up your own Postgres in VM or a managed cloud service as RDS or GCP, more information:
https://cloud.google.com/blog/products/databases/to-run-or-not-to-run-a-database-on-kubernetes-what-to-consider
It seems migrating Postgres database using data directory is viable:
How To Move a PostgreSQL Data Directory to a New Location on Ubuntu 16.04
But in case of RDS because SSHing to the RDS is not possible I am here to ask if there is any way to speed up my Postgres database migration.
You could use the AWS Database Migration Service:
AWS Database Migration Service (AWS DMS) is a cloud service that makes it easy to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores. You can use AWS DMS to migrate your data into the AWS Cloud, between on-premises instances (through an AWS Cloud setup), or between combinations of cloud and on-premises setups.
We want to migrate an external postgresql database into amazon RDS however for some time we need to keep both of them working and in sync. I have found ways of doing it but only with RDS being the master and not the Slave. Is there a good and viable solution which could help us?
There is AWS service Database Migration Service which can be used to migrate external databases to Amazon AWS RDS.
https://docs.aws.amazon.com/dms/latest/userguide/Welcome.html
It supports postgresql 9.4 and higher.
There are different task types and you need to use Ongoing Replication to keep them sync.
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Task.CDC.html
To setup that task you need to enable Logical Replication for source database