Is there anyway to enable logical replication wal_level = logical on Google Cloud Postgres instance? I want to do CDC (change data capture) to stream WAL Record to Apache Kafka. Thanks for your help!
Update:
This feature now is enabled on Google Cloud SQL. To enable wal_level to replication you just need to set the flag cloudsql.logical_decoding as on.
For more details:
https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication
Note: this issue was solved in August, 30th release.
No, this is not supported by Cloud SQL for PostgreSQL.
The Cloud SQL engineering team is working on this issue, here you can find a Feature Request. You can star it in order to get notified when it gets updated. Also you can write a comment there and ask a question. Please, bear in mind that we don't have an ETA for it.
Related
I wanted to check if there is a way to stop and resume PostgreSQL replication using pglogical? For some reason, if either the publisher or subscriber needs to restart and go offline for sometime (or connectivity issues because of some n/w issues), is there a way to stop the replication and resume it again? I know it is not a relevant example but AWS DMS (which used Postgres native logical replication) gives you an option to stop/resume the replication. Wanted to check if there is a similar option available in pglogical.
Thanks
Sure there is. I presume You have pglogical extension installed and there is standard logical replication:
select pglogical.alter_subscription_disable('subscription_name');
You can use a WHERE clause to aim it specifically:
select pglogical.alter_subscription_disable(subscription_name) from pglogical.subscriptions where writer = 'name_of_writer';
Since Citus is not going to be available as a Managed Service in AWS, I am trying move the database to RDS (not the whole history but only the transactional portion as an OLTP). The migration from Citus is not clear because the data does not reside in a single node. I want to check the options we might have to move data from Citus to RDS.
Amazon DMS: This option is good for the supported databases (PostgreSQL) but we do not know what behavior this will have in Citus from the distributed nature of the engine. Has someone migrated the data to S3, to another DB or something in these lines?
I saw this paper from AWS https://d1.awsstatic.com/whitepapers/aws-cloud-data-ingestion-patterns-practices.pdf?did=wp_card&trk=wp_card on how to ingest data from different sources and DMS seems like a good option but I do not know the internals of Citus that well to tell if we will get all the data and gather the CDC correctly.
A Custom migration: Via a support ticket, we can access the S3 buckets that Citus uses for Disaster recovery where the WAL logs are available and we could use something like WAL-G to take those logs and replicate them in a Postgres instance. The issue here is that this is a very custom migration and the development time might be too high.
Is there any other option to move data from Citus to RDS or Aurora in AWS, what looks like a good path to make the database migration? All the documents refer to move data the other way around, from Aurora or RDS to Citus.
Sumedh from Citus Cloud here. Please go ahead and open a support ticket with us to further investigate solutions. We can evaluate if using DMS is a viable approach for your use-case.
Is replication not supported for Azure Postgres Flexible Server?
Zone redundancy can not be treated as alternative for Disaster Recovery. Such an important feature is missing. May be until it is in preview?
Hi #Abhijeet Azure Postgres Flexible server supports logical replication. You can refer to this documentation for more details. Let me know if you have additional queries.
Regards,
Oury
If by "replication" you mean "read replicas" then indeed this is currently not supported.
High Availability options, in theory, do implement replication, but as of 10.2022 still do not support read access and are meant for fail-over purposes.
On 12th of Oct 2022 they did announce that they are working on Read Replicas, and that the feature will enter public preview "soon-ish" but no specific dates are yet communicated.
See: https://techcommunity.microsoft.com/t5/azure-database-for-postgresql/ignite-2022-continued-innovation-with-azure-database-for/ba-p/3651653
I have created a replicated Postgresql database (Master - Slave). I did this with an already existing Ansible Playbook (Role) , which I don't fully understand yet. The cluster currently consists of only 2 databases on different VMs.
So I want to test this replication now.
Unfortunately I have little experience with Postgresql.
How can I control whether they connect stable?
If the slave really takes over the task if the master should fail?
Many thanks for any information, tips & tricks.
Postgresql v. 9.6
Official PostgreSQL does not yet support automatic failover (Although there are multiple third-party projects which support this feature). Therefore if the deployment you have mentioned is only official PostgreSQL, after master failure, none of replicas take over the write task. But they can answer read queries if they are configured as hot_standby.
If you want to check the state of replication, in master you can check out pg_stat_replication in master.
Also these official docs would help you understand Postgres streaming replication & failover better:
https://www.postgresql.org/docs/9.6/warm-standby.html#STREAMING-REPLICATION
https://www.postgresql.org/docs/9.6/warm-standby-failover.html
I am currently working on google cloud sql. where i have make one single instance of postgres server. i want to build two node postgres cluster with both nodes active.
How can i built it. any help/tutorial or link regarding to it will be highly appreciated.
Thanks
Cloud SQL for Postgres Beta currently does not support any form of High Availability [1].
If this is something you require immediately, you would need to run the nodes yourself or use another solution.
[1] https://cloud.google.com/sql/docs/postgres/high-availability