I have a problem replicating postgres to clickhouse. I can easily create materialized postgresql engine and it works fine. But we need to have synchronous commits work so that any write to our main postgres db should wait for the same writes to be written to clickhouse before the transaction is completed.
I tried adding the following setting to postgresql config. But this prevents clickhouse from replicating.
synchronous_commit: "on"
synchronous_standby_names: "'*'"
Related
I have several postgres databases which need to be replicated as-is to a single aws redshift.
We have currently set up DMS services to the same. However, we keep encountering issues such as source database full, large column issues and most importantly the issue in DMS when new columns with defaults are added on postgres databases(This does not replicate with ongoing replication)
So, are there any other ways that we can set up this ongoing replication?
my primary server is Postgresql-9.4 and the secondary server is Postgresql-13, I followed all the steps but while restarting the secondary server, I am facing the error "An old version of the database format was found. You need to dump and reload before using PostgreSQL 13." how I should resolve it.
You cannot have streaming replication between different PostgreSQL versions, and you cannot have logical replication with versions less than v10.
You will have to use trigger-based replication like with Slony-I.
I have a requirement of replicate data from AWS RDS Postgres(9.6) Database to On-Premise Postgres(9.5) Database. I have found stuff about replication from On-premise to On-premise. But How can we implement it for AWS RDS to On-premise?
I do this using Bucardo.
Check-out this: https://bucardo.org/Bucardo/
With Bucardo you can replicate RDS postgres instance to a slave postgres present somewhere, only configuring slave, so without the needs to configure RDS stuff.
Also you can do this with zero downtime.
Anyway I am not sure this will work using different versions of Postgresql. You should use same version if possible. I tested it with 9.4.x and it is working.
UPDATE
I can confirm that this is working also using different version of Postgres, for example I was able to replicate with these versions:
AWS RDS postgresql 9.4.x
On-premise postgresql 9.6.x
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.FeatureSupport.LogicalReplication
Beginning with PostgreSQL version 9.4, PostgreSQL supports the
streaming of WAL changes using logical replication slots. Amazon RDS
supports logical replication for a PostgreSQL DB instance version
9.4.9 and higher and 9.5.4 and higher. Using logical replication, you can set up logical replication slots on your instance and stream
database changes through these slots to a client like pg_recvlogical.
Logical slots are created at the database level and support
replication connections to a single database.
mind possible problems eg https://dba.stackexchange.com/questions/173267/aws-rds-postgres-logical-replication
Currently I have 1 postgres instance which is starting to receive too much load and want create a cluster of 2 postgres nodes.
From reading the documentation for postgres and pgpool, it seems like I can only write to a master and read from a slave or run parallel queries.
What I'm looking for is a simple replication of a database but with master/slave based on which table is being updated. Is this possible? Am i missing it somewhere in the documentation?
e.g.
update users will be executed on server1 and replicated to server2
update big_table will be executed on server2 and replicated back to server1
What you are looking for is called MASTER/MASTER replication. This is supported natively (without PgPool) since 9.5. Note, that it's an "eventually consistent" architecture, so your application should be aware of possible temporary differences between the two servers.
See PG documentation for more details and setup instructions.
I tried replication in postgres 9.3, and successfully implemented it.
After that i am unable to create new database in slave. It says "cannot execute CREATE TABLE in a read-only transaction." Does this mean i cannot create or alter database in slave?
That is correct. The slave is in read-only mode and all changes you want to make should be done on the master server that propagates it to its slaves. Making the slave read-write would defeat the purpose of the replication (if we are not considering master-master or multi-master replication)