Currently we are building replication of the primary db with hotstand by referring the link
https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby
Is there a way to replicate only a single table from the primary database?
Only with logical replication but I don't know if this is possible on GCP.
Cloud SQL Postgresql currently doesn’t support logical replication. There is a feature request that is publicly viewable here which you can up-vote with the +1 and star to get any updates.
Related
How to migrate my whole database which is currently in AWS RDS Postgres to AWS Redshift and also can you please help me out how can I keep both these DBs in sync. I want to sync even if any column is updated in RDS so it must get updated in Redshift also.
I know we can achieve it with AWS Glue, but the above scenario is mandatory in my case. Migration task is easy to do but to to the CDC migration is bit challenging. I am also aware about the bookmark key but my situation is bit different, I do not have any sequential column in the tables, but it has updated_at field in all the tables so this column is the only field on which I can check whether the record is processed or not so that duplicate processing may not occur and if any new data is inserted it should also get replicated in RedShift.
So, would anyone help me out to do this even by using pyspark script?
Thanks.
I have created two publications one on each PostgreSQL instance but on the same table. Then I subscribed to these publications from each instance. I didn't see an error in the logs of two instances but I can see replication works on one side i.e. uni-directional.
Does logical replication support bi-directional or not?
Thanks
No, you cannot use logical replication to replicate the same table in both directions. This would necessitate complicated conflict resolution, which PostgreSQL doesn't have at the moment.
I am new to PostgreSQL, I am trying to replicate single database like only one database need to replicate from Master to Slave Server.
I think you can find more information here: https://www.postgresql.org/docs/10/static/logical-replication.html
I am not sure without a sample code (from your side) what is your code/error/etc.
I have 5 users which uses 5 different servers(using openerp), each one uses the same database copy. Whenever the user enters data to his database, then it should sync to the 6th server's database an then sync the data in the 6th database to all other 4 users databases. So any data entry in any database should sync to the other databases running in different servers. Is that possible? how can I achieve it?
Edit:
I found Bucardo, but it need primary key for every table. But in openerp, there are many2many relation tables which doesn't have a primary key.
This is called master-master or multi-master replication:
http://en.wikipedia.org/wiki/Multi-master_replication#PostgreSQL
http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling
You can also check out SymmetricDS or Daffodil which let you sync over HTTP.
A pure PostgreSQL installation supports currently only single master replication.
However you may achieve multi-master replication by installing additional tools http://wiki.postgresql.org/wiki/Clustering.
You should also check out Pg documentation on this topic Chapter 25. High Availability, Load Balancing, and Replication
What would be the best way to replicate individual DB tables from a Master postgresql server to a slave machine? It can be done with cron+rsync, or with whatever postgresql might have build in, or some sort of OSS tool, but so far the postgres docs don't seem to cover how to do table replication. I'm not able to do a full DB replication because some tables have license->IP stuff connected to it, and I can't replicate those on the slave machine. I don't need instant replication, hourly would be acceptable as well.
If I need to just rsync, can someone help identify what files within the /var/lib/pgsql directory would need to be synced, or how I would know what tables they are.
Starting with Postgres 10, logical replication is built into Postgres! This is often a better solution than external solutions. The Postgres docs are great and easy to follow. It's very easy. See the quick setup docs, which in essense boils down to running this:
-- On publisher DB
CREATE PUBLICATION mypub FOR TABLE users, departments;
-- On subscriber DB
CREATE SUBSCRIPTION mysub CONNECTION 'dbname=foo host=bar user=repuser' PUBLICATION mypub;
You might want to try Bucardo, which is an open source software to synchronize rows between tables even if they are in a remote location. It's a very simple software, and it is capable of creating one-way synchronization relationships as well.
Check out http://bucardo.org/wiki/Bucardo
You cannot get anything useful by copying individual tables files in the data directory. If you want to replicate selected tables, there are a number of good options.
http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling