I have 5 users which uses 5 different servers(using openerp), each one uses the same database copy. Whenever the user enters data to his database, then it should sync to the 6th server's database an then sync the data in the 6th database to all other 4 users databases. So any data entry in any database should sync to the other databases running in different servers. Is that possible? how can I achieve it?
Edit:
I found Bucardo, but it need primary key for every table. But in openerp, there are many2many relation tables which doesn't have a primary key.
This is called master-master or multi-master replication:
http://en.wikipedia.org/wiki/Multi-master_replication#PostgreSQL
http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling
You can also check out SymmetricDS or Daffodil which let you sync over HTTP.
A pure PostgreSQL installation supports currently only single master replication.
However you may achieve multi-master replication by installing additional tools http://wiki.postgresql.org/wiki/Clustering.
You should also check out Pg documentation on this topic Chapter 25. High Availability, Load Balancing, and Replication
Related
I need to create a replica of existing database, that would copy any changing operation from master to slave, I.e create a mirror some sort of. I found a lot of examples in web but they all describes process when master and slave are on different servers.
I would like to create a write replica on the same server where master is located , without spinning up second instance of Postgres.
Is it possible to do so and could you point me a direction where I could find a solution how to do it?
Thank you.
P.S. I understand that replication on 2 servers is better, but I just need to do it on one common server.
If you want physical replication, you will need to run two instances of PostgreSQL. If they are on the same server machine, they will need to have different port numbers. The different port numbers is the only complexity, otherwise it is just like running on two different servers.
If you want logical replication, you can do that within a single instance, but you will need to jump through some hoops to create the subscription intra-instance, as described in the "Notes" section
You could consider using a simple trigger to insert/update/delete data on the other database as soon as the main one get modified.
A more "professional" way would be to use synchronous replication.
Currently we are building replication of the primary db with hotstand by referring the link
https://cloud.google.com/community/tutorials/setting-up-postgres-hot-standby
Is there a way to replicate only a single table from the primary database?
Only with logical replication but I don't know if this is possible on GCP.
Cloud SQL Postgresql currently doesn’t support logical replication. There is a feature request that is publicly viewable here which you can up-vote with the +1 and star to get any updates.
I'd like to preface this by saying I'm not a DBA, so sorry for any gaps in technical knowledge.
I am working within a microservices architecture, where we have about a dozen or applications, each supported by its Postgres database instance (which is in RDS, if that helps). Each of the microservices' databases contains a few tables. It's safe to assume that there's no naming conflicts across any of the schemas/tables, and that there's no sharding of any data across the databases.
One of the issues we keep running into is wanting to analyze/join data across the databases. Right now, we're relying on a 3rd Party tool that caches our data and makes it possible to query across multiple database sources (via the shared cache).
Is it possible to create read-replicas of the schemas/tables from all of our production databases and have them available to query in a single database?
Are there any other ways to configure Postgres or RDS to make joining across our databases possible?
Is it possible to create read-replicas of the schemas/tables from all of our production databases and have them available to query in a single database?
Yes, that's possible and it's actually quite easy.
Setup one Postgres server that acts as the master.
For each remote server, create a foreign server then you then use to create a foreign table that makes the data accessible from the master server.
If you have multiple tables in multiple server that should be viewed as a single table in the master, you can setup inheritance to make all those tables appear like one. If you can define a "sharding" key that identifies a distinct attribute between those server, you can even make Postgres request the data only from the specific server.
All foreign tables can be joined as if they were local tables. Depending on the kind of query, some (or a lot) of the filter and join criteria can even be pushed down to the remote server to distribute the work.
As the Postgres Foreign Data Wrapper is writeable, you can even update the remote tables from the master server.
If the remote access and joins is too slow, you can create materialized views based on the remote tables to create a local copy of the data. This however means that it's not a real time copy and you have to manage the regular refresh of the tables.
Other (more complicated) options are the BDR project or pglogical. It seems that logical replication will be built into the next Postgres version (to be released a the end of this year).
Or you could use a distributed, shared-nothing system like Postgres-XL (which probably is the most complicated system to setup and maintain)
I want all tables to be replicated by bucardo (at least for a given database), but it looks like I have to add them all manually:
bucardo_ctl add all tables
Can I have it so that every table in a database is replicated, or added to bucardo automatically?
If not, is there another replication strategy in Postgresql that might be better fit for me? I'm hoping to have all nodes available for reads/writes, to avoid administering any routing process to route the writes to the master. If the routing of writes can be done natively in Postgresql, then that could be a solution as well.
Use below command for add all
bucardo add all tables
What would be the best way to replicate individual DB tables from a Master postgresql server to a slave machine? It can be done with cron+rsync, or with whatever postgresql might have build in, or some sort of OSS tool, but so far the postgres docs don't seem to cover how to do table replication. I'm not able to do a full DB replication because some tables have license->IP stuff connected to it, and I can't replicate those on the slave machine. I don't need instant replication, hourly would be acceptable as well.
If I need to just rsync, can someone help identify what files within the /var/lib/pgsql directory would need to be synced, or how I would know what tables they are.
Starting with Postgres 10, logical replication is built into Postgres! This is often a better solution than external solutions. The Postgres docs are great and easy to follow. It's very easy. See the quick setup docs, which in essense boils down to running this:
-- On publisher DB
CREATE PUBLICATION mypub FOR TABLE users, departments;
-- On subscriber DB
CREATE SUBSCRIPTION mysub CONNECTION 'dbname=foo host=bar user=repuser' PUBLICATION mypub;
You might want to try Bucardo, which is an open source software to synchronize rows between tables even if they are in a remote location. It's a very simple software, and it is capable of creating one-way synchronization relationships as well.
Check out http://bucardo.org/wiki/Bucardo
You cannot get anything useful by copying individual tables files in the data directory. If you want to replicate selected tables, there are a number of good options.
http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling