Has anyone had much experience using londiste? It is an alternative to slony for postgres replication. I have been beating my head against the wall trying to get slony to work the way I need it and was looking for any easier way.
londiste seems like a good alternative, but I wanted to see if anyone has any pros/cons before I commit to a switch.
I have used both and for my requirements Londiste is a good option.
We have a simple set up where a subset of tables is replicated from a staging server to live by large batch updates and insert and also intraday smaller updates running on postgres 8.4 and Centos 5.5 and skytools 2 and we also use it as the queue component for event based actions. Previously I have used Slony from the 1.* series so I can't comment on more recent versions.
Some Pros for Londiste
Simple to set up
Generally simple to administer
Haven't had any issues with robustness of replication in 8 months of production use
Also can be used as a generic queing system outside of replication and it is quite simple to write your own consumer
Some Cons
Documentation is pretty scant
You need to be careful when implementing ddl changes
It won't stop you from making changes in the slave
Can't be used for cascading replication or failover/switchover use case
I will limit my comment on Slony to my experience that it was complex to set up and administer and the version I used did not compare favourably on tolerance to network issues with Londiste but could have been used for cascading replication and switchover use cases.
As mentioned before, Londiste is simpler to use, indeed. And as of version 3, released in March 2012, Londiste supports cascading replication and failover/switchover, as well as a bunch of other new cool features.
Related
Is Postgres-XL fully backwards compatible with PostgreSQL? If so, what extra setup is required to take advantage of the automatic sharding?
Is Postgres-XL production-ready or will it be merged with core?
Not sure what you mean by backwards compatible. You can not take Postgres-XL tables and use them in Postgres if that is what you are asking. It does use the standard Postgres engine for the data nodes so you can use the standard Postgres drivers and tools with it, you just have to connect to the coordinator nodes.
While it is possible it could some day be merged in I would not count on it, and even it is that will be a REALLY long way off.
We use PostgreSQL 9.3 in our application. We want to setup PostgreSQL active-active clustering with DRBD. I google it and see a lot of resource about active-passive.
Does PostgreSQL support Active-Active Clustering with DRBD?
No, PostgreSQL does not support active/active clustering with DRBD.
PostgreSQL does not support any form of shared-storage clustering in any way - active/active, active/passive, or otherwise.
It's rather implausible to support shared storage clustering with the architecture in PostgreSQL. Lots of things would need to change. In particular, Pg couldn't lazily write buffers to disk anymore, which would be brutal for performance.
You'll need to use replication. You can use read-replicas (with a few limitations) that way.
There's no support for multi-master, nor is there any support for auto-relaying write queries to the master from a replica. Some people use PgPool-II for routing queries, though it also has some significant limitations.
(I'm involved in work on bi-directional replication, which seeks to offer another alternative, but it's very much alpha. See BDR on the PostgreSQL wiki)
Help me to chouse a simple (lightweight) solution to the master-slave replication of one table between two Postgresql databases. The table contains a large object.
Here you'll find a very good overview of the replication tools for PostgreSQL. Please, take a look and hopefully you'll be able to pick one.
Otherwise, if you need something really lightweight, you can do it yourself. You'll need a trigger and a couple of functions, a dblink module if you need almost immediate changes propagation, otherwise you can survive with cron.
I have an application that can not afford to lose data, so Postgresql is my choice for database (ACID)
However, speed and query advantages of MongoDB are very attractive, but based on what I've read so far, MongoDB can report a successful write which may not have gone to disk, so I can't make it my mission critical db (I'll also need transactions)
I've seen references to people using mysql and MongoDB together, one for the transactions and the other for queries. Please not that I'm not talking about keeping some data in one DB and the rest in another. I want to use Postgresql as a gateway to data entry, and MongoDB for reads.
Are there any resources that offer an architecture/guide for Postgresql + MongoDB usage in this way? I can remember seeing this topic in Postgresql conference agenda, but I could not find the link.
I don't think you'll get much speed using MongoDB just as a cache. It's strengths are replication and horizontal scalability. On one computer you'd make Mongo and Postgres compete for memory, IO bandwidth and processor time.
As you can not afford to loose transactions you'll be better with Postgres only. Its has efficient caching, sophisticated query planner, prepared queries and wide indexing support cause that read-only queries will be very fast - really comparable to MongoDB on a single computer.
Postgres can even scale horizontally now using asynchronous, or, from version 9.1, synchronous replication.
One way to achieve this would be to set up a master-slave replication with the PostgreSQL database as master, and the MongoDB database as slave. You would then do all reads from MongoDB, and all writes to PostgreSQL.
This post discusses such a setup using a tool called Bucardo:
http://blog.endpoint.com/2011/06/mongodb-replication-from-postgres-using.html
You may also be able to do it with Tungsten Replicator, although it seems designed to be used with MySQL:
http://code.google.com/p/tungsten-replicator/wiki/TRCHeterogeneousReplication
I can remember seeing this topic in Postgresql conference agenda, but I could not find the
link.
Maybe, you are talking about this: https://www.postgresqlconference.org/content/hybrid-applications-using-mongodb-and-postgres
Depending how important transactions are to you, one option is to use MongoDb driver's safe mode and drop Postgresql.
http://www.mongodb.org/display/DOCS/getLastError+Command
How can you expect transactional consistency from Postgres but trust MongoDB for reads? How would you support rollbacks in this scenario? How do you detect when they've gotten out of sync?
I think you're better off going with memcache and implementing a higher level object cache. Alternatively, you could consider a replication slave for reads. If you have performance needs beyond what a dedicated read slave can provide, consider denormalizing your tables on your slave system.
Make sure that any of this is actually needed. For thin tables with PK lookups most modern database engines like Postgres or InnoDB are going to generally keep up with NoSQL solutions. Don't fall into the ROFLSCALE trap
http://www.youtube.com/watch?v=b2F-DItXtZs
I think you can run a mongo replica set.. Let say 3 Slave and 1 Master.. Then in your app you should run all write transactions on Postgresql and then on Mongo ReplicaSet.. After that you can query read operations on Mongo Replica set..
But Synchronizing will be a problem, you should work on it..
you may find some replacement for mongo in here or here that is safer and fast as well.
but I advise to simplify your solution instead of making a complicated design.
Visual Guide to NoSQL Systems
lucky
In mongodb we can specify writeConcern property to specify that it should write to journal/ instances and then send confirmation/ acknowledgement and i think even mongodb has teh concept of transactions. Not sure why we need postgres behind it.
Is there any an easy (built-in, add-on, open-source or commercial) to do replication on Postgresql (Master-slave) to have the data inside the slave be scrubbed for PCI compliance while being replicated across? How about ETL tools? It does not have to be instantaneous ... up to an hour lag is acceptable but the faster the better of course.
If this doesn't work, how about possibly using triggers on the slave database to achieve this?
Perhaps you should try creating a view of the tables you wish to scrub (performing your scrubbing in the SELECT), and then replicate the view to your offsite location.
I believe triggers on the slave would put you at risk for non-compliance, since data could leak out. If you want a packaged solution, I'd probably look at Bucardo, looking specifically into doing custom replication hooks into slave, to filter out (or modify) the columns you don't need/want. If that won't work, the idea to use views is probably your next best bet.
Yes. Use slony, add triggers to the master to materialize what you want to replicate and replicate only those materialized views. If you scrub on the master, that should do what you want. Since Slony will happily replicate only part of your database, that should work fine (on the other hand, remember, Slony will happily replicate only part of your database).