I have sharded and replicated Postgre database. I use CLUSTER + RUN way of running functions. I define target (master/replica) using CLUSTER param and shard using RUN ON. How can I make PL/Proxy run or not to run function on master, if originally target was replica, but it failed?
In PL/Proxy, you define shards via a libpq connection string. Now if a shard is replicated, you can simply use a connection string like
host=slave.myorg.com,master.myorg.com port=5432,5432 dbname=...
This will try to connect to the first host, and if that fails, it will fall back to the second host. PostgreSQL v14 has the additional connection string parameter target_session_attrs=prefer-standby to preferable connect to the standby server, even if it is not the first in the list.
Related
We are using mongodb cluster with set of 3 members , and all consumers are able to read/write data.
Use mongo.conf file to create cluster && configure replica set, which are executed in ansible script as part of system restart.
Our system does heavy writes/read operation on mongo clusters ,and sometime replica get broken.
Then it is no longer feasible to restore replica without deleting data from one of the corrupted server.
Here are my queries:
what is better way to notify when replica is broken using script/some code?We can not use Mongo atlas
etc.
What is efficient way to restore data on corrupted member.
What is better way to notify when replica is broken using script/some code?
Use the replSetGetStatus database command.
You can run this command from any driver or the shell. Among other things, it returns the state (primary/secondary/startup/recovery/unreachable) and last applied operation time for each member.
What is efficient way to restore data on corrupted member.
Check out the docs for a few options: https://docs.mongodb.com/manual/tutorial/resync-replica-set-member/#resync-a-member-of-a-replica-set
I've one primary host as mongo1.ppshein.net, secondary host as mongo1.ppshein.net and arbiter as mongo3.ppshein.net, and configured MongoDB replica as above shown in AWS EC2. And in each of MongoDB config file, bindIP is as its host name and App Server host.
To access that MongoDB replica from python, I thought I could be able to use following code-snippet,
>>> from pymongo import MongoClient
>>> db = MongoClient('mongodb://serverA:27017, serverB:27017, serverC:27017/?replicaSet=foo').db_name
But problem is if serverA is down/unhealthy, I'm not sure whether above code-snippet would be working properly or not. That's why I'm curious to know how to get primary host of MongoDB instead of adding multiple hosts in connection string?
Once you connect to a replica set the driver will always reconnect you to the primary if one can be elected (i.e. you have a quorum of nodes). the only reason you give a list of nodes in the argument is to prevent a situation where you are attempting to connect to a node that is down. if that node was down you would get a server timeout.
The full specification for the server discovery and monitoring protocol is here.
Currently I have 1 postgres instance which is starting to receive too much load and want create a cluster of 2 postgres nodes.
From reading the documentation for postgres and pgpool, it seems like I can only write to a master and read from a slave or run parallel queries.
What I'm looking for is a simple replication of a database but with master/slave based on which table is being updated. Is this possible? Am i missing it somewhere in the documentation?
e.g.
update users will be executed on server1 and replicated to server2
update big_table will be executed on server2 and replicated back to server1
What you are looking for is called MASTER/MASTER replication. This is supported natively (without PgPool) since 9.5. Note, that it's an "eventually consistent" architecture, so your application should be aware of possible temporary differences between the two servers.
See PG documentation for more details and setup instructions.
I know the way to set up a Master/Slave DB in Postgres is having 2 DB servers, but unfortunately i have only one server for now.
How can i mirror my production db into another "backup db" in "real_time"? I want to give access to another user to the mirrored db, so even if he does something there it will not affect production.
Nothing stops you setting up hot standby streaming replication, or another replication option like Londiste, between two PostgreSQL instances on the same computer.
The two copies of PostgreSQL must use different ports, but that's the only real restriction.
How to set up the second PostgreSQL instance depends on your operating system and how you installed PostgreSQL, which you have not mentioned.
You'll want to use streaming replication with hot standby if you want a read-only replica. If you want it to be read/write, then you can do a one-off copy of the database with pg_basebackup and not keep them in sync after that. Or you can use a tool like Londiste to replicate changes selectively.
You can run multiple instances of PostgreSQL on the same computer, by using different ports.
I have a server that I want to use for testing new app verson (say staging server), but at same time I want to use it as replication slave for MongoDB. So, there is two roles:
always replicate an database to this server (only one database, original, with real data)
after deployment, make a copy of original db, to a new one (*-staging db), and test my deployment against this database
I see from docs how to replicate only specified database from one server to another, seems that it's working fine. But the problem that when i've tried to make a copy of existing database, on slave server, it fails with error not master. I don't want to make this database copy on master server, because it means that all staging tests will be executed against master server, that doesn't work for me.
Does it mean that I can't have MongoDB master for one database, and slave for another?
Slaves by default are read only but you can achieve what you are trying to do by making it master and slave at the same time by passing both --master and --slave when starting your server:
mongod --slave --source master:1234 --master