I have pgpool-II for HA and repmgr for automatic failover. Pgpool-II also can also run failover I am just wondering doing automatic failover using pgpool or repmgr? If pgpool can do the failover do I need to use repmgr? and use shell scripts to the promotion of new master?
I want to do the automatic failover via repmgrd and use pgpool only as entry point for the applications. It almost works: I have set the failover_command in pgpool configuration file to an empty string (''). When the primary database fails pgpool executes the failover_command (which does nothing because failover_command is empty), and then it will repeatedly try to find a new master. When repmgrd has done the failover, then pgpool see the new master and it is fine.
If the master is stopped, repmgrd will do the failover after x seconds (depending on reconnect interval and attemps parameters). You have to make sure that the health check interval specified in pgpool is bigger than that otherwise pgpool will detach the stopped database too early : the parameters are health_chek_max_retries and health_check_retry_delay (there is also health_check_period which is not clear). You must also set failover_on_backend_error to no.
I am not very confident in this set-up though, I still need to test it more intensively.
pgpool-II: Connection Pooling, Load Balancing, Automated failover, Limiting Exceeding Connections
In pgpool virtual IP is their application connect to VIP( a delegated_ip parameter on pgpool. conf file) primary goes down then failover happens standby promote to primary ( automatic failover )then there are no-changes on application end.
failover_command to write a script then configure the failover.sh script location on pgpool.conf file failover_command="" parameter
repmgr: replication and failover
In Repmgr If the primary is stopped, it will do the failover after xx seconds (depending on reconnecting interval connect_timeout parameter on /etc/repmgr.conf
Related
I have a postgresql 10 master db with 2 hot standby servers with streaming replication, and the replication is working correctly. The synchronous_commit is setted to remote_write
Also I have a pgpool 3.7.5 configured with the params:
delay_threshold = 1
sr_check_period = 1
And the the following weights:
master: 1
node1: 3
node2: 3
In the log I can see the node1 and node2 are lagging:
Replication of node:1 is behind 75016 bytes from the primary server (node:0)
The pgpool docs says:
delay_threshold (integer)
Specifies the maximum tolerance level of replication delay in WAL bytes on the standby server against the primary server. If the delay exceeds this configured level, Pgpool-II stops sending the SELECT queries to the standby server and starts routing everything to the primary server even if load_balance_mode is enabled, until the standby catches-up with the primary. Setting this parameter to 0 disables the delay checking. This delay threshold check is performed every sr_check_period. Default is 0.
The problem it's that pgpool sends queries to the hot standbys before they obtained the new data from master through streaming replication.
I enabled the log_per_node_statement = on temporally to be able to see which node the query executes and I can see that queries are sent to the nodes even if there aren't sync when delay_threshold should avoid that.
Am I missing something? When the nodes are behind master the queries are not supposed to go the master?
Thanks in advance.
Other config values of pgpool are:
num_init_children = 120
max_pool = 3
connection_cache = off
load_balance_mode = on
master_slave_sub_mode = 'stream'
replication_mode = off
sr_check_period = 1
first, I think you should check the result of "show pool_nodes" and check if three nodes are properly set with right role (primary, standby, standby).
second, did you set "app_name_redirect_preference_list" or "database_redirect_preference_list" ? If so, That can affect on selecting the node for SELECT query.
And in my opinion, I think delay_threshold = 1 is strict, the unit is bytes and in my case, I use "10000000" on PROD. why don't you just put "/NO LOAD BALANCE/" comment to send specific queries to only master?
And I simply recommend you to upgrade the version of pgpool to 4.0.0 (2018-10-19 released). 3.7.x has mysterious bug on load balancing.
I also faced a similar problem that load balancing is not working properly with the version (3.7.5) even when our configuration has no problem. The pgpool randomly We even contact pgpool developer team to solve this problem but they couldn't find the root cause.
You can check the details in the link below.
https://www.pgpool.net/mantisbt/view.php?id=435.
And this was resolved like charm by upgrading to version 4.0.0.
I'm searching for HA solutions without load balancing in the master-slave model, using postgresql. My favorite solution so far is log shipping synchronous replication. But I have one main concern, and that is, if my slave server becomes unavailable, will my master server continue it's operation? Or will it wait for the acknowledgment of my slave server until it's up again?
If you have only one standby, the master will halt ( by design ).
The master will still serve read-only statements, but all writes will be blocked until the standby comes back.
You can avoid this scenario by providing multiple candidates in synchronous_standby_names.
See SYNCHRONOUS-REPLICATION in the PostgreSQL Docs.
I found another way to prevent the master halt at slave crash. We can use wal_sender_timeout in masters postgresql.conf file to disconnect from the slave if it's been crashed.
I am running two databases (PostgreSQL 9.5.7) in a master/slave setup. My application is connecting to a pgpool instance which routes to the master database (and slave for read only queries).
Now I am trying to scale out some data to another read-only database instance containing only a few tables.
This works perfectly using pglogical directly on the master database.
However if the master transitions to slave for some reason, pglogical can't replicate any longer because the node is in standby.
Tried following things:
subscribed on the slave since it's less likely to go down, or overheated: Can't replicate on standby node.
subscribed via pgpool server: pgpool doesn't accept replication connections.
subscribed to both servers: pglogical config gets replicated along, so can't give them different node names.
The only thing I can think of now is to write my own tcp proxy which regularly checks for the state of the server to which I can subscribe to.
Is there any other/easier way I can solve this ?
Am I using the wrong tools perhaps ?
Ok so it seems that there are no solutions for this problem just yet.
Since the data in my logically replicated database is not changing fast, there is no harm if the replication stops for a moment.
Actions on failover could be:
Re-subscribe to the promoted master.
or promote standby node back to master after failover.
I am new in using pgbouncer 1.7 and I want to configure it with master slave configuration.
I have configured postgres 9.3 streaming replication using repmgr and I want to use pgbouncer for load balancing and connection pooling so that it automatically switches to slave if master goes down . So how should I configure it for the same. I have both master and slave on diff servers and og bouncer on diff servers. Do i need to install pgbouncer on both master and slave servers also for it to work or just installing on a diff server will work .
I have tried many online tutorial for it but sadly didnt found any suggestions. Please if anyone can help.
Thanks in advance,
Mohit
PgBouncer does not have automatic failover, propagation and ex-master rebuild handling. You can change IP for same hostname to failover though:
https://pgbouncer.github.io/faq.html
How to failover
PgBouncer does not have internal failover-host configuration nor detection. It is possible via some external tools:
DNS reconfiguration - when ip behind DNS name is reconfigured, pgbouncer will reconnect to new server. This behaviour can be tuned
via 2 config parameters - dns_max_ttl tunes lifetime for one hostname,
and dns_zone_check_period tunes how often zone SOA will be queried for
changes. If zone SOA record has changed, pgbouncer will re-query all
hostnames under that zone.
Write new host to config and let PgBouncer reload it - send SIGHUP or use RELOAD; command on console. PgBouncer will detect changed host
config and reconnect to new server.
Pgpool has automatic failover if you wnat to try.
PostgreSql 9.1 has master-slave synchronous replication. Suppose the master is machine A and the slave is machine B.
If the master fails, how does PostgreSQL know when to make the slave the master? What if the slave incorrectly thought the master was down because of a temporary network glitch on the master where the client program could still contact the master though.
And moreover, how would my client program know the slave in the new master and more importantly is ready to accept writes. Does the slave send a message to the client?
Check repmgr, it's one of its jobs is to deal with this issue.
Typically you want to use a promotion-management system like repmgr or patroni. Then you want to use some sort of a high availability proxy (could be pgbouncer or haproxy) to handle the actual abstraction so your applications do not need to know what system is master.
In answer to your question, most of these systems use a heartbeat to determine if there is a problem. Patroni goes out over the etcd heartbeat. Repmgr has its own heartbeat check. With Repmgr you need to write hook scripts to take care of stonith, and so forth.