I've setup streaming replication and wonder if it's synchrounous.. if it's blocking.
Which implys when slave goes down, the synchronous rep will be blocked and there will be problems service client requests.
Or do I need not to worry about such a scenario?
Yes, with synchronous replication a COMMIT will block until the required standby servers have reported that they have received the information.
That leads to reduced availability if you only have a single standby server. This is not a PostgreSQL shortcoming, but a fundamental necessity; read about the CAP theorem for more.
The way to deal with that is to have more than one standby server, so that life can go on if a standby server fails.
Related
I have Postgresql 14 and I made streaming replication (remote_apply) for 3 nodes.
When the two standby nodes are down, if I tried to do an insert command this will show up:
WARNING: canceling wait for synchronous replication due to user request
DETAIL: The transaction has already committed locally, but might not have been replicated to the standby.
INSERT 0 1
I don't want to insert it locally. I want to reject the transaction and show an error instead.
Is it possible to do that?
No, there is no way to do that with synchronous replication.
I don't think you have thought through the implications of what you want. If it doesn't commit locally first, then what should happen if the master crashes after sending the transaction to the replica, but before getting back word that it was applied there? If it was committed on the replica but rejected on the master, how would they ever get back into sync?
I made a script that checks the number of standby nodes and then make the primary node read-only if the standby nodes are down.
I had set up several NoSQL streams this morning. Around mid-day, there were issues with the network connectivicity in the environment where Oracle NoSQL Database was deployed. I saw some anomalies in my applications.
How does Oracle NoSQL DB Streams API handle failures like unstable network connection, master transfer, and rebalance at the server?
Please advise.
Thanks for trying Oracle NoSQL Database Streams API.
Oracle NoSQL Streams handles failures in different ways depending on the nature of the failure. In the case of unstable network connectivity, the Streams API would reconnect to the master node of each shard at the host and port cached in the Streams API, therefore, when network connectivity restores, the Streams API would reconnect successfully and resume streaming from the last streamed operation. If the attempts to reconnect fail after three times, the Streams API would refresh the topology from the Oracle NoSQL Database, and reconnect with the master node found in the latest store topology. This happens when the master transfers and the old master is no longer accessible.
Handling store rebalancing is similar to handling master transfer, the Streams API would pull the new topology from the store, and locate the new master of each shard to reconnect. After the rebalancing is complete and the new topology is ready, the Streams API would be able to reconnect and resume streaming.
The description above is for the latest version Oracle NoSQL Database 21.1. In previous versions, there is a limit that caps the number of reconnects in Stream API, and the Streams API will terminate the stream and signal NoSQLSubscriber.onError() if it runs out of the maximum attempts. In 21.1, such limit is removed by default and the Streams API would just keep reconnecting till the connection is restored. The users can override the default behavior by setting NoSQLSubscriptionConfig.Builder.setMaxReconnect(long maxReconnect)
Currently we are using postgres streaming replication to sync/replicate databases on Primary and replica server. We are planning to use one of the application to sync data from our secondary or replica server to our data warehouse which needs logical replication to be enabled for tracking the changes and syncing the data from our replica server to data warehouse. Can we enable logical replication on top of streaming replication ? Is it possible or good practice to enable both on the same server or database ? If so, will there be any performance impact or what are the considerations or best practices to be followed?
There is no problem with using streaming (physical) replication and logical replication at the same time, but a physical standby server cannot be a logical primary. So you will have to use the same server as primary server for both physical and logical replication. But that shouldn't be a problem, since streaming replication primary and standby are physically identical anyway.
Version : ActiveMQ Artemis 2.10.1
When we use ha-policy and replication, is the synchronization mode between the master and the slave full synchronization? Can we choose full synchronization or asynchronization?
I'm not 100% certain what you mean by "full synchronization" so I'll just explain how the brokers behave...
When a master broker receives a durable (i.e. persistent) message it will write the message to disk and send the message to the slave in parallel. The broker will then wait for the local disk write operation to complete as well as receive a response from the slave that it accepted the message before it responds to the client who originally sent the message.
This behavior is not configurable.
What is the consistency of Postgresql HA cluster with Patroni?
My understanding is that because the fail-over is using a consensus (etc or zookeeper) the system will stay consistent under network partition.
Does this mean that transaction running under the serializable Isolation Level will also provide linearizability.
If not which consistency will I get Sequential Consistency, Causal Consistency .. ?
You shouldn't mix up consistency between the primary and the replicas and consistency within the database.
A PostgreSQL database running in a Patroni cluster is a normal database with streaming replicas, so it provides the eventual consistency of streaming replication (all replicas will eventually show the same values as the primary).
Serializabiliy guarantees that you can establish an order in the database transactions that ran against the primary such that the outcome of a serialized execution in that order is the same as the workload had in reality.
If I read the definition right, that is just the same as “linearizability”.
Since only one of the nodes in the Patroni cluster can be written to (the primary), this stays true, no matter if the database is in a Patroni cluster or not.
In a distributed context, where we have multiple replicas of an object’s state, A schedule is linearizable if it is as if they were all updated at once at a single point in time.
Once a write completes, all later reads (wall-clock time) from any replica should see the value of that write or the value of a later write.
Since PostgreSQL version 9.6 its possible to have multiple synchronous standy node. This mean if we have 3 server and use num_sync = 2, the primary will always wait for write to be on the 2 standby before doing commit.
This should satisfy the constraint of linearizable schedule even with failover.
Since version 1.2 of Patroni, When synchronous mode is enabled, Patroni will automatically fail over only to a standby that was synchronously replicating at the time of the master failure.
This effectively means that no user visible transaction gets lost in such a case.