Does it work like master-slave way, where xmpp clients connect to a master node, and master node uses the slave nodes to distribute the load?
If not, how load balancing can be done after clustering of ejabberd nodes?
All nodes are equal and there is no master. State is kept in mnesia or mysql (like roster table, session etc.). Configuration is replicated over all nodes.
Usually it means there is LB in front of the whole cluster. One cluster is represented by one domain. You can have more and federate them.
Related
Imagine a scenario where I have 3 classes of worker node (A,B,C) and 2 master nodes (X,Y) as part of a Kubernetes cluster. There maybe multiple worker nodes of each class. Is it possible to route the traffic such that traffic arriving at the different master nodes is routed to a different set of worker nodes. For example I want master node X to route traffic to workers of class A and potentially fail over to class B under heavy load and for master node Y to route traffic to nodes of class C but also use B as a failover when needed.
As correctly pointed out in the comments, the traffic is not handled by the master nodes; its responsibility, at a high level, is to observe and maintain the desired state of the Kubernetes resources deployed in the worker nodes, as well as to push any changes to them.
That being said, yes, it is possible to discriminate and send the traffic to different worker nodes using Topology Aware Hints:
Topology Aware Hints enable topology aware routing by including
suggestions for how clients should consume endpoints. This approach
adds metadata to enable consumers of EndpointSlice and / or Endpoints
objects, so that traffic to those network endpoints can be routed
closer to where it originated.
This feature is specifically designed to handle traffic in a multi-zone environment and has a series of built-in Safeguards to prevent situations like overload of an endpoint, insufficient endpoints in a zone, etc.
In HA kubernetes clusters, we configure multiple control planes(master nodes), but how does multiple control planes sync their data? When we create a pod using kubectl command, the request went through the cloud load balancer to one of the control plane. I want to understand how other control planes sync their data with the one that got the new request?
First of all, please note that API Server is the only component that directly talks with the etcd.
Every change made on the Kubernetes cluster ( e.g. kubectl create) will create appropriate entry in etcd database and everything you will get from a kubectl get command is stored in etcd.
In this article you can find detailed explanation of communication between API Server and etcd.
Etcd uses RAFT protocol for leader election and that leader handles all client requests which need cluster consensus ( requests that do not require consensus can be processed by any cluster member ):
etcd is built on the Raft consensus algorithm to ensure data store consistency across all nodes in a cluster—table stakes for a fault-tolerant distributed system.
Raft achieves this consistency via an elected leader node that manages replication for the other nodes in the cluster, called followers. The leader accepts requests from the clients, which it then forwards to follower nodes. Once the leader has ascertained that a majority of follower nodes have stored each new request as a log entry, it applies the entry to its local state machine and returns the result of that execution—a ‘write’—to the client. If followers crash or network packets are lost, the leader retries until all followers have stored all log entries consistently.
More information about etcd and raft consensus algorithm can be found in this documentation.
To my knowledge, etcd uses Raft as a consensus and leader selection algorithm to maintain a leader that is in charge of keeping the ensemble of etcd nodes in sync with data changes within the etcd cluster. Among other things, this allows etcd to recover from node failures in the cluster where etcd runs.
But what about etcd managing other clusters, i.e. clusters other than the one where etcd runs?
For example, say we have an etcd cluster and separately, a DB (e.g. MySQL or Redis) cluster comprised of master (read and write) node/s and (read-only) replicas. Can etcd manage node roles for this other cluster?
More specifically:
Can etcd elect a leader for clusters other than the one running etcd and make that information available to other clusters and nodes?
To make this more concrete, using the example above, say a master node in the MySQL DB cluster mentioned in the above example goes down. Note again, that the master and replicas for the MySQL DB are running on a different cluster from the nodes running and hosting etcd data.
Does etcd provide capabilities to detect this type of node failures on clusters other than etcd's automatically? If yes, how is this done in practice? (e.g. MySQL DB or any other cluster where nodes can take on different roles).
After detecting such failure, can etcd re-arrange node roles in this separate cluster (i.e. designate new master and replicas), and would it use the Raft leader selection algorithm for this as well?
Once it has done so, can etcd also notify client (application) nodes that depend on this DB and configuration accordingly?
Finally, does any of the above require Kubernetes? Or can etcd manage external clusters all by its own?
In case it helps, here's a similar question for Zookeper.
etcd's master election is strictly for electing a leader for etcd itself.
Other clusters, however can use a distributed strongly-consistent key-value store (such as etcd) to implement their own failure detection, leader election and to allow clients of that cluster to respond.
Etcd doesn't manage clusters other than its own. It's not magic awesome sauce.
If you want to use etcd to manage a mysql cluster, you will need a component which manages the mysql nodes and stores cluster state in etcd. Clients can watch for changes in that cluster state and adjust.
We have 2 kubernetes clusters hosted on different data centers and we're deploying the applications to both these clusters. We have an external load balancer which is outside the clusters but the the load balancer only accepts static IPs. We don't have control over the clusters and we can't provision a static IP. How can we go about this?
We've also tried kong as an api gateway. We were able to create an upstream with targets as load balanced application endpoints and providing different weights but this doesn't give us active/passive or active/failover. Is there a way we can configure kong/nginx upstream to achieve this?
Consider using HA proxy, where you can configure your passive cluster as backup upstream, and you will get active/passive cluster working. As mentioned in this nice guide about HA proxy
backup meaning it won’t participate in the load balance unless both
the nodes above have failed their health check (more on that later).
This configuration is referred to as active-passive since the backup
node is just sitting there passively doing nothing. This enables you
to economize by having the same backup system for different
application servers.
Hope it helps!
here is example of clustered-durable-subscription and here is clustered-static-discovery, In clustered-static-discovery connecting with only one server (cluster auto connected with another server using cluster configuration).
As per doc
Normally durable subscriptions exist on a single node and can only
have one subscriber at any one time, however, with ActiveMQ Artemis
it's possible to create durable subscription instances with the same
name and client-id on different nodes of the cluster, and consume from
them simultaneously. This allows the work of processing messages from
a durable subscription to be spread across the cluster in a similar
way to how JMS Queues can be load balanced across the cluster
Should i need to add additional config for static cluster, or durable-subscription will work fine with static cluster without set the client id and subscription for all node of(As i have mentioned in static cluster we only make connection with one node)
The "static" part of the "clustered-static-discovery" really only refers to cluster node discovery (as the name suggests). Once the cluster nodes are discovered and the cluster is formed then the cluster will behave the same as if the discovery were dynamic (e.g. using UDP multicast). In other words, a clustered durable subscription should work the same no matter what mechanism was used on the server-side for cluster node discovery.