i want to build a 3 nodes (avoid split brain) symmetric cluster with high availability using replication. In addition I would like to be able to load balanced messages between nodes
how should this be achieved?
option 1: 1 master with 2 slaves
option 2: 3 colocated master/slave
nodes
Option 1 isn't really an option as the slaves will not participate in the voting process which means split-brain will not be mitigated. The only option you have left (of the 2 you listed, of course) is to use 3 colocated master/slaves.
Related
I've already setup MongoDB sharding and now I need to setup replication for availability. How do I do this? I've currently got this:
2 mongos instances running in different datacenters
2 mongod config servers running in different datacenters
2 mongod shard servers running in different datacenters
all communication is over a private network setup by my provider that is available cross-datacenter
Do I just setup replication on each server (by assigning each a secondary)?
I would build the whole system in 3 DCs', for redundancy.
Every data center would have three servers with services of:
1x mongoS at Server1
1x node of config server replica set at Server1
1x node of shard1 replica set at Server2
1x node of shard2 replica set at Server3
So, a total of 9 nodes (physical or virtual).
If we "lose" one DC, everything works still, because we have a majority in all three replica sets.
You need 3 servers in each replica set for redundancy. Either put the third one in one of the data centers or get a third data center.
The config replica set needs 3 servers.
Each of the shard replica sets needs 3 servers.
You can keep the 2 mongoses.
After reading through the suggestions from D. SM and JJussi (thanks by the way), I'll be implementing the following infrastructure:
3 monogs instances spaced across different datacenters
3 config servers spaced across different datacenters
2 shards with 2 storage servers spaced across different datacenters with an arbiter each (to cut down on costs for now) each
Thanks once again for your input.
I am learning kubernetes by following the official documentation and in the Creating Highly Available clusters with kubeadm part it's recommended to use 3 masters and 3 workers as a minimum required to set a HA cluster.
This recommendation is given with no explanation about the reasons behind it. In other words, why a 2 masters and 2 workers configuration is not ok with HA ?
You want an uneven number of master eligible nodes, so you can form a proper quorum (two out of three, three out of five). The total number of nodes doesn't actually matter. Smaller installation often make the same node master eligible and data holding at the same time, so in that case you'd prefer an uneven number of nodes. Once you move to a setup with dedicated master eligible nodes, you're freed from that restriction. You could also run 4 nodes with a quorum of 3, but that will make the cluster unavailable if any two nodes die. The worst setup is 2 nodes since you can only safely run with a quorum of 2, so if a node dies you're unavailable.
(This was an answer from here which I think is a good explanation)
This why exactly:
https://etcd.io/docs/v3.3/faq/#why-an-odd-number-of-cluster-members
check then the concept of quorum, you can find plenty of info especially in the pacemaker/corosync documentation
I have one RabbitMQ cluster which consists of 3 nodes (each node has : Rabbitmq version :3.3.5, Erlang R14B0 ). Let's call this as Group1. Each node has n number of queues.
I have another cluster with set of 3 nodes with specifications as RabbitMQ 3.6.6, Erlang R16B03-1.Let's call this as Group2.
I need to replicate the queue structure of Group1 (node 1,node 2,node 3) exactly for Group2(node 1,node 2,node 3).
Is there any easy way to perform this without having to add each queue explicitly. Please suggest some measures.
Thank you.
Regards,
Shweta
We want to create a MongoDB shard (v. 2.4). The official documentation recommends to have 3 config servers.
However, the policies of our company won't allow us to get 3 extra servers for this purpose. Since we have already 3 application servers (1 web node, 2 process nodes) we are considering to put the configuration servers in the same application servers, with the mongos. Availability is not critical for us.
What do you think about this configuration? Can we face some problem or is it discouraged for some reason?
Given that Availability is not critical for your use case, I would say it should be fine to place the config servers in the same application servers and mongos.
If one of the process nodes is down, you will lose: 1 x mongos, 1 application server and 1 config server. During this down time, the other two config servers will be read-only , which means there won't be balancing of shards, modification to cluster config etc. Although your other two mongos should still be operational (CRUD wise). If your web-node is down, then you have a bigger problem to deal with.
If two of the nodes are down (2 process nodes, or 1 web server and process node), again, you would have bigger problem to deal with. i.e. Your applications are probably not going to work anyway.
Having said that, please consider the capacity of these nodes to be able to handle a mongos, an application server and a config server. i.e. CPU, RAM, network connections, etc.
I would recommend to test the deployment architecture in a development/staging cluster first under your typical workload and use case.
Also see Sharded Cluster High Availability for more info.
Lastly, I would recommend to check out MongoDB v3.2 which is the current stable release. The config servers in v3.2 are modelled as a replica set, see Sharded Cluster config servers for more info.
The question might seem ridiculous but it seems to me that a "yes" would be a little crazy.
MongoDB suggests to have replication sets of 3 machines. So if the database can stand on 1 computer, I need 3 machines, and if tomorrow I need to shard and need 2 machines I will actually need 6, right ?
Or is there something smarter that can be done and that comes for free with mongoDB ? (with coding theory like Hamming, ... the number of extra bits that we need is not linear in the size of the total number of bits)
Please don't hesitate to ask me to reformulate if what I say is not clear
Thanks in advance for your answers,
Thomas
So there is some really good documentation which is the recommended cluster setup in terms of phisycal instance separation. There should be considered two things (at least) separately. One is replication and for this one see this documentation : http://docs.mongodb.org/manual/core/replica-set-members/
Which means you have to have at least two data nodes (due to HA) in a replicaset and can have one arbiter which is not holding data just participate in election as it is described in the docs linked above. You need an odd number of setmembers due to the primary has to be elected by a majority inside the replicaset.
The other aspect is sharding. Sharding needs some additional metadata maintaining layer which is achived through additional processes these are configuration servers and mongos routers. For sharded production cluster see : http://docs.mongodb.org/manual/core/sharded-cluster-architectures-production/. In this setup the three configservers have to be on separated instances. Also the two mongos processes cannot reside on the same instance.
So for the minimal alignment. Have to be considered :
You must not collocate data nodes (each two datanodes in each shard have to be on a separated instance)
The arbiter node belonging to a specific shards replicaset have to be on a separated instance from the two datanodes
The three configservers should reside on separated instances from each other
The minimal two mongos processes have to reside on separated nodes from each other
However datanodes cannot be collocated, configservers and mongos processes can be on the same instances as the datanodes.
So theoretically one can align a sharded cluster without braking any of the recomendations on 4 instances with two shards like this:
Instance 1:
datanode replicaset 1, configserver 1, arbiter replicaset 2
Instance 2:
datanode replicaset 1, configserver 2, mongos 1
Instance 3:
datanode replicaset 2, configserver 3, arbiter replicaset 1
Instance 4:
datanode replicaset 2, mongos 2
Where replicaset 1 represents the first shard and replicaset 2 represents the second.
datanode is not a terminology which is used for mongoDB in general just i am likely to address with this name those mongod process which are handling real data, so the (Primaries and secondaries in a replicaset).
Just as a sidenote i would not do this. Just start micro instances for the configservers and keep mongos processes on the application servers.