In a regular sharded replica cluster, it consists of 10 mongos, 5 config servers and 10 shards. I use mongo client to connect to multiple mongos instances.
I have two questions.
The first question: What is load policy in this situation? Is it round-robin scheduing?
The second one: What if one onf the mongos instances is down, what is the move that mongoclient would take? Will it still connect to this mongos instance or drop this one from the list.
Please help with these.thanks
The mongos servers provide a routing service to direct read/write queries to the appropriate shard(s).
You are specifying multiple mongos's to connect to the MongoDB sharded cluster. An available mongos will be used to connect to the server.
The first question: What is load policy in this situation? Is it
round-robin scheduling?
The client will connect to the server with an available mongos. There is no "load policy" and there is no round-robin scheduling. You use multiple mongos's for high availability.
See: Number of mongos and Distribution
The second one: What if one of the mongos instances is down, what is
the move that mongoclient would take? Will it still connect to this
mongos instance or drop this one from the list.
If a mongos is down, the client will connect to the server using another available mongos from the list (you have more than one mongos to connect with).
Related
I have a mongo cluster of 1 Main routing mongoserver, with 3 shards server en 3 config servers. Each shard have a primary mongo database, a secundairy mongo database and an arbiter mongo database.
The shards are redundant (Primary mongoserver and Replication mongoserver).
The Config servers are redundant. (3 config server)
Only the main routing mongoserver is not redundant.
So this is a single point of failure.
Is there a way within mongo to expend this routing mongoserver, so that there is a failover/redundant functionality?
Regards,
If your "routing mongoserver" is mongos, you can add as many of them as you like. Typical setup is one per app server, collocated with the app server to reduce network latency.
I have a mongos to route my queries to two different mongo clusters running on two different ec2 instances so that if one ec2 instance goes down, i have a backup.
The challenge is, where should I put my mongos query router? I do not want to put my mongos query router on a 3rd EC2 instance upstream, because EC2 instances can fail and break. I've had this happen to me. Ec2 instances do not recover on their own and spin themselves up again right?. If the ec2 instance that my mongos query router is on goes down, then all the redundancy upstream that is built for high-availability becomes irrelevant.
So is there another amazon service (like an ec2) that is small, and would only be dedicated to one server (a mongos query distributor), that can spin itself up again if it goes down due to hardware failures, or auto-grow its own RAM and disk-space to give the mongos query router more resources due to software consuming system resources?
Looks like ec2 instances can auto-recover now via
https://aws.amazon.com/blogs/aws/new-auto-recovery-for-amazon-ec2/
So having a mongos query router on a mini ec2 instance with auto-scaling and auto-recovery should be safe.
Also, though not an out-of-box solution for ec2 instances, it looks like you can now also scale up the RAM and disk-size of your ec2 instance by using custom cloud-watch alarms to trigger these actions via
http://aws.amazon.com/code/8720044071969977
Can we have this type of configuration?
Two server running the following things each-
1.Mongo Config Server.
2.Mongo Router.
3.Application.
Total 4 EC2 servers-
First Server-Running the web application & mongos.
Second Server-Running the web application & mongos.
Third Server-Running the First Shard with complete DB(Say for
example Demo).
Forth Server-Running The Second Shard with complete DB(Say for
example Demo).
Both the Mongos should point to one shard named Shard1?
Yes, You can have multiple mongos instances running against a single shard. Think of the mongos instances as clients for the sharded cluster which have to run as a daemon process in order to keep metadata and heartbeats up to date.
Edit: as for having a complete DB, this is only possible for a single DB. You can have one DB on shard1 and the other DB on shard2, for example. but you can never have a single complete DB on two shards. To achieve the goal of having db1 on shard1 and db2 on shard2, you simply make the respective shard the primary shard of the respective database and don't shard any collection. Please read the docs for the movePrimary command for details.
A bit OOT:
However, running a single config server is strongly advised against, and for a good reason. If the single config server goes down or gets corrupted, your cluster will be impossible to use - and recreating the sharded cluster will not an easy task to be done. And it's going to be a lengty process. So please, use three config servers.*
I want to make continuous backup of my Sharded Cluster on a single MongoDB server somewhere else.
So, is it possible to create Replica Set with Sharded Cluster (mongos instance) and single MongoDB server?
Did anyone experience creating Replica Sets with two Sharded Clusters or with one Sharded Cluster and one Single Server?
How does it work?
By the way, the best (and for now, the only) way to continuously backup Sharded Cluster is by using MongoDB Management Service (MMS).
I were also facing the same issue sometime back. I wanted to take replica of all sharded cluster into one MongoDB. But, i didn't found any solution for this scenario, and I think this is true, because -
If you configure the multiple shard server (say 2 shard server) with
one replica set, then this will not work because in a replica set (say
rs0) only 1-primary member is possible. And In this scenario, we will
have multiple primary depend on number of shard server.
To take the backup of your whole sharded cluster, you must stop all writes to the cluster. You can refer to MongoDB documentation on this - http://docs.mongodb.org/manual/tutorial/backup-sharded-cluster-with-database-dumps/
I'm trying to figure out how different instances of mongos server work together.
If I have 1 configserver and some shards, for example four, each of them composed by only one node (a master of course), and have four mongos server... do the mongos server communicate between them? Is it possible that one mongos redirect its load to another mongos?
When you have multiple mongos instances, they do not automatically load-balance between each other. They don't even know about each others existence.
The MongoDB drivers for most programming languages allow to specify multiple mongos instances when creating a connection. In that case the driver will usually ping all of them and connect to the one with the lowest latency. This will usually be the one which is closest geographically. When all have the same network distance, the one which is least busy right now will usually respond first. The driver will then stay connected to that one mongos, unless the program explicitely reconnects or the mongos can no longer be reached (in that case the driver will usually automatically pick another one from the initial list).
That means using multiple mongos instances is normally only a valid method for scaling when you have a large number of low-load clients, not one high-load client. When you want your one high-load client to make use of many mongos instances, you need to implement this yourself by creating a separate connection to each mongos instance and implement your own mechanism to distribute queries among them.
Short answer
As of MongoDB 2.4, the mongos servers only provide a routing service to direct read/write queries to the appropriate shard(s). The mongos servers discover the configuration for your sharded cluster via the config servers. You can find out more details in the MongoDB documentation: Sharded Cluster Query Routing.
Longer scoop
I'm trying to figure out how different instances of mongos server work togheter.
The mongos servers do not currently talk directly to each other. They do coordinate activity some activity via your config servers:
reading the sharded cluster metadata
initiating a balancing round (any mongos can start a balancing round, but only one round can be active at a time)
If I have 1 configserver
You should always have 3 config servers in production. If you somehow lose or corrupt your config server, you will have to combine your data and re-shard your database(s). The sharded cluster metadata saved on the config servers is the definitive source for what sharded data ranges should live on each shard.
some shards, for example four, each of them composed by only one node (a master of course)
Ideally each shard should be backed by a replica set if you want optimal uptime. Replica sets provide for auto-failover and can be very useful for administrative purposes (for example, taking backups or adding indexes offline).
Is it possible that one mongos redirect its load to another mongos?
No, the mongos do not perform any load balancing. The typical recommendation is to deploy one mongos per app server.
From an application/driver point of view you can specify multiple mongos in your connect string for failover purposes. The application drivers will generally connect to the nearest available mongos (by network ping time), and attempt to reconnect to in the event the current mongos connection fails.