I'm editting the config file of a mongos. And I have a replicaSet of n config servers. To balance the load among all of them, I have to write all config servers in the config file, or it's enough to add the master?
sharding:
configDB: <configReplSetName>/cfg1.example.net:27017, cfg2.example.net:27017,...
The documentation is not very clear about this. Thank you.
Edit - Possible solution: I've found this doc, where I interpret, that you should add all your config servers to your mongo's config file. Otherway, if the master config server cannot be accessed, the cluster won't be able to operate. This makes sense, since, in mongodb architecture, only mongos can act as load balancers. If someone can confirm this, I will vote him.
Related
I'm really new into learning MongoDB and I'm having a hard time trying to understand sharding.
So I have my PC who is the host and I have created two VMs using VirtualBox. In my PC (host) there is a DB with some data. So my issue is which of those 3 components should be the the Config Server, the Shard and the Query Router (mongo). Can somebody please help me explaining this ? (I have read the documentation and still haven't understand it completely).
A sharded cluster needs to run at least three processes, a "decent" sharded cluster runs at least 10 processes (1 mongos Router, 3 Config servers, 2 x 3 Shard servers). In production, they run on different machines, but it is no problem to run them all on the same machine.
I would suggest these steps for learning:
Deploy a Stand alone MongoDB
Deploy a Replica Set, see Deploy a Replica Set or Deploy Replica Set With Keyfile Authentication
Deploy a Sharded Cluster, see Deploy a Sharded Cluster or Deploy Sharded Cluster with Keyfile Authentication
In the beginning it is an overkill to use different machines. Use localhost for all.
You can run multiple mongod/mongos instances on one machine, just ensure they are using different ports (e.g. the default port 27017), different dbPath and preferable also different log files.
You may run your first trial without authentication. Once you got that working, use authentication with keyfile and as next step use authentication by x.509 certificates.
If you use Windows, then have a look at https://github.com/Wernfried/mongoDB-oneclick
Not able to connect to mongo standalone node experiencing below error.
ERROR [cluster-ClusterId{value='asdfs', description='null'}-x.x.x.x:27017] org.mongodb.driver.cluster - Expecting replica set member, but found a STANDALONE. Removing x.x.x.x:27017 from client view of cluster.
Is it okay to give multiple Ip's in config file while only one mongo node is there?
Is it okay to give multiple Ip's in config file while only one mongo node is there?
Not for standalone connections, no.
Specify the address of your standalone only.
I created a mongodb replicaset (using 3.2) and on each server, I set up MongoDB as a service
"C:\Program Files\MongoDB\Server\3.2\bin\mongod.exe" --config "C:\program files\mongodb\server\3.2\mongod.cfg" --service
So far so good. I recently set up users and need to set up MongoDB so that a user has to be supplied. From what I've read, I would start mongodb with the --auth parameter. However, since the service is already created, is there an equivalent in the config file? Based on the Configuration File Options Documentions, I have tried security.authorization set to enable
security:
authorization: enabled
But when I restarted the service on both servers, it appears that neither could talk to each other. I also tried
setParameter:
auth
but MongoDB wouldn't start up with that configuration.
What's the right way to do this?
Since you are using a replica set, merely setting security.authorization is not sufficient as you need to allow for cluster members to authenticate which is referred to as "Internal Authentication" in the docs.
The easiest way to do this is using a keyfile which is essentially a shared secret/password among the cluster members. After you've created your keyfile and copied it to all of the replica set members, you'll need to specify it's location in your config via the security.keyFile setting or using --keyFile.
For reference, you may also want to read Enforce Keyfile Access Control on Existing Replica Set for more detail on these steps.
I'm newbie to Zookeeper.Trying to setup clustering server for zookeeper exhibitor to modify the data. I have tried the server setup with 3 nodes, but data modification not reflected on all the zookeeper.
I refereed the following url & also setup the server in the same way. But no use, some thing i'm missing in that config to run it correctly.
http://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#sc_zkMulitServerSetup
Exhibitor startup command is:
java -jar exhibitor-war-1.0-jar-with-dependencies.jar -c file --nodemodification true --port 9090
Farther I need to add any other config with this to get my data modification reflect on all the zookeeper.
Advance thanks for ur kind time!
I have similar scenario, and I'm on Windows. So I get some problems due to the fact that Exhibitor is Unix-oriented - it tries to restart zkServer.sh (instead of zkServer.bat). So I:
1. have manually started ZK ensemble (all instances get data modifications from each other).
2. set up Exhibitor above every ZK instance - with single network config file.
Hope it helps. If not, give more details.
After reading the official documentation for the MongoDB sharding architecture I have not found out why you need to have one or three config servers, and not another number.
The MongoDB documentation on Config Servers says:
"If one or two config servers become unavailable, the cluster’s metadata becomes read only. You can still read and write data from the shards, but no chunk migrations or splits will occur until all three servers are available."
Hence the reflection: one server is equivalent to a single point of failure, but with two servers we have the same behavior as three, right?
So, why absolutely three servers and not only two or more, in example?
Because the doc says also:
Config servers do not run as replica sets.
Config Server Protocols
MongoDB 3.0 and earlier only support a single type of config server deployment protocol which is referred to as the legacy SCCC (Sync Cluster Connection Configuration) as of MongoDB 3.2. An SCCC deployment has either 1 config server (development only) or 3 config servers (production).
MongoDB 3.2 deprecates the SCCC protocol and supports a new deployment type: Config Servers as Replica Sets (CSRS). A CSRS deployment has the same limits as a standard replica set, which can have 1 config server (development only) or up to 50 servers (production) as at MongoDB 3.2. A minimum of 3 CSRS servers is recommended for high availability in a production deployment, but additional servers may be useful for geographically distributed deployments.
SCCC (Sync Cluster Connection Configuration)
With SCCC, the config servers are updated using a two-phase commit protocol which requires consensus from multiple servers for a transaction. You can use a single config server for testing/development purposes, but in production usage you should always have 3. A practical answer for why you cannot use only 2 (or more than 3) servers in MongoDB is that the MongoDB code base only supports 1 or 3 config servers for an SCCC configuration.
Three servers provide a stronger guarantee of consistency than two servers, and allows for maintenance activity (for example, backups) on one config server while still having two servers available for your mongos to query. More than three servers would increase the time required to commit data across all servers.
The metadata for your sharded cluster needs to be identical across all config servers, and is maintained by the MongoDB sharding implementation. The metadata includes the essential details of which shards currently hold ranges of documents (aka chunks). In a SCCC configuration, config servers are not a replica set, so if one or more config servers are offline then the config data will be read only -- otherwise there is no means for the data to propagate to the offline config servers when they are back online.
Clearly 1 config server provides no redundancy or backup. With 2 config servers, a potential failure scenario is where the servers are available but the data on the servers does not agree (for example, one of the servers had some data corruption). With 3 config servers you can improve on the previous scenario: 2/3 servers might be consistent and you could identify the odd server out.
CSRS (Config Servers as Replica Sets)
MongoDB 3.2 deprecates the use of three mirrored mongod instances for config servers, and starting in 3.2 config servers are (by default) deployed as a replica set. Replica set config servers must use the WiredTiger 3.2+ storage engine (or another storage engine that supports the new readConcern read isolation semantics). CSRS also disallows some non-default replica set configuration options (e.g. arbiterOnly, buildIndexes, and slaveDelay) that are unsuitable for the sharded cluster metadata use case.
The CSRS deployment improves consistency and availability for config servers, since MongoDB can take advantage of the standard replica set read and write protocols for sharding config data. In addition, this allows a sharded cluster to have more than 3 config servers since a replica set can have up to 50 members (as at MongoDB 3.2).
With a CSRS deployment, write availability depends on maintaining a quorum of members that can see the current primary for a replica set. For example, a 3-node replica set would require 2 available members to maintain a primary. Additional members can be added for improved fault tolerance, subject to the same election rules as a normal replica set. A readConcern of majority is used by mongos to ensure that sharded cluster metadata can only be read once it is committed to a majority of replica set members and a readPreference of nearest is used to route requests to the nearest config server.