Sharding in MongoDB - mongodb

I try to test sharding in MongoDB. For example, I use host1.com and host2.com instead real server names.
So I created config server at host1.com:
mongod --dbpath /path/to/configdb/ --configsvr
Started mongos at the same machine:
mongos --configdb host1.com --port 27020
And started mongod at two machines (host1.com and host2.com):
mongod --dbpath /path/to/test_shard_db/ --shardsvr
I added shards, enabled sharding for database test and collection test with shard key {'name': 1} (collection has only this field and _id for test) as explained in tutorial . But after all this operations all my data writes only to one shard, which is primary for database.
Here is config:
Sharding status:
mongos> db.printShardingStatus()
--- Sharding Status ---
sharding version: { "_id" : 1, "version" : 3 }
shards:
{ "_id" : "shard0000", "host" : "host1.com:27018", "maxSize" : NumberLong(1) }
{ "_id" : "shard0001", "host" : "host2.com:27018", "maxSize" : NumberLong(10) }
databases:
...
{ "_id" : "test", "partitioned" : true, "primary" : "shard0000" }
test.test chunks:
shard0001 1
{ "name" : { $minKey : 1 } } -->> { "name" : { $maxKey : 1 } } on : shard0001 Timestamp(1000, 0)
Collection stats:
mongos> db.printCollectionStats()
test
{
"sharded" : false,
"primary" : "shard0000",
"size" : 203535788,
...
}
Balancer status:
mongos> sh.isBalancerRunning()
true
So why all data in collection reside only at one shard though I added more than 1 megabyte of data? And why db.printCollectionStats() show me that test database "sharded" : false. What I did wrong?

The default chunk size is 64MB so you have room to grow before a split will occur. You can split the shard key range yourself beforehand which can allow writes to go to multiple shards from the start. See the MongoDB Split Chunks documentation for more info.
On the difference between chunk size and maxSize:
maxSize will limit the volume of data on a given shard. When reached the balancer will look to move chunks to a shard where maxSize has not been reached. A chunk is a collection of documents that all fall within a section of the shard key range. The MongoDB balancer will move data between shards at the chunk level to balance. When a chunk approaches the maxSize value, it will be split into 2 which may result in a move.

Related

When there is a mongod that already has data, data is not visible when configuring a shard

When there is a mongod that already has data, the data is not visible when configuring the shard
In detail
Two mongods have a collection with the same name and the following data is stored
Info(A,B mongod)
DB : db_test
Collection : test_log
A mongod
{ "_id" : ObjectId("5f44cfa372c25aeff48e66d3"), "skey" : 100, "data" : 1 }
B mongod
{ "_id" : ObjectId("5f44cf731eadcef49cd93e7b"), "skey" : 200, "data" : 2 }
In Mongos, I did the following command
mongos> sh.addShard("A.MASTER:21011")
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> sh.addShard("B.SLAVE1:21011")
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> sh.addShardTag("shard0000", "master")
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
mongos> sh.addShardTag("shard0001", "slave_1")
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
mongos> sh.enableSharding("db_test")
{ "ok" : 1 }
mongos> sh.shardCollection("db_test.test_log", {skey:1})
{ "collectionsharded" : "db_test.test_log", "ok" : 1 }
mongos> sh.addTagRange("db_test.test_log", {skey:100}, {skey:199}, "master")
WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : {
"ns" : "db_test.test_log",
"min" : {
"skey" : 100
}
}
})
mongos> sh.addTagRange("db_test.test_log", {skey:200}, {skey:299}, "slave_1")
WriteResult({
"nMatched" : 0,
"nUpserted" : 1,
"nModified" : 0,
"_id" : {
"ns" : "db_test.test_log",
"min" : {
"skey" : 200
}
}
})
And I looked up data in mongos ('find, aggregate ..' command was executed)
result
mongos> db.test_log.find()
{ "_id" : ObjectId("5f44cfa372c25aeff48e66d3"), "skey" : 100, "data" : 1 }
before sharding, I executed Command (create index (to use shardkey))
mongos> db.test_log.createIndex({ "skey": 1 });
{
"raw" : {
"A.MASTER:21011" : {
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"ok" : 1
}
},
"ok" : 1
}
I wonder what's the problem
When creating an index in mongos, I am also curious about the part allocated only to A.master.
Please..
The collection data should all reside on 1 shard before the collection is sharded.
When you run shardCollection any initial chunks are created on the database's primary shard.
Additional chunks are created by splitting an already existing chunk.
When the balancer moves a chunk from one shard to another, it first deletes the document range covered by that chunk from the target collection.
When querying via mongos, the router checks with the config servers to find out which shards hold chunks that might be needed for the query, and directs the query to only those shards.
Tags and tag ranges are consulted by the balancer to determine where to move chunks. These are not consulted by the mongos routers.
So what happened in your scenario is you had a database with a collection that existed on more than one mongod node, each with an index on {skey: 1}
You then created a sharded cluster by adding both of those mongod nodes as individual shards.
At this point, the config servers would not have had any information about the database or the collection.
When you ran enableSharding, the config servers created a config document for the database, and assigned one of the shards to be the database's primary shard.
At this point any query for any unsharded collection in that database would be handled by the primary shard.
When you ran shardCollection, the config servers created a document for the collection that identifies the shard key. It also created a single chunk from MinKey to MaxKey, identified as residing on the database's primary shard.
Your subsequent query for the data was serviced by the mongos, which consulted the chunk list for the collection, finding that there was only 1 chunk. It then forwarded that request to the owning shard, which returned the data it had. The other shard would have never been consulted.
You can check how the chunks are distributed by running sh.status(true).
At this point, splitting the existing chunk and letting it balance will not help, because the range will be cleared before moving the chunk. This means any documents on the other shard will be deleted before the empty chunk is moved.
There is no simple method to take 2 separate replica sets that already contain data and join them together in a sharded cluster. In that scenario, the simplest approach would be to mongodump the data from one replica set, create a sharded cluster with the other, after enabling sharding on all databses, then add the remaining replica set as a second shard, and then mongorestore the remaining data.
For a single replica set that contains data, there is a documented procedure to Convert a Replica Set to a Sharded Cluster

Is there a primary shard in DBs in which sh.enableSharding() has not been yet executed?

MongoDB sharding cluster uses a "primary shard" to hold collection data in DBs in which sharding has been enabled (with sh.enableSharding()) but the collection itself has not been yet enabled (with sh.shardCollection()). The mongos process choses automatically the primary shard, except if the user state it explicitly as parameter of sh.enableSharding()
However, what happens in DBs where sh.enableSharding() has not been executed yet? Is there some "global primary" for these cases? How can I know which one it is? sh.status() doesn't show information about it...
I'm using MongoDB 4.2 version.
Thanks!
The documentation says:
The mongos selects the primary shard when creating a new database by picking the shard in the cluster that has the least amount of data.
If enableSharding is called on a database which already exists, the above quote would define the location of the database prior to sharding being enabled on it.
sh.status() shows where the database is stored:
MongoDB Enterprise mongos> use foo
switched to db foo
MongoDB Enterprise mongos> db.foo.insert({a:1})
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5eade78756d7ba8d40fc4317")
}
shards:
{ "_id" : "shard01", "host" : "shard01/localhost:14442,localhost:14443", "state" : 1 }
{ "_id" : "shard02", "host" : "shard02/localhost:14444,localhost:14445", "state" : 1 }
active mongoses:
"4.3.6" : 2
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
{ "_id" : "foo", "primary" : "shard02", "partitioned" : false, "version" : { "uuid" : UUID("ff618243-f4b9-4607-8f79-3075d14d737d"), "lastMod" : 1 } }
{ "_id" : "test", "primary" : "shard01", "partitioned" : false, "version" : { "uuid" : UUID("4d76cf84-4697-4e8c-82f8-a0cfad87be80"), "lastMod" : 1 } }
foo is not partitioned and stored in shard02.
If enableSharding is called on a database which doesn't yet exist, the database is created and, the primary shard is specified, the specified shard is used as the primary shard. Test code here.

Sharding in a replicaset MongoDB

I have mongoDb replica set , One primary one secondary and an arbiter to vote. I'm planning to implement sharding as the data is expected to grow exponentially.I find difficult in following mongoDb document for sharding. Could someone explain it clearly to set it up. Thanks in advance.
If you could do accomplish replicaset, sharding is pretty simple. Pretty much repeating the mongo documentation in fast forward here:
Below is for a sample setup: 3 configDB and 3 shards
For the below example you can run all of it one machine to see it all working.
If you need three shards setup three replica sets. (Assuming 3 Primary's are 127:0.0.1:27000, 127.0.0.1:37000, 127.0.0.1:47000)
Run 3 instances mongod as three config servers. (Assuming: 127.0.0.1:27020, 127.0.0.1:27021, 127.0.0.1:270122)
Start mongos (note the s in mongos) letting it know where your config servers are. (ex: 127.0.0.1:27023)
Connect to mongos from mongo shell and add the three primary mongod's of your 3 replica sets as the shards.
Enable sharding for your DB.
If required enable sharding for a collection.
Select a shard key if required. (Very Important you do it right the first time!!!)
Check the shard status
Pump data; connect to individual mongod primarys and see the data distributed across the three shards.
#start mongos with three configs:
mongos --port 27023 --configdb localhost:27017,localhost:27018,localhost:27019
mongos> sh.addShard("127.0.0.1:27000");
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> sh.addShard("127.0.0.1:37000");
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> sh.addShard("127.0.0.1:47000");
{ "shardAdded" : "shard0002", "ok" : 1 }
mongos> sh.enableSharding("db_to_shard");
{ "ok" : 1 }
mongos> use db_to_shard;
switched to db db_to_shard
mongos>
mongos> sh.shardCollection("db_to_shard.coll_to_shard", {collId: 1, createdDate: 1} );
{ "collectionsharded" : "db_to_shard.coll_to_shard", "ok" : 1 }
mongos> show databases;
admin (empty)
config 0.063GB
db_to_shard 0.078GB
mongos> sh.status();
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("557003eb4a4e61bb2ea0555b")
}
shards:
{ "_id" : "shard0000", "host" : "127.0.0.1:27000" }
{ "_id" : "shard0001", "host" : "127.0.0.1:37000" }
{ "_id" : "shard0002", "host" : "127.0.0.1:47000" }
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "test", "partitioned" : false, "primary" : "shard0000" }
{ "_id" : "db_to_shard", "partitioned" : true, "primary" : "shard0000" }
db_to_shard.coll_to_shard
shard key: { "collId" : 1, "createdDate" : 1 }
chunks:
shard0000 1
{ "collId" : { "$minKey" : 1 }, "createdDate" : { "$minKey" : 1 } } -->> { "collId" : { "$maxKey" : 1 }, "createdDate" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0)

Mongodb sharded replica

I have 3 nodes for sharding and configserver (sharding servers run on standard port 27017 and configserver running on port 27019)
stage-mongo1-tmp, stage-mongo2-tmp, and stage-mongo3-tmp
and a query router
stage-query0-mongo
in my current setup.
Sharding is working perfect as expected.
--- Sharding Status ---
sharding version: {
"_id" : 1,
"version" : 3,
"minCompatibleVersion" : 3,
"currentVersion" : 4,
"clusterId" : ObjectId("5321a5cc8a18e5280f7c9d5a")
}
shards:
{ "_id" : "shard0000", "host" : "stage-mongo1-tmp:27017" }
{ "_id" : "shard0001", "host" : "stage-mongo2-tmp:27017" }
{ "_id" : "shard0002", "host" : "stage-mongo3-tmp:27017" }
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "testdb", "partitioned" : true, "primary" : "shard0000" }
testdb.testcollection
shard key: { "_id" : "hashed" }
chunks:
shard0001 28
shard0002 31
shard0000 28
too many chunks to print, use verbose if you want to force print
Now, I was enabling replica set on these nodes. I logged in to stage-mongo1-tmp, and ran
rs.initiate()
and added stage-mongo2-tmp and stage-mongo3-tmp as the replica members as
rs.add("stage-mongo2-tmp")
Log files says replication enabled and elected one primary.
rs.conf()
was showing good output
[rsBackgroundSync] replSet syncing to: stage-mongo1-tmp:27017
[rsSync] oplog sync 2 of 3
[rsSyncNotifier] replset setting oplog notifier to stage-mongo1-tmp:27017
[rsSync] replSet initial sync building indexes
[rsSync] replSet initial sync cloning indexes for : ekl_omniscient
[rsSync] build index ekl_omniscient.serviceability { _id: "hashed" }
[rsSync] build index done. scanned 571242 total records. 3.894 secs
replSet RECOVERING
replSet initial sync done
replSet SECONDARY
However, when I test the High-availability by taking one node down, mongos on the query node is returning error saying
mongos> show dbs;
Thu Mar 13 20:17:04.394 listDatabases failed:{
"code" : 11002,
"ok" : 0,
"errmsg" : "exception: socket exception [CONNECT_ERROR] for stage-mongo1-tmp:27017"} at src/mongo/shell/mongo.js:46
When I connect to one of the other node, one has automatically elected as primary. But still, my queries are returning errors.
What am I doing wrong here in replica set? Why is it not high-available? Do I need to add more servers to make it high-available? I am looking for a minimum set of servers to implement this.
Figured out. We add shard for the replica set.
sh.addShard("rs0/:port,..)
Once this is done, we need to enable sharding on db level and collection level. This would enable sharding and replica.

MongoDB Sharding Policy

I am trying to understand the following behavior displayed by my sharding setup. The data seems to only increase on a single shard as I continuously add data. How does MongoDB shard or distribute data across different servers? Am I doing this correctly? MongoDB version 2.4.1 used here on OS X 10.5.
As requested, sh.status() as follows:
mongos> sh.status()
sharding version: {
"_id" : 1,
"version" : 3,
"minCompatibleVersion" : 3,
"currentVersion" : 4,
"clusterId" : ObjectId("52787cc2c10fcbb58607b07f") }
shards:
{ "_id" : "shard0000", "host" : "xx.xx.xx.xxx:xxxxx" }
{ "_id" : "shard0001", "host" : "xx.xx.xx.xxx:xxxxx" }
{ "_id" : "shard0002", "host" : "xx.xx.xx.xxx:xxxxx" }
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "newdb", "partitioned" : true, "primary" : "shard0001" }
newdb.prov
shard key: { "_id" : 1, "jobID" : 1, "user" : 1 }
chunks:
shard0000 43
shard0001 50
shard0002 43
Looks like you have chosen a very poor shard key. You partitioned along the values of { "_id" : 1, "jobID" : 1, "user" : 1 } - this will not be a good distribution for inserts since _id values are monotonically increasing since you are using ObjectId() values for _id.
You want to select a shard key that represents how you access the data - it doesn't make sense that you have two more fields after _id - since _id is unique the other two fields are never going to be used to partition the data.
Did you perhaps intend to shard on jobID, user? It's hard to know what the best shard key would be in your case, but it's clear that all the inserts are going into the highest chunk (top value through maxKey) since every new _id is a higher value than the previous one.
Eventually they should be balanced to other shards, but only if the balancer is running, if all your config servers are up and if secondaries are caught up. Best to pick a better shard key and have inserts be distributed evenly across the cluster from the start.