MongoDB error when sharding a collection: ns not found - mongodb

I've setup a simple server configuration for testing sharding functionnalities purpose and i get the error above.
My configuration is pretty simple: one config server, one shard server and one mongos (respectively in 127.0.0.1:27019, 127.0.0.1:27018, 127.0.0.1:27017).
Everything looks to work well until i try to shard a collection, the command gives me the following:
sh.shardCollection("test.test", { "test" : 1 } )
{
"ok" : 0,
"errmsg" : "ns not found",
"code" : 26,
"codeName" : "NamespaceNotFound",
"operationTime" : Timestamp(1590244259, 5),
"$clusterTime" : {
"clusterTime" : Timestamp(1590244259, 5),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
The config server and shard server outputs show no errors:
2020-05-23T10:39:46.629-0400 I SHARDING [conn11] about to log metadata event into changelog: { _id: "florent-Nitro-AN515-53:27018-2020-05-23T10:39:46.629-0400-5ec935b2bec982e313743b1a", server: "florent-Nitro-AN515-53:27018", shard: "rs0", clientAddr: "127.0.0.1:58242", time: new Date(1590244786629), what: "shardCollection.start", ns: "test.test", details: { shardKey: { test: 1.0 }, collection: "test.test", uuid: UUID("152add6f-e56b-40c4-954c-378920eceede"), empty: false, fromMapReduce: false, primary: "rs0:rs0/127.0.0.1:27018", numChunks: 1 } }
2020-05-23T10:39:46.620-0400 I SHARDING [conn25] distributed lock 'test' acquired for 'shardCollection', ts : 5ec935b235505bcc59eb60c5
2020-05-23T10:39:46.622-0400 I SHARDING [conn25] distributed lock 'test.test' acquired for 'shardCollection', ts : 5ec935b235505bcc59eb60c7
2020-05-23T10:39:46.637-0400 I SHARDING [conn25] distributed lock with ts: 5ec935b235505bcc59eb60c7' unlocked.
2020-05-23T10:39:46.640-0400 I SHARDING [conn25] distributed lock with ts: 5ec935b235505bcc59eb60c5' unlocked.
Of course the collection exists on primary shard:
rs0:PRIMARY> db.test.stats()
{
"ns" : "test.test",
"size" : 216,
"count" : 6,
"avgObjSize" : 36,
"storageSize" : 36864,
"capped" : false,
...
}
I have no idea what could be wrong here, i'd much appreciate any help :)
EDIT:
Here is the detail about steps i follom to run servers, i probably misunderstand something :
Config server:
sudo mongod --configsvr --replSet rs0 --port 27019 --dbpath /srv/mongodb/cfg
mongo --port 27019
Then in mongo shell
rs.initiate(
{
_id: "rs0",
configsvr: true,
members: [
{ _id : 0, host : "127.0.0.1:27019" }
]
}
)
Sharded server:
sudo mongod --shardsvr --replSet rs0 --dbpath /srv/mongodb/shrd1/ --port 27018
mongo --port 27018
Then in shell:
rs.initiate(
{
_id: "rs0",
members: [
{ _id : 0, host : "127.0.0.1:27018" }
]
}
)
db.test.createIndex({test:1})
Router:
sudo mongos --configdb rs0/127.0.0.1:27019
mongo
Then in shell:
sh.addShard('127.0.0.1:27018')
sh.enableSharding('test')
sh.shardCollection('test.test', {test:1})

That error happens sometimes when some routers have out of date ideas of what databases/collections exist in the sharded cluster.
Try running https://docs.mongodb.com/manual/reference/command/flushRouterConfig/ on each mongos (i.e. connect to each mongos sequentially by itself and run this command on it).

I just misunderstood one base concept: config servers and shard servers are distinct and independant mongodb instances, so each must be part of distinct replicasets .
So replacing
sudo mongod --configsvr --replSet rs0 --port 27019 --dbpath /srv/mongodb/cfg
with
sudo mongod --configsvr --replSet rs0Config --port 27019 --dbpath /srv/mongodb/cfg
makes the configuration work.

Related

Unable to initiate replica set while createing config server to deploy sharded cluster

I am trying to deploy a sharded cluster in MongoDB in mac. I am following this page. For creating a config server for sharded cluster I did the following steps
mkdir -p /data/config/config-a /data/config/config-b /data/config/config-c
mongod --logpath "cfg-a.log" --dbpath /data/config/config-a --port 57040 --fork --configsvr --smallfiles
mongod --logpath "cfg-b.log" --dbpath /data/config/config-b --port 57041 --fork --configsvr --smallfiles
mongod --logpath "cfg-c.log" --dbpath /data/config/config-c --port 57042 --fork --configsvr --smallfiles
and after this I tried initiating the replica set, as follows
$ mongo --port 57040
> config = { _id : "cs", members : [{ _id:0, host:"localhost:57040"}, { _id:1, host: "localhost:57041"}, { _id:2, host:"localhost:57042"}]};
{
"_id" : "cs",
"members" : [
{
"_id" : 0,
"host" : "localhost:57040"
},
{
"_id" : 1,
"host" : "localhost:57041"
},
{
"_id" : 2,
"host" : "localhost:57042"
}
]
> rs.initiate(config)
{
"ok" : 0,
"errmsg" : "This node was not started with the replSet option",
"code" : 76,
"codeName" : "NoReplicationEnabled"
}
Why am I getting this error ? I did not get any error while initiating other replica sets but I am getting error in this one. Could someone help me with this.
The guide you are following is outdated. MongoDB documentation has a tutorial that configures a sharded cluster. It gives both --replSet and --configsvr arguments:
mongod --configsvr --replSet <replica set name> --dbpath <path> --bind_ip localhost,<hostname(s)|ip address(es)>

Error while creating Replica set - MongoDb

I am trying to create replica but unable to proceed
Script to create 3 mongod instance :
sudo mkdir -p /data/rs1 /data/rs2 /data/rs3
sudo mongod --replSet rs1 --logpath "1.log" --dbpath /data/rs1 --port 27017 --fork
sudo mongod --replSet rs2 --logpath "2.log" --dbpath /data/rs2 --port 27018 --fork
sudo mongod --replSet rs3 --logpath "3.log" --dbpath /data/rs3 --port 27019 --fork
This executes successfully but after this i try to provide rs1 information about rs2 and rs3 via below script :
init_replica.js :
config = {
_id:"rs1",members:[
{_id:0,host:"grit-lenevo-pc:27017",priority:0,slaveDelay:5},
{_id:1,host:"grit-lenevo-pc:27018"},
{_id:2,host:"grit-lenevo-pc:27019"}]
}
rs.initiate(config)
rs.status()
Now when i try to run :
mongo --port 27018 < init_replica.js
I am getting :
MongoDB shell version: 3.2.8
connecting to: 127.0.0.1:27018/test
{
"_id" : "rs1",
"members" : [
{
"_id" : 0,
"host" : "grit-lenevo-pc:27017",
"priority" : 0,
"slaveDelay" : 5
},
{
"_id" : 1,
"host" : "grit-lenevo-pc:27018"
},
{
"_id" : 2,
"host" : "grit-lenevo-pc:27019"
}
]
}
{
"ok" : 0,
"errmsg" : "Attempting to initiate a replica set with name rs1, but command line reports rs2; rejecting",
"code" : 93
}
{
"info" : "run rs.initiate(...) if not yet done for the set",
"ok" : 0,
"errmsg" : "no replset config has been received",
"code" : 94
}
bye
Note : The same command works fine if i try below command :
mongo --port 27017 < init_replica.js
Following tutorials : M101 Mongo Db For Java Developers
It's right about there:
"Attempting to initiate a replica set with name rs1, but command line reports rs2; rejecting"
You should supply all members with the same replica set name as the seed (s1). For the second member:
sudo mongod --replSet rs1 ...
and not
sudo mongod --replSet rs2 ...
Same principal goes for third member
I had a similar name mismatch issue, but it was a bit more subtle.
In my mongo.conf I used "rs0" (quoted) for the RS name, and then ran rs.initiate({_id : "rs0"...}), which failed with
"Attempting to initiate a replica set with name rs0, but command line reports \"rs0\"; rejecting"
It took a while to notice the extra quotes - don't use them in the RS name in mongo.conf.

authentication failed from httpinterface and Robomongo

edit:
Ah bad news, Robomongo 0.8.x doesn't support SCRAM-SHA-1
https://github.com/paralect/robomongo/issues/766. Good news is that V0.9 they're working hard with promises support for it.
And also the http interface in Mongo 3.0 doesn't work with SCRAM-SHA-1 user documents, because "(it) is generally considered insecure".
https://jira.mongodb.org/browse/SERVER-17527
I've just set up a mongo3.0 replica set, and enabled authentication, and created an userAdminAnyDatabase admin and a normal readWrite user.
./mongod --dbpath=/usr/local/mongo/mongodb/data/data1 --logpath=/usr/local/mongo/mongodb/logs/log1/mongodb.log --port 27017 --replSet jv_mongo --smallfiles --fork --rest --httpinterface --keyFile /usr/local/mongo/mongodb/key/mongodb.pem
./mongod --dbpath=/usr/local/mongo/mongodb/data/data2 --logpath=/usr/local/mongo/mongodb/logs/log2/mongodb.log --port 27018 --replSet jv_mongo --smallfiles --fork --rest --httpinterface --keyFile /usr/local/mongo/mongodb/key/mongodb.pem
./mongod --dbpath=/usr/local/mongo/mongodb/data/data3 --logpath=/usr/local/mongo/mongodb/logs/log3/mongodb.log --port 27019 --replSet jv_mongo --smallfiles --fork --rest --httpinterface --keyFile /usr/local/mongo/mongodb/key/mongodb.pem
jv_mongo:PRIMARY> use admin
switched to db admin
jv_mongo:PRIMARY> db.getUser("mongoAdmin");
{
"_id" : "admin.mongoAdmin",
"user" : "mongoAdmin",
"db" : "admin",
"roles" : [
{
"role" : "userAdminAnyDatabase",
"db" : "admin"
}
]
}
jv_mongo:PRIMARY> use comment
switched to db comment
jv_mongo:PRIMARY> db.getUser("comment");
{
"_id" : "comment.comment",
"user" : "comment",
"db" : "comment",
"roles" : [
{
"role" : "readWrite",
"db" : "comment"
}
]
}
And access the shell without any problem.
./mongo --port 27017 -u mongoAdmin -p PASSWORD --authenticationDatabase admin
./mongo --port 27017 -u comment -p PASSWORD --authenticationDatabase comment
jv_mongo:PRIMARY> db.user_login.find();
{ "_id" : ObjectId("5506a9de41e1073435ff06b3"), "id" : NumberLong(2), "user_id" : 9527, "login_time" : ISODate("2015-03-16T10:01:02.378Z"), "login_ip" : "127.0.0.1" }
{ "_id" : ObjectId("5506a9de41e1073435ff06b4"), "id" : NumberLong(3), "user_id" : 9538, "login_time" : ISODate("2015-03-16T10:01:02.380Z"), "login_ip" : "127.0.0.1" }
{ "_id" : ObjectId("5506a9de41e1073435ff06b5"), "id" : NumberLong(4), "user_id" : 9549, "login_time" : ISODate("2015-03-16T10:01:02.382Z"), "login_ip" : "127.0.0.1" }
And also successfully accessed mongo via java driver
But I received auth fail when trying Robomongo or 192.168.106.152:28017.
I'm not very familiar with Mongo or Mongo3.0, maybe I'm missing some key configuration?
Use MongoChef, it will work for mongodb 3.0+

Mongodb Sharding not working - what is causing Collection not sharded

I am trying to set up mongodb sharding with two nodes. I have enabled 3 configuration process and a router process. I am extracting data (With 50 columns - 650 MB - _id as the key) from SQL server and putting in mongodb. In the pentaho configuration I have enabled "Use all Replica sets" and enter the primary node's host name and the port. When I run the transformation, all the data are getting into primary node and the other node is not getting data. When I entered, db.table.getShardDistribution(), I get the following message "Collection not sharded".
Also the status of is.BalancerRunning() gives me false status. I am very sure that background process balancer is not working here.
Mean while i tried to insert a sample test records 10,00,000 records with name as the key , the sharding setup was working fine and each shard got data distributed.
So,I am missing something or doing something wrong while I run pentaho transformation to populate data in mongodb. Any help is appreciated.
My set up
C:\Mongodb\bin\mongod.exe --shardsvr --port 10001 --dbpath C:\Mongodb\shard1 > C:\Mongodb\Log\shard1.log
C:\Mongodb\bin\mongod.exe --shardsvr --port 10002 --dbpath C:\Mongodb\shard2 > C:\Mongodb\Log\shard2.log
C:\Mongodb\bin\mongod.exe --configsvr --port 20000 --dbpath C:\Mongodb\configdb > C:\Mongodb\Log\config.log
C:\Mongodb\bin\mongos.exe --configdb 10.231.34.105:
--chunkSize 1 > C:\Mongodb\Log\mongos.log
mongos> use admin
switched to dbadmin
mongos> db.runCommand( { addshard : "10.231.34.105:40001" } );
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> db.runCommand( { addshard : "10.231.34.106:40002" } );
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> db.runCommand( { enablesharding : "dbTest" } );
{ "ok" : 1 }
mongos> db.runCommand( { shardcollection : "dbTest.cTest", key : { Date_ID: 1 } } );
{ "collectionssharded" : "dbTest.cTest", "ok" : 1 }
mongos> use dbTest;
db.cTest.ensureIndex({ Date_D : 1 });```
I am not sure from your post if you shard database and collections.
Once you setup shards did you enable sharding for database. like below
db.runCommand({
enablesharding : "dbname"
});
Do db.stats() to confirm.
After enable sharding for collections. like below.
db.runCommand({
shardcollection : "collection_name",
key : {
shardKey : "hashed"
}
});
Then to confirm db.collections.stats()

mongoDB sharding example

Newbie using mongo 2.0.1 32-bit on windows tried testing out shards as follows:
(4) processes: 2 shards + config srver + mongos w tiny chunksize
mongod.exe --shardsvr --port 10001 --dbpath <folder1> > shard1.log
mongod.exe --shardsvr --port 10002 --dbpath <folder2> > shard2.log
mongod.exe --configsvr --port 20000 --dbpath <configfolder> > config.log
mongos.exe --configdb localhost:20000 --chunkSize 1 > mongos.log
I ran the shell and set up 2 shards:
mongos> use admin
switched to dbadmin
mongos> db.runCommand( { addshard : "localhost:10001" } );
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> db.runCommand( { addshard : "localhost:10002" } );
{ "shardAdded" : "shard0001", "ok" : 1 }
Then I enabled sharding for a test database (dbTest) and collection (cTest):
mongos> db.runCommand( { enablesharding : "dbTest" } );
{ "ok" : 1 }
mongos> db.runCommand( { shardcollection : "dbTest.cTest", key : { Name : 1 } } );
{ "collectionssharded" : "dbTest.cTest", "ok" : 1 }
Finally I populated the cTest collection (indexed by Name) with 1,000,005 sample records:
mongos> use dbTest
switched to db dbTest
db.cTest.drop();
db.cTest.ensureIndex({ Name : 1 });
db.cTest.save({Name: "Frank", Age:56, Job: "Accountant", State: "NY"});
db.cTest.save({Name: "Bill" , Age:23, State: "CA"});
db.cTest.save({Name: "Janet", Age:34, Job: "Dancer" });
db.cTest.save({Name: "Andy", Age:44 });
db.cTest.save({Name: "Zach", Age:23, Job: "Fireman", State: "CA"});
i=1;
while(i<=1000)
{
j=1;
while (j<=1000)
{
db.cTest.save({Name:"Person("+i+","+j+")", Age:i+j});
j = j+1
};
i=i+1;
};
HOWEVER ...
It appears that nothing actually got sharded. In the config database, db.chunks.count() is zero, and I can see from windows explorer file sizes that all the data went into the physical file setup setup for the first shard, and none to the second.
Can anyone spot what I've done wrong, and also provide some tips on how to admin & debug this type of thing & see what's going on ?
Thanks
Once you "shardcollection", don't drop it. It will remove metadata about sharded collection.