I know there is a ShardingTest() object that can be used to create a testing sharding environment (see https://serverfault.com/questions/590576/installing-multiple-mongodb-versions-on-the-same-server), eg:
mongo --nodb
cluster = new ShardingTest({shards : 3, rs : false})
However, given that the disk space in my testing machine is limited and I'm getting "Insufficient free space for journal files" errors when using the above command, I'd like to set the smallfiles option. I have tried with the following with no luck:
cluster = new ShardingTest({shards : 3, rs : false, smallfiles: true})
How smallfiles can be enabled for a sharding test, please? Thanks!
A good way to determine how to use a MongoDB shell command is to type the command without the parentheses into the shell and instead of running it will print the source code for the command. So if you run
ShardingTest
at the command prompt you will see all of the source code. Around line 30 you'll see this comment:
// Allow specifying options like :
// { mongos : [ { noprealloc : "" } ], config : [ { smallfiles : "" } ], shards : { rs : true, d : true } }
which gives you the correct syntax to pass configuration parameters for mongos, config and shards (which apply to the non replicaset mongods for all the shards). That is, instead of specifying a number for shards you pass in an object. Digging further in the code:
else if( isObject( numShards ) ){
tempCount = 0;
for( var i in numShards ) {
otherParams[ i ] = numShards[i];
tempCount++;
}
numShards = tempCount;
This will take an object and use the subdocuments within the object as option parameters for each shard. This leads to, using your example:
cluster = new ShardingTest({shards : {d0:{smallfiles:''}, d1:{smallfiles:''}, d2:{smallfiles:''}}})
which from the output I can see is starting the shards with --smallfiles:
shell: started program mongod --port 30000 --dbpath /data/db/test0 --smallfiles --setParameter enableTestCommands=1
shell: started program mongod --port 30001 --dbpath /data/db/test1 --smallfiles --setParameter enableTestCommands=1
shell: started program mongod --port 30002 --dbpath /data/db/test2 --smallfiles --setParameter enableTestCommands=1
Alternatively, since you now have the source code in front of you, you could modify the javascript to pass in smallfiles by default.
A thorough explanation of the invoking modes of ShardingTest() is to be found in the source code of the function itself.
E.g., you could set smallFiles for two shards as follows:
cluster = new ShardingTest({shards: {d0:{smallfiles:''}, d1:{smallfiles:''}}})
Related
I'm following the tutorial in the MongoDB: The Definitive Guide, 2nd edition, for a databases course, and it appears not to work in version 3.6.2.
Basically I have two mongo shells opened using mongo --nodb.
Then, in the first, I run cluster = new ShardingTest({"shards": 3, "chunksize": 1}) (which works and produces a steady stream of output).
In the second shell, the book says to run db = (new Mongo("localhost:30999")).getDB("test") which fails. I was told by a colleague instead to run db = (new Mongo("localhost:20000")).getDB("test"), which worked.
Then, I inserted data which worked as well. However, when trying sh.status(), I got the message printShardingStatus: this db does not have sharding enabled. be sure you are connecting to a mongos from the shell and not to a mongod.
After searching online, I figured I'd run sh.enableSharding(db) which also gave me the following error:
2018-03-01T11:05:22.654-0500 E QUERY [thread1] Error: not connected to a mongos :
sh._checkMongos#src/mongo/shell/utils_sh.js:8:15
sh._adminCommand#src/mongo/shell/utils_sh.js:18:9
sh.enableSharding#src/mongo/shell/utils_sh.js:98:12
#(shell):1:1
I'm running on a Windows 10 machine, and have the correct environmental variables set up and the db folder created, so any help/pointers would be much appreciated!
EDIT 1:
This error persists even if db.collection.ensureIndex() is run first.
Try connecting to the shell on port 20006 by opening a new mongo --nodb, then do db = (new Mongo("localhost:20006")).getDB("test")
This should open the mongos for all the shards, so now the command sh.status() should work, as well as other commands like setting balancer state and starting balancer.
The commands bellow show how to run 3 shards instances and 3 config instances in your localhost. For each of these shards also are created 3 replica sets (mongod instances), may it can help you:
clean everything up
echo "killing mongod and mongos"
killall mongod
killall mongos
echo "removing data files"
rm -rf /data/config
rm -rf /data/shard*
start a replica set and tell it that it will be shard0
echo "starting servers for shard 0"
mkdir -p /data/shard0/rs0 /data/shard0/rs1 /data/shard0/rs2
mongod --replSet s0 --logpath "s0-r0.log" --dbpath /data/shard0/rs0 --port 37017 --fork --shardsvr
mongod --replSet s0 --logpath "s0-r1.log" --dbpath /data/shard0/rs1 --port 37018 --fork --shardsvr
mongod --replSet s0 --logpath "s0-r2.log" --dbpath /data/shard0/rs2 --port 37019 --fork --shardsvr
sleep 5
connect to one server and initiate the set
echo "Configuring s0 replica set"
mongo --port 37017 << 'EOF'
config = { _id: "s0", members:[
{ _id : 0, host : "localhost:37017" },
{ _id : 1, host : "localhost:37018" },
{ _id : 2, host : "localhost:37019" }]};
rs.initiate(config)
EOF
start a replicate set and tell it that it will be a shard1
echo "starting servers for shard 1"
mkdir -p /data/shard1/rs0 /data/shard1/rs1 /data/shard1/rs2
mongod --replSet s1 --logpath "s1-r0.log" --dbpath /data/shard1/rs0 -port 47017 --fork --shardsvr
mongod --replSet s1 --logpath "s1-r1.log" --dbpath /data/shard1/rs1 --port 47018 --fork --shardsvr
mongod --replSet s1 --logpath "s1-r2.log" --dbpath /data/shard1/rs2 --port 47019 --fork --shardsvr
sleep 5
echo "Configuring s1 replica set"
mongo --port 47017 << 'EOF'
config = { _id: "s1", members:[
{ _id : 0, host : "localhost:47017" },
{ _id : 1, host : "localhost:47018" },
{ _id : 2, host : "localhost:47019" }]};
rs.initiate(config)
EOF
start a replicate set and tell it that it will be a shard2
echo "starting servers for shard 2"
mkdir -p /data/shard2/rs0 /data/shard2/rs1 /data/shard2/rs2
mongod --replSet s2 --logpath "s2-r0.log" --dbpath /data/shard2/rs0 --port 57017 --fork --shardsvr
mongod --replSet s2 --logpath "s2-r1.log" --dbpath /data/shard2/rs1 --port 57018 --fork --shardsvr
mongod --replSet s2 --logpath "s2-r2.log" --dbpath /data/shard2/rs2 --port 57019 --fork --shardsvr
sleep 5
echo "Configuring s2 replica set"
mongo --port 57017 << 'EOF'
config = { _id: "s2", members:[
{ _id : 0, host : "localhost:57017" },
{ _id : 1, host : "localhost:57018" },
{ _id : 2, host : "localhost:57019" }]};
rs.initiate(config)
EOF
now start 3 config servers
echo "Starting config servers"
mkdir -p /data/config/config-a /data/config/config-b /data/config/config-c
mongod --replSet csReplSet --logpath "cfg-a.log" --dbpath /data/config/config-a --port 57040 --fork --configsvr
mongod --replSet csReplSet --logpath "cfg-b.log" --dbpath /data/config/config-b --port 57041 --fork --configsvr
mongod --replSet csReplSet --logpath "cfg-c.log" --dbpath /data/config/config-c --port 57042 --fork --configsvr
echo "Configuring configuration server replica set"
mongo --port 57040 << 'EOF'
config = { _id: "csReplSet", members:[
{ _id : 0, host : "localhost:57040" },
{ _id : 1, host : "localhost:57041" },
{ _id : 2, host : "localhost:57042" }]};
rs.initiate(config)
EOF
now start the mongos on a standard port
mongos --logpath "mongos-1.log" --configdb csReplSet/localhost:57040,localhost:57041,localhost:57042 --fork
echo "Waiting 60 seconds for the replica sets to fully come online"
sleep 60
echo "Connnecting to mongos and enabling sharding"
add shards and enable sharding on the test db
mongo <<'EOF'
use admin
db.runCommand( { addshard : "s0/localhost:37017" } );
db.runCommand( { addshard : "s1/localhost:47017" } );
db.runCommand( { addshard : "s2/localhost:57017" } );
db.runCommand( { enableSharding: "test" } );
db.runCommand( { shardCollection: "test.some_collection", key: { some_id:1 } } );
EOF
I created a one node Mongo database, everything went fine and I'm able to enter mongo shell and execute commands like "db","use test" , but that's about it , I'm not able to execute any other commands like "show dbs" or insert anything. I googled as there are lot of threads on this error with some suggesting to run rs.slaveOk() and some others asking to create user, I tried creating user as below:
db.createUser(
{
user: "okkadu",
pwd: "abc123",
roles: [ { role: "root", db: "test" } ]
}
)
However this also failed with below error:
Error: couldn't add user: not authorized on test to execute command
Could someone help me out what else can I try ?, please find the copy if my config file as well:
##
**### Basic Defaults
##
port = 27017
dbpath = /opt/mongodb/data
logappend = true
logpath = /var/log/mongodb/mongodb.log
rest = true
fork = true
#bind_ip = 127.0.0.1
pidfilepath = /var/log/mongodb/mongodb.pid
journal = true
httpinterface = true
storageEngine = wiredTiger
auth=true**
I'm setting up a sharded mongo cluster. I have two replica sets consisting of two nodes each, a replica set of three config servers, and a single mongos instance.
I have been able to add the replica set to the mongos instance:
sh.addShard("rs1/shard-rs01-s01");
This returns {"ok" : 1} and the same is true of the second replica set.
However when I try to do any database operations such as db.test.insert(...) I receive this error:
2017-02-23T01:17:28.599+0000 I ASIO [CatalogManagerReplacer]
Connecting to shard-RS01-S01:27017
2017-02-23T01:17:28.600+0000 I ASIO [CatalogManagerReplacer]
Connecting to config-01:27019
2017-02-23T01:17:28.603+0000 I ASIO [CatalogManagerReplacer]
Successfully connected to config-01:27019
2017-02-23T01:17:48.600+0000 I ASIO [CatalogManagerReplacer] Failed to connect to shard-RS01-S01:27017 - ExceededTimeLimit: Operation timed out
I double checked that the firewall wasn't blocking the connection by disabling it on all of the systems. For what it is worth, on the node that contains the mongos instance I can connect to the replica-set directly through the command like using this command regardless of the firewall state:
mongo --host rs1/shard-rs01-s01:27017
So I am fairly sure it is not a firewall issue. Anyone have any ideas?
Here's a shard map of the setup if it is useful for anyone able to help...
mongos> db.runCommand("getShardMap")
{
"map" : {
"config" : "rs0/config-01:27019,config-02:27019,config-03:27019",
"config-01:27019" : "rs0/config-01:27019,config-02:27019,config-03:27019",
"config-02:27019" : "rs0/config-01:27019,config-02:27019,config-03:27019",
"config-03:27019" : "rs0/config-01:27019,config-02:27019,config-03:27019",
"rs0/config-01:27019,config-02:27019,config-03:27019" : "rs0/config-01:27019,config-02:27019,config-03:27019",
"rs1" : "rs1/shard-RS01-S01:27017,shard-RS01-S02:27017",
"rs1/shard-RS01-S01:27017,shard-RS01-S02:27017" : "rs1/shard-RS01-S01:27017,shard-RS01-S02:27017",
"rs2" : "rs2/shard-RS02-S03:27017,shard-RS02-S04:27017",
"rs2/shard-RS02-S03:27017,shard-RS02-S04:27017" : "rs2/shard-RS02-S03:27017,shard-RS02-S04:27017",
"shard-RS01-S01:27017" : "rs1/shard-RS01-S01:27017,shard-RS01-S02:27017",
"shard-RS01-S02:27017" : "rs1/shard-RS01-S01:27017,shard-RS01-S02:27017",
"shard-RS02-S03:27017" : "rs2/shard-RS02-S03:27017,shard-RS02-S04:27017",
"shard-RS02-S04:27017" : "rs2/shard-RS02-S03:27017,shard-RS02-S04:27017"
},
"ok" : 1
}
you need to initialize your mongos.
rs.initiate( { _id: "configReplSet", configsvr: true, members: [ { _id: 0, host: "mongo-config-1:27017" }] } )
I am trying to get MongoDB running on my localhost (Windows) with authentication.
To do so, I first have to add a user, right?
I did so by starting the daemon using this command:
C:\[…]\mongod.exe -f C:\[…]\mongo.config
mongo.config contains the following:
# Basic database configuration
dbpath = C:\[…]\db\
bind_ip = 127.0.0.1
port = 20571
# Security
noauth = true
# Administration & Monitoring
nohttpinterface = true
After that I connected via this command:
C:\[…]\mongo.exe --port 20571 127.0.0.1
There I added a user:
> use admin
switched to db admin
> db.addUser('test', 'test')
{ "n" : 0, "connectionId" : 1, "err" : null, "ok" : 1 }
{
"user" : "test",
"readOnly" : false,
"pwd" : "a6de521abefc2fed4f5876855a3484f5",
"_id" : ObjectId("50db155e157524b3d2195278")
}
To check if everything worked I did the following:
> db.system.users.find()
{ "_id" : ObjectId("50db155e157524b3d2195278"), "user" : "test", "readOnly" : false, "pwd" : "a6de521abefc2fed4f5876855a3484f5" }
Which seemed OK to me.
After that I changed "noauth = true" to "auth = true" in the mongo.config file and restarted the daemon.
Now I expected to be able to connect with user and password:
C:\[…]\mongo.exe --port 20571 -u test -p test 127.0.0.1
Which denied access to me with this message:
MongoDB shell version: 2.0.4
connecting to: 127.0.0.1:20571/127.0.0.1
Wed Dec 26 16:24:36 uncaught exception: error { "$err" : "bad or malformed command request?", "code" : 13530 }
exception: login failed
So here's my question: Why does the login fail?
I can still connect without providing user and password, but can't access any data because "unauthorized db:admin lock type:-1 client:127.0.0.1". Which is actually what I expected.
As Andrei Sfat told me in the comments on the question I made 2 major errors.
First, I thought I could pass the IP to the Client as a simple argument. But you have to use --host for that.
Instead, the parameter I thought was the IP address actually should be the db name.
So the correct command to connect to a Server is as follows:
C:\[…]\mongo.exe --port 20571 -u test -p test --host 127.0.0.1 admin
Second, users are per database. As I only added the user "test" to the db "admin", it only works there.
Obviously the auth = true configuration wasn't load successfully. Did you forget the -f paramter when you restart the mongod.exe?
C:\[…]\mongod.exe -f C:\[…]\mongo.config
I am new to mongoDB and i have the following query as follows "db.runCommand( { addshard : "sf103", maxSize:100000 } );" Why we are using sf103 If i use this command in my Environment i am Getting following errors>>> db.runCommand( { addshard : "sf103", maxSize:100000 } );
{
"ok" : 0,
"errmsg" : "couldn't connect to new shard dbconnectionpool: connect failed sf103 : "
}
>
Here, The Sf103 represents what???? Please help me.......
Advance Thanks,
Kumar.
addshard takes the first parameter as "serverhostname[:port]". sf103 is a example host name they specified. What is the mongo host you want to add to the cluster. Please read the doc to understand how to configure the mongo cluster.
Mongo sharded cluster config
--Sai