Mongo shell not creating indexes on running databases - mongodb

We dumped the definition of indexes using mongo-index-exporter and we try to apply them at each launch of our application, so to achieve a reproducible environment, by executing
mongo --verbose "mongodb://$MONGO_DB_USER:$MONGO_DB_PASSWORD#$MONGO_DB_HOST/$MONGO_DB_NAME?authSource=$MONGO_AUTH_SOURCE&authMode=scram-sha1" indexes.js
on a file like so:
print('Using db ' + db)
setVerboseShell(true)
print('Creating indexes on mycoll1')
db.mycoll1.createIndex({"_id":1}, {"name":"_id_", "background": true});
db.mycoll1.createIndex({"createdAt":1,"field1.field2.field3":1}, {"name":"coll1_field123_idx"});
print('Creating indexes on mycoll2')
db.mycoll2.createIndex({"_id":1}, {"name":"_id_", "background": true});
db.mycoll2.createIndex({"createdAt":1,"field1.field2.field3":1}, {"name":"coll1_field123_idx"});
To create indexes on a replicaSet with five nodes, we are performing the
following command:
mongo --verbose "mongodb://$MONGO_DB_USER:$MONGO_DB_PASSWORD#$MONGO_DB_HOST/$MONGO_DB_NAME?authSource=$MONGO_AUTH_SOURCE&authMode=scram-sha1" indexes.js
this works fine for us in QA, where MONGO_DB_HOST is a single node and we are using a fresh new empty database, and doesn't work in production where additionally the database is already existing (and the collection have contents).
Additional relevant information:
background mode doesn't make any difference
The verbose mode doesn't work, i.e. although we launch mongo client with -v and we do a setVerbose(true) nothing is logged on the console
Our mongo shell is 3.6.11 and our mongo server is 3.2.9
Copy pastying the commands inside a mongo shell executed on the server works terminates immediately and results in the following output
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 11,
"numIndexesAfter" : 11,
"note" : "all indexes already exist",
"ok" : 1
}

Related

How to write log file for individual collection in mongodb database?

I have a peculiar question that I haven't been able to find an answer to yet. I have a database and a collection in it. Let's call them "test database" and "test collection". There are a range of users from root to a user with read-only access to the "test database". I need to output a special log file for operations that are carried out in a "test database".
I have already found "mongo.log" in the path "/var/log/mongodb/". But all possible requests from all "databases" are loaded into this file at once, which is not convenient. Is it possible somehow to create a separate file for writing logs only from the "test database"?
I also found out earlier that in order to keep track of all requests, I need to set the following settings:
$ mongo
MongoDB shell version: 2.4.9
connecting to: test
> use myDb
switched to db myDb
> db.getProfilingLevel()
0
> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 1, "ok" : 1 }
> db.getProfilingLevel()
2

Mongo Copy Database Collect Exist Error

I am trying to copy a database from a remote host over to the current database. I am using this command:
rs0:PRIMARY> db.copyDatabase("olddb", "newdb", "xx.xx.xx.137", "user1", "abc123");
But I am getting this error:
{
"done" : true,
"ok" : 0,
"errmsg" : "failed to create collection \"newdb.email_batches\": collection already exists"
}
Except the collect does not exist. What could I be doing wrong in this command?
The problem was different database versions. The DB I was on was Mongo 3.0 and the database I was trying to transfer from was 3.2
What is very strange about that is the command copyDatabase as functionaliy for transferring data from 2.6 to 3.2, yet for some reason does not work with 3.0.

How to convert a MongoDB replica set to a stand alone server

Consider, I have 4 replicate sets and the config is as follows:
{
"_id": "rs_0",
"version": 5,
"members" : [
{"_id": 1, "host": "127.0.0.1:27001"},
{"_id": 2, "host": "127.0.0.1:27002"},
{"_id": 3, "host": "127.0.0.1:27003"},
{"_id": 4, "host": "127.0.0.1:27004"}
]
}
I am able to connect to all sets using
mongo --port <port>
There are documents for getting information on Convert a Standalone to a Replica Set, but can anyone tell me how to convert back to standalone from replica set?
Remove all secondary hosts from replica set (rs.remove('host:port')), restart the mongo deamon without replSet parameter (editing /etc/mongo.conf) and the secondary hosts starts in standalone mode again.
The Primary host is tricky one, because you can't remove it from the replica set with rs.remove.
Once you have only the primary node in the replica set, you should exit mongo shell and stop mongo. Then you edit the /etc/mongo.conf and remove the replSet parameter and start mongo again.
Once you start mongo you are already in standalone mode, but the mongo shell will prompt a message like:
2015-07-31T12:02:51.112+0100 [initandlisten] ** WARNING: mongod started without --replSet yet 1 documents are present in local.system.replset
to remove the warning you can do 2 procedures:
1) Droping the local db and restarting mongo:
use local
db.dropDatabase();
/etc/init.d/mongod restart
2)Or if you don't want to be so radical, you can do:
use local
db.system.replset.find()
and it will prompt a message like:
{ "_id" : "replicaSetName", "version" : 1, "members" : [ { "_id" : 0, "host" : "hostprimary:mongoport" } ] }
then you will erase it using:
db.system.replset.remove({ "_id" : "replicaSetName", "version" : 1, "members" : [ { "_id" : 0, "host" : "hostprimary:mongoport" } ] })
and it will probably prompt:
WriteResult({ "nRemoved" : 1 })
Now, you can restart the mongo and the warning should be gone, and you will have your mongo in standalone mode without warnings
Just remove a host from replica set (rs.remove('host:port')), relaunch it without replSet parameter and it's standalone again.
On an Ubuntu Machine
Stop your mongo server
open /etc/mongod.conf
Comment the replication and replSetName line
#replication:
#replSetName: rs0
Start your mongo server and go to mongo shell
drop local database
use local
db.dropDatabase()
Restart mongo
The MongoDB Documentation suggests the following to perform maintenance on a replica set member, which brings the the replica set member into standalone mode for further operations. With little modification it can be made standalone:
If node in concern is the only node in a shard, drain the chunks to other shards as per MongoDB documentation here, or else the sharded database will break, i.e.
Make sure balancer is enabled by connecting to mongos and run sh.startBalancer(timeout, interval)
For the shard in concern, go to admin database and db.adminCommand( { removeShard: "mongodb0" } )
Check draining status by repeating above removeShard command, wait for draining to complete
If node in concern is primary, do rs.stepDown(300)
Stop the node by running db.shutdownServer()
Change the yaml config by:
commenting out replication.replSetName (--replSetName in command line)
commenting out sharding.clusterRole for shard or config server (--shardsvc and --configsvr in command line)
commenting out net.port, then change it to a different port (--port in command line)
Start the mongod instance
If change is permanent, go to other mongod instance and run rs.remove("host:port")
After this, the node in concern should be up and running in standalone mode.
Follow below steps :
Go to mongo shell on Secondary servers
Stop the secondary servers by using below command :
use admin
db.shutdownServer()
Go to Linux shell- on secondary servers and type below command :
sudo service mongod stop
Starting the MongoDB replication -
Go to Linux shell - on secondary servers and type below command :
sudo service mongod start
Starting the MongoDB replication -
Go to primary and type below commands to start the replication :
a] rs.initiate()
b] rs.add("Secondar -1:port no")
c] rs.add("Secondary-2:port no")
d] rs.add({ "_id" : 3, "host" : "Hidden_member:port no", "priority" : 0,
"hidden" : true })
e] rs.status()

What is the BSON doc (query) for the cmd `show dbs` in MongoDB?

According to Getting Started with MongoDB, we can use show dbs to get the list of existing databases.
But this is a kind of command running in the mongo shell.
My question is that how to write a mongodb query (bson) for the list of databases, and where this query should be sent to?
It's not a query, but you can run the listDatabases command against the admin database to get a list of databases in BSON:
> use admin
switched to db admin
> db.runCommand({listDatabases : 1})
{
"databases" : [
{
"name" : "local",
"sizeOnDisk" : 83886080,
"empty" : false
},
],
"totalSize" : 83886080,
"ok" : 1
}
Most programming language engines for MongoDB can also run commands in addition to queries given the right syntax. For example, in Java:
DB db = mongo.getDB("admin");
DBObject cmd = new BasicDBObject("listDatabases", 1);
CommandResult result = db.command(cmd);
Interestingly, if you're really insistent on it being a query, you can query the virtual collection $cmd on the admin database to run commands by submitting queries:
> use admin
switched to db admin
> db.$cmd.findOne({"listDatabases":1})

MongoDb DropDatabase Not Working

I have a 200gb data base on a sharded four node cluster and and I would like to drop the databse and delet all the files associated to it from the node. I am connecting to my mongos and call dropDatabase on it. The system comes back with ok but if call show dbs, it will show the database again and shows that it is still occupying the 200gb. What I am doing wrong?
I think you are running into this issue:
https://jira.mongodb.org/browse/SERVER-4804
In most cases it seems like the database is in fact removed but the mongos still reports it as being there. You can verify it is gone by either trying to use the DB and getting an error or by logging into the shards directly and checking.
The bug refers to issues with dropping databases while a migration is happening. You can workaround the cause of the issue by doing something like this (sub in your own dbname):
mongos> use config
switched to db config
// 1. stop the balancer
mongos> db.settings.update({_id: "balancer"}, {$set: {stopped: true}}, true)
// 2. wait for in-progress migrations to finish, this may take a few seconds
mongos> while (db.locks.findOne({_id: "balancer", state: {$ne: 0}}) != null) { sleep(1000); }
// 3. now you can safely drop the database
mongos> use <dbname>
switched to db <dbname>
mongos> db.dropDatabase()
{ "dropped" : "<dbname>", "ok" : 1 }
You may want to run the flushRouterConfig on the mongoses to refresh the config info:
mongos> use config
switched to db config
mongos> var mongoses = db.mongos.find()
mongos> while (mongoses.hasNext()) { new Mongo(mongoses.next()._id).getDB("admin").runCommand({flushRouterConfig: 1}) }
{ "flushed" : true, "ok" : 1 }
Of course, the real fix will only come along when the fix is committed - looks like it is targeted for 2.1
If you are in a broken state, you can try this, but it is tricky:
To "reset" the sharding metadata to recover from this issue, please try to do the following
First, stop the balancer (as above) and wait for migrations to finish (also as above)
Next, ensure there is no activity from the app servers on the database in question
Now, ensure that there are no collection entries in config.collections for namespaces beginning with "TestCollection." If so, remove those entries through a mongos:
mongos> use config
mongos> db.collections.find({_id: /^TestCollection\./})
// if any records found, delete them
mongos> db.collections.remove({_id: /^TestCollection\./})
Next up, ensure there is no database entry in config.databases for "TestCollection", and if so, remove it through mongos:
mongos> use config
switched to db config
mongos> db.databases.find({_id: "TestCollection"})
// if any records found, delete them
mongos> db.databases.remove({_id: "TestCollection"})
Now, ensure there are no entries in config.chunks for any namespaces in the database (like the default test namespace). If there are any, remove through mongos:
mongos> use config
switched to db config
mongos> db.chunks.find({ns: /^test\./})
// if any records found, delete them
mongos> db.chunks.remove({ns: /^test\./})
Then, flushRouterConfig on all mongoses:
mongos> use config
switched to db config
mongos> var mongoses = db.mongos.find()
mongos> while (mongoses.hasNext()) { new Mongo(mongoses.next()._id).getDB("admin").runCommand({flushRouterConfig: 1}) }
{ "flushed" : true, "ok" : 1 }
...
Finally, manually connect to each shard primary, and drop the database on the shards (not all shards may have the database, but it's best to be thorough and issue the dropDatabase() call on all
Regarding in-progress migrations, you can use this snippet:
// 2. wait for in-progress migrations to finish, this may take a few seconds
mongos> while (db.locks.findOne({_id: "balancer", state: {$ne: 0}}) != null) { sleep(1000); }
When done, don't forget to reenable the balancer:
mongos> use config
switched to db config
mongos> db.settings.update({_id: "balancer"}, {$set: {stopped: false}}, true)
I had this exact same problem, and discovered that issuing the db.dropDatabase as a regular user failed silently but doing the same as sudo worked, in case that helps anyone.