create index on existing collection on a MongoDB replica set (PRIMARY) - mongodb

I want to add an index on an existing collection in a MongoDB replica set.
This replica set has only a Primary, at the moment, and is configured in this way to be able to use MongoDB change streams.
I log into Mongo shell using:
mongo -u <Account>
where <Account> is an account with role 'userAdminAnyDatabase '.
Then issue the commands:
use dbOfInterest
db.collectionOfInterest.createIndex({fieldOne:-1})
But after issuing the command above the console seems not responding anymore (after several minutes I had to stop it using CTRL+C).
The considered collection is a test one with very few documents.
What am I doing wrong?

Finally solved!!.
The issue is indeed related to "replica set node with auth can connect to itself" (more here).
I had to modify my mongod (file /etc/mongod.conf) configuration as follow:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
# how the process runs
processManagement:
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/user1/Desktop/mongo.keyfile
authorization: enabled
#operationProfiling:
replication:
replSetName: AirHeritage
#sharding:
## Enterprise-Only Options:
#auditLog:
#snmp:
Note keyFile section into YAML mongod.conf file (pay attention to white spaces).
To correctly generate this mongo.keyfile I used:
openssl rand -base64 756 > <path-to-keyfile>
chmod 400 <path-to-keyfile>
Then do:
sudo chown mongodb:mongodb /home/user1/Desktop/mongo.keyfile
Check that results are something as:
ls -al /home/user1/Desktop/mongo.keyfile
-r-------- 1 mongodb mongodb 1024 gen 13 09:19 /home/user1/Desktop/mongo.keyfile
Then stop and restart mongod using:
sudo systemctl stop mongod
sudo systemctl start mongod
Check status:
sudo systemctl status mongod
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-01-13 09:25:59 CET; 44min ago
Docs: https://docs.mongodb.org/manual
Main PID: 10156 (mongod)
Memory: 348.6M
CGroup: /system.slice/mongod.service
└─10156 /usr/bin/mongod --config /etc/mongod.conf
gen 13 09:25:59 eanx-XPS-13-9350 systemd[1]: Started MongoDB Database Server.
Then log into mongo console as root (not sure it is necessary root, should suffice ClusterAdmin role):
mongo -u root
use dbOfInterest
db.collectionOfInterest.createIndex({field:1})
Having done all that before, index creation should have worked without hanging the console with a result as:
db.collectionOfInterest.createIndex({'field':1})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"commitQuorum" : "votingMembers",
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1610526522, 5),
"signature" : {
"hash" : BinData(0,"qSqkVswHQA/IzWGYCd8HwNhXoQk="),
"keyId" : NumberLong("6877458498892857349")
}
},
"operationTime" : Timestamp(1610526522, 5)
}
To check MongoDB logs use:
sudo tail /var/log/mongodb/mongod.log | jq
(if not installed in your system use sudo apt install jq, jq is very useful to pretty print json files)
Finally check indexes on collection with:
db.collectionOfInterest.getIndexes()
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_"
},
{
"v" : 2,
"key" : {
"field" : 1
},
"name" : "field.name_1"
}
Note that two key fields are reported: "_id" (by default on collection creation) and "field"!!
Hope this can help someone else having a similar issue.
Only move the mongo.keyfile to a more suitable location (someone can suggest where?)

What you do is creating index "foreground" and that blocks usage of DB, until creation is ready.
You need to do it "background".
db.collectionOfInterest.createIndex({fieldOne:-1},{background:1})

Related

MongoDB unable to connect to the newly created admin user in admin db

We have created a new admin user in mongodb version 3.6.21
[root#MONGODB01 mongodb]# mongo --port 37017
MongoDB shell version v3.6.21
connecting to: mongodb://127.0.0.1:37017/?gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("54681232-fa6b-4dbd-bfe5-c1bde274d430") }
MongoDB server version: 3.6.21
s1:PRIMARY> use admin
switched to db admin
s1:PRIMARY> db.createUser(
... {user: "admin",
... pwd: "reco#123",
... roles:[{role: "root" , db:"admin"}]})
Successfully added user: {
"user" : "admin",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
s1:PRIMARY> exit
bye
After that we are trying to login but facing below error. Please help
[root#MONGODB01 mongodb]# mongo --port 37017 –u admin –p reco#123 -authenticationDatabase admin
MongoDB shell version v3.6.21
connecting to: mongodb://127.0.0.1:37017/%E2%80%93u?authSource=admin&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("729e70cf-d594-4bd1-b6dc-73d75fd20f89") }
MongoDB server version: 3.6.21
loading file: admin
2021-10-18T15:35:07.305+0530 E - [main] file [admin] doesn't exist
failed to load: admin
Below is the config file for primary mongod instance.
[root#MONGODB01 mongodb]# cat /etc/mongod.conf
# mongod.conf
systemLog:
destination: file
logAppend: true
path: /data/mongodb/primary1/mongod.log
storage:
dbPath: /data/mongodb/primary1/db
journal:
enabled: true
# mmapv1:
# # Reduce data files size and journal files size
# smallFiles: true
wiredTiger:
engineConfig:
cacheSizeGB: 4
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
net:
port: 37017
bindIp: 0.0.0.0
security:
# authorization: enabled
keyFile: /data/mongodb/key/mongodb.key
replication:
replSetName: s1
sharding:
clusterRole: shardsvr
FYI - We are running Primary shard1, Replica Shard 2 and Replica Shard 3 on single node.
You need two hyphen --, should be this:
mongo --port 37017 --authenticationDatabase admin -u admin -p reco#123
You may enclose the password by quotes, i.e.
mongo --port 37017 --authenticationDatabase admin -u admin -p "reco#123"
For username and password you use either short or long form, i.e.
-u admin -p "reco#123"
or
--username admin --password "reco#123"
I was able to solve this issue by using below command instead
mongo --port 37017 -authenticationDatabase admin -u admin -p reco#123

mongodb remote authentication failure Mongod v4

I have recently installed mongodb on one of our windows 2016 server with below configuration and running mongod as
mongod --config C:\mongo_db\config\mongo_db_configuration.conf --auth
--config file
net:
# MongoDB server listening port
#bindIp: 0.0.0.0,192.168.43.250
port: 27017
bindIpAll: true
storage:
# Data store directory
dbPath: "C:\\mongo_db\\db"
mmapv1:
# Reduce data files size and journal files size
smallFiles: true
systemLog:
# Write logs to log file
destination: file
path: "C:\\mongo_db\\logs\\mongodb.log"
security:
authorization: enabled
We have created a root user in admin database
MongoDB Enterprise > db.runCommand({connectionStatus:1})
{
"authInfo" : {
"authenticatedUsers" : [
{
"user" : "root",
"db" : "admin"
}
],
"authenticatedUserRoles" : [
{
"role" : "dbAdminAnyDatabase",
"db" : "admin"
},
{
"role" : "root",
"db" : "admin"
}
]
},
"ok" : 1
}
I am able to connect to the mongod instance locally (mongo --port 27017 -u "root" -p "12345" --authenticationDatabase admin) but getting error while connecting remotely.the server firewall is disabled.
"C:\Mongodb>mongo.exe 192.168.0.171:27017/admin -u root -p 12345
MongoDB shell version v4.0.4
connecting to: mongodb://192.168.0.171:27017/admin
Implicit session: session { "id" : UUID("c68e9dd5-6f7b-4356-bc3e-688a7e2b3e1d") }
MongoDB server version: 4.0.4
2018-11-26T23:50:47.534+0800 E QUERY [js] Error: Authentication failed. :
DB.prototype._authOrThrow#src/mongo/shell/db.js:1685:20
#(auth):6:1
#(auth):1:2
exception: login failed"
I stumbled upon this when I was having the same problem. I was connecting to a MongoDB server version 4.2.6 from a MongoDB shell version v3.6.9. In case anyone else finds this question like I did, here's the syntax that worked:
mongo --host mongodb://username:password#IP:PORT/ --authenticationDatabase admin
I'm also having similar issue with my virtual machines. I have 2 linux and 1 windows machine (which are remotely connected to my windows machine). Let's say Linux A has mongo with credentials. When I try to login from my windows machine, I got the same error that you mention.
However, when I connect mongo from linux B to linux A, I can successfully connect from shell using:
mongo --host 192.168.1.6 --port 27017 -u username -p 'pwd'
I use exact same command from windows as well but somehow it fails to login. Maybe this command could work on your environment but waiting for proper solution as well.

Replica - MongoDB EC2 instances

Added 3 EC2 instances in in AWS cloud.
Installed 3 MongoDB instances in 3 different instances and all are running fine.
I am adding replication for the same DBs.
below are the steps i followed:
sudo service mongod start --sslPEMKeyFile "/etc/ssl/mongodb.pem" --sslMode "requireSSL" --replSet "myReplSet" ( in all 3 mongo instances)
config = {_id : "myReplSet" , members: [{_id: 0, host:"ip-10-0-0-68:27017"},{_id: 1, host:"ip-10-0-0-247:27017"},{_id: 2, host:"ip-10-0-0-148:27017"}]}
rs.initiate(config)
Error as below:
MongoDB Enterprise > rs.initiate(config)
{
"ok" : 0,
"errmsg" : "This node was not started with the replSet option",
"code" : 76,
"codeName" : "NoReplicationEnabled"
}
EDIT
I added same security groups and Elastic IP for all 3 instances.
All 3 are new DBs.
Now I am getting error as:
{
"ok" : 0,
"errmsg" : "'ip-10-0-0-148:27017' has data already, cannot initiate set.",
"code" : 110,
"codeName" : "CannotInitializeNodeWithData"
}
EDIT-2
Config file for one the servers:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
# network interfaces
net:
port: 27017
# bindIp: 127.0.0.1,13.58.225.174,13.58.180.226 # Listen to local interface only, comment to listen on all interfaces.
#security:
#operationProfiling:
replication:
replSetName: rs0
The service start command does not accept parameters. So
sudo service mongod start --sslPEMKeyFile "/etc/ssl/mongodb.pem" --sslMode "requireSSL" --replSet "myReplSet"
Is equivalent to
sudo service mongod start
If you want to add those settings to the mongod configuration, you will need to edit /etc/mongod.conf to add them in. Since the names of some options vary between the command line and the configuration file, you should take a look at the Configuration File Options documentation to ensure you use the correct settings.
EDIT
Based on your edits and the error message, it appears the mongod process (ip-10-0-0-148:27017) already has data on it. You cannot add a mongod process that already has data to a replica set. You can, however, initiate the replica set on a mongod process that already has data. You should connect to that mongod process and run the rs.initiate() there. Then you run rs.add() to add the remaining, empty, mongod processes to the replica set.
Note: MongoDB does offer MongoDB Cloud Manager and MongoDB Atlas which can do some or all of this for you. However, they are not free, so its up to you if you want to use them.
This way it works fine.
rs.initialize()
rs.add()
rs.add()
i reffed here:
https://devops.profitbricks.com/tutorials/configure-mongodb-replica-set/

Authenticate mongodb replica set

I have 4 instances of mongodb running(replica set) with following mongodb.conf file for each instance:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /root/mongodata/log/mongod.log
# Where and how to store data.
storage:
dbPath: /root/mongodata/db1 # also have db2 and so on for rest of the instances
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /root/mongodata/db1/mongod.pid # location of pidfile, different for 4 instances
# network interfaces
net:
port: 30000 #port different for 4 different instances
bindIp: 12.123.321.432(example ip, same for all 4 .conf files)
# security
security:
KeyFile: /path to my keyfile location
# authorization: enabled
#operationProfiling:
replication:
replSetName: testReplica #have this same for all 4
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
I also created a keyfile for internal authentication as follows:
openssl rand -base64 756 > <path-to-keyfile>
chmod 400 <path-to-keyfile>
After all the instances are running I opened mongoShell as follows:
mongo --host 12.123.321.432 --port 30000
I am able to open the shell but when I try to create a user, I get the following exception:
2016-12-22T20:55:38.396-0500 E QUERY [thread1] Error: couldn't add user: not authorized on test to execute command { createUser: "root", pwd: "xxx", roles: [ { role: "root", db: "admin" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 30000.0 } } :
_getErrorWithCode#src/mongo/shell/utils.js:23:13
DB.prototype.createUser#src/mongo/shell/db.js:1230:11
#(shell):1:1
I tried switching to admin db but still says unauthorized, I also tried to run rs.initiate() command to define primary and secondary dbs, says unauthorized. I read even if i start the mongod with authentication disabled the internal authentication via keyfile will force the role based authentication. What am I missing here and how would i resolve it? thanks in advance.
The error message shows that you have not privileges to execute the command.
After set keyfile parameter in .conf file, mongo need you login with auth user.
So if you have not root user.
start mongod without auth config (don't set keyfile and disable security.authorization )
create use with root role
db.createUser({user:"root",pwd:"rootpassword",roles:[{role:"root",db:"admin"}]})
if you have root user, you should use db.auth() or login mongo with root privileges, for executing rs.conf() command.
mongo --port portnumber -u root -p --authenticationDatabase admin
or after login with command mongo --port portnumber
db.auth("root", "rootpassword")
ps. KeyFile is for replica set inner communication security.

unable to launch config server for mongo shard

I am trying to get the config server running for mongo shard using the following command and config file:
mongod --config /etc/mongod2.conf
mongod2.config
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb2
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
processManagement:
pidFilePath: /srv/mongodb/db2.pid
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27019
bindIp: 127.0.0.1
processManagement:
fork: true
#security:
# authorization: enabled
replication:
replSetName: configReplSet
sharding:
clusterRole: configsvr
When I run the command above, I get the following output (with no log output in the mongo logs):
about to fork child process, waiting until server is ready for connections.
forked process: 14246
ERROR: child process failed, exited with error number 1
I am running mongo v3.2.2 on ubuntu v14. The port 27019 is not bound by another process.
The issue was caused since I never created a separate data directory. I saw better error messages when I ran the following:
mongod --dbpath /var/lib/mongodb2 --port 27019
This command gave better error messages (it showed that the data directory was not present etc.). This helped solve the issue and I was able to run the config server (output below):
2016-06-29T12:21:30.216-0400 I CONTROL [initandlisten] MongoDB starting : pid=16060 port=27019 dbpath=/var/lib/mongodb2 64-bit host=abcdefghi
2016-06-29T12:21:30.216-0400 I CONTROL [initandlisten] db version v3.2.0