Added 3 EC2 instances in in AWS cloud.
Installed 3 MongoDB instances in 3 different instances and all are running fine.
I am adding replication for the same DBs.
below are the steps i followed:
sudo service mongod start --sslPEMKeyFile "/etc/ssl/mongodb.pem" --sslMode "requireSSL" --replSet "myReplSet" ( in all 3 mongo instances)
config = {_id : "myReplSet" , members: [{_id: 0, host:"ip-10-0-0-68:27017"},{_id: 1, host:"ip-10-0-0-247:27017"},{_id: 2, host:"ip-10-0-0-148:27017"}]}
rs.initiate(config)
Error as below:
MongoDB Enterprise > rs.initiate(config)
{
"ok" : 0,
"errmsg" : "This node was not started with the replSet option",
"code" : 76,
"codeName" : "NoReplicationEnabled"
}
EDIT
I added same security groups and Elastic IP for all 3 instances.
All 3 are new DBs.
Now I am getting error as:
{
"ok" : 0,
"errmsg" : "'ip-10-0-0-148:27017' has data already, cannot initiate set.",
"code" : 110,
"codeName" : "CannotInitializeNodeWithData"
}
EDIT-2
Config file for one the servers:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
# network interfaces
net:
port: 27017
# bindIp: 127.0.0.1,13.58.225.174,13.58.180.226 # Listen to local interface only, comment to listen on all interfaces.
#security:
#operationProfiling:
replication:
replSetName: rs0
The service start command does not accept parameters. So
sudo service mongod start --sslPEMKeyFile "/etc/ssl/mongodb.pem" --sslMode "requireSSL" --replSet "myReplSet"
Is equivalent to
sudo service mongod start
If you want to add those settings to the mongod configuration, you will need to edit /etc/mongod.conf to add them in. Since the names of some options vary between the command line and the configuration file, you should take a look at the Configuration File Options documentation to ensure you use the correct settings.
EDIT
Based on your edits and the error message, it appears the mongod process (ip-10-0-0-148:27017) already has data on it. You cannot add a mongod process that already has data to a replica set. You can, however, initiate the replica set on a mongod process that already has data. You should connect to that mongod process and run the rs.initiate() there. Then you run rs.add() to add the remaining, empty, mongod processes to the replica set.
Note: MongoDB does offer MongoDB Cloud Manager and MongoDB Atlas which can do some or all of this for you. However, they are not free, so its up to you if you want to use them.
This way it works fine.
rs.initialize()
rs.add()
rs.add()
i reffed here:
https://devops.profitbricks.com/tutorials/configure-mongodb-replica-set/
Related
I want to add an index on an existing collection in a MongoDB replica set.
This replica set has only a Primary, at the moment, and is configured in this way to be able to use MongoDB change streams.
I log into Mongo shell using:
mongo -u <Account>
where <Account> is an account with role 'userAdminAnyDatabase '.
Then issue the commands:
use dbOfInterest
db.collectionOfInterest.createIndex({fieldOne:-1})
But after issuing the command above the console seems not responding anymore (after several minutes I had to stop it using CTRL+C).
The considered collection is a test one with very few documents.
What am I doing wrong?
Finally solved!!.
The issue is indeed related to "replica set node with auth can connect to itself" (more here).
I had to modify my mongod (file /etc/mongod.conf) configuration as follow:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1
# how the process runs
processManagement:
timeZoneInfo: /usr/share/zoneinfo
security:
keyFile: /home/user1/Desktop/mongo.keyfile
authorization: enabled
#operationProfiling:
replication:
replSetName: AirHeritage
#sharding:
## Enterprise-Only Options:
#auditLog:
#snmp:
Note keyFile section into YAML mongod.conf file (pay attention to white spaces).
To correctly generate this mongo.keyfile I used:
openssl rand -base64 756 > <path-to-keyfile>
chmod 400 <path-to-keyfile>
Then do:
sudo chown mongodb:mongodb /home/user1/Desktop/mongo.keyfile
Check that results are something as:
ls -al /home/user1/Desktop/mongo.keyfile
-r-------- 1 mongodb mongodb 1024 gen 13 09:19 /home/user1/Desktop/mongo.keyfile
Then stop and restart mongod using:
sudo systemctl stop mongod
sudo systemctl start mongod
Check status:
sudo systemctl status mongod
● mongod.service - MongoDB Database Server
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-01-13 09:25:59 CET; 44min ago
Docs: https://docs.mongodb.org/manual
Main PID: 10156 (mongod)
Memory: 348.6M
CGroup: /system.slice/mongod.service
└─10156 /usr/bin/mongod --config /etc/mongod.conf
gen 13 09:25:59 eanx-XPS-13-9350 systemd[1]: Started MongoDB Database Server.
Then log into mongo console as root (not sure it is necessary root, should suffice ClusterAdmin role):
mongo -u root
use dbOfInterest
db.collectionOfInterest.createIndex({field:1})
Having done all that before, index creation should have worked without hanging the console with a result as:
db.collectionOfInterest.createIndex({'field':1})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"numIndexesAfter" : 2,
"commitQuorum" : "votingMembers",
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1610526522, 5),
"signature" : {
"hash" : BinData(0,"qSqkVswHQA/IzWGYCd8HwNhXoQk="),
"keyId" : NumberLong("6877458498892857349")
}
},
"operationTime" : Timestamp(1610526522, 5)
}
To check MongoDB logs use:
sudo tail /var/log/mongodb/mongod.log | jq
(if not installed in your system use sudo apt install jq, jq is very useful to pretty print json files)
Finally check indexes on collection with:
db.collectionOfInterest.getIndexes()
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_"
},
{
"v" : 2,
"key" : {
"field" : 1
},
"name" : "field.name_1"
}
Note that two key fields are reported: "_id" (by default on collection creation) and "field"!!
Hope this can help someone else having a similar issue.
Only move the mongo.keyfile to a more suitable location (someone can suggest where?)
What you do is creating index "foreground" and that blocks usage of DB, until creation is ready.
You need to do it "background".
db.collectionOfInterest.createIndex({fieldOne:-1},{background:1})
I am not able to start mongo db service, if I try to start mongod using systemctl restart mongod getting error
Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.
Logs: vi /var/log/mongo/mongod.log
In log file, if I see below message, it looks like port is already in use
exception in initAndListen std::exception: listen: Address already in use, terminating
netstat -apt| grep "27017"
No process id.
Below is mongod status:
I am not getting any solution, your help will be greatly appreciated. Thanks.
Trying starting mongodb on different port and with different data directory, that might be helpful, use below command to start with different port and different data directory
mongod --port 27027 --dbpath </path/to/your/data/directory>
If this work then some process might be using 27017 or or you might have mongod running on background, so first check the process id and manually kill it as below
ps -Aef | grep 27017
or
ps -Aef | grep mongod
Find the process id and kill it
sudo kill -9 <processId>
Then try to restart it as you are doing:
systemctl restart mongod
or:
/etc/init.d/mongodb restart
Hope this help you out.
One issue was the lock file used by MongoDB might have an incorrect ownership:
-rw-r--r-- 1 root root 0 Mar 16 02:27 /var/lib/mongo/mongod.lock
Suggest you to simply removed it which will allow MongoDB to recreate it properly. This file is only needed by MongoDB if it's currently running, and ideally the file should only exist if it's running too, since it only contains a copy of the current process ID which MongoDB is running under.
The second issue will be the configuration file itself: /etc/mongod.conf.
MongoDB expects the file to be in YAML format where apparently blank spaces matter. I have share the sample configuration file which worked in our server.
Basically, each indentation should be made using two spaces for any directives underneath their parent directive.
Additinal notes:
For example, the "net:" heading is not indented because it's the top of its configuration tier, as where "port:" and "bindIp:" are each indented with 2 spaces since "net:" is their parent directive.
The "ssl:" directive is also indented with two spaces since its parent is "net:", but since "mode:", "CAFile:" and "PEMKeyFile:" along belong to "ssl:" they go underneath it and again each take 2 additional spaces for further indentation for a total of 4 spaces.
Finally the following mongo restart will work:
systemctl restart mongod
or
systemctl restart mongod.service
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile
timeZoneInfo: /usr/share/zoneinfo
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0 # Listen to local interface only, comment to listen on all interfaces.
ssl:
mode: preferSSL
CAFile: /etc/ssl/mongodb/RootCA/root-ca.pem
PEMKeyFile: /etc/ssl/mongodb/host.omkieitsolutions.com.pem
security:
authorization: enabled
clusterAuthMode: keyFile
#operationProfiling:
#replication:
"/etc/mongod.conf" 50L, 990C
I have 4 instances of mongodb running(replica set) with following mongodb.conf file for each instance:
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /root/mongodata/log/mongod.log
# Where and how to store data.
storage:
dbPath: /root/mongodata/db1 # also have db2 and so on for rest of the instances
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /root/mongodata/db1/mongod.pid # location of pidfile, different for 4 instances
# network interfaces
net:
port: 30000 #port different for 4 different instances
bindIp: 12.123.321.432(example ip, same for all 4 .conf files)
# security
security:
KeyFile: /path to my keyfile location
# authorization: enabled
#operationProfiling:
replication:
replSetName: testReplica #have this same for all 4
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
I also created a keyfile for internal authentication as follows:
openssl rand -base64 756 > <path-to-keyfile>
chmod 400 <path-to-keyfile>
After all the instances are running I opened mongoShell as follows:
mongo --host 12.123.321.432 --port 30000
I am able to open the shell but when I try to create a user, I get the following exception:
2016-12-22T20:55:38.396-0500 E QUERY [thread1] Error: couldn't add user: not authorized on test to execute command { createUser: "root", pwd: "xxx", roles: [ { role: "root", db: "admin" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 30000.0 } } :
_getErrorWithCode#src/mongo/shell/utils.js:23:13
DB.prototype.createUser#src/mongo/shell/db.js:1230:11
#(shell):1:1
I tried switching to admin db but still says unauthorized, I also tried to run rs.initiate() command to define primary and secondary dbs, says unauthorized. I read even if i start the mongod with authentication disabled the internal authentication via keyfile will force the role based authentication. What am I missing here and how would i resolve it? thanks in advance.
The error message shows that you have not privileges to execute the command.
After set keyfile parameter in .conf file, mongo need you login with auth user.
So if you have not root user.
start mongod without auth config (don't set keyfile and disable security.authorization )
create use with root role
db.createUser({user:"root",pwd:"rootpassword",roles:[{role:"root",db:"admin"}]})
if you have root user, you should use db.auth() or login mongo with root privileges, for executing rs.conf() command.
mongo --port portnumber -u root -p --authenticationDatabase admin
or after login with command mongo --port portnumber
db.auth("root", "rootpassword")
ps. KeyFile is for replica set inner communication security.
I am trying to get the config server running for mongo shard using the following command and config file:
mongod --config /etc/mongod2.conf
mongod2.config
# Where and how to store data.
storage:
dbPath: /var/lib/mongodb2
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
processManagement:
pidFilePath: /srv/mongodb/db2.pid
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# network interfaces
net:
port: 27019
bindIp: 127.0.0.1
processManagement:
fork: true
#security:
# authorization: enabled
replication:
replSetName: configReplSet
sharding:
clusterRole: configsvr
When I run the command above, I get the following output (with no log output in the mongo logs):
about to fork child process, waiting until server is ready for connections.
forked process: 14246
ERROR: child process failed, exited with error number 1
I am running mongo v3.2.2 on ubuntu v14. The port 27019 is not bound by another process.
The issue was caused since I never created a separate data directory. I saw better error messages when I ran the following:
mongod --dbpath /var/lib/mongodb2 --port 27019
This command gave better error messages (it showed that the data directory was not present etc.). This helped solve the issue and I was able to run the config server (output below):
2016-06-29T12:21:30.216-0400 I CONTROL [initandlisten] MongoDB starting : pid=16060 port=27019 dbpath=/var/lib/mongodb2 64-bit host=abcdefghi
2016-06-29T12:21:30.216-0400 I CONTROL [initandlisten] db version v3.2.0
I have the issue with running MongoDB 3.0 service in Windows 7.
I have created the data/db and data/log inside MongoDB root and configured mongod.conf file:
Edited
# mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: C:\Program Files\MongoDB\data\log\mongod.log
# Where and how to store data.
storage:
dbPath: C:\Program Files\MongoDB\data\db
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger:
# how the process runs
processManagement:
fork: true
pidFilePath: C:\Program Files\MongoDB\mongod.pid
# network interfaces
# net:
# port: 27017
# bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces.
#security:
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
Then I have created a windows service which will run the mongodb on startup:
sc.exe create MongoDB binPath= "C:\MongoDB\bin\mongod.exe
--service --config=C:\MongoDB\mongod.conf" displayname= "MongoDB 3.0 Standard"
start= "auto"
I have tried also:
sc.exe create MongoDB binPath= "\"C:\MongoDB\bin\mongod.exe\" --service --config=\"C:\MongoDB\mongod.conf\"" DisplayName= "MongoDB" start= "auto"
MongoDB service properties show me path to executable:
C:\MongoDB\bin\mongod.exe --service --config=C:\MongoDB\mongod.conf
Attempts to run service(net start MongoDB) show me error message:
The service is not responding to the control function.
More help is available by typing NET HELPMSG 2186.
I know this issue is quite popular but most of solutions led to wrong paths, file names/extensions which don't solve my problem.
UPDATE:
Suggestion to create windows service under specified user doesn't help me as well (the same message during service start):
C:\Users\Administrator>sc.exe create MongoDB binPath= "C:\MongoDB\bin\mongod.exe
--service --config=C:\MongoDB\mongod.conf" displayname= "MongoDB 3.0 Standard"
start= "auto" obj= ".\Administrator" password= "#gdgsfg1"
Moreover, I have tried to specify username and password via services.msc -> MongoDB -> Properties->Log on
Windows Event log:
Error 6.05.2015 19:54:25 Service Control Manager 7009 None
A timeout was reached (30000 milliseconds) while waiting for the MongoDB service to connect.
Error 6.05.2015 19:54:25 Service Control Manager 7000 None
The MongoDB service failed to start due to the following error:
The service did not respond to the start or control request in a timely fashion.
I have found the solution why MongoDB service doesn't start successfully.
First of all, install windows mongodb service by this way to see all kind of errors with your YAML config file or anything else during service install(sc.exe doesn't provide enough ifnormation during mongodb service install):
mongod.exe --config "C:\Program Files\MongoDB\mongod.cfg" --install
Actually, my issue was with processManagement.fork option which is not exist in Windows since it causes error Unrecognized processManagement.fork option. So, I have remove this option from my config file and service starts fine.
The final config file:
systemLog:
destination: file
logAppend: true
path: C:\Program Files\MongoDB\data\log\mongod.log
timeStampFormat: iso8601-utc
storage:
dbPath: C:\Program Files\MongoDB\data\db
journal:
enabled: true
processManagement:
pidFilePath: C:\Program Files\MongoDB\mongod.pid
net:
port: 27017
bindIp: 127.0.0.1