[Mongodb][OpsManager]Mongodb secondary instances are not adding to replica set - mongodb

Question

I have created 3 mongodb instances with below mongod.conf [Primary1,Secondary1,Secondary2]
net:
port: 27017
bindIp: 0.0.0.0
replication:
replSetName: Replica1
I have started the 3 instance using " sudo service mongod start "
If I connect primary server to other 2 servers using mongo –host "ip" –port 27017..its working and connected.
Issue 1:
After I do rs.initiate() and rs.conf()
Secondary 1 is included in members but when I use rs.status() but showing stateStr" : "STARTUP"
And If I check its log means below error :
"msg":"Failed to reap transaction table","attr":{"error":"NotYetInitialized: Replication has not yet been configured"}}
Issue 2:
Because of issue 1 if I try to rs.add() for Secondary 2 ,its not working
rs.status() reponse
{
"_id" : 0,
"name" : "primary:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
.......
}
{
"_id" : 1,
"name" : "secondary1:27017",
"health" : 1,
"state" : 0,
"stateStr" : "STARTUP",
"uptime" : 1579,
"optime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDurable" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2021-01-08T04:47:35.455Z"),
"lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"configVersion" : -2,
"configTerm" : -1
}
Log Output of secondary 1's mongodb.log
{"t":{"$date":"2021-01-08T04:39:16.634+00:00"},"s":"I", "c":"CONNPOOL", "id":22576, "ctx":"ReplNetwork","msg":"Connecting","attr":{"hostAndPort":"primary:27017"}}
{"t":{"$date":"2021-01-08T04:39:18.004+00:00"},"s":"I", "c":"CONTROL", "id":20714, "ctx":"LogicalSessionCacheRefresh","msg":"Failed to refresh session cache, will try again at the next refresh interval","attr":{"error":"NotYetInitialized: Replication has not yet been configured"}}
{"t":{"$date":"2021-01-08T04:39:18.004+00:00"},"s":"I", "c":"CONTROL", "id":20712, "ctx":"LogicalSessionCacheReap","msg":"Sessions collection is not set up; waiting until next sessions reap interval","attr":{"error":"NamespaceNotFound: config.system.sessions does not exist"}}
{"t":{"$date":"2021-01-08T04:39:36.634+00:00"},"s":"I", "c":"CONNPOOL", "id":22576, "ctx":"ReplNetwork","msg":"Connecting","attr":{"hostAndPort":"primary:27017"}}

Answer 1

Issue got fixed. Primary database was able to connect to secondary hosts but in return secondary host can't. So I adjusted the security groups and connected. Takeaway is all the primary and secondary hosts should be able to connect with each other in mongodb port (27017)

Answer 2

Same issue happened for me , networks and everything is fine . Issue with host entry missing for primary server, still secondary was trying to connect DNS name
"ctx":"ReplNetwork","msg":"Connecting","attr":{"hostAndPort":"MONGODB01:27717"}}
Immediately have added into host entry everything is synced