I'm trying to define custom timeout values when first establishing a connection with mongoose.connect(), but am seeing some strange results:
If I use basic options (without any timeouts specified), then everything works fine:
options = { server:{ auto_reconnect: true, } }
However, if I try to specify socketTimeoutMS (e.g. 5000ms), then the connection repeatedly times out.
options = {
server:{
auto_reconnect: true,
socketOptions:{
connectTimeoutMS : 30000,
socketTimeoutMS : 5000,
keepAlive : 1
}
}
}
However, despite the [Error: connection to xxx timed out] errors that I get, the application still works!
Can anyone explain this behaviour?
Other info:
Mongoose v3.8.12 (Native driver 1.4.5)
MongoDb Server v2.4.5
Connecting to server on localhost (Windows 7 64bit)
Related
We have 4 mongoDB servers of which the first one is currently the primary with 3 replicas. If I specify all 4 servers in the connection string it fails to connect at all, but if I just specify the first one it connects fine. This is bad because if the first server fails, it will not be able to connect.
This works:
mongodb://login:password#server1:27017/admin?readPreference=Primary
This does NOT work:
mongodb://login:password#server1:27017,server2:27017,server3:27017,server4:27017/admin?readPreference=Primary
Exception:
A timeout occured after 30000ms selecting a server using CompositeServerSelector{ Selectors = WritableServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 } }. Client view of cluster state is { ClusterId : "1", ConnectionMode : "Automatic", Type : "ReplicaSet", State : "Connected", Servers : [{ ServerId: "{ ClusterId : 1, EndPoint : "Unspecified/server1:27017" }", EndPoint: "Unspecified/server1:27017", State: "Disconnected", Type: "Unknown", HeartbeatException: "MongoDB.Driver.MongoConnectionException: An exception occurred while opening a connection to the server.
The service trying to connect runs on Kube.
Any idea why this would be?
you need to add the mode to the connection string: replicaSet=myRepl
I followed
DigitalOcean Mongodb Install
sudo ufw allow from your_other_server_ip/32 to any
I set your other server as 127.0.01 as I will be connecting it with express local device as localhost:3000
sudo ufw allow from 127.0.0.1/32 to any
and created admin user.
I have also updated mongodb.conf to
logappend=true
bind_ip = 127.0.0.1,139.**.*.**
port = 27017
How can I make connection with mongoose now.
I tried with a gui with ssh connection and it worked.
How can I connect it with HTTP URL.
EDIT - 1
I installed mongodb
https://www.digitalocean.com/community/tutorials/how-to-install-mongodb-on-ubuntu-18-04
And with
https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-mongodb-on-ubuntu-16-04#part-two-securing-mongodb
Step 3 — Testing the Remote Connection
I am getting
MongoDB shell version v3.6.3
Enter password:
connecting to: mongodb://139.**.*.***:27017/
MongoDB server version: 3.6.3
And running
> show users
{
"_id" : "testdb.testusr",
"user" : "testusr",
"db" : "testdb",
"roles" : [
{
"role" : "readWrite",
"db" : "testdb"
}
]
}
If i try to connect with mongoose with below code
var connectionString = "mongodb://testusr:testpwd#139.**.*.***:27017/testdb";
mongoose
.connect(connectionString, {
keepAlive: 1,
useUnifiedTopology: true,
useNewUrlParser: true,
})
.then(() => console.log('DB Connected!'))
.catch(err => {
console.log(`DB Connection Error: ${err.message}`);
});
I am getting below output
DB Connection Error: Server selection timed out after 30000 ms
I'm not a Go guy, just need to use a plugin written in Go and I'm having some trouble between plugin and MongoDB.
The error is:
server selection error: server selection timeout
current topology: Type: Unknown
Servers:
Addr: localhost:27017, Type: Unknown, State: Connected, Avergage RTT: 0, Last error: dial tcp 127.0.0.1:27017: connect: connection refused
exit status 1
My configuration:
time=“2019-09-03T16:29:35Z” level=debug msg=“Host: ip-XXX-XX-XX-XXX.sa-east-1.compute.internal”
time=“2019-09-03T16:29:35Z” level=debug msg=“Port: 27017”
time=“2019-09-03T16:29:35Z” level=debug msg=“Username: user”
time=“2019-09-03T16:29:35Z” level=debug msg=“Password: user123*”
time=“2019-09-03T16:29:35Z” level=debug msg=“DBName: dbBackend”
The plugin snippet that performs the connection:
addr := fmt.Sprintf("mongodb://%s:%s", m.Host, m.Port)
to := 60 * time.Second
opts := options.ClientOptions{
ConnectTimeout: &to,
}
opts.ApplyURI(addr)
if m.Username != "" && m.Password != "" {
opts.Auth = &options.Credential{
AuthSource: m.DBName,
Username: m.Username,
Password: m.Password,
PasswordSet: true,
}
}
client, err := mongo.Connect(context.TODO(), &opts)
if err != nil {
return m, errors.Errorf("couldn't start mongo backend. error: %s\n", err)
}
err1 := client.Ping(context.TODO(), nil)
if err1 != nil {
log.Fatal(err1) // error happens here
}
log.Debugf("MONGO CONNECTED")
m.Conn = client
return m, nil
I just can't realize why the mongo driver is looking on localhost if I'm setting the address of my mongoDB server.
EDIT 1
My db has replica set configured only to use change streams.
This is my RS configuration:
{
"_id" : "rs0",
"version" : 69559,
"protocolVersion" : 1,
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 0,
"host" : "localhost:27017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : 0,
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("5cf684c3c0db3f53727d1bb4")
}
}
Any help solving it appreciated.
Thanks
why the mongo driver is looking on localhost if I'm setting the address of my mongoDB server.
When mongo-go-driver's client is connecting to a MongoDB deployment, it will perform Server Discovery and Monitoring to discovers one or more servers (MongoDB being a distributed database by nature). One of the early steps is to begin monitoring the topology by invoking isMaster command on all servers. Based on the output of isMaster the client will try to contact those servers. In the case of Replica Set (your case), the client strives to connect to the primary server (from isMaster.primary).
However, the hostname address is not a Fully Qualified Domain Name (FQDN) to be resolvable from the client's machine. The client's machine trying to connect to localhost defined as the replica set primary, thus failed to make a connection. Also, this is why you're seeing a message status where current topology: Type: Unknown but State: Connected. It failed to discover the deployment topology even before able to select a server to execute the command (ping)
You can solve this by setting resolvable hostnames for the value of the members field in the replica set configuration. In addition, when possible, use a logical DNS hostname instead of an ip address, as this avoids configuration changes due to ip address changes.
You can change the replica set hostnames using rs.reconfig() i.e:
cfg = rs.conf()
cfg.members[1].host = "<RESOLVABLE HOSTNAME>:<PORT NUMBER>"
rs.reconfig(cfg)
In your case, where there's only one replica set member it's quite straight forward. However if you're in production mode and have more than one members you can follow the steps outlined in Change Hostnames in a Replica Set where there are two options:
Change Hostnames without disrupting availability
Change Hostnames at the same time (one-go)
Having said all the explanation above,
alternatively, as your replica set deployment is only one server (development mode) you can set the connection mode to direct via ClientOptions.SetDirect(). Which specifies whether the client should connect directly to a server instead of auto-discovering other servers in the cluster (although this means you have no redundancy) i.e.:
opts := options.ClientOptions{ ConnectTimeout: &timeoutVariable}
opts.SetDirect(true)
opts.ApplyURI(addr)
client, err := mongo.Connect(connect.TODO(), &opts)
I'm connecting to the mongo with reconnect options on the startup and using created db over the whole app.
var options = {
"server": {
"auto_reconnect": true,
"poolSize": 10,
"socketOptions": {
"keepAlive": 1
}
},
"db": {
"numberOfRetries": 60,
"retryMiliSeconds": 5000
}
};
MongoClient.connect(dbName, options).then(useDb).catch(errorHandler)
When I restart mongo server, driver reconnect successful. If I stop server and start it after a 30 second I get MongoError "topology was destroyed" on every operation. This 30 second seems to me is a default value for numberOfRetries = 5 and my given option doesn't have effect. Am I doing something wrong? How can I manage reconnection for a long time?
According to this answer, in order to fix this error, you should increase connection timeout in the options:
var options = {
"server": {
"auto_reconnect": true,
"poolSize": 10,
"socketOptions": {
"keepAlive": 1,
"connectTimeoutMS": 30000 // increased connection timeout
}
},
"db": {
"numberOfRetries": 60,
"retryMiliSeconds": 5000
}
};
I have a node-mysql pool configuration of
var db_init={
host : 'ip_address_of_GCS_SQL',
user : 'user_name_of_GCS_SQL',
password : 'password here',
database : 'db here',
supportBigNumbers: true,
connectionLimit:100
};
Pool was created using
GLOBAL.db_foobar = mysql.createPool(db_init);
I basically just left the connection on for a couple of hours and I saw this error reported by my connection.query Request (after getConnection of course):
prodAPI-104 (out): { status: 'Error',
prodAPI-104 (out): details: '[foobar_function]Error in query',
prodAPI-104 (out): err: '{ [Error: read ETIMEDOUT]\n code: \'ETIMEDOUT\',\n errno: \'ETIMEDOUT\',\n syscall: \'read\',\n fatal: true }',
prodAPI-104 (out): query: 'SELECT * FROM `foobar_table`;' }
Why is this happening? The MySQL in Google-Cloud-SQL didn't report a query taking too long to create so I dunno why this happened.
I suspect the reason is that keepalive is not enabled on the connection to the MySQL server.
node-mysql does not have an option to enable keepalive and neither does node-mysql2, but node-mysql2 provides a way to supply a custom function for creating sockets which we can use to enable keepalive:
var mysql = require('mysql2');
var net = require('net');
var pool = mysql.createPool({
connectionLimit : 100,
host : '123.123.123.123',
user : 'foo',
password : 'bar',
database : 'baz',
stream : function(opts) {
var socket = net.connect(opts.config.port, opts.config.host);
socket.setKeepAlive(true);
return socket;
}
});