Configure mongodb property maxWaitQueueSize in Spring boot application? - mongodb

Question

I get the error com.mongodb.MongoWaitQueueFullException: Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 500 has been exceeded. while doing a stress test on my application.
So I am thinking of configuring the maxWaitQueueSize property via configuration.
I am using spring boot to configure mongodb connection. I am using #EnableAutoConfiguration in my Application and I have declared only spring.data.mongodb.uri=mongodb://user:password#ip:27017 in the application.properties file.
How do I configure the maxWaitQueueSize property with spring boot?
How do I decide a good value for the maxWaitQueueSize?

Answer 1

If you're using MongoDB 3.0+, you can set waitQueueMultiple in your mongouri :
spring.data.mongodb.uri=mongodb://user:password#ip:27017/?waitQueueMultiple=10
waitQueueMultiple is a number that the driver multiples the maxPoolSize value to, to provide the maximum number of threads allowed to wait for a connection to become available from the pool.
How do I decide a good value for the maxWaitQueueSize?
It's not directly related to MongoDB but you can read more about Pool Sizing in Hikari github wiki.

Answer 2

In com.mongodb.MongoClientURI, you can find the parameters which can be used in MongoClientOption.
if (key.equals("maxpoolsize")) {
builder.connectionsPerHost(Integer.parseInt(value));
} else if (key.equals("minpoolsize")) {
builder.minConnectionsPerHost(Integer.parseInt(value));
} else if (key.equals("maxidletimems")) {
builder.maxConnectionIdleTime(Integer.parseInt(value));
} else if (key.equals("maxlifetimems")) {
builder.maxConnectionLifeTime(Integer.parseInt(value));
} else if (key.equals("waitqueuemultiple")) {
builder.threadsAllowedToBlockForConnectionMultiplier(Integer.parseInt(value));
} else if (key.equals("waitqueuetimeoutms")) {
builder.maxWaitTime(Integer.parseInt(value));
} else if (key.equals("connecttimeoutms")) {
builder.connectTimeout(Integer.parseInt(value));
} else if (key.equals("sockettimeoutms")) {
builder.socketTimeout(Integer.parseInt(value));
} else if (key.equals("autoconnectretry")) {
builder.autoConnectRetry(_parseBoolean(value));
} else if (key.equals("replicaset")) {
builder.requiredReplicaSetName(value);
} else if (key.equals("ssl")) {
if (_parseBoolean(value)) {
builder.socketFactory(SSLSocketFactory.getDefault());
}
}

Answer 3

I am using spring boot starter webflux. This issue also happens.
I tried to add MongoClientFactoryBean. It doesn't work.
The whole application is located in https://github.com/yigubigu/webfluxbenchmark. I tried to test performance benchmark of webflux and original mvc.
#Bean
public MongoClientFactoryBean mongoClientFactoryBean() {
MongoClientFactoryBean factoryBean = new MongoClientFactoryBean();
factoryBean.setHost("localhost");
factoryBean.setPort(27017);
factoryBean.setSingleton(true);
MongoClientOptions options = MongoClientOptions.builder()
.connectionsPerHost(1000)
.minConnectionsPerHost(500)
.threadsAllowedToBlockForConnectionMultiplier(10)
.build();
factoryBean.setMongoClientOptions(options);
return factoryBean;
}

Answer 4

you can achieve this by injecting an object of MongoOptions to your MongoTemplate.

Answer 5

This maxQueueSize limit is computed here in the Java client source code :
https://github.com/mongodb/mongo-java-driver/blob/3.10.x/driver-core/src/main/com/mongodb/connection/ConnectionPoolSettings.java#L273
It is the product of maxConnectionPoolSize and threadsAllowedToBlockForConnectionMultiplier and hence can be modified through ?maxPoolSize= and ?waitQueueMultiple= in the connection URI.