vertx-mongo: Max number of operations (maxWaitQueueSize) of 500 has been exceeded.

[gillbates]

Hi all,

during the development with mongo and vertx, we happened to meet a problem by failed setting the value of maxWaitQueueSize.
i.e, it seemed the vertx-mongo hasnot utilize the pooling for the connections to mongo, which is implemented in the official mongo java driver.

JsonObject config = new JsonObject();
config.put("host", "127.0.0.1").put("port", 27017).put("db_name", "btc").put("waitQueueMultiple", 10000);

Using the following code with mongo default setup, the vertx-mongo will throw

com.mongodb.MongoWaitQueueFullException: Too many operations are already waiting for a server. Max number of operations (maxWaitQueueSize) of 500 has been exceeded.
	at com.mongodb.connection.BaseCluster.createWaitQueueFullException(BaseCluster.java:384)
	at com.mongodb.connection.BaseCluster.notifyWaitQueueHandler(BaseCluster.java:431)
	at com.mongodb.connection.BaseCluster.selectServerAsync(BaseCluster.java:142)
	at com.mongodb.binding.AsyncClusterBinding.getAsyncClusterBindingConnectionSource(AsyncClusterBinding.java:75)
	at com.mongodb.binding.AsyncClusterBinding.getReadConnectionSource(AsyncClusterBinding.java:65)
	at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:446)
	at com.mongodb.operation.FindOperation.executeAsync(FindOperation.java:546)
	at com.mongodb.async.client.MongoClientImpl$2.execute(MongoClientImpl.java:123)
	at com.mongodb.async.client.OperationIterable.batchCursor(OperationIterable.java:132)
	at com.mongodb.async.client.OperationIterable.into(OperationIterable.java:64)
	at com.mongodb.async.client.FindIterableImpl.into(FindIterableImpl.java:169)
	at io.vertx.ext.mongo.impl.MongoClientImpl.findWithOptions(MongoClientImpl.java:267)
	at io.vertx.ext.mongo.impl.MongoClientImpl.find(MongoClientImpl.java:249)
	at com.xxx.play.test.MongoWaitQueueFull.main(MongoWaitQueueFull.java:20)

we've attached a sample code to reproduce the issue, anyone may have a look or having the same issue?

sample code:

public class MongoWaitQueueFull {
    public static void main(String[] args) {
        JsonObject config = new JsonObject();
        config.put("host", "127.0.0.1").put("port", 27017).put("db_name", "btc").put("waitQueueMultiple", 10000);
        Vertx vertx = Vertx.vertx();
        MongoClient mongoClient = MongoClient.createShared(vertx, config);
        for (int i = 0; i < 1000; i++) {
            Future<List<JsonObject>> future = Future.future();
            mongoClient.find("tick", new JsonObject(), future.completer());
            future.setHandler(r -> {
                if (r.succeeded()) {
                    System.out.println("find success");
                } else {
                    System.out.println(r.cause());
                }
            });
        }
        vertx.setTimer(100000, l -> System.out.println("close"));
    }
}

ps: It seemed that the vertx-mongo failed provide pooling for developers or am I using in in a wrong way? Otherwise, we don't have to manually specify this value to increase the size ...

[karianna]

Fixed by #113 - Thanks @johnoliver

@gillbates 3.5.0 won't be released for sometime but you can compile from source and get the patched binary

[gillbates]

Actually I think the implementation of our pool is somehow buggy maybe? Otherwise, we don't have to change this value as we use the original mongo driver?

[gaurav-m]

Hi @karianna @gillbates

I am still facing this issue after setting waitQueueMultiple in mongo client config.
500 limit doesn't change.

Also the doc on https://vertx.io/docs/vertx-mongo-client/java/ says

waitQueueMultiple
The maximum number of waiters for a connection to become available from the pool. Default value is 500

Although from the code in PR (https://github.com/vert-x3/vertx-mongo-client/pull/113/files) it appears that this int value is a multiplier, and not the exact value.

[karianna]

@gaurav-m Can you open a new issue for this please?

[gaurav-m]

@karianna #218
thanks)

[MosheAvdiel]

Hello, any update on this issue? I'm unable to set this value (waitQueueMultiple).
How to update the connection pool settings?
Using Vertx and MongoClient with json options.
Thanks in advance.