[BUG?] gremlin-driver 3.2.0 - Opened connections above maximum pool size - Thread safe ?

[Anthony Perinot]

Hello,

We noticed an abnormal number of connections from the gremlin driver.

Here is a sample code able to leverage the issue :

public class App {
    private static final int RECONNECT_INITIAL_DELAY = 50;
   
    private static final String GREMLIN_HOST = "localhost";
    private static final int GREMLIN_PORT = 8182;
   
    private static final int MIN_POOL_SIZE = 2;
    private static final int MAX_POOL_SIZE = 8;

    public static void main(String[] args) {

        /* Client initialization */
        final Builder builder = Cluster.build().serializer(new GryoMessageSerializerV1d0(GryoMapper.build().create())).addContactPoint(GREMLIN_HOST).port(GREMLIN_PORT);

        builder.reconnectIntialDelay(RECONNECT_INITIAL_DELAY);
        builder.minConnectionPoolSize(MIN_POOL_SIZE);
        builder.maxConnectionPoolSize(MAX_POOL_SIZE);

        final Cluster cluster = builder.create();

        final Client client = cluster.connect();
        client.init();

        final String request = "g.V()";

        final ExecutorService executorService = Executors.newFixedThreadPool(100);

        for (int i = 0; i < 1000; ++i) {
            executorService.submit(new Runnable() {

                @Override
                public void run() {
                    try {
                        client.submit(request, null).all().get(500, TimeUnit.MILLISECONDS);
                    } catch (final TimeoutException | InterruptedException | ExecutionException | RuntimeException exception) {
                        System.err.println("Execution failed, an error occured: " + exception.getMessage());
                    }
                }
            });
        }
    }
}


By running this code and opening another terminal using a netstat -p | grep 8182 we can see the number of connections going above 8. (we saw around 150 sockets opened in our application)

So my question are :
* Is the gremlin driver thread safe ?
* If it is supposed to be thread safe, then this is an issue, otherwise, a misuse from me (Also maybe some documentation was missing ?).

Thank you for your time.

Regards,
Anthony PERINOT

[Stephen Mallette]

You should be able to use the driver as you are using it. I've run your code on a two different versions of TinkerPop and netstat doesn't show any more open connections than expected. I'm seeing just 2 open (or more if increase that lower bound) when I have a max of 8 configured. I guess I'm not able to recreate your problem. I did alter your script slightly so that I could run in from the Gremlin Console:

cluster = Cluster.build().minConnectionPoolSize(4).maxConnectionPoolSize(8).create()

client = cluster.connect()

client.init()

executorService = java.util.concurrent.Executors.newFixedThreadPool(100);

for (int i = 0; i < 1000; ++i) {

  executorService.submit{

    try {

      client.submit("1+1", null).all().get(500, java.util.concurrent.TimeUnit.MILLISECONDS);

    } catch (Exception exception) {

      System.err.println("Execution failed, an error occured: " + exception.getMessage());

    }

  }

}

but it's pretty much what you had. thoughts?

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/5fc88910-1afd-43c7-bc16-a6785d22d0e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anthony Perinot]

Weirdly enough, I'm not able to reproduce the problem when using your sample of code in the gremlin console.

Also, to be able to leverage the problem more easily, we can reduce the maxConnectionPoolSize and increase the thread pool.

from my previous code, replacing :

    private static final int MAX_POOL_SIZE = 4;

and

        final ExecutorService executorService = Executors.newFixedThreadPool(500);

When I run it, i can see 13 (amount depending on the race condition i guess) opened connections.

sudo netstat -p | grep 8182 | grep 46097 | wc -l
13

I'll try to investigate further next week to identify the root cause, and try to get another test case triggering the problem.
(My guess from now is that the bug is located in the ConnectionPool class)

[Stephen Mallette]

I tried this again with the settings you mentioned (plus others) tried complex traversals, etc and can't recreate the problem. Please let me know if you are able to learn any new information as you continue looking at this.

> (My guess from now is that the bug is located in the ConnectionPool class)

That is a good guess - if there is a problem, I would look there.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/161556e0-8bbf-4066-af92-5e8b8d277ffc%40googlegroups.com.