New feature: "Network Connection Pool" now is stable

99 views
Skip to first unread message

Luca Garulli

unread,
Jun 23, 2011, 3:31:50 PM6/23/11
to orient-database
Hi all,
today I've released a new feature in OrientDB that helps a lot when you've a multi-thread client that use a remote connection to a OrientDB server: the Network Connection Pool! This was implemented about 1 month ago, but the beta-test period is finished only today, so I'm too excited to tell you about it. And this fixes also the problem about 100% of CPU consumed by the client app reported in this ML.

I've tested a lot today on a multi-core machine and the improvement is quite high. A massive insertion of 1,000,000 of records executed by 5 concurrent threads against a remote database on the same machine (localhost). The full source code is http://code.google.com/p/orient/source/browse/trunk/tests/src/test/java/com/orientechnologies/orient/test/database/speed/LocalCreateDocumentMultiThreadIndexedSpeedTest.java

These the times:

maxPool = 1 (default) : 272 secs
maxPool = 3 : 228 secs
maxPool = 5 : 215 secs

And below how to use it ;-)

By the ways it's present in the 1.0rc3-SNAPSHOT just released on MAVEN repository.

Thanks a lot to Emanuele Tagliaferri (the creator of one of two C drivers) for the fundamental support about this.

Lvc@

Network Connection Pool

Each client, by default, uses only one network connection to talk with the server. Multiple threads on the same client share the same network connection pool.

When you've multiple threads could be a bottleneck since a lot of time is spent on waiting for a free network connection. This is the reason why is much important to configure the network connection pool.

The configurations is very simple, just 2 parameters:

  • minPool, is the initial size of the connection pool. The default value is configured as global parameters "client.channel.minPool" (see parameters)
  • maxPool, is the maximum size the connection pool can reach. The default value is configured as global parameters "client.channel.maxPool" (see parameters)

At first connection the minPool is used to pre-create network connections against the server. When a client thread is asking for a connection and all the pool is busy, then it tries to create a new connection until maxPool is reached.

If all the pool connections are busy, then the client thread will wait for the first free connection.

Example of configuration by using database properties:


database = new ODatabaseDocumentTx("remote:localhost/demo");
database
.setProperty("minPool", 2);
database
.setProperty("maxPool", 5);

database
.open("admin", "admin");


Curtis Stanford

unread,
Jun 23, 2011, 4:22:15 PM6/23/11
to orient-...@googlegroups.com
Will this help with multiple local connection threads?

Luca Garulli

unread,
Jun 23, 2011, 4:26:23 PM6/23/11
to orient-...@googlegroups.com
No,
this is only for remote connections. But local now works well even in multi-threads contexts (tested up to 200 threads!)

Lvc@

Curtis Stanford

unread,
Jun 23, 2011, 4:37:51 PM6/23/11
to orient-...@googlegroups.com
Sorry I meant connections on the same thread. I just checkout the latest SVN and can't create two connections at the same time on the same thread. It get confused because of the ThreadLocal instance.

Luca Garulli

unread,
Jun 23, 2011, 4:58:06 PM6/23/11
to orient-...@googlegroups.com
Yeah,
I found a way to overcome it, maybe tomorrow could be released.

Lvc@

Curtis Stanford

unread,
Jun 23, 2011, 5:04:50 PM6/23/11
to orient-...@googlegroups.com
You are amazing. Thanks!
Reply all
Reply to author
Forward
0 new messages