Database Connection and Connection Pooling in verticle

[Wolfgang]

I created a simple example with two verticals.

1. WebServer

2. DataAccess

The WebServer verticle communicates with the DataAccess verticle via the event bus. In order to make sure that I don't use up too many connections to the database and reuse connections I included a ConnectionPool in the DataAccess verticle which I save in a static field. Somehow it seems like there is always only one connection (re-)used. Even if I put very heavy concurrent load on the WebServer.

I am not sure about the class loader separation and concurrency model in vert.x. A verticle instance is only accessed by one thread at a time? So this means that all work is performed in serial order if I am configuring only 1 instance per verticle?

 

Maybe I am missing something ...  or do you have another recommendation for setting up a similar simple architecture (Webserver --> Database).

[Jaime Yap]

My understanding is that If you are using the webserver that is bundled with the vert.x distribution, you can actually spawn N instances of the verticle that creates said webserver (and if I understand the model right, will allocate a new thread per verticle instance), and that the webserver will share a single port and route connections across the N threads that you allocated.

 

So to answer your question, the webserver you can scale by adding multiple instances of the verticle. The DataAccess one you are on your own.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To view this discussion on the web, visit https://groups.google.com/d/msg/vertx/-/tcWH5mJhEF0J.
To post to this group, send an email to ve...@googlegroups.com.
To unsubscribe from this group, send email to vertx+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/vertx?hl=en-GB.

[Tim Fox]

When you're talking about "database" do you mean JDBC? Or some other kind of database?

Are you using a worker verticle? If you have some code to share would be easier to answer...

[Wolfgang]

Yes, I want to access a relational MySQL database.

I included some sample code here: https://gist.github.com/2823791

[Tim Fox]

Sorry, I missed that the first time :(

For blocking databases (e.g. JDBC), I'd recommend creating a verticle, each instance of which manages a single database connection.

Don't use statics to share a database connection pool - this won't work anyway since each verticle instance runs in its own classloader so you'll actually end up with each instance having its own pool.

Then, create an instance of work queue (see modules manual), and instantiate a single instance of that. Instantiate X instances of your database verticle (tune X to see where best performance lies). These X instances then form your "pool".

Have each instance of your verticle register itself with the work queue.

Then clients just send messages to the work queue, and this will distribute work among the available database verticles as they are available.

[Tim Fox]

And make sure you start your verticle as a worker (e.g. set worker to true in mod.json)

[Wolfgang]

I implemented the version with the queue worker and i receive the "incoming" message in my DataAccess instance "worker". 

My WebServer also get the "status:ok" after registering with the queue worker in the reply Handler so far the setup of the worker-queue and my "worker" instance is fine.

But unfortunately the reply messages from the DataAccess "worker" instance are not received back in the WebServer once the query results are available and need to be provided in the response object. The callback handler is not executed ...

The manual says:

"When a processor receives some work, and has completed its processing. It should reply to the message with an empty reply. This tells the work queue that the work has been processed and can be forgotten about."

Does this mean that workers behind a worker-queue can't send "real" reply objects?

[Tim Fox]

Right, I would need to make a small change to allow the replies to get
back to the sender. Right now, the sender gets a reply when the work has
been *accepted* by the queue, not when the work has been processed.

> --
> You received this message because you are subscribed to the Google
> Groups "vert.x" group.
> To view this discussion on the web, visit

> https://groups.google.com/d/msg/vertx/-/Ka-r7x4ZrgEJ.

> To post to this group, send an email to ve...@googlegroups.com.
> To unsubscribe from this group, send email to
> vertx+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/vertx?hl=en-GB.

--
Tim Fox

Vert.x - effortless polyglot asynchronous application development
http://vertx.io
twitter:@timfox

[Tim Fox]

Please add a github issue

> vertx+unsubscribe@googlegroups.com.

[Mario Gutierrez]

Is there an example eclipse or maven project that uses JDBC? I'm a node.js guy and just need to a working example that connects to MySQL to get started. Seems like this is a much needed vert.x example since most of the Java ecosystem is not async.

[Tim Fox]

I agree this is an important case, it's been on the TODO list for some time.

For now we don't have any specific JDBC module, however, you can write normal JDBC code (as you would in Java, say) and put that in a worker verticle where it's allowable to block, you can then communicate with that verticle via the event bus from event loop (non blocking) code. That should work for now.

The idea is, eventually we will provide a generic JDBC worker which can be used against any JDBC compliant database.