Google Groups

Taking Vert.x to the next level - Vert.x 2.0

Tim Fox Nov 21, 2012 5:12 AM
Posted in group: vert.x
During Devoxx I started to think about what we need to do to take Vert.x to the next level. There is currently a lot of interest in Vert.x that we need to build on, but there's also quite a lot we need to do to gain acceptance in the enterprise.

I would therefore like to start working in the next few weeks on the next major release - vert.x 2.0.

Some of the things I'd like to consider including in this release are:

1. Classloader refactoring.

Currently each verticle/module runs in its own classloader instance. The main reason for this is to provide isolation by preventing instances to share data using statics.

However this has problems including permgen getting easily exhausted as each instance has its own copy of classes including copies of classes from any other jars that it uses. This doesn't scale.

Other issues include not being able to put user defined classes in a Shared map and share them between module instances.

I'd therefore like to change the classloading module such that each module/verticle *type* (not instance) has its own classloader instance. This means classes would only be loaded for a module once, not once per instance. This is a more traditional model you'd find in most app servers.

The down side of this is users will be able to share data using static members between instances of the same module/verticle. But I think this is a price worth paying. (We can warn against this).

2. IDE integration.

We need good Eclipse/IntelliJ integration.

Personally I don't use IDEs for anything much more than editing files, but I'm probably in a minority.

We need the Gradle tasks to work properly for creating Eclipse/IntelliJ projects (currently they are broken).

We need an effortless way of easily testing Vert.x verticles/modules from within an IDE. I'm guessing we need an Eclipse + IntelliJ plugin here.

We need a way of effortlessly creating Vert.x projects from an IDE or command line in a "best-practice" project layout. We need to decide what the best practice is first!

3. Management / monitoring

Ops need to be able to manage Vert.x servers in production.

Pid has started some good work here including a GUI. I would like to see this in Vert.x 2.0

4. Multi-threaded workers

(This came up after some feedback from Chris Richardson @ VMware)

Writing some kinds of worker verticles that use traditional Java APIs such as JDBC is quite awkward since worker verticles are single threaded. It would be much simpler to be able to write a multi-threaded worker verticle which managed a JDBC pool directly. A multi-threaded verticle could receive messages on the same handler instance concurrently.

This wouldn't be commonly used and wouldn't be the default, but would be very useful in writing plugin modules for stuff like JDBC, other database stuff, mailers etc. 

5. Avoiding callback spaghetti.

It's a well known issue of asynchronous frameworks/platforms such as Vert.x/Node.js/Akka/Play etc that direct use of asynchronous APIs can result in so-called "callback spaghetti" and it becomes hard to reason about control flow.

Some frameworks (e.g. Play/Akka also stuff like kaolan-async JS) tackle this by providing some kind of "promises" or "deferreds" API which allows you to combine asynchronous actions into different control flows. We had something like this in earlier versions of Vert.x but I removed it because it was quite clunky.

Creating a good promises API is tricky and we'd actually have to produce multiple APIs - one for each language we support!

Another way of solving this problem is by using continuations/coroutines to provide an old-fashioned direct-style (i.e. logically synchronous) API, but which doesn't actually block OS threads. The problem with this is the JVM has no first class support for continuations so you have to do this via byte code manipulation. There are a few projects that handle this e.g. JavaFlow/Kilim but it's not clear to me at this point how feasible it would be to do this.

If we could use a continuations approach this could be provided in a module which uses the core API.

6. Separate language support out into modules. 

This would mean the core Vert.x installation would be small and you'd only download and install particular language support if you actually used it.

7. HA support.

I am leaning towards an Erlang-like supervisor model. But we need a solution here if we're to be seriously considered in many production environments.

8. Scala and Clojure Support.

9. Various other smaller stuff including:

* Some improvements to the core API, e.g. HttpClient
* Upgrade to latest version of SockJS
* Upgrade to Netty 4

If you have any more suggestions about stuff you really think should be included in Vert.x 2.0, please suggest away!

In particular I'm interested in hearing from people who have already used Vert.x in real projects, and the problems you found, so we can fix them for Vert.x 2.0