Getting the ball rolling: Some thoughts about Vert.x 3.0

[Tim Fox]

Throwing some ideas out there...

I've been thinking recently about Vert.x 3.0. One thing that has become
clear to me over the past year or so, is that we have done some things
well in Vert.x, but other things not so well. I think it's been a
learning experience and I think we should take the opportunity with
Vert.x 3.0 to improve things.

Some of the proposals I'm going to make for Vert.x 3.0 are quite
radical. But I think it's necessary that we make some big changes if
we're going to continue to gain popularity, and take Vert.x to the next
level.

So here are my *main* proposals (there are lots of other smaller bits
and pieces, but I'm going to leave those for now)

I'd like to hear your comments on this, and if you have any more
suggestions about other improvements we can make in Vert.x 3.0, please
share them :)

1) Massively simplify the module system.

One of the purposes of the Vert.x module system is to provide a
consistent way for users to encapsulate and reuse Vert.x functionality
across different languages. Another feature of the module system is to
resolve module dependencies at run-time.

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Forcing all these different users to do things in a different Vert.x way
isn't attractive to these different communities, and has caused some
confusion.

I'm therefore proposing that instead of forcing a "one size fits-all"
module on Vert.x users, that we drop the idea of a module (i.e. a module
zip containing a mod.json) altogether and simply embrace whatever module
system is in common use by the relevant community. That way users from
different language communities can do things in the way they already
know without having to understand a different module format.

For Java users that means simply creating plain jars, and putting them
in Maven. For JavaScript users, putting them in npm. For Ruby users,
putting them in rubygems. etc.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a
nice feature, but in practice is it really worth it? Especially if we
resolve all dependencies at build time, Maven will catch any such clashes.

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Also, if a member of the community creates a module with a Java API,
it's a PITA to have to manually create wrappers for that API in
different languages. In many cases people don't bother, resulting in the
API being only usable in a single language.

I'm proposing that instead of manually maintaining API shims, we only
maintain the core APIs in Java and we provide a tool which generates API
shims for different languages. That way everything stays up to date with
little effort, and should be a huge win for language module maintainers.

It's not an easy task, but I believe it is possible and it means we
won't constantly be spending our time in the community with the tedious
job of keeping APIs up to date and we can spend our time on fun stuff
like innovating new functionality. We can also generate other language
API documentatiom using the same tool.

4) Automatic generation of location transparent proxies for services

Let's say we have a reusable component such as a database persistor.
Currently with Vert.x this will listen on the event bus and interact
with the rest of the world by receiving and sending messages.

However, it would be much nicer if the client could use a rich API, e.g.

api.store(object, {
// stored
})

instead of:

eventbus.send(address, storeMessage, {
// stored
}

I'm proposing that if a simple Java API is provided by the component,
that we can generate proxies that the client can use, which handle the
marshalling and marshalling of the data over the event bus automatically.

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

public void start() {
System.out.println("My verticle is starting");
vertx.createHttpServer().requestHandler(...).listen(8080);
}

public static void main(String[] args) {
Vertx vertx = VertxFactory.newVertx();
vertx.deployVerticle(new MyVerticle());
}

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

7) Reification of different dispatchers

Currently events in Vert.x are dispatched either on an event loop or a
worker thread depending on what type of verticle it is. It's currently a
bit clunky to configure parameters for the dispatchers, e.g. number of
threads in the pool. Also the worker pool is currently global to each
Vert.x instance.

By re-ifying the dispatchers you will be able to maintain multiple
different worker pools and configure them programmatically and also
configure event loops, e.g.

Dispatcher dispatcher = new EventLoopDispatcher(numberOfEventLoops);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

Dispatcher dispatcher = new WorkerPoolDispatcher(maxThreads, minThreads,
etc);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

you'll be able to provide your own dispatcher implementations that, say,
use a Java executor

Executor myExecutor = ...;
Dispatcher dispatcher = new ExecutorDispatcher(myExecutor);

vertx.deployVerticle(new MyVerticle(), dispatcher);

8) Java 8 only

I propose that we target Java 8 and later for Vert.x 3.0, so we have
access to all the new features (e.g. Lambas, functional stuff,
completablefuture)

[Jordan Halterman]

1) I think a flat class hierarchy would greatly simplify project management and lower the barrier to entry as we've all seen plenty of lengthy discussion on classloaders. I'm curious about what removing the module system will look like in projects that use multiple languages (at my work we use a pretty even mixture of Java, JS, Python, and eventually PHP in large projects). I have had my quibbles with the module system, but I always thought there was potential.

2) Controversial indeed. I like the current API :-)

3) having been developing and maintaining polyglot Vert.x APIs, I have often pondered the feasibility of such a feature. The event bus provides an API for a lot of projects, but I think this would make life much easier for a lot APIs built on top of Vert.x. I think it would be well worth the time and effort and would make life easier in open- and close-sourced projects. Ahh, that would be awesome!

4) It seems like a pretty standard Json event bus protocol has emerged, and it would be great to wrap that in some sort of API.

5, 6, 7, and 8) Yes, yes, yes, and yes!

> --
> You received this message because you are subscribed to the Google Groups "vert.x" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Rick Hight]

The module system is one of the things that attracted me to Vertx. That said I don't use it at all except as a sort of single war file type thingy. 

I just use Verticles. 

I like the idea of embracing maven/gradle instead of modules. We still need fat jars. We still need jars that have jars. Don't we?

So at least for the java case we need something like a war file and so far you have been calling that a module. :)

So it seems to me for Java you need something like a module. Nasty classloader junk... 

I guess I can just start only using fatJars. :) Wait... you will still have fatJars right?

inline....

So just have maven build an executable jar file with everything you need packaged up.. .

A microservice? (stupid name, but that is what the cool kids are calling it)... 

 

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

hmmmm.... 

 

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

cleaner? More Ruby, JS, Dart like.. a hell of lot less Java-ish. :)

IDK Seems like heresy. Time to get out the pitch forks. 

** I like that.

 

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

So a standard wire protocol and a standard marshaling protocol?

One that will adapt to what type of client is on the other end?

The protocol gets generated from the Java interfaces.

 

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

+1

I've written JSON marshalers for doing this kind of stuff with my vertx projects so I can send messages between verticles.

It is in Boon. 

It needs a rewrite. 

https://github.com/RichardHightower/boon/wiki/Method-marshaling-by-list-with-Boon---important-for-JSON-method-marshaling

http://rick-hightower.blogspot.com/2014/02/added-respondsto-invoke-from-ruby-to.html

Boon JSON converts things to a list or Map of Value objects which are semi-parsed JSON thing-ma-bobs which are called index overlays.

I can take a JSON call and push it on a bus and invoke in another tier. 

The invoker needs improvements as does the object mapper, but it does some crazy complex mapping and it does it pretty damn fast.

http://rick-hightower.blogspot.com/2014/04/groovy-and-boon-provide-fastest-json.html

For another project I allows strongly typed maps to be marshaled as message headers on the vertx bus (I don't use JSON, I use a binary stream).

Anyway... I like the idea.

I still owe you a pull request. 

I can split the JSON parser, invoker, etc. out and donate it with improvements for speed and code coverage. 

 

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

I have been pitching Vertx as a solution for clients who were interested in OSGi (trying to talk them down).

This is an about face, but seems to be the prevailing wind in the industry the so called micro-services.

Once you have some decent marshaling between tiers... doesn't the value of modules go up?

So you have a high-speed binary marshaling between two Java tiers... you can send message in-memory, in-process PDQ with the cost of some serialization, but still faster than leaving the process space.... Then you have modules with separate names-space jar files all deployed in one VM... but maybe that just does not matter at all. Maybe the answer is IPC between processes... Your modules are just other JVM processes that happen to be on the same machine or not. Doesn't Java 8 support unix domain sockets? Maybe the answer to modules is never do it.. just use fatJars and marshal calls.... keep them course grained micro services... sorry... I digress. I think it is a good direction to explore.

 

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

:)

<bean id="foo.VerticleMain">

    <property name="holySh1tReallyStill?"> ...

@Inject

int shootMe

Sounds awesome.

 

[Paulo Lopes]

In a way i do agree with this since it makes users more at home when using other languages than Java but i can also see this as a source of new problems. e.g.: I am a JS developer and have developed some module for node.js and published it on npm, if vert.x advertises that supports npm modules there is a big chance that my module does not work on vert.x because I've native code or use some V8 specific stuff that rhino/nashorm does not support and even the API is not compatible (which forces another hidden dependency in nodyn).

Other potential issue is "native bindings", for example in Yoke I support JS but I don't code most of the stuff in JS but in Java where I create native Rhino objects for better performance and better integration with the API how does one package such module? It should not end up on npm since it is not compatible at all with node.js but how does it handle the internal jar that should be in the classpath in order to the remain JS code to work?

 

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

It is an interesting approach but for a Java developer this does not feel natural and there is always the chance of errors since we are not relying on the compiler to validate the method but on strings and resolving linkage at runtime. I'd rather have the strings replaced with enums in java and keep strings for other languages where it feels natural like JS and Ruby. 

I like the idea (i've implemented it on mod-redis manually) but could you detail more the marshalling stuff? I mean will the event bus use something else than JSON? I am still not happy with the current JSON api and if this will bring more types to what can go over the bus, i am all for it. (still cannot use mod-mongo-persistor since cannot use dates to filter collections or create self expiring indexes for example...) 

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

My personal preference is to go with JDK APIs such as CompletableFutures in order to avoid having to many dependencies on my platform that I might not even use. 

Yay! 

[田传武]

On Mon, Apr 14, 2014 at 12:45 AM, Tim Fox <timv...@gmail.com> wrote:

3) Automatic generation of other language APIs

4) Automatic generation of location transparent proxies for services

I am curious about how to implement automatic generation. I wrote a native java eventbus over vertx's RawWebSocketTransport, translated the java code to Objective-C for the iOS platform using J2ObjC, and translated to javascript using GWT. Is there any similar way to do this?

[JC]

1) agree to this, for me the module system - while reasonable and functional - was still just "another" module system to learn. Who wants different classloaders can still start different instances and there it is. However the caveat above is very valid: how could one avoid wrong expectations about these fresh vertx-only libraries?

2) as a mainly Java programmer I feel this as a step backwards - I'm used to rely on compile time checks for once. Here less is not more (for me).

3) sounds good as idea, but I guess it will depend a lot on the actual implementation to say whether it's good or not.

4) I'm not so sure about this, to me the actual "send" API sounds better as it leaves to the module the flexibility to implement different or new actions. You might land otherwise needing two APIs for the same module, eww... If only Json encoding is the culprit (and I agree it is a small annoyance) why not just making pluggable encoders easier to plug in?

5) 6) 7) interesting indeed and in spirit of the time.

8) even for those not needing/using them?

[Simone Scarduzio]

Great ideas Tim, thanks for sharing. I have some comments inline:

I really agree with this point, including dependencies is a hard process to streamline and definitely you don't want to solve this problem twice in your project.

Maven, Gem, Npm are amazing tools and it makes all the sense of the world to keep on using them also for vert.x related dependencies.

This does not mean vert.x dependencies should suffer fragmentation: something gets published in npm, but not in maven, etc.

There's definitely room for automation here: if I write a useful db connector for Vert.x, I should be able submit it and see it published in multiple formats so that other developers can install it with Npm, Gem or Maven.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a
nice feature, but in practice is it really worth it? Especially if we
resolve all dependencies at build time, Maven will catch any such clashes.

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This is the most IDE unfriendly thing ever. If you're going to go down this path, I think we should have at least an Enum representing the event (Event.CONNECT, Event.CLOSE, ..) in place of the raw string. I'm pretty sure this is a step backwards not only for Java, but for anybody using an IDE.  

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

3) Automatic generation of other language APIs

BRILLIANT.

This is so much needed. I believe at this stage, anybody that has put together a non trivial DB-backed Vert.x project, either is not using the Vert.x db connector module or has written a small internal API that takes care of (un)marshalling the JSON messaging part.

  

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

Reactive extensions is a well defined interface and is already been ported to almost all languages Vert.x supports. 

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

YES YES YES YES PLEASE YES :)

I'm 100% on this. Imagine maintaining two versions of the example code syntax for Java in the documentation pre and post Java8.. Let's just stick to Java8 syntax. Every example code will look so much simpler and more elegant.

[Witold Szczerba]

8 times "yes"!

I would add to this: do everything what's possible to make Vert.x
compatible with Node.js. I know not everything can be done in this
area, especially when it comes to native extensions, but there are
many use cases where "as much as possible" would be fine.

Example: we are creating application with modules in different
technologies and languages. Some parts are in JEE (especially those
which require XA Data sources), then we have some Python and finally
Vert.x and Node.js. We have lot of code we would like to share between
Node.js and Vert.x, but now it is not possible or very limited. Stupid
things like lack of setTimeout or setImmediate exclude libraries like
"Q" which we use everywhere. Other use cases include using superb
testing infrastructure like Mocha test runner, Chai expectations,
Sinon mocks and stubs... It would be awesome to be able to use it
within Vert.x with NPM. Our solution for now is to cease Vert.x
development and switch to Node.js, as it's much more matured and full
of modules.

Regards,
Witold Szczerba

[Jonathan Wagner]

1) Seems like a good idea.  Not sure about implications for users actively using a variety of languages, it might require a pile of different tools to build stuff.  Flat classloader will certainly solve a variety of problems for me.  

2)  Kind of ambivalent about this one.   

3)  +1 here.  I have previously had to deal with projects where I maintained bindings for lots of languages and generation is certainly the way to go when you can.  If we are able to come up with a *REALLY GOOD* way to do the docs across languages, that would in itself be a pretty epic tool in itself.

4)  I have written something like this and have been using it for a bit.  You can see sort of a basic example here: https://gist.github.com/tigeba/10653070  I would be happy to make the code available/contribute it if this approach seems suitable.  It handles serialization of all basic types + JsonObject and has support for pluggable serializers for other random objects.  One issue I will raise is that for this to work well it seems that you also need to deal with the issue of service location.  I picked up a copy of "Distributed Algorithms for Message-Passing Systems" over Christmas but still haven't found time to read it, so my Byzantine Generals still have problems if you know what I mean..

5) +1.  Been using Rx a bit here and there as well.

6,7 sound good.

8) My only concern here is that Android will continue to lag behind and we won't be able to use Vert.x on Android as a result.

[bytor99999]

One word. Perfect. 

So when will it be done?

Mark

[steven citron-pousty]

Wait you mean it's not being released this week ;)

--

[Danny Kirchmeier]

1) Yes. At my work, we have a private maven server where we host all of 

our modules and download them using gradle. We don't use the vertx module 

system.

2) No. I love IDE completion and hate having to remember the names for 

different events. If this proposal is just because you do not want to 

copy the multiple handler definitions to several classes, you could

have a wrapper handler container like so:

httpServer.on.connect(handler);

httpServer.on.disconnect(handler);

4) YES. As it stands in my projects, I create facade services which

provide an API interface and simply delegate to the eventbus. I would

love for these to be automatically generated somehow, especially by

vertx.

5) Yes.

[Amit Kumar]

Does (1) by any chance mean, I will not be able to run different versions (possibly with different version dependencies) of my module at the same time ?

[Anindya Chatterjee]

Nice and interesting features to have, except no. 2. Remembering the name would be a pain and would almost become impossible for a big project. 

No. 1 is the most simple and powerful feature. It will help to avoid lot many confusions as normal java developers are accustomed with a single classloader normally. It will also attract more use of DI frameworks like Spring. It will help debugging a lot.

No. 7 is also powerful. It will provide the developers with a great amount of control over execution.

One more idea to improve vertx experience, is to prefer java objects to pass as a message over Json by default. I mean json should be there under the hood, but automatic marshalling and unmarshalling should abstract it away, because java has very poor language support for json unlike javascript. It would be a lot easier that way to write code for vertx in java . Same thing can be done for other languages as well where there is poor native json support.

Introducing message interceptor api would be a great feature to have. Just some interfaces; user will implement their own custom interceptors based on their need.

It would also be nice to introduce SpEL like expression languages. It will help to write a lot cleaner code.

Thanks for reading this.

Regards,

Anindya Chatterjee

[Frank Reiter]

On Apr 13, 2014 9:45 AM, "Tim Fox" <timv...@gmail.com> wrote:

> instead of
>
> httpServer.connectHandler(handler);
>
> we do
>
> httpServer.on("connect", handler);

I like that the compiler catches typos in the first (current) version.

> 8) Java 8 only

+1

Frank.

[Frank Reiter]

+1 to what Paulo writes below.
Frank.

[Joern Bernhardt]

1) Massively simplify the module system.

If this means that we are forced to have a Java database connector, a JS database connector, a Ruby database connector, a Scala database connector, a Python/PHP/yeti/whatever database connector... I won't sign this.

I like to write modules in a non-Java language where I most probably don't have a code generator for other languages. Do I need to update/recompile the module with every single Vert.x update then, so it stays compatible? Maybe the simplified API you propose will simplify this, but I'm not 100% sure about the implications. To get rid of mod.json and all of that would be great, but using the Gradle/Maven template is not too bad either (even though it changes too often ;)).

What will happen if you have one module using Scala for Vert.x 3.0 and another module using Scala for Vert.x 3.1 - if these are incompatible, you won't be able to use them together? We can something like this with the current module system, can't we?

Generally about the module system: NodeJS is split into two parts: node and npm. node runs all the things and npm tackles the module system around it. There is a lot of stuff that npm does, what vertx doesn't right now. For example when installing modules, it can add this as a dependency into your package.json (the mod.json equivalent). It will also grab and install this into a sub-directory called node_modules/<module_name> where it stores the modules own dependencies again, effectively creating the ClassLoader we should have in Vert.x right now: Every module has its own classes/dependencies and doesn't need to rely on others.

I don't really see the need for Maven or Gradle when the code is structured like this, you could just copy the whole directory and run it like a "vertx fatjar". Yes, this means it results in modules that may have the same dependency included multiple times and there is probably a lot of room for improvement. But it works quite well and is much easier to understand.

Maybe it would be easier for Vert.x to go a similar route and provide a CLI that you can use to download libraries from maven into your module, add other modules as dependencies into mod.json, etc.

 

2) Refactoring of the core interfaces.

I consider the IDE problem a valid one, but I think using strings here is especially cool if you want to be able to add custom events. Using strings would help here a quite a bit as it lets us reuse a lot of error handling code for example. So +1 on the string proposal, even though I'm an IDE user, too.

 

3) Automatic generation of other language APIs

Sounds great - not sure whether this will work out well in practice, but in theory +1.

 

4) Automatic generation of location transparent proxies for services

So all modules need to be written in Java from now on?

Or can I at least write the core functionality in Scala and provide a simple "to be translated into other languages"-Java-wrapper? If we don't kick the mod.json completely but add our API in there, this could be easier to maintain a Java wrapper for most. Something like this: https://gist.github.com/Narigo/710226847a7f2f73220d

 

5) Promises/Future/Rx support

At least for Scala it would be nicer to have the native Scala Future/Promise types. I'm not sure what Rx is adding if Java8 also provides Futures?

The magical API generator can hopefully generate into native Future/Promise types, too? If yes, +1.

 

6) No differentiation between "core" and "platform".

Sounds nice, +1.

 

7) Reification of different dispatchers

Guess this works better for people in need, so +1. I just hope that the documentation will have a lot of information about this and how to use it, so I can understand it at some point, too. ;)

 

8) Java 8 only

Yes please.  

One additional suggestion:

Currently there are a few async drivers popping up relying on either Netty or Akka. I have to admit I really don't understand too much about the underlying threading that needs to be done there, but it would be really really great if Vert.x could somehow provide a way to use these drivers in a simple way. Right now, using Netty drivers is not really simple and modules around them need to use some implementation details that shouldn't be used directly. If Vertx could provide a VertxEventLoopGroup (or whatever Netty drivers need) and a VertxActorSystem (Akka drivers usually want an ActorSystem), so these drivers can be used with a simple event bus wrapper, this would lead to a lot of new async drivers.

I've talked with Norman about this already and for Netty, this won't work currently. He said there might be a chance to get there with Netty 4.1, but he wasn't sure about this. It would be nice to have this on the radar as we could have access to a lot of real async drivers then. Most of the currently available modules really are worker modules or using ugly hacks to get async drivers to work (Normans suggestion for the MySQL/PostgreSQL module is to create a single threaded NioEventLoopGroup now, meaning the module creates a new thread for each instance of it). I'm actually not aware of any 100% async driver written in Vert.x yet...?

[bytor99999]

"2. Remembering the name would be a pain and would almost become impossible for a big project. "

But it wouldn't be different for a big or small project. These names are internal to Vert.x connectHandler == "connect", dataHandler = "data", closeHandler = "close".

Those aren't names of your addresses or something that make up. The are a finite list from Vert.x

"One more idea to improve vertx experience, is to prefer java objects to pass as a message over Json by default. "

What if you aren't writing in Java. JSON was chosen because it works for all languages regardless of direct support. We are using Groovy and dealing with adapting JSON to Java objects and back is a NON-issue for us because we get the JSON immediately and as a Map that we can use dot notation and use it immediately. If you prefer Java objects instead, then you just negated a huge benefit of Groovy.

Now what I would support is a way to pass Java "domain" objects instead of having to convert them back and forth between JsonObject or the like for Java developers, but not at the cost of other languages.

Mark

[dgo...@squaredfinancial.com]

1) Massively simplify the module system.

I'm at least a little wary of (1) anyway.  The vertx platform part and its lightweight (but existent) module system were definitely a reason we went with vertx in particular.  It going away is not something I anticipated (though maybe I should have given moving under the eclipse umbrella?).

Our current vertx-based internal project is structured as 10+ vertx modules (plus third-party deps like yoke).    It's not the end of the world if it is dropped (even leaving aside the "don't upgrade to vertx 3" option), but I imagine it would probably go something like "hello, osgi. <sigh...>" and a sweeping restructuring, rather than "yay let's juggle classpaths like it's 1999".

> Another purpose of the Vert.x module system is to provide isolation of  module classes. This allows different modules to use different versions
> of the same jars at the same time in the same JVM instance. This is a  nice feature, but in practice is it really worth it?

...erm... as it happens we specifically wanted something that worked the way vertx worked...

If the vertx-land answer became "well use eclipse virgo if you want that sort of thing" (or something, I completely stopped looking into it when we found vertx), that's the way it goes I guess, just saying the vertx platform with its module system was a positive for us that just worked, certainly not something we were fighting with particularly. 

> I'm therefore proposing that instead of forcing a "one size fits-all"  module on Vert.x users, that we drop the idea of a module (i.e. a module  zip containing a mod.json)
> altogether and simply embrace whatever module  system is in common use by the relevant community.

Hmm. I'm not sure about this either.  I was up until now regarding vertx modules as a vertx platform feature, with the language-level stuff like pip (python) or npm (javascript) at a "different level" altogether (except in the case of java itself, sortof, though even a java module of course can have non-module deps bundled into it), with vertx quite agnostic about them.

The way I imagined it working for the python/jython case in conjunction with the existing vertx 2 way of doing things was using pip* during some part of the vertx module build - to install python deps right into the vertx module (yes, pip install takes alternate paths. it looks like npm does too). Or put another way, a vertx module would be acting a bit like a prepopulated intra-jvm python virtualenv (which is after all kinda what it is doing for us for java stuff too).

That's still not a hard argument against abandoning vertx modules as such - just replacing them with osgi bundles doing similar duty is very probably feasible, I imagine a similar strategy would work there for python/jython i.e. install the python dependencies into the bundle.

... Anyway, sorry if that seemed a bit contrary.  Losing the vertx platform part and its module system for us definitely means having to replace it with something else, not nothing.

(* or currently more manually, though it looks like pip will soon properly work with jython: http://sourceforge.net/p/jython/mailman/message/32174800/ )

[Vinay Samudre]

On Tuesday, April 15, 2014 8:03:57 AM UTC+10, Anindya Chatterjee wrote:

Nice and interesting features to have, except no. 2. Remembering the name would be a pain and would almost become impossible for a big project. 

No. 1 is the most simple and powerful feature. It will help to avoid lot many confusions as normal java developers are accustomed with a single classloader normally. It will also attract more use of DI frameworks like Spring. It will help debugging a lot.

No 4. - will make vert.x development lot lot simpler  

No. 7 is also powerful. It will provide the developers with a great amount of control over execution.

One more idea to improve vertx experience, is to prefer java objects to pass as a message over Json by default. I mean json should be there under the hood, but automatic marshalling and unmarshalling should abstract it away, because java has very poor language support for json unlike javascript. It would be a lot easier that way to write code for vertx in java . Same thing can be done for other languages as well where there is poor native json support.

+1 

Introducing message interceptor api would be a great feature to have. Just some interfaces; user will implement their own custom interceptors based on their need.

+1 

[Dean Pehrsson-Chapman]

Our current vertx-based internal project is structured as 10+ vertx modules (plus third-party deps like yoke).    It's not the end of the world if it is dropped (even leaving aside the "don't upgrade to vertx 3" option), but I imagine it would probably go something like "hello, osgi. <sigh...>" and a sweeping restructuring, rather than "yay let's juggle classpaths like it's 1999".

+1

I can see why people find the module system complex, but the truth is it's a lot simpler than the alternatives, and being part of the platform makes the whole infrastructure simpler. It's possible that the people using vert.x are not enterprise (because enterprise adoption lags) and this is more of an enterprisey feature.

Rather than drop it totally, is there any way it could be made optional?

+1 for interceptors

[petermd]

1) Massively simplify the module system. 

Even though I’m one of those who struggled (struggles) with the class loader / isolation early on I’d still be worried that you’re throwing out the proverbial baby here. The threading model of VertX works in part because a lot of the common ways to subvert it are prevented by the class loader design. Using a traditional shared classloadeder may quickly introduce a lot of equally problematic thread-safety issues.

That said, I do think there’s a lot that could be done to assist debugging issues with class-loaders and conflict resolution.

2) Refactoring of the core interfaces. 

I prefer strongly typed interfaces, unless the type of every handler is the same you’ll end with more run-time type conflicts on Handler parameters which are easy to create / hard to debug.

3) Automatic generation of other language APIs 

Only concern is the combinatorial explosion if you have other wrappers / non-runnable jars / frameworks. RxJava does this but each API shim has the same classname so you are limited to one lang per class loader (module class loader could help here). This is a little bit more restrictive but makes writing utility code possible with needing to use messy inheritance (where every lang inherits from java e.g.)

4) Automatic generation of location transparent proxies for services 

For general module interop there’s lots of existing libs for this already (protocol buffers, thrift, bson, ObjectMapper etc etc) not sure the core needs to introduce a new one. It would be good if the EventBus allowed modules to provide their own encoded message (e.g. json/bson etc) that could be read by other lang modules without them needing a custom codec though.

I do think there is a problem with drivers for third-party services (esp. databases with complex formats) but I’m personally averse to generated stubs as they tend to promote breaking the wire protocol (because its too easy to make inadvertent breaking changes) and create more scope for compatibility conflicts between modules that use the same persistor.

It also maintains the double indirection where the DB module inevitably has to decode the message and re-encode using the DB native protocol using its own client lib. This leads to problems with loss of data typing or structure (issues around dates etc). If the app itself uses Bson on the network then you can have a chain of lossy transformations (network bson -> vertx api -> (event bus) vertx encoding -> (persister) vertx pojo -> db encoding) when it fact the original object could most likely be encapsulated directly into the DB request (subject to suitable schema validation).

As Joern says encouraging an async driver model (esp those written on netty) to separate protocol encoding from the network transport so they could be used independently would provide a lot of potential benefits. Developers can use the native async drivers (with proper typing) but the underlying networking would use VertX scheduler to ensure the threading model is maintained. Obviously its non-trivial to implement but it would be effort shared across all async platforms so more likely to gain support from the DB Vendors / Contributors?

5) Promises/Future/Rx support 

Sounds great to me! Obviously the way RxJava does code-gen would become a factor (language class loader isolation etc). There is a whole other set of issues with Future/Promises libs that do their own internal threading that could benefit from platform support. Currently grappling with this for rxvertx.

7) Reification of different dispatchers 

Having to size/configure thread pools is one of the problems I was hoping to avoid with an async platform :-)

[Tim Fox]

On 13/04/14 12:55, Rick Hight wrote:

The module system is one of the things that attracted me to Vertx. That said I don't use it at all except as a sort of single war file type thingy. 

I just use Verticles. 

I like the idea of embracing maven/gradle instead of modules. We still need fat jars. We still need jars that have jars. Don't we?

So at least for the java case we need something like a war file and so far you have been calling that a module. :)

So it seems to me for Java you need something like a module. Nasty classloader junk... 

I guess I can just start only using fatJars. :) Wait... you will still have fatJars right?

Yeah, no plans to get rid of fatjars.

[Tim Fox]

When I suggested NPM I didn't mean that our modules would use the node.js API, just that NPM might be a place where we could put Vert.x JS modules. I'm not sure if that will work though,

Other potential issue is "native bindings", for example in Yoke I support JS but I don't code most of the stuff in JS but in Java where I create native Rhino objects for better performance and better integration with the API how does one package such module?

I'd say using Rhino specific stuff is problematic. This stuff is going to break anyway as Nashorn gets more popular.

[Tim Fox]

On 14/04/14 23:41, Joern Bernhardt wrote:

1) Massively simplify the module system.

If this means that we are forced to have a Java database connector, a JS database connector, a Ruby database connector, a Scala database connector, a Python/PHP/yeti/whatever database connector... I won't sign this.

Quite the opposite ;) Generation means you would just have to maintain a single Java module and everything else is generated for you.

I like to write modules in a non-Java language where I most probably don't have a code generator for other languages. Do I need to update/recompile the module with every single Vert.x update then, so it stays compatible? Maybe the simplified API you propose will simplify this, but I'm not 100% sure about the implications. To get rid of mod.json and all of that would be great, but using the Gradle/Maven template is not too bad either (even though it changes too often ;)).

What will happen if you have one module using Scala for Vert.x 3.0 and another module using Scala for Vert.x 3.1 - if these are incompatible, you won't be able to use them together?

In practice though, how many people really do that. IMHO we're adding a lot of pain for a feature that is rarely used.

We can something like this with the current module system, can't we?

Generally about the module system: NodeJS is split into two parts: node and npm. node runs all the things and npm tackles the module system around it. There is a lot of stuff that npm does, what vertx doesn't right now. For example when installing modules, it can add this as a dependency into your package.json (the mod.json equivalent). It will also grab and install this into a sub-directory called node_modules/<module_name> where it stores the modules own dependencies again, effectively creating the ClassLoader we should have in Vert.x right now: Every module has its own classes/dependencies and doesn't need to rely on others.

I don't really see the need for Maven or Gradle when the code is structured like this, you could just copy the whole directory and run it like a "vertx fatjar". Yes, this means it results in modules that may have the same dependency included multiple times and there is probably a lot of room for improvement. But it works quite well and is much easier to understand.

Maybe it would be easier for Vert.x to go a similar route and provide a CLI that you can use to download libraries from maven into your module, add other modules as dependencies into mod.json, etc.

 

2) Refactoring of the core interfaces.

I consider the IDE problem a valid one, but I think using strings here is especially cool if you want to be able to add custom events. Using strings would help here a quite a bit as it lets us reuse a lot of error handling code for example. So +1 on the string proposal, even though I'm an IDE user, too.

 

3) Automatic generation of other language APIs

Sounds great - not sure whether this will work out well in practice, but in theory +1.

 

4) Automatic generation of location transparent proxies for services

So all modules need to be written in Java from now on?

Or can I at least write the core functionality in Scala and provide a simple "to be translated into other languages"-Java-wrapper? If we don't kick the mod.json completely but add our API in there, this could be easier to maintain a Java wrapper for most. Something like this: https://gist.github.com/Narigo/710226847a7f2f73220d

 

5) Promises/Future/Rx support

At least for Scala it would be nicer to have the native Scala Future/Promise types. I'm not sure what Rx is adding if Java8 also provides Futures?

The magical API generator can hopefully generate into native Future/Promise types, too? If yes, +1.

 

6) No differentiation between "core" and "platform".

Sounds nice, +1.

 

7) Reification of different dispatchers

Guess this works better for people in need, so +1. I just hope that the documentation will have a lot of information about this and how to use it, so I can understand it at some point, too. ;)

 

8) Java 8 only

Yes please.  

One additional suggestion:

Currently there are a few async drivers popping up relying on either Netty or Akka. I have to admit I really don't understand too much about the underlying threading that needs to be done there, but it would be really really great if Vert.x could somehow provide a way to use these drivers in a simple way. Right now, using Netty drivers is not really simple and modules around them need to use some implementation details that shouldn't be used directly. If Vertx could provide a VertxEventLoopGroup (or whatever Netty drivers need) and a VertxActorSystem (Akka drivers usually want an ActorSystem), so these drivers can be used with a simple event bus wrapper, this would lead to a lot of new async drivers.

I've talked with Norman about this already and for Netty, this won't work currently. He said there might be a chance to get there with Netty 4.1, but he wasn't sure about this. It would be nice to have this on the radar as we could have access to a lot of real async drivers then. Most of the currently available modules really are worker modules or using ugly hacks to get async drivers to work (Normans suggestion for the MySQL/PostgreSQL module is to create a single threaded NioEventLoopGroup now, meaning the module creates a new thread for each instance of it). I'm actually not aware of any 100% async driver written in Vert.x yet...?

[S C]

On Thu, Apr 17, 2014 at 4:11 PM, Tim Fox <timv...@gmail.com> wrote:

On 14/04/14 23:41, Joern Bernhardt wrote:

What will happen if you have one module using Scala for Vert.x 3.0 and another module using Scala for Vert.x 3.1 - if these are incompatible, you won't be able to use them together?

In practice though, how many people really do that. IMHO we're adding a lot of pain for a feature that is rarely used.

Even then, there's nothing stopping you to start another vertx instance and cluster them. Voila, incompatible modules talking over the bus. Or am I getting it completely wrong?

S

[Tim Fox]

On 15/04/14 19:46, dgo...@squaredfinancial.com wrote:

1) Massively simplify the module system.

I'm at least a little wary of (1) anyway.  The vertx platform part and its lightweight (but existent) module system were definitely a reason we went with vertx in particular.  It going away is not something I anticipated (though maybe I should have given moving under the eclipse umbrella?).

Our current vertx-based internal project is structured as 10+ vertx modules (plus third-party deps like yoke).    It's not the end of the world if it is dropped (even leaving aside the "don't upgrade to vertx 3" option), but I imagine it would probably go something like "hello, osgi. <sigh...>"

I'm not suggesting using another module system, and certainly not OSGI.

and a sweeping restructuring, rather than "yay let's juggle classpaths like it's 1999".

> Another purpose of the Vert.x module system is to provide isolation of  module classes. This allows different modules to use different versions
> of the same jars at the same time in the same JVM instance. This is a  nice feature, but in practice is it really worth it?

...erm... as it happens we specifically wanted something that worked the way vertx worked...

If the vertx-land answer became "well use eclipse virgo if you want that sort of thing" (or something, I completely stopped looking into it when we found vertx), that's the way it goes I guess, just saying the vertx platform with its module system was a positive for us that just worked, certainly not something we were fighting with particularly. 

> I'm therefore proposing that instead of forcing a "one size fits-all"  module on Vert.x users, that we drop the idea of a module (i.e. a module  zip containing a mod.json)
> altogether and simply embrace whatever module  system is in common use by the relevant community.

Hmm. I'm not sure about this either.  I was up until now regarding vertx modules as a vertx platform feature, with the language-level stuff like pip (python) or npm (javascript) at a "different level" altogether (except in the case of java itself, sortof, though even a java module of course can have non-module deps bundled into it), with vertx quite agnostic about them.

The way I imagined it working for the python/jython case in conjunction with the existing vertx 2 way of doing things was using pip* during some part of the vertx module build - to install python deps right into the vertx module (yes, pip install takes alternate paths. it looks like npm does too). Or put another way, a vertx module would be acting a bit like a prepopulated intra-jvm python virtualenv (which is after all kinda what it is doing for us for java stuff too).

That's still not a hard argument against abandoning vertx modules as such - just replacing them with osgi bundles doing similar duty is very probably feasible, I imagine a similar strategy would work there for python/jython i.e. install the python dependencies into the bundle.

... Anyway, sorry if that seemed a bit contrary.  Losing the vertx platform part and its module system for us definitely means having to replace it with something else, not nothing.

(* or currently more manually, though it looks like pip will soon properly work with jython: http://sourceforge.net/p/jython/mailman/message/32174800/ )

[Dean Pehrsson-Chapman]

On Thursday, 17 April 2014 15:17:44 UTC+1, Tim Fox wrote:

On 15/04/14 19:46, dgo...@squaredfinancial.com wrote:

Our current vertx-based internal project is structured as 10+ vertx modules (plus third-party deps like yoke).    It's not the end of the world if it is dropped (even leaving aside the "don't upgrade to vertx 3" option), but I imagine it would probably go something like "hello, osgi. <sigh...>"

I'm not suggesting using another module system, and certainly not OSGI.

I think the author's point was that those who found the module system useful would need to use another module system like OSGI, instead of vert.x's simple one.

[Tim Fox]

I'm interested in understanding what feature(s) of the Vert.x module system would be missed...

[Mumuney Abdlquadri]

Hi Tim,

I use vert.x modules system to separate functionality. I never got OSGI, Eclipse module system? not enough documentation. Netbeans module system ok but more complex than vertx.

Please dont drop the module system.

Regards.

--

[Tim Fox]

On 17/04/14 09:28, Mumuney Abdlquadri wrote:

Hi Tim,

I use vert.x modules system to separate functionality.

Aren't plain jars enough to "separate functionality"?

[Mumuney Abdlquadri]

No. that is why we OSGI and the rest...

If using jars, how do I undeploy a functionality from a management console.

[Tim Fox]

On 17/04/14 09:51, Mumuney Abdlquadri wrote:

No. that is why we OSGI and the rest...

If using jars, how do I undeploy a functionality from a management console.

The proposal I've made wouldn't affect your ability to deploy / undeploy stuff. Instead of deploy/undeployModule you can just use deploy/undeployVerticle

[Mumuney Abdlquadri]

But I still want a way to mark certain verticles as being in a group(module) and treat them as a unit.

[Mumuney Abdlquadri]

Still on modules. If we drop the idea of module.

How does my Java vertx app deployed in maven depend on a Javascript vertx app deployed npm?

[Tim Fox]

On 17/04/14 10:04, Mumuney Abdlquadri wrote:

But I still want a way to mark certain verticles as being in a group(module) and treat them as a unit.

Surely, jars accomplish that?

[Tim Fox]

On 17/04/14 10:07, Mumuney Abdlquadri wrote:

Still on modules. If we drop the idea of module.

How does my Java vertx app deployed in maven depend on a Javascript vertx app deployed npm?

If you mean "How can I deploy a JavaScript Vert.x app from Java", the answer is you can just use deployVerticle pretty much like you do today.

[S C]

Well not quite, I can today just deploy a module by name and not bother what language it's written in or for. I can't imagine doing that across dependency managers.
S

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

[Rick Hight]

I want to take another crack at this.

Inline.

On Sunday, April 13, 2014 9:45:17 AM UTC-7, Tim Fox wrote:

Throwing some ideas out there...

I've been thinking recently about Vert.x 3.0. One thing that has become
clear to me over the past year or so, is that we have done some things
well in Vert.x, but other things not so well. I think it's been a
learning experience and I think we should take the opportunity with
Vert.x 3.0 to improve things.

Some of the proposals I'm going to make for Vert.x 3.0 are quite
radical. But I think it's necessary that we make some big changes if
we're going to continue to gain popularity, and take Vert.x to the next
level.

So here are my *main* proposals (there are lots of other smaller bits
and pieces, but I'm going to leave those for now)

I'd like to hear your comments on this, and if you have any more
suggestions about other improvements we can make in Vert.x 3.0, please
share them :)

1) Massively simplify the module system.

Not sure. We need something like a war file that has jar files, and Java code. This was the mod-zip.

Since OSGi is a pretty much a complete failure, Vertx modules are attractive.

In a world of micro-services, you deploy each "module" as its own micro-service.

Verticals really are the important concept here, not modules. 

Java needs a way to ship a unit of app that includes the jar files for that app.

fatJar could be that unit. 

If you can explain more about how you can replace this functionality.

What does a vertx world look like without modules.

Does this mean that vertx is just geared towards microservices?

So each module just deploys its own vertx instance?

You communicate by wiring up instances in the same cluster?

One concern is speed. What is the cost of every call to another mirco-service if that call (and by call, I mean posting a message on the boss I suppose) has to go over the TCP/IP stack.

http://www.cl.cam.ac.uk/research/srg/netos/ipc-bench/details/tmpwoNtJN.html

There seems to be very little difference between UDS and TCP/IP, but mempipe seems to be 2x to 4x.

The point being... the cost of micro-service vs. module is the cost of IPC. 

So with microservice vs. module you have an vertx instance (JVM really) per microsevice.

Am I totally off base. I'd love some clarifying comments.

 

One of the purposes of the Vert.x module system is to provide a
consistent way for users to encapsulate and reuse Vert.x functionality
across different languages. Another feature of the module system is to
resolve module dependencies at run-time.

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Forcing all these different users to do things in a different Vert.x way
isn't attractive to these different communities, and has caused some
confusion.

I'm therefore proposing that instead of forcing a "one size fits-all"
module on Vert.x users, that we drop the idea of a module (i.e. a module
zip containing a mod.json) altogether and simply embrace whatever module

system is in common use by the relevant community. That way users from
different language communities can do things in the way they already
know without having to understand a different module format.

For Java users that means simply creating plain jars, and putting them
in Maven. For JavaScript users, putting them in npm. For Ruby users,
putting them in rubygems. etc.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a
nice feature, but in practice is it really worth it?

Worth it? Can't really live without it? Are you proposing a move from modules to pure microservices?

How do you get past this issue?

Sorry.. I know I am being a Java grandpa here, but I don't understand.

Without modules. I will need a vertx instance per service or I will need to make sure that every jar I deploy is using the same versions of gak.

In most Java shops I work with... projects resemble archeology. There was this project developed six years ago that uses Spring 2.x and Hibernate 3, and now the latest versions that use Spring 11, and Hibernate 14. The reason they have war files is so they don't stomp on each other.

In reality... they have war files and then each service has its own Tomcat instance so the war files are redundant. 

So I want to be clear... Are you proposing a microservice approach, in which case modules are redundant since each service gets its own instance anyway.

(I hate the terms like microservice since it is just a term for what people mostly/actually do. But giving something a name, NoSQL, Reactive, AJAX seems to help so I will use this name with distaste).

 

Especially if we
resolve all dependencies at build time, Maven will catch any such clashes.

????

To build what... a mod-zip? Maven builds jar files. You have maven build mod-zip. Maven can also build war files.

Your comment makes sense in the mod-zip sense and in the fat jar sense, but not in the jar sense.

If you don't build modules, what are you building with maven?

fatjar?

mod-zip?

something new?

 

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

As long as it is easy to use from gradle and/or maven... I think it is already the Java way.

I can't speak to other languages. Java is my home turf. 

 

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

Please no. If you have to do that.. Please at least have on take an enum or a string. 

httpServer.on(CONNECT, handler); 

 

httpServer.on("connect", handler); 

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Also, if a member of the community creates a module with a Java API,
it's a PITA to have to manually create wrappers for that API in
different languages. In many cases people don't bother, resulting in the
API being only usable in a single language.

I'm proposing that instead of manually maintaining API shims, we only
maintain the core APIs in Java and we provide a tool which generates API
shims for different languages. That way everything stays up to date with
little effort, and should be a huge win for language module maintainers.

It's not an easy task, but I believe it is possible and it means we
won't constantly be spending our time in the community with the tedious
job of keeping APIs up to date and we can spend our time on fun stuff
like innovating new functionality. We can also generate other language
API documentatiom using the same tool.

No opinion.

 

4) Automatic generation of location transparent proxies for services

Let's say we have a reusable component such as a database persistor.
Currently with Vert.x this will listen on the event bus and interact
with the rest of the world by receiving and sending messages.

However, it would be much nicer if the client could use a rich API, e.g.

api.store(object, {
   // stored
})

instead of:

eventbus.send(address, storeMessage, {
   // stored
}

I'm proposing that if a simple Java API is provided by the component,
that we can generate proxies that the client can use, which handle the
marshalling and marshalling of the data over the event bus automatically.

+1 I spent some cycles improving my JSON marshaling code yesterday.

:)

https://github.com/RichardHightower/boon/commit/1d5f2da5eb865038be555e538c659a59a51567ba

 

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

+1

This is something I have been thinking about for years and years.

 

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

I think you just want a subset that is RX like and then let someone else add/contribute RX modules.

Vertx has to be careful not to jump the shark. 

RxJava is too big.

Take the 20% that makes sense and leave the rest on the table. IMHO.

 

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

+1

 

7) Reification of different dispatchers

Currently events in Vert.x are dispatched either on an event loop or a
worker thread depending on what type of verticle it is. It's currently a
bit clunky to configure parameters for the dispatchers, e.g. number of
threads in the pool. Also the worker pool is currently global to each
Vert.x instance.

By re-ifying the dispatchers you will be able to maintain multiple
different worker pools and configure them programmatically and also
configure event loops, e.g.

Dispatcher dispatcher = new EventLoopDispatcher(numberOfEventLoops);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

Dispatcher dispatcher = new WorkerPoolDispatcher(maxThreads, minThreads,
etc);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

you'll be able to provide your own dispatcher implementations that, say,
use a Java executor

Executor myExecutor = ...;
Dispatcher dispatcher = new ExecutorDispatcher(myExecutor);

vertx.deployVerticle(new MyVerticle(), dispatcher);

Damn. I wish I understood what you meant.

http://en.wikipedia.org/wiki/Reification_(computer_science)

More words. 

This sounds pretty advanced. Hopefully there will be an easy button. 

The one thing I love about the vertx model is that the constructs are basic and easy.

Push something here and it comes out here.

 

8) Java 8 only

I propose that we target Java 8 and later for Vert.x 3.0, so we have
access to all the new features (e.g. Lambas, functional stuff,
completablefuture)

And long before that we convert the examples to Java 8. :)

Add some things...

TOML for config.

Boon for JSON parsing (or a Boon merge). 

As soon as you accept my pull request for Hazelcast 3.2, I am going to work on a pull request for Boon. 

Then after that I am going to brainstorm some ideas for marshaling.

:)

 

[Tim Fox]

On 17/04/14 10:29, S C wrote:

Well not quite, I can today just deploy a module by name and not bother what language it's written in or for. I can't imagine doing that across dependency managers.

Not sure I understand what you mean by "across dependency managers", but if you're asking how would one deployVerticles from Java, JS etc I was thinking something like this:

vertx.deployVerticle(javaClassname)

vertx.deployverticle(commonJSModulename)

etc

So it's not really so different from today

[Rick Hight]

On Thursday, April 17, 2014 10:12:43 AM UTC-7, Tim Fox wrote:

On 17/04/14 10:04, Mumuney Abdlquadri wrote:

But I still want a way to mark certain verticles as being in a group(module) and treat them as a unit.

Surely, jars accomplish that?

No. No they don't. fatJars do. mod-zip does. A jar is just a jar. It does not mean it will run. It can depend on other jars which are not present or that are present but are the wrong version.

fatJar does solve this but it is fat. 

mod-zip does solve this, but you might be getting rid of it.

Most Java projects have dependencies on average 20 jars.

There has to be a way to package up an app or microservice as a single unit.

fatJar does this. mod-zip does this. A jar file does NOT do this.

[S C]

The problem is to FIND and fetch them. Starting is simple, but somebody has to do the resolution. And when you have both npm and maven how will they talk to each other to figure out where it's your module, in the java or in the javascript world?

[Rick Hight]

My wish list for Vertx 3.0.

• Micro-service architecture in - modules out
• With more modules, verticles are the center of the vertx universe
• Embrace Gradle/Maven/RubyGem/NPM (actually that is not really my wish list.. the first two are... the other two are just regurgitating what Tim said)
• High-speed IPC (In a world without modules, processes need to talk to each other very quickly)
• Object marshaling between verticles and micro-services (callbacks, Reification, micro-service to address resolution)
• TOML or TONL for config https://github.com/mojombo/toml
• Embrace JDK 8 API
• RxJava support pulled into core (either 20% pareto of RxJava features or more integration)
• Ability to more easily upgrade switch clustering support (so you can have Vertx 3 but Hazelcast can be upgraded independently, this is mostly there but not quite. Vertx 2 for example could support both 2.x and 3.x Hazelcast)

On Sunday, April 13, 2014 9:45:17 AM UTC-7, Tim Fox wrote:

Throwing some ideas out there...

I've been thinking recently about Vert.x 3.0. One thing that has become
clear to me over the past year or so, is that we have done some things
well in Vert.x, but other things not so well. I think it's been a
learning experience and I think we should take the opportunity with
Vert.x 3.0 to improve things.

Some of the proposals I'm going to make for Vert.x 3.0 are quite
radical. But I think it's necessary that we make some big changes if
we're going to continue to gain popularity, and take Vert.x to the next
level.

So here are my *main* proposals (there are lots of other smaller bits
and pieces, but I'm going to leave those for now)

I'd like to hear your comments on this, and if you have any more
suggestions about other improvements we can make in Vert.x 3.0, please
share them :)

1) Massively simplify the module system.

One of the purposes of the Vert.x module system is to provide a
consistent way for users to encapsulate and reuse Vert.x functionality
across different languages. Another feature of the module system is to
resolve module dependencies at run-time.

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Forcing all these different users to do things in a different Vert.x way
isn't attractive to these different communities, and has caused some
confusion.

I'm therefore proposing that instead of forcing a "one size fits-all"
module on Vert.x users, that we drop the idea of a module (i.e. a module
zip containing a mod.json) altogether and simply embrace whatever module
system is in common use by the relevant community. That way users from
different language communities can do things in the way they already
know without having to understand a different module format.

For Java users that means simply creating plain jars, and putting them
in Maven. For JavaScript users, putting them in npm. For Ruby users,
putting them in rubygems. etc.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a

nice feature, but in practice is it really worth it? Especially if we

resolve all dependencies at build time, Maven will catch any such clashes.

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Also, if a member of the community creates a module with a Java API,
it's a PITA to have to manually create wrappers for that API in
different languages. In many cases people don't bother, resulting in the
API being only usable in a single language.

I'm proposing that instead of manually maintaining API shims, we only
maintain the core APIs in Java and we provide a tool which generates API
shims for different languages. That way everything stays up to date with
little effort, and should be a huge win for language module maintainers.

It's not an easy task, but I believe it is possible and it means we
won't constantly be spending our time in the community with the tedious
job of keeping APIs up to date and we can spend our time on fun stuff
like innovating new functionality. We can also generate other language
API documentatiom using the same tool.

4) Automatic generation of location transparent proxies for services

Let's say we have a reusable component such as a database persistor.
Currently with Vert.x this will listen on the event bus and interact
with the rest of the world by receiving and sending messages.

However, it would be much nicer if the client could use a rich API, e.g.

api.store(object, {
   // stored
})

instead of:

eventbus.send(address, storeMessage, {
   // stored
}

I'm proposing that if a simple Java API is provided by the component,
that we can generate proxies that the client can use, which handle the
marshalling and marshalling of the data over the event bus automatically.

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

7) Reification of different dispatchers

Currently events in Vert.x are dispatched either on an event loop or a
worker thread depending on what type of verticle it is. It's currently a
bit clunky to configure parameters for the dispatchers, e.g. number of
threads in the pool. Also the worker pool is currently global to each
Vert.x instance.

By re-ifying the dispatchers you will be able to maintain multiple
different worker pools and configure them programmatically and also
configure event loops, e.g.

Dispatcher dispatcher = new EventLoopDispatcher(numberOfEventLoops);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

Dispatcher dispatcher = new WorkerPoolDispatcher(maxThreads, minThreads,
etc);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

you'll be able to provide your own dispatcher implementations that, say,
use a Java executor

Executor myExecutor = ...;
Dispatcher dispatcher = new ExecutorDispatcher(myExecutor);

vertx.deployVerticle(new MyVerticle(), dispatcher);

[Tim Fox]

On 17/04/14 10:40, S C wrote:

The problem is to FIND and fetch them.

I'm proposing that's done at build time, using e.g. Maven. I.e. no different to what 99% of Java applications do anyway.

[Rick Hight]

Wait! I get it. So you are saying that vertx will look at the pom.xml file in META-INF of the jar files and then just download everything you need for that jar file so it will run.

Ok... so if I deploy three jar files. And there are conflicts?

Do you really want that hassle?

Sounds like a PIA and if you think classloader issues and classloader isolations were an issue before wait until you have someone deploy two incompatible jar files to vertx and there are incompatibility of cglib.

If this is what you had in mind, I think it will set vertx back. 

IDK. Modules are looking pretty appealing all of a sudden.

[Tim Fox]

On 17/04/14 11:01, Rick Hight wrote:

Wait! I get it. So you are saying that vertx will look at the pom.xml file in META-INF of the jar files and then just download everything you need for that jar file so it will run.

No, I'm not suggesting Vert.x does any dependency management.

I'm just suggesting that dependency resolution happens at build-time not run-time using whatever tools are used as a standard in that community (e.g. Maven). It's really not a strange proposal - I'm simply proposing that we do things in the way that 99% of apps are build anyway.

[S C]

So I have my nice java project using maven and I want to pull the nice database driver written in JavaScript available with npm. Using modules - straightforward. Using dependency managers, how?

[Rick Hightower]

RE: No, I'm not suggesting Vert.x does any dependency management.

** Thankfully! (I orginally wrote thank God, but after reading your rants on twitter thought better of it).

RE: I'm just suggesting that dependency resolution happens at build-time not run-time using whatever tools are used as a standard in that community (e.g. Maven).

+1 +100 yes. yes. yes.

** Right. So produce a jar file. But a jar file does not run without dependencies. So... you must mean produce oneJar or some sort of fatJar. Right? jar file will not do shit.

RE: It's really not a strange proposal - I'm simply proposing that we do things in the way that 99% of apps are build anyway.

Sure it is. But strange is good. Think different. Or just.. Think. I like it.

99% of apps do not build giant jar files. 99% of java web apps build war files (ok maybe 90% but still). :) We are inherently the 1% so we are inherently strange. If you want to be bigger than 1%, we probably need a way to build a jar like thing that has all of the dependencies bundled in. You said you were going to continue to support fatJar so fatJar will get used a lot more I imagine because you need a way to bundle libs with the runnable jar.....

I am not arguing. I just want a little clarity. If you get rid of modules, we need a good no-module story, and building a jar file with maven aint it.

Vertx 2

 Build = build fatJar or mod-zip with maven or gradle

Run = deploy mod-zip and run module or java -jar mymodule.fatjar.jar

So you are suggesting this....

Vertx 3

 Build = jar with maven or gradle

 Run = ? how do I include all of the jar files that my verticle needs?

RE: whatever tools are used as a standard in that community (e.g. Maven). 

By this you must mean....

https://code.google.com/p/jarjar/

fatJar

http://one-jar.sourceforge.net/

https://code.google.com/p/onejar-maven-plugin/

IS THIS WHAT YOU MEAN?

Rick Hightower
(415) 968-9037
Profile 

[Lance Ball]

On Sunday, April 13, 2014 12:45:17 PM UTC-4, Tim Fox wrote:

Throwing some ideas out there...

1) Massively simplify the module system.

Yes, this sounds like a welcome change.

 

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Most definitely. From the point of view of JS developers, npm is definitely the way to go. For the nodyn project I wrote dynjs-npm[0], an npm module loader that conforms to node.js' API [1]. It was written with DynJS as the target Javascript runtime, but I think it should work just fine with Nashorn and Rhino as well - as long as there is an existing require() function in the global scope - which it falls back to if nothing is found in the typical npm locations. With Nodyn, I just load the npm require() function in the global scope when a verticle starts up [2]. It should be pretty simple to adapt this or something similar to the existing JS language modules.

[0] https://github.com/projectodd/dynjs-npm

[1] http://nodejs.org/api/modules.html 

[2] https://github.com/nodyn/nodyn/blob/master/src/main/java/org/projectodd/nodyn/NodeJSVerticleFactory.java#L50

 

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

Speaking as someone who writes a lot of javascript, I prefer the .on('some_event') style. The strongly typed handler API in Java bothered me a lot while I was shepherding the lang-js module. However, I can understand the Java developer's point of view regarding type checking. It's somewhat idiomatic to the language in many ways. An alternate approach may be to have untyped event handling in languages where that is the norm. Especially if those languages can support a dynamically dispatched style. For example, the SockJS bridge hooks I implemented in lang-js for vert.x 2.1 features take advantage of Javascript's fast and loose typing to provide SockJSServer.on() already [3]. Maybe the answer is that more dynamically typed languages could be encouraged to implement the API's handler styles in ways that are more idiomatic to the language itself.

 [3] http://vertx.io/mod-lang-js/docs/1.0.0/sockjs.SockJSServer.html

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Boy do we ever. :) 

I think this is an ambitious goal. Especially given the other goals above. For language modules to truly reflect the coding style of a given community, they really need to be tailored to the target language's current usage idioms and styles. I think it will be difficult to do that with a language generation tool. A generator may be good and useful for a fledgling language implementation, but I think it may not be the final step in a language module's production. If you can pull it off, though, more power to you.

 

4) Automatic generation of location transparent proxies for services

I don't have strong feelings on this, but it's an interesting idea. I wonder if it's maybe the first step in the creation of the code generating tool in 3). It kind of sounds like the same idea but with the initial source being 3rd party modules instead of vertx core. 

 

5) Promises/Future/Rx support

I have reservations about the magical API generator, so not much to add here.

 

6) No differentiation between "core" and "platform".

Speaking from a limited point of view, this would be nice for nodyn. With vertx being an embedded component, the platform bits just kind of get in the way. I want the nodyn experience to be as much like a node.js experience as possible for the end user, and the platform tends to change all of that.

 

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application. 

This is my usage scenario, so yes this would be an improvement for me. That said, I realize this isn't _everyone's_ usage scenario. I think there is a place for the vert.x platform, because not everyone is going to use its services simply as an embedded component. 

I guess I'm on both sides of this debate. Personally, I don't want or need the vert.x module system in nodyn and that's a big part of why I'm keeping vertx-platform.jar out of it for the time being. That said, if the module system is gone and platform is nothing more than deploy/undeploy then maybe it makes sense to smush them together.

 

7) Reification of different dispatchers

I'm curious to see where you go with this. It could be a very interesting feature that may really open up potential interoperability with 3rd party libraries/systems/whathaveyou. 

 

8) Java 8 only

Yes.

And, one more thing...

9) Published system stats/usage data. 

That's probably not the right title for it, but what I mean is, it would be nice to have an API that allowed me to query vertx and find out things like how many verticles are running and some information about them.

[Tim Fox]

On 17/04/14 11:45, S C wrote:

So I have my nice java project using maven and I want to pull the nice database driver written in JavaScript available with npm. Using modules - straightforward. Using dependency managers, how?

You'd add a dependency in your pom.xml to the database driver jar, just like you would in any Maven project.

[Lance Ball]

On Thursday, April 17, 2014 9:59:05 AM UTC-4, Tim Fox wrote:

On 13/04/14 23:42, Paulo Lopes wrote:

In a way i do agree with this since it makes users more at home when using other languages than Java but i can also see this as a source of new problems. e.g.: I am a JS developer and have developed some module for node.js and published it on npm, if vert.x advertises that supports npm modules there is a big chance that my module does not work on vert.x because I've native code or use some V8 specific stuff that rhino/nashorm does not support and even the API is not compatible (which forces another hidden dependency in nodyn). 

When I suggested NPM I didn't mean that our modules would use the node.js API, just that NPM might be a place where we could put Vert.x JS modules. I'm not sure if that will work though,

I think it can work and mentioned it in another post. It's definitely possible to support npm style packaging and deployment locally. And it should also be possible to publish npm modules to the main npm repository that are based on lang-js code. But I'm sure will lead to confusion like Paulo describes. 

Some modules in the main npm repository are pure JS and don't depend on the node.js API. These should work fine on both vert.x and node.js. Some npm modules depend on node.js APIs. They're clearly usable in node.js, but to use them in vert.x would introduce a nodyn dependency. If vert.x users start publishing JVM-based modules to the main npm repository, then we'd have a 3rd category of modules that will only run on vert.x. And even a 4th category of npm modules that are JVM based, taking advantage of both the node.js and vert.x APIs. These would only be runnable using nodyn. Complimicated!

This is all kind of similar to the native/java rubygem duality in Ruby. In the long run it would be great if npm supported both Java and native modules, and that support were somewhat seamless - or as seamless as it is in Ruby/JRuby. Frankly, I think if you want to keep attracting the attention of node.js developers it's a good idea to publish to the npm repository in spite of the confusion. Maybe, somewhere over the rainbow, there will be a time when there are java based implementations of many of the current native npm modules.

Lance

 

[Tim Fox]

On 17/04/14 11:49, Rick Hightower wrote:

RE: No, I'm not suggesting Vert.x does any dependency management.

** Thankfully! (I orginally wrote thank God, but after reading your rants on twitter thought better of it).

RE: I'm just suggesting that dependency resolution happens at build-time not run-time using whatever tools are used as a standard in that community (e.g. Maven).

+1 +100 yes. yes. yes.

** Right. So produce a jar file. But a jar file does not run without dependencies. So... you must mean produce oneJar or some sort of fatJar. Right? jar file will not do shit.

RE: It's really not a strange proposal - I'm simply proposing that we do things in the way that 99% of apps are build anyway.

Sure it is. But strange is good. Think different. Or just.. Think. I like it.

99% of apps do not build giant jar files. 99% of java web apps build war files (ok maybe 90% but still). :) We are inherently the 1% so we are inherently strange. If you want to be bigger than 1%, we probably need a way to build a jar like thing that has all of the dependencies bundled in. You said you were going to continue to support fatJar so fatJar will get used a lot more I imagine because you need a way to bundle libs with the runnable jar.....

I am not arguing. I just want a little clarity. If you get rid of modules, we need a good no-module story, and building a jar file with maven aint it.

Vertx 2

 Build = build fatJar or mod-zip with maven or gradle

Run = deploy mod-zip and run module or java -jar mymodule.fatjar.jar

So you are suggesting this....

Vertx 3

 Build = jar with maven or gradle

 Run = ? how do I include all of the jar files that my verticle needs?

That's a good question. I guess it depends on how you're using Vert.x.

If you're embedding it then it's up to you how you run it, if not embedding then maybe a fat jar or just a non fat executable jar are options.

[Rick Hightower]

Thanks boss. Sounds good. Sorry for the long drawn out banter. Trying to get my head around it.

[Tim Fox]

On 17/04/14 12:31, Rick Hightower wrote:

Thanks boss. Sounds good. Sorry for the long drawn out banter. Trying to get my head around it.

Me too.

None of this is a done deal - I just want to experiment with some ideas and see if they make sense or not :)

[S C]

No no no no no :-) my problem is when all Java developers will publish their stuff in Maven and all JS developers in npm. How on earth will I pull dependencies (jars, whatever) in my project from BOTH of these? Or is this so unlikely to happen once there's no system to rule them all?

[Tim Fox]

On 17/04/14 12:43, S C wrote:

No no no no no :-) my problem is when all Java developers will publish their stuff in Maven and all JS developers in npm. How on earth will I pull dependencies (jars, whatever) in my project from BOTH of these? Or is this so unlikely to happen once there's no system to rule them all?

I suppose you would just use Maven for the jars and npm for the JS bits. Although if you prefer to use Maven for everything I don't see why we can't support putting jars containing CommonJS modules in Maven too (although this might be weird for most JavaScript developers).

[petermd]

I suppose you would just use Maven for the jars and npm for the JS bits. Although if you prefer to use Maven for everything I don't see why we can't support putting jars containing CommonJS modules in Maven too (although this might be weird for most JavaScript developers).

there's already something that does this

http://www.webjars.org/ 

might even be something re-usable for vert.x?

-peter

[Rick Hightower]

Brilliant. 

Looks like you just need one of these for vertx

http://www.webjars.org/documentation

And then it already supports it client side.

Then we need one more for serverside JS support.

(All of the examples are how to load your JS resource from a JAR file from SpringMVC, Dropwizard, J.....SF [please jesus never repeat that acronym that is JS..F I add the dots because I don't want anymore recruiters asking for JSF consulting... ever... ]

We need something equiv but for the serverside.

And we need client side resource loader for JS probably. 

Wait do you have that already?

--

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rick Hightower]

So we can use org.webjars instead of NPM. (Sorry... JS is not my thing)

Maybe we need something like Servlet 3 for the client-side in vertx 3 (maybe it exists already)...

http://www.webjars.org/documentation

If you don't have it, I have it in Boon and can merge patch the relevant bits into Vertx.

(Boon can treat the classpath like a file system including list directories and files, and ... and ... and .. yeah that too.)

Instructions for Servlet 3

With any Servlet 3 compatible container, the WebJars that are in the WEB-INF/lib directory are automatically made available as static resources. This works because anything in aMETA-INF/resources directory in a JAR in WEB-INF/lib is automatically exposed as a static resource.

Maven Example (example app)

First add a WebJar as a dependency of your application in the pom.xml file, like:

<dependencies>
    <dependency>
        <groupId>org.webjars</groupId>
        <artifactId>bootstrap</artifactId>
        <version>3.1.0</version>
    </dependency>
</dependencies>

Then simply reference the resource like:

<link rel='stylesheet' href='webjars/bootstrap/3.1.0/css/bootstrap.min.css'>

[Rick Hightower]

I am noticing something on this list.

JavaScript people are speaking up so I imagine that JS is a common thing in Vertx land.

Java people are speaking up.

I have not heard squat from Scala land or Jython land or Groovy land.

I am guessing that if you are in Groovy land you are in Spring land and Reactor land and Grails land, and if you are in Jython land... you are probably not.. you are probably using Python and you are using Django and Tornado..... 

So am I guessing... maybe Vertx 3 really just needs to focus on Java and JavaScript. Groovy can run fine in Java land and whatever you do to for Java land also applies to Groovy and making the API nice for JS will make it nice for Java. So maybe the core should just focus on JS and Java, and let 3rd parties do the Jython and Groovy bits. Scala has AKKA, Play and Spudge (ok... I forget the actual name... but is it Spark, Spunk, Spaz, Spot, Spray, something like that)

Just a thought. I have nothing against Python (I love it), and nothing against Ruby (anymore), I love Groovy, I tolerate Scala... but maybe this Polyglot thing aint really much of a thing after all.

Have you done a recent poll of the vertx user-base?

What is the demographics for 

Ruby

Groovy

Java

JavaScript

Python

Scala

Because from this email exchange it seems like...

Ruby                 2%

Groovy              2%

Java                  51%

JavaScript         41%

Python              2%

Scala                2%

It might be good to have awesome JS and Java support, and sacrifice a little bit of polyglot-ness.

I think the 2% is being generous.

More compatibility with Node will bring you more eyeballs than Scala support and who still uses Jython (sure there are people who do, and I wrote a book on Jython so I am no Jython basher, but in the scheme of things, its days of glory never existed and are rapidly declining).  Because Scala has all those things I mentioned.

If I had to pick something that would be good for the success of Vertx it would to be to really focus on JS and Java, and allow more and more Node modules to be dropped into Vertx land.

I say this with almost complete ignorance of Node. I personally detest JS. I write it under duress. But vertx is very much Java answer to Node and a damn good answer at that.....

http://www.techempower.com/benchmarks/

I did mention that if you switched to Boon you would be number 1 on the JSON benchmark by 1% didn't I?

If I am using Groovy, I am probably going to focus on 

http://spring.io/blog/2013/05/13/reactor-a-foundation-for-asynchronous-applications-on-the-jvm

Since Groovy is from Interface21, I mean SpringSource, I mean VMWare... I mean Pivotal.... 

( I love groovy btw)

Scala guys are going to gravitate towards

http://spray.io/ and AKKA and/or play... which has a big banner that says it is reactive so it must be right? :)

Caveat:

Not a fan of JavaScript but that is where the market is going

I think Vertx should double down on JavaScript, Java and JSON. 

more compatibility with Node and let the others go where they must....

The JVM supports all. They are all inherently able to participate in Vertx. They can use Java APIs and JS APIs for that matter.

"Success is often built on a reflexive habit of saying “yes” to opportunities that come your way. Eventually, as you succeed, you must prioritize the many opportunities that present themselves, or else you’ll be overwhelmed, overcommitted, and ineffective. These steps can help you say “no” more comfortably:" --Harvard Business Review Adapted from “Learning to Say "No" Is Part of Success” by Ed Batista.

Just a thought since we are talking about 3.0....

Maybe Polyglot means just focus on JS and Java. :)

[Rick Hightower]

Oh and Java 8 will probably help use be more compatible with Scala anyway and Groovy so... so...

It should be ok to focus on Java 8 and JS support for Vertx3.

Again.. just brainstorming here.

[Tim Fox]

On 17/04/14 14:03, Rick Hightower wrote:

I am noticing something on this list.

JavaScript people are speaking up so I imagine that JS is a common thing in Vertx land.

Java people are speaking up.

I have not heard squat from Scala land or Jython land or Groovy land.

I am guessing that if you are in Groovy land you are in Spring land and Reactor land and Grails land, and if you are in Jython land... you are probably not.. you are probably using Python and you are using Django and Tornado..... 

So am I guessing... maybe Vertx 3 really just needs to focus on Java and JavaScript. Groovy can run fine in Java land and whatever you do to for Java land also applies to Groovy and making the API nice for JS will make it nice for Java. So maybe the core should just focus on JS and Java, and let 3rd parties do the Jython and Groovy bits. Scala has AKKA, Play and Spudge (ok... I forget the actual name... but is it Spark, Spunk, Spaz, Spot, Spray, something like that)

Just a thought. I have nothing against Python (I love it), and nothing against Ruby (anymore), I love Groovy, I tolerate Scala... but maybe this Polyglot thing aint really much of a thing after all.

Have you done a recent poll of the vertx user-base?

What is the demographics for 

Ruby

Groovy

Java

JavaScript

Python

Scala

Because from this email exchange it seems like...

Ruby                 2%

Groovy              2%

Java                  51%

JavaScript         41%

Python              2%

Scala                2%

It might be good to have awesome JS and Java support, and sacrifice a little bit of polyglot-ness.

I've actually had similar thoughts recently. I can tell from the google analytics on the docs pages which Vert.x language docs are accessed the most - Java and JavaScript are very popular, Groovy is actually quite popular too - we have quite a few Groovy Vert.x users, Ruby and Python are less popular (by a long way). Scala and Clojure are kind of an unknown factor still as they are new.

If we have to manually maintain the other language APIs (as we currently do) then I would be concerned about devoting a lot of time in keeping APIs that aren't used much up date - this was one of the main motivations for proposing generation of other language APIs - then they're not such a resource drain.

I think the 2% is being generous.

More compatibility with Node will bring you more eyeballs than Scala support and who still uses Jython (sure there are people who do, and I wrote a book on Jython so I am no Jython basher, but in the scheme of things, its days of glory never existed and are rapidly declining).  Because Scala has all those things I mentioned.

If I had to pick something that would be good for the success of Vertx it would to be to really focus on JS and Java, and allow more and more Node modules to be dropped into Vertx land.

I say this with almost complete ignorance of Node. I personally detest JS. I write it under duress. But vertx is very much Java answer to Node and a damn good answer at that.....

http://www.techempower.com/benchmarks/

I did mention that if you switched to Boon you would be number 1 on the JSON benchmark by 1% didn't I?

I haven't forgotten ;) I promise I'll get around to looking at it soon (spending most of my time travelling at the moment).

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[Tim Fox]

On 17/04/14 13:37, Rick Hightower wrote:

Brilliant. 

Looks like you just need one of these for vertx

http://www.webjars.org/documentation

+1. Already on my list of things to look at :)

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[Rick Hightower]

I was not sure how you would take that comment since Polyglot is sort of a vertx calling card....

I am glad to hear that Groovy is active. I have not used the Groovy support for vert.x yet, but I have clients who love Groovy.

Three is easier to support than six especially when 1 of the three already endeavors to be compatible with Java (Groovy). :)

So if Clojure and Python go the way of Scala support (done by 3rd parties and contributed as libraries since we don't have modules), we are less polyglot but still in the ethos if not in the effort.

Java, Groovy, JavaScript.

Then that gives more cycles to improve core around JavaRx, marshaling and Node compatibility (assertion brainstorming...).

I like it. 

http://www.techempower.com/benchmarks/

RE: I haven't forgotten ;) I promise I'll get around to looking at it soon (spending most of my time travelling at the moment).

No worries. It is probably easier if I send a PR for it anyway.

I will modify Vertx. to use Boon and see what blows up.

I had to get my head around gradle before I could work on vertx. I now know gradle well enough to use it on a few projects so I am ready to work on a PR.

Also I sent a PR for upgrading hazelcast to 3.2. I usually wait for a PR to get accepted before doing another one. (I once spent a ton of time preparing a JSF patch for a popular project which I was assured would get accepted and then just before I submitted it they accepted another patch. Bitter. They made me jump through hoops too.)

https://github.com/eclipse/vert.x/pull/813

[bytor99999]

We use Groovy. Not Groovy Verticles as scripts though, as actual classes. 

Hence my comment earlier in this thread about the who focus on Java part.

Mark

[dgo...@squaredfinancial.com]

On Thursday, 17 April 2014 23:01:36 UTC+1, Tim Fox wrote:

> Python are less popular (by a long way)

Well, vertx's official python support is currently based on jython 2.5.  It's not really vertx's fault given the endless-beta status of jython 2.7, of course.  And I'm not saying there'd be an amazing uptick with 2.7 - Rick Hightower kinda has a point that a lot of python people are likely to favor one of the usual python async frameworks (after all, they've been doing it over twice as long as node...) - but the "urgh. 2.5?" factor shouldn't be underestimated either.   In python terms, it's an antique, whereas, while 2.7 is not 3.x, at least it's to have some support python-wise to 2020 [1].    It's really up to Jordan Halterman of course, but providing an alternate build of mod-lang-jython with jython 2.7beta (that could be enabled in langs.properties if the user is willing to take the beta risk) would probably be positive.


[1] http://legacy.python.org/dev/peps/pep-0373/#update

[dgo...@squaredfinancial.com]

On Wednesday, 16 April 2014 11:07:19 UTC+1, Dean Pehrsson-Chapman wrote:

I can see why people find the module system complex, but the truth is it's a lot simpler than the alternatives, and being part of the platform makes the whole infrastructure simpler. It's possible that the people using vert.x are not enterprise (because enterprise adoption lags) and this is more of an enterprisey feature.

While nonflat classloader structure is something that people may find confusing and unnecessary for simpler needs (maybe it could be optional with an "isolate" flag in mod.json and/or deploy though?), I do wonder about other aspects of modules like the packaging, where there might be perceived complexity:

It has struck me in the past that packaging-wise, the recommendation for creating a module tend to go straight to "use the maven/gradle template".   But those templates have a fair bit of stuff in them. Particularly to someone coming from a scripting-language background I'd say they can look quite involved and java-y (even if they're a lot more lightweight than a lot of java-land stuff, and doing useful things).

When you get down to it, a vertx platform module archive is a .zip with a mod.json *. Now, the modules manual doesn't somehow claim different. The point is that, while a lot of the other stuff in the templates is (if you're used to the conventions and understand what they're for) no doubt nice-to-have,  any old thing should be able to spit out a .zip with a mod.json in it, right?**. Including e.g. a "python setup.py vertx_mod" command (proof-of-concept - https://github.com/dgoldensquared/pyvertxmod )

* Yes, osgi bundles are just .jar files (themselves .zips of course) and a manifest.mf with the relevant osgi fields, I know...

** Actually, wrong right now, since vertx platform's unzipper seems to be upset by absence of separate "directory/" entries in a .zip file if "directory/blah" is there (which is probably a bug akin to https://github.com/vert-x/mod-unzip/issues/2 , I only just noticed as a python-generated zip initially did not to include them), but that's possible to workaround.

[Paulo Lopes]

On Thursday, April 17, 2014 7:32:50 PM UTC+2, Rick Hight wrote:

I want to take another crack at this.

Inline.

On Sunday, April 13, 2014 9:45:17 AM UTC-7, Tim Fox wrote:

Throwing some ideas out there...

I've been thinking recently about Vert.x 3.0. One thing that has become
clear to me over the past year or so, is that we have done some things
well in Vert.x, but other things not so well. I think it's been a
learning experience and I think we should take the opportunity with
Vert.x 3.0 to improve things.

Some of the proposals I'm going to make for Vert.x 3.0 are quite
radical. But I think it's necessary that we make some big changes if
we're going to continue to gain popularity, and take Vert.x to the next
level.

So here are my *main* proposals (there are lots of other smaller bits
and pieces, but I'm going to leave those for now)

I'd like to hear your comments on this, and if you have any more
suggestions about other improvements we can make in Vert.x 3.0, please
share them :)

1) Massively simplify the module system.

Not sure. We need something like a war file that has jar files, and Java code. This was the mod-zip.

Since OSGi is a pretty much a complete failure, Vertx modules are attractive.

In a world of micro-services, you deploy each "module" as its own micro-service.

Verticals really are the important concept here, not modules. 

Java needs a way to ship a unit of app that includes the jar files for that app.

fatJar could be that unit. 

If you can explain more about how you can replace this functionality.

What does a vertx world look like without modules.

Does this mean that vertx is just geared towards microservices?

So each module just deploys its own vertx instance?

You communicate by wiring up instances in the same cluster?

One concern is speed. What is the cost of every call to another mirco-service if that call (and by call, I mean posting a message on the boss I suppose) has to go over the TCP/IP stack.

http://www.cl.cam.ac.uk/research/srg/netos/ipc-bench/details/tmpwoNtJN.html

There seems to be very little difference between UDS and TCP/IP, but mempipe seems to be 2x to 4x.

The point being... the cost of micro-service vs. module is the cost of IPC. 

So with microservice vs. module you have an vertx instance (JVM really) per microsevice.

Am I totally off base. I'd love some clarifying comments.

 

One of the purposes of the Vert.x module system is to provide a
consistent way for users to encapsulate and reuse Vert.x functionality
across different languages. Another feature of the module system is to
resolve module dependencies at run-time.

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Forcing all these different users to do things in a different Vert.x way
isn't attractive to these different communities, and has caused some
confusion.

I'm therefore proposing that instead of forcing a "one size fits-all"
module on Vert.x users, that we drop the idea of a module (i.e. a module
zip containing a mod.json) altogether and simply embrace whatever module
system is in common use by the relevant community. That way users from
different language communities can do things in the way they already
know without having to understand a different module format.

For Java users that means simply creating plain jars, and putting them
in Maven. For JavaScript users, putting them in npm. For Ruby users,
putting them in rubygems. etc.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a
nice feature, but in practice is it really worth it?

Worth it? Can't really live without it? Are you proposing a move from modules to pure microservices?

How do you get past this issue?

Sorry.. I know I am being a Java grandpa here, but I don't understand.

Without modules. I will need a vertx instance per service or I will need to make sure that every jar I deploy is using the same versions of gak.

In most Java shops I work with... projects resemble archeology. There was this project developed six years ago that uses Spring 2.x and Hibernate 3, and now the latest versions that use Spring 11, and Hibernate 14. The reason they have war files is so they don't stomp on each other.

In reality... they have war files and then each service has its own Tomcat instance so the war files are redundant. 

So I want to be clear... Are you proposing a microservice approach, in which case modules are redundant since each service gets its own instance anyway.

(I hate the terms like microservice since it is just a term for what people mostly/actually do. But giving something a name, NoSQL, Reactive, AJAX seems to help so I will use this name with distaste).

 

Especially if we
resolve all dependencies at build time, Maven will catch any such clashes.

????

To build what... a mod-zip? Maven builds jar files. You have maven build mod-zip. Maven can also build war files.

Your comment makes sense in the mod-zip sense and in the fat jar sense, but not in the jar sense.

If you don't build modules, what are you building with maven?

fatjar?

mod-zip?

something new?

 

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

As long as it is easy to use from gradle and/or maven... I think it is already the Java way.

I can't speak to other languages. Java is my home turf. 

 

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

Please no. If you have to do that.. Please at least have on take an enum or a string. 

httpServer.on(CONNECT, handler); 

 

httpServer.on("connect", handler); 

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Also, if a member of the community creates a module with a Java API,
it's a PITA to have to manually create wrappers for that API in
different languages. In many cases people don't bother, resulting in the
API being only usable in a single language.

I'm proposing that instead of manually maintaining API shims, we only
maintain the core APIs in Java and we provide a tool which generates API
shims for different languages. That way everything stays up to date with
little effort, and should be a huge win for language module maintainers.

It's not an easy task, but I believe it is possible and it means we
won't constantly be spending our time in the community with the tedious
job of keeping APIs up to date and we can spend our time on fun stuff
like innovating new functionality. We can also generate other language
API documentatiom using the same tool.

No opinion.

 

4) Automatic generation of location transparent proxies for services

Let's say we have a reusable component such as a database persistor.
Currently with Vert.x this will listen on the event bus and interact
with the rest of the world by receiving and sending messages.

However, it would be much nicer if the client could use a rich API, e.g.

api.store(object, {
   // stored
})

instead of:

eventbus.send(address, storeMessage, {
   // stored
}

I'm proposing that if a simple Java API is provided by the component,
that we can generate proxies that the client can use, which handle the
marshalling and marshalling of the data over the event bus automatically.

+1 I spent some cycles improving my JSON marshaling code yesterday.

:)

https://github.com/RichardHightower/boon/commit/1d5f2da5eb865038be555e538c659a59a51567ba

 

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

+1

This is something I have been thinking about for years and years.

 

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

I think you just want a subset that is RX like and then let someone else add/contribute RX modules.

Vertx has to be careful not to jump the shark. 

RxJava is too big.

Take the 20% that makes sense and leave the rest on the table. IMHO.

 

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

+1

 

7) Reification of different dispatchers

Currently events in Vert.x are dispatched either on an event loop or a
worker thread depending on what type of verticle it is. It's currently a
bit clunky to configure parameters for the dispatchers, e.g. number of
threads in the pool. Also the worker pool is currently global to each
Vert.x instance.

By re-ifying the dispatchers you will be able to maintain multiple
different worker pools and configure them programmatically and also
configure event loops, e.g.

Dispatcher dispatcher = new EventLoopDispatcher(numberOfEventLoops);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

Dispatcher dispatcher = new WorkerPoolDispatcher(maxThreads, minThreads,
etc);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

you'll be able to provide your own dispatcher implementations that, say,
use a Java executor

Executor myExecutor = ...;
Dispatcher dispatcher = new ExecutorDispatcher(myExecutor);

vertx.deployVerticle(new MyVerticle(), dispatcher);

Damn. I wish I understood what you meant.

http://en.wikipedia.org/wiki/Reification_(computer_science)

More words. 

This sounds pretty advanced. Hopefully there will be an easy button. 

The one thing I love about the vertx model is that the constructs are basic and easy.

Push something here and it comes out here.

 

8) Java 8 only

I propose that we target Java 8 and later for Vert.x 3.0, so we have
access to all the new features (e.g. Lambas, functional stuff,
completablefuture)

And long before that we convert the examples to Java 8. :)

Add some things...

TOML for config.

Boon for JSON parsing (or a Boon merge). 

As soon as you accept my pull request for Hazelcast 3.2, I am going to work on a pull request for Boon. 

Then after that I am going to brainstorm some ideas for marshaling.

That is quite interesting, one thing I can suggest about marshaling is to have some kind of "reviver" handler like on JavaScript JSON where one can plug a handler to convert from the base json types to some language specific type. The example I always give is that from the browser I send dates which the ecmaspec encodes as:  YYYY-MM-DDTHH:mm:ss.sssZ what i want is that in my module I could have a reviver that given a string i would use a dateformatter.parse and get a Date object again...

This would allow modules like the mongodb to perform searches on time intervals, or using expiring indexes... as a simple example...

  

:)

 

[Paulo Lopes]

On Thursday, April 17, 2014 7:53:36 PM UTC+2, Rick Hight wrote:

My wish list for Vertx 3.0.

• Micro-service architecture in - modules out
• With more modules, verticles are the center of the vertx universe
• Embrace Gradle/Maven/RubyGem/NPM (actually that is not really my wish list.. the first two are... the other two are just regurgitating what Tim said)
• High-speed IPC (In a world without modules, processes need to talk to each other very quickly)
• Object marshaling between verticles and micro-services (callbacks, Reification, micro-service to address resolution)
• TOML or TONL for config https://github.com/mojombo/toml

If you suggest TOML for config when we used to use json everywhere what would be the relation to the message types on the event bus? why not also propose toml as a message type too? I see the benefit there since it is more descriptive than json... 

• Embrace JDK 8 API
• RxJava support pulled into core (either 20% pareto of RxJava features or more integration) 

• Ability to more easily upgrade switch clustering support (so you can have Vertx 3 but Hazelcast can be upgraded independently, this is mostly there but not quite. Vertx 2 for example could support both 2.x and 3.x Hazelcast)

On Sunday, April 13, 2014 9:45:17 AM UTC-7, Tim Fox wrote:

Throwing some ideas out there...

I've been thinking recently about Vert.x 3.0. One thing that has become
clear to me over the past year or so, is that we have done some things
well in Vert.x, but other things not so well. I think it's been a
learning experience and I think we should take the opportunity with
Vert.x 3.0 to improve things.

Some of the proposals I'm going to make for Vert.x 3.0 are quite
radical. But I think it's necessary that we make some big changes if
we're going to continue to gain popularity, and take Vert.x to the next
level.

So here are my *main* proposals (there are lots of other smaller bits
and pieces, but I'm going to leave those for now)

I'd like to hear your comments on this, and if you have any more
suggestions about other improvements we can make in Vert.x 3.0, please
share them :)

1) Massively simplify the module system.

One of the purposes of the Vert.x module system is to provide a
consistent way for users to encapsulate and reuse Vert.x functionality
across different languages. Another feature of the module system is to
resolve module dependencies at run-time.

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Forcing all these different users to do things in a different Vert.x way
isn't attractive to these different communities, and has caused some
confusion.

I'm therefore proposing that instead of forcing a "one size fits-all"
module on Vert.x users, that we drop the idea of a module (i.e. a module
zip containing a mod.json) altogether and simply embrace whatever module
system is in common use by the relevant community. That way users from
different language communities can do things in the way they already
know without having to understand a different module format.

For Java users that means simply creating plain jars, and putting them
in Maven. For JavaScript users, putting them in npm. For Ruby users,
putting them in rubygems. etc.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a

nice feature, but in practice is it really worth it? Especially if we

resolve all dependencies at build time, Maven will catch any such clashes.

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Also, if a member of the community creates a module with a Java API,
it's a PITA to have to manually create wrappers for that API in
different languages. In many cases people don't bother, resulting in the
API being only usable in a single language.

I'm proposing that instead of manually maintaining API shims, we only
maintain the core APIs in Java and we provide a tool which generates API
shims for different languages. That way everything stays up to date with
little effort, and should be a huge win for language module maintainers.

It's not an easy task, but I believe it is possible and it means we
won't constantly be spending our time in the community with the tedious
job of keeping APIs up to date and we can spend our time on fun stuff
like innovating new functionality. We can also generate other language
API documentatiom using the same tool.

4) Automatic generation of location transparent proxies for services

Let's say we have a reusable component such as a database persistor.
Currently with Vert.x this will listen on the event bus and interact
with the rest of the world by receiving and sending messages.

However, it would be much nicer if the client could use a rich API, e.g.

api.store(object, {
   // stored
})

instead of:

eventbus.send(address, storeMessage, {
   // stored
}

I'm proposing that if a simple Java API is provided by the component,
that we can generate proxies that the client can use, which handle the
marshalling and marshalling of the data over the event bus automatically.

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

7) Reification of different dispatchers

Currently events in Vert.x are dispatched either on an event loop or a
worker thread depending on what type of verticle it is. It's currently a
bit clunky to configure parameters for the dispatchers, e.g. number of
threads in the pool. Also the worker pool is currently global to each
Vert.x instance.

By re-ifying the dispatchers you will be able to maintain multiple
different worker pools and configure them programmatically and also
configure event loops, e.g.

Dispatcher dispatcher = new EventLoopDispatcher(numberOfEventLoops);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

Dispatcher dispatcher = new WorkerPoolDispatcher(maxThreads, minThreads,
etc);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

you'll be able to provide your own dispatcher implementations that, say,
use a Java executor

Executor myExecutor = ...;
Dispatcher dispatcher = new ExecutorDispatcher(myExecutor);

vertx.deployVerticle(new MyVerticle(), dispatcher);

[Dean Pehrsson-Chapman]

I'm interested in understanding what feature(s) of the Vert.x module system would be missed...

The separation of modules by classloader.  We can have two modules that depend on x.jar.  One of them wants to upgrade, the other one doesn't care.  If we don't have classloader separation we have to upgrade both.  When we upgrade both, we force a manual retest on the one that has not changed at all.

If this happened as part of a high priority bug fix, the difference it would make to the business would be dramatic.   

[S C]

I sent ages ago a pull request fixing mod-unzip but for whatever reasons didn't get through...

[David Dossot]

Agreed: the current share nothing architecture of Vert.x, for which the per-module classloader is an enabler, is a major plus of Vert.x and certainly one of the main reasons why we picked it. I'd be sad to see this go and be propelled back to resolving version compatibility issues like I did a decade ago in J2EE containers of the time.

I personally like the current module structure: it's simple and does its job well. It provides a versionable standalone self-contained unit of deployment .

Moreover it doesn't imply any particular layout for a project. For example, for our Ruby projects, we do not use the src/main/... Maven style but a flatter structure that is more palatable to Ruby devs. We bring the gem dependencies in the mod as a jar. We bring shared Ruby mods as included dependencies so Ruby classes can be re-used. All in all, a pretty elegant model enabled thanks to the clean and simple structure of mods.

My main concerns around mods is the discovery model: using Maven to locate mods has some annoying implications regarding hot reloading snapshots, I wish I could just give a URL to a mod.zip sometimes, like in the context of continuous deployment with CI server produced mods.

So Vert.x becomes less of a platform, and more of a library with a flat classpath that can be trivially embedded into any application.

This concerns me TBF. I was in fact about to ask for more platform :) i.e the same way application servers have a "deploy" dir where apps can be dropped or removed, I'd like Vert.x to have the same...

All in all, I think I like Vert.x 2 because it's opinionated on some things. I'm a little leery to see Vert.x 3 losing some of these core values that makes Vert.x different and attractive.

My 2c
D.

 

[Rick Hightower]

Is it faster to serialize Java objects with Jackson JSON/Text or ObjectOutputStream binary?http://www.dzone.com/links/which_is_faster_serializing_to_json_with_jackson.html Jackson is hands down faster, and Boon is faster than Jackson. So JSON over a message bus would be faster than RMI most likley...

I have two forms of JSON serialization. One is traditional. You map Java objects to JSON objects the other one is sideways. You map Java objects to JSON lists so it skips the key names. I have not benchmarked the sideways serialization but I have been optimizing it quite a bit. It was the sideways serialization that led me to writing the marshaler. Marshaling serialization to constructors is similar to marshaling method calls to methods. Anyway with sideway serialization, the constructors form the schema definition instead of the fields. It works surprisingly well and Boon can mix/match sideways and normal serialization of JSON. It makes for some tight, small JSON feeds and method invocations. I use it for REST and WebSocket projects. Often times the front end vertx merely checks parameters and headers and then forwards the HTTP call to the bus. It works well.



--

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rick Hightower]

ATM it makes more sense for config. I doubt any of the parsers are geared for speed and efficiency. They are probably more geared towards being a good config file processor. You would have to benchmark them. It would be an easy parser to write not as easy as JSON. It is missing a key type IMO. I needs a tuple/list. It has a uniform array. Both are warranted for this type of serialization. I don't see what it buys us for serialization. Config: hell yes.

JSON and JSON loose and sideways JSON would be just as good. JSON loose is JSON without quotes for keys (and other stuff), JSON sideways is serializing Java objects into JSON arrays instead of JSON objects (Boon can do this, and it can mix and match). 

We can write something pretty damn tight with JSON and it can be pretty damn close to Kryo speeds but with the interoperability of JSON. 

Also Boon and Jackson and others check for cyclic references at some level by default. We can have a mode where we do not check for cyclic references which will speed up serialization. 

Compare this:

https://github.com/RuedigerMoeller/fast-serialization/wiki/TestPage

To this:

http://www.dzone.com/links/which_is_faster_serializing_to_json_with_jackson.html

http://rick-hightower.blogspot.com/2014/04/which-is-faster-java-object.html

JSON sideways serialization can get us at about 50% compression of normal JSON.

I think we can get Boon JSON serialization fairly close to the fastest binary serializer, and in human readable text.

Then we can always just use Kryo or FST for cases where we want additional speed, but Boon JSON should be fast enough where the difference is just dust on the scale.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rick Hightower]

I got confused by that benchmark. I thought FastJSON was Jackson (which is under the group name FastXML which is odd since it is JSON but I digress).

So FastJSON (not Jackson) was pretty damn close to Kryo and FST, so I was not sure how close Boon is to FastJSON so I added it to the JMH benchmarks.

http://rick-hightower.blogspot.com/2014/04/fastjson-rocky-mr-t-boon-wants-to-take.html

:)

Sorry lack of sleep and all...

But you get the idea.

[Alex Ardelean]

Hi, 

I am not a regular vertx user, I just like to play with it on my free time. Haven't found a way or a place where to put it in the project where I am working on at my current job, but I hope I will soon find it :).

Anyway, here are my thoughts after my short experience until now.

1,3,4,5,6,7,8 -YES, I think this will be a great addition.

2- I do not see any benefit from a java developer to do such a thing. If you really want it for easing up integration with JS or closing the gap in the api with Nodejs (why would you want that anyway?) maybe you could keep both approaches.

Also, as for module dependencies, which is the most valuable out of all these points, I would kindly suggest to take a look if you haven't already done it, to the way spring-boot has solved this problem. I was very reluctant to try spring-boot out, but after weeks of pressure from my colleagues and friends, I have finally done, and it seems like a huge step into the future of development. No more hassle with dependencies and all that configurations. Maybe that could be inspirational for the new vertx 3.0 module refactoring feature. 

I don't want to sound like a spring fan boy, trying to push what I like to other frameworks, but I really feel that setting up a vertx 3.0 project in the manner we could set up a spring boot project would greatly decrease the the time of development for small-medium size projects (what vertx is or could be best at) and add a breeze of fun in starting new projects in this futuristic manner.

Best Regards, Alex.

[Tim Fox]

On 19/04/14 10:46, Alex Ardelean wrote:

Hi, 

I am not a regular vertx user, I just like to play with it on my free time. Haven't found a way or a place where to put it in the project where I am working on at my current job, but I hope I will soon find it :).

Anyway, here are my thoughts after my short experience until now.

1,3,4,5,6,7,8 -YES, I think this will be a great addition.

2- I do not see any benefit from a java developer to do such a thing. If you really want it for easing up integration with JS or closing the gap in the api with Nodejs (why would you want that anyway?) maybe you could keep both approaches.

Also, as for module dependencies, which is the most valuable out of all these points, I would kindly suggest to take a look if you haven't already done it, to the way spring-boot has solved this problem. I was very reluctant to try spring-boot out, but after weeks of pressure from my colleagues and friends, I have finally done, and it seems like a huge step into the future of development. No more hassle with dependencies and all that configurations. Maybe that could be inspirational for the new vertx 3.0 module refactoring feature. 

I don't want to sound like a spring fan boy, trying to push what I like to other frameworks, but I really feel that setting up a vertx 3.0 project in the manner we could set up a spring boot project would greatly decrease the the time of development for small-medium size projects (what vertx is or could be best at) and add a breeze of fun in starting new projects in this futuristic manner.

Alex,

Could you elaborate about what Spring Boot does which you like and could learn from? I don't know much about it but I thought it was basically a way to do a "fat jar" (which we already have).

--

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[Tim Fox]

It's a double edged sword. Yes, having classloader isolation allows problems like this to be solved, but it also introduces other problems, e.g. in your example if you then passed different versions of the same class from one module to another you would get ClassCastExceptions.

I'd say having two versions of the same jar in an app often causes more problems than it solves.

[Rick Hightower]

Spring boot has project templates and it is similar to Roo or Forge or maven archetypes or even play or appfuse. One of its "features" is it can be run from the command line with maven or gradle so no longer will you mvn jetty:run instead you will mvn springboot:run (I kid you not).  So this very new and exciting thing has been done about 100 times (I wrote several.. Presto and later Crank). 

In short it helps people get started.

One of the "amazing" features is it supports thyme leaf instead of JSP because JSPs are "deprecated". Having used thymeleaf this makes me sad. Thymeleaf makes me want to stab myself in the eye. What a load of ....

That said one thing that vertx is missing is hey mr developer here is how you use this to build a webapp or a service or a Websocket doo hickey which sort of leaves vertx out if the reach of many. Having written 30 or so tech courses over the years the mere thought of this makes me want to ... You guessed it.

The maven archetype is really bloated. The gradle template is hard to find. Vertx has the rest like mvn vertx:runMod etc.

Idk. 

[Tim Fox]

Another way of tackling this issue is to refactor your app into a set of loosely coupled services each running in their own JVM and talking over the event bus instead of a single blob. This should reduce the chances of any particular Vert.x instance requiring multiple versions of the same jar at the same time.

On 18/04/14 13:26, Dean Pehrsson-Chapman wrote:

[Rick Hightower]

This is where service method marshaling could come in handy. You talk to an interface methods that has handlers instead of returns.

Develop and unit test with mocks that are collocated. Deploy with skeletons and stubs that marshal calls to other microservices.

Or just use the bus... 

I wrote a method marshaler. I am sure it is one of many. 

On Saturday, April 19, 2014, Tim Fox <timv...@gmail.com> wrote:

You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Paulo Lopes]

On Sat, Apr 19, 2014 at 12:23 PM, Rick Hightower <richardh...@gmail.com> wrote:

Spring boot has project templates and it is similar to Roo or Forge or maven archetypes or even play or appfuse. One of its "features" is it can be run from the command line with maven or gradle so no longer will you mvn jetty:run instead you will mvn springboot:run (I kid you not).  So this very new and exciting thing has been done about 100 times (I wrote several.. Presto and later Crank). 

In short it helps people get started.

One of the "amazing" features is it supports thyme leaf instead of JSP because JSPs are "deprecated". Having used thymeleaf this makes me sad. Thymeleaf makes me want to stab myself in the eye. What a load of ....

That said one thing that vertx is missing is hey mr developer here is how you use this to build a webapp or a service or a Websocket doo hickey which sort of leaves vertx out if the reach of many. Having written 30 or so tech courses over the years the mere thought of this makes me want to ... You guessed it.

Humm... I've kind of created a similar thing for Yoke, (and to some extent Vert.x :P)

wget http://goo.gl/OMuYRo

or curl if you prefer...

java -jar yoke-tools.jar [--java|groovy|js] group:artifact:version

And you get a gradle project with an hello world web application with a simple unit test...

after that you do:

./gradlew runMod and you have a running app... so i don't see the "amazingness" of spring boot, also I've included a bunch of template engines, groovy template, handlebars and jade to name the most popular... adding thymeleaf could be achieved i just did not spend time on it...

maybe we could merge yoke under the vert.x umbrella and profit from the existing functionality :)

just saying :P

 

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Paulo Lopes
www.jetdrone.com

[S C]

The point Boot is trying to make is right there on their frontpage and applying it to vert.x...
- Create stand-alone Spring applications - Yoke is one, I'm not aware of other options
- Embed Tomcat or Jetty directly (no need to deploy WAR files) - vert.x has vert.x already
- Provide opinionated 'starter' POMs to simplify your Maven configuration - vert.x templates are rather kitchen-sink
- Automatically configure Spring whenever possible - ha.
- Provide production-ready features such as metrics, health checks and externalized configuration - quite far from it
- Absolutely no code generation and no requirement for XML configuration - well the price of being opinionated...

Generally speaking, to start with a new technology you need a STRONG motivation. Boot is trying to ease it for Spring. How about vert.x?

PS: indeed, this post is not about the platform 3.0. But a better platform management interface would be very welcome (statistics, metrics without killing performance, reconfigure, remote control, restarts...)

[Tim Fox]

On 19/04/14 23:12, S C wrote:

The point Boot is trying to make is right there on their frontpage and applying it to vert.x...
- Create stand-alone Spring applications - Yoke is one, I'm not aware of other options

Not sure what you mean here... Yoke is a Spring application?

- Embed Tomcat or Jetty directly (no need to deploy WAR files) - vert.x has vert.x already
- Provide opinionated 'starter' POMs to simplify your Maven configuration - vert.x templates are rather kitchen-sink

Can you add some more detail to this point?

- Automatically configure Spring whenever possible - ha.
- Provide production-ready features such as metrics, health checks and externalized configuration - quite far from it
- Absolutely no code generation and no requirement for XML configuration - well the price of being opinionated...

Generally speaking, to start with a new technology you need a STRONG motivation. Boot is trying to ease it for Spring. How about vert.x?

Reading the above, we already have everything there other than metrics and templates could be improved (?). Is that basically what is missing?

PS: indeed, this post is not about the platform 3.0. But a better platform management interface would be very welcome (statistics, metrics without killing performance, reconfigure, remote control, restarts...)

It's on the TODO list.

--

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[pml...@gmail.com]

-----Original Message-----
From: Tim Fox
Sent: 20/04/2014, 11:08
To: ve...@googlegroups.com
Subject: Re: [vertx:17822] Re: Getting the ball rolling: Some thoughts about Vert.x 3.0


On 19/04/14 23:12, S C wrote:
> The point Boot is trying to make is right there on their frontpage and
> applying it to vert.x...
> - Create stand-alone Spring applications - Yoke is one, I'm not aware
> of other options

Not sure what you mean here... Yoke is a Spring application?

No, no, no. Yoke is *not* a spring app, has no dependency to spring and was developed with vert.x api in mind.

Developers can use spring or any DI lib they want, but at its core, yoke only depends on vert.x.

> - Embed Tomcat or Jetty directly (no need to deploy WAR files) -
> vert.x has vert.x already
> - Provide opinionated 'starter' POMs to simplify your Maven
> configuration - vert.x templates are rather kitchen-sink

Can you add some more detail to this point?

> - Automatically configure Spring whenever possible - ha.
> - Provide production-ready features such as metrics, health checks and
> externalized configuration - quite far from it
> - Absolutely no code generation and no requirement for XML
> configuration - well the price of being opinionated...
> Generally speaking, to start with a new technology you need a STRONG
> motivation. Boot is trying to ease it for Spring. How about vert.x?

Reading the above, we already have everything there other than metrics
and templates could be improved (?). Is that basically what is missing?

>
> PS: indeed, this post is not about the platform 3.0. But a better
> platform management interface would be very welcome (statistics,
> metrics without killing performance, reconfigure, remote control,
> restarts...)

It's on the TODO list.

> --
> You received this message because you are subscribed to the Google
> Groups "vert.x" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to vertx+un...@googlegroups.com

> <mailto:vertx+un...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

[Tim Fox]

On 20/04/14 12:43, pml...@gmail.com wrote:
> -----Original Message-----
> From: Tim Fox
> Sent: 20/04/2014, 11:08
> To: ve...@googlegroups.com
> Subject: Re: [vertx:17822] Re: Getting the ball rolling: Some thoughts about Vert.x 3.0
>
>
> On 19/04/14 23:12, S C wrote:
>> The point Boot is trying to make is right there on their frontpage and
>> applying it to vert.x...
>> - Create stand-alone Spring applications - Yoke is one, I'm not aware
>> of other options
> Not sure what you mean here... Yoke is a Spring application?
>
> No, no, no. Yoke is *not* a spring app, has no dependency to spring and was developed with vert.x api in mind.
>
> Developers can use spring or any DI lib they want, but at its core, yoke only depends on vert.x.

I know that Yoke is not a Spring application :)

I was just questioning the sentence "Create stand-alone Spring

applications - Yoke is one, I'm not aware

of other options" ;)

[Tim Fox]

On 17/04/14 20:01, Lance Ball wrote:

On Sunday, April 13, 2014 12:45:17 PM UTC-4, Tim Fox wrote:

Throwing some ideas out there...

1) Massively simplify the module system.

Yes, this sounds like a welcome change.

 

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Most definitely. From the point of view of JS developers, npm is definitely the way to go. For the nodyn project I wrote dynjs-npm[0], an npm module loader that conforms to node.js' API [1]. It was written with DynJS as the target Javascript runtime, but I think it should work just fine with Nashorn and Rhino as well - as long as there is an existing require() function in the global scope - which it falls back to if nothing is found in the typical npm locations. With Nodyn, I just load the npm require() function in the global scope when a verticle starts up [2]. It should be pretty simple to adapt this or something similar to the existing JS language modules.

[0] https://github.com/projectodd/dynjs-npm

[1] http://nodejs.org/api/modules.html 

[2] https://github.com/nodyn/nodyn/blob/master/src/main/java/org/projectodd/nodyn/NodeJSVerticleFactory.java#L50

 

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

Speaking as someone who writes a lot of javascript, I prefer the .on('some_event') style. The strongly typed handler API in Java bothered me a lot while I was shepherding the lang-js module. However, I can understand the Java developer's point of view regarding type checking. It's somewhat idiomatic to the language in many ways. An alternate approach may be to have untyped event handling in languages where that is the norm.

+1

Especially if those languages can support a dynamically dispatched style. For example, the SockJS bridge hooks I implemented in lang-js for vert.x 2.1 features take advantage of Javascript's fast and loose typing to provide SockJSServer.on() already [3]. Maybe the answer is that more dynamically typed languages could be encouraged to implement the API's handler styles in ways that are more idiomatic to the language itself.

 [3] http://vertx.io/mod-lang-js/docs/1.0.0/sockjs.SockJSServer.html

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Boy do we ever. :) 

I think this is an ambitious goal. Especially given the other goals above. For language modules to truly reflect the coding style of a given community, they really need to be tailored to the target language's current usage idioms and styles.

+1

I think it will be difficult to do that with a language generation tool. A generator may be good and useful for a fledgling language implementation, but I think it may not be the final step in a language module's production. If you can pull it off, though, more power to you.

 

4) Automatic generation of location transparent proxies for services

I don't have strong feelings on this, but it's an interesting idea. I wonder if it's maybe the first step in the creation of the code generating tool in 3). It kind of sounds like the same idea but with the initial source being 3rd party modules instead of vertx core. 

 

5) Promises/Future/Rx support

I have reservations about the magical API generator, so not much to add here.

 

6) No differentiation between "core" and "platform".

Speaking from a limited point of view, this would be nice for nodyn. With vertx being an embedded component, the platform bits just kind of get in the way. I want the nodyn experience to be as much like a node.js experience as possible for the end user, and the platform tends to change all of that.

Can you elaborate on this? AIUI, whether you embed the Vert.x platform or Vert.x core in NoDyn should make no difference to the NoDyn node.js user experience at all. I really want to understand this objection, because right now I really don't ;)

 

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application. 

This is my usage scenario, so yes this would be an improvement for me. That said, I realize this isn't _everyone's_ usage scenario. I think there is a place for the vert.x platform, because not everyone is going to use its services simply as an embedded component. 

I guess I'm on both sides of this debate. Personally, I don't want or need the vert.x module system in nodyn and that's a big part of why I'm keeping vertx-platform.jar out of it for the time being. That said, if the module system is gone and platform is nothing more than deploy/undeploy then maybe it makes sense to smush them together.

 

7) Reification of different dispatchers

I'm curious to see where you go with this. It could be a very interesting feature that may really open up potential interoperability with 3rd party libraries/systems/whathaveyou. 

 

8) Java 8 only

Yes.

And, one more thing...

9) Published system stats/usage data. 

That's probably not the right title for it, but what I mean is, it would be nice to have an API that allowed me to query vertx and find out things like how many verticles are running and some information about them.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[S C]

When you start developing an application with plain vert-x you basically always start from scratch. It's just a fact and not a bad point as vert.x is a platform. Yoke is already a framework which means by definition an opinionated "best practice"-based way of building applications because it already offers you "standard" ways to do it. Now, do we have in the vert.x environment any other "best practice" ways to build vert.x applications? There are quite a few power users on this list, but how do I, Joe Developer, benefit from their infinite wisdom if there's no "framework" where they would/could contribute this wisdom? That's the point of Boot for the Spring world, and in the vert.x world the only attempt at "vert.x for the masses"  is Yoke.

S

[S C]

On Sun, Apr 20, 2014 at 11:08 AM, Tim Fox <timv...@gmail.com> wrote:

On 19/04/14 23:12, S C wrote:

The point Boot is trying to make is right there on their frontpage and applying it to vert.x...
- Create stand-alone Spring applications - Yoke is one, I'm not aware of other options

Not sure what you mean here... Yoke is a Spring application?

Nope. See other email.

 

- Provide opinionated 'starter' POMs to simplify your Maven configuration - vert.x templates are rather kitchen-sink

Can you add some more detail to this point?

The Gradle template brings in 4 languages. The only reason I can think of is because you couldn't tell who wants what and well let's offer everything. Furthermore, the vert.x Gradle template does not offer anything beyond a "hello world" application, while the Boot starters are already real-world usable applications with different module integration.

 

- Automatically configure Spring whenever possible - ha.
- Provide production-ready features such as metrics, health checks and externalized configuration - quite far from it
- Absolutely no code generation and no requirement for XML configuration - well the price of being opinionated...

Generally speaking, to start with a new technology you need a STRONG motivation. Boot is trying to ease it for Spring. How about vert.x?

Reading the above, we already have everything there other than metrics and templates could be improved (?). Is that basically what is missing?

True, I really miss that management console which I know it's on the TODO list since quite a while. But I also miss the ways to make the start easier - with "usable" templates, or a framework, anything...

 

S

[sANTo L]

1) Massively simplify the module system.

One of the purposes of the Vert.x module system is to provide a
consistent way for users to encapsulate and reuse Vert.x functionality
across different languages. Another feature of the module system is to
resolve module dependencies at run-time.

I think we have learnt that in practice, Java users want to do things in
the Java way they know (i.e. put jars in Maven and resolve at build
time), JavaScript users want to do things in the server-side JavaScript
way (i.e. use npm / CommonJS modules), Ruby users want to use gems etc.

Forcing all these different users to do things in a different Vert.x way
isn't attractive to these different communities, and has caused some
confusion.

I'm therefore proposing that instead of forcing a "one size fits-all"
module on Vert.x users, that we drop the idea of a module (i.e. a module
zip containing a mod.json) altogether and simply embrace whatever module
system is in common use by the relevant community. That way users from
different language communities can do things in the way they already
know without having to understand a different module format.

For Java users that means simply creating plain jars, and putting them
in Maven. For JavaScript users, putting them in npm. For Ruby users,
putting them in rubygems. etc.

Another purpose of the Vert.x module system is to provide isolation of
module classes. This allows different modules to use different versions
of the same jars at the same time in the same JVM instance. This is a
nice feature, but in practice is it really worth it? Especially if we

resolve all dependencies at build time, Maven will catch any such clashes.

The module classloader hierarchy has historically been a source of
complexity and confusion. Removing the classloader hierarchy will mean
that Vert.x applications will run using a simple flat classpath. This
should dramatically simply running, debugging, embedding and sharing
classes between different parts of an application.

I really like the fact that Vert.x has "a" module system which allows to create an application based on separate modules that can interact with each other through the message bus, while still being able
to share some commonly used code. (cfr non-runnable modules)
The other advantage of the module system is the fact that you can hot-replace a module without having to stop/reload your entire application.

So I would prefer that this functionality remains available in future versions as well.
If this can be achieved by replacing the current module system with a more streamlined/simplified alternative, then that's fine for me, but please don't throw it out without providing an alternative

2) Refactoring of the core interfaces.

This might be controversial.... Vert.x objects such as HttpServer have
different handler methods for different events, e.g. connectHandler,
webSocketHandler.

Instead of that I suggest we refactor to use something more like the
Node.js style, e.g.

instead of

httpServer.connectHandler(handler);

we do

httpServer.on("connect", handler);

This should provide a cleaner API, and allow different events to be
handled in a more consistent way.

On the one hand I would prefer the refactored approach, but a consequence of this is that we will lose compile time checking, which is rather unnatural for a java developer :-)

3) Automatic generation of other language APIs

As language module maintainers will know, one of the main pain points of
Vert.x is in keeping the other language APIs up to date.

Also, if a member of the community creates a module with a Java API,
it's a PITA to have to manually create wrappers for that API in
different languages. In many cases people don't bother, resulting in the
API being only usable in a single language.

I'm proposing that instead of manually maintaining API shims, we only
maintain the core APIs in Java and we provide a tool which generates API
shims for different languages. That way everything stays up to date with
little effort, and should be a huge win for language module maintainers.

It's not an easy task, but I believe it is possible and it means we
won't constantly be spending our time in the community with the tedious
job of keeping APIs up to date and we can spend our time on fun stuff
like innovating new functionality. We can also generate other language
API documentatiom using the same tool.

It sounds good, but I'm not sure it would be that easy to implement that.
Also as I am mostly using java and sometimes javascript, but not any of the other languages, I actually don't care that much about it.
If it would mean that java and javascript will be treated as wore languages while the other languages are completely separated from the framework, then I am in favor of this because that would mean
that you could concentrate more on the core functionality of the framework without losing precious time on less used languages.

4) Automatic generation of location transparent proxies for services

Let's say we have a reusable component such as a database persistor.
Currently with Vert.x this will listen on the event bus and interact
with the rest of the world by receiving and sending messages.

However, it would be much nicer if the client could use a rich API, e.g.

api.store(object, {
   // stored
})

instead of:

eventbus.send(address, storeMessage, {
   // stored
}

I'm proposing that if a simple Java API is provided by the component,
that we can generate proxies that the client can use, which handle the
marshalling and marshalling of the data over the event bus automatically.

This means the component simply needs to provide a simple rich interface
in Java, and Vert.x will handle creating idiomatic APIs in each language
which can be used remotely from anywhere on the network to access the
component. It should be possible for this to work from client side JS
too, so basically any Vert.x component API would be automatically usable
from there too (subject to security constraints).

Another advantage of this is you won't have to worry about encoding and
decoding stuff manually in JSON in order to send/receive it from a
component.

Sounds great, but only if it doesn't mean we have to  sacrifice on performance

5) Promises/Future/Rx support

We can use our magical API generator to take any core API and create a
wrapper that exposes the API in terms of RxJava observables or Java 8
CompletableFutures instead.

Vert.x definitely needs something like this so we can get rid of the callback trouble.

 

6) No differentiation between "core" and "platform".

Most of the platform stuff in Vert.x is related to implementing the
module system. If the module system disappears then we basically just
have core and a few methods for deploying/undeploying verticles. All of
these methods can be rolled into the core API.

So Vert.x becomes less of a platform, and more of a library with a flat
classpath that can be trivially embedded into any application.

While we can still support the "vertx" command, many Vert.x applications
will be simple Java programs that can be run directly with 'java', e.g.

public class MyVerticle implements Verticle {

    public void start() {
       System.out.println("My verticle is starting");
       vertx.createHttpServer().requestHandler(...).listen(8080);
    }

    public static void main(String[] args) {
      Vertx vertx = VertxFactory.newVertx();
      vertx.deployVerticle(new MyVerticle());
    }

}

java org.foo.MyVerticle -cp ...

Note that we will allow deployVerticle to take an actual Verticle
*instance* as well as the Verticle *class* which is currently supported.
This will allow verticles to be injected by DI frameworks more easily.

While I'm currently not using Vert.x in an embedded way, I can imagine this would be a welcome improvement, so you have my vote on this one.

7) Reification of different dispatchers

Currently events in Vert.x are dispatched either on an event loop or a
worker thread depending on what type of verticle it is. It's currently a
bit clunky to configure parameters for the dispatchers, e.g. number of
threads in the pool. Also the worker pool is currently global to each
Vert.x instance.

By re-ifying the dispatchers you will be able to maintain multiple
different worker pools and configure them programmatically and also
configure event loops, e.g.

Dispatcher dispatcher = new EventLoopDispatcher(numberOfEventLoops);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

Dispatcher dispatcher = new WorkerPoolDispatcher(maxThreads, minThreads,
etc);

vertx.deployVerticle(new MyVerticle(), dispatcher);

or

you'll be able to provide your own dispatcher implementations that, say,
use a Java executor

Executor myExecutor = ...;
Dispatcher dispatcher = new ExecutorDispatcher(myExecutor);

vertx.deployVerticle(new MyVerticle(), dispatcher);

Sounds interesting, but one concern might be that it can become more complex, unless you make this completely optional and keep the default implementation similar to how it works in the current version.

8) Java 8 only

I propose that we target Java 8 and later for Vert.x 3.0, so we have
access to all the new features (e.g. Lambas, functional stuff,
completablefuture)

Sounds great for the vert.x platform itself, but I'm not so sure for the applications that will be written on top of the platform.
E.g. in practice it happens that you have to integrate with systems that don't use the newest java version, meaning that you have to use an older java version to be able to use the api of that system.
If vert.x only supports java 8 and later, I'm not sure this will still be possible, as the module that will integrate with the non-java-8 system will run inside vert.x, meaning it needs java 8, no ?

One other thing I would like to see is a changelog for each nog version.
Currently there is no easy way to know what has changed in a new version (e.g. from 2.1 RC1 to RC2 to RC3)

Santo

[dgo...@squaredfinancial.com]

On Saturday, 19 April 2014 11:23:38 UTC+1, Tim Fox wrote:

Another way of tackling this issue is to refactor your app into a set of loosely coupled services each running in their own JVM and talking over the event bus instead of a single blob. This should reduce the chances of any particular Vert.x instance requiring multiple versions of the same jar at the same time.

It was mentioned earlier that that architecture will probably affect performance characteristics. Though actual significance for a particular app will presumably vary, it's not like the jvm is cpython with its annoying gil.

[Arno Schulz]

Tim what you are describing are great features, but perhaps the question are what are the benefits that are desired in vert.x. For me

it's concise it forces you to use and build reusable modules, this could be taken further as node.js does in reducing their core by separating by moving the file system, timers, http client/server, dns client and sockets into their own modules. Even the modules or package manager could be moved out a la NPM.

It's got good performance, by reducing the core size this could be a further focus would using chronicle or LMAX disruptor for the local eventbus help, in a similar vain as Rick suggested would switching to boon or protostuff or bson help.

It's a platform, I especially like how vertigo is evolving as an equivalent to noflo.js (based on node.js) perhaps embracing Vertx as a platform more we could see our own express.js and http-proxy and passport etc...

It's polyglot, I've started using it as javascript platform that would allow me to hook into existing java libraries. And I've seen interesting work use groovy and clojure.

Over all I believe that Vertx is mature but it could grow on the community side (bigger following), refocusing on multiple package managers (maven for Java, NPM for js, leigning for clojure, gems for ruby, etc...) while logical for the developers seems at odds with creating a polyglot platform - on the other hand the zipped module is unifying for Vert.x and it really says KISS.

[Rick Hightower]

RE: It's got good performance, by reducing the core size this could be a further focus would using chronicle or LMAX disruptor for the local eventbus help, in a similar vain as Rick suggested would switching to boon or protostuff or bson help.

Kryo and Boon JSON would be my picks for binary and JSON/text serialization. Boon, you probably guess why... :) 

Though... protostuff is pretty close to Kryo

https://github.com/eishay/jvm-serializers/wiki

Kryo seems a bit faster overall. But then again, it does not really matter. Either would work, and we could roll our own that was cross lang. It is not that hard to write a fast serializer. I've done it a few times. They are fast enough that differences would be dust on the scale. 

LMAX disruptor is great but it is bit hard to tune for a generic use case like a general purpose event bus like vertx provides.

Chronicle... seems a bit off topic... 

https://github.com/peter-lawrey/Java-Chronicle

https://github.com/OpenHFT/Java-Chronicle

(Unless you can tie that in... Off topic BTW why did Apache Apollo decide to use LevelDB? Did Java-Chronicle not exist or is LevelDB faster?)

Back to LMAX disruptors....

Disruptor API is a bit unwieldily to say the least. 

Also unless you really know what you are doing, it very hard to get the disruptor to outperform the LinkedTransferQueue which is part of JDK 7, and LinkedTransferQueue is in a word: AWESOME! 

See 

http://php.sabscape.com/blog/?p=557

I did not write what he wrote, but have witnessed the power of LinkedTransferQueue, and have deployed multi-million user apps uses LinkedTransferQueue as a key component of a super-high-speed writer. 

I've burnt holes in disks with LinkedTransferQueue. You can do amazing things with LinkedTransferQueue, and the API is easier to grok (follow the link above). I've done some NIO without the aid of Netty (websocket load testing client), and I am no expert but I was able to get things to really, really fly. 

My guess is if you are creating a general purpose event bus, disruptor would just get in your way. If you are writing a trading system and can control and have the time to write a custom ring buffer for you very specific data structures then that is a horse of a different color. Please prove me wrong.

It is really a formula 1 versus a Porshe SUV, except you need to haul groceries so... pretty much the porsche suv is the answer.

I have not dug into the internal (other than a cursory glance to see what / how) of how the vertx event bus is constructed but I assume since they are at the top of the techempower benchmark, they probably got that handled. 

Arno,

I don't know you from Adam. So forgive me my indulgence. Tim and the Netty guy have put some amazingly fast stuff in vertx. It is whipping the pants off just about everything out there. Personally if I were going to talk about how much faster the event bus could be, I would fork vertx, modify it, setup some JMH tests, show the before and the after, and then talk about how this or that could make the event bus faster. Because I think.... it can be done, but it is not easy and it is certainly not going to be easy to create a general purpose bus using LMAX disruptor as a basis. But this is all opinion and conjecture. I have some experience, but am not expert by a long shot (but this has never stopped me before).

--Rick Hightower

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/nLXpM5WM9qQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Arno Schulz]

Yes you're right Vertx has a lot in place and the event bus part came across wrong, I meant that by focusing on a smaller core then tests to increase performance as you suggest regarding different frameworks for serialization or inter process communication could be done.

Personally I'm working in HL7 message transfers and chronicle's approach with the journaling on disk would be beneficial (especially when messages don't make it through)

You received this message because you are subscribed to the Google Groups "vert.x" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.

[bytor99999]

But we can say the exact same thing going the other way. If they are separated and you wanted to upgrade all/both. Now you have to do that work twice. 

Not that either argument is a third world problem.

Mark

p.s. If you are replying to this thread and unless you are posting comments inline. Be careful on the size of the post you are creating. Usually it will have excess stuff making us scroll down really far to get to something usefull. (Again, this is just a first world problem.)

On Friday, April 18, 2014 5:26:38 AM UTC-7, Dean Pehrsson-Chapman wrote:

[bytor99999]

On Saturday, April 19, 2014 3:20:07 AM UTC-7, Tim Fox wrote:

It's a double edged sword. Yes, having classloader isolation allows problems like this to be solved, but it also introduces other problems, e.g. in your example if you then passed different versions of the same class from one module to another you would get ClassCastExceptions.

I'd say having two versions of the same jar in an app often causes more problems than it solves.

I wish I read this before I posted my reply as this one says it better.

Mark

[bytor99999]

Having used Spring ROO for a while to find out its main issues (Also have used archetypes, JBoss SeamGen. Grails own built in builders and a few others) these types of tools are great to a point. But the main point where they fail and will always fail, is the tight coupling with certain other technologies and not being able to swap it out correctly. Thymeleaf and tag libs in Spring ROO was a huge one for me. These tools are very very opinionated on what should be used. And Tim has said this a lot of times, is that Vert.x is un-opinionated.

Thanks

Mark

[David Dossot]

It's a double edged sword. Yes, having classloader isolation allows problems like this to be solved, but it also introduces other problems, e.g. in your example if you then passed different versions of the same class from one module to another you would get ClassCastExceptions.

I'd say having two versions of the same jar in an app often causes more problems than it solves.

In the same app, totally agreed. But for two independent apps (i.e. mods) deployed in the same Vert.x container, surely classloader isolation is a must.

[Dean Pehrsson-Chapman]

I'd say having two versions of the same jar in an app often causes more problems than it solves.

Some of those problems are easier to solve than others :)

Tim, I can see the reason for getting rid of the module system, but you must have thought it was important originally.  Can you talk about the original aims and why you don't think they are worth it anymore?

[S C]

Before Tim has a chance to speak about the classloader :)
for me it was important to have a dependencies system working ACROSS technologies. Why? If my app uses both Javascript and Java modules as it often does, I don't want to have to deal with BOTH npm and maven at the same time. It's just very naive to think javascript developers will publish modules in maven (and java modules in npm? fat chance) so the actual module system solved this for me. If webjars brings it the same way, I'm all ears.

S