Vertigo Module Registery - Anyone likes join?

[Oliver Rolle]

Hi community,

intro: the motivation behind the ui is to make software development easier through combining high-level modules with each other and therefore enforce modular design and module re-usability & module sharing. But how does the developer know which modules are available and suitable for her problem?

Lets invert the problem: why does she not know the modules?
If I use Eclipse + Java to define a vertigo network, the IDE cannot help me which modules are available. I could inform myself through the module registry of vertx, browse mailing lists or use google. Most of the time I search for 3-5 minutes than I decide, I could code it myself (if its not too big) - its more fun than searching. The core point is, the search for modules is too time-consuming with respect to the fun coding provides - can you confirm this behaviour with your experience?

I think vertx module registry is not fully thought-out. Its a yellow page, which has a very simple search function, but I cannot look into mod.json, its not easy to share a module with the community (maven one liner would be awesome!) and it has no API for machine-to-machine interaction.

proposal: I want to improve the vertx module registry to solve above issues. The registry should be made especially for vertigo - make the search and sharing of vertigo modules and networks as easy as possible, while being compatible to vertx modules. The last step is to integrate the registry in the ui therefore making the registry available for the development process (enable the ui to share and reuse networks). Do you like to join or support this sub-project?

[dgo...@squaredfinancial.com]

On Thursday, 24 July 2014 17:00:35 UTC+1, Oliver Rolle wrote:

I think vertx module registry is not fully thought-out. Its a yellow page, which has a very simple search function, but I cannot look into mod.json, its not easy to share a module with the community (maven one liner would be awesome!) and it has no API for machine-to-machine interaction.

Beware this is potentially misinformed FUD, I'm just a user of vertx: There is maybe a bit of an elephant in the room there with the vert.x 3.0 plans, which last I checked included  abandonment of the "vertx-platform" part of vertx! [1]. AFAIUI that means an end to the brief life of vertx modules as a thing.  There'll presumably still be a need for some sort of directory for human-level discovery of vertx-oriented libraries if/when that happens, though.

We (my employers, I don't mean vertigo or vertx) had started to think vaguely about existing similar (for some value of similar) systems like maybe osgi + a p2 server (i.e. like eclipse plugins)  as a contingency plan. There was a 2012-era investigation (not by us) of vertx plus osgi [2][3][4].  Not saying it's ideal or even a good idea...

[1] https://groups.google.com/d/msg/vertx/DHL9cAg4SYo/FAqi1iBimOwJ
[2] http://underlap.blogspot.com2012/06/osgi-case-study-modular-vertx.html
[3] https://github.com/glyn/vert.x.osgi
[4] http://www.slideshare.net/glynnormington/is-osgi-modularity-always-worth-it

[Amit Kumar]

+1 for sharing vertigo 'pre-assembled workflows' as well as 'cooked components' as modules in a separate module/component/workflow repository.

[Oliver Rolle]

Am Montag, 28. Juli 2014 18:23:47 UTC+2 schrieb dgo...@squaredfinancial.com:

Beware this is potentially misinformed FUD, I'm just a user of vertx: There is maybe a bit of an elephant in the room there with the vert.x 3.0 plans, which last I checked included  abandonment of the "vertx-platform" part of vertx! [1]. AFAIUI that means an end to the brief life of vertx modules as a thing.  There'll presumably still be a need for some sort of directory for human-level discovery of vertx-oriented libraries if/when that happens, though.

I think Jordan knows more about the development of vertx. I remember that vertx and vertigo might or do converge?
 

We (my employers, I don't mean vertigo or vertx) had started to think vaguely about existing similar (for some value of similar) systems like maybe osgi + a p2 server (i.e. like eclipse plugins)  as a contingency plan. There was a 2012-era investigation (not by us) of vertx plus osgi [2][3][4]. Not saying it's ideal or even a good idea...

I think osgi support is an advantage for vertx as it handles configurations and dependencies for deployments. AFAIUI the module system seems to last in vertx 3.
The vertigo modulereg is more about sharing vertigo networks. Networks are configurations - clearly there is some major overlap between modulereg and osgi.
 

[Jordan Halterman]

Sorry Oliver. I just realized Google filtered that post as spam. All fixed.

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.
To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/fc347fe3-8361-42c4-9963-2b58e804e02c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[dgo...@squaredfinancial.com]

On Monday, 28 July 2014 20:29:30 UTC+1, Oliver Rolle wrote:

I think Jordan knows more about the development of vertx. I remember that vertx and vertigo might or do converge?

Little doubt he does.

I would idly speculate at least the cluster-wide shared data api (or functionally similar) that vertigo currently provides (which we locally borrowed - the more we use hazelcast, the more we, uh, use hazelcast - I dare say we could swap out vertx itself from our system before hazelcast at this stage...) will appear in core vertx sooner or later.

I think osgi support is an advantage for vertx as it handles configurations and dependencies for deployments.

 Yeah, Sadly osgi is ...maybe a leetle bit more bureaucratic...

 AFAIUI the module system seems to last in vertx 3.

I was just pawing through the current vertx master branch...  vertx-platform is gone, cut down to vertx-dist, which just provides a little script to launch class Starter [1], which handles verticle launch only.  Method .deployVerticle() just hanging on class Vertx [2] now. Nope ...not looking good for modules. Though a classloader isolation facility is still there, which was one important aspect of what modules were doing (since it allows deployment of multiple versions of the "same" class) (but then if you end up having to use osgi anyway for other aspects, say, you'll probably end up using osgi's isolation instead [3], ho hum...).

> The vertigo modulereg is more about sharing vertigo networks. Networks are configurations - clearly there is some major overlap between modulereg and osgi.

Another idea (I see you're already discovering from local maven repo so you may have already considered) is maybe pushing other artifact types to good ol' remote/central maven repos?

 Maven repo servers including maven central currently provide documented search query apis [4][5] over and above a "raw" maven repo's "dumb but maybe  crawlable" structure, though they unfortunately differ from eachother (but hey, that could be hidden behind an abstraction layer). We're presently using an archiva instance in-house, it too has a search api [6].

In the case of the public repos I suppose too many automated search queries in too short a time might be perceived as abuse (though in the maven central case I suspect anything vertigo/vertigo-flow could generate would be a tiny drop in a vast ocean, and caching could be done...).

You can get a pretty good list of vertx modules from maven central just by searching with the classifier "mod" they use, as in [5].

If the mod.json, say, were to be uploaded too, outside the mod.zip itself, that would avoid having to grab the full mod.zip to inspect and discover which ones are vertigo components.

Similarly json network specs could be pushed (with another distinct classifier or something). Archiva apparently happily stores arbitrary .json artifacts (tested), can't speak for maven central or nexus-based repos but I imagine they can too.


[1] https://github.com/eclipse/vert.x/blob/master/vertx-core/src/main/java/io/vertx/core/Starter.java
[2] https://github.com/eclipse/vert.x/blob/master/vertx-core/src/main/java/io/vertx/core/Vertx.java
[3] http://spring.io/blog/2009/01/19/exposing-the-boot-classpath-in-osgi/#class-space

[4] http://blog.sonatype.com/2011/06/you-dont-need-a-browser-to-use-maven-central/
[5] http://search.maven.org/solrsearch/select?q=l:%22mod%22&rows=500&wt=json
[6] http://archiva.apache.org/docs/2.1.0/adminguide/webservices/rest.html#Search_Service

[Jordan Halterman]

Alright. I'm slowly coming back to my open source projects. I'm moving this weekend, but after that I plan on having lots of time to get back in to it.

The Hazelcast cluster API is definitely going to be in Vert.x itself soon. Basically, I think Vertigo actually provides a lot of these types of features just a little ahead of them becoming part of Vert.x core. That's why Vertigo 1.0 will be developed for Vert.x 3, and since it looks like they've gotten a lot of the core changes figured out, I'll probably start digging into it pretty soon.

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/167322ea-8ac7-400b-b307-6b8afaff6480%40googlegroups.com.

[Oliver Rolle]

Am Freitag, 1. August 2014 03:13:03 UTC+2 schrieb dgo...@squaredfinancial.com:

> The vertigo modulereg is more about sharing vertigo networks. Networks are configurations - clearly there is some major overlap between modulereg and osgi.

Another idea (I see you're already discovering from local maven repo so you may have already considered) is maybe pushing other artifact types to good ol' remote/central maven repos

 Maven repo servers including maven central currently provide documented search query apis [4][5] over and above a "raw" maven repo's "dumb but maybe  crawlable" structure, though they unfortunately differ from eachother (but hey, that could be hidden behind an abstraction layer). We're presently using an archiva instance in-house, it too has a search api [6].

In the case of the public repos I suppose too many automated search queries in too short a time might be perceived as abuse (though in the maven central case I suspect anything vertigo/vertigo-flow could generate would be a tiny drop in a vast ocean, and caching could be done...).

You can get a pretty good list of vertx modules from maven central just by searching with the classifier "mod" they use, as in [5].

If the mod.json, say, were to be uploaded too, outside the mod.zip itself, that would avoid having to grab the full mod.zip to inspect and discover which ones are vertigo components.

Similarly json network specs could be pushed (with another distinct classifier or something). Archiva apparently happily stores arbitrary .json artifacts (tested), can't speak for maven central or nexus-based repos but I imagine they can too.

 

I had to think about it a couple of days, because your suggestions destroyed my original plan. Not too bad, as it saved me a couple of work days!
 

I did not thought to use remote maven, but you are totally right! (The local maven was a practical decision. It just contained my locally developed mod.zip's.)

The usage of the module registry as a search engine for vertigo & vertx modules is a very clever idea! I will hack a prototype together.

[Oliver Rolle]

Welcome in your new home! I hope nothing important broke during the move ;-)

[Oliver Rolle]

Found some time to implement a prototype of the registry. Its based on joke and jade4j. In total ~250 lines of code :-)
See: http://partypeer.de:8080/

• crawls all coordinates which contain mod.zip file form search.maven.org
• downloads the mod.zip and extracts mod.json
• makes mod.json browsable