Idea: closures and actor-like components in vertigo

[Oliver Rolle]

Hi community,

as the module system is gone in vertx 3 (and its unclear if it might re-appear in some other form) I focus on another problem which drove me insane a couple of month ago. (If vertx3 is stable-released I will re-evaluate the UI / registry stuff with the hope to be able to reuse it - atm I have written them off; f!#? sunk costs!)

Problem: I want to realize a web application with vertigo. I have a web server source (emits web requests to the network) and a web server sink (responds to the web request). Between source and sink is my application logic. If I implement this with vertigo I have to ensure that a request id is contained in each message from source to sink so that the web server sink can distinguish between parallel requests and route the message to right response. The id cannot by pass the app logic and flow directly to the sink, because the sink does not know which (app logic) message corresponds to which id. This is very impractical, because the components have to ensure that they forward the id. This is impossible to control as components are black boxes.

Solution: If it would be possible to put the id in a closure after the web server source emits the message to the application logic components (a closure-enter operator of a connection), than the components in the app logic do not need to take care of the id explicitly - its taken care by the closure. When the application logic computed the message, the closure is exited (closure-exit operator) and the id is mixed in the computed message so that the web server sink can deliver the web response. The closure enforces that a message is computed within its own scope. The scope should have a defined life time as it might not exist per request but as long as a user is logged in. A component lives as long as its network lives we might need something different to implement closures. I think of actors. A special component which creates / restores actors on enter operation and destories / persists them on exit operation - maybe this component builds its own network of components for each closure or the components spawn / destroy actors on enter / exit of the closure - I have no idea which design is better.

I play around with this closure-actor idea and I like it because I have not seen it in fbp, it makes building business logic in a web environment easy as the closure takes care of synchronizing the computation per closure and it makes components easier to reuse in different applications.

Do you think this closure-actor model is worth of implementing?
Is it possible to be implemented in/with vertigo?
I like to code a proof of concept. Which vertigo classes might be a good starting point to implement this closure-actor model?

best regards
Oli

[Arno Schulz]

Personally I go back to the FBP book http://www.jpaulmorrison.com/fbp/concepts.shtml

All components should respect that Information Packets (IP) are part of streams and sub-streams (essentially a hierarchy of IPs delimited by special bracket packages), and the default action should be to let the packets through untouched unless they contain the triggers required to perform the blocks action.

Otherwise as you've mentioned you go for solutions to manage unknown input outputs.

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.
To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/af92aa3c-085e-4c0e-a82b-8718c5cd2450%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jordan Halterman]

I'm definitely reevaluating different models for Vertigo 1.0. I doubt that the API will change much (aside from the necessary changes for Vert.x 3) but I do intend to create a framework that is more state oriented. I have looked a lot at the actor model, but the problem I have with it is that actors are not really composable, and that's one of the aspects of Vertigo that I've always found to be useful. So, I'd like to implement a model that can easily handle fault tolerance and state management without impacting combosability/reusability.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/CA%2BJY%2B9bsVPq2Jcw4Cnp6qHUKLUEepYTwjCeKEx9nAQrNwdBM3A%40mail.gmail.com.

[Oliver Rolle]

I have not much experience with actor model. Can you explain why its not composable?

The only problem I see with actors is that the actor itself decides to whom to communicate. That's hard to manage in big / complex systems. The solution to this is to keep the fbp communication style between components but instantiate components on a condition specified in the network. This is different to the actor model as the condition is defined in the network and not by the actor. It's like a group selector but group only routes - here we have an instantiation & route to the instance.

The group selector of a connection can emulate a closure if the target component deploys a sub network for each (different) group field. Than the target comp forwards the received (group-selected) msg to the deployed sub net and the computed result is emitted on the out port of the target comp. I will test this this weekend.

> You received this message because you are subscribed to a topic in the Google Groups "vertigo" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx-vertigo/k1la0Ee5Zkk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to vertx-vertig...@googlegroups.com.

> To post to this group, send email to vertx-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/vertx-vertigo.

> To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/DF16D694-1836-4263-BC5D-ABB26A17BC7C%40gmail.com.

[Jordan Halterman]

Well, as I understand it (and I'm no expert) actors are not pure functions in the way that Vertigo components can be viewed. Whereas Vertigo components receive a message on an input port and then send the message on an output port and, as you mentioned, the routing logic is a level above components, actors do have knowledge about what happens to their output. 

However, I am somewhat interested in the way that Akka handles faults. I think that a lot could be done in the way of failure handling. 

Additionally, something I have prototyped quite a bit is the idea of making connections more configurable. Right now, Vertigo provides pretty rigid tools of communication and routing. But in reality, different systems may have widely varying requirements, each of which impact performance and reliability. Just as components are composable, so too should be communication channels with regards to guarantees like ordering, queuing (push/pull), persistence, replication, etc. The question is where this configuration should occur: at the network level or in the implementation, and if in the implementation on the sending or receiving side?

These are all really high level concepts at the moment. I still have a bit more research to do on the subject, and it's likely I'll be focusing on the cluster and state management aspects of Vertigo 1.0 first anyways. I'm certainly open to examining different models *for ideas*, but primarily with a focus on the requirements of modern distributed systems, probably not with the intent of changing Vertigo's communication model.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/CAA4x-u9veqHE9FB0cbgMLOw0D8kBHV9weVm%2BxF5a6ttur6F1qA%40mail.gmail.com.

[Oliver Rolle]

Sorry, for my late answer. I moved and shit hit the fan...

Am Freitag, 26. September 2014 03:13:50 UTC+2 schrieb Jordan Halterman:

Additionally, something I have prototyped quite a bit is the idea of making connections more configurable. Right now, Vertigo provides pretty rigid tools of communication and routing. But in reality, different systems may have widely varying requirements, each of which impact performance and reliability.

Configurable connections are an awesome idea! Than I could implement my custom connection operator and add it to the run time to use it in my application, similar to components - neat :-) The only problem might be performance, as the underlying network topology is not efficient for every connection operator - but non the less a very interesting idea. I had that problem in p2p networks where I had to implement different routing mechanisms. An huge problem was that the routing performance degenerates if it does not leverage the logical network topology (p2p networks are typically organized in hierarchies, hash rings or unstructured). Vertx uses tupel spaces to organize its distributed data structures? I do not have  any experience with tupel space :-/
 

Just as components are composable, so too should be communication channels with regards to guarantees like ordering, queuing (push/pull), persistence, replication, etc. The question is where this configuration should occur: at the network level or in the implementation, and if in the implementation on the sending or receiving side?

I prefer network level, because we know that actor model has disadvantages because its solved routing on implementation level. 2nd if its on network level it is modular and easier to customize.
 

These are all really high level concepts at the moment. I still have a bit more research to do on the subject, and it's likely I'll be focusing on the cluster and state management aspects of Vertigo 1.0 first anyways. I'm certainly open to examining different models *for ideas*, but primarily with a focus on the requirements of modern distributed systems, probably not with the intent of changing Vertigo's communication model.

You are right with the priority! 1.0 should be stable.

We might fork and add experimental stuff there?

[Jordan Halterman]

I know what you mean. My move to LA was a disaster! It took us about a month to actually unpack all of our stuff. But it's over now.

I say fork away! I'm working pretty hard on cluster management stuff ATM. I've also been working a bit on getting some of Vert.x 3's language support done. That is going to be huge for Vertigo, so I think contributing to getting some languages going is a worthy cause. 

Experimentation is the way forward! Hopefully I can get a basic port of Vertigo for Vert.x 3 done soon so we can start refactoring. Vert.x 3 doesn't make too many changes. Most of it will likely be in the cluster manager. After that we can refactor the messaging implementation to accomplish whatever it is we decide we want to accomplish.

I have a ton to say on messaging, but I'll save it for a later discussion since it's already 2am here :-)

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/4a381cc4-1964-48a4-83d4-93cdbd8f4769%40googlegroups.com.

[Oliver Rolle]

Am Mittwoch, 1. Oktober 2014 10:50:53 UTC+2 schrieb Jordan Halterman:

I know what you mean. My move to LA was a disaster! It took us about a month to actually unpack all of our stuff. But it's over now.

haha, same here. I have only work stuff unboxed. I hope I do not need 1 month ^^

I say fork away! I'm working pretty hard on cluster management stuff ATM. I've also been working a bit on getting some of Vert.x 3's language support done. That is going to be huge for Vertigo, so I think contributing to getting some languages going is a worthy cause. 

I did not know that vertx3 multi language support is this easily done. If this works out we have nearly unlimited flexibility for integration in jvm applications. I am dreaming of vertigo with scala.

Experimentation is the way forward! Hopefully I can get a basic port of Vertigo for Vert.x 3 done soon so we can start refactoring. Vert.x 3 doesn't make too many changes. Most of it will likely be in the cluster manager. After that we can refactor the messaging implementation to accomplish whatever it is we decide we want to accomplish.

I will wait with the connection stuff for your vertx3 port (beside a small PoC to validate the closure idea)

I have a ton to say on messaging, but I'll save it for a later discussion since it's already 2am here :-)

I got your email on 10am with my first coffee in my hand, thinking "if I told this my ancestors 200 years ago that it will be possible to communicate with somebody on the other side of the earth in a blink of an eye, they would have burned me on a stake screaming witcher"

[Jordan Halterman]

Lol awesome.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/f949ceca-e1ce-41ca-bcd5-b27f5f9e2305%40googlegroups.com.