Component State Machine

[Oliver Rolle]

Hi Community,

I work on a method to translate a sbpm process model to source code. sbpm uses 2 models: a subject interaction model which is much like a vertigo network and a subject behavior which is like a finite state machine (FSM). I searched a modelling tool for FSM and found SMC (State Machine Compiler; http://smc.sourceforge.net/) which compiles a model of a FSM (*.sm file) into Java-Source (or JavaScript, Python and many more langs) which controls the control flow of a underlying user programmed class. For a deeper look into SMC see http://smc.sourceforge.net/slides/SMC_Tutorial.pdf

I think SMC might be even outside sbpm interesting as Jordan defines the recovery algo for all deterministic components which is equal to a FSM. Why not modelling the FSM explicitly? One draw back has SMC, it does not provid any maven artefacts by the project itself. There are some 3rd party maven-plugins which are used to compile *.sm files to *.java files via maven plugin. See for example https://github.com/sfdc-hhildebrand/SMC but I do not get this plugin to work (which is from 2012 and uses its own repo).  A different, working but older plugin is found here http://mojo.codehaus.org/smc-maven-plugin/howto.html - the generated source is not compatible to a later SMC version and has less features.

Here is my scrum backlog / roadmap for the next 2-4 weeks:

User story: As a developer I like to define a high-level component behavior so that I can model component behavior before implementing the behavior.
1. register domain so that I can publish software on sonatype
2. create a SMC github fork and publish this on sonatype
3. fork https://github.com/sfdc-hhildebrand/SMC, update to latest SMC and publish it on maven
3.a. Optional: add a SMC code gen for vertigo and / or vertx3 (overlaps?)
4. Integrate network model and component model so that it can be shared easily

Any suggestions, additions, changes in priority?

best regards
Oli

[Oliver Rolle]

@Jordan: Do you have a script which pushes multi-module mvn projects artifacts to sonatype?

1. done

[Jordan Halterman]

This seems very interesting!

The current Vertigo master pushes all the Maven modules to Central as separate projects. I just use the Maven release plugin, version each module, and then let it to its thing. The parent pom just needs to be setup properly (which can be a pain, but once it's done it's done)

Sent from my iPhone

On Nov 6, 2014, at 1:30 PM, Oliver Rolle <oliver...@googlemail.com> wrote:

@Jordan: Do you have a script which pushes multi-module mvn projects artifacts to sonatype?

1. done

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.
To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/c5b2beb3-ce3c-4c16-af86-a841c2138f79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Oliver Rolle]

2. done
3. done
See https://github.com/orolle/SMC. Binaries should be available on maven central soon. I wait for feedback from Sonatype.

3.a SMC code generation for vertigo / vertx is a little bit complicated. I will delay this task after the vertigo 1.0 api is fixed as I do not like to re-write the code gen on every api change ;-)

4. Change this task to: Re-write the PoC with SMC code generation. This task gives me a better perspective how SMC is used in conjunction with the network model and how yoke can be integrated. This will prove the ability of SMC to model component behavior and implement the model on a lower level.

PS: Beside the model-abstraction of SMC, it might increase the performance of vertigo components as finite state machines are typically fast and failure-resilient. This should be tested / benchmarked.

[Jordan Halterman]

That's probably a good idea. Things are changing quickly and will continue to change quickly for a few weeks. It's a lot different since I last pushed to the 1.0 branch. I think tomorrow I'll push what I've done which is quite a bit with still a long way to go.

SMC seems incredibly interesting, and forking it was a great idea :-) FSM is such a perfect application for Vertigo, I think we can move more and more towards making it a core concept with fault tolerance and features revolving around the concepts of FSM.

--

You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/18e30d77-7a08-4764-82de-a3b9f0a1725b%40googlegroups.com.

[Oliver Rolle]

Am Sonntag, 9. November 2014 14:47:03 UTC+1 schrieb Jordan Halterman:

SMC seems incredibly interesting, and forking it was a great idea :-) FSM is such a perfect application for Vertigo, I think we can move more and more towards making it a core concept with fault tolerance and features revolving around the concepts of FSM.

I think it should be an additional concept making it easy to develop FSM-components, but not every vertigo component is a FSM - e.g. a database client.

My core contribution to vertigo is to make vertigo easy to use for high-level designed applications. Similar to the UI which allows to model networks, the SMC allows to model deterministic components. If vertigo 1.0, UI and SMC is put together we have everything to model flow-based / process-based applications, support the implementation of flow-components via FSM and execute it as a distributed application on a cluster with strong QoS. Up to this point we have a nice pipeline from development to operation (good for continuous integration). What is missing is to seamlessly integrate a powerful web framework (like yoke) - than we should be able to provide a very useful tool to implement process-oriented web-scale apps.

[Jordan Halterman]

On Nov 9, 2014, at 8:27 AM, Oliver Rolle <oliver...@googlemail.com> wrote:

Am Sonntag, 9. November 2014 14:47:03 UTC+1 schrieb Jordan Halterman:

SMC seems incredibly interesting, and forking it was a great idea :-) FSM is such a perfect application for Vertigo, I think we can move more and more towards making it a core concept with fault tolerance and features revolving around the concepts of FSM.

I think it should be an additional concept making it easy to develop FSM-components, but not every vertigo component is a FSM - e.g. a database client.

Agree. Obviously core Vertigo "components" will always remain low level. I'm still exploring ways to provide higher level component-based APIs like FSM and event-sourced. The challenge in this is doing so in a polyglot manner. Various languages handle polymorphism in a variety of ways, so other patterns have to be explored to build an API that makes sense in all languages. The few ideas I've played with are controller verticles, decorators, and strategies. I think I probably favor using a decorator type interface for proving statefullness for the simple fact that it allows more modular code and is really easy to accommodate in all languages. In fact, I'll probably just continue to directly port Vertigo to Vert.x 3 and then provide those features in separate modules.

My core contribution to vertigo is to make vertigo easy to use for high-level designed applications. Similar to the UI which allows to model networks, the SMC allows to model deterministic components. If vertigo 1.0, UI and SMC is put together we have everything to model flow-based / process-based applications, support the implementation of flow-components via FSM and execute it as a distributed application on a cluster with strong QoS. Up to this point we have a nice pipeline from development to operation (good for continuous integration). What is missing is to seamlessly integrate a powerful web framework (like yoke) - than we should be able to provide a very useful tool to implement process-oriented web-scale apps.

Another idea I've played around with is integrating a web framework into Vertigo. This is obviously one of the areas that is sorely lacking, and I'm willing to make any changes necessary to accommodate making it easier to feed information into a network or expose a network via an HTTP interface.

I have done a lot of research on Akka and have been interested in a couple of concepts there. I added a ComponentReference interface to the current API which allows you to communicate directly with a component's input/output ports. So, you can deploy a network and send messages to or receive messages from its components. This I think is especially important for testing.

The other component of Akka in which I've been interested is the concept of supervisors which creates essentially a hierarchical system of actors wherein "parents" determine how to handle the failures of their children. I think this makes for a uniquely solid foundation for actor systems and I'm mostly interested by the fact that it forces the user to think about how to handle failures. However, because of the differences between Akka's actor model and FBP, I haven't found a good place to apply those types of concepts to Vertigo. But I would definitely like to find a way to expose failures to the user and integrate strategies for handling those failures. It may just be that the network/component configuration API should support failure strategies on a per-component basis, or the NetworkReference and ComponentReference interfaces should expose methods for handling failures.

One thing that I did integrate from Akka, though, is its excellent configuration library. I'm quite fond of the syntax and think it will make it much easier to define networks and components in configuration. It allows you to include other configuration files and even reference other configuration values. I am sort of leaning towards trying to use this syntax to provide metadata for various elements of the Vertigo application, including cluster, networks, components, state machines, and other services.

Another interesting feature that I added to Vertigo 1.0 was a fluent network DSL. This is only experimental, so I don't even know if it should stick around, but it's sort of growing on me:

Network.builder()

  .connect()

    .component("foo")

      .main("foo.js")

  .and()

    .component("bar")

      .main("bar.py")

  .to()

    .component("baz")

      .main("baz.rb"):

Or...

Network.builder()

  .component("foo")

    .main("foo.js")

  .component("bar")

    .main("bar.py")

  .component("baz")

    .main("baz.rb")

  .connect("foo")

    .port("out")

  .and("bar")

    .port("out")

  .to("baz")

    .port("in")

Or a combination of both of course :-)

My only concern is that there are too many ways to define a network. Configuration file, configuration class, and fluent DSL.

I'll still try to push this stuff tonight. Out visiting family right now.

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/ac46e6c0-d0d8-4f28-a477-d20d27058c61%40googlegroups.com.

[Jordan Halterman]

While we're on the topic, I want to reiterate that anyone with any gripes about the current API PLEASE make a proposal now!

I'm wondering if the input/output APIs should be changed...

From:

input.port("in").messageHandler(...)

To:

input.messageHandler("in", ...)

...in keeping with the event bus API (though I think the event bus API may be changing with EventBus.consumer(String address) returning a MessageConsumer, which is sort of synonymous to a port. Maybe not :-)

I'm also looking to make component configuration files a much more integral part of Vertigo 1, with ports being declared explicitly. I'm wondering what other useful metadata could be required.

By the way, ports in Vertigo 1.0 are typed, which I like :-)

I think over the next couple of weeks the API should start taking real shape, so I want to get any syntactic changes in ASAP. Always keep in mind that APIs need to be language agnostic, so things like abstract base classes don't work - interfaces only. Of course, abstract base classes can (and are) provided for Java, so long as other languages can access a lower level interface.

On Nov 9, 2014, at 8:27 AM, Oliver Rolle <oliver...@googlemail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/ac46e6c0-d0d8-4f28-a477-d20d27058c61%40googlegroups.com.

[Jordan Halterman]

I should add: I mentioned typed ports in relation to configurations because ports are now typed via the Java compiler (generics), but I'm wondering if it would be useful to add type support to ports in component configuration files. This could allow us to do configuration time type checking, i.e. throw an exception if an int port is connected to a String port. I'm always in favor of strong typing wherever possible. Perhaps types can be optional (supporting an Object port as the default).

On Nov 8, 2014, at 2:59 PM, Oliver Rolle <oliver...@googlemail.com> wrote:

--

You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/18e30d77-7a08-4764-82de-a3b9f0a1725b%40googlegroups.com.

[Oliver Rolle]

Thanks Jordan for your comments! I will answer each of your comment inline :-)

I am back online after I got a "programmer"-disease and my doc had ordered me not to use a keyboard for a week so that my arm can regenerate. Now everything is fine :-)

[Jordan Halterman]

On Nov 18, 2014, at 9:24 AM, Oliver Rolle <oliver...@googlemail.com> wrote:

Thanks Jordan for your comments! I will answer each of your comment inline :-)

I am back online after I got a "programmer"-disease and my doc had ordered me not to use a keyboard for a week so that my arm can regenerate. Now everything is fine :-)

Don't forget to stretch before each time you take the field! :-)

If you have never been wounded programming, you're not doing it right!

Am Montag, 10. November 2014 02:14:33 UTC+1 schrieb Jordan Halterman:

I should add: I mentioned typed ports in relation to configurations because ports are now typed via the Java compiler (generics), but I'm wondering if it would be useful to add type support to ports in component configuration files. This could allow us to do configuration time type checking, i.e. throw an exception if an int port is connected to a String port. I'm always in favor of strong typing wherever possible. Perhaps types can be optional (supporting an Object port as the default).

On Nov 8, 2014, at 2:59 PM, Oliver Rolle <oliver...@googlemail.com> wrote:

2. done
3. done
See https://github.com/orolle/SMC. Binaries should be available on maven central soon. I wait for feedback from Sonatype.

3.a SMC code generation for vertigo / vertx is a little bit complicated. I will delay this task after the vertigo 1.0 api is fixed as I do not like to re-write the code gen on every api change ;-)

4. Change this task to: Re-write the PoC with SMC code generation. This task gives me a better perspective how SMC is used in conjunction with the network model and how yoke can be integrated. This will prove the ability of SMC to model component behavior and implement the model on a lower level.

PS: Beside the model-abstraction of SMC, it might increase the performance of vertigo components as finite state machines are typically fast and failure-resilient. This should be tested / benchmarked.

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.
To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/18e30d77-7a08-4764-82de-a3b9f0a1725b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/8fe8eeac-5c87-4597-94a9-e015ddda0b1d%40googlegroups.com.

[Oliver Rolle]

Am Dienstag, 18. November 2014 21:26:28 UTC+1 schrieb Jordan Halterman:

Don't forget to stretch before each time you take the field! :-)

If you have never been wounded programming, you're not doing it right!

I never ever thought I could be wounded by programming itself!
you can bet an eye that I will stretch before writing a single line of code. The worst thing during my time afk was having good ideas all the time but I could not code  them. It feels like being on a diet on an all-inclusive vacation :-D

damn it! its after 2 am. I answer your comments tomorrow.

[Jordan Halterman]

Yeah I know that's gotta be torturous! There's no way I could comply for that long!

--
You received this message because you are subscribed to the Google Groups "vertigo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx-vertig...@googlegroups.com.
To post to this group, send email to vertx-...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx-vertigo.

To view this discussion on the web visit https://groups.google.com/d/msgid/vertx-vertigo/a9e77f54-5f21-485e-bb63-d28d0cc0c314%40googlegroups.com.

[Oliver Rolle]

I had a discussion with the inventor of sbpm (Albert Fleischmann) and he proofed me that the original plan to marry vertigo with sbpm on the same abstraction level will not work as imagined :-( sbpm is a modelling language, whereas the model can be realized in vertigo, akka and other frameworks. My core assumption was that the sbpm subject communication model is equal to a vertigo network and therefore I can marry them, but that was not 100% the case. From sbpm perspective subjects are like classes, whereas from vertigo perspective subjects are like a concrete instance of a class - the ScopeSelector should do the translation. Problem is that the ScopeSelector cannot do this translation in every case as it mixes 2 different abstraction layers. Therefore we have to separate them. Actually its no big deal, as we can reuse everything! Good news is that we can separate the sbpm modelling from sbpm execution more clearly, but it adds degrees of freedom during the model to execution translation which increases software complexity. Damn trade-offs!

The sbpm model is an abstract design of interacting FSMs (a subject is a FSM). In the model layer we have no database and web interactions as the sbpm model only models the behavior (not the doing itself). The doing is part of the implementation. As we translate the model into a vertigo "exexcutable" network, we have to translate everything to a network and its components.

Lets assume the model can be translated into a network. (I think it can for most cases.) Next we have to translate the FSMs to components. I think the decorator pattern is perfect for our approach! Lets compile a FSM example.sm to a controller class (SMC provided), an vertx-codegen-annotated interface (yet to be done) and a vertigo component (yet to be done). The interface is implemented by ImplIExample, whereas the component routes communication between Controller and ImplIExample and other vertigo components.

The controller is generated by SMC and controls ImplIExample within the component.

@VertexCodeGen
public interface IExample extends SbpmBehavior {
  // Transition Events, would be received on inport("onEvent1") and routed to FSM-controller
  public void onEvent1(JsonObject event1);

  // Transition guards to prevent FSM from switching in wrong state; called by controller
  public boolean isEvent1Ok(JsonObject event1);

  // do Methods are called by controller if a Transition has been done
  public void doSomethingAfterTransition();
}

Component:

public class ConcreteExampleComponent extends ComponentVerticle {
  IExample stateful = new ImplIExample();
  ExampleController fsm = new ExampleController(stateful);

  public void start(){
    inport("onEvent1").onMessage((JsonObject o) -> fsm.onEvent1(o));
  }
}

Lets say we want a web-enabled component, than WebIExample implements IExample and IExportState (Interface to export the state for html rendering) and WebStateRenderer (WebStateRenderer uses the state and renders a html document; maybe auto generated or manual written?):
Here is the web-enabled Component:

public class ConcreteExampleComponent extends ComponentVerticle {
  IExample stateful = new WebIExample();
  ExampleController fsm = new ExampleController(stateful);
  WebStateRenderer render = new WebStateRenderer(stateful, fsm);

  public void start(){
    inport("onEvent1").onMessage((JsonObject o) -> fsm.onEvent1(o));
    inport("http").onMessage((HttpRequest req) -> render.handle(req)); // processes http request
  }
}

I am not happy with this design... it might get tricky for an user to determine what 

Am Montag, 10. November 2014 01:24:50 UTC+1 schrieb Jordan Halterman:

My core contribution to vertigo is to make vertigo easy to use for high-level designed applications. Similar to the UI which allows to model networks, the SMC allows to model deterministic components. If vertigo 1.0, UI and SMC is put together we have everything to model flow-based / process-based applications, support the implementation of flow-components via FSM and execute it as a distributed application on a cluster with strong QoS. Up to this point we have a nice pipeline from development to operation (good for continuous integration). What is missing is to seamlessly integrate a powerful web framework (like yoke) - than we should be able to provide a very useful tool to implement process-oriented web-scale apps.

Another idea I've played around with is integrating a web framework into Vertigo. This is obviously one of the areas that is sorely lacking, and I'm willing to make any changes necessary to accommodate making it easier to feed information into a network or expose a network via an HTTP interface.

Very interesting idea to expose a network via an HTTP interface! Maybe it is compatible with an http-inport on component level. It would make network / components restful?
 

I have done a lot of research on Akka and have been interested in a couple of concepts there. I added a ComponentReference interface to the current API which allows you to communicate directly with a component's input/output ports. So, you can deploy a network and send messages to or receive messages from its components. This I think is especially important for testing.

This is very useful!
 

One thing that I did integrate from Akka, though, is its excellent configuration library. I'm quite fond of the syntax and think it will make it much easier to define networks and components in configuration. It allows you to include other configuration files and even reference other configuration values. I am sort of leaning towards trying to use this syntax to provide metadata for various elements of the Vertigo application, including cluster, networks, components, state machines, and other services.

I agree! I like the library the more I do with it. Its very intuitive and expressive :)

My only concern is that there are too many ways to define a network. Configuration file, configuration class, and fluent DSL.

 Do you have a prefered way to build networks? Too many ways will be hard to maintain :-(

[Oliver Rolle]

I have no changes to suggest. I like it the way it is :-)

[Oliver Rolle]

I agree to add types to config files, as we want runtime configuration and deployment of networks and this will make it easier to accomplish?

Am Montag, 10. November 2014 02:14:33 UTC+1 schrieb Jordan Halterman:

[Oliver Rolle]

2.a. done github: https://github.com/orolle/SMC
2.b. done sonatype / maven central binaries
3. done smc-maven-plugin (http://mojo.codehaus.org/smc-maven-plugin/howto.html) ported to SMC 6.4.0 ; synced with github and binaries pushed to sonatype
3.a. vertigo smc compiler: skipped / wait for more specific requirements (task 4) & api stabilization
4. in progress