Re: Workflow Engine related questions

[Herman Peeren]

The onAfterExecute event is specific for CLI-applications and not used elsewhere. But I think a plugin is not the way to go. In Joomla! at the end of a controller the next controller to execute is set by a redirection-url. That is the time to call the workflow-engine to ask what the next step is.

I think there are 2 points where the workflow engine must be called:

• when starting the main controller. That is: after user-input. Now, depending on state and user-input, the right (sub)controller and task has to be determined
  
• when having executed the (sub)controller-task: to determine the next redirection

You say "during the workflow, the engine has to be in charge of the execution". That is true, but because it is implemented in a stateless webworld the logic is a bit the other way around: the controller is in charge and delegates some decisions to the workflow engine.

On Friday, 25 May 2012 08:04:34 UTC+2, Florian Voutzinos wrote:

Hello,

I have a question about my GSOC project the Workflow engine.

I use Sebastian Bergmann low level workflow engine which is very well written.

There are two parts :

- the graph definition or workflow with nodes, different types of branching and merging.
- the execution engine allowing to execute the graph.

The goal is to execute controller tasks which are mapped to graph nodes.

Let's say a user enters a checkout process of an e-commerce extension.
The corresponding task starts the workflow execution by calling the engine, then the engines knows task A of Controller B has to be executed, then task C of Controller C on success, task D on failure (it depends on the graph...).

So during the workflow, the engine has to be in charge of the execution : the executed task provides values to the engine (for example : payment success, failure) so he knows which branch to choose for example (so which task as a follower).

I think a plugin with the onAfterExecute event would do the trick : supposing the engine knows which task to execute from which controller, he can call it.

I'm wondering if this is the best way, and how to do it ? It should work with both JApplicationWeb and JApplicationCli if possible and I don't know if instanciating a controller and execute a task from a plugin is a good way...

Thanks.

Regards,

Florian

[Herman Peeren]

Elin points out (thanks) that JApplicationWeb gives you onBeforeExecute and onAfterExecute. Sorry, had overlooked that; I usually make CMS-applications and in CMS 2.5 JApplicationWeb is not used (yet), if I'm not mistaken. Thank you for the correction. Strange though: I see Elin's reaction in my e-mail, but not (yet) online; would expect it the other way around...

But my main points are:

• that I think a plugin is not the best way to call your workflow engine.
  
• that the the "Joomla!-way", what ever that might mean exactly, would probably be to use the setting of redirection in the subcontroller.
• I suggested to flip the thinking about who is in charge.

[Elin Waring]

You're right it isn't used in CMS 2.5, though if Michael can pull off what he is working on (thread on the CMS list) it will be there for 3.0. If not it is all of a piece with the new MVC and UCM and thus future oriented libraries and platform applications should be using it. So that's what this project is aiming for. This is a big project (not unlike ACL was in many ways and of course it would not be possible to even think about in the CMS without ACL and remember even once ACL was in the framework it took a long time for it to be implemented in the CMS and in many ways it is still not completely implemented there) and I would not expect to see it in the CMS in the short term.  Also, one thing to remember with the newer paradigm is that there is a 1:1 controller and task relationship.  In other words we are going to potentially be a lot of controllers and the CMS will have a lot of work to do to to recreate JControllerAdmin's functionality for example. 

In terms of the workflow engine I think this is really an interesting question because we want to think of the Joomla way as not necessarily the current Joomla CMS way but really a more general Joomla Platform way because this engine will need to work not only for the CMS 4.0 but also for other platform applications and possibly coordinating between them (since thanks to Florian I've been reading about workflow patterns my understanding of the issues involved has shifted ).  The whole thing is rather far reaching as the separate discussion of business logic (on the UCM thread) where Andrew proposed a plugin solution seems highly relevant since a lot of workflow is about business rules. 

So ... in terms of loading the graph using plugins which are easy to make and customize seems like a very useful approach. 

In terms of who is in charge is seems too me that if you are using a workflow engine when, for example, you save your save controller would probably hand off to a workflow controller to do the next steps and then it will hand back whatever is appropriate at the end.  I'm just thinking out loud  here but it seems like in the cms the workflow engine might be implemented as what we call an extension but in other contexts there might be an application that really does nothing but manage work flows for one or more applications.

Elin

[Amy Stephen]

On Fri, May 25, 2012 at 1:04 AM, Florian Voutzinos <voutzino...@gmail.com> wrote:

Let's say a user enters a checkout process of an e-commerce extension.
The corresponding task starts the workflow execution by calling the engine, then the engines knows task A of Controller B has to be executed, then task C of Controller C on success, task D on failure (it depends on the graph...).

Normally, I would recommend using an event/plugin to set state for later actions in a workflow process, but, given you are indicating the actions are defined by controller/task pairs, I would agree with Herman's recommendation to use the redirect.

However, I think linking these to controller/tasks is going to create some problems in that you only have one action that can ever be taken as a response to another action.

Consider a possible "publish" action.
1. Notify subscribers.
2. Ping.
3. Tweet.
4. Update cache.

If these events are activated by a controller/task pair, then 4 redirects would be needed, or one task will be needed for every combination of tasks that are responses to a workflow. That gets into duplicate code, and so on.

In my thinking, controller/tasks should be used to define and activate a single, specific activity, while an event/plugin is better for firing off an unknown/unlimited number of activities (some of which may result in a task being fired.)

Always good to keep in mind that there are always many ways to accomplish the same thing.

Good luck w your project!
Amy

[Herman Peeren]

You are completely right, Florian! I didn't read the sentence in your first posting well: "It should work with both JApplicationWeb and JApplicationCli". I thought you wanted to implement a workflow application, but it is more a part of the Framework/Platform you want to develop, so that it can be used in other applications.

I like that very much: in fact I don't like the way the redirections are done now in Joomla (for instance after a save-action). A standard workflow-engine in the Platform could do that more efficiently and can then be used instead of the redirect, as is "the Joomla! way" right now. I'm enthusiastic about that change.

Then indeed the onAfterExecute can be useful to determine what the next task is. In that way, the user is free to determine if (s)he wants to use steps in a workflow process or not. A first step then could be to determine if the user-input has influence on the choice of the controller/task. For if step D is for instance to get a payment from a customer, it depends on success/faillure input coming back what next step should be taken. I first thought you should use onBeforeExecute for that, but that is not necessary if the first step is to determine what the client-input was (which also avoids a possible loop).

To summarise, what I understand as the goal of your work is:

• to provide an API to build a workflow-graph
• to trigger the workflow-engine on each onAfterExecute, to get us to the next point in the graph (if available)

That sounds great to me! And by implementing an AND-split in the graph, parallel tasks - as Amy describes - can be easily done. 

Maybe we have to think a bit about the exact way the controllers will be used. For now we mostly have a main-controller, a kind of dispatcher, and sub-controllers, that are instantiated by the main-controller. I can imagine that when a workflow-engine is available we can reorganise that structure too. But that is for later.

[Herman Peeren]

I meant "... the developer is free to ..." instead of "... the user is free to ...".

[Herman Peeren]

On Saturday, 26 May 2012 21:18:02 UTC+2, Florian Voutzinos wrote:

It will be based on ezcWorkflow by Sebastian Bergmann which is open source.

Yes, I've read that (and related work). It is very promising to use it in Joomla!

On Saturday, 26 May 2012 21:18:02 UTC+2, Florian Voutzinos wrote:

I don't know exactly how it must be used inside a MVC but my idea is to create a type of node that can be mapped to a controller task, so during the graph execution, the engine knows which task to execute. The definition of the graph and nodes, tasks mapping will be done via the Definition API (during the extension installation for example) or xml (I'm not sure If I will have enough time). For web applications, there are two possibilities : the engine is in charge of the execution (the master) or it helps to set the redirection dynamically (I think it's better).

The point I tried to make earlier is: try to flip the thinking about who is in charge. I think it has advantages to delegate the decision what will come next to the workflow-engine. So, of course the workflow-engine has the ultimate decision in what the next step is, but it is organised the other way around from what you would think first: not the engine on top of the hierarchy saying what has to be done and then it will be executed, but the onAfterExecute-event asking the engine what is the next step. Not the workflow-engine as a central "dispatcher" initiating the rest of the tasks, but the engine as a roadmap that can be asked for the right direction all the time. It has some similarities to the way you are inclide to implement a state-machine until you flip your thinking when you discover the State Design Pattern.

On Saturday, 26 May 2012 21:18:02 UTC+2, Florian Voutzinos wrote:

The eventual problem is : during the execution there are sql queries otherwise the workflow state is not persistent, so it can slower the execution.

We are talking about  one db-query after each controller-execution; no large amounts of data, just persisting the state of the graph (current and activated nodes). I don't see much of a performance problem there.

 On Saturday, 26 May 2012 21:18:02 UTC+2, Florian Voutzinos wrote:

For daemons applications the engine must master the execution, instanciate controllers, execute tasks, fork processes if needed. So there is an other case.

Well, in fact, what I was trying to say before; you can use exactly the same delegated way as in a web-application. I think it is a great advantage if you could use one and the same way in different situations. I thought that's exactly why you wanted to use the onAfterExecute-event: because it is the same vor a webapplication or a daemon. The only thing that is different is, that no persistence of the graph's state is needed while waiting for user-input.

Hope my thoughts are helpful.

Regards,
Herman

[Herman Peeren]

FYI: the workflow in the installation of the Joomla!-CMS is controlled client-side: Javascript-code manages the sequence while keeping in touch with the server (Ajax). That could also be a way to implement a workflow-engine for a web-application, although I don't think it is useful for what you try to achieve in your project: something that works as well for a web-application as a daemon. I just mention it for completeness; always good to know the other tricks up the sleeve.

[Rouven Weßling]

On 27.05.2012, at 15:00, Herman Peeren wrote:

FYI: the workflow in the installation of the Joomla!-CMS is controlled client-side: Javascript-code manages the sequence while keeping in touch with the server (Ajax). That could also be a way to implement a workflow-engine for a web-application, although I don't think it is useful for what you try to achieve in your project: something that works as well for a web-application as a daemon. I just mention it for completeness; always good to know the other tricks up the sleeve.

Not quite. The server side application tells the client side which view will be the next step. The client side is than responsible for fetching that view.

Best regards

Rouven

[Elin Waring]

You can see what Florian has so far here:

https://github.com/florianv/joomla-platform/tree/workflow/libraries/joomla/workflow

Elin

[Herman Peeren]

Thank you!

[Herman Peeren]

It will not exactly be a state pattern, but you can get some inspiration from it in the way it uses delegation instead of inheritance. That has some similarities to what I called the "flip in thinking" about who is in charge.

In fact the new Joomla!-controllers, with only one task to be executed, are an implementation of a command pattern.

See http://www.slideshare.net/HermanPeeren/design-patterns-illustrated for our illustrations of those "classical" GoF design patterns (state on slide 42, command on slide 33).


On Friday, 1 June 2012 12:28:48 UTC+2, Florian Voutzinos wrote:

Thanks for the answers.

I think I will implement the state patern for the Engine (maybe Command also) and the nodes.

I'm going on a fork, trying to simplify some things and adapt some concepts to Joomla.

Regards,

Florian Voutzinos

On Friday, 1 June 2012 12:28:48 UTC+2, Florian Voutzinos wrote:

Thanks for the answers.

I think I will implement the state patern for the Engine (maybe Command also) and the nodes.

I'm going on a fork, trying to simplify some things and adapt some concepts to Joomla.

Regards,

Florian Voutzinos