Workflow Project daily progress report [GSOC 2013]

[hardik juneja]

Hello Dominic --

Today i implemented the approach you suggested tomorrow.
Its working smooth.

Tomorrow, I will try to extend the functionality of the model developed till now.
This include taking create/view/search/update as inputs and generating the workflow from them.

Roadblocks : For update we need an id of the particular record that user want to update, but in our menu we will just show option to update
so, we can show them a list view first and then they can open the record to update it.

But, what if there are no record? 
Then i think we should check the db for that and with notice can move further.

One more thing, I think the list views will be stops.
How to move further once a list view is showed? how long will we wait on a list view?
I still will have to think on this.

Regards,
Hardik Juneja

[Dominic König]

tisdagen den 25 juni 2013 11.31.27 skrev hardik juneja:
> Roadblocks : For update we need an id of the particular record that user
> want to update, but in our menu we will just show option to update
> so, we can show them a list view first and then they can open the record to
> update it.

A menu item to update records doesn't make sense.

Typically, we have a table or list view of records where each entry has a
button/link/icon to edit it - /that/ would go to update.

IMHO, the menu item would be the workflow itself.

> One more thing, I think the list views will be stops.
> How to move further once a list view is showed? how long will we wait on a
> list view?

Hmm, interaction is a two-player game. If the system has completed a step
(whether it provided a form or a list doesn't really make a difference), then
it's the user's turn to initiate the next request - and that's what the system
is waiting for.

So the next step is initiated by the user clicking an action item on the page.
That could be

- a form-action (=submit, if the page contains a form)
- a record-action (e.g. read/update/delete, if the page contains records)
- a workflow action
- implicit (e.g. back/skip/cancel/continue/checkout)
- explicit (menu action)

For each of these actions, the page provides the necessary action items:
buttons, links, menus...whatever - even a list view.

As soon as the user picks an action - the workflow continues. If he does never
pick an action - then it does not continue: no request - no response. That's
how servers work.

I mean, yes - an asynchronous task could run a timer, and if the user takes
too long to respond, automatically alert his neighbour to check whether he has
died of a heart attack...but hey: that's probably beyond GSoC. For now, we
accept the user's sudden death and do nothing until they serve us a new one.

Dominic

[Dominic König]

tisdagen den 25 juni 2013 11.31.27 skrev hardik juneja:

> One more thing, I think the list views will be stops.
> How to move further once a list view is showed? how long will we wait on a
> list view?
> I still will have to think on this.

What about a popup box "DO SOMETHING!!!" accompanied with a nasty siren sound?

Dominic

[Dominic König]

I mean - it's called a "work"flow, not a "sit-and-think-"flow

:D

Yeah - ok ok, I'm going to get more serious again soon, I promise.

Dominic

[Dominic König]

tisdagen den 25 juni 2013 11.31.27 skrev hardik juneja:

> One more thing, I think the list views will be stops.
> How to move further once a list view is showed?

From my brother's handbook for interaction design:

"Stay calm and wait for someone to resuscitate the user."

So, not just my idea :D

Dominic

[hardik juneja]

Hmm, interaction is a two-player game. If the system has completed a step

(whether it provided a form or a list doesn't really make a difference), then
it's the user's turn to initiate the next request - and that's what the system
is waiting for.

So the next step is initiated by the user clicking an action item on the page.
That could be

- a form-action (=submit, if the page contains a form)
- a record-action (e.g. read/update/delete, if the page contains records)
- a workflow action
    - implicit (e.g. back/skip/cancel/continue/checkout)
    - explicit (menu action)

For each of these actions, the page provides the necessary action items:
buttons, links, menus...whatever - even a list view.

As soon as the user picks an action - the workflow continues. If he does never
pick an action - then it does not continue: no request - no response. That's
how servers work.

My point was -

If user is going to define route for the workflow.

So, lets say he chooses

1) req/req/create

2) org/org/create

3) org/org/list

4) pr/pr/create

now after completing two steps 
The third one is a list view.

So now he should be given freedom to do anything 

which include following tasks-

- a form-action (=submit, if the page contains a form)
- a record-action (e.g. read/update/delete, if the page contains records)
- a workflow action
    - implicit (e.g. back/skip/cancel/continue/checkout)
    - explicit (menu action)

and then workflow continue with the forth step

okhay :)

 

I mean, yes - an asynchronous task could run a timer, and if the user takes
too long to respond, automatically alert his neighbour to check whether he has
died of a heart attack...but hey: that's probably beyond GSoC. For now, we
accept the user's sudden death and do nothing until they serve us a new one.

Right, thats beyond GSOC ;)

[hardik juneja]

I know that was a foolish question but i wanted to say that once we mare on a list view and when user perform certain actions and if that includes say updating a record as he reaches the update view then get_vars (?workflow = ) is gone.

So there must be some way to tell the workflow request handler, that the user has not yet left the workflow and will come back. :)

--

Hardik Juneja

[Dominic König]

Correct,

a page that is rendered by a workflow must include the workflow ID in any
possible next action that is initiated /from/ this page.

One option is to store the current workflow in the session, and handle any
subsequent request in this session as step within this workflow until the user
explicitly exits the workflow.

But this isn't quite right: if the user navigates away, e.g. by choosing a
menu item outside of the workflow, or from his browser bookmarks, then you may
connect a request to the workflow that doesn't belong to it.

The better option is to store the workflow ID in the page, and have any
subsequent request /from that page/ that belongs to the workflow re-submit it -
whilst other requests which do not belong to the workflow, do not re-submit it.

For example:

If a workflow renders a datatable, then it renders all action buttons in the
datatable with the workflow ID in the action URL.

When the user then clicks on an action button for a record, then the workflow
ID is automatically re-submitted (in the request URL, i.e. get_vars).

If he instead navigates away to another page outside of the workflow, then the
action buttons on that page would not include the workflow ID, hence requests
from that page are not handled by the workflow anymore.

---

The question wasn't really "stupid" - it comes back to what you said about
where to store the "workflow information" (i.e. the identity of the workflow
instance): and in fact, the best option is to store it in the page, and that
is, in any action item on that page that belongs to the workflow.

Usually, there aren't really many different action items - for a start it seems
sufficient to modify the action buttons in datatables according to the workflow
settings (because a "list view" is actually not a stop, but a "choose record
and next action" step), and in all POST forms to add a hidden INPUT that
carries the workflow identity.

You may need additional elements for workflow navigation: "back", "cancel",
"skip", "continue", "cancel" and the like, probably as additional header or
footer elements (e.g. "buttons") - and of course (optional? configurable!) info
elements to display the workflow status (could be a header, for example) and
instructions for the user.

Furthermore, you may need menu items to return to pending workflows after
navigating away. This isn't always required, though.

---

Sorry for having made fun of it - but it was very hard to resist... :S

Dominic

[Dominic König]

tisdagen den 25 juni 2013 22.48.41 skrev Dominic König:
> But this isn't quite right: if the user navigates away, e.g. by choosing a
> menu item outside of the workflow, or from his browser bookmarks, then you
> may connect a request to the workflow that doesn't belong to it.

Typical catch:

User starts a workflow in one browser tab, and an additional (non-workflow) page
in another tab. This typical habit of users with modern browsers speaks
strongly against session.

> Sorry for having made fun of it - but it was very hard to resist... :S

It could be worse: it could be my brother typing :D these customer service
guys compensate their constant hyper-friendliness with quite a dark kind of
humor.

Dominic

[hardik juneja]

If a workflow renders a datatable, then it renders all action buttons in the
datatable with the workflow ID in the action URL.

When the user then clicks on an action button for a record, then the workflow
ID is automatically re-submitted (in the request URL, i.e. get_vars).

If he instead navigates away to another page outside of the workflow, then the
action buttons on that page would not include the workflow ID, hence requests
from that page are not handled by the workflow anymore.

Appending the workflow id in the action button looks sufficient for now.

 

You may need additional elements for workflow navigation: "back", "cancel",
"skip", "continue", "cancel" and the like, probably as additional header or
footer elements (e.g. "buttons") - and of course (optional? configurable!) info
elements to display the workflow status (could be a header, for example) and
instructions for the user.

I have all of these in my mind. :)

do you want this to be done earlier?

 

Furthermore, you may need menu items to return to pending workflows after
navigating away. This isn't always required, though.

So if we are going to have a menu then it will be good to save the workflow id and its name, with its status (current and completed) in the DB too.

Sorry for having made fun of it - but it was very hard to resist... :S

Its okay, I too found it funny :)

what's the name of the handbook that you mentioned earlier?

[Dominic König]

Can we have our meeting on IRC please?

Dominic

[hardik juneja]

yes!!

On Wed, Jun 26, 2013 at 1:21 PM, Dominic König <dom...@nursix.org> wrote:

Can we have our meeting on IRC please?

Dominic

--

Hardik Juneja

[hardik juneja]

H

ello Dominic,

Sorry for the delay

Things got delayed due to git conflicts other git issues.

please review the pull request 

https://github.com/flavour/eden/pull/522

--

Hardik Juneja

[Dominic König]

Already done.

[Dominic König]

Hardik--

let me re-iterate my "Shopping List" example:

In this (imaginary) workflow, we start with a request to GET a list of supply
items:

/supply/item/wf_id=reqitems

This initiates a new workflow "reqitems" - S3Request.__call__ will detect the
"wf_id" and hand over to the workflow engine, which executes the request in the
context of the workflow.

Normally, /supply/item just gives you a list view of supply items - it's not
leading anywhere.

But with the workflow identifier in the URL, this presents a list view of supply
items *plus* additional action items:

1) an action button for each record to add it to the "shopping list"
2) a menu to see the currently "selected" items with option to "deselect" them
again
3) a button "Request these items"

So, if the user clicks on a "Add to shopping list" button, then this initiates
a request to

supply/item/4?wf_id=reqitems:1&wf_action=select

(This time it has an instance-ID for the workflow)

Again: the S3Request detects the workflow qualifier and hands this request over
to the workflow engine for processing.

If the user clicks on "Request these items", then this button links to

GET req/req/create?wf_id=reqitems:1

And again, S3Request detects the workflow qualifier and hands the request over
to the workflow engine. This will return a req/req create-form that is
prepopulated with the previously selected items. The user can fill in this form
with the request details, and then submit to

POST req/req/create?wf_id=reqitems:1

And again, this goes to the workflow engine which now creates the new request.

None of the requests ever goes to a dedicated workflow-controller - but through
regular RESTful controllers. That is, the opposite of what you did:

Instead of:
/workflow/index?c=supply&f=item&wf_id=req_items

we call:
/supply/item?wf_id=reqitems

This has a lot of advantages:

a) controller settings can be applied even by the workflow engine, e.g. you
have the prep and postp of that controller available
b) controller-based authorization rules apply automatically
c) you get the right S3Request/S3Resource combo - no need to "fake" anything
d) you get the right menu/breadcrumbs status

The tricky part is to connect the steps by rendering the action items into the
page, and to realize the interim data storage (e.g. the "list of selected
items").

What we need right now is:

a) tweak of S3Request.__call__() to call the workflow engine if the request is
addressed at a workflow
b) the workflow engine to instantiate the respective workflow (or return to the
regular S3Request if no such workflow can be found)
c) a configuration pattern for workflows: this can be a Python-class pattern for
now like workflow_x = S3WorkflowNode(...) & S3WorkflowNode(...) &
(S3WorkflowNode(...) | S3WorkflowNode(...))

If we have that, we have to discuss how we match a request to the right
workflow node, and how we store the current workflow status.

And then we discuss a framework how we render workflow action items into the
page.

Once all this works, we can discuss how to serialize workflows to store them in
the DB (or export/import them via XML) - but for now we really need the engine
first.

The key people I know who are keen to implement workflows in Eden are all
Python-capable - and only when they are happy, we extend it beyond that
audience.

Dominic

torsdagen den 27 juni 2013 00.47.29 skrev hardik juneja:

[hardik juneja]

Hmmm,

This is getting interesting :)

I almost got every thing from the start to the end.

Instead of:
/workflow/index?c=supply&f=item&wf_id=req_items

No, my url pattern was same as yours.

Something like 

so /workflow/index was a place which shows you a form to enter the route so if you enter

-> req

-> create

-> commit

-> list

and then click submit button

[ But now we will use define a configuration pattern for workflows: this can be a Python-class pattern for

now like workflow_x = S3WorkflowNode(...) & S3WorkflowNode(...) &

(S3WorkflowNode(...) | S3WorkflowNode(...))

 

]

then it first takes you to req/req/create, as you click submit then to req/commit/list
so the url will for above was - 

req/req/commit/create?workflow=req_items:1Ab34e:w3sd:234asd

then, 

req/req/commit?workflow=req_items:1Ab34e:w3sd:234asd


I have got what we are trying to do
Thanks a lot
for such a detailed example :)

[Fran Boon]

On 26 June 2013 22:41, Dominic König <dom...@nursix.org> wrote:
> let me re-iterate my "Shopping List" example:
> In this (imaginary) workflow

I don't think this is imaginary at all - I think it's an excellent
example which would be well worth coding :)

F

[hardik juneja]

Hi Dominic --

I have cleaned up the pull request and moved the condition check into Request.__Call__()
its working fine.

And, I will like to discuss the design for defining configurations for the workflow engine.

You suggested something like -

 workflow_x = S3WorkflowNode(...) & S3WorkflowNode(...) &

(S3WorkflowNode(...) | S3WorkflowNode(...))

​can we this be -

workflow_1 = [ S3WorkflowNode("req", "req", "create"), 

[S3WorkflowNode("req", "req", "list"), S3WorkflowNode("supply", "item", "create")], 

S3WorkflowNode("req", "commit", "list") ]​

or 

workfow_1 = [ ("req", "req", "create"),

[("supply", "items", "create"), ("req", "req", "list")] ,

("req", "commit", "list") ] 

​First one would be better​, I think.

​After defining this we will have to save it somewhere.

And then our S3Workflow class will acess it and will pop each step and will execute it.
and after popping if it found that its is a list then it will check for the status and will execute the required status.

​

​Regards,

Hardik juneja​

[Dominic König]

Hmm --

that's your call, actually.

I'm not so much a friend of "Can we..." questions - of course we "can".
Instead, I'd like to see a good reasoning behind such design decisions, i.e.
an answer to the question "why should we"?

I'm not so convinced that a nested list structure will be sufficient - there may
be more operators required to specify connections between steps than one
(=comma is the only available operator in nested lists).

But as I say - your decision.

Dominic

[hardik juneja]

I'm not so much a friend of "Can we..." questions - of course we "can".
Instead, I'd like to see a good reasoning behind such design decisions, i.e.
an answer to the question "why should we"?

First of all i will like to know how what we are going to do with the configuration variable, i mean in this case "workflow_x"

workflow_x = S3WorkflowNode(...) & S3WorkflowNode(...) &
(S3WorkflowNode(...) | S3WorkflowNode(...))

what i had in my mind was if we use a nested list pattern something like -

workflow_1 = [ S3WorkflowNode("req", "req", "create"), 

[S3WorkflowNode("req", "req", "list"), S3WorkflowNode("supply", "item", "create")], 

S3WorkflowNode("req", "commit", "list") ]

then we can store this variable somewhere and we make workflow class to access this list and and then our workflow class should be intelligent enough to decide the order of the node to call.

so it takes each elements from the list and execute it

and if it found a sublists then it check for the workflow data and look for correct element in the sublist and execute it.

but, the problem is how will it come to know what actually a condition is.
i mean the condition on the basis of which the element in the sublist will be chosen.

And,
if we consider this design 

workflow_x = S3WorkflowNode(...) & S3WorkflowNode(...) &
(S3WorkflowNode(...) | S3WorkflowNode(...))

Then i will like to know how what we will do next
are we going to store this somewhere?
or pass it to a class?
if yes, then is that class going to break it on the basis of operators?

Sorry, i am taking time on the design but i think this is important before i can move further

Looking forward to hear from you,
Hardik Juneja

[Fran Boon]

I would have thought nested dicts was a little more flexible than lists

F

[Fran Boon]

On 27 June 2013 21:34, Fran Boon <fr...@sahanafoundation.org> wrote:
> I would have thought nested dicts was a little more flexible than lists

& as I mentioned before, JSON, seems a good storage mechanism - both
for data in page (either jquery.data or a hidden field) & later into
the DB

You definitely need to be able to handle conditional logic...lists
just aren't flexible enough IMHO
With a dict you have a lot more options...

F

[hardik juneja]

You definitely need to be able to handle conditional logic...lists

just aren't flexible enough IMHO
With a dict you have a lot more options...

Nested Dictionary would be much better, I agree.

Are you suggesting json to define the configuration ?
We are discussing about the way to define the configuration for the workflow engine.

Hardik Juneja

[Fran Boon]

On 27 June 2013 22:27, hardik juneja <hardikj...@gmail.com> wrote:
> Nested Dictionary would be much better, I agree.

:)

> Are you suggesting json to define the configuration ?
> We are discussing about the way to define the configuration for the workflow
> engine.

Way to store it

F

[Dominic König]

torsdagen den 27 juni 2013 21.35.50 skrev Fran Boon:
> On 27 June 2013 21:34, Fran Boon <fr...@sahanafoundation.org> wrote:
> > I would have thought nested dicts was a little more flexible than lists
>
> & as I mentioned before, JSON, seems a good storage mechanism - both
> for data in page (either jquery.data or a hidden field) & later into
> the DB

Hmm, right - but storing workflow status/data isn't really what I'm after here.

This is primarily about the workflow definition itself, i.e. how to implement a
workflow. This will need serialization some day, but for now we've agreed to
hardcode it in Python.

> You definitely need to be able to handle conditional logic...lists
> just aren't flexible enough IMHO

This is definitely more important.

But let me become a little more generic here:

Typically, a workflow defines data objects, and certain events for these data
objects which trigger actions.

The workflow engine is to identify the event and the data objects it involves,
determine the status of the objects and from that determine and invoke an
event handler (from a pre-defined set of event handlers) to handle the event.

The event handler would then introspect the data object and determine which
actions to perform and how, then perform the actions and perhaps update the
object's status (if the action is complete), and exit.

---

Or, from a different perspective:

The workflow engine is controlled by events - and if a certain event and a
certain object status come together, a certain event handler is called for the
data object.

If the event handler is successful, it would update the object's status, so
when a next event for the same data object occurs, it would have a different
status, thus be handled by another event handler.

With this general principle, the workflow engine moves a data object from one
event handler (node) to another, until a final status is reached.

---

The first class of events we're focussing on are RESTful requests. The event
hook is what Hardik put into S3Request.__call__ - this detects the event and
hands it over to the workflow engine (=event manager). Additionally, the
S3Request also identifies the data object the event is about(=S3Resource).

We may later get other event classes connected (e.g. status transitions of
data objects can be events in themselves = internal events).

---

The data we need to store in addition to the mere data object is the data
object's current status (within a certain workflow). This is ideally a JSON
structure which can easily be serialized both into the page and the DB.

However, /this/ discussion thread is about how to implement a workflow, which
we agreed to have hardcoded in Python for now (serialization to come later).

The focus for that should be to make it /easy/ for developers to define a
workflow, yet flexible.

Dominic

[Dominic König]

torsdagen den 27 juni 2013 23.47.26 skrev du:
> The first class of events we're focussing on are RESTful requests. The
> event hook is what Hardik put into S3Request.__call__ - this detects the
> event and hands it over to the workflow engine (=event manager).
> Additionally, the S3Request also identifies the data object the event is
> about(=S3Resource).

Well...and the type of event: the request with all it's parameters.

Dominic

[Dominic König]

But in the standard case, S3Request does not take the workflow /status/ of the
resource into account - i.e. it does not connect any previous actions
performed on the same object to the current event.

This is a basic principle of RESTful API's: the interface is stateless, it
forgets what you (or anyone) did before and handles every request as if it was
the first ever.

However, the data resources are not "stateless" - they can be changed by
requests so subsequent requests find a modified resource. This does not violate
the principles of the API.

Therefore (and this comes with an exclamation mark): workflow is focussed
around the "status" of the /resource/ (and not the status of the interface!).

It does not really matter what your last request was - the critical point is
what the current status of the resource is, and what "next" status is to be
achieved within a certain workflow.

Additionally, events in a workflow don't have to involve always one and the
same user - in fact, it is quite common that it wouldn't be the same user.

E.g., in our shopping list example, after sending the request the workflow
isn't really complete yet ;)

Dominic

[hardik juneja]

Dominic --

What about my question :(

https://groups.google.com/d/msg/sahana-eden/uWXty9xQiqg/oVIg4nzOGTIJ

[Dominic König]

So,

the workflow "nodes" are not really requests, but statuses of the data object
(in our example the "request"):

new => prepare => requested => committed => fulfilled

Alternative routes are possible:

new => prepare => cancel
new => prepare => requested => cancel

Statuses could also be more complex, of course.

Whenever a node renders a page, it offers the action items to achieve a status
transition. That is the whole point of an interactive workflow node.

E.g., in our example, for the "new => prepare" transition that would be a
button that says "Request the selected items". Clicking this button moves the
workflow data object from the "new" status to the "prepare" status, thereby
moving it forward.

Of course, the same user could have multiple "shopping list" workflow instances
in process at the same time, each at a different status (e.g. one new, another
requested).

Dominic

[Dominic König]

Where to put the workflow configuration?

Good question - I don't know yet.

Since they need to be accessible cross-controller, it can't be in controllers.
And since they need to be template-specific, it can't be models either.

Seems they are best placed in a separate module á la "menus.py" or
"layouts.py" (workflows.py?).

Another option is of course to put them into separate models.

Whichever option you choose - ideally, a workflow doesn't produce any overheads
whatsoever unless/until it's needed. It was therefore I suggested a class-
pattern, because that only happens when the object really gets requested (like
method handlers, which are mere class handles until they have to process a
request).

I would really like to hear your suggestions first before I answer this,
otherwise I'm developing this instead of you - and that's not really what you
want is it?

Dominic

[Fran Boon]

On 27 June 2013 23:53, Dominic König <dom...@nursix.org> wrote:
> Where to put the workflow configuration?

> Since they need to be accessible cross-controller, it can't be in controllers.
> And since they need to be template-specific, it can't be models either.
> Seems they are best placed in a separate module á la "menus.py" or
> "layouts.py" (workflows.py?).

+1

Just like we have parser.py too...

F

[hardik juneja]

Sorry i slept yesterday

Whichever option you choose - ideally, a workflow doesn't produce any overheads
whatsoever unless/until it's needed. It was therefore I suggested a class-
pattern, because that only happens when the object really gets requested (like
method handlers, which are mere class handles until they have to process a
request).

I would really like to hear your suggestions first before I answer this,
otherwise I'm developing this instead of you - and that's not really what you
want is it?

Storing the configuration in modules say workflow.py like we do in menu.py is what i had in mind too

[Fran Boon]

On 28 June 2013 11:13, hardik juneja <hardikj...@gmail.com> wrote:
> Storing the configuration in modules say workflow.py like we do in menu.py
> is what i had in mind too

* templates not modules

Core functionality in modules, actual workflows in templates...

F

[Dominic König]

To close the loop:

In a request to

/supply/item?wf_id=<name>:<id>

- the request (=URL+HTTP method) determines the *Event*
- the <name> determines the workflow type
- <name>:<id> identifies the particular data object (workflow instance)

The status and current content of the workflow instance comes (most likely)
from the database. It is very likely to be a JSON-serializable data structure,
e.g. nested dicts or something.

The workflow engine retrieves the instance, and maps its current status and the
event (request) to a certain pre-defined event handler (=node).

That's basically all the workflow engine does.

The event handler responds to the event, either by performing a certain action
or by providing a page with action items for the user to perform an action -
or both.

When the action is complete, the event handler udpates the status of the
workflow instance (= status transition).

Which action the event handler performs, and what the next status of the
workflow instance is depends on the event data - which is always both, the data
submitted by the user (if any) /and/ the related resource(s).

Let's assume that "actions" are standard REST methods (S3Methods):

Then the event handler can introspect /and/ manipulate the event data - at
least before and after the method has been performed, using a preprocess- and
a postprocess-hook equivalently to s3.prep and s3.postp (of course, we may
want to introduce additional hooks over time - but these are the two I was
thinking of first).

However, other "actions" could also be custom functions specific for the
particular workflow - it would be good to leave that open yet have a standard
interface. I do though believe that the S3Method interface could serve well
even for that purpose (yeah - we may need to extend it a little bit).

For every event, the event handler may need to perform multiple "actions" -
either sequentially, or conditionally or even in loops (e.g. if a component
has multiple instances).

The key points to resolve:

- configuration of event handlers (nodes) in a workflow
- storing and retrieving of workflow instances
- mapping of instance status + current event to a certain node
- interface between workflow engine (=event manager) and workflow nodes (=event
handler)
- interface between nodes and "actions" (=S3Methods?)
- workflow widgets (=widgets for users to perform workflow actions), and how to
put them into a page

I think these are the 6 top-stories of the implementation part.

We can certainly break them down into individual jobs, but I'd like to leave
that to you.

For the first story, we need to decide whether we have multiple event handler
(node) classes, or just one.

I'm currently in favour of a single event handler class with options to
customize it, rather than having custom event handler classes.

The advantage would be that you could serialize the options into e.g. XML, so
that they can be exported/imported easily, or modified using the GUI - thus
separated from Python code in a way that they can be worked on by application
designers without them having to touch the code.

However, that may be a /huge/ overhead to parse and handle all the possible
node config options - or the event handler class becomes too inflexible, so that
workflow definitions become rocket science.

Additionally - if the node configuration becomes more complicated than
programming Python, then this doesn't really make any sense. Python is
relatively easy and most people who design workflows in Eden are actually
Python-literate in some way.

So - I think this is pretty much all I have in mind right now.

Regards,
Dominic

[Dominic König]

fredagen den 28 juni 2013 13.01.23 skrev du:
> - configuration of event handlers (nodes) in a workflow

Note that this /includes/ the configuration of the possible/desired status
transitions - and since that is where the workflow design usually starts, I'd
like to see this as the basis of the configuration pattern (even though it is
not mandatory - but much easier to see the "flow" then).

Dominic

[hardik juneja]

The key points to resolve:

- configuration of event handlers (nodes) in a workflow
- storing and retrieving of workflow instances
- mapping of instance status + current event to a certain node
- interface between workflow engine (=event manager) and workflow nodes (=event
handler)
- interface between nodes and "actions" (=S3Methods?)
- workflow widgets (=widgets for users to perform workflow actions), and how to
put them into a page

Noted

 

For the first story, we need to decide whether we have multiple event handler
(node) classes, or just one. 

 

I'm currently in favour of a single event handler class with options to
customize it, rather than having custom event handler classes.

Are you talking about S3WorkflowNode class here?
if yes then i am in favour of having one class.

Just one more question?

if we make make developers define all the workflow in one file say workflow.py.
let say this way

workflow_x = S3WorkflowNode("req","req","create") & S3WorkflowNode("req" , "commit", "list") & (S3WorkflowNode(...) | S3WorkflowNode(...))

Then are we going to split this syntax some way?

And one more foolish question
is this correct syntax wise? 

[Dominic König]

fredagen den 28 juni 2013 04.46.30 skrev hardik juneja:
> Just one more question?
> if we make make developers define all the workflow in one file say
> workflow.py.
> let say this way
>
> workflow_x = S3WorkflowNode("req","req","create") & S3WorkflowNode("req" ,
> "commit", "list") & (S3WorkflowNode(...) | S3WorkflowNode(...))
>
> Then are we going to split this syntax some way?
>
> And one more foolish question
> is this correct syntax wise?

Again - this is for you to flesh out in detail. I'm just trying to give a high-
level steer.

The syntax is something that you would design, and it should be easy to learn
and robust, but also flexible enough for future extensions.

Massimo (the web2py creator), but also many others, have made use of standard
object operators to define complex structures, for example:

The DAL class overrides __getattr__() to give access to the tables in the
current database:

table = db.my_table

The Table class overrides __getattr__() to give access to the fields in the
table:

field = table.my_field

The Field class defines a __eq__() method which returns a Query instance, so
that you can do:

query = (field == 2)

Then, the DAL class (which represents a DB Adapter) defines a __call__() method
that accepts a Query instance and returns a Set:

set = db(query)

The Set class defines a method select(), which accepts Field instances and
returns a Rows object:

rows = set.select(field1, field2)

Now, you can put that all into one statement, which gives the classic web2py
DB query syntax:

rows = db(field==2).select(field1, field2)

The Rows class overrides the __getitem__ method to allow iteration over the
individual rows, while their Row class also overrides __getitem__() to give
access to the individual values in a row based on the Field instance:

for row in rows:
value1 = row[field1]

As you can see, there is not a single constructor involved in this sequence
despite it involves 6 different classes - and only a minimum of explicit
function calls.

This isn't really magic - but it is easy to learn, almost intuitive and hides
the complexity of the DAL enough so that you can focus on your actual goal.

If course, this "hidden complexity" can bite you sometimes - but the risk for
headaches is significantly higher where options don't follow the overall
pattern.

Consistency is key, both with Python itself and the overall pattern: if you
pass 9 options as keyword arguments, and one as a global variable, then that
one global variable will keep confusing others and causing bugs - even if it
seems so much easier than a keyword argument at a first glance.

But this in turn means you need to make good decisions to allow future
extension without having to re-design the syntax pattern. The problem is that
it isn't easy to back out a syntax pattern once it is in production - code
migrations are a major pain in....sorry.

Anyway...

What I want to say is that you don't have to follow my (very rough) syntax
outline - I didn't really think it through, but just typed something.

Rather, follow your intuition: try your syntax out to implement the "shopping
list" or whatever example you want - this gives you an impression.

Dominic

[Dominic König]

fredagen den 28 juni 2013 15.00.22 skrev du:
> Rather, follow your intuition: try your syntax out to implement the
> "shopping list" or whatever example you want - this gives you an
> impression.

i.e. if you need a lot of comments to explain what your example workflow
definition does, then it's crap ;)

Dominic

[Fran Boon]

On 28 June 2013 14:00, Dominic König <dom...@nursix.org> wrote:
> What I want to say is that you don't have to follow my (very rough) syntax
> outline - I didn't really think it through, but just typed something.
> Rather, follow your intuition: try your syntax out to implement the "shopping
> list" or whatever example you want - this gives you an impression.

This is great general advice for ALL students:
What a mentor advises in 5 minutes may take you 5 hours to experiment
with & come back with the suggested improvements.
You are working full-time (40 hours/week) on the projects whilst us
mentors are volunteering just a few hours a week.

Our role is NOT to be prescriptive in the first instance (although
yes, that can come later at merge time), but rather give more general
guidance & then review your work & suggest improvements..

F

[hardik juneja]

This is great general advice for ALL students:
What a mentor advises in 5 minutes may take you 5 hours to experiment
with & come back with the suggested improvements.
You are working full-time (40 hours/week) on the projects whilst us
mentors are volunteering just a few hours a week.

Our role is NOT to be prescriptive in the first instance (although
yes, that can come later at merge time), but rather give more general
guidance & then review your work & suggest improvements..

I am really sorry,
I respect your time, I will this in mind next time.


Dominic--

Lets have a class let say S3WorkflowConfig .
So all the workflow will be defined under this class (like we have in  s3menu)

these will be defined like -

def req_items(self):

       return dict(node1 = S3WorflowNode( . . . ), node2 = dict(choice1 = S3WorkflowNode( . . . ), choice2( . . . )), node3 = S3WorkflowNode( . . . ) )

here req_items is the name of the workflow.

So our workflow engine will look for Workflow name in the URL and then will call "req_items".
It will return the full configuration to the workflow engine and then we can save it in a temporary place and then will execute the steps accordingly.

[hardik juneja]

And Yes,

def req_items(self):

       return dict(node1 = S3WorflowNode( . . . ), node2 = dict(choice1 = S3WorkflowNode( . . . ), choice2 = S3WorkflowNode( . . . )), node3 = S3WorkflowNode( . . . ) )

We are using dictionary here-

The disadvantage of this will be that dictionary have no order , so the order in which we define the nodes will be gone, which is not at all good.

Solution for this can be, either we use use the names as defined in the above example i.e keeping it like node1, node2, node3 . . 

This way we can sort the dictionary and can easily use it.

Or we can use Ordered Dictionary 
Ordered Dict

[Dominic König]

Maybe then you take a closer look at the syntax I suggested ;)

This allows for both, a defined order and branching/alternatives.

Dominic

[hardik juneja]

Dominic--

Today-
I tried to extend the functionality of your workflow engine, I made it take configuration from the a file private/template/default/workflow.py, where configuration will be defined by the developers.

def req_items():

      workflow = dict(node1=S3WorkfowNode('' req ", " req ", " create"), node2 = S3WorkflowNode(...))

Here req_items is workflow name

[ I know this is still Dictionary here :(

  We will not move further without having a good syntax. ]

Good News, It runs smoothly it takes the configuration from the file and store it in the database.

[BUT JUST FOR NOW I am storing the configuration in session later i will use json-serialised data structure to store it] 

and then continue the workflow by executing nodes.

The workflow name is taken from the configuration itself, So once the request goes on the URL with the same workflow name as defined in the configuration then that workflow will start executing.

Ex.
The URL

/req/req/create?wf_id:req_items

This will take you to the workflow "req_items" and will start the workflow as defined in the configuration files after adding the uuid.

Roadblock- I am still not able to decide a good syntax is close to the one the you suggested and we will only be able to move further on once we have a good syntax. can you please give me some more hint on the one u suggested?
I am totally blank with ideas right now :(

I am storing the Configuration in the session instead of Json serialisable data [Just for now, because i don't want to move further without having a good syntax], Please, Will you like to review the code?

Thanks
Hardik Juneja