Getting started with ruote-web2

[Alia K]

Hello,

I've recently just installed ruote-web2 on a test vm, but I am a bit
confused on how to test the workflow functionality?

For example there are two existing processes: simple-sequence and
leave-request:

With simple sequence, I can 'launch' a process as admin, but don't
know how to complete it... Is leave-request functional or is just an
example?

In any case, is there are any starting documentation on how one should
proceed or is it too early to play around with this package?

AK

[John Mettraux]

On Tue, Jan 13, 2009 at 7:58 PM, Alia K <alia_...@yahoo.com> wrote:
>
> I've recently just installed ruote-web2 on a test vm, but I am a bit
> confused on how to test the workflow functionality?
>
> For example there are two existing processes: simple-sequence and
> leave-request:
>
> With simple sequence, I can 'launch' a process as admin, but don't
> know how to complete it... Is leave-request functional or is just an
> example?

Hi,

leave-request is just an example. You can still complete it by
"proceeding" its workitem.

> In any case, is there are any starting documentation on how one should
> proceed or is it too early to play around with this package?

There is for now no documentation about this package, it's meant to be
a bit self-descriptive (sorry I know, it's not always obvious, but
your suggestions are welcome).

Ruote-web2 will reach the "beta" stage very soon. I'm currently
working on a rudimentary form handling capability
(http://github.com/jmettraux/ruote-forms), I hope to integrate it into
ruote-web2 by tomorrow.

The main source of documentation (for Ruote) is at
http://openwferu.rubyforge.org

Of course, your questions and suggestions are welcome in this mailing list.

Best regards,

--
John Mettraux - http://jmettraux.wordpress.com

[Diego Moreno]

Hi John!

I have just seen ruote-forms subproject. It looks very interesting. I wonder how you will integrate it into workflow language, because as you know we are doing something similar. Is there some documentation about it?

Kind regards.
Diego.

2009/1/13 John Mettraux <jmet...@openwfe.org>

[John Mettraux]

On Tue, Jan 13, 2009 at 11:13 PM, Diego Moreno <dmo...@dit.upm.es> wrote:
>
> I have just seen ruote-forms subproject. It looks very interesting. I wonder
> how you will integrate it into workflow language, because as you know we are
> doing something similar. Is there some documentation about it?

Hi Diego,

ruote-forms is just a naive / rudimentary generic form handler. There
is for now no documentation about it (it's quite straightforward as
well). Ruote-forms is a plain javascript library you can glimpse at
how it's used via
http://github.com/jmettraux/ruote-forms/blob/master/public/index.html
(RuoteForms.renderForm('div_name', json_data)).

There is no integration at all for it in the workflow language.

Currently the only way to influence 'forms' via the engine/participant
is via the filters :

http://openwferu.rubyforge.org/expressions.html#exp_filter
http://openwferu.rubyforge.org/expressions.html#exp_filter_definition

I'll probably adapt ruote-forms to make it follow the directives in
the filters (passed in the workitem) (did some work about that today
http://github.com/jmettraux/ruote/commit/fd48192a94649b0b35ee144b3c6687cb690a4248).

As I write these lines, ruote-web2 form handling is limited to a text
area for editing the JSON representation of the workitem payload.
Ruote-forms intends to replace that textarea approach. But, in my
opinion, the 'form' engine should be pluggable, determined by
participant and/or workitem, like :

<participant ref="toto"
form="http://server.example.com/forms/customer_order.xform" />

for example. The form selection could as well be hidden from the
engine, left to the "worklist" part of the application (why not use a
decision table / rule engine to select the form to use based on the
participant, the workflow name/revision and the payload...)

I'm trying to avoid having 'form' stuff in the process definition
language itself (a bit like when we try avoid to have style directives
within HTML documents).

As hinted by the form attribute above, xforms (thanks to firefox)
could be used (https://developer.mozilla.org/en/XForms) for forms
smarter than the default ones proposed by ruote-forms.

Ruby makes it easy for people to plugin the logic they need, they can
do the last mile the way they want.

What do you think ?

By the way is the work of your team available / visible somewhere ?

Kind regards,

[Alia K]

Thanks for your reply. I'm quite interested in building a web based
workflow approval system based on ruote. I've looked at Seam/Jboss/
JBPM but am not too keen on immersing myself in java's verbosities as
I've mostly python/ruby centric.

What would be great is to have a simple demo app (perhaps leave-
request with forms) so that newcomers can use it as an example on how
to build ruote-web based apps. Perhaps later, users can contribute to
an example directory to demo different workflow applications.

Best,

AK

[John Mettraux]

Hello Alia,

working hard on ruote-web2 these days, your feedback is welcome. Thanks !

I saw you were working with GoFlow and Python, very interesting.

The rudimentary ruote-forms system got integrated this week and
Torsten helped me with some QA on the install steps.

http://github.com/jmettraux/ruote-web2

Best regards,

John

[Alia K]

Hi John,

> working hard on ruote-web2 these days, your feedback is welcome. Thanks !

Thank you for sharing ruote. (-:

> I saw you were working with GoFlow and Python, very interesting.

Yes, I spent some time working on GoFlow, (on the refactoring branch
specifically), but to be honest, it's still fairly early-stage as a
project and my contribution kinda tapered off when I had some other
priorities...

When I started revisiting the field: I've checked out Windows
workflow, Seam/JBPM, YAWL and finally ruote, which I find quite
evolved, and being written in a dynamic language, much more up my
alley.

> The rudimentary ruote-forms system got integrated this week and
> Torsten helped me with some QA on the install steps.
>
> http://github.com/jmettraux/ruote-web2

Will be sure to follow its progress...

Best Regards,

AliaK

[Diego Moreno]

Hi John,

Just to clarify a bit, and give a clear overview of our project: we are trying to develop all the necessary tools, mechanisms, hooks, and... "everything" so that our final users can define the workflows AND their forms completely from an XML language (and thus avoiding any contact with "lower-level" Ruby). Even more so, our next step would be building a graphical tool to generate that XML.

So, although our work is not public for the moment, we can give you a "sneak-preview".

As I have said, we work about defining rich forms into workflow. As of this moment, we define the form by setting some special fields in openWFEru language in external files. These files are included into the main workflow definition with a line that looks like the following one:

<!-- INCLUDE _v_view01.xml -->

This way we have separated the workflow definition from view definition (just look at the attached file "_v_view01.xml"). This is processed by the engine, and the result in the FLEX client looks like the screenshot we attach (although it is in Spanish, you will be able to identify most fields, for sure).

Of course, one issue would be that the form definition language is "home-made", and not standard like CSS or XFORMS, so we are considering the migration to these solutions... but that is not immediate in the case of FLEX.

What do you think about our approach to provide the end user/designer with all the means to define everything in XML inside the workflow definition, for the sake of easiness?

Best regards,
Diego.

2009/1/13 John Mettraux <jmet...@openwfe.org>

[emil...@gmail.com]

Hello John, all,

We (Diego Moreno and me) have found some _performance_ issues when launching
a workflow. We need to initialize a significant number (20-40) of
fields/variables at the beginning before giving the first <participant> tag.
However, if we query immediately the engine about which participant has the
turn, as the engine is still loading the fields/variables, we get no valid
response. The 'workaround' is to introduce a delay (sleep) before asking the
engine for the participant, which is not very desirable, as the numbers
show:

Initialize 40 fields - sleep 10s - OK
Initialize 40 fields - sleep 15s - OK
Initialize 40 variables - sleep 15s - fail

The conclusion we get is that it is very time consuming to initialize a lot
of fields, and that using variables is even worse, and we were wondering if
there is any way to improve that (e.g. loading a lot of fields at once). Can
you help us?

Thanks for your help, and best regards,

Emilio

[John Mettraux]

On Tue, Jan 20, 2009 at 7:50 PM, <emil...@gmail.com> wrote:
>
> The conclusion we get is that it is very time consuming to initialize a lot
> of fields, and that using variables is even worse, and we were wondering if
> there is any way to improve that (e.g. loading a lot of fields at once). Can
> you help us?

Hi Emilio,

which version of Ruby ? (1.8.7 xxx ?)
which version of the Engine ? (0.9.18 ?)
which flavour of the Engine ? (db_persisted, file_persisted,
cached_file_persisted, ?)
which error_journal are you using ?

Thanks in advance for that information, it will be very helpful.

I will give you some hints on how to optimize that in a further email
(after dinner).

[John Mettraux]

On Tue, Jan 20, 2009 at 7:50 PM, <emil...@gmail.com> wrote:
>
> The conclusion we get is that it is very time consuming to initialize a lot
> of fields, and that using variables is even worse, and we were wondering if
> there is any way to improve that (e.g. loading a lot of fields at once). Can
> you help us?

OK,

a certain number of points :

1). use a cached version of the engine

the cached version of the engine maintains a cache of the expressions
composing the process instances (the latest used are kept in memory)
so less reads are needed, it also optimizes disk writes, ie access is
buffered and a minimal amount of writes is performed.

There is a cached version of the file persisted engine and also one
for the db persisted engine.

2). use the set-field expression

http://openwferu.rubyforge.org/expressions.html#exp_restore

it can set a certain number of fields in 1 sweep.

3). set the initial fields of the engine via the launch item

a certain number of ways to launch a process :

engine.launch('http://data.example.com/procdefs/definition1.xml')

li = OpenWFE::LaunchItem.new
li.wfdurl = 'http://data.example.com/procdefs/definition1.xml')
li.fields = { 'x' => 0, 'y' => 1 }
engine.launch(li)

In the second example, the fields are passed in 1 sweep, at launch time.

4). if you're already using ruote 0.9.19, you can pass variables at
launch time :

engine.launch('http://data.example.com/procdefs/definition1.xml',
:variables => { 'var0' => 'val0' })

li = OpenWFE::LaunchItem.new
li.wfdurl = 'http://data.example.com/procdefs/definition1.xml')
li.fields = { 'x' => 0, 'y' => 1 }
engine.launch(li, :variables => { 'var0' => 'val0' })

5). you can wrap the field setting in a participant :

engine.register_participant 'initializator' do |workitem|
workitem.fields['f0'] = 'val0'
# ...
end

(you can do a similar thing for the variables, but settings variables
is less performant).

6). live in a web world

pass the references (URIs) to the information, not the information
itself. In an active_record world, it's OK as well to pass the db id
of records.

As always if there are points in the documentation that are difficult
to understand or missing, pointing at them or providing better
formulations would be appreciated. Questions are welcome.

OK, still waiting for your version information.

[emil...@gmail.com]

Hi John,

just to clarify, we are using:

-ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
-OpenWFEru 0.9.18
-We are using openferu engine with the ruote-web configuration. Some things
are in db (e.g. workitem fields) and other things are in file system
(work_development directory).
-error_journal? I suppose we are using error_journal included with
ruote-web.

Thanks again,

Emilio

-----Mensaje original-----
De: openwfe...@googlegroups.com
[mailto:openwfe...@googlegroups.com] En nombre de John Mettraux
Enviado el: martes, 20 de enero de 2009 12:28
Para: openwfe...@googlegroups.com
Asunto: [openwferu-users] Re: Workflow start-up time

[emil...@gmail.com]

Hello John,

set-fields seems like a good idea, it seems appropriate for our situation,
but we have the following problem:

-In Ruby:

set_fields :value => { "name" => "", "addr" => ""}

-In JSON:

['set_fields',{'value':{'name': '', 'addr': ''}}]

But... how would you write it in XML notation?

<set-fields value="..." />

Thanks for your help,

Kind regards,

Emilio and Diego

-----Mensaje original-----
De: openwfe...@googlegroups.com
[mailto:openwfe...@googlegroups.com] En nombre de John Mettraux

Enviado el: martes, 20 de enero de 2009 13:17

Para: openwfe...@googlegroups.com
Asunto: [openwferu-users] Re: Workflow start-up time

[John Mettraux]

On Tue, Jan 20, 2009 at 10:28 PM, <emil...@gmail.com> wrote:
>
> just to clarify, we are using:
>
> -ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
> -OpenWFEru 0.9.18
> -We are using openferu engine with the ruote-web configuration. Some things
> are in db (e.g. workitem fields) and other things are in file system
> (work_development directory).
> -error_journal? I suppose we are using error_journal included with
> ruote-web.

Hi,

thanks for sharing this information.

> work_development directory

Have you tried with Rails in 'production' mode ?

I did my homework and adapted a bench to fit your 40 fields/variables test :

http://gist.github.com/49480

my system (a 2008 macbook) :

ruby : ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
memory : 4GB
processor : 2.4 GHz Intel Core 2 Duo
os : mac OS X 10.5.6

the results (ruote 0.9.20) :

http://gist.github.com/49485

I'm setting 40 fields or 40 variables, with debug on or off. One thing
I'm not testing is participant insertion. Having 40 fields mean 41
inserts for active_record, but you indicated that variables yielded
worse performance.

In the worst case, the engine takes 4.6 seconds to set 40 variables.

Since you are using ruote 0.9.18, here are the results for that version :

http://gist.github.com/49488

(not using the db persisted expression system, you guys aren't using it either).

I was expecting worse performance since I have improved Ruote a lot
lately, but nothing really visible.

Is your process definition only setting fields before reaching the
first participant ?
Maybe my benchmark is too kind on the engine (launching only one
process) or my machine too powerful...

I hope you'll share your observations once this is solved, it would be fair.

[John Mettraux]

On Tue, Jan 20, 2009 at 11:06 PM, <emil...@gmail.com> wrote:
>
> set-fields seems like a good idea, it seems appropriate for our situation,
>

> But... how would you write it in XML notation?
>
> <set-fields value="..." />

Ah yes, this is not documented, but you can do :

<set-fields>
<hash>
<entry>
<string>field0</string><string>value0</string>
<string>field1</string><string>value1</string>
</entry>
</hash>
</set-fields>

(I will document that
http://rubyforge.org/tracker/index.php?func=detail&aid=23627&group_id=2609&atid=10195)

Thanks, don't forget to look at the other improvement points and to
share your discoveries.

[emil...@gmail.com]

Hi John,

we have followed your tip, but using:

<set-fields>
<hash>
<entry>
<string>p_01</string><string>value0</string>
<string>p_02</string><string>value1</string>
</entry>
</hash>
</set-fields>

we get this error:

class: RuntimeError
message: apply
stacktrace:
exception : unknown expression 'hash'
.../vendor/openwfe/expressions/raw.rb:96:in `instantiate_real_expression'
.../vendor/openwfe/expressions/raw.rb:134:in `apply'
.../vendor/openwfe/expool/expressionpool.rb:885:in `send'
.../vendor/openwfe/expool/expressionpool.rb:885:in `do_apply_reply'
.../vendor/openwfe/util/workqueue.rb:73:in `send'
.../vendor/openwfe/util/workqueue.rb:73:in `service_init'
.../vendor/openwfe/util/workqueue.rb:65:in `loop'

Any hint?

Thanks, and regards,

Emilio

-----Mensaje original-----
De: openwfe...@googlegroups.com
[mailto:openwfe...@googlegroups.com] En nombre de John Mettraux

Enviado el: martes, 20 de enero de 2009 15:19

Para: openwfe...@googlegroups.com
Asunto: [openwferu-users] Re: Workflow start-up time

[John Mettraux]

On Tue, Jan 20, 2009 at 1:28 AM, Diego Moreno <dmo...@dit.upm.es> wrote:
>
> What do you think about our approach to provide the end user/designer with
> all the means to define everything in XML inside the workflow definition,
> for the sake of easiness?

Hi,

you're building a product and I am merely driving a project, you're
walking the last mile, if you think that it's the best way to do it
for you and your customer, then it's perfect.

Best regards,

[John Mettraux]

On Tue, Jan 20, 2009 at 10:28 PM, <emil...@gmail.com> wrote:
>
> just to clarify, we are using:
>
> -ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
> -OpenWFEru 0.9.18
> -We are using openferu engine with the ruote-web configuration. Some things
> are in db (e.g. workitem fields) and other things are in file system
> (work_development directory).
> -error_journal? I suppose we are using error_journal included with
> ruote-web.

Hello Emilio,

a few more benchmarks. I have added a upm.xml process to the demo of
ruote-web at http://difference.openwfe.org:3000/
You can test it by yourself as well, just get there and launch that
upm.xml flow.

When running in dev mode, it needs around 2 seconds to launch that
"set 40 fields" process, it drops to around 1 second when running in
production mode. It's a shared virtual server somewhere in Texas, not
much memory. Anyway, nothing like the up to 9 seconds that you are
experiencing. The version of ruote[-web] there is 0.9.19.

I was wondering, you guys seem to have implemented a custom XML
include technique (see
http://groups.google.com/group/openwferu-users/msg/c495096d56402099)
what is the performance cost of that ?

(maybe more classical techniques
http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=xml+include
are better handled by REXML...)

OK, I did my homework.

[John Mettraux]

On Wed, Jan 21, 2009 at 1:49 AM, <emil...@gmail.com> wrote:
>
> we have followed your tip, but using:
>
> <set-fields>
> <hash>
> <entry>
> <string>p_01</string><string>value0</string>
> <string>p_02</string><string>value1</string>
> </entry>
> </hash>
> </set-fields>
>
> we get this error:
>
> class: RuntimeError
> message: apply
> stacktrace:
> exception : unknown expression 'hash'
> .../vendor/openwfe/expressions/raw.rb:96:in `instantiate_real_expression'

Sorry, my bad :

---8<---
<set-fields>
<a>
<hash>
<entry>
<string>customer_name</string><string>Zigue</string>
</entry>
<entry>
<string>customer_age</string><string>34</string>
</entry>
<entry>
<string>approved</string><false />
</entry>
</hash>
</a>
</set-fields>
--->8---

this <a> thing also understands YAML and JSON :

---8<---
<set-fields>
<a>{"customer":{"name":"Zigue","age":34},"approved":false}</a>
</set-fields>
--->8---

Best regards,

[emil...@gmail.com]

Hi John,

We are using your benchmarking program to do some tests. Thanks for the
code! Our results seem interesting.

ruby : ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
memory : 2GB

processor : 2.4 GHz Intel Core 2 Duo

os : Ubuntu GNU/Linux 8.10 Intrepid Ibex

With ruote-0.9.20: http://gist.github.com/49614

There are two different scenarios:
- Workflow with 40 fields/variables loaded from XML
- Workflow with 40 fields/variables loaded from "big XML", which is
something like this:

<process-definition revision="0.1" name="flow">
<description> flow </description>
<sequence>

<set field="f_01" value="var01"/>
<set field="f_02" value="var02"/>
...
<set field="f_40" value="var40"/>

<if>
<equals field-value="one" value="two" />
<sequence>
<!-- 300 lines of flow that are not run, because the condition is
not met -->
</sequence>
</if>

</sequence>
</process-definition>

The two values that are significant are those for big
XML/fields/FilePersistedEngine/(debug off), with a run time of 13.283186s,
and big XML/variables/FilePersistedEngine/(debug off), with a run time of
24.425080s.

This is representative of our problem. Our set-fields take a long time
(10-15 seconds) to initialize when they are into a big XML. John, in your
opinion, what is the best solution for this problem?

Best regards,

Emilio

-----Mensaje original-----
De: openwfe...@googlegroups.com
[mailto:openwfe...@googlegroups.com] En nombre de John Mettraux

Enviado el: miércoles, 21 de enero de 2009 4:36

Para: openwfe...@googlegroups.com
Asunto: [openwferu-users] Re: Workflow start-up time

[John Mettraux]

Hi,

you seem to be using a default ruote-web (densha) 0.9.18, so the
engine is the "CachedFilePersistedEngine" one.

> This is representative of our problem. Our set-fields take a long time
> (10-15 seconds) to initialize when they are into a big XML. John, in your
> opinion, what is the best solution for this problem?

Have you measured this to be the "set-fields" time ?

I'd suggest adding this at the end of your config/environment.rb to
get timestamps in your logs :

---8<---
class Logger
def format_message(severity, timestamp, progname, msg)
#"#{timestamp} (#{$$}) #{msg}\n"
"#{timestamp.strftime('%Y/%m/%d
%H:%M:%S')}.#{timestamp.usec.to_s[0, 3]} (#{$$}) #{msg}\n"
end
end
--->8---

Then you can precisely answer the question : is set-fields taking such
a long time ?

It would be interesting to time each operation, especially your custom
include technique. How long does it take for your system to read the
whole XML and then to turn it into its syntax tree. Then, how long
does the <set-fields> itself takes ?

When running in development mode, the engine is very verbose. With the
modification to Logger described a few paragraphs back, you can get a
detailed picture of what is happening (log/openwferu_development.log).

Maybe the best solution, is to design your processes not to
"transport" too much raw data, but only what is relevant for process
routing (<if> and such), data for forms can always be retrieved at
form rendering time. Passing reference to data instead of data itself
is not a sin.

[emil...@gmail.com]

Hello John,

Thank you very much for your tip about using a more standard way to include
XML files. I understand that using:

<!DOCTYPE process-definition [
<!ENTITY view1 SYSTEM "_view_file.xml">
]>

before the "process-definition", and then including "&view1;" in the
workflow should work with WEXML. However, I have tried it and have not
managed to get it working, so my question is: should it be working (and thus
it is my fault)? or is it something that must be implemented (and thus we
could volunteer to add)? and is there a DTD to be used/validated for
OpenWFEru language?

Thanks, and kind regards,

Emilio

-----Mensaje original-----
De: openwfe...@googlegroups.com
[mailto:openwfe...@googlegroups.com] En nombre de John Mettraux

Enviado el: miércoles, 21 de enero de 2009 4:36

Para: openwfe...@googlegroups.com
Asunto: [openwferu-users] Re: Workflow start-up time

[Diego Moreno]

Hi John,

At first, thanks for the logger code. It has been very useful.

For next tests I have used a basic ruote-web installation (downloaded today). Tests involve a execution of a workflow loaded from big XML, which is something like this:

<process-definition revision="0.1" name="flow">
 <description> flow </description>
 <sequence>

   <set field="f_01" value="var01"/>
   <set field="f_02" value="var02"/>
   ...
   <set field="f_40" value="var40"/>

   <if>
     <equals field-value="one" value="two" />
     <sequence>

       <!-- XXX lines of flow that are not run, because the condition is not met -->
     </sequence>
   </if>
 </sequence>
</process-definition>

I have made four tests. Time was measured from the first set-field operation timestamp (seen with your logger code) to the last set-field timestamp.

1) 40 set-fields from a XML workflow definition with 340 lines. 5 seconds.

2) 40 set-fields from a XML workflow definition with 1170 lines. 11 seconds.

3) 40 set-fields, in one sweep using a hash, from a XML workflow definition with 1170 lines. 2 seconds.

4) 40 set-variables from a XML workflow definition with 1170 lines. 14 seconds.

A hash of set-fields seems to be the solution. But it is strange that the size of the workflow part that is not executed influences in the final runtime.

By the way, related to ruote-web installation I have found two problems:

- I had to disable the rufus-eval git repository from lib/task/ruote.rake because it did not exist.
- I had to comment line 454 from vendor/openwfe/extras/participants/active_participants.rb
#  ActiveRecord::Base.connection_pool.release_connection
because connection_pool method is missing for ActiveRecord-2.0.2 (recommended version for run ruote-web)

Kind regards.
Diego.

2009/1/21 John Mettraux <jmet...@openwfe.org>

[John Mettraux]

On Thu, Jan 22, 2009 at 12:20 AM, <emil...@gmail.com> wrote:
>
> Hello John,

Hello Emilio,

next time, could you please not "recycle" a thread :

http://groups.google.com/group/openwferu-users/browse_frm/thread/f7d3c2781782e002

The initial subject was "getting started with ruote-web2"

If you want to start a discussion, post a new message, don't fetch an
old one in your Outlook and hit "reply".

Thanks in advance, my best regards,