Inputs scoping and validation

[Raphael Simon]

Hello all, please bare with me if that topic has already been covered, a quick search did not yield anything useful. I'm looking at putting together processes using potentially many sub-processes. In our use case it's possible that a sub-process  be authored by a 'user' contributing back to the overall process (for example to allow users to specify custom steps that need to be taken when certain events occur). One pain point we are hitting is how to clearly specify what the "inputs" and "outputs" of such sub-processes should be.

Ideally we would want to be able to specify which workitem fields should be set and how they should be set prior to invoking a subprocess. It should also be able to then validate the workitem after the sub-process finishes for all the specified outputs. A related issue is that it would seem 'safer' if the subprocess did not have access to the overall workitem but only to the inputs it has specified. Potentially it would be nice to be able to map the parent process workitem fields to the subprocess workitem fields since the field names may defer and map back on the way out.

I understand it's possible to specify variables for a subprocess by using the expression attributes however in this case I'm really interested in the workitem. I see that FlowExpression#launch_sub can take a workitem as option but that doesn't seem to be surfaced to the subprocess expression. If it was we could build the mapping mechanism described above.

In terms of validation of fields I also could not find anything readily available. I put a FieldsValidationParticipant together, wonder if that's going in the right direction? so to summarize:

1. How does one validate workitem fields in ruote (e.g. the initial workitem fields)?

2. How would a sub-process define the fields it expects and how would the parent process do the mapping?

I could be that I'm completely missing the 'philosophy' being ruote, please let me know if that's the case...

Thank you!

--

Raphael.

[John Mettraux]

On Thu, Jan 27, 2011 at 03:35:34PM -0800, Raphael Simon wrote:
>
> In terms of validation of fields I also could not find anything readily
> available. I put a FieldsValidationParticipant

> <https://gist.github.com/799298>together, wonder if that's going in the
> right direction?

Hello Raphael, welcome back.

Back a million year ago, when ruote was OpenWFE, a Java workflow engine, participants could be tied to filters. Those filters controlled which fields went to the real participant and then handled the merge when the workitem came back.

When I moved to Ruby, I wanted (maybe still want (https://github.com/jmettraux/ruote/blob/7675a47da31485fffd6d160ab2f5592a08a0affe/TODO.txt#L290-291)) to have this feature in ruote, but I realized people built very smart participant implementations, very quickly (it's Ruby), and they all had slightly different needs.

Now I realize you want something a bit different, in fact you want this filtering but you also want the "subprocess signature enforcement", am I right ?

> so to summarize:
>
> 1. How does one validate workitem fields in ruote (e.g. the initial workitem
> fields)?

Ah validation at launch time... I've been resisting this feature for a while :

http://groups.google.com/group/openwferu-users/browse_thread/thread/4ea83ccfbc1caa87
http://groups.google.com/group/openwferu-users/msg/8504cf9e065a4607

I guess the launch call would immediately fail. Patrice, in that thread, really wants the process to fail immediately (not asynchronously).

What is your take ?

> 2. How would a sub-process define the fields it expects and how would the
> parent process do the mapping?

Via a participant is right.

What happens when the expected fields are not here ? Error ?

> I could be that I'm completely missing the 'philosophy' being ruote, please
> let me know if that's the case...

The philosphy on that, is more like trying to delay the implementation decision until it's really needed.

So let's look at the current 'way' :

---8<---
Ruote.define do
define 'y' do
filterTwoIn
participany 'z'
filterTwoOut
end
sequence do
filterOneIn
participant 'x'
subprocess 'y'
filterOneOut
end
end
--->8---

Quite verbose.

One thing we could do is introduce a :filter attribute (on any expression), that would trigger a participant upon applying an yet another upon leaving (replying to the engine).

---8<---
sequence :filter => 'one' do
# ...
end

class LevelOneFilter
def consume(workitem)
# filter (and keep original)
end
def on_reply(workitem)
# unfilter (merge incoming and original)
end
end

engine.register_participant 'one', LevelOneFilter
--->8---

Maybe it's "perverting" participants a bit too much, but it could shorten our process definitions :

---8<---
Ruote.define do
define 'y' do
participany 'z', :filter => 'two'
end
sequence :filter => 'one' do
participant 'x'
subprocess 'y'
end
end
--->8---

What do you think ?

Best regards,

--
John Mettraux - http://jmettraux.wordpress.com

[Raphael Simon]

Hello John, thanks for the quick reply! Hopefully it won't be another million years before Ruote can support some of this ;)

>Now I realize you want something a bit different, in fact you want this filtering but you also >want the "subprocess signature enforcement", am I right ?

Yes what I'm thinking is a mechanism by which a sub-process could specify what its 'inputs'/'output' are and a process could then map the current workitem fields to these inputs and map back the 'outputs' to the current workitem field.

To take a concrete example: imagine a process used to launch servers in the cloud. That process requires a list of server ids 'server_ids' and a cloud id 'cloud_id'. This process would be used in many other processes - anytime servers need to be launched. However how that list of server ids could be retrieved in many different ways. In one process it may be an initial field but in another it may be the result of looking up servers using a 'lookup' process which would set that list in e.g. the __result__ field.

We would want that 'launch_servers' process to clearly state what the required initial fields are and enforce that the fields are indeed there before handing it off to it (or have a participant check it right at the beginning). I think a difference with the use case you linked to is that we would like this definition to be explicit in the workflow, a sort of 'contract' that is defined by the sub-process. The FieldsDefinitionParticipant was an attempt at defining such a contract.

We can implement this validation today but the one piece we would be missing is the scoping, that is making sure the sub-process has its own workitem.

>I guess the launch call would immediately fail. Patrice, in that thread, really wants the process >to fail immediately (not asynchronously).

>What is your take ?

That makes sense to me, at some level this would be an error in the workflow definition itself rather than an error in the execution so the earlier the failure the better.

>Via a participant is right.
>What happens when the expected fields are not here ? Error ?

Yes, it's a contract violation.

The filtering mechanism you describe could help 'reuse' the contract definitions for multiple sub-processes but we would want the definition to be explicit in the workflow rather than embedded in a participant. The filtering would give some of the scoping too. I'm wondering: where would the fields "filtered out" be stored so that they can be merged back after the participant replies?

--
Raphael.

--
you received this message because you are subscribed to the "ruote users" group.
to post : send email to openwfe...@googlegroups.com
to unsubscribe : send email to openwferu-use...@googlegroups.com
more options : http://groups.google.com/group/openwferu-users?hl=en

[John Mettraux]

On Thu, Jan 27, 2011 at 06:02:02PM -0800, Raphael Simon wrote:
>
> > I guess the launch call would immediately fail. Patrice, in that thread,
> > really wants the process >to fail immediately (not asynchronously).
> >
> > What is your take ?
>
> That makes sense to me, at some level this would be an error in the workflow
> definition itself rather than an error in the execution so the earlier the
> failure the better.

Hello Raphael,

In that case, the check_and_launch approach I describe is handy, but it's not applicable to subprocesses.

> > Via a participant is right.
> > What happens when the expected fields are not here ? Error ?
>
> Yes, it's a contract violation.

OK.

> The filtering mechanism you describe could help 'reuse' the contract
> definitions for multiple sub-processes but we would want the definition to
> be explicit in the workflow rather than embedded in a participant.

what should it look like ?

---8<---
define 'x' do
signature do
server_id :required => true, :type => String
#...
end
# body...
end
--->8---

?

> The filtering would give some of the scoping too. I'm wondering: where would
> the fields "filtered out" be stored so that they can be merged back after the
> participant replies?

Each expression stores the workitem ('applied_workitem' field) upon getting applied, so the storage already occurs.

So we have filters and signatures... Filters have a scope, signatures behave more like checkpoints.

I think I'd like to implement the filter feature. The signature one needs refining.

Cheers,

[Raphael Simon]

Hello John,

>what should it look like ?
>
>---8<---
> define 'x' do
>   signature do
>     server_id :required => true, :type => String
>     #...
>   end
>   # body...
> end
>--->8---
>
>?

Exactly, although I'm hoping it's possible to define a more compact syntax (this is how my first iteration at FieldsValidationParticipant looked like and I found it a bit 'heavy' to use).

>Each expression stores the workitem ('applied_workitem' field) upon getting applied, so the >storage already occurs.

Ah yes that makes sense, then it fits the use case perfectly.

>So we have filters and signatures... Filters have a scope, signatures behave more like >checkpoints.
>
>I think I'd like to implement the filter feature. The signature one needs refining.

And the signature can be fully implemented as a participant but the filter cannot so this makes sense. Thank you for the quick feedback loop, really appreciate it.

Regarding the filters, would it make sense to also allow defining them inline, something like:

---8<---
Ruote.process_definition :name => 'my_uber_process' do
  sequence do
    do_something
    a_subprocess :in => [ 'a_field', 'some_other_field', 'orignal_name' => 'new_name' ],
                           :out => [ '__result__' => 'new_field' ]
    do_something_else
  end
end
--->8---

where anything not explicitly listed in the filters list is not given to the subprocess (in) or merged back into the workitem (out).

I guess if you provide the ability to filter using a participant this could then be implemented doing something like:

---8<---
mappings :input => ...., :output => ...
a_subprocess :filter => :mappings
--->8---

with the right implementation for the mappings participant. Maybe the former syntax then just becomes a shortcut for the later.

Thanks again,

--
Raphael.

[John Mettraux]

On Thu, Jan 27, 2011 at 08:48:58PM -0800, Raphael Simon wrote:
>
> > So we have filters and signatures... Filters have a scope, signatures
> > behave more like checkpoints.
> >
> > I think I'd like to implement the filter feature. The signature one needs
> > refining.
>
> And the signature can be fully implemented as a participant but the filter
> cannot so this makes sense. Thank you for the quick feedback loop, really
> appreciate it.
>
> Regarding the filters, would it make sense to also allow defining them
> inline, something like:
>
> ---8<---
> Ruote.process_definition :name => 'my_uber_process' do
> sequence do
> do_something
> a_subprocess :in => [ 'a_field', 'some_other_field', 'orignal_name' =>
> 'new_name' ],
> :out => [ '__result__' => 'new_field' ]
> do_something_else
> end
> end
> --->8---

Hello Raphael,

in this example, shouldn't the filter be replaced with a signature ? (ie defined in the subprocess ?)

> where anything not explicitly listed in the filters list is not given to the
> subprocess (in) or merged back into the workitem (out).
>
> I guess if you provide the ability to filter using a participant this could
> then be implemented doing something like:
>
> ---8<---
> mappings :input => ...., :output => ...
> a_subprocess :filter => :mappings
> --->8---
>
> with the right implementation for the mappings participant. Maybe the former
> syntax then just becomes a shortcut for the later.

I like the inlining idea.

I'd like to use :filter (and :signature maybe) both with participants and { :in => [], :out => [] } one-liners.

Ruote will lookup the name given to it, it could map to a participant, or a process variable or an engine variable (for filters/signatures shared by the whole engine).

Participants are powerful because you have the whole Ruby power at your disposal for filtering/merging/enforcing.

Inlines are quick are limited to a process instance (except when they're bound in engine variables).

One thing I don't like about the DSL behind the inlines and theFieldsValidationParticipant is that it's Ruby-bound. I'd prefer to limit ourselves to the JSON schema (sorry no DateTime or Hash<Hash<String, String>, Fixnum> recursive fun). I don't want to tie the process definitions to Ruby.

I'd like to avoid implementing (and/or maintaining) a complete type system in ruote.

I will look at the expanded version of the "inlines".

Give me a bit of time to come up with something we can refine further.

We also have to be clear and differentiate transformations from validations.

:type => 'String', :default => 'run'

is a transformation, while

:type => 'Array<String>', :required => true

is a validation.

Sorry if I drag you in too general a discussion, but I'd like to discuss that with sufficient altitude in the point of view.

[Raphael Simon]

>in this example, shouldn't the filter be replaced with a signature ? (ie defined in the subprocess ?)

I don't think so: the subprocess defines its contract/signature independently of what the workitem may be at the point it gets called. The purpose of the filter/mapping here is to define how to match that contract. It has to be done in the parent process. The signature belongs to a given sub-process while a parent process has to define the mapping matching the current workitem when it calls it.

>I like the inlining idea.

Great!

>I'd like to use :filter (and :signature maybe) both with participants and { :in => [], :out => [] } one-liners.

I'm missing something: how would you use :filter with :in, :out ?

>Ruote will lookup the name given to it, it could map to a participant, or a process variable or an engine variable (for filters/signatures shared by the whole engine).

Makes sense.

>One thing I don't like about the DSL behind the inlines and theFieldsValidationParticipant is >that it's Ruby-bound. I'd prefer to limit ourselves to the JSON schema (sorry no DateTime or >Hash<Hash<String, String>, Fixnum> recursive fun). I don't want to tie the process definitions >to Ruby.
>
>I'd like to avoid implementing (and/or maintaining) a complete type system in ruote.

That makes a lot of sense, I wasn't sure where to set the limit but thinking in terms of JSON helps a lot.

>We also have to be clear and differentiate transformations from validations.

>  :type => 'String', :default => 'run'

>is a transformation, while

> :type => 'Array<String>', :required => true

>is a validation.

I am not sure it's that clear, the point being that if a field has a default value then it doesn't matter that it is required. So a field is either required (and it's an error if the value is missing) or has a default value (used if the value is missing). But you're right this wasn't meant to be pure validation (so the name isn't great), this was more meant to describe what the interface of the sub-process is.

>Sorry if I drag you in too general a discussion, but I'd like to discuss that with sufficient altitude in the point of view.

Please, by all means :) If anything, it's me dragging you...

--
Raphael.

[John Mettraux]

On Thu, Jan 27, 2011 at 10:10:44PM -0800, Raphael Simon wrote:
>
> > in this example, shouldn't the filter be replaced with a signature ? (ie
> > defined in the subprocess ?)
>
> I don't think so: the subprocess defines its contract/signature
> independently of what the workitem may be at the point it gets called. The
> purpose of the filter/mapping here is to define how to match that contract.
> It has to be done in the parent process. The signature belongs to a given
> sub-process while a parent process has to define the mapping matching the
> current workitem when it calls it.

Understood. Transformation it seems.

> > I'd like to use :filter (and :signature maybe) both with participants and {
> > :in => [], :out => [] } one-liners.
>
> I'm missing something: how would you use :filter with :in, :out ?

For now, I'm thinking something like :

---8<---
engine.variables['filter1'] = { :in => [], :out => [] }

# ...

Ruote.define do
x :filter => { :in => [], :out => [] }
set 'v:filter0' => { :in => [], :out => [] }
x :filter => 'filter0'
x :filter => 'filter1'
x :filter => 'participant_name'
end
--->8---

> > One thing I don't like about the DSL behind the inlines and
> > theFieldsValidationParticipant is >that it's Ruby-bound. I'd prefer to limit
> > ourselves to the JSON schema (sorry no DateTime or >Hash<Hash<String,
> > String>, Fixnum> recursive fun). I don't want to tie the process definitions
> > to Ruby.
> >
> > I'd like to avoid implementing (and/or maintaining) a complete type system
> > in ruote.
>
> That makes a lot of sense, I wasn't sure where to set the limit but thinking
> in terms of JSON helps a lot.

Maybe we could have a look at things like JSONpath or equivalents.

> > We also have to be clear and differentiate transformations from
> > validations.
> >
> > :type => 'String', :default => 'run'
> >
> > is a transformation, while
> >
> > :type => 'Array<String>', :required => true
> >
> > is a validation.
>
> I am not sure it's that clear, the point being that if a field has a default
> value then it doesn't matter that it is required. So a field is either
> required (and it's an error if the value is missing) or has a default value
> (used if the value is missing). But you're right this wasn't meant to be
> pure validation (so the name isn't great), this was more meant to describe
> what the interface of the sub-process is.

Understood.

Best regards,

[Raphael Simon]

Bonjour John,

>Understood. Transformation it seems.

Yes at the parent process level it's mapping the workitem to what the sub-process/participant expects. In the sub-process/participant is where the signature/contract lives which does the validation.

>For now, I'm thinking something like :
>
>---8<---
>engine.variables['filter1'] = { :in => [], :out => [] }
>
># ...
>
>Ruote.define do
> x :filter => { :in => [], :out => [] }
> set 'v:filter0' => { :in => [], :out => [] }
> x :filter => 'filter0'
> x :filter => 'filter1'
> x :filter => 'participant_name'
>end
>--->8---

This looks great.

>Maybe we could have a look at things like JSONpath or equivalents.

Makes sense, so you could do something like:

x :filter => { :in => [ 'field.sub_field' => 'sub_process_field', 'array_field[2]' => 'another_sub_process_field' ] }

Ideally the syntax should be consistent with the 'WorkItem#lookup' syntax (so you can document it / implement it as doing a standard lookup in the workitem and assigning the result to the workitem being passed to the participant).

Please let me know if I can help with code reviews or testing as you make progress.

Merci!

--
Raphael.

[John Mettraux]

On Fri, Jan 28, 2011 at 10:46:05AM -0800, Raphael Simon wrote:
>
> x :filter => { :in => [ 'field.sub_field' => 'sub_process_field',
> 'array_field[2]' => 'another_sub_process_field' ] }
>
> Ideally the syntax should be consistent with the 'WorkItem#lookup' syntax
> (so you can document it / implement it as doing a standard lookup in the
> workitem and assigning the result to the workitem being passed to the
> participant).
>
> Please let me know if I can help with code reviews or testing as you make
> progress.

Hello Raphael,

I've started work on the 'filter' :

https://github.com/jmettraux/ruote/commit/afc772cbe022a617a7846615aeddab1af016d847

For now, a filter looks like :

[
{ 'field' => 'x', 'default' => 1 },
{ 'field' => 'y', 'remove' => true }
]

I was wondering about an alternate (transposed) variant for the filters :

{
'require' => [ 'x', 'y', 'z' ],
'remove' => [ 'a', 'b' ],
'copy' => { 'c' => 'd', 'e' => 'f' },
'default' => { 'g' => {}, 'h' => "twelve" }
}

The two formats could coexist (though since order matters, we couldn't have strict equality...). Well, for now I'm going with the first version (list of rules).

I think I'll go on with mixing validation and transformation. It's OK. When I look at your filter at

https://gist.github.com/799298

I wonder if I should use a field_error like you do and place the process instance in error or simply raise an error on the first error encountered when filtering.

I also found

http://json-schema.org/
http://tools.ietf.org/html/draft-zyp-json-schema-03

and two ruby implementations

https://github.com/Constellation/ruby-jsonchema
https://github.com/hoxworth/json-schema (active)

Looking for it, at least as an inspiration, for the validation aspect of filters. (of course, your gist is in as well).

Cheers,

[John Mettraux]

Hello,

it's progressing well.

https://github.com/jmettraux/ruote-kit/commit/93ec14d10f8e9730a53cde28b79af33291faf1f://github.com/jmettraux/ruote/commits/ruote2.1/

To get an overview of the filtering (validations + transformations) capabilities :

https://github.com/jmettraux/ruote/blob/ec683dce3cd9577835df0672f90a5536d44bed19/test/unit/ut_22_filter.rb

For now, the filter are only "one-way", via the 'filter' expression :

https://github.com/jmettraux/ruote/blob/ec683dce3cd9577835df0672f90a5536d44bed19/test/functional/eft_35_filter.rb

I will start working on the :filter attribute discussed in this thread soon.

Comments are welcome, especially about validation errors stacking and so.

[Raphael Simon]

Hello John,

This looks great and seems to provide a lot of flexibility! I see that you did end-up supporting arrays and hashes :)

I will give this a whirl and adapt our workflows to use it.

--
Raphael.

[John Mettraux]

On Fri, Feb 04, 2011 at 05:51:09PM -0800, Raphael Simon wrote:
>
> This looks great and seems to provide a lot of flexibility! I see that you
> did end-up supporting arrays and hashes :)
>
> I will give this a whirl and adapt our workflows to use it.

Hello,

just pushed a change that lets you do things like

filter :fields => "a, b, c", :type => 'string'
filter :fields => %w[ a b c ], :type => 'string'
filter :fields => '/^user_/', :type => 'string'

filter :fields => '/^x_(.+)$/', :copy_to => 'y_\1'

...

https://github.com/jmettraux/ruote/commit/f496d53ba0962c3f3048b09b5015e19fe07cc928

Best regards,

[Raphael Simon]

I was just about to require something like that (we have some dynamically named fields). Thank you!

--
Raphael.

[Raphael Simon]

Hello John et. all,

So I have moved our workflows to use the built-in filters on 'edge' ruote and it works great. 

Stepping back a little though there is a little bit of confusion in my mind about what this does in relation to what we had discussed. Specifically what I had understood what that you were going to work on a way to filter and/or transform the workitem fields being "given to" and/or "retrieved from" a participant. That is a participant would have an implicit contract/interface and the filters could be used to "force" the workitem that it receives to abide that contract. So you could have a filter that does:

some_participant :filter => { :in => { 'global_field' => 'a_field' }, :out => { 'a_field' => 'a_result' } }

This would have filtered out all workitem fields other than 'global_field' from the workitem that 'some_participant' consumes and would have renamed the 'global_field' workitem field to 'a_field' as far as 'some_participant' is concerned. Once 'some_participant' replies the initial workitem fields would be "restored" and the workitem field 'a_field' would now be named 'a_result' for the next expression to pick up.

Validation would have been nice to be built in but could have been done through participants (like my weak attempt with the FieldDefinitionsParticipant). However it seems that you went the other route though were filters really only validate but don't filter nor transform at the moment. While it's great to have validation built-in into Ruote I can't help but think that this should not be the job of filters, maybe a validate expression would be more appropriate.

Obviously I could be completely off-base and miss the grand vision behind the current design :) Anyway I would appreciate it if you could share some of the rationale behind the current choices you made and where actual filtering and transforming fit.

Thank you !

--

Raphael.

[John Mettraux]

On Mon, Feb 07, 2011 at 04:02:15PM -0800, Raphael Simon wrote:
>
> So I have moved our workflows to use the built-in filters on 'edge' ruote
> and it works great.

Hello Raphael,

> Stepping back a little though there is a little bit of confusion in my mind
> about what this does in relation to what we had discussed. Specifically what
> I had understood what that you were going to work on a way to filter and/or
> transform the workitem fields being "given to" and/or "retrieved from" a
> participant. That is a participant would have an implicit contract/interface
> and the filters could be used to "force" the workitem that it receives to
> abide that contract. So you could have a filter that does:
>
> some_participant :filter => { :in => { 'global_field' => 'a_field' }, :out
> => { 'a_field' => 'a_result' } }
>
> This would have filtered out all workitem fields other than 'global_field'
> from the workitem that 'some_participant' consumes and would have renamed
> the 'global_field' workitem field to 'a_field' as far as 'some_participant'
> is concerned. Once 'some_participant' replies the initial workitem fields
> would be "restored" and the workitem field 'a_field' would now be named
> 'a_result' for the next expression to pick up.

For now, it's limited to

---8<---
sequence do
filter :field => 'global_field', :move_to => 'a_field'
some_participant
filter :field => 'a_field', :move_to => 'a_result'
end
--->8---

I haven't had the time to implement the :filter as an attribute now, I only did the 'filter' as an expression.

http://groups.google.com/group/openwferu-users/msg/de967598edea5b5d

---8<---
some_participant
:in => { :field => 'global_field', :move_to => 'a_field' },
:out => { :field => 'a_field', :move_to => 'a_result' }

# or

some_participant
:in => { :f => 'global_field', :mv_to => 'a_field' },
:out => { :f => 'a_field', :mv_to => 'a_result' }
--->8---

> Validation would have been nice to be built in but could have been done
> through participants (like my weak attempt with the
> FieldDefinitionsParticipant). However it seems that you went the other route
> though were filters really only validate but don't filter nor transform at
> the moment.

filtering :

https://github.com/jmettraux/ruote/blob/97e83972e1419c4114e81f619c696a65dcf7a0d5/test/unit/ut_22_filter.rb#L31-46

transform :

https://github.com/jmettraux/ruote/blob/97e83972e1419c4114e81f619c696a65dcf7a0d5/test/unit/ut_22_filter.rb#L178-299

I also not had the time to document transformations via the 'filter' expression yet :

https://github.com/jmettraux/ruote/blob/97e83972e1419c4114e81f619c696a65dcf7a0d5/lib/ruote/exp/fe_filter.rb#L241-247

> While it's great to have validation built-in into Ruote I can't
> help but think that this should not be the job of filters, maybe a validate
> expression would be more appropriate.

'validate' vs 'filter', makes sense. I mixed both to be able to do things like :

https://github.com/jmettraux/ruote/blob/97e83972e1419c4114e81f619c696a65dcf7a0d5/test/unit/ut_22_filter.rb#L84-143

('and' and 'or')

> Obviously I could be completely off-base and miss the grand vision behind
> the current design :) Anyway I would appreciate it if you could share some
> of the rationale behind the current choices you made and where actual
> filtering and transforming fit.

I have the impression I'm not too far off the mark, I just need some time to do the :filter attribute (now that I have the filter mecha shared by the filter expression).

Wdyt ?

[Raphael Simon]

Sorry I missed all that (I had actually looked at these tests but completely forgot after reading the filter expression doc today), looks like you over-delivered ;)

This definitely fits the bill. I'm looking forward to the filter attribute as this is probably going to be how we will be mostly using filters (in the spirit of mapping 'inputs' and 'outputs' to the workitem). I'm thinking the standalone expression is great to do validation (e.g. at the beginning of the workflow and all sub-workflows) while the attribute is what to use to do the mapping for each participant that needs it.

I also like the fact that the filter mechanism is easy to use (via Ruote::Filter) since I'm thinking our participants will use this directly to validate the incoming workitem (don't know if it's overkill but at some level this is nicely independent of everything else and could even live in its own gem, rufus-validate anyone?)

--

Raphael.

[John Mettraux]

On Mon, Feb 07, 2011 at 05:52:10PM -0800, Raphael Simon wrote:
>
> This definitely fits the bill. I'm looking forward to the filter attribute
> as this is probably going to be how we will be mostly using filters (in the
> spirit of mapping 'inputs' and 'outputs' to the workitem). I'm thinking the
> standalone expression is great to do validation (e.g. at the beginning of
> the workflow and all sub-workflows) while the attribute is what to use to do
> the mapping for each participant that needs it.

Hello Raphael,

sorry for the late reply : the :filter attribute is in.

https://github.com/jmettraux/ruote/blob/e5952b53957642b443327761203f949015710398/test/functional/ft_56_filter.rb

It only understands filter passed via variables for now (I'll add filter participants) later.

Please tell me if it fits your bill and how I could improve it.

> I also like the fact that the filter mechanism is easy to use (via
> Ruote::Filter) since I'm thinking our participants will use this directly to
> validate the incoming workitem (don't know if it's overkill but at some
> level this is nicely independent of everything else and could even live in
> its own gem, rufus-validate anyone?)

+1 for using Ruote::Filter from your participants.

rufus-validate : why not, thinking about it, maybe after I release ruote 2.2.0

Cheers,

[John Mettraux]

On Thu, Feb 10, 2011 at 02:20:45PM +0900, John Mettraux wrote:
>
> On Mon, Feb 07, 2011 at 05:52:10PM -0800, Raphael Simon wrote:
> >
> > This definitely fits the bill. I'm looking forward to the filter attribute
> > as this is probably going to be how we will be mostly using filters (in the
> > spirit of mapping 'inputs' and 'outputs' to the workitem). I'm thinking the
> > standalone expression is great to do validation (e.g. at the beginning of
> > the workflow and all sub-workflows) while the attribute is what to use to do
> > the mapping for each participant that needs it.
>

> sorry for the late reply : the :filter attribute is in.
>
> https://github.com/jmettraux/ruote/blob/e5952b53957642b443327761203f949015710398/test/functional/ft_56_filter.rb
>
> It only understands filter passed via variables for now (I'll add filter participants) later.
>
> Please tell me if it fits your bill and how I could improve it.
>
> > I also like the fact that the filter mechanism is easy to use (via
> > Ruote::Filter) since I'm thinking our participants will use this directly to
> > validate the incoming workitem (don't know if it's overkill but at some
> > level this is nicely independent of everything else and could even live in
> > its own gem, rufus-validate anyone?)
>
> +1 for using Ruote::Filter from your participants.
>
> rufus-validate : why not, thinking about it, maybe after I release ruote 2.2.0

Hello,

I've added participants as valid filters for the :filter attribute

https://github.com/jmettraux/ruote/commit/ceae194e14447ce857bce996fcac7913adb9b30c
https://github.com/jmettraux/ruote/commit/4e31fb648e963893389351f0fa7aa3e328bdbf8e

I think I'm done with the filters (expression and attribute) for now.

It's documented at

http://ruote.rubyforge.org/exp/filter.html
http://ruote.rubyforge.org/common_attributes.html#filter

I haven't implemented the 'record' feature found in the 'filter' expression for the :filter attribute. It doesn't seem necessary, in other words, validation filters are probably not necessary for filtered sections.

I hope to release 2.2.0 very soon.

Best regards,

[Raphael Simon]

Thanks for the hard work John. I had a look at the changes you made to the docs yesterday and it looks great.

--
Raphael.