Partial failure

[ko...@tresata.com]

i have tasks a, b and c, with b depending on a, and c also depending on a. i do not enable parallelism, and the tasks are executed in the order a,b,c

what happens if b fails? since c does not depend on b, will it still run?

[Aaron Crow]

c will still run. you can test with the following Drakefile

===

a.out <-

  echo "running a" > $OUTPUT

b.out <- a.out

  blech

c.out <- a.out

  echo "running c" > $OUTPUT

On Wed, Mar 4, 2015 at 6:49 PM, <ko...@tresata.com> wrote:

i have tasks a, b and c, with b depending on a, and c also depending on a. i do not enable parallelism, and the tasks are executed in the order a,b,c

what happens if b fails? since c does not depend on b, will it still run?

--
You received this message because you are subscribed to the Google Groups "drake-workflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to drake-workflo...@googlegroups.com.
Visit this group at http://groups.google.com/group/drake-workflow.
For more options, visit https://groups.google.com/d/optout.

[Artem Boytsov]

Hm... Aaron, are you sure it's correct behavior - is it how it was implemented from the start? It seems like without parallelization it must halt. I can see how something like --ignore_errors mode can be useful (it could basically mean "ignore step errors to the extent possible"), but it probably shouldn't be the default, and we gotta take good care of how we report back to stdout/err, i.e. return the error code to the parent process, print a summary of failed steps at the end, etc. Otherwise it can be pretty confusing and potentially dangerous. 

Or am I missing something?

--

Artem.

[Koert Kuipers]

if i have a very large graph that starts with a, and b depends on a, and then c-z all also depend on a but have nothing to do with b, then why would i want c-z to not run just because b failed?

agreed that the reporting can get confusing in that case.

i did try the example aaron provided and it worked as he said.

[Artem Boytsov]

On Thu, Mar 5, 2015 at 4:16 PM, Koert Kuipers <ko...@tresata.com> wrote:

if i have a very large graph that starts with a, and b depends on a, and then c-z all also depend on a but have nothing to do with b, then why would i want c-z to not run just because b failed?

Because we should follow the principle of least surprise. I wasn't trying to suggest Drake shouldn't have a mode for ignoring errors, we could even mention this in the output, for example:

Workflow aborted: failure building B. 

Tasks aborted due to --ignore-errors=false: C, ..... Z

We could also have ignore-errors as a task option.

--

Artem.

[Koert Kuipers]

i am not sure if "ignore errors" is the best way to capture this. it suggests you might continue with the next step (that depends on the step that has an error) despite the error. what i want is the steps to run that have not had any errors in their dependencies. more like a "greedy" form of running drake with respect to errors: do as much as you can, despite the errors, without ignoring them.

[Artem Boytsov]

Yes, --ignore-errors would be a misleading name. I think --greedy is better as Drake cmd line option, but it can also be somewhat confusing as a step's option. Would be nice if we had one name that worked in both situations. 

Yours,

  Artem.

--

Artem.

[Alan Malloy]

Personally I'd copy the option and name from make: drake -k, drake --keep-going.

To unsubscribe from this group and stop receiving emails from it, send an email to drake-workflow+unsubscribe@googlegroups.com.

Visit this group at http://groups.google.com/group/drake-workflow.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "drake-workflow" group.

To unsubscribe from this group and stop receiving emails from it, send an email to drake-workflow+unsubscribe@googlegroups.com.

Visit this group at http://groups.google.com/group/drake-workflow.
For more options, visit https://groups.google.com/d/optout.

--

Artem.

--

Artem.

--

Artem.

[Koert Kuipers]

i like it!

[Artem Boytsov]

totally! i didn't know make had the same option - best proposed name by far! and works as a step option, too

i like it!

To unsubscribe from this group and stop receiving emails from it, send an email to drake-workflo...@googlegroups.com.

Visit this group at http://groups.google.com/group/drake-workflow.
For more options, visit https://groups.google.com/d/optout.

--

Artem.