Creating custom error handling on multiple tasks?

[My Display Name]

I need to have some custom error handling, where if any of my tasks fail from a specific module, custom Python code will be run. I was introduced to event handling in Luigi from this Stack Exchange question:

https://stackoverflow.com/questions/32590364/can-luigi-propagate-exception-or-return-any-result

However, the suggested example code:

import luigi

from my_tasks import MyTask


@MyTask.event_handler(luigi.Event.FAILURE)
def mourn_failure(task, exception):
    """Will be called directly after a failed execution
    of `run` on any MyTask subclass
    """

    do_something()


luigi.run()

applies to one task. I'd like to have a custom event handler that applies to all tasks within a module. For example, let's say my module is:

class Task1(luigi.Task):

    stuff

class Task2(luigi.Task):

    stuff

def mourn_failure(task, exception):

    this will run custom code if Task1 or Task2 fail in any way

Any suggestions on how to implement this?

[Chris Palmer]

When you register an event handler on a Task class it applies to all instances of that class or any subclasses. So you could achieve what you want with something like the following:

import luigi

class MournsFailures(luigi.Task):

    pass

@MournsFailures.event_handler(luigi.Event.FAILURE)

def mourn_failure(task, exception):

    handle_failure()

class Task1(MournsFailures):

    .....

class Task2(MournsFaillures):

    ....

Hope that is clear

Chris

--
You received this message because you are subscribed to the Google Groups "Luigi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to luigi-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Brooks]

This is extremely helpful, thank you!

To unsubscribe from this group and stop receiving emails from it, send an email to luigi-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

--------------------------

Andrew Brooks, PhD

Data Scientist, Scientific Games International

www.andrew-brooks.com