Workflows with defused FWs never reach the COMPLETED state. How to delete them automatically ?

[David Michéa]

Hi,

Often I have to remove completed WFs. To do this I use the command :

lpad delete_wflows -s COMPLETED (in crontab)

But, sometimes inside a task I detect that the run can not continue (for a reason or an other). In those cases, my PyTask returns FWAction(defuse_children = True) because I don't want to set it as FIZZLED since I periodically rerun FIZZLED operations. I just want the underlying tasks to never run.

But it defuses only the direct child, not the childs of its child which remains in the WAITING state.

By consequence the workflow never reach the COMPLETED state and I can not delete it automatically.

I don't know the rule for setting the state of the WF according to the state of its FWs.  I would like to the workflow being completed as soon as there are no more READY and FIZZLED FWs...

Do you think it is possible ? Or do you think to a better solution to clean up my completed WFs ?

Thanks a lot,
David

[Anubhav Jain]

Hi David,

The intention behind the "defuse" was to essentially *pause* a workflow. Only the immediate child gets defused because the idea is that you can reignite just that child, and the rest of that workflow will continue to run as normal. This was the simplest solution to implement, although it does lead to some issues (for example, the children of DEFUSED fireworks are in the "WAITING" state, although they may never be eligible to run unless you reignite it).

In terms of rules for the state of the overall workflow, the code is relatively straightforward to look through: fireworks.core.firework.Workflow#state in fireworks/core/firework.py:760

For example, if *any* of the fireworks are DEFUSED, then the overall workflow state will be DEFUSED.

Thus, if you simply want to delete any workflows that have either completed or for which you've defused any firework inside, you could do:

lpad delete_wflows -s COMPLETED

lpad delete_wflows -s DEFUSED

Let me know if that works ...