Handeling Failed Task

[parlad neupane]

Hi Everyone,

I have 5 tasks in a abc.py file. Is there a way to failed all task if one task failed. I tried putting it on required function but it ran and updated the successful tasks. For example, i have task1, task2, task3, task4, task5. Now task3 fails, but task1, and task2 runs successful. Is there way to prevent task1 and task2 from updating cause task 3 fails. If any task fails, is it possible to fail all the task?

 

[HadoopMarc]

Hi,

No. If task1 and task2 cannot be reused in a rerun (after correcting error of task3), you should pass a parameter (e.g. a timestamp) to task1 and task2 and include it in the output target name. This way there is no result for task1 and task2 if you run it with a different input parameter.

Best wishes,     Marc

Op dinsdag 24 augustus 2021 om 17:04:57 UTC+2 schreef parlad...@gmail.com: