Handeling Failed Task

11 views
Skip to first unread message

parlad neupane

unread,
Aug 24, 2021, 11:04:57 AM8/24/21
to Luigi
Hi Everyone,

I have 5 tasks in a abc.py file. Is there a way to failed all task if one task failed. I tried putting it on required function but it ran and updated the successful tasks. For example, i have task1, task2, task3, task4, task5. Now task3 fails, but task1, and task2 runs successful. Is there way to prevent task1 and task2 from updating cause task 3 fails. If any task fails, is it possible to fail all the task?
 

HadoopMarc

unread,
Aug 24, 2021, 11:22:38 AM8/24/21
to Luigi
Hi,

No. If task1 and task2 cannot be reused in a rerun (after correcting error of task3), you should pass a parameter (e.g. a timestamp) to task1 and task2 and include it in the output target name. This way there is no result for task1 and task2 if you run it with a different input parameter.

Best wishes,     Marc

Op dinsdag 24 augustus 2021 om 17:04:57 UTC+2 schreef parlad...@gmail.com:
Reply all
Reply to author
Forward
0 new messages