Luigi does not try Failed Tasks

[Nhat Bui Minh]

This the luigi.cfg file that I has set up

[core]

logging_conf_file=logging.conf

[scheduler]

retry_count=100

retry_delay=5

[worker]

keep_alive=true

max_reschedules=100

However, luigi does not retry any  failed tasks in my pipeline. Really appreciate if anyone could point to me the correct way of telling luigi to retry failed tasks !

[Tashrif B.]

Hi Nhat,

Check out the last line in my configuration and see if that solves your problem:

https://github.com/pnlbwh/luigi-pnlpipe/blob/diag-cte/luigi.cfg

-Tashrif

.

--
You received this message because you are subscribed to the Google Groups "Luigi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to luigi-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/luigi-user/47cc7b0d-2316-4830-88fa-5e24008f0410n%40googlegroups.com.

[Nhat Bui Minh]

Hi Tashrif,

Thank you but that does not seem to work.

One thing I notice is Luigi keeps marking those failed tasks as pending when it asks scheduler for work. For example : I have 137 failed tasks in one run

DEBUG - Asking scheduler for work...

DEBUG - http://localhost:8082 "POST /api/get_work HTTP/1.1" 200 160

DEBUG - Done

DEBUG - There are no more tasks to run at this time

DEBUG - There are 137 pending tasks possibly being run by other workers

DEBUG - There are 137 pending tasks unique to this worker

DEBUG - There are 137 pending tasks last scheduled by this worker

[Nhat Bui Minh]

Actually, please disregard my prev email. The process works! Thank you so much!!!!!