To run each task in a new python process

[beni...@gmail.com]

Hi,

I have just started to use Luigi. Because of memory restrictions, (I just have one machine to run the entire pipeline) I would like to know if it possible to configure Luigi to run any task in a new python process so the memory is cleaned up after the execution of each task.

Thanks in advance.

[Arash Rouhani]

It does that by default if you supply --workers >1

--
You received this message because you are subscribed to the Google Groups "Luigi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to luigi-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[beni...@gmail.com]

Thanks very much