Luigi

1–30 of 1023

0 selected

tashri...@gmail.com2

Jul 20

/api/ping on a browser

Okay, I realized how the URL should be constructed: http://localhost:8082/api/ping?data={"worker

unread,

/api/ping on a browser

Okay, I realized how the URL should be constructed: http://localhost:8082/api/ping?data={"worker

Jul 20

Hui Zheng, … Diego Fabiano13

Jun 30

failed tasks are not marked properly by scheduler

Same issue for me to this date. for me, the task throws an exception in the requires() but the exact

unread,

failed tasks are not marked properly by scheduler

Same issue for me to this date. for me, the task throws an exception in the requires() but the exact

Jun 30

Nahal Mirzaie, … Lars Albertsson4

Jun 17

Schedule more than 1k tasks with Luigi

1k tasks should not be a problem. What problems are you experiencing? In a large scale batch

unread,

Schedule more than 1k tasks with Luigi

1k tasks should not be a problem. What problems are you experiencing? In a large scale batch

Jun 17

Eamonn Faherty, Lars Albertsson2

Jun 17

How can I reduce the time taken to schedule tasks?

This is a fairly common problem with large DAGs or high cadence jobs. These are some tricks that we

unread,

How can I reduce the time taken to schedule tasks?

This is a fairly common problem with large DAGs or high cadence jobs. These are some tricks that we

Jun 17

Jun 14

Simple luigi.Task statistics

Hi all! I was wondering if there was a way to save some statistics related to the runtime of a task

unread,

Simple luigi.Task statistics

Hi all! I was wondering if there was a way to save some statistics related to the runtime of a task

Jun 14

Ankita Shrivastava, Tashrif3

May 21

Requires method running more then one my workflow in dynamic dependency yield.

Hi Tashrif, I have not set up this parameter anywhere. in my code. is it somehow related to yield

unread,

Requires method running more then one my workflow in dynamic dependency yield.

Hi Tashrif, I have not set up this parameter anywhere. in my code. is it somehow related to yield

May 21

Ricardo Wagenmaker, Lars Albertsson2

May 18

Processes vs Threads

Luigi is essentially single-threaded. The only exception is the worker's KeepAliveThread, which

unread,

Processes vs Threads

Luigi is essentially single-threaded. The only exception is the worker's KeepAliveThread, which

May 18

Adrien Berchet, Lars Albertsson3

May 4

Extend contrib or a new repository?

Ok, thank you for your answer. I will try to submit a first PR soon and see how it goes then. Le sam.

unread,

Extend contrib or a new repository?

Ok, thank you for your answer. I will try to submit a first PR soon and see how it goes then. Le sam.

May 4

Marchello Lippi, … Lars Albertsson5

Apr 13

Luigi does not support python 2.7. You will need to install a recent version of python. Regards, Lars

unread,

Luigi does not support python 2.7. You will need to install a recent version of python. Regards, Lars

Apr 13

Friscian Viales, Tashrif B.3

Mar 8

Application Hosting

Great, thanks for the info! On Mon, Mar 8, 2021, 20:54 Tashrif B. <tashrifbillah@gmail.com>

unread,

Application Hosting

Great, thanks for the info! On Mon, Mar 8, 2021, 20:54 Tashrif B. <tashrifbillah@gmail.com>

Mar 8

Paul S. Johnson, Lars Albertsson4

Mar 1

Getting list of files on FTP site

I'm sorry, I gave bad advice. We cannot overload the path property, since it is assigned in

unread,

Getting list of files on FTP site

I'm sorry, I gave bad advice. We cannot overload the path property, since it is assigned in

Mar 1

Vasco Patarata, … Lars Albertsson24

Mar 1

Create dynamically sequential tasks in Luigi

Yes, you have to specify WrapperTask1 as a dependency for all tasks under WrapperTask2. If

unread,

Create dynamically sequential tasks in Luigi

Yes, you have to specify WrapperTask1 as a dependency for all tasks under WrapperTask2. If

Mar 1

Christoph Thiede, Tashrif B.3

Feb 19

luigi.format.TextFormat: How to preserve line endings/carriage return?

Ah, now I see. Unfortunately, the built-in open() method does not support luigi's MockTargets

unread,

luigi.format.TextFormat: How to preserve line endings/carriage return?

Ah, now I see. Unfortunately, the built-in open() method does not support luigi's MockTargets

Feb 19

Rafał Wojdyła

Feb 18

Official python support >3.7

Hi, I wonder if anyone has been using Luigi on python 3.8 or 3.9? And could provide any valuable

unread,

Official python support >3.7

Hi, I wonder if anyone has been using Luigi on python 3.8 or 3.9? And could provide any valuable

Feb 18

Arash Rouhani, Lars Albertsson2

Feb 17

Handing over the Maintainership of the Luigi Codebase

Thanks for sticking around to maintain Luigi for so long! We are grateful for all the time that you

unread,

Handing over the Maintainership of the Luigi Codebase

Thanks for sticking around to maintain Luigi for so long! We are grateful for all the time that you

Feb 17

Zoltan, … Lars Albertsson4

Feb 10

Hi, This is easy to achieve without explicit support in Luigi with ExternalTasks, as you mention.

unread,

Hi, This is easy to achieve without explicit support in Luigi with ExternalTasks, as you mention.

Feb 10

eldad marciano2

Jan 14

How to let pending task to run when a worker finish ?

Solved the issue, please ignore the Q On Thursday, January 14, 2021 at 12:35:50 PM UTC+2 eldad

unread,

How to let pending task to run when a worker finish ?

Solved the issue, please ignore the Q On Thursday, January 14, 2021 at 12:35:50 PM UTC+2 eldad

Jan 14

Tashrif B., … pete4

12/13/20

How to run Luigi unittests?

Tashrif Sorry for not being more responsive. You're doing thorough work here and you deserve some

unread,

How to run Luigi unittests?

Tashrif Sorry for not being more responsive. You're doing thorough work here and you deserve some

12/13/20

12/9/20

Set spark config for one task only

This is not a question but something I could find neither on this list, nor in the docs. If you have

unread,

Set spark config for one task only

This is not a question but something I could find neither on this list, nor in the docs. If you have

12/9/20

Felix Kunzweiler, Lars Albertsson2

11/27/20

Limiting remote connections

Hi, Resources are meant for this use case. Luigi allocates the resource per task, so only your tasks

unread,

Limiting remote connections

Hi, Resources are meant for this use case. Luigi allocates the resource per task, so only your tasks

11/27/20

11/18/20

How to access column id in luigi-task-hist.db

Hi all, I figured a way by querying the SQL database file. Here is my gist. Thanks. On Wed, Nov 11,

unread,

How to access column id in luigi-task-hist.db

Hi all, I figured a way by querying the SQL database file. Here is my gist. Thanks. On Wed, Nov 11,

11/18/20

kar...@hackerearth.com, … Lars Albertsson5

11/16/20

RPCError: Errors (3 attempts) when connecting to remote scheduler 'http://localhost:8082'

Your Luigi worker fails to contact the central scheduler, which it believes to run on the same

unread,

RPCError: Errors (3 attempts) when connecting to remote scheduler 'http://localhost:8082'

Your Luigi worker fails to contact the central scheduler, which it believes to run on the same

11/16/20

Itay Feldman, Peter Weissbrod2

11/13/20

luigi ETL + MS SQL tutorial

I have code which happens to interact with sql server via pymssql but it is insert code and isnt

unread,

luigi ETL + MS SQL tutorial

I have code which happens to interact with sql server via pymssql but it is insert code and isnt

11/13/20

Matthew Lueder2

11/12/20

How do you set the luigi configuration from within code?

It appears this was a celery issue and not a Luigi one. I managed to fix this by moving the code to

unread,

How do you set the luigi configuration from within code?

It appears this was a celery issue and not a Luigi one. I managed to fix this by moving the code to

11/12/20

Hari Krishna Dara, Lars Albertsson4

10/28/20

Workflow that includes manual steps

That sounds like a good approach to me. Looking forward to the blog post. ;-) Lars Albertsson Data

unread,

Workflow that includes manual steps

That sounds like a good approach to me. Looking forward to the blog post. ;-) Lars Albertsson Data

10/28/20

Nhat Bui Minh, Lars Albertsson2

10/24/20

Parallelizing Multiple Tasks which share the same dependencies

Yes, what you attempt to do is a standard pattern, so something has gone wrong. If you share a

unread,

Parallelizing Multiple Tasks which share the same dependencies

Yes, what you attempt to do is a standard pattern, so something has gone wrong. If you share a

10/24/20

Nhat Bui Minh, Tashrif B.4

10/18/20

Luigi does not try Failed Tasks

Actually, please disregard my prev email. The process works! Thank you so much!!!!! On Sunday,

unread,

Luigi does not try Failed Tasks

Actually, please disregard my prev email. The process works! Thank you so much!!!!! On Sunday,

10/18/20

tr...@dose.com, … Lars Albertsson6

9/30/20

Luigi across multiple hosts

Nowadays, the easiest way to scale out to multiple nodes for Luigi pipelines is to run Luigi in

unread,

Luigi across multiple hosts

Nowadays, the easiest way to scale out to multiple nodes for Luigi pipelines is to run Luigi in

9/30/20

9/13/20

Copying data from DB source

Hi all, I currently have a number of tasks that copies data from Oracle, does some transformation/

unread,

Copying data from DB source

Hi all, I currently have a number of tasks that copies data from Oracle, does some transformation/

9/13/20

Franz Pucher, HadoopMarc6

9/7/20

How to use a tf.data.Dataset object as output target for reuse in other Tasks?

Thank you for the suggestions. Another solution that I'll probably try is to create the tf.data.

unread,

How to use a tf.data.Dataset object as output target for reuse in other Tasks?

Thank you for the suggestions. Another solution that I'll probably try is to create the tf.data.

9/7/20

Search

Clear search

Close search

Google apps

Main menu