Luigi

1–30 of 1040

0 selected

João Victor Davim, Lars Albertsson2

Mar 22

S3Target with custom endpoint url

If you add some extra keyword arguments to the S3Client constructor, it seems they will be passed on

unread,

S3Target with custom endpoint url

If you add some extra keyword arguments to the S3Client constructor, it seems they will be passed on

Mar 22

Mar 18

authentication of users and permissions (how-to, documentation, samples?)

Dumb question? Does luigi support users, we have a small team, but we'd like to know who did what

unread,

authentication of users and permissions (how-to, documentation, samples?)

Dumb question? Does luigi support users, we have a small team, but we'd like to know who did what

Mar 18

Niko Strongioglou

Mar 16

Successfull task run with Unfulfilled dependency at run time

I have the following setup ```python class RootTask(luigi.WrapperTask): def requires(self):

unread,

Successfull task run with Unfulfilled dependency at run time

I have the following setup ```python class RootTask(luigi.WrapperTask): def requires(self):

Mar 16

Niko Strongioglou

Mar 9

TypeError: requires() missing 1 required positional argument: 'self'

Hello fellow Luigi users. I am stuck on this error for quite some time now. This is the code that

unread,

TypeError: requires() missing 1 required positional argument: 'self'

Hello fellow Luigi users. I am stuck on this error for quite some time now. This is the code that

Mar 9

Filippe Gonçalves, Lars Albertsson2

Mar 7

Documentation: luigi.contrib.azureblob

Yes, that's the way to contribute documentation. There are some additional instructions at https:

unread,

Documentation: luigi.contrib.azureblob

Yes, that's the way to contribute documentation. There are some additional instructions at https:

Mar 7

Steffen Köhler, Lars Albertsson2

Feb 10

rerun a task if a earlier required input has changed?

By default, completeness in Luigi's dependency model is based on existence only. This differs

unread,

rerun a task if a earlier required input has changed?

By default, completeness in Luigi's dependency model is based on existence only. This differs

Feb 10

Stephen Sun, … Stephen Sun10

Feb 8

How to concatenate results from multiple tasks

Finally solved the issue by using MockTarget. Just generate whatever file format you want and it

unread,

How to concatenate results from multiple tasks

Finally solved the issue by using MockTarget. Just generate whatever file format you want and it

Feb 8

Massy Bourennani, Lars Albertsson2

12/25/21

Best Luigi reads

There are two presentations and a blog post under "batch processing" in this list: https://

unread,

Best Luigi reads

There are two presentations and a blog post under "batch processing" in this list: https://

12/25/21

pur...@gmail.com

12/1/21

Is there a way to change the font color of debug console messages?

Given the lack of mention on documentation, the answer seems to be No, but I thought I would ask.

unread,

Is there a way to change the font color of debug console messages?

Given the lack of mention on documentation, the answer seems to be No, but I thought I would ask.

12/1/21

Massy Bourennani

11/23/21

How to use instance attributes in Luigi Task

Hi ! What is the best way to use instance attributes in Luigi Task ? More details in SO https://

unread,

How to use instance attributes in Luigi Task

Hi ! What is the best way to use instance attributes in Luigi Task ? More details in SO https://

11/23/21

Stephen Sun, … Lars Albertsson4

10/11/21

Luigi Task fails with output as a dictionary. (Under Docker env)

-9 means that the process was killed by the operating system or some other process using the SIGKILL

unread,

Luigi Task fails with output as a dictionary. (Under Docker env)

-9 means that the process was killed by the operating system or some other process using the SIGKILL

10/11/21

Bill Tucker, … Adrien Berchet6

10/1/21

event handlers, how should they really behave?

It is possible to keep the functions inside the class and setup the event handler in the __init__

unread,

event handlers, how should they really behave?

It is possible to keep the functions inside the class and setup the event handler in the __init__

10/1/21

Fiorella Zampetti

9/20/21

Help needed :-)

Hello, We're a team of Software Engineering researchers investigating technical debt. Technical

unread,

Help needed :-)

Hello, We're a team of Software Engineering researchers investigating technical debt. Technical

9/20/21

9/14/21

Avoid waiting on long running remote tasks

I have a use case where I'm using Luigi to schedule tasks running on a remote cluster. I'm

unread,

Avoid waiting on long running remote tasks

I have a use case where I'm using Luigi to schedule tasks running on a remote cluster. I'm

9/14/21

parlad neupane, HadoopMarc2

8/24/21

Handeling Failed Task

Hi, No. If task1 and task2 cannot be reused in a rerun (after correcting error of task3), you should

unread,

Handeling Failed Task

Hi, No. If task1 and task2 cannot be reused in a rerun (after correcting error of task3), you should

8/24/21

Friscian Viales, Lars Albertsson2

8/19/21

Getting TimeoutError while scheduling tasks

I recall creating a custom GCSClient that reused a single connection at some point. IIRC, I discarded

unread,

Getting TimeoutError while scheduling tasks

I recall creating a custom GCSClient that reused a single connection at some point. IIRC, I discarded

8/19/21

tashri...@gmail.com

8/19/21

[Show and tell] Luigi server behind authentication

Hello all, For a long time, I have been looking for an authentication mechanism to host a public

unread,

[Show and tell] Luigi server behind authentication

Hello all, For a long time, I have been looking for an authentication mechanism to host a public

8/19/21

8/18/21

Grouped and stacked tasklist

Hi all, Can someone explain what the grouped and stacked tasklist visualizations are? I see that the

unread,

Grouped and stacked tasklist

Hi all, Can someone explain what the grouped and stacked tasklist visualizations are? I see that the

8/18/21

tashri...@gmail.com, Lars Albertsson3

8/13/21

/api/ping on a browser

You shouldn't access the luigid web api from your code. Luigi worker code calls the web api. Your

unread,

/api/ping on a browser

You shouldn't access the luigid web api from your code. Luigi worker code calls the web api. Your

8/13/21

Hui Zheng, … Diego Fabiano13

6/30/21

failed tasks are not marked properly by scheduler

Same issue for me to this date. for me, the task throws an exception in the requires() but the exact

unread,

failed tasks are not marked properly by scheduler

Same issue for me to this date. for me, the task throws an exception in the requires() but the exact

6/30/21

Nahal Mirzaie, … Lars Albertsson4

6/17/21

Schedule more than 1k tasks with Luigi

1k tasks should not be a problem. What problems are you experiencing? In a large scale batch

unread,

Schedule more than 1k tasks with Luigi

1k tasks should not be a problem. What problems are you experiencing? In a large scale batch

6/17/21

Eamonn Faherty, Lars Albertsson2

6/17/21

How can I reduce the time taken to schedule tasks?

This is a fairly common problem with large DAGs or high cadence jobs. These are some tricks that we

unread,

How can I reduce the time taken to schedule tasks?

This is a fairly common problem with large DAGs or high cadence jobs. These are some tricks that we

6/17/21

6/14/21

Simple luigi.Task statistics

Hi all! I was wondering if there was a way to save some statistics related to the runtime of a task

unread,

Simple luigi.Task statistics

Hi all! I was wondering if there was a way to save some statistics related to the runtime of a task

6/14/21

Ankita Shrivastava, Tashrif3

5/21/21

Requires method running more then one my workflow in dynamic dependency yield.

Hi Tashrif, I have not set up this parameter anywhere. in my code. is it somehow related to yield

unread,

Requires method running more then one my workflow in dynamic dependency yield.

Hi Tashrif, I have not set up this parameter anywhere. in my code. is it somehow related to yield

5/21/21

Ricardo Wagenmaker, Lars Albertsson2

5/18/21

Processes vs Threads

Luigi is essentially single-threaded. The only exception is the worker's KeepAliveThread, which

unread,

Processes vs Threads

Luigi is essentially single-threaded. The only exception is the worker's KeepAliveThread, which

5/18/21

Adrien Berchet, Lars Albertsson3

5/4/21

Extend contrib or a new repository?

Ok, thank you for your answer. I will try to submit a first PR soon and see how it goes then. Le sam.

unread,

Extend contrib or a new repository?

Ok, thank you for your answer. I will try to submit a first PR soon and see how it goes then. Le sam.

5/4/21

Marchello Lippi, … Lars Albertsson5

4/13/21

Luigi does not support python 2.7. You will need to install a recent version of python. Regards, Lars

unread,

Luigi does not support python 2.7. You will need to install a recent version of python. Regards, Lars

4/13/21

Friscian Viales, Tashrif B.3

3/8/21

Application Hosting

Great, thanks for the info! On Mon, Mar 8, 2021, 20:54 Tashrif B. <tashrifbillah@gmail.com>

unread,

Application Hosting

Great, thanks for the info! On Mon, Mar 8, 2021, 20:54 Tashrif B. <tashrifbillah@gmail.com>

3/8/21

Paul S. Johnson, Lars Albertsson4

3/1/21

Getting list of files on FTP site

I'm sorry, I gave bad advice. We cannot overload the path property, since it is assigned in

unread,

Getting list of files on FTP site

I'm sorry, I gave bad advice. We cannot overload the path property, since it is assigned in

3/1/21

Vasco Patarata, … Lars Albertsson24

3/1/21

Create dynamically sequential tasks in Luigi

Yes, you have to specify WrapperTask1 as a dependency for all tasks under WrapperTask2. If

unread,

Create dynamically sequential tasks in Luigi

Yes, you have to specify WrapperTask1 as a dependency for all tasks under WrapperTask2. If

3/1/21

Search

Clear search

Close search

Google apps

Main menu