1.6.1 - Subdag Operator not entering error state when child operators fail

552 views
Skip to first unread message

Steven Yvinec-Kruyk

unread,
Nov 18, 2015, 2:35:34 AM11/18/15
to Airflow
On a clean install of 1.6.1

My SubDagOperators remain in running state when the contained operators fail (following retry). Anyone else observing this behavior ??

Thanks

Maxime Beauchemin

unread,
Nov 18, 2015, 11:20:43 AM11/18/15
to Airflow
We should get to the bottom of this, but in production we use the SequentialExecutor() for SubDagOperator. We set it that way early and never looked back since it's been working well. It simplifies by preventing too much redirection (CeleryExecutor) or parallelization and subthreads within a taskinstance (LocalExecutor) 

task =    SubDagOperator(
        task_id="foo",
        subdag=subdag,
        executor=SequentialExecutor(),
        dag=dag)

Steven Yvinec-Kruyk

unread,
Nov 18, 2015, 11:53:39 AM11/18/15
to Airflow
Max,

Just to confirm ... is this a behavior you have observed with your subdags as well ? If it's specific to my DAG I can start digging for a root cause.

Thanks

Steven Yvinec-Kruyk

unread,
Nov 18, 2015, 6:27:29 PM11/18/15
to Airflow
Hi Max,

I've changed my subdags to use the sequential executor ... but the issue persists ... the SubDagOperator will succeed ... but does not fail.

Maxime Beauchemin

unread,
Nov 19, 2015, 12:06:19 AM11/19/15
to Airflow
Interesting, we had a SubDag fail this morning and it definitely did bubble up in the parent task. So yeah it's a bug. Please debug if you have time, otherwise creating a failing unit test would help for us to help fix it.

Max

Steven Yvinec-Kruyk

unread,
Nov 19, 2015, 3:15:52 PM11/19/15
to Airflow
Max,

The attached dag has a subdag with two tasks ... the second task is designed to fail. The subdagoperator does not fail however.

Tried on a server as well as on my mac with 1.6.1 from Pypi.

Let me know if you get the same result.

main.py

Steven Yvinec-Kruyk

unread,
Nov 19, 2015, 3:24:51 PM11/19/15
to Airflow
Also ... just downgraded to 1.5.2 and the issue goes away... it appears the issue is caused somewhere between 1.5.2 and 1.6.1.

Steven Yvinec-Kruyk

unread,
Nov 23, 2015, 3:56:13 PM11/23/15
to Airflow
Max,

Were you able to recreate the issue ? Just checking so I know if I should be looking at my dag code or AF code.

Thanks

Maxime Beauchemin

unread,
Nov 23, 2015, 8:41:40 PM11/23/15
to Airflow
Haven't gotten to it yet and I'm not sure when I'll do. I cannot guarantee this week since it's a 2 days week for me...
Reply all
Reply to author
Forward
0 new messages