on_failure_callback in default_args in DAG

[Keeyong Han]

I was using this well in Apache Airflow installation:

DAG = DAG(

  dag_id='failcallback_example_dag',

  start_date=datetime(2018, 9, 30),

  schedule_interval='@once',

  on_failure_callback=on_failure_callback,

)

But in the Cloud Composer, I got this error:

 __init__() got an unexpected keyword argument 'on_failure_callback'

When I moved it to each task's configuration like the following, it worked though:

task0 = PythonOperator(

    task_id='task0',

    python_callable=task0_python_callable,

    on_failure_callback=on_failure_callback,

    provide_context=True,

    dag=DAG

)

Cloud Composer doesn't support on_failure_callback in DAG definition? 

[Keeyong Han]

Somehow adding this option to default_args parameter worked. 

Cheers! 

[Wilson Lian]

on_failure_callback is only supported in Operators in Airflow 1.9.0. I noticed that the Airflow documentation is out of date and shows that it's a valid parameter for DAGs. I suspect you're using an old version of Airflow in your local installation?

--
You received this message because you are subscribed to the Google Groups "cloud-composer-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloud-composer-discuss+unsub...@googlegroups.com.
To post to this group, send email to cloud-composer-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloud-composer-discuss/44c5eb98-e39a-439c-bbc5-faa30bf960ea%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Keeyong Han]

You are right Wilson. Thanks!

To post to this group, send email to cloud-compo...@googlegroups.com.