how to update an existing dag without affecting re-try?

71 views
Skip to first unread message

Frank Wang

unread,
Mar 24, 2016, 2:19:43 PM3/24/16
to Airflow
I have two questions

1. if I have a DAG used by our production system now, I need to release the same DAG(same ID) with some changes(e.g. add new operators), whats the best way to do it?  should I create a new DAG for the new changes and turn off the old DAG? I also wonder how will this affect the retry logic, will airflow system try to run the re-try on the new DAG if the new DAG uses the same name/ID

2. my DAG has 3 operators, can I set retry logic for one of the operators, but not for the whole DAG? if so, how to do it


Thanks

Frank 

Or Sher

unread,
Apr 18, 2016, 6:31:40 AM4/18/16
to Airflow
I'm at the same point when I need to add some Operators to my prod DAG but I don't want the scheduler to automatically backfill these operators.
What happens when deploying the same DAG with new operators?
What's the way to deal with these changes?

Or Sher

unread,
Apr 18, 2016, 9:14:12 AM4/18/16
to Airflow
OK, tried it.
It doesn't kick old schedules for the new Operators.
Just the way I hoped it will act!
Reply all
Reply to author
Forward
0 new messages