Task fails with state "upstream failed" even though upstream executed successfully

89 views
Skip to first unread message

rich...@rezco.co.za

unread,
May 25, 2018, 3:23:59 AM5/25/18
to composer-users
Hi there

I'm testing running several bigquery operator tasks (very basic queries writing to tables in this example) with one level of dependency. My DAG (bg_flatten_table_new3, attached) has been running on composer for a few days, mostly working 100%. However, every now and then it fails to execute a task with state "upstream failed" even though upstream executed successfully. 

Composer log (attached) for this task is empty and instance details (attached) also doesn't give much info. 

Any help would be much appreciated.  
Example task failed.png
bq_flatten_table_new3.py
task instance details.png
log.png
Reply all
Reply to author
Forward
0 new messages