Airflow jobs are stuck

1 view
Skip to first unread message

Aman Jain

unread,
Jun 1, 2019, 3:28:24 AM6/1/19
to composer-users
Problem:
I am running Cloud composer on n1-standard-2 machine with 3 nodes. I am executing 4 dags every 1 minute and airflow.cfg is default settings. But the dag is not executing from last 3 days(27 May, 2019). 
There is no task instance after the 27 May and more than 17 task instances are in running state but there is no log.

I have checked the airflow-worker pods and all 3 pods are in running state.

Also, I have tried to replace the airflow scheduler pod and cleared last running job. Dag started to run but after some run it stuck again as before.
Should I change the machine type. But how to decide which machine to choose.

Reply all
Reply to author
Forward
0 new messages