Google Cloud Composer - Unable to read remote log

152 views

Skip to first unread message

Paweł Lonca

unread,

May 19, 2021, 5:18:33 AM5/19/21

to composer-users

I'm trying to run my Airflow DAG on the Cloud Composer by Google (composer-1.16.4-airflow-1.10.15). Tasks can't be completed because Composer can't access log files. The error I get is as follows:

*** Unable to read remote log from gs://europe-west1-XXXXXXXXXXXXXXXX-bucket/logs/main_jira_dag/get_project_tasks_TBA/2021-05-19T08:02:32.739900+00:00/1.log *** 'NoneType' object has no attribute 'download' *** Log file does not exist: /home/airflow/gcs/logs/main_jira_dag/get_project_tasks_TBA/2021-05-19T08:02:32.739900+00:00/1.log *** Fetching from: http://airflow-worker-YYYYYYYYYYYY:<port>/log/main_jira_dag/get_project_tasks_TBA/2021-05-19T08:02:32.739900+00:00/1.log *** Failed to fetch log file from worker. HTTPConnectionPool(host='airflow-worker-YYYYYYYYYYYY', port=<port>): Max retries exceeded with url: /log/main_jira_dag/get_project_tasks_TBA/2021-05-19T08:02:32.739900+00:00/1.log (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6343208eb8>: Failed to establish a new connection: [Errno -2] Name or service not known',))