In the following example in line c (the bash command) which looks like
c = 'echo LOG_HOUR_LOAD={ {{ ti.xcom_pull("hour_calc")[2] }} , {{ ti.xcom_pull("hour_calc")[3] }} } >> parameters.txt;'
I'm trying to render something like:
'echo LOG_HOUR_LOAD={20151006 , 20151008} >> parameters.txt;'However the first and last curly bracket causes the dag task to fail. I have tried various escape characters without result ( \ and \\ ). Does anyone know how to write:
LOG_HOUR_LOAD={ {{ ti.xcom_pull("hour_calc")[2] }} , {{ ti.xcom_pull("hour_calc")[3] }} } to parameters.txt using the BashOperatorHere's the full code example:
import airflow
import time
from datetime import datetime, timedelta
def calc(ds, execution_date, ti, **kwargs):
ed = int(time.mktime(execution_date.timetuple()))
hour = ((int(ed)/3600) * 3600) - 3600
hour_1 = hour - 3600
hour_p1 = hour + 3600
log_hour_load = time.strftime("%H", time.gmtime(hour))
log_hour_before = time.strftime("%H", time.gmtime(hour_1))
log_hour_p1 = time.strftime("%H", time.gmtime(hour_p1))
return hour, log_hour_load, log_hour_before, log_hour_p1
args = {'owner': 'webstats', 'start_date': datetime(2015, 10, 7, 9, 40, 0), 'depends_on_past': False}
dag = airflow.DAG(dag_id='webstats', schedule_interval=timedelta(hours=1), default_args=args)
t1 = airflow.operators.PythonOperator(task_id='hour_calc', provide_context=True, python_callable=calc, dag=dag)
c = 'echo LOG_HOUR_LOAD={ {{ ti.xcom_pull("hour_calc")[2] }} , {{ ti.xcom_pull("hour_calc")[3] }} } >> parameters.txt;'
t2 = airflow.operators.BashOperator(task_id='parameters', bash_command=c, dag=dag)
t2.set_upstream(t1)