How to retrieve a value in xcom pushed via BashOperator ?

10,158 views
Skip to first unread message

Hao Ren

unread,
Mar 23, 2016, 1:32:06 PM3/23/16
to Airflow
Here my code for retrieve a string pushed via xcom:

```
default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2016, 6, 1)
}

dag = DAG('xcom-test', default_args=default_args, schedule_interval=timedelta(1))

push = BashOperator(
    task_id='push',
    bash_command="echo value",
    xcom_push=True,
    dag=dag)

pull = BashOperator(
    task_id='pull',
    bash_command="echo {{ params.task.xcom_pull(task_ids='push') }}",
    params={'task': push},
    xcom_push=True,
    dag=dag)

pull.set_upstream(push)
```

pull task should display "value" as output.

It does not work, since xcom_pull call on BashOperator needs a context argument.
I do not find any clues on how this context argument is retrieved, and its type, etc.
Any idea on this ?

Hao

Jeremiah Lowin

unread,
Mar 24, 2016, 6:32:14 PM3/24/16
to Airflow
Hi Hao,

Call xcom_pull on the TaskInstance (the object that is actually being executed), rather than the task (which is essentially the class of the TaskInstance). To do that, just change your line to:
   
bash_command="echo {{ ti.xcom_pull(task_ids='push') }}"

Jeremiah

Jeremiah Lowin

unread,
Mar 24, 2016, 6:33:39 PM3/24/16
to Airflow
One more thing, you shouldn't need to supply any params to your second operator (remove the line params={'task': push})

Hao Ren

unread,
Mar 30, 2016, 5:40:26 AM3/30/16
to Airflow
Thank you. It works.
Reply all
Reply to author
Forward
0 new messages