ERROR - 'bool' object is not callable in airflow/models.py", line 1000, in run

[Serega Sheypak]

Hi, one of my custom operators fails with exception, I can't understand why, can you help me please?

[2016-02-09 16:48:13,173] {models.py:1041} ERROR - 'bool' object is not callable

Traceback (most recent call last):

  File "/app/.heroku/python/lib/python2.7/site-packages/airflow/models.py", line 1000, in run

    result = task_copy.execute(context=context)

  File "/app/extensions/untar_operator.py", line 49, in execute

    self.xcom_push(context, self.artifact_tar_gz_name, unpack_path)

TypeError: 'bool' object is not callable

[2016-02-09 16:48:13,196] {models.py:1077} ERROR - 'bool' object is not callable

Here is my Operator https://gist.github.com/seregasheypak/74c8fee735cefc85feb0 

I try to extend existing BashOperator.

airflow/models.py: 

# If a timout is specified for the task, make it fail

# if it goes beyond

result = None

if task_copy.execution_timeout:

    with utils.timeout(int(

            task_copy.execution_timeout.total_seconds())):

        result = task_copy.execute(context=context)

else:

    result = task_copy.execute(context=context) # Exception here...

[Serega Sheypak]

Something weird with inheritance, have no idea how to debug it. I just extracted BashOperator execute as standalone routine and made my custom operator be child of BaseOperator.

It works now, provably i inherit BaseOperator in a wrong way or something else is broken :)

[Maxime Beauchemin]

It looks like xcom_push is an attribute of BashOperator that expects a boolean. The method xcom_push you seem to be looking for is a method of the TaskInstance class, so you'd want to call it from the reference in the context object, as in `context['task_instance'].xcom_push(...)`

http://pythonhosted.org/airflow/code.html#airflow.operators.BashOperator

As a side note, while "untaring" is a good reusable component conceptually, think that your operator is a task factory, and that your task is a unit of work. A unit of work should not leave trace behind on the worker (like an untared file) as you can't predict whether the next step will take place on the same worker. Untaring would be more of a building block to use within an operator. 

Max

[Serega Sheypak]

Oh, self.xcom_push is "shadowed"? I use self.xcom_push to send messages to other DAG stages and it doesn't work only when I extend BashOperator. BashOperator "hides" xcom_push for some reason with boolean attribute. What is the reason? I didn't understand the idea of that trick. I do inherit "xcom_push" from the BaseOperator...

[Maxime Beauchemin]

I think it's just a collision, someone created an attribute that interferes with the parent class. We should fix that. Mind creating an issue on Github?

Max

[Serega Sheypak]

Please have a look at PR:

https://github.com/airbnb/airflow/pull/992

Hope it shouldn't break backward compatibility.