What we can do for: PendingDeprecationWarning: .... BigQueryOperator. ... dropped in Airflow 2.0

[sgio...@bitbang.com]

Hi,

when I pass a query to make a dml on BigQuery with 

Es:

table_to_dwh_task = BigQueryOperator(
    dag=self.dag,
    task_id=p_task_id,
    sql=p_query,
    # Airflow dice che il parametro è obsloleto ma il dag di GCP lo vuole
    bql=p_query,
    destination_dataset_table="{0}.{1}.{2}".format(self.project, self.bq_dwh_dataset, p_table_name),
    allow_large_results=True,
    use_legacy_sql=False,
    write_disposition=write_disposition,
    priority="BATCH"
)

where p_query is a sql command:

f"select a.* except (day) from {view_name} a where day = parse_timestamp('%Y%m%d','{p_day}')"

I got this warning:

/usr/local/lib/airflow/airflow/models.py:2159: PendingDeprecationWarning: Invalid arguments were passed to BigQueryOperator. Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {'sql': "select a.* except (day) from .... <omitted> parse_timestamp('%Y%m%d','20181231')", 'priority': 'BATCH'}
  category=PendingDeprecationWarning

We followed the cloud.google.com examples to build the dag, so the question is?

The warning depends by the bsql parameter o what ever?

Thanks

We work on GCP with Composer and Airflow 1.9.

[Sergei Guschin]

Why you passing both sql and bql? try to use just sql.

[Stefano Giostra]

Hi Sergei,

On GCC Airflow is 1.9 version, so i must use bsql, so i set bsql and sql variable.

I can't not use bsql until we will adopt AF 1.10

[Sergei Guschin]

i see, in 1.9 it's just a bql

is it just complaining that 'sql' is not in parameters list? what happen if you drop 'sql'

[Stefano Giostra]

Thanks Segei.

On my PC i have AF 1.10, so i have sql, bql and priority.

I've commented sql and priority and now I haven't more warning messagges.

Thanks again.

Stefano

[Leah Cole]

Hi - the bql parameter is going to be deprecated in Airflow 2.0. You're completely right - the GCP example use it, and it's in our backlog to change them to use the sql parameter instead! You should be good if you use the sql parameter instead of the bql one. 

[Stefano Giostra]

Hi, 

with the new project we create a new cloud-composer so now we have AF 1.10 and now we can use sql and priority

Thanks

[Leah Cole]

Just as a further update, I've changed the samples to use the sql parameter instead of the bql one.