Airflow ->Greenplum

[Erlan Tayshabarov]

In the old DAG code, there are many database queries similar to:
df1 = pd.read_sql('''                                
    select d.id, d.is_group, t.table_name, d.is_parallel
    from map_table d
    join etl_user.dwh_all_table t on t.table_id = d.target_table_id
    where d.is_complex = 1 and d.is_active = 1 and task_type_id = 15;
''')
Each of these queries creates many transactions, resulting in performance issues. For those who have similar experience, how did you optimize them? Please share your experience.