How can I disable logs from airflow?

4,493 views
Skip to first unread message

Praveenkumar Venkatesan

unread,
Dec 7, 2015, 1:30:32 PM12/7/15
to Airflow
I have too many INFO logs being printed by airflow modules. I am not an expert with the use of "logging" framework. But basing off of examples online, I've tried doing the following

```
airflow_log = logging.getLogger("airflow")
airflow_log.setLevel(logging.WARNING)
```

I've also tried playing with the `propagate` flag. Is airflow the logger name? I've tried changing the logger name to `airflow.hooks` but that doesn't work either.




Praveenkumar Venkatesan

unread,
Dec 8, 2015, 1:16:53 PM12/8/15
to Airflow, maximebe...@gmail.com
@Max - any thoughts on this?

Maxime Beauchemin

unread,
Dec 8, 2015, 1:30:23 PM12/8/15
to Airflow, maximebe...@gmail.com
I just landed a PR that might help with some of this. There's also the LOGGIN_LEVEL constant in `settings.py` that you can overwrite in your local config.

Dmitriy Lee

unread,
Feb 17, 2016, 7:54:00 PM2/17/16
to Airflow, maximebe...@gmail.com
I think it would be great to have an option for loading logging configuration from an external file (either cfg or yaml, preferably yaml). It provides an easy way to control behavior of individual loggers. For example you can have airflow logger outputting at INFO level while suppressing output of boto logger (which is way too verbose) to WARNINGS.

Example config file 

version: 1
disable_existing_loggers: false

root:
level: NOTSET
handlers: [console, info_file_handler, error_file_handler]

formatters:

standard:
format: "%(asctime)s [%(levelname)s] %(name)s: %(message)s"
datefmt: '%Y-%m-%d %H:%M:%S%z'

handlers:

default:
level: INFO
class: logging.StreamHandler

console:
class: logging.StreamHandler
level: DEBUG
formatter: standard
stream: ext://sys.stdout

info_file_handler:
class: logging.handlers.RotatingFileHandler
level: INFO
formatter: standard
filename: ../log/mongodb_cdc_agent.log
maxBytes: 10485760 # 10MB
backupCount: 20
encoding: utf8

loggers:
paramiko.transport:
level: WARNING

boto3:
level: WARNING

botocore:
level: WARNING
Reply all
Reply to author
Forward
0 new messages