Proposal: Adding Model Interpretability & Explainability Library and/or Examples to Keras

30 views

Skip to first unread message

Nikola Savic

unread,

Mar 26, 2025, 7:10:30 AMMar 26

to Keras-users

Hello,

Are there any plans on creating a model interpretability and explainability library for Keras? There are a few reasons I think Keras should have such a library.

The first is because most users who use Keras value simplicity and ease of use, so having a library where someone could just import a tool like occlusionSensitivity and use it on their custom or imported model would be really valuable.

The second reason is that Keras supports multiple backends, so it could be the interpretability tool of choice for users coming from various backgrounds. Additionally, it could leverage JAX for performance.

The Keras docs provide some good examples of model interpretability, mostly for vision models. I was also thinking about expanding those examples with different vision and LLM models from Keras Hub.

I am currently working on adding an example on interpreting model decisions using occlusion sensitivity. I'd be happy to contribute to this effort more by helping create example notebooks and possibly by contributing to the development of an interpretability library. Please let me know your thoughts on this.