Hello,
Are there any plans on creating a model interpretability and explainability library for Keras? There are a few reasons I think Keras should have such a library.
The first is because most users who use Keras value simplicity and ease of use, so having a library where someone could just import a tool like occlusionSensitivity and use it on their custom or imported model would be really valuable.
The second reason is that Keras supports multiple backends, so it could be the interpretability tool of choice for users coming from various backgrounds. Additionally, it could leverage JAX for performance.
The Keras docs provide some good examples of model interpretability, mostly for vision models. I was also thinking about expanding those examples with different vision and LLM models from Keras Hub.
I am currently working on adding an example on interpreting model decisions using occlusion sensitivity. I'd be happy to contribute to this effort more by helping create example notebooks and possibly by contributing to the development of an interpretability library. Please let me know your thoughts on this.
Best Regards,
Nikola Savic