Can the attribution values be visualized in Tensorboard mode?

[Tanmay Gupta]

I worked with the Notebook mode and was successfully able to project attribution scores (using Shapely algorithm) on WIT dashboard. Due to a bigger data size, I then tried the visualization in the Tensorboard mode. The instructions given on the documentation page, mentions only two requirements: 1. ML model in TF serving format and, 2. TFRecord file of the example data set. There isn't any mention of generating or uploading attribution values (generated by Google Integrated Gradient or SHAP) in the Tensorboard mode. Please suggest if it's possible to add attribution values in Tensorboard mode or am I missing anything.

[James Wexler]

Thanks for reaching out Tanmay and glad you were able to get attributions working with WIT in notebook mode.

Unfortunately, WIT in TensorBoard doesn't currently support attributions. This is because WIT in TensorBoard queries the models through TF Serving's PredictionService API, which only returns prediction results from the served model, and not other custom calculated values such as shap values. We rely on the model being queried to provide the attribution results (just as you did with a custom prediction fn when using attributions in notebook mode).

When you mention wanting to use TB mode due to a bigger data size, what exactly do you refer to? Are you able to load more data points when using WIT in TB vs in notebook mode, or were you just experimenting with the limits of each mode?

On Wed, Apr 1, 2020 at 12:48 AM Tanmay Gupta <tanmay...@gmail.com> wrote:

I worked with the Notebook mode and was successfully able to project attribution scores (using Shapely algorithm) on WIT dashboard. Due to a bigger data size, I then tried the visualization in the Tensorboard mode. The instructions given on the documentation page, mentions only two requirements: 1. ML model in TF serving format and, 2. TFRecord file of the example data set. There isn't any mention of generating or uploading attribution values (generated by Google Integrated Gradient or SHAP) in the Tensorboard mode. Please suggest if it's possible to add attribution values in Tensorboard mode or am I missing anything.

--
You received this message because you are subscribed to the Google Groups "What-If Tool" group.
To unsubscribe from this group and stop receiving emails from it, send an email to what-if-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/what-if-tool/696bc2d5-6de6-4502-933f-9593348226af%40googlegroups.com.

[Tanmay Gupta]

Thanks a lot for replying and clarifying James, really appreciate your prompt response.

When I say bigger data size, I meant a test dataset of 100000 data points and ~2000 features. In the notebook mode, I experimented with a subset of my dataset of 500 data points and 50 features and was able to get the desired results. But when I triggered the WIT tool with complete dataset, the kernel kept dying repeatedly, so I thought it might be memory issue for notebook and thus wanted to experiment with the dataset in Tensorboard mode.

Please suggest for better ways to handle bigger sets for WIT, if possible. Regards.

To unsubscribe from this group and stop receiving emails from it, send an email to what-i...@googlegroups.com.