AutoML tables

[Rit]

How can I use the What-if tool with a model trained on GCP AutoML tables? 

[jwe...@google.com]

Thanks for reaching out. I actually just looked into how to do this for someone else. Let me copy my response here so its visible to all:

I had to use a Jupyter notebook as opposed to a Colab notebook because running an exported AutoML Tables model requires serving the model in their cloud-automl-tables-public model server docker container, and I'm not aware of a way to run docker containers inside of a colab kernel. I couldn't find a way to run the exported model directly, without using the docker container approach, as the model seems to use TensorFlow operations that aren't available in the public installation of TensorFlow.

At the bottom of this message I've included the code from the notebook I used which includes some instructional comments.

The basic approach is:

1) Train a AutoML Tables model

2) Export it and copy the exported model to your computer, as described at https://cloud.google.com/automl-tables/docs/model-export

3) Serve the model on your computer using the docker method described at that same link

4) Install and enable the witwidget package on your local computer, as per https://github.com/pair-code/what-if-tool#how-do-i-enable-it-for-use-in-a-jupyter-notebook

5) Install Jupyter (https://jupyter.org/install) and launch into jupyter and open the attached notebook (or a similar one since your model/data will be slightly different).

6) Run the notebook cells (after making appropriate changes to load your data and set the name of your "target" feature the model is trying to predict. If your model isn't regression, you'll need to update the set_model_type call in the notebook, and also adjust your custom prediction function to return a list of class scores for each datapoint, as opposed to a single score, as we do for regression models (more details at https://pair-code.github.io/what-if-tool/learn/tutorials/notebooks/)

Let me know any follow-up questions you have and if these instructions work for you.

Jupyter notebook code:

# Before running the "jupyter notebook" command to bring up a notebook, install WIT

# for jupyter notebooks in your environment through instructions here:

# https://github.com/pair-code/what-if-tool#how-do-i-enable-it-for-use-in-a-jupyter-notebook

import tensorflow as tf

import json

import numpy as np

import pandas as pd

data_path = './bank-marketing.csv' # This was the CSV my auto ml regression model was trained on.

df = pd.read_csv(data_path)

# Turn the dataset into a list of JSON dictionaries, as WIT accepts that format.

records = json.loads(df.to_json(orient="records"))

from witwidget.notebook.visualization import WitConfigBuilder

from witwidget.notebook.visualization import WitWidget

import requests

import json

# Custom prediction function to call to the served model.

# The AutoML tables model should be exported and then locally served

# as described at https://cloud.google.com/automl-tables/docs/model-export

def custom_predict(examples):

# Send the predict request to the model served through docker.

request_data = {"instances": examples}

response = requests.post("http://localhost:8080/predict", data=json.dumps(request_data))

predictions = response.json()["predictions"]

# The served model seems to return predictions as a list if multiple examples are provided,

# but if only one example is provided in the examples list, then the predictions is a single

# value, instead of a list of length 1. In that case, we turn it into a list of length 1

# before returning it, since WIT's API is to return a list of numbers for regression models.

if not isinstance(predictions, list):

predictions = [predictions]

return predictions

# Display WIT - in this case on 1000 examples, for a regression model trying to predict the 'Balance' feature.

WitWidget(WitConfigBuilder(records[0:1000]).set_custom_predict_fn(custom_predict).set_model_type('regression').set_target_feature('Balance'), height=800) 

[Rit]

Hello, 

Thank you for your response, it's been incredibly helpful!

I've been able to follow up to step 3, however the steps for "3) Serve the model on your computer using the docker method described at that same link" weren't entirely clear on the instructions. While I've downloaded docker onto my desktop now I don't have experience with it, is there a more granular breakdown available for this? 

Thanks in advance! 

[jwe...@google.com]

After installing docker, run steps 1 through 4 from https://cloud.google.com/automl-tables/docs/model-export#run-server. That should accomplish what I listed in my above message as "step 3".