How to send a tf.example into a TensorFlow Serving gRPC predict request

[jjen...@kinetica.com]

I have data in tf.example form and am attempting to make requests in predict form (using gRPC) to a saved model. I am unable to identify the method call to effect this.

I am starting with the well known Automobile pricing DNN regression model (https://github.com/tensorflow/models/blob/master/samples/cookbook/regression/dnn_regression.py) which I have already exported and mounted via the TF Serving docker container

import grpc

import numpy as np

import tensorflow as tf

from tensorflow_serving.apis import predict_pb2, prediction_service_pb2_grpc

stub = prediction_service_pb2_grpc.PredictionServiceStub(grpc.insecure_channel("localhost:8500"))

tf_ex = tf.train.Example(

    features=tf.train.Features(

        feature={

            'curb-weight': tf.train.Feature(float_list=tf.train.FloatList(value=[5.1])),

            'highway-mpg': tf.train.Feature(float_list=tf.train.FloatList(value=[3.3])),

            'body-style': tf.train.Feature(bytes_list=tf.train.BytesList(value=[b"wagon"])),

            'make': tf.train.Feature(bytes_list=tf.train.BytesList(value=[b"Honda"])),

        }

    )

)

request = predict_pb2.PredictRequest()

request.model_spec.name = "regressor_test"

# Tried this:

request.inputs['inputs'].CopyFrom(tf_ex)

# Also tried this:

request.inputs['inputs'].CopyFrom(tf.contrib.util.make_tensor_proto(tf_ex))

# This doesn't work either:

request.input.example_list.examples.extend(tf_ex)

# If it did work, I would like to inference on it like this:

result = self.stub.Predict(request, 10.0)

Thank you for any suggestions of how to correctly do this. I'm hoping to achieve a really generalized robust approach to serving different kinds of models with TF.

- Julian

[Robert Crowe]

You might want to look at this example:

https://www.tensorflow.org/serving/tutorials/Serving_REST_simple

It's REST instead of gRPC, but it might help.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/9bed6b5c-6e7f-4607-b550-1b45e7925ce0%40tensorflow.org.