New Dataset API in Serving

[Alex Egg]

I have a pre-processing pipeline setup for my model using the new Datasets API which works great during training. However, I would also like to utilize this same pipeline suing inference too. However, it seems difficult to wire this up using TensorFlow serving. Namely, I can't figure out how to get Serving to initialize the Iterator. I was able to partially retrofit it using the legacy_init_op parameter:

            iterator = Iterator.from_structure(dataset.output_types, dataset.output_shapes)

            next_example = iterator.get_next()

            inferance_init_op = iterator.make_initializer(dataset)

            ...

        #need to add an op here to init the dataset iterator

        legacy_init_op = self.iterator_init_op

        

        with self.session as sess:

            _builder.add_meta_graph_and_variables(

            sess, [tf.saved_model.tag_constants.SERVING],

            signature_def_map={signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: prediction_signature}, 

            clear_devices=True,

            legacy_init_op=legacy_init_op)

However, I get this error b/c there is no data in the pipeline yet: 

AbortionError: AbortionError(code=StatusCode.FAILED_PRECONDITION, details="GetNext() failed because the iterator has not been initialized. Ensure that you have run the initializer operation for this iterator before getting the next element.
	 [[Node: IteratorGetNext = IteratorGetNext[_output_shapes=[[?,224,224,3]], output_shapes=[[?,224,224,3]], output_types=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](Iterator)]]")

I did see something in master called `tf.contrib.data.get_single_element` which might be what I'm looking for, but it's not released yet.

What is the pattern for using DataSet API in serving?

[Hala Neji]

can you send me the file of the network alexnet.py please 

[Alex Egg]

I'm not using alexnet, but here is my code that creates the graph:

https://gist.github.com/eggie5/1f42bb6364e5b4bd98b419ddbf2a0f45

[Alex Egg]

mystery solved: https://github.com/tensorflow/tensorflow/issues/14143

[wu wei]

Hi, have u solved this problem yet? I also want to use data iterator init in tensorflow serving.

在 2017年10月31日星期二 UTC+8上午2:29:51，Alex Egg写道：

[Lili Ghosh]

If you want to refresh your mood find the best Delhi Escorts at our site.

[Alex Egg]

Yes,

Upgrade your TF version to include this fix: https://github.com/tensorflow/tensorflow/commit/af8a5507937108a41781ba117fa16edd3b1091b5

or set clear_devices to `False`

[gan...@gmail.com]

Hi, Alex. 

is this working with latest tensorflow 1.11? I replicated your gist in my code. 

It saves model successfully.  However when I try to start serving server it throws error

Failed to start server. Error: Unknown: 1 servable(s) did not become available: {{{name: ppredix version: 1539165545311} due to error: Invalid argument: You must feed a value for placeholder tensor 'src_placeholder' with dtype float and shape [?,?,10]

[[{{node src_placeholder}} = Placeholder[_output_shapes=[[?,?,10]], dtype=DT_FLOAT, shape=[?,?,10], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]}, }

my export function

https://gist.github.com/ganni/1981f954d349941b75bea5f7a534a6c0