Are there any easy way to convert NCHW-PB-TensorFlow model to the NHWC-TFLITE model?

276 views
Skip to first unread message

Alexey Bochkovskiy

unread,
Oct 7, 2020, 6:13:56 PM10/7/20
to TensorFlow Lite
Are there any easy way to convert NCHW-PB-TensorFlow model to the NHWC-TFLITE model, without Python using source code of model?
Without adding extra transpose-layers beofre and after each conv-layer, which slows down inference.

So it can be used for conversion: (NCHW) Pytorch -> ONNX -> PB-TensorFlow -> TFLITE (NHWC)

It seems it doesn't work properly: https://github.com/tensorflow/tensorflow/issues/41608

Karim Nosseir

unread,
Oct 7, 2020, 11:19:51 PM10/7/20
to Alexey Bochkovskiy, TensorFlow Lite
Hey Alexey,

We added recently basic support for NCHW->NHWC in TFLiteConverter. It is basic support (doesn't handle all cases), it starts by adding transposes and then tries to optimize away the extra transpose if possible.
So, if you have a TF graph with NCHW, try converting without manipulating the graph.
If you got hit by an issue, please file an issue on github and add me and we can check what we can do.



--
You received this message because you are subscribed to the Google Groups "TensorFlow Lite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tflite+un...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tflite/6e9a2088-b1ab-4664-8e58-2ce97360e54bn%40tensorflow.org.


--
Regards,
Karim Nosseir

Alexey Bochkovskiy

unread,
Oct 8, 2020, 1:04:32 AM10/8/20
to TensorFlow Lite, Karim Nosseir, TensorFlow Lite, Alexey Bochkovskiy
Hi Karim,
Thanks! It seems to be working well now. I just have to use tf.lite.TFLiteConverter.from_saved_model instead of ./toco.








четверг, 8 октября 2020 г. в 06:19:51 UTC+3, Karim Nosseir:

Karim Nosseir

unread,
Oct 8, 2020, 3:28:55 PM10/8/20
to Alexey Bochkovskiy, TensorFlow Lite
Great. Good to know it worked.
Toco is deprecated and is not maintained and will be removed. Please always use TFLliteConverter.

Thanks
--
Regards,
Karim Nosseir

andrey malyshev

unread,
Nov 25, 2020, 5:51:51 PM11/25/20
to TensorFlow Lite, Karim Nosseir, TensorFlow Lite, Alexey Bochkovskiy
Karim,
> We added recently basic support for NCHW->NHWC in TFLiteConverter. It is basic support (doesn't handle all cases), it starts by adding transposes and then tries to optimize away the extra transpose if possible.
How does it work? I do not see any API in TFLiteConverter regarding changing of the input layout. Does it change it and waterfall new shapes down over the topology or converter just wraps layers which should be executed in NHWC by couple transposes before and after?

Thanks,
Andrey

четверг, 8 октября 2020 г. в 06:19:51 UTC+3, Karim Nosseir:
Hey Alexey,

Karim Nosseir

unread,
Nov 27, 2020, 2:25:35 AM11/27/20
to andrey malyshev, TensorFlow Lite, Alexey Bochkovskiy
Hey Andrey,

No API exposed. This happens silently by default.
As I said it changes the Ops to have NHWC and add transposes. Then it tries to optimize these transposes away.
--
Regards,
Karim Nosseir

chen liu

unread,
Aug 6, 2021, 4:45:18 PMAug 6
to TensorFlow Lite, Karim Nosseir, TensorFlow Lite
Hello Karim,
I have a similar question in this direction with another way around.
I have a trained NCHW model and I want to force tflite keeping NCHW data flow. Are there any tricks I can apply? I know tflite prefers NHWC format for efficiency reasons on mobile devices, but I have custom hardware, which is more friendly to NCHW. So I want to ask if this is possible?
Best regards,
Chen

Jaesung Chung

unread,
Aug 12, 2021, 5:32:43 PMAug 12
to TensorFlow Lite, chenst...@gmail.com, Karim Nosseir, TensorFlow Lite
Hi Chen,

Currently, there is no easy-to-use flag to generate the NCHW format weights in the TFLite converter. However, there is a chance to modify the TFLite MLIR converter to generate such models but looks like it is not a simple task.

If you want that feature, could you file a feature request at TensorFlow GitHub?

Best regards,
Jaesung

Karim Nosseir

unread,
Aug 15, 2021, 11:29:04 AMAug 15
to Jaesung Chung, TensorFlow Lite, chenst...@gmail.com
Hi Chen,

Sorry for the late reply as I am OOO. We don't have an exposed API to do this, please file request as Jaesung mentioned.
But note that, TFLite kernels doesn't know how to handle NCHW weights, do you will need to implement these ops as custom ops or leverage TF (using TF SELECT mode) 
--
Regards,
Karim Nosseir

chen liu

unread,
Aug 26, 2021, 6:28:30 AMAug 26
to TensorFlow Lite, Karim Nosseir, TensorFlow Lite, Jaesung Chung
Hello Karim,
thanks for your reply. It's no problem as I'm OOO for weeks as well. Currently, I'm using tflite results as a reference to test a custom AI hardware. If there is no simple flag to expose NCHW, I could manually move axes. That's okay.

Best regards,
Chen

Karim Nosseir

unread,
Sep 14, 2021, 2:16:08 PMSep 14
to chen liu, TensorFlow Lite, Jaesung Chung
Hi Chen,

There is no flag currently to do this, but if there is a flag to expose it, you will end up with a model that has TF ops not TFLite ops, since TFLite doesn't support NCHW.
--
Regards,
Karim Nosseir
Reply all
Reply to author
Forward
0 new messages