Feb 28, 2023, 3:12:37 AMFeb 28
Sign in to reply to author
Sign in to forward
You do not have permission to delete messages in this group
Sign in to report message as abuse
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to TensorFlow Lite, Advait Jain, TensorFlow Lite, blaine...@femtosense.ai, YoungSeok Yoon
Thank you for raising the issue.
Although we don't currently have any concrete support plans, we've recently added an experimental 16x8 quantization feature for LSTM, and here's how you could try using it.
[What's Done and Known Limitations]
We had worked with LSTM for int16x8 quantization recently. It uses the MLIR quantizer at C++ level. And we made several unit tests and e2e test for the LSTM int16x8 MLIR quantization. 16x8 MLIR quantizer currently supports unidirectional_sequence_lstm, fully_connected, softmax Ops for 16x8, but because of the MLIR itself, we only support 32-bit bias (not 64-bit).
[How to try it at python level]
There is no python level interface to use the 16x8 MLIR quantizer. You can still try using it by changing your lite.py and run the converter by referring to this PR
As Advait said above, we'd be open to collaborating on 16x8 quantization support and getting community contributions.