Quantization Codes in MLIR TFLite Converter

14 views
Skip to first unread message

Alex Jiang

unread,
Aug 4, 2021, 8:39:06 PMAug 4
to TensorFlow Lite
Hi all,

May I know the detailed quantization codes (uint 8) in MLIT Converter? For example, how to quantize a conv2d operation?

Thank you so much!

Best,
Alex

Tei (Taehee) Jeong

unread,
Aug 4, 2021, 9:48:58 PMAug 4
to Alex Jiang, TensorFlow Lite
Hello Alex,

MlirQuantizeModel is a python entry point, which calls mlir::lite::QuantizeModel as a MLIR entry point. The passes added contains actual logics for quantization.

  pm.addPass(TFL::CreatePrepareQuantizePass(quant_specs));
  pm.addPass(TFL::CreateQuantizePass(
      verify_numeric, whole_model_verify, legacy_float_scale,
      blocklisted_mlir_op_names, blocklisted_nodes));
  pm.addPass(TFL::CreatePostQuantizePass(/*emit_quant_adaptor_ops=*/true));
  pm.addPass(TFL::CreateOptimizeOpOrderPass());
  pm.addPass(TFL::CreateModifyIONodesPass(input_mlir_type, output_mlir_type));
  if (!blocklisted_ops.empty() || !blocklisted_nodes.empty()) {
    // If the first or final ops are not quantized, remove QDQ.
    pm.addPass(TFL::CreatePostQuantizeRemoveQDQPass());
  }


Each pass has their own share of quantization steps, and especially (Prepare|Post)?QuantizePass are the three major ones responsible for. To inspect each pass in detail, you can use tf-opt to manually run MLIR passes on your calibrated model.

Regards,
Tei

--
You received this message because you are subscribed to the Google Groups "TensorFlow Lite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tflite+un...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tflite/48421912-1a7a-44f3-85d9-2239211abcdbn%40tensorflow.org.
Reply all
Reply to author
Forward
0 new messages