Quantization Codes in MLIR TFLite Converter

[Alex Jiang]

Hi all,

May I know the detailed quantization codes (uint 8) in MLIT Converter? For example, how to quantize a conv2d operation?

Thank you so much!

Best,

Alex

[Tei (Taehee) Jeong]

Hello Alex,

MlirQuantizeModel is a python entry point, which calls mlir::lite::QuantizeModel as a MLIR entry point. The passes added contains actual logics for quantization.

  pm.addPass(TFL::CreatePrepareQuantizePass(quant_specs));
  pm.addPass(TFL::CreateQuantizePass(
      verify_numeric, whole_model_verify, legacy_float_scale,
      blocklisted_mlir_op_names, blocklisted_nodes));
  pm.addPass(TFL::CreatePostQuantizePass(/*emit_quant_adaptor_ops=*/true));
  pm.addPass(TFL::CreateOptimizeOpOrderPass());
  pm.addPass(TFL::CreateModifyIONodesPass(input_mlir_type, output_mlir_type));
  if (!blocklisted_ops.empty() || !blocklisted_nodes.empty()) {
    // If the first or final ops are not quantized, remove QDQ.
    pm.addPass(TFL::CreatePostQuantizeRemoveQDQPass());
  }

Each pass has their own share of quantization steps, and especially (Prepare|Post)?QuantizePass are the three major ones responsible for. To inspect each pass in detail, you can use tf-opt to manually run MLIR passes on your calibrated model.

Regards,

Tei

--
You received this message because you are subscribed to the Google Groups "TensorFlow Lite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tflite+un...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tflite/48421912-1a7a-44f3-85d9-2239211abcdbn%40tensorflow.org.