Positioning of iree w.r.t. ONX runtime?

[Do Po]

Hello,

Has someone thought of the positioning of iree w.r.t. ONNX runtime?

They seem to have gathered a serious user and support base, what would be the advantages of using iree?

Best regards,

Dumitru

[Jacques Pienaar]

Does the ONNX runtime support anything but ONNX? I have never used ONNX as it doesn't exist in any of the flows or workloads from any of the folks I've worked with. I think that's probably where most folks start their decision process.

ONNX runtime got started a few years before IREE with Microsoft owner and user here. I think in isolation MS is a serious user already 🙂 Now for how much it is used and in which cases I don't know. Again, no ONNX in anything I've done or groups I've been interacting with for last 9 years in ML infra so didn't have any value for me to evaluate.

I'd say if both suit your purposes, evaluate what each bring*. I think folks criteria is so varied and I'd really not want to do marketing or dissuade others to consider their needs and see what fits in their evaluations. So the question is what do you need from a runtime here? On which devices? What in ONNX runtime satisfies your needs?

-- Jacques 

* Cursory glance seems to indicate ONNX runtime > 7 MB while IREE one is ~30 kB, but I'm guessing they may have default kernels and the like where it depends on if one can use them, if it's customizable to not use them, how easy to extend, does it support async in meaningful way, how can one incorporate codegen, etc. So doing a proper eval would take being diligent here across many platforms and use cases, and that only matters if it supports what one needs to do.

--
You received this message because you are subscribed to the Google Groups "iree-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iree-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iree-discuss/059f6f22-d88d-4c23-8fa3-04ed83b24e5cn%40googlegroups.com.

[flo-th]

Hi Jacques,

To follow up on Dumitru's question, is ONNX format is planned to be supported by IREE import tool ?

Floris

[Renato Golin]

On Thu, 21 Sept 2023 at 08:59, flo-th <floris...@gmail.com> wrote:

Hi Jacques,

To follow up on Dumitru's question, is ONNX format is planned to be supported by IREE import tool ?

Why not just use ONNX-MLIR?

https://github.com/onnx/onnx-mlir

ONNX RT is a whole different beast, implementing many of the same functionality as IREE, I don't think it makes sense to merge the two.

--renato

[Jacques Pienaar]

The closest would be if the ONNX to TOSA (I don't recall a LinAlg or StableHLO path) in onnx-mlir was used and then one used TOSA as an supported input format of IREE. But you'd probably run into ops that can't be converted, much of frontend work is needs based out of practicality.

-- Jacques 

--

You received this message because you are subscribed to the Google Groups "iree-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iree-discuss...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/iree-discuss/CAPH-gfedFA8s3QCycESU5ji2kAQV5T1BrkXp4tPs1gmROC4e3w%40mail.gmail.com.