Stop tracking VMVX benchmarks in IREE benchmark CI

[Che-Yu Wu]

Hi all,

Hanhan and I came up with the idea to disable benchmarks with VMVX backend in IREE benchmark CI to save benchmark time on mobile phones, as we think currently we are not actively looking into those performances. The test run shows that ~20mins (49mins vs. 1h12mins) can be saved by disabling those benchmarks.

So we want to check if everyone is fine with this decision? Or is there still a need to track those numbers?

Alternatively, we can move the VMVX benchmarks to run on x86_64 instead of mobile phones, where we have more resources. 

Best,

CheYu Wu

[Jacques Pienaar]

That makes sense to me. We aren't near actually using for performance on mobile so SGTM.

-- Jacques 

--
You received this message because you are subscribed to the Google Groups "iree-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iree-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iree-discuss/0cf073a2-3aac-4f58-979e-54ed8f70dc2en%40googlegroups.com.

[Che-Yu Wu]

After the discussion on discord, we moved VMVX benchmarks to run on x86_64 runners: https://github.com/openxla/iree/pull/15969

Best,

CheYu Wu