CUDA HAL refresh update

[Lei Zhang]

Hello everyone,

Just a heads up that since mid last year, we started to refresh the CUDA HAL driver, see https://github.com/openxla/iree/issues/13245 for details regarding the needs and goals.

Now we have completed the impl and moved it to the default hal/drivers directory with https://github.com/openxla/iree/pull/14620. Next steps would be start replacing the v1 cuda HAL driver concretely:

* Replace various tests/benchmarks in-tree to use cuda2 and address potential issues

* And after being stable, delete cuda and rename cuda2 as cuda

I'll give updates along the way about the above.

cuda2 improves lots of aspects of the current cuda impl and we believe it should be a strict improvement, esp regarding async behavior and graph usage. It's needed for the long-term direction of IREE; and having a solid foundation right now is important. Hopefully this won't be too disruptive; but certainly let us know if you have any future questions or issues. :)

Thanks,
Lei

[Jacques Pienaar]

Great news so early in new year 🙂

--
You received this message because you are subscribed to the Google Groups "iree-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iree-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iree-discuss/CAEA22XN9P%2BFDGAUuxuB%3D-TFp_AzaDii1sd8nGMr2gMoe5xS_jA%40mail.gmail.com.

[Lei Zhang]

Update on this--I've landed https://github.com/openxla/iree/pull/16107 to make cuda2 impl on and cuda1 impl off by default. So from now on when you use the cuda hal driver you get the new one. All tests are passing and for benchmarks, just two minor regressions. I'd wait for some extra time, and then start deleting the existing cuda1 code and rename various cuda2 symbols back to cuda.

Thanks,
Lei