Does TFRT support multiple CUDA streams?

[Haibin]

If I understand it correctly, native TF only use 1 CUDA stream for computation per device. Is/will TFRT use multiple CUDA streams? 

Thanks,

Haibin 

[Idan Mintz]

Hi Haibin,

TFRT uses a single stream when executing in eager (op by op) mode. In graph execution mode, TFRT exposes kernels that enable multi-stream computation but it is up to the model creator and/or their compiler to do multi-stream assignment. The TF MLIR compiler will be capable of doing stream assignment.

Thanks,

Idan