Rework the libxev epoll backend to play nice with zigcoro based
scheduler.
In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.
Should have a slight performance increase, too.
This PR is a collection of fixes related to CUDA and ROCm sandbox
discovery.
In the process, update the epoll libxev patch which caused 100% CPU on
linux.
While fixing CUDA add `xla_gpu_enable_latency_hiding_scheduler` as per
NVIDIA's recommendations.
Finally, bump cuDNN to 9.6.0 while we're here.
---------
Co-authored-by: Corentin Godeau <corentin@zml.ai>
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.
API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)
Mostly, also, it's pretty fast.