mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
Replaces the `zml$flash_attn` custom call with Metal DSL kernels over
contiguous K/V:
- `fa_vec` + `fa_vec_reduce` for bf16 decode, at most 256 threads per
threadgroup.
- `fa_tiled` for prefill and f16.
- Batched queries work (the old path panicked). Other dtypes and head
dims fall back to sdpa.
Bumps the plugins to
[202611101651.363.1.d46c7471c419](
|
History
|
|
|---|---|---|
| .. | ||
| cpu | ||
| cuda | ||
| furiosa | ||
| metal | ||
| neuron | ||
| oneapi | ||
| rocm | ||
| tpu | ||
| BUILD.bazel | ||
| packages.bzl | ||
| platforms.zig | ||