0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

Compare commits

...

23 commits

Author SHA1 Message Date
erwanvivien
1a26a051a5
zml/tensor: return a distinct tensor from toMemory on single-memory platforms (#1087)
On cpu, neuron and metal, `toMemory` returned self. Donating a result to
a device buffer and its `toMemory` copy to a host buffer then called
`reuseBuffer` twice on the same tensor id: the second donation replaced
the first, and compilation panicked because both outputs aliased the
host buffer. This broke llmd async scheduling on Metal and CPU.

Return the same value under a new tensor id instead.
2026-11-16 14:17:19 +01:00
Raphael Nahoum
8fcff8c462
workspace: bump ZML plugins (#1082)
Bump plugins to

[202611132003.375.1.7d765f57a407](2026111320.375.1.7d765f57a407)

oneAPI: weight-only FP8 and FP4 (NVFP4, MXFP4) scaled dots now run in a
Triton kernel that reads the
weights in their stored format.
2026-11-13 22:27:25 +01:00
Raphael Nahoum
f65072a740
platforms/furiosa: use all RNGD chips by default (#1083) 2026-11-13 22:09:27 +01:00
Benjamin Rabier
9c9161bac6
platforms/oneapi: update pjrt plugin and bundle libmkl_def (#1070)
Update PJRT OneAPI plugin with fixes for Triton
2026-11-13 17:59:02 +01:00
Kevin Barre
d2a69b9718
zml/moe: [refacto] consolidate Furiosa TCL kernel setup (#1066)
Separate dispatch from kernel generation. Use one factory to select BF16
and block-128 FP8 kernels by format and schedule, and reuse the shared
ActivationKind.
2026-11-13 17:34:09 +01:00
Kevin Barre
4c8f520251
zml/moe: support BF16 GEGLU-tanh on Furiosa (#1063)
Pass the requested activation to both BF16 TCL MoE kernels and compute
tanh GELU on the gate before multiplying by the up projection. This lets
Gemma 4 compile and run with its required activation.

Verified both kernel paths against a CPU reference and confirmed
coherent Gemma 4 generation on two and four RNGD cards.
2026-11-13 17:34:09 +01:00
Kevin Barre
3ecf1ff68b
zml/moe: mask nonlocal BF16 routes on Furiosa (#1062)
Replace nonlocal expert IDs with an in-range placeholder and zero their
routing weights before dispatch. This prevents the activation-stationary
and blockwise kernels from using -1 in gathers and scatters.

Verified BF16 outputs against the CPU reference and coherent
Qwen3.5-35B-A3B generation on two and four RNGD cards.
2026-11-13 17:34:08 +01:00
erwanvivien
4af281a0be
stdx/flags: cleanup flag_name using mem.replaceScalar (#1068)
Just porting 5253ea1362 from #1065
2026-11-13 15:38:46 +01:00
Damien Mehala
a0533da7b8
zml/kernel: introduce packages per DSL (#1061)
Split `kernel.zig` into one package per DSL
2026-11-13 14:13:10 +01:00
Hugo Mano
8c41b89801
zml/ops: manualComputation v2 (#1045)
Last PR for manualComputation v2 which promote as default and only one
API for manual computation. Custom calls now require to be wrapped in a
manual computation too.
2026-11-13 13:52:58 +01:00
Damien Mehala
2d45ef0804
zml/platform: fix zml$print leak (#1060) 2026-11-13 13:52:07 +01:00
Corentin Godeau
dc829cfb7d
[Modularization] zml/kernels/moe: replace options.zig with api.zig (#1059)
Also moved `LocalRouting` to `api.zig`
2026-11-13 10:50:15 +01:00
Kevin Barre
c40fa52d2d
zml/moe [Platform:tcl]: support block-128 FP8 SwiGLU experts (#1053)
- Select TCL for plain E4M3FN block-128 experts with matching BF16 or
  F32 scales. Preserve rejection of unsupported quantization.

- Quantize activations per block and apply checkpoint scales during
  native projections. Keep BF16 materialization and F32 down and
  routing accumulation.

- Reuse local expert routing and AllReduce for expert parallelism.
  Choose expert-stationary for small FP8 token counts and use the
  existing activation-stationary and blockwise recipes for larger
  counts.

- Share FP8 and BF16 dispatch preparation while retaining the token
  padding and BF16 SiLU rounding workarounds.
2026-11-12 13:47:48 +01:00
Guillaume Wenzek
8ebed54029
zml: refactor safetensors (#1048)
Refactor `zml/safetensors.zig`

* Drop metadata parsing
* refactor code to be more "object oriented", code is owned by the
`TensorRegistry`
* implement a new format `model.safetensors.full_index.json` that is
similar to `index.json` but actually has data inside it:

```
{
"metadata": {"num_tensors":96085,"total_size":510286023000},
"weight_map": {
  "aligner.w1.bias": {"filename":"model-00001-of-00048.safetensors","shape":[5120],"data_offsets":[27384,37624],"dtype":"BF16"},
  "aligner.w1.weight": {"filename":"model-00001-of-00048.safetensors","shape":[5120,9216],"data_offsets":[37624,94409464],"dtype":"BF16"},
  "aligner.w2.bias": {"filename":"model-00001-of-00048.safetensors","shape":[5120],"data_offsets":[94409464,94419704],"dtype":"BF16"},
  "aligner.w2.weight": {"filename":"model-00001-of-00048.safetensors","shape":[5120,5120],"data_offsets":[94419704,146848504],"dtype":"BF16"},
  "vision.blocks.0.attn.wo.bias": {"filename":"model-00001-of-00048.safetensors","shape":[1024],"data_offsets":[146848504,146850552],"dtype":"BF16"},
  "vision.blocks.0.attn.wo.weight": {"filename":"model-00001-of-00048.safetensors","shape":[1024,1024],"data_offsets":[146850552,148947704],"dtype":"BF16"},
  "vision.blocks.0.attn.wqkv.bias": {"filename":"model-00001-of-00048.safetensors","shape":[3072],"data_offsets":[148947704,148953848],"dtype":"BF16"},
 ...
 ```
 
 I followed conventions of `model.saftensors.index.json`
2026-11-12 13:12:25 +01:00
Raphael Nahoum
2b604ebfc6
zml/kernels/flash_attention/metal: port the kernels to the Metal DSL (#1054)
Replaces the `zml$flash_attn` custom call with Metal DSL kernels over
contiguous K/V:

- `fa_vec` + `fa_vec_reduce` for bf16 decode, at most 256 threads per
threadgroup.
- `fa_tiled` for prefill and f16.
- Batched queries work (the old path panicked). Other dtypes and head
dims fall back to sdpa.

Bumps the plugins to
[202611101651.363.1.d46c7471c419](2026111016.363.1.d46c7471c419)
2026-11-10 18:51:08 +01:00
Raphael Nahoum
9f8da93eaf
zml/kernels/paged_attention/metal: fit decode in 256 threads and split it over the GPU cores (#1046)
Fixes Metal decode on M1/M2: `fa_vec_paged` launched up to 512 threads
per threadgroup, over the 384 those GPUs accept.

- A threadgroup holds up to 4 query heads of one KV head times position
splits, at most 256 threads.
- The split count follows the PJRT core count and the number of rows.
- Head dims 32 and 64 now use the vector kernel instead of the tiled
one.

llmd decode on M5 Max vs master: no change at short context, +4–27% at
8K, +12–74% at 24K, bs16 unchanged.
Head-dim-64 models: 2–7.6× at long context.

Should fix https://github.com/zml/zml/issues/1037 but needs some testing
on a real M1/M2
2026-11-10 18:51:08 +01:00
Guillaume Wenzek
6ee4f84ea7
zml: Exe.memoryStats (#1052) 2026-11-10 14:24:41 +01:00
Louis Lechevalier
b61eede864
zml/kernels: Enable MegaMoe support in llmd (#1032)
This PR enable the MegaMoe kernel support in llmd by :
- Making the symmetric workspace an arg of CuteOps
- Adding cuda.grid_constant arg to the DSL builder, enabling GPU kernels
to receive TMA tensormap
- Adapting the existing mxfp4 kernel (non mega) to the new shared
swizzling `interleaved_8`
2026-11-10 14:23:27 +01:00
Raphael Nahoum
7b40b1b0f8
zml/io: honor slices and splat in DirectMemoryWriter.drain (#1050)
`DirectMemoryWriter.drain` only wrote the first slice of data and
ignored the splat count. The `std.Io.Writer` contract is to write every
slice in order, then repeat the last one splat times.

- A one-byte splat advanced one byte per drain call.
- After a partial write, `splatBytesAll` passes an empty first slice
followed by the pattern. The old drain returned 0 for that forever, so
the fill never finished.

`drain` now writes every slice and repeats the last one splat times.
2026-11-09 20:48:45 +01:00
Corentin Godeau
b23dbe334a
[Modularization] zml/kernels/moe: break cycle between flashinfer_cutlass and platform (#1042)
* Stopped using `zml.ops.CustomCall` because it creates cycles
impossible to break.
* renamed some `cutlass_flashinfer` to `flashinfer_cutlass` for
consistency

I didn't replace `CustomCall` because flashinfer_cutlass is the last
user and will be removed soon.
2026-11-09 19:24:38 +01:00
Corentin Godeau
2c57959202
[Modularization] zml/kernels/moe: break cycle (#1041)
`zml/kernels/moe.zig` is not imported by backend implementation anymore.

I introduced `options.zig` to contain the shared options. I also
introduced `weights.zig` to expose the unpacking helpers.
2026-11-09 19:24:38 +01:00
Corentin Godeau
b73de969eb
[Modularization] zml: break cycle in zml/kernels/paged_attention (#1039) 2026-11-09 19:24:37 +01:00
Corentin Godeau
7fbd15eee4
[Modularization] zml: break cycle in zml/attention (#1038)
Break cycle between `zml/attention/attention.zig` and
`zml/kernels/flash_attention/flash_attention.zig`
2026-11-09 19:24:37 +01:00
97 changed files with 7031 additions and 4957 deletions