0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00

Compare commits

..

14 commits

Author SHA1 Message Date
erwanvivien
1a26a051a5
zml/tensor: return a distinct tensor from toMemory on single-memory platforms (#1087)
On cpu, neuron and metal, `toMemory` returned self. Donating a result to
a device buffer and its `toMemory` copy to a host buffer then called
`reuseBuffer` twice on the same tensor id: the second donation replaced
the first, and compilation panicked because both outputs aliased the
host buffer. This broke llmd async scheduling on Metal and CPU.

Return the same value under a new tensor id instead.
2026-11-16 14:17:19 +01:00
Raphael Nahoum
8fcff8c462
workspace: bump ZML plugins (#1082)
Bump plugins to

[202611132003.375.1.7d765f57a407](2026111320.375.1.7d765f57a407)

oneAPI: weight-only FP8 and FP4 (NVFP4, MXFP4) scaled dots now run in a
Triton kernel that reads the
weights in their stored format.
2026-11-13 22:27:25 +01:00
Raphael Nahoum
f65072a740
platforms/furiosa: use all RNGD chips by default (#1083) 2026-11-13 22:09:27 +01:00
Benjamin Rabier
9c9161bac6
platforms/oneapi: update pjrt plugin and bundle libmkl_def (#1070)
Update PJRT OneAPI plugin with fixes for Triton
2026-11-13 17:59:02 +01:00
Kevin Barre
d2a69b9718
zml/moe: [refacto] consolidate Furiosa TCL kernel setup (#1066)
Separate dispatch from kernel generation. Use one factory to select BF16
and block-128 FP8 kernels by format and schedule, and reuse the shared
ActivationKind.
2026-11-13 17:34:09 +01:00
Kevin Barre
4c8f520251
zml/moe: support BF16 GEGLU-tanh on Furiosa (#1063)
Pass the requested activation to both BF16 TCL MoE kernels and compute
tanh GELU on the gate before multiplying by the up projection. This lets
Gemma 4 compile and run with its required activation.

Verified both kernel paths against a CPU reference and confirmed
coherent Gemma 4 generation on two and four RNGD cards.
2026-11-13 17:34:09 +01:00
Kevin Barre
3ecf1ff68b
zml/moe: mask nonlocal BF16 routes on Furiosa (#1062)
Replace nonlocal expert IDs with an in-range placeholder and zero their
routing weights before dispatch. This prevents the activation-stationary
and blockwise kernels from using -1 in gathers and scatters.

Verified BF16 outputs against the CPU reference and coherent
Qwen3.5-35B-A3B generation on two and four RNGD cards.
2026-11-13 17:34:08 +01:00
erwanvivien
4af281a0be
stdx/flags: cleanup flag_name using mem.replaceScalar (#1068)
Just porting 5253ea1362 from #1065
2026-11-13 15:38:46 +01:00
Damien Mehala
a0533da7b8
zml/kernel: introduce packages per DSL (#1061)
Split `kernel.zig` into one package per DSL
2026-11-13 14:13:10 +01:00
Hugo Mano
8c41b89801
zml/ops: manualComputation v2 (#1045)
Last PR for manualComputation v2 which promote as default and only one
API for manual computation. Custom calls now require to be wrapped in a
manual computation too.
2026-11-13 13:52:58 +01:00
Damien Mehala
2d45ef0804
zml/platform: fix zml$print leak (#1060) 2026-11-13 13:52:07 +01:00
Corentin Godeau
dc829cfb7d
[Modularization] zml/kernels/moe: replace options.zig with api.zig (#1059)
Also moved `LocalRouting` to `api.zig`
2026-11-13 10:50:15 +01:00
Kevin Barre
c40fa52d2d
zml/moe [Platform:tcl]: support block-128 FP8 SwiGLU experts (#1053)
- Select TCL for plain E4M3FN block-128 experts with matching BF16 or
  F32 scales. Preserve rejection of unsupported quantization.

- Quantize activations per block and apply checkpoint scales during
  native projections. Keep BF16 materialization and F32 down and
  routing accumulation.

- Reuse local expert routing and AllReduce for expert parallelism.
  Choose expert-stationary for small FP8 token counts and use the
  existing activation-stationary and blockwise recipes for larger
  counts.

- Share FP8 and BF16 dispatch preparation while retaining the token
  padding and BF16 SiLU rounding workarounds.
2026-11-12 13:47:48 +01:00
Guillaume Wenzek
8ebed54029
zml: refactor safetensors (#1048)
Refactor `zml/safetensors.zig`

* Drop metadata parsing
* refactor code to be more "object oriented", code is owned by the
`TensorRegistry`
* implement a new format `model.safetensors.full_index.json` that is
similar to `index.json` but actually has data inside it:

```
{
"metadata": {"num_tensors":96085,"total_size":510286023000},
"weight_map": {
  "aligner.w1.bias": {"filename":"model-00001-of-00048.safetensors","shape":[5120],"data_offsets":[27384,37624],"dtype":"BF16"},
  "aligner.w1.weight": {"filename":"model-00001-of-00048.safetensors","shape":[5120,9216],"data_offsets":[37624,94409464],"dtype":"BF16"},
  "aligner.w2.bias": {"filename":"model-00001-of-00048.safetensors","shape":[5120],"data_offsets":[94409464,94419704],"dtype":"BF16"},
  "aligner.w2.weight": {"filename":"model-00001-of-00048.safetensors","shape":[5120,5120],"data_offsets":[94419704,146848504],"dtype":"BF16"},
  "vision.blocks.0.attn.wo.bias": {"filename":"model-00001-of-00048.safetensors","shape":[1024],"data_offsets":[146848504,146850552],"dtype":"BF16"},
  "vision.blocks.0.attn.wo.weight": {"filename":"model-00001-of-00048.safetensors","shape":[1024,1024],"data_offsets":[146850552,148947704],"dtype":"BF16"},
  "vision.blocks.0.attn.wqkv.bias": {"filename":"model-00001-of-00048.safetensors","shape":[3072],"data_offsets":[148947704,148953848],"dtype":"BF16"},
 ...
 ```
 
 I followed conventions of `model.saftensors.index.json`
2026-11-12 13:12:25 +01:00
69 changed files with 3830 additions and 3535 deletions