0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

1,078 commits

Author SHA1 Message Date
Guillaume Wenzek
5a18757368
zig 0.17 (#693)
Bump for zig 0.17

Deps:
- uucode bumped to
1fb73433bb
- zio bumped to:
417757935e
- rules_zig upgraded to [version
0.17](https://github.com/hermeticbuild/rules_zig/releases/tag/v0.17.0)

There is one Codex generated patch:
- zigimg: minor fixes to allow zig 0.17 to compile it, until they
officially support
2026-10-07 11:08:06 +02:00
Tristan Gautié
948f21c26c
zml/attention: introduce new AttentionMask entry (#904)
Follow-up to https://github.com/zml/zml/pull/888:

We introduce a new `.standard` entry to `AttentionMask` enum, that
accepts both flags `causal` and `sliding_window`. This makes the
interface more flexible, to allow cases with both non-causal _and_
sliding_window options.
2026-10-06 18:07:42 +02:00
Guillaume Wenzek
857c59d331
fix: remove debug log (#906)
of course
2026-10-06 17:29:50 +02:00
Guillaume Wenzek
b2f0e03043
zml/Tensor: explicit sharding in Tensor code (#868)
In Buffer code we already have explicit sharding, but not for tensor.

I'm adding a `zml.Sharding` to `.withPartitioning` to make it explicit
which mesh we are working with.
To simplify the migration, `.withPartitioning` in Tensor code can use
`.model` syntax to lookup a sharding by name.
This replace the previous logic of `selectSharding(shape)` which would
look up a sharding that matches the axes of the given partitioning which
could yield surprising behavior.

This also modifies how sharding are stored in zml.Shape.
Before it was storing a list of axis names + a tag (2 * 8 * 64bits), now
I'm storing a zml.Sharding (64 bits) and a compact PartitionArray (32
bits) that references the logical axis of the sharding by offset instead
of by name.

There is an impact on the `TensorRegistry` because it also needs to be
aware of the possible shardings, instead of delegating the matching
partition spec -> sharding to loading. This means error will be detected
sooner.

`store.createTensor` accepts `@enumLiteral()` to lookup the relevant
sharding.

This also simplifies `zml.Buffer` because the full sharding is now
stored in the shape, so it doesn't need to be passed independently. It
also means that it will propagate from the tensor who are graph inputs,
to the buffers that are eg inputs.

`Placement` logic also got a bit simpler.
2026-10-06 15:37:22 +02:00
Steeve Morin
3831492aa0 third_party/zio: bump zio=v0.19.0 2026-10-05 18:11:42 +02:00
Kevin Barre
3b70cc23a0
zml/workspace: bump llvm to 0.8.24 (#901)
Bump the hermetic LLVM Bazel dependency from 0.8.9 to 0.8.24
2026-10-05 17:51:01 +02:00
Damien Mehala
108c2d429b
zml/attention: support sparse attention (#892) 2026-10-05 17:23:18 +02:00
Damien Mehala
3b8b05b9d9
zml/attention: unify masking options under AttentionMask (#888)
Replace `is_causal` and `sliding_window` fields in `AttentionOptions`
with a tagged union. This prepares the API for a subsequent commit that
will introduce sparse attention mask. A tagged union makes these
mutually exclusive behaviour explicit.
2026-10-05 17:23:16 +02:00
Benjamin Rabier
177f81f21b
stdx: Add cpu.zig with cpu utilities (#902) 2026-10-05 16:48:54 +02:00
Corentin Godeau
11aceb7430
zml/moe: refactor MoE (#884)
This PR removes a lot of slop in the MoE part of ZML, as well as some
global improvements.

Among the improvements:
* Unify activation functions and parameters across backends
* Unify activation computation across backends
* Replace error bubbling with explicit asserts (because we have no
meanignful way to handle them)
* Add a StableHLO MoE backend
2026-10-05 16:02:38 +02:00
Corentin Godeau
98dc929608
zml/moe: cleanup entrypoint (#865)
Move logic out of the entry point while keeping everything the same
2026-10-05 16:02:37 +02:00
Damien Mehala
daa0c0dca8
zml/quantization: MXFP8 and MXFP4 (#842)
Introduce MXFP8 and MXFP4 quantization functions.

Changes:
   - add `QuantizedInput.reuseBuffer`
2026-10-05 13:52:44 +02:00
Benjamin Rabier
427499e81e
zml/exe: Exe.Runner destroys donated input buffer handles (#896)
Record donated input indices during compilation and destroy their
PJRT_Buffer handles once execution is enqueued for `zml.Exe.Runner`.

I'm not doing this automatically in `call()` because it's a bit
invasive. Not all the code uses the pattern of overwriting an input
buffer like `run(.{cache}, .{&cache})`, but most do. So I'm cleaning it
up on easy path and leave `call` as is for finer control.
2026-10-05 13:52:13 +02:00
Benjamin Rabier
af4e079f34
zml/exe: callOpts/zml.testing.autoCall replaced by eval + remove run|callOpts (#897)
Removing callOpts & runOpts to simplify the API and changing `io` to be
required for API clarity. I'm replacing `zml.testing.autoCall` with
`Exe.eval` which solves most needs in a simpler way IMHO
2026-10-05 13:51:36 +02:00
Kevin Barre
daac9c8833
zml/model: [refactor] Change llama to a whole forward pass (#881) 2026-10-05 13:21:19 +02:00
Benjamin Rabier
699add8fd2
zml/io: fix error handling when a direct shard writer fails to init (#894)
DirectShardWriter.init treated retrieveBuffer errors as unreachable and
leaked its transfer manager and buffer when a later step failed.
DirectMemoryWriter.init counted a shard writer as initialized even when
its init failed, so its errdefer deinitialized an undefined writer.
2026-10-05 11:16:39 +02:00
Benjamin Rabier
8c2e4c6dc9
pjrt,zml/platform: expose devices' NUMA node (#891)
Allows placing all the threads/memory on the right numa nodes. It's
faster for a device to access memory on the same numa node and overall
the it's better to have all threads on the same numa node if all devices
are on the same one.
2026-10-05 11:15:57 +02:00
Benjamin Rabier
773fb8f2f3
platforms/oneapi: update latest pjrt artifact (#883) 2026-10-02 22:30:34 +02:00
Louis Lechevalier
75aa9878e1
mlir/dialects: keep pointerType general in LLVM dialect (#889)
NVPTXAddressSpace is NVPTX specific. In LLVM, addressSpace is defined
for each target (nvptx, amdgpu)
The pointerType is general in llvm dialect by accepting an anytype.
2026-10-02 15:45:47 +02:00
Louis Lechevalier
9e6793c604
mlir/dialects: rename NVPTX specifics in LLVM dialect (#887) 2026-10-02 13:33:21 +02:00
Louis Lechevalier
e2a882077d
zml/attention: CuTe flashmla kernel for sm100, sparse MLA related backend (#882)
- Implement the flashmla kernel for sm100 using the cute dsl in zig.
- Extend the cute dsl with specialized sm100 (tcgen05) ops.
- Introduces a new cute backend which calls the cute kernel.
- Uses a new cache for MLA (latent or quantized).
2026-10-02 10:13:50 +02:00
Olivier Boulant
8cbd3bd38c
platform/metal: support toolchain extraction on Linux x86_64 (#880)
Even if `exec_compatible_with = ["@platforms//os:macos"],` is removed
from `platforms/metal/BUILD.bazel`, fetching the pinned Metal toolchain
on Linux currently fails:
```
ERROR: /home/oboulant/.cache/bazel/_bazel_oboulant/4ea653214b23f6728bdb9f5b40de2f50/external/zml+/bazel/http_aea_archive.bzl:20:13: An error occurred during the fetch of repository 'zml++http_aea_archive+metal_toolchain':
   Traceback (most recent call last):
        File "/home/oboulant/.cache/bazel/_bazel_oboulant/4ea653214b23f6728bdb9f5b40de2f50/external/zml+/bazel/http_aea_archive.bzl", line 43, column 19, in _http_aea_archive_impl
                _require_macos(rctx)
        File "/home/oboulant/.cache/bazel/_bazel_oboulant/4ea653214b23f6728bdb9f5b40de2f50/external/zml+/bazel/http_aea_archive.bzl", line 20, column 13, in _require_macos
                fail("http_aea_archive is macOS-only; current repository OS is %r" % rctx.os.name)
Error in fail: http_aea_archive is macOS-only; current repository OS is "linux"
```

Add checksum-pinned `ipsw` and `7-Zip` binaries for Linux x86_64.
`bazel/http_aea_archive.bzl` now selects the right binaries according to
running host. It preserves the existing macOS extraction path.
Linux ARM64 remains uncovered.

Validation:

```bash
bazel  build \
  # selects compiler binaries from the extracted external repository \
  # filegroup defined in platforms/metal/metal_toolchain.BUILD.bazel \
  @metal_toolchain//:metal_compile_tools \
  # selects the compiler’s runtime files \
  # filegroup also defined in platforms/metal/metal_toolchain.BUILD.bazel \
  @metal_toolchain//:metal_compile_runtime \
  # copies compiler binaries and runtime files into the layout consumed by ZML \
  //platforms/metal:metal_toolchain \
  # selects macOS ARM64 as the build target \
  --platforms=//platforms:macos_arm64 \
  --lockfile_mode=off
```

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2026-10-01 18:41:45 +02:00
Olivier Boulant
676f26a4e6
platforms/cuda: fix libunwind downloads blocking Linux ARM64 build (#875)
The error it fixes:
```bash
 bazel build //llmd:upload_sentry_symbols_linux_amd64 //llmd:upload_sentry_symbols_linux_arm64 --config=remote
...
ERROR: no such package '@@zml++cuda_packages+llvm-libunwind1_linux_arm64//': java.io.IOException: Error downloading [https://apt.llvm.org/bookworm/pool/main/l/llvm-toolchain-23/llvm-libunwind1_23.1.2~++20260916013112+21a77e7bb6bf-1~exp1~20260916133123.74_arm64.deb] to /Users/oboulant/Library/Caches/bazel/_bazel_oboulant/50730f1e2ab11dae9d7f564452062ab5/external/zml++cuda_packages+llvm-libunwind1_linux_arm64/.downloaded.llvm-libunwind1_23.1.2~++20260916013112+21a77e7bb6bf-1~exp1~20260916133123.74_arm64.deb: GET returned 404 Not Found
...
```

The `platforms/cuda/packages.lock.json` was updated with `bazel run
@apt_cuda//:lock`.
2026-09-30 22:28:09 +02:00
Benjamin Rabier
4d6c61f595
platforms/oneapi: update pjrt plugin (#877) 2026-09-30 19:24:34 +02:00
Raphael Nahoum
a34428e05c
workspace: bump plugins (#876) 2026-09-30 18:24:24 +02:00
Corentin Godeau
f37e845672
zml/safetensors: support hf snapshots (#872)
This PR adds support for models downloaded via `hf download` without the
`--local-dir` flag.

This is done by keeping the given path instead of getting it from the
`std.Io.File`. The problem with `File.realpath` is that in the case of
models downloaded from HF, the files points to blobs whose name are hex
strings, thus losing the required file extension. Unfortunately, we rely
on the file extension to detect the kind of repository the path
represents.

I also added `//examples/io:playground` to the `//:completion` target so
that we have completion when working on it.
2026-09-29 18:23:08 +02:00
Damien Mehala
35d5bf9282
zml/tokenizer: remove homemade tokenizer (#870) 2026-09-29 14:37:42 +02:00
Damien Mehala
8392254013
zml/attention: support sink (#864)
Required for Mimo 2.6
2026-09-29 13:57:37 +02:00
Benjamin Rabier
94278ee4ed
zml/quantization: Improve FP8 quantization (#859)
The changed code needs to round up the scale to the next power of two.
The math is:
```
(bits + 0x7fffff) & 0xff800000
```

Since 0x7fffff is \(2^{23}-1\):
- When \(m=0\), adding it fills the fraction bits without carrying into
the exponent.
- When \(m>0\), the addition carries exactly once into the exponent.
- The mask then clears all 23 fraction bits.
2026-09-29 12:45:01 +02:00
Benjamin Rabier
8745ef8b30
Compiler: disable cudnn fusion (#867)
Too many problems for now in XLA:
- https://github.com/zml/zml/pull/863
- a bf16 dot failed

We have fixes incoming for those, but unclear whether it's enough
2026-09-29 12:43:12 +02:00
Damien Mehala
337ce099fc
zml/tokenizer: add function to retrieve the token corresponding using its ID (#851) 2026-09-29 10:29:37 +02:00
Louis Lechevalier
74c590b0b0
zml/moe: Refactoring MoE selection and packing contracts (#862)
Refactoring of the MoE selection and add repacking contract required by
the model inits.
This solve a design problem in the model init :
- Before, backend selection required the resulting MoE after repacking
(dtype), and repacking required to know the selected backend.
- Now, backend selection is based on raw weights loaded from HF
(quantization scheme and dtype). Then it induces the packing.
Add useful structs for packing that are specific to MoE
(`ExpertsLayout`).
2026-09-28 18:53:26 +02:00
Louis Lechevalier
90618b8a7d
zml/ops: Up tensors size to trigger the gemm fusion dot lowering in XLA (#863) 2026-09-28 18:36:44 +02:00
Raphael Nahoum
c6530f08d3
workspace: bump plugins (#857) 2026-09-25 13:52:26 +02:00
Steeve Morin
2397718ca3 zml: enable xla_gpu_cudnn_gemm_fusion_level=2 on cuda 2026-09-25 07:15:52 +00:00
Steeve Morin
94e79b1ea7 third_party/zio: add dynamic page size support
this enables GB300 (notably) to work
2026-09-24 23:03:14 +00:00
Steeve Morin
9b7d8fb58c bazel: rework zig_options to be config_settting compatible
zig_option can now take a `flag` dependency to expose existing flags or
`bool`, `int or `string` for config_setting flags
2026-09-24 22:21:44 +02:00
Benjamin Rabier
cabcdd8f59
platforms/oneapi: Use latest pjrt artifact (#855) 2026-09-24 20:13:04 +02:00
Damien Mehala
90f914ad07
platforms/cuda: use info level instead of warn (#827) 2026-09-24 17:57:54 +02:00
Raphael Nahoum
778a5ab3c4
zml/compiler: enable experimental all-reduce kernels on CUDA and bump plugins (#852)
Enable XLA's experimental all-reduce kernels on CUDA (custom one-shot /
two-shot kernels instead of NCCL)
2026-09-24 17:56:20 +02:00
Louis Lechevalier
e2da790b1c
zml/moe: CuTe MoE mxfp4 backend (#853)
Introduces the CuTe mxfp4 moe backend. The kernels use the cute dsl and
are optimized (and so at the moment constraint to its shapes).
Introduces the cute launch host func and so the associated cuda dialect
in zml.
2026-09-24 13:12:08 +02:00
Guillaume Wenzek
50e68877a6
ci: use bazel --experimental_disk_cache_gc_max_size (#824) 2026-09-24 10:31:19 +02:00
Kevin Barre
58cda1283b
buffer: ensure proper deinitialization of host buffer events (#854) 2026-09-23 23:17:48 +02:00
Raphael Nahoum
1e045ec017
mlir/dialects/cute_ir: carry the DSL compiler's CuTe dialects (#849)
The CuTe dialects no longer build on a git checkout with a private
overlay.

- Every operation, type, attribute and enum of the compiler's cute and
cute_nvgpu dialects is typed, with its documentation, operand
constraints, traits and result inference. Nothing is kept as an opaque
payload, and the compiler's IR round-trips through it unchanged.
- The definitions are generated following `REGENERATING.md`, from the
release's Python bindings and by probing the compiler.
- The C API and the Zig builders follow: atom types take their
parameters instead of a text payload, constrained integers are one type,
and pointers take an address space enum.
2026-09-23 17:54:37 +02:00
Damien Mehala
11d49d8bc5
zml/tokenizer: add normalizer support (#847)
Support normalization from HF JSON and typed normalizer sequences.

Changes:
   - Patch IREE memory ownership and chunk-boundary bugs
   - Add regression tests for IREE bugs.
2026-09-23 16:57:04 +02:00
Guillaume Wenzek
e1356ee4ed
zml: Compiler.pushLoc (#846)
Add explicit debug info in ZML.
Model author can now add `pushLocation` to mark specific sections of the
IR

---------

Co-authored-by: Guillaume Wenzek <gwenzek@users.noreply.github.com>
2026-09-23 00:17:41 +02:00
Hugo Mano
ca9a342377
platforms: update plugins and tune ROCm graph execution (#848)
Update CPU, CUDA, Metal, and ROCm plugin references to release
202609221707.75.1.0cf3c5a71e94.

Default ROCm to one HIP graph stream and one hardware queue per device
to reduce synchronization overhead and throughput variability. Disable
rocprofiler queue interposition by default, while preserving explicit
environment overrides.

Restrict collective command-buffer capture to all-reduce and all-gather.
Other collective types execute outside HIP graphs.
2026-09-22 21:05:06 +02:00
Damien Mehala
71396f389a
zml/moe: fix cutlass expert local routing (#845)
Prevent non local experts to fallback on expert 0.
2026-09-22 16:31:08 +02:00
Corentin Godeau
397aa43e0b
pjrt: properly interpret error when an event is ready (#844)
A ready event can still contain an error that we need to report.
2026-09-21 19:42:46 +02:00
Steeve Morin
d0cf9562a2 zml: enable nccl user buffers by default
With SKIP_TEMP, captured NCCL collectives can retain stale physical
mappings if VMM reclaims their backing under memory pressure, causing
silent corruption.

Enable user buffers to assign color 1 and receive the reclaim exemption:
https://github.com/openxla/xla/pull/46029
2026-09-21 19:26:09 +02:00