0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

58 commits

Author SHA1 Message Date
Raphael Nahoum
5b079127bb
workspace: bump ZML plugins (#1034)
Bump plugins to
[202611060925.355.1.2ca211fa5fbd](2026110609.355.1.2ca211fa5fbd)
2026-11-06 10:56:05 +01:00
Raphael Nahoum
3285b7e0f4
zml: FP8 block activations and paged attention query padding on Furiosa and bump plugins (#1033)
- Quantize block FP8 activations to E4M3FN on Furiosa, so scaled dots
run natively. Other block sizes keep BF16 activations and XLA's
fallback.

- Pad paged attention query groups to a power of two with zero queries,
as TCC fails other group sizes. Marked as a workaround.

- Bump to
[202611051630.351.1.48d539621544](2026110516.351.1.48d539621544)
2026-11-05 17:38:31 +01:00
Hugo Mano
47f50667d8
zml/attention: declare manual partitioning in the shared API instead of kernels (#1016)
Move manual regions out of flash attention kernels and use
`manualComputation2` for every shared attention backend.

Also fully replicate the embedding executable outputs in Qwen and Llama.
Partial graph constraints leave their batch and sequence dimensions
open, which is invalid at executable boundaries.

Fix two further blockers exposed by the Qwen run that we forgot to move
recently: softplus comparisons must use the input dtype and ROCm DNN
runfiles must include the MIOpen kernel archives.
2026-11-05 13:00:00 +01:00
Louis Lechevalier
69a6b2bb22
workspace: bump ZML plugins (#1031) 2026-11-05 11:44:39 +01:00
Raphael Nahoum
ca39742f14
zml/platform: bump plugins and preallocate Metal memory again (#1001)
- Turns preallocation back on by default on Metal, as before #979: not
preallocating seems to cause memory issues we haven't looked into yet.
- Bumps the PJRT plugins to `202610290951.309.1.785550677e75`.
2026-10-29 12:52:21 +01:00
Raphael Nahoum
70f1bd4af4
dsl/metal: add the Metal DSL (#989)
Adds a Zig DSL for Metal kernels over the AIR dialect, and moves the
Metal paged attention kernels onto it.

- `mlir/dialects/air`: a `thread` address space. Each op has an
`llvmBuilder` that calls its `air.*` builtin with Apple's attributes,
and the dialect's translation interface builds the `!air.kernel`
metadata, so lowering to LLVM IR is MLIR's own translation. The verifier
rejects what Apple's compiler can't take: bool, scalable and n-D
vectors, buffers of non-Metal scalars, scalar f32 converts, and
`air.kernel` on a declaration.
- `dsl/metal`: an MSL-shaped builder. It emits `air` plus the
`arith`/`cf`/`vector`/`llvm` IR that clang produces for MSL, and covers
MSL's math, SIMD-group, barrier and `simdgroup_matrix` functions.
- `zml.kernel.metal` / `ops.metal`: wrap a kernel into a
`__gpu$xla.gpu.air` custom call. XLA lowers it, compiles it to a
metallib and launches it.
2026-10-29 12:52:20 +01:00
Damien Mehala
aa30110932
platforms/rocm: default rocprofiler log level to error (#992)
By default rocprofiler log level is `warning`, which can produce
unnecessary log output. Now, we override this log level to error, unless
`ROCPROFILER_LOG_LEVEL` environment variable is already set.
2026-10-27 13:01:40 +01:00
Raphael Nahoum
79bcc6af03
workspace: bump plugins (#985)
Update the metal plugin to improve performance under load
2026-10-23 17:08:20 +02:00
Louis Lechevalier
2e7871a802
zml/platforms: bump ZML plugins (#984) 2026-10-23 15:30:55 +02:00
Raphael Nahoum
5d85f0f521
platforms: bump plugins and stop preallocating by default on Metal (#979)
Metal does not need to pre-allocate memory
2026-10-22 14:25:28 +02:00
Louis Lechevalier
94ead7094e
zml/platforms: bump ZML plugins and enable all reduce collective in cmd buffer (#959)
This enable Lamport all reduce collective if compute capabilities matched on CUDA and ROCm
2026-10-19 15:18:23 +02:00
Benjamin Rabier
c36b3f8f1e
platforms: Bump plugins for reduced Thunk Prepare/Initialize overhead (#946)
For DeepSeek 4.1-Flash on 4 GB300
```
|                                    |       current |     new |
|------------------------------------|--------------:|--------:|
| Start of XlaModule → cuGraphLaunch | 1,008 µs      | 132 µs  |
| ...of which on CPU                 | 896 µs        | 70 µs   |
| Prepare + Initialize               | ~790 µs       | ~2 µs   |
```
2026-10-16 13:43:03 +02:00
Hugo Mano
bc9d6c2c8d
workspace: bump ZML plugins and CUDA/ROCm runtimes (#942)
Update XLA plugins to release 202610151602.227.1.e215eb02db73
for CPU, CUDA, ROCm, Metal, and Furiosa.

Updated CUDA to 13.4.3 and ROCm with TheRock 10.1.

Now we use CUDA compatibility libraries and NCCL 2.32.3 through NVIDIA's
APT repositories. Starting with CUDA 13.4's redistribution manifest
omits the generic cuda_compat package
(https://docs.nvidia.com/deploy/cuda-compatibility/why-cuda-compatibility.html).
So we resolve it through NVIDIA's APT repositories instead.
These repositories expose flat package indexes at separate x86_64
and SBSA URLs, so we extend rules_distroless to support flat sources.

For NCCL we leverage the same APT source workflow, replacing
separate cryptic wheel URLs.
2026-10-16 10:31:34 +02:00
Raphael Nahoum
e385d9c272
workspace: bump plugins to fix Metal GDN overflow (#940)
Metal GDN used to overflow on state indexes since the rework.
2026-10-15 14:45:20 +02:00
Raphael Nahoum
cddc9cf607
zml/io: load Furiosa weights through pinned host memory (#928)
Staging blocks are uninitialized `.host_pinned` buffers, and tensors are
sent with the direct writer. The plugin backs `.host_pinned` with SDK
pinned (dma-buf) memory, which the SDK recognizes by address and
transfers without pinning pages on every request.

Pinned memory comes from `/dev/dma_heap/system`, so the loading user
needs access to it.
2026-10-14 19:32:13 +02:00
Hugo Mano
82e800dc5c
workspace: bump ZML plugins (#937)
This introduce the new tiling path support.
2026-10-14 15:18:04 +02:00
Corentin Godeau
c0992073fd
workspace: bump other plugins (#922) 2026-10-09 17:36:28 +02:00
Tristan Gautié
734ec46230
workspace: bump ZML plugins to 202610081055.168.1.56e99704ada4 (#915) 2026-10-08 14:22:40 +02:00
Olivier Boulant
daab01572f
platforms: switch LLVM libunwind packages from 23 to 22 (#907)
`apt.llvm.org` removed some links, breaking downloads from our lockfiles
and resulting in 404.

Switch CPU/CUDA to `llvm-toolchain-bookworm-22` and ROCm to
`llvm-toolchain-jammy-22`, and regenerate the lockfiles.

All test passed on:
- macOS
- 9985wx-5090x4
- gb300-2* 
- mi300

* See reverted commit
[f7eb1cb](https://github.com/zml/zml/pull/907/commits/f7eb1cbe3b715da465bd4e9cba8a2b071499b045)
@hugomano

---

It is a follow up from https://github.com/zml/zml/pull/875
2026-10-07 19:13:33 +02:00
Hugo Mano
b148f0b919
workspace: bump XLA + LLVM + IRs (#903) 2026-10-07 16:33:44 +02:00
Raphael Nahoum
a34428e05c
workspace: bump plugins (#876) 2026-09-30 18:24:24 +02:00
Raphael Nahoum
c6530f08d3
workspace: bump plugins (#857) 2026-09-25 13:52:26 +02:00
Steeve Morin
9b7d8fb58c bazel: rework zig_options to be config_settting compatible
zig_option can now take a `flag` dependency to expose existing flags or
`bool`, `int or `string` for config_setting flags
2026-09-24 22:21:44 +02:00
Raphael Nahoum
778a5ab3c4
zml/compiler: enable experimental all-reduce kernels on CUDA and bump plugins (#852)
Enable XLA's experimental all-reduce kernels on CUDA (custom one-shot /
two-shot kernels instead of NCCL)
2026-09-24 17:56:20 +02:00
Raphael Nahoum
1e045ec017
mlir/dialects/cute_ir: carry the DSL compiler's CuTe dialects (#849)
The CuTe dialects no longer build on a git checkout with a private
overlay.

- Every operation, type, attribute and enum of the compiler's cute and
cute_nvgpu dialects is typed, with its documentation, operand
constraints, traits and result inference. Nothing is kept as an opaque
payload, and the compiler's IR round-trips through it unchanged.
- The definitions are generated following `REGENERATING.md`, from the
release's Python bindings and by probing the compiler.
- The C API and the Zig builders follow: atom types take their
parameters instead of a text payload, constrained integers are one type,
and pointers take an address space enum.
2026-09-23 17:54:37 +02:00
Hugo Mano
ca9a342377
platforms: update plugins and tune ROCm graph execution (#848)
Update CPU, CUDA, Metal, and ROCm plugin references to release
202609221707.75.1.0cf3c5a71e94.

Default ROCm to one HIP graph stream and one hardware queue per device
to reduce synchronization overhead and throughput variability. Disable
rocprofiler queue interposition by default, while preserving explicit
environment overrides.

Restrict collective command-buffer capture to all-reduce and all-gather.
Other collective types execute outside HIP graphs.
2026-09-22 21:05:06 +02:00
Steeve Morin
fc0eb7d878
platforms: bump plugins to 202609181631.57.1.f601577e1ab1 (#837)
Notably, adds LTO
2026-09-18 20:00:56 +02:00
Steeve Morin
df3a43d29f
bazel: add zig_options rule (#820)
Introduce `zig_option` and `zig_options` rules to generate the bazel
equivalent of option modules plugged on bazel flags.

On `string` flags, if the flag has a `allowed_values`, the `string` is
converted into a `enum` in zig code.

Also, port platforms to this and remove the ugly `defines` hack to
comptime test for platforms enablement.

In the process, port `arocc` to the `zig_options`.
2026-09-16 21:50:22 +02:00
Benjamin Rabier
51235e7eb6
zml: block32 FP8 with E8M0 scales, and an explicit Linear output dtype (#811)
Two things, the second one falling out of the first.

**`fp8_block32`**: E4M3FN weights with one scale per 32x32 tile, the
layout
DeepSeek V4.1 ships. `quantizeBlockFp8` takes a block size and a scale
dtype;
the E8M0 path floors amax at 1e-4 and rounds the exponent up, so the
activation
side is MXFP8: one E8M0 scale per row per 32 contracted values. That
pairing is
what XLA's fp8_block_gemv arm needs to hand both scales to
`tt.dot_scaled`
(`Fp8BlockGemvScaleInMma` requires E8M0 on *both* sides); F32 activation
scales
would quietly fall to the per-block multiply. The Triton MoE path gets
the same
scheme at `block_size_k = 32`.

`Linear.forward` also stops rounding the activation to BF16 before
quantizing,
so amax is computed on the input as given.

**An explicit output dtype** on `scaledDot` and `Linear.forward`.
Inference
cannot answer what an FP8 x FP8 product should return: the accumulation
is F32
either way, so BF16 or F32 back is the caller's choice. `forwardScaled`
now
hands the caller's type to `scaledDot` rather than rounding through BF16
and
converting after. Narrow outputs are rejected.

Separately, `Linear.forward` converts the weight and bias to the
activation
dtype. `dotGeneral` asserts both operands share one, which broke the
moment a
model raised its activation to F32 against a BF16 weight; the wrappers
this API
replaced converted for exactly that reason.

**Breaking:** `Linear.forward` takes a second argument and `scaledDot` a
fifth.

---------

Co-authored-by: Raphael Nahoum <raphael@zml.ai>
2026-09-16 18:13:08 +02:00
Hugo Mano
4adb31b4d6
platforms: bump plugins for CPU, ROCm, CUDA and Metal (#812)
Bump to build `202609161226.43.1.b992d049cec2`
2026-09-16 14:53:44 +02:00
Raphael Nahoum
94257d3747
kernels/cute: Zig CuTe DSL and XLA custom call and bump xprof (#810)
Adds a Zig front end for NVIDIA's CuTe DSL, shaped after `cutlass.cute`,
plus the `__gpu$xla.gpu.cute` custom call that compiles and launches it
on CUDA.

- `mlir/dialects/cute_ir`: bindings for the `cute` and `cute_nvgpu`
dialects
- `zml.ops.cute`: emits the custom call with the textual module, `grid`
and `block`.
- `kernels/cute` + `zml.kernel.cute.Kernel`: builder, values, tensors
with coordinate `get`/`set`, thread/block indices, static smem, sync,
`scf`. See `kernels/cute/README.md`.
- bump `xprof-nightly`

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-09-16 14:02:46 +02:00
Corentin Godeau
91c8b31502
platforms: bump plugins for CPU, ROCm, CUDA and Metal (#809)
Bump to `202609151311.37.1.f1b6432d55a0`
2026-09-15 16:07:39 +02:00
Hugo Mano
0ebb0c21a9 platforms: bump plugins for CPU, ROCm, CUDA and Metal (#788) 2026-09-10 14:34:40 +00:00
Benjamin Rabier
fdb6d5cd23
platforms/rocm: Update package.lock.json (#687) 2026-07-29 18:01:59 +02:00
Hugo Mano
3e589c9772
platforms/rocm: bump to rocm-jaxlib-v0.10.2 with TheRock 7.14 (#671)
Linked PR: https://github.com/zml/pjrt-artifacts/pull/96
2026-07-21 16:18:28 +02:00
Corentin Kerisit
076417363a
workspace: add missing loads (#606)
nits
2026-06-27 00:51:55 +02:00
Hugo Mano
13cee57bf2
zml/profiling: unify profiling flow and tracing (#443)
This PR modernizes ZML profiling in four areas:

- replaces the old C++ `xspace_to_perfetto` tool with a native Zig
exporter built on upb
- integrates trace export directly into `zml/profiler.zig`
- introduces a scoped tracing API in `zml.tracer` and wires it through
examples
- adds platform-specific annotation support and Bazel-native profiler
wrappers for CUDA and ROCm

The result is a simpler profiling flow, better large-trace behavior, and
a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
Steeve Morin
b230be2f3f platforms: bump PJRT=manual-2026-04-30T17-13-00Z
notably, this bumps fixes TTIR gen problems now that we switched away
from python based TTIR
2026-04-30 17:34:49 +00:00
elogir
5300905228
platforms/rocm: update amdgpuids URL and bump llvm version (#510)
This PR fixes two issues:

- Update `amdgpu.ids` URL to Gitlab, now that the old link is dead
- Bump LLVM version to `0.7.3`
2026-04-27 19:59:31 +02:00
Corentin Kerisit
241aaf7d3b
platforms/cuda: add arm64 support (#493)
Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-04-22 12:37:48 +02:00
Benjamin Rabier
154a2cb641
workspace/xla: Use rolledback version of pjrt (#491)
Rolling back XLA to versions that work with triton 3.6
2026-04-21 15:06:39 +02:00
Benjamin Rabier
f258280ab8
platforms/cuda+rocm: update pjrt artifact (#487)
Update with the XLA patches for improved parallel compilation.

`//examples/llm` goes from 27s to 8s for llama3.1 8B compilation
2026-04-20 18:24:01 +02:00
Benjamin Rabier
c1a902b131
workspace/xla: update to nightly-2026-04-20 (#485) 2026-04-20 17:16:45 +02:00
Steeve Morin
da99fe93b5 platforms/rocm: bump ROCm=7.2.2 2026-04-17 16:48:08 +00:00
elogir
67b4fbdbb8
bin: Introduce zml-smi (#440)
This PR introduces **zml-smi**, a tool to monitor metrics of any
platform supported by ZML, featuring but not limited to:

- real time `--top` mode
- per process tracking
- responsive layout

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-03-31 12:47:07 +02:00
elogir
4ecdd92cb0
platforms/rocm: merge rocm and mesa amdgpu.ids (#441)
This PR introduces a script to merge `amdgpu.ids` from two sources: Mesa
and ROCm.

Both provide GPU ID-to-name mappings, but each may contain entries the
other lacks. For example, Mesa includes `AMD Radeon 8060S Graphics`
while ROCm does not, and ROCm lists newer GPUs absent from Mesa.

The script merges the two sources together, with ROCm entries taking
priority in case of conflicts.
2026-03-27 14:45:35 +01:00
Hugo Mano
6e7cb90728
zml/platforms/rocm: update to ROCm 7.2.1 with full hermeticity (#439)
Fix ROCm runtime hermeticity in the packaged LLM sandbox by patching transitive ROCm libraries to use zmlxrocm_dlopen and by adding missing libdw/liblzma/libbz2 runtime deps for distroless execution. Also update the ROCm packaging to 7.2.1 and refresh the pinned libpjrt_rocm
2026-03-26 17:52:48 +01:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Benjamin Rabier
3b76703430
workspace: fixes for build and test //... green
Fixes:
- Added `target_compatible_with = ["@platforms//os:linux"],` for neuron
& rocm
- Updated the zig in neuron
- Updated neuron to use Python 3.12, it's at best a mess to handle two
different python version with `build //...`
- renamed `third_party/flashattn/BUILD.bazel` to
`third_party/flashattn/flashattention.BUILD.bazel` as it's not part of
this module.
- Used HTTPS & fallback URL for swig, sometimes we would end up with an
incorrect archive (wrong sha256)
- Using `--@rules_python//python/config_settings:bootstrap_impl=script`
for hermetic Python build, it relied on the system Python otherwise.
- Used a separate pip extension for neuron to avoid the extra pip args
to leak to other installations.
- Fix `zml/testing` `fn autoCall` function

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-03-23 14:55:48 +01:00
Steeve Morin
13c7c4bb33 platforms/rocm: bump PJRT plugin, switch to hipBLASLt instead of triton
faster compile times!

but also +10tok/s on Llama 3.1 8B (125tok/s -> 136tok/s)
2026-03-21 15:30:23 +00:00