0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

44 commits

Author SHA1 Message Date
Raphael Nahoum
5b079127bb
workspace: bump ZML plugins (#1034)
Bump plugins to
[202611060925.355.1.2ca211fa5fbd](2026110609.355.1.2ca211fa5fbd)
2026-11-06 10:56:05 +01:00
Raphael Nahoum
3285b7e0f4
zml: FP8 block activations and paged attention query padding on Furiosa and bump plugins (#1033)
- Quantize block FP8 activations to E4M3FN on Furiosa, so scaled dots
run natively. Other block sizes keep BF16 activations and XLA's
fallback.

- Pad paged attention query groups to a power of two with zero queries,
as TCC fails other group sizes. Marked as a workaround.

- Bump to
[202611051630.351.1.48d539621544](2026110516.351.1.48d539621544)
2026-11-05 17:38:31 +01:00
Louis Lechevalier
69a6b2bb22
workspace: bump ZML plugins (#1031) 2026-11-05 11:44:39 +01:00
Raphael Nahoum
ca39742f14
zml/platform: bump plugins and preallocate Metal memory again (#1001)
- Turns preallocation back on by default on Metal, as before #979: not
preallocating seems to cause memory issues we haven't looked into yet.
- Bumps the PJRT plugins to `202610290951.309.1.785550677e75`.
2026-10-29 12:52:21 +01:00
Raphael Nahoum
70f1bd4af4
dsl/metal: add the Metal DSL (#989)
Adds a Zig DSL for Metal kernels over the AIR dialect, and moves the
Metal paged attention kernels onto it.

- `mlir/dialects/air`: a `thread` address space. Each op has an
`llvmBuilder` that calls its `air.*` builtin with Apple's attributes,
and the dialect's translation interface builds the `!air.kernel`
metadata, so lowering to LLVM IR is MLIR's own translation. The verifier
rejects what Apple's compiler can't take: bool, scalable and n-D
vectors, buffers of non-Metal scalars, scalar f32 converts, and
`air.kernel` on a declaration.
- `dsl/metal`: an MSL-shaped builder. It emits `air` plus the
`arith`/`cf`/`vector`/`llvm` IR that clang produces for MSL, and covers
MSL's math, SIMD-group, barrier and `simdgroup_matrix` functions.
- `zml.kernel.metal` / `ops.metal`: wrap a kernel into a
`__gpu$xla.gpu.air` custom call. XLA lowers it, compiles it to a
metallib and launches it.
2026-10-29 12:52:20 +01:00
Raphael Nahoum
79bcc6af03
workspace: bump plugins (#985)
Update the metal plugin to improve performance under load
2026-10-23 17:08:20 +02:00
Louis Lechevalier
2e7871a802
zml/platforms: bump ZML plugins (#984) 2026-10-23 15:30:55 +02:00
Raphael Nahoum
5d85f0f521
platforms: bump plugins and stop preallocating by default on Metal (#979)
Metal does not need to pre-allocate memory
2026-10-22 14:25:28 +02:00
Louis Lechevalier
94ead7094e
zml/platforms: bump ZML plugins and enable all reduce collective in cmd buffer (#959)
This enable Lamport all reduce collective if compute capabilities matched on CUDA and ROCm
2026-10-19 15:18:23 +02:00
Benjamin Rabier
c36b3f8f1e
platforms: Bump plugins for reduced Thunk Prepare/Initialize overhead (#946)
For DeepSeek 4.1-Flash on 4 GB300
```
|                                    |       current |     new |
|------------------------------------|--------------:|--------:|
| Start of XlaModule → cuGraphLaunch | 1,008 µs      | 132 µs  |
| ...of which on CPU                 | 896 µs        | 70 µs   |
| Prepare + Initialize               | ~790 µs       | ~2 µs   |
```
2026-10-16 13:43:03 +02:00
Hugo Mano
bc9d6c2c8d
workspace: bump ZML plugins and CUDA/ROCm runtimes (#942)
Update XLA plugins to release 202610151602.227.1.e215eb02db73
for CPU, CUDA, ROCm, Metal, and Furiosa.

Updated CUDA to 13.4.3 and ROCm with TheRock 10.1.

Now we use CUDA compatibility libraries and NCCL 2.32.3 through NVIDIA's
APT repositories. Starting with CUDA 13.4's redistribution manifest
omits the generic cuda_compat package
(https://docs.nvidia.com/deploy/cuda-compatibility/why-cuda-compatibility.html).
So we resolve it through NVIDIA's APT repositories instead.
These repositories expose flat package indexes at separate x86_64
and SBSA URLs, so we extend rules_distroless to support flat sources.

For NCCL we leverage the same APT source workflow, replacing
separate cryptic wheel URLs.
2026-10-16 10:31:34 +02:00
Raphael Nahoum
e385d9c272
workspace: bump plugins to fix Metal GDN overflow (#940)
Metal GDN used to overflow on state indexes since the rework.
2026-10-15 14:45:20 +02:00
Raphael Nahoum
cddc9cf607
zml/io: load Furiosa weights through pinned host memory (#928)
Staging blocks are uninitialized `.host_pinned` buffers, and tensors are
sent with the direct writer. The plugin backs `.host_pinned` with SDK
pinned (dma-buf) memory, which the SDK recognizes by address and
transfers without pinning pages on every request.

Pinned memory comes from `/dev/dma_heap/system`, so the loading user
needs access to it.
2026-10-14 19:32:13 +02:00
Hugo Mano
82e800dc5c
workspace: bump ZML plugins (#937)
This introduce the new tiling path support.
2026-10-14 15:18:04 +02:00
Corentin Godeau
c0992073fd
workspace: bump other plugins (#922) 2026-10-09 17:36:28 +02:00
Tristan Gautié
734ec46230
workspace: bump ZML plugins to 202610081055.168.1.56e99704ada4 (#915) 2026-10-08 14:22:40 +02:00
Olivier Boulant
daab01572f
platforms: switch LLVM libunwind packages from 23 to 22 (#907)
`apt.llvm.org` removed some links, breaking downloads from our lockfiles
and resulting in 404.

Switch CPU/CUDA to `llvm-toolchain-bookworm-22` and ROCm to
`llvm-toolchain-jammy-22`, and regenerate the lockfiles.

All test passed on:
- macOS
- 9985wx-5090x4
- gb300-2* 
- mi300

* See reverted commit
[f7eb1cb](https://github.com/zml/zml/pull/907/commits/f7eb1cbe3b715da465bd4e9cba8a2b071499b045)
@hugomano

---

It is a follow up from https://github.com/zml/zml/pull/875
2026-10-07 19:13:33 +02:00
Hugo Mano
b148f0b919
workspace: bump XLA + LLVM + IRs (#903) 2026-10-07 16:33:44 +02:00
Raphael Nahoum
a34428e05c
workspace: bump plugins (#876) 2026-09-30 18:24:24 +02:00
Raphael Nahoum
c6530f08d3
workspace: bump plugins (#857) 2026-09-25 13:52:26 +02:00
Steeve Morin
9b7d8fb58c bazel: rework zig_options to be config_settting compatible
zig_option can now take a `flag` dependency to expose existing flags or
`bool`, `int or `string` for config_setting flags
2026-09-24 22:21:44 +02:00
Raphael Nahoum
778a5ab3c4
zml/compiler: enable experimental all-reduce kernels on CUDA and bump plugins (#852)
Enable XLA's experimental all-reduce kernels on CUDA (custom one-shot /
two-shot kernels instead of NCCL)
2026-09-24 17:56:20 +02:00
Raphael Nahoum
1e045ec017
mlir/dialects/cute_ir: carry the DSL compiler's CuTe dialects (#849)
The CuTe dialects no longer build on a git checkout with a private
overlay.

- Every operation, type, attribute and enum of the compiler's cute and
cute_nvgpu dialects is typed, with its documentation, operand
constraints, traits and result inference. Nothing is kept as an opaque
payload, and the compiler's IR round-trips through it unchanged.
- The definitions are generated following `REGENERATING.md`, from the
release's Python bindings and by probing the compiler.
- The C API and the Zig builders follow: atom types take their
parameters instead of a text payload, constrained integers are one type,
and pointers take an address space enum.
2026-09-23 17:54:37 +02:00
Hugo Mano
ca9a342377
platforms: update plugins and tune ROCm graph execution (#848)
Update CPU, CUDA, Metal, and ROCm plugin references to release
202609221707.75.1.0cf3c5a71e94.

Default ROCm to one HIP graph stream and one hardware queue per device
to reduce synchronization overhead and throughput variability. Disable
rocprofiler queue interposition by default, while preserving explicit
environment overrides.

Restrict collective command-buffer capture to all-reduce and all-gather.
Other collective types execute outside HIP graphs.
2026-09-22 21:05:06 +02:00
Steeve Morin
fc0eb7d878
platforms: bump plugins to 202609181631.57.1.f601577e1ab1 (#837)
Notably, adds LTO
2026-09-18 20:00:56 +02:00
Steeve Morin
df3a43d29f
bazel: add zig_options rule (#820)
Introduce `zig_option` and `zig_options` rules to generate the bazel
equivalent of option modules plugged on bazel flags.

On `string` flags, if the flag has a `allowed_values`, the `string` is
converted into a `enum` in zig code.

Also, port platforms to this and remove the ugly `defines` hack to
comptime test for platforms enablement.

In the process, port `arocc` to the `zig_options`.
2026-09-16 21:50:22 +02:00
Benjamin Rabier
51235e7eb6
zml: block32 FP8 with E8M0 scales, and an explicit Linear output dtype (#811)
Two things, the second one falling out of the first.

**`fp8_block32`**: E4M3FN weights with one scale per 32x32 tile, the
layout
DeepSeek V4.1 ships. `quantizeBlockFp8` takes a block size and a scale
dtype;
the E8M0 path floors amax at 1e-4 and rounds the exponent up, so the
activation
side is MXFP8: one E8M0 scale per row per 32 contracted values. That
pairing is
what XLA's fp8_block_gemv arm needs to hand both scales to
`tt.dot_scaled`
(`Fp8BlockGemvScaleInMma` requires E8M0 on *both* sides); F32 activation
scales
would quietly fall to the per-block multiply. The Triton MoE path gets
the same
scheme at `block_size_k = 32`.

`Linear.forward` also stops rounding the activation to BF16 before
quantizing,
so amax is computed on the input as given.

**An explicit output dtype** on `scaledDot` and `Linear.forward`.
Inference
cannot answer what an FP8 x FP8 product should return: the accumulation
is F32
either way, so BF16 or F32 back is the caller's choice. `forwardScaled`
now
hands the caller's type to `scaledDot` rather than rounding through BF16
and
converting after. Narrow outputs are rejected.

Separately, `Linear.forward` converts the weight and bias to the
activation
dtype. `dotGeneral` asserts both operands share one, which broke the
moment a
model raised its activation to F32 against a BF16 weight; the wrappers
this API
replaced converted for exactly that reason.

**Breaking:** `Linear.forward` takes a second argument and `scaledDot` a
fifth.

---------

Co-authored-by: Raphael Nahoum <raphael@zml.ai>
2026-09-16 18:13:08 +02:00
Raphael Nahoum
ace231388d
platforms/cpu: fix sandbox lib path after plugin layout change (#819)
The CPU plugin archives now ship the shared library under lib/, and
copy_to_directory outputs to sandbox/ instead of sandbox/lib. Look up
the sandbox directory in runfiles and join lib/ when building the
library path, matching metal, rocm and cuda.
2026-09-16 17:48:02 +02:00
Hugo Mano
4adb31b4d6
platforms: bump plugins for CPU, ROCm, CUDA and Metal (#812)
Bump to build `202609161226.43.1.b992d049cec2`
2026-09-16 14:53:44 +02:00
Raphael Nahoum
94257d3747
kernels/cute: Zig CuTe DSL and XLA custom call and bump xprof (#810)
Adds a Zig front end for NVIDIA's CuTe DSL, shaped after `cutlass.cute`,
plus the `__gpu$xla.gpu.cute` custom call that compiles and launches it
on CUDA.

- `mlir/dialects/cute_ir`: bindings for the `cute` and `cute_nvgpu`
dialects
- `zml.ops.cute`: emits the custom call with the textual module, `grid`
and `block`.
- `kernels/cute` + `zml.kernel.cute.Kernel`: builder, values, tensors
with coordinate `get`/`set`, thread/block indices, static smem, sync,
`scf`. See `kernels/cute/README.md`.
- bump `xprof-nightly`

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-09-16 14:02:46 +02:00
Corentin Godeau
91c8b31502
platforms: bump plugins for CPU, ROCm, CUDA and Metal (#809)
Bump to `202609151311.37.1.f1b6432d55a0`
2026-09-15 16:07:39 +02:00
Hugo Mano
0ebb0c21a9 platforms: bump plugins for CPU, ROCm, CUDA and Metal (#788) 2026-09-10 14:34:40 +00:00
Hugo Mano
833f42029d
platforms: bump cpu + cuda plugins and align xla (#626)
Related PR: https://github.com/zml/monorepo/pull/304
2026-07-03 14:22:23 +02:00
Steeve Morin
b230be2f3f platforms: bump PJRT=manual-2026-04-30T17-13-00Z
notably, this bumps fixes TTIR gen problems now that we switched away
from python based TTIR
2026-04-30 17:34:49 +00:00
Benjamin Rabier
e79311ccf4
platforms/cpu: remove sha256: prefix from the pjrt artifact's sh256 (#499) 2026-04-23 11:17:40 +02:00
Benjamin Rabier
154a2cb641
workspace/xla: Use rolledback version of pjrt (#491)
Rolling back XLA to versions that work with triton 3.6
2026-04-21 15:06:39 +02:00
Benjamin Rabier
c1a902b131
workspace/xla: update to nightly-2026-04-20 (#485) 2026-04-20 17:16:45 +02:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Steeve Morin
06f13f9a22 platforms: uncompress http_archives using http_bsdtar_archive
Roughly a 2x speedup according to @cerisier. So take it for what it is.
2026-03-20 16:22:26 +00:00
Benjamin Rabier
a011fb0e52
Green CI (#419)
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
2026-03-20 10:31:57 +01:00
Steeve Morin
dcbf5f2e27 workspace: update to 0.16.0-dev.2611+f996d2866, add remote exec
also:
- use toolchains_llvm_bootstrapped for macOS builds
2026-02-16 12:05:01 +01:00
Steeve Morin
1b644887b7 bazel,platforms: centralize runfiles loading
runfiles are process-wide in space and time.
so it makes sense to keep them around

because they are bound to the process, we use the smp_allocator and
except the memory to be kept around during the lifetime of the process.
2026-01-19 18:14:06 +01:00
Steeve Morin
21a7c5fec8 platforms: pass allocator to load 2026-01-19 17:41:57 +01:00
Steeve Morin
9e2c09066b runtimes: rename to platforms 2026-01-19 17:41:38 +01:00