0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

91 commits

Author SHA1 Message Date
Benjamin Rabier
86ab457d0b
dsl/cute: add SM100 primitives and CTA-group support (#1007)
Add helpers for TMA gather/prefetch, weight-stationary MMA,
shared-memory descriptors, packed FP32 arithmetic, async cluster stores,
and register limits.

Make CTA groups and barrier scopes explicit across TMA, block-scaled
MMA, TMEM, and synchronization, including two-CTA operations and
multicast completion. Consolidate named CTA barriers and update the MoE
and paged-attention callers.

Cover instruction operands, descriptor encoding, and MLIR round-trips in
builder tests.
2026-11-03 10:26:31 +01:00
Raphael Nahoum
70f1bd4af4
dsl/metal: add the Metal DSL (#989)
Adds a Zig DSL for Metal kernels over the AIR dialect, and moves the
Metal paged attention kernels onto it.

- `mlir/dialects/air`: a `thread` address space. Each op has an
`llvmBuilder` that calls its `air.*` builtin with Apple's attributes,
and the dialect's translation interface builds the `!air.kernel`
metadata, so lowering to LLVM IR is MLIR's own translation. The verifier
rejects what Apple's compiler can't take: bool, scalable and n-D
vectors, buffers of non-Metal scalars, scalar f32 converts, and
`air.kernel` on a declaration.
- `dsl/metal`: an MSL-shaped builder. It emits `air` plus the
`arith`/`cf`/`vector`/`llvm` IR that clang produces for MSL, and covers
MSL's math, SIMD-group, barrier and `simdgroup_matrix` functions.
- `zml.kernel.metal` / `ops.metal`: wrap a kernel into a
`__gpu$xla.gpu.air` custom call. XLA lowers it, compiles it to a
metallib and launches it.
2026-10-29 12:52:20 +01:00
Raphael Nahoum
b37ba31500
mlir/dialects: build every remaining op and attribute through the dialect packages (#993)
The DSLs and the kernels no longer make ops or set dialect attributes by
name: each dialect package gets the builder or helper they were missing,
and every consumer goes through it. This now fix ZML core files as well
2026-10-27 17:53:31 +01:00
Raphael Nahoum
b71982fda7
mlir/dialects: add AIR dialect (#988)
Adds Apple's AIR dialect, for writing Metal kernels in MLIR.

- `mlir/dialects/air`: the AIR that Apple's Metal compiler emits for
MSL. Each op is named after its `air.*` intrinsic without the type
suffix (`fast_exp`, `max.u`, `simd_shuffle_xor`, `wg.barrier`,
`simdgroup_matrix_8x8_load`, ...), with fastmath flags on float ops.
Kernel arguments are described the way that compiler describes them:
builtins, buffers with access/noalias, and `air.arg_name`.
- `llvm`, `arith`: bind array and named struct types, linkage, loop
annotations, and integer overflow flags.
2026-10-26 11:41:06 +01:00
Raphael Nahoum
c74d5ff637
mlir/dialects: add TCL dialect and DSL (#986)
Support Furiosa's TCL dialect and add the builder as well to write
custom tcl kernels
2026-10-26 11:41:06 +01:00
Raphael Nahoum
3f60e626e4
mlir/dialects: split into one package per dialect and unify them (#982)
Splits the monolithic mlir/dialects library into one Bazel package per
dialect, then makes every package follow the same layout, C API and Zig
API. Each package owns its Zig bindings, its C API when upstream doesn't
provide one, and its tests. Dialect registration moves out of the core
mlir package, so a target only pulls in the dialects it actually uses.
The conventions are documented in `mlir/dialects/README.md`, with a
porting checklist and conformance checks.

Along the way:

- cute_ir is flattened, and its Cuda, CuteNVGPU and NVVM sub-dialects
become their own packages
- fly's ROCDL extension moves to fly_rocdl
- new packages for ROCDL and UB
- kernels/ is renamed to dsl/, since it holds the DSLs
- every package registers through its own registerDialect, and each DSL
through registerDialects, instead of dialect name strings
- the C and C++ follow the LLVM and MLIR C API conventions, formatted
with a new mlir/.clang-format
- BUILD files share one template
- accessors are named as nouns like in the rest of ZML
- each package tests itself in its own test.zig, including that every
bound operation is registered and that every wrapper method compiles

Fixes:

- memref atomic_rmw emitted the wrong kind, because the AtomicRMWKind
values didn't match the pinned LLVM
- the pinned Triton and Mosaic no longer have tt.cat and tpu.repeat, so
the DSL's cat and repeat now lower the way tl.cat and pltpu.repeat do
- the ROCDL bindings are completed: buffer address spaces, vector types,
waves_per_eu range

Breaking changes:

- register dialects through their package instead of
DialectHandle.fromString, and mlir.registerFuncExtensions becomes
func.registerExtensions
- accessors drop the get prefix, so getValue() becomes value()
- ttir.cat and mosaic_tpu.repeat are removed, and stablehlo pad and
mosaic_tpu sort take their options before the location
- zml.kernel.fly.rocdl becomes fly_rocdl
- Mesh.sdyMeshAttr takes a context and returns an attribute instead of
text
- func.func no longer prints public, its default visibility

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-10-26 11:41:05 +01:00
Guillaume Wenzek
7bb457ebeb
zml: refactor manualComputation (#925)
here I'm promoting a pattern I would like to extend to more of ZML:

function that takes `anytype` argument should do as little work as
possible and called a fully typed function.

I kept `manualComputation` signature to not break existing code,
I moved the code to `zml/ops/manual_computation.zig` to better
encapsulate the required helper functions.
(and `zml/ops.zig` is pretty big atm)

Similarly `Sharding.Partitioning.parse` now calls `const logical =
comptime Logical.parse(partitioning);` then `logical.applyUseAll(tags,
mesh, default_value);` which is fully typed.

I'm also documenting `manualComputation` better
2026-10-16 12:30:05 +02:00
Guillaume Wenzek
00a0af4a55
zig 0.17 (second attempt) (#923)
Bump for zig 0.17

Deps:
- uucode bumped to
1fb73433bb
- zigimg bumped to
734fa919fb
- libvaxis bumped to
14b7684272
- zio bumped to:
417757935e
- rules_zig upgraded to [version
0.17](https://github.com/hermeticbuild/rules_zig/releases/tag/v0.17.0)
- zls is using a commit from master, but they don't have yet a tagged
release so Codex did a bit of packaging. I confirmed it correctly
autocompletes on my machine
2026-10-13 17:11:38 +02:00
Raphael Nahoum
b774b7ad16
mlir/dialects: bind missing attributes and types through the C API (#905)
Some dialects (i.e. `gpu`) were missing the C bindings, this is now
done.
2026-10-08 10:39:26 +02:00
Hugo Mano
b148f0b919
workspace: bump XLA + LLVM + IRs (#903) 2026-10-07 16:33:44 +02:00
Guillaume Wenzek
303ff3e6a4
Revert "zig 0.17 (#693)" (#909)
This reverts commit 5a18757368.

Our internal repos has a few zig patches, I need to also upgrade them
before we ugprade ZML
2026-10-07 11:34:51 +02:00
Guillaume Wenzek
5a18757368
zig 0.17 (#693)
Bump for zig 0.17

Deps:
- uucode bumped to
1fb73433bb
- zio bumped to:
417757935e
- rules_zig upgraded to [version
0.17](https://github.com/hermeticbuild/rules_zig/releases/tag/v0.17.0)

There is one Codex generated patch:
- zigimg: minor fixes to allow zig 0.17 to compile it, until they
officially support
2026-10-07 11:08:06 +02:00
Louis Lechevalier
75aa9878e1
mlir/dialects: keep pointerType general in LLVM dialect (#889)
NVPTXAddressSpace is NVPTX specific. In LLVM, addressSpace is defined
for each target (nvptx, amdgpu)
The pointerType is general in llvm dialect by accepting an anytype.
2026-10-02 15:45:47 +02:00
Louis Lechevalier
9e6793c604
mlir/dialects: rename NVPTX specifics in LLVM dialect (#887) 2026-10-02 13:33:21 +02:00
Louis Lechevalier
e2a882077d
zml/attention: CuTe flashmla kernel for sm100, sparse MLA related backend (#882)
- Implement the flashmla kernel for sm100 using the cute dsl in zig.
- Extend the cute dsl with specialized sm100 (tcgen05) ops.
- Introduces a new cute backend which calls the cute kernel.
- Uses a new cache for MLA (latent or quantized).
2026-10-02 10:13:50 +02:00
Louis Lechevalier
e2da790b1c
zml/moe: CuTe MoE mxfp4 backend (#853)
Introduces the CuTe mxfp4 moe backend. The kernels use the cute dsl and
are optimized (and so at the moment constraint to its shapes).
Introduces the cute launch host func and so the associated cuda dialect
in zml.
2026-09-24 13:12:08 +02:00
Raphael Nahoum
1e045ec017
mlir/dialects/cute_ir: carry the DSL compiler's CuTe dialects (#849)
The CuTe dialects no longer build on a git checkout with a private
overlay.

- Every operation, type, attribute and enum of the compiler's cute and
cute_nvgpu dialects is typed, with its documentation, operand
constraints, traits and result inference. Nothing is kept as an opaque
payload, and the compiler's IR round-trips through it unchanged.
- The definitions are generated following `REGENERATING.md`, from the
release's Python bindings and by probing the compiler.
- The C API and the Zig builders follow: atom types take their
parameters instead of a text payload, constrained integers are one type,
and pointers take an address space enum.
2026-09-23 17:54:37 +02:00
Guillaume Wenzek
e1356ee4ed
zml: Compiler.pushLoc (#846)
Add explicit debug info in ZML.
Model author can now add `pushLocation` to mark specific sections of the
IR

---------

Co-authored-by: Guillaume Wenzek <gwenzek@users.noreply.github.com>
2026-09-23 00:17:41 +02:00
Raphael Nahoum
94257d3747
kernels/cute: Zig CuTe DSL and XLA custom call and bump xprof (#810)
Adds a Zig front end for NVIDIA's CuTe DSL, shaped after `cutlass.cute`,
plus the `__gpu$xla.gpu.cute` custom call that compiles and launches it
on CUDA.

- `mlir/dialects/cute_ir`: bindings for the `cute` and `cute_nvgpu`
dialects
- `zml.ops.cute`: emits the custom call with the textual module, `grid`
and `block`.
- `kernels/cute` + `zml.kernel.cute.Kernel`: builder, values, tensors
with coordinate `get`/`set`, thread/block indices, static smem, sync,
`scf`. See `kernels/cute/README.md`.
- bump `xprof-nightly`

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-09-16 14:02:46 +02:00
Raphael Nahoum
b78c0f03d7
kernels/fly: FlyDSL Zig DSL (#799)
Adds a Zig front end for the FlyDSL dialect, so AMD GPU kernels can be
written in Zig alongside the existing Mosaic, Triton and tile ones, and
dispatched the same way.

Types and layouts are built through the dialect's own C entry points
rather than by printing and reparsing assembly. That holds in both
directions: host-side layout literals become attributes directly, and
anything the dialect computes is read back through typed accessors.

Launch geometry is derived from the device rather than hard-coded, so
the same kernels run on both AMD architecture families and the tests
stay architecture agnostic. Everything builds and tests on any host, no
GPU needed.

Also carries one unrelated cleanup, dropping a needless null terminator
from the Triton and tile kernel strings.
2026-09-14 09:57:36 +02:00
Raphael Nahoum
77e41879d3
zml/kernel: add a CUDA Tile IR (cuda_tile) Zig DSL (#779)
A third kernel language beside Triton and Mosaic TPU. It emits NVIDIA's
cuda_tile dialect and reaches the GPU through XLA's
`__gpu$xla.gpu.cuda_tile` custom call.

- `third_party/cuda_tile`: cuda-tile v13.3.3
- `mlir/dialects/cuda_tile`: every type and enum through the dialect's C
API, one constructor per op (all 100, in Ops.td order), and a small C
shim for the TileView interface the C API does not expose.
- `kernels/cuda_tile`: views, tokens, for/if/loop, reduce/scan (single
and variadic), atomics, MMA, every elementwise op.
- `zml.kernel.cuda_tile.Kernel` and `ops.cudaTile`: the custom call,
with a grid and no num_warps/num_stages (those are optimization_hints in
the IR).
2026-09-08 14:57:40 +02:00
Damien Mehala
a05d7f920a
mlir/func: support up to 64 results (#731) 2026-08-27 11:13:09 +02:00
Raphael Nahoum
14a0a8f507
zml: start of quantization support (#690)
Add MLIR composites, `scaledDot`, and a Linear path for NVFP4 weights
(unpack, optional activation quant on Blackwell, global scales). Skip
fused multi-source tensors in the generic loader so pack/`loadExecute`
can fill them (may be used later for fused QKV etc).
2026-07-31 11:02:07 +02:00
Guillaume Wenzek
648c5fd2ee
zml/attention: stablehlo paged attention (#688)
Implement paged_attention in pure stablehlo.

introduces `zml.ops.if` and `zml.ops.if2`
The first one is similar to `zml.ops.while` and requires a full struct
with two functions. it's a bit heavy to use, but it follows the
stablehlo API, and if one of the branch has side effects it will
correctly evaluate the side effect only in that branch.

`zml.ops.if2` is a simpler version that doesn't require to create
intermediary struct, but as a consequence, if you have side effect they
will always evaluate, and it relies on the compiler to correctly avoid
duplicated compute by moving relevant compute inside the branches.

(open to feedback  about if/if2)
2026-07-31 10:57:02 +02:00
Hugo Mano
1fead1c014
platforms/neuron: introduce NKI stack with SDPA kernel (#528)
**Summary**

This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.

The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.

On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
loupicaaa
b6e2b026bc
zml/moe: Add sharding for triton MoE (#552)
This PR introduces the sharding as expert parallelism for MoE :

- Introduce a .experts logical sharding (required to express an other
semantic than TP (.model))
- Wrap the whole MoE impl into a manualComputation
- Implement a manual stableHlo all reduce since shardy can not infer
the correct communication after a manualComputation (blackbox)
- Partition qwen 3.5 moe example with TP for the rest of the model (non
MoE)

---------

Co-authored-by: loupicaaa <louis@zml.ai>
2026-05-27 10:13:38 +02:00
elogir
cebc4cebf8
kernels: extend Mosaic TPU + Triton kernel DSL (#545)
This PR changes the following:

- Name-based `output_operand_aliases` for MosaicTPU and Triton DSLs
- New `kernels/mosaic_tpu/pipeline.zig`, a Zig port of `pltpu.emit_pipeline`
- Some fixes to MosaicTPU
2026-05-25 09:41:08 +02:00
Guillaume Wenzek
37ef0a87c0
mlir: fluent api (#533) 2026-05-15 10:53:54 +02:00
Corentin Godeau
3b718a5a89
zml/tensor: add Tensor.uninitialized() (#535)
This PR exposes the XLA-specific custom calls that allow to create
uninitialized tensors (thus not paying the cost of a kernel to
essentially do a memset).

The `stablehlo` part of it was already done, so I added a wrapper in
`tensor.zig`
2026-05-13 10:40:20 +02:00
elogir
c917acb572
kernels/triton: add LLVM dialect, pipelining hints and fixes (#525)
This PR introduces the following:

- Wire up the LLVM dialect so the Triton builder can emit
`llvm.intr.assume` so we can match
[`triton.language.assume`](https://triton-lang.org/main/python-api/generated/triton.language.assume.html#triton.language.assume)
- Thread `tt.num_stages` through `scf.for` for pipelining hints
- Add an inline-return mode to `ReturnIfScope`. Not necessary but better
for IR matching with Python reference
- Build DSL kernels IR in a new MLIR context

Also fixes i1 extension to use `extui` instead of `extsi`.
2026-05-12 14:54:42 +02:00
Guillaume Wenzek
5d94f9b83c
mlir: add shardy dialect (#526)
Add bindings to shardy dialect,
remove the string generation code from zml.Sharding, and replace it with
mlir calls.
2026-05-11 09:37:05 +02:00
elogir
8e4c333007
zml: replace Python Triton/Mosaic IR generation with native Zig DSLs (#517)
Drops the Python build-time dependency for Triton and Mosaic TPU
kernels. ZML now emits TTIR and Mosaic TPU IR directly from Zig, against
the same MLIR context as the rest of the compiler. So no more JAX,
Triton at build time and a lighter sandbox.

Split into two commits:

- **`c72f44884` — Add Triton and Mosaic Zig DSL.** Additive: IR builders
(`kernels/triton/`, `kernels/mosaic_tpu/`), upstream dialect bindings
(`ttir`, `mosaic_tpu`, plus `arith`/`scf`/`math`/`memref`/`vector`/
`affine`/`cf`), and the high-level
`zml.kernel.{triton,mosaic_tpu}.Kernel(Config, spec)` declarative form.
Comes with author guides in each `README.md`.

- **`cb61e529f` — Remove Python and replace with Zig kernels.** Migrates
the two production kernels (`unified_attention` for Triton,
`ragged_paged` for Mosaic TPU) and deletes everything Python related in
the sandbox and what was generating the IR.

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-04-30 16:13:48 +02:00
Guillaume Wenzek
9211a1770e
Remove BoundedArray.init (#506)
When looking at the new Sharding code I was triggered by the high number
of "try".

I then realized most of the try where traced back to `axes = try
.init(0)` which can never fail.
So I removed BoundedArray.init (aka Andrew mode)

It's generally unsafe because it makes `slice()` return undefined.;
I replaced`try init(0)` by `.empty`
I replaced `try init(n)` by `.fromSlice` or with `.{ .buffer =
undefined, .len = n }` to explicitly show the unsafeness

This leads to widespread changes because it means that most of the
sharding operations can't fail anymore.
2026-04-27 10:31:52 +02:00
Benjamin Rabier
70c5d1c588
zml/ops: Stronger typing of custom calls and more code re-use beyond flashattn (#454)
Today, creating a custom call is a bit tedious and error prone. The
primary goal of this PR is to only accept structs as
input/output/attributes and rely more strongly on the names everywhere
when creating as custom call. I've migrated the `fa2` custom to provide
an example. Can migrate the rest.
- Custom call relies on the position of the input/outputs/attributes to
be in the correct order to work. The `output_operand_aliases` also maps
the outputs to the input field position.
- If the custom call is "sharding aware", as in it knows it works on a
single partition on multi devices setup, one must use
`zml.ops.manualComputation` which is a bit tedious to set up.
- `flashattn.zig` has a bunch of logic to register said custom call but
also to map inputs into something more usable which can be shared.
- both `zml.ops.customCall` and `zml.ops.manualComputation` tend to
accept to many different types IMHO which makes it a bit hard to follow.
But well, to be fair not sure how much simpler my code is. I do
constrain myself to only accepting simple structs for the generic types.

I've re-used the existing `manualComputationInternal` and
`customCallInteral` for this new API without touching them. The only
exception is the `output_operand_alises` where I needed a bit more
flexibility.
2026-04-07 15:17:33 +02:00
Hugo Mano
463ca4a762
zml: add zml.ops.while and zml.nn.GatedDeltaNet (#427) 2026-03-24 14:28:34 +01:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Benjamin Rabier
a011fb0e52
Green CI (#419)
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
2026-03-20 10:31:57 +01:00
Corentin Godeau
70e4c9953d
Bump zig to 0.16.0-dev.2722+f16eb18ce (#409)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-03-10 15:58:48 +01:00
Hugo Mano
abde0a04dd
zml: sharding (#407)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-10 11:17:57 +01:00
Hugo Mano
9f1f46f686 mlir: introduce func extension registration via C++ bridge 2026-03-05 11:24:55 +00:00
Steeve Morin
dcbf5f2e27 workspace: update to 0.16.0-dev.2611+f996d2866, add remote exec
also:
- use toolchains_llvm_bootstrapped for macOS builds
2026-02-16 12:05:01 +01:00
Corentin Godeau
1fd90f1e45
Various improvements (#379)
This pull request includes several improvements and bug fixes across the
codebase, focusing on memory handling, correctness in tensor testing,
and API consistency. The most significant changes are grouped below:

### Tensor Testing and Memory Management Improvements

* Fixed a bug in `zml/testing.zig`'s `testlayer` by updating the
callback signature to take a mutable context and increment the index
correctly, ensuring tensors are correctly initialized.
* Improved memory management for expected output slices in `testlayer`
by switching to `zml.Slice.alloc` for allocation, using `readSliceAll`,
and freeing only the data instead of the whole slice, removing the
potential alignment problems.
* Enhanced test logging to include the tolerance value in success
messages for better diagnostics.

### API and Code Quality Enhancements

* Increased the evaluation branch quota in `MapRestrict` to support more
complex compile-time type mapping, preventing quota exceeded errors.
* Updated the `fromCStruct` function in `pjrt/pjrt.zig` to accept a
`meta.Struct`.

### StableHLO Output Operand Alias Handling

* Improved handling of output operand aliases in
`mlir/dialects/stablehlo/stablehlo.zig::custom_call` by passing
`output_tuple_indices` for multi-result functions, increasing
correctness in custom call attribute construction.
2026-02-09 12:01:22 +01:00
Corentin Godeau
7bbce6c3d0
Attention Backend on ZML V2 (#373) 2026-01-16 10:59:12 +01:00
Corentin Godeau
3581770d26 Fix tests 2025-12-19 11:09:34 +01:00
Corentin Godeau
dc00398d1c Restored ops.triton 2025-12-11 18:23:11 +01:00
Corentin Godeau
51ec792d87 Migrate to @EnumLiteral() 2025-12-11 18:11:58 +01:00
Corentin Godeau
145a802491 Re-enabled argsort tests 2025-12-09 18:03:14 +01:00
Corentin Godeau
09e3ddf55d Re-enabled gather tests 2025-12-09 16:56:29 +01:00
Corentin Godeau
edc451fcf3 Various fixes in stablehlo bindings 2025-12-09 14:59:47 +01:00
Corentin Godeau
45c94dd5b5 Small fix in stablehlo.zig 2025-12-08 17:45:18 +01:00