0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

7 commits

Author SHA1 Message Date
Guillaume Wenzek
a7a4fcd3f8
zml/sharding: use logical meshes (#943)
## Motivation

This is the big sharding rewrite we need to achieve complex sharding.

Currently we always resolve a mesh logical axis to a mesh physical axis
(or several of them).
But on GPU we always have 1D topology, so we can't have 2D logical
meshes.
So this PR drops the logical -> physical axis bindings, and instead
store a logical -> devices binding.
This is more flexible, and has the nice side effect of simplifying
`Buffer.from`

On stablehlo side this changes how we generate the MLIR because we now
use logical meshes, and `manualComputation` uses logical axes.
(In shardy, GSPM is not affected AFAICT because it uses device ID
directly)

## Buffer.from

This also accelerates `Buffer.from` that I rewrote to be FAST.
Version | Fixed cost | Per device
-- | -- | --
origin/master | 109.6 ns | 35.39 ns
Rewrite | 35.8 ns | 4.38 ns
Improvement | 3.1× | 8.1×

## Creating mesh

I'm adding a new syntax to create LogicalMesh, to allow the more complex
meshes we want to do, and in particular, I now allow to force a specific
number of dimensions.
You can see it in action in `mnist.zig`. 

```
const dp_mp = try platform.registerMesh("DP-MP", try .init(
        .{ .data, .model },
        &.{ .low_bandwidth, .high_bandwidth },
        &.{ 2, -1 },
    ));
 
info: topology used: Mesh(name=DP-MP, partitions=4, replicas=1)
┌ Mappings (logical coordinates from physical coordinates):
├── data (low_bandwidth) -> (bus / 2)
└── model (high_bandwidth) -> (bus % 2)

┌ PhysicalMesh(platform=cpu shardable_axes={ .bus } num_devices=4)
└── [bus] 0..4 (tree)
    ├── Device(id=0, coords={ 0 })
    ├── Device(id=1, coords={ 1 })
    ├── Device(id=2, coords={ 2 })
    └── Device(id=3, coords={ 3 })

┌ Logical coordinates:
└── data=0
    └── model=0,1 -> devices=0,1
└── data=1
    └── model=0,1 -> devices=2,3
```

The previous syntax is still available, and ZML will align logical axes
to physical axes like we do before this PR.
This part as unit tests, and I checked it work as I expect on CUDA but I
would need to double check the logics still works on TPU/Neuron, because
there is a few platform specific branches in that code.

## Test plan

As part of the benchmark Codex generated a test suite with hard-coded
offsets, allowing to check that Buffer.from didn't change behavior
through the commits, checking we correctly compute each individual
shard.

TODO check that LLMD perf isn't degraded.
2026-10-16 15:27:30 +02:00
Guillaume Wenzek
7bb457ebeb
zml: refactor manualComputation (#925)
here I'm promoting a pattern I would like to extend to more of ZML:

function that takes `anytype` argument should do as little work as
possible and called a fully typed function.

I kept `manualComputation` signature to not break existing code,
I moved the code to `zml/ops/manual_computation.zig` to better
encapsulate the required helper functions.
(and `zml/ops.zig` is pretty big atm)

Similarly `Sharding.Partitioning.parse` now calls `const logical =
comptime Logical.parse(partitioning);` then `logical.applyUseAll(tags,
mesh, default_value);` which is fully typed.

I'm also documenting `manualComputation` better
2026-10-16 12:30:05 +02:00
Guillaume Wenzek
807a9116d4
zml: make Buffer.from faster (#556)
`Buffer.from` is in the hot path everytime we want to send smth to the
GPU.
Sometimes we can overlap it with GPU compute but not always, and
sometimes it's quite complicated to do so.

I've rewrote Sharding.Placement to be device agnostic instead of device
specific.
This allow to reuse most of the logic for multiple devices.

According to LLVM MCA (with script included here), we got from:

```
Iterations:        100
Instructions:      324200
Total Cycles:      175191
Total uOps:        369700
```

to 

```
    Iterations:        100
    Instructions:      162900
    Total Cycles:      85585
    Total uOps:        182600
```

So about twice as fast for this part of the code. I would need some help
to confirm the numbers on TPU.


(Once the PR is approved, I'll remove the mca script as well as the
build.zig)

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-06-01 10:51:32 +02:00
Guillaume Wenzek
9211a1770e
Remove BoundedArray.init (#506)
When looking at the new Sharding code I was triggered by the high number
of "try".

I then realized most of the try where traced back to `axes = try
.init(0)` which can never fail.
So I removed BoundedArray.init (aka Andrew mode)

It's generally unsafe because it makes `slice()` return undefined.;
I replaced`try init(0)` by `.empty`
I replaced `try init(n)` by `.fromSlice` or with `.{ .buffer =
undefined, .len = n }` to explicitly show the unsafeness

This leads to widespread changes because it means that most of the
sharding operations can't fail anymore.
2026-04-27 10:31:52 +02:00
Hugo Mano
abde0a04dd
zml: sharding (#407)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-10 11:17:57 +01:00
Corentin Godeau
715b0f745a Commented Writer in stdx.BoundedArray
It was causing compile issue in tests
2025-12-09 14:59:14 +01:00
Steeve Morin
57fd9cda27
workspace: zig 0.15 (#316)
Much of the work was done by @gwenzek

Sadly we drop the torch loader in the process, to be added back later !

---------

Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00