0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

13 commits

Author SHA1 Message Date
Kevin Barre
74562d518d
zml/platform: Add furiosa platform (#871)
Co-authored-by: Raphael Nahoum <raphael@zml.ai>
2026-10-09 12:21:16 +02:00
Guillaume Wenzek
b2f0e03043
zml/Tensor: explicit sharding in Tensor code (#868)
In Buffer code we already have explicit sharding, but not for tensor.

I'm adding a `zml.Sharding` to `.withPartitioning` to make it explicit
which mesh we are working with.
To simplify the migration, `.withPartitioning` in Tensor code can use
`.model` syntax to lookup a sharding by name.
This replace the previous logic of `selectSharding(shape)` which would
look up a sharding that matches the axes of the given partitioning which
could yield surprising behavior.

This also modifies how sharding are stored in zml.Shape.
Before it was storing a list of axis names + a tag (2 * 8 * 64bits), now
I'm storing a zml.Sharding (64 bits) and a compact PartitionArray (32
bits) that references the logical axis of the sharding by offset instead
of by name.

There is an impact on the `TensorRegistry` because it also needs to be
aware of the possible shardings, instead of delegating the matching
partition spec -> sharding to loading. This means error will be detected
sooner.

`store.createTensor` accepts `@enumLiteral()` to lookup the relevant
sharding.

This also simplifies `zml.Buffer` because the full sharding is now
stored in the shape, so it doesn't need to be passed independently. It
also means that it will propagate from the tensor who are graph inputs,
to the buffers that are eg inputs.

`Placement` logic also got a bit simpler.
2026-10-06 15:37:22 +02:00
Guillaume Wenzek
d6489fbb84
zml/attention: implement batching in cu_fa2 backend (#641) 2026-07-17 14:10:41 +02:00
elogir
f9183137a1
examples/llm: fix single-chip TPU sharding crash (#627)
A 1-chip TPU mesh collapses to rank 1 (only `link_x`), so the
unconditional `link_y` fold tripped InvalidPhysicalAxis. Only fold
`link_y` when the physical mesh actually has it.
2026-07-03 14:32:41 +02:00
elogir
33a3b37900
platforms/metal: add Metal support (#593)
Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-06-24 09:54:11 +02:00
Louis Lechevalier
be29637c07
zml/moe: Mosaic TPU MoE backend. (#571)
This PR add the TPU backend for MoE. It uses the GMM Expert parallelism
kernel to perform the grouped gemms on gate_up and down weights (these
weights must have a precise shape to run optimally).

This backend is currently optimized for small batches. A specialized
mosaic tpu kernel will be introduced to handle bigger ones.

This implementation come with few fixes concerning the TPU sharding and
the Qwen 3.5 moe model which uses the backend :
- TPU sharding : PJRT reports TPU device coordinates in a max rank form.
So it breaks folding. To avoid this we compute the Rank by not counting
where coords =0.
- Linear attention was reading padding token in the Qwen 3.5 model. So
we introduce masking of them.

n.b. : The kv head repetition to satisfy the sharding will be handled a
more generic way in a next PR

---------

Co-authored-by: Corentin Godeau <corentin@zml.ai>
2026-06-16 10:06:52 +02:00
Hugo Mano
950c80dee8
examples/llm: split Llama and Qwen 3.5 with LFM cleanup (#555)
This PR splits Llama in `examples/llm` into separate `embed_tokens`,
`layer`, and `lm_head` kernels for both prefill and decode, and brings
LFM2 onto the same composed execution path. And done same work for Qwen
3.5 .

It removes the old LFM2 `--single` mode, reworks Llama KV-cache access
to use explicit layer indices, and reuses more decode-time state through
a persistent runner and prebuilt token index buffers.

It also tightens sharding/partitioning in LFM2 for multi-device runs,
adds clearer per-phase compile logging/program names, removes the unused
`--single` CLI flag.
2026-05-28 10:10:29 +02:00
loupicaaa
b6e2b026bc
zml/moe: Add sharding for triton MoE (#552)
This PR introduces the sharding as expert parallelism for MoE :

- Introduce a .experts logical sharding (required to express an other
semantic than TP (.model))
- Wrap the whole MoE impl into a manualComputation
- Implement a manual stableHlo all reduce since shardy can not infer
the correct communication after a manualComputation (blackbox)
- Partition qwen 3.5 moe example with TP for the rest of the model (non
MoE)

---------

Co-authored-by: loupicaaa <louis@zml.ai>
2026-05-27 10:13:38 +02:00
Guillaume Wenzek
0979289ffe
zml/sharding: pass sharding by pointer and simplify replicated usage (#514)
This PR simplifies the common replicated case while making sharding
intent explicit and use sharding by pointer ( sharding struct is quite
big: ~600 bytes).

Using pointers for shardings also avoids unnecessary sharding copies and
makes it easier to share stable sharding instances across compilation,
loading, and runtime paths.

- `Platform` owns the canonical replicated sharding
- `Buffer` APIs now take explicit sharding specs:
    - `.replicated`
    - `.{ .sharded = sharding_ptr }`
- `createTensor` / `maybeCreateTensor `now require explicit partitioning
instead of accepting null
- compilation/loading now use pointer-based sharding lists

This makes simple cases like mnist and benchmark much cleaner: callers
no longer need to manually construct and thread a replicated sharding
just to use the default case, but they still state placement explicitly
at the call site.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-05-06 12:46:15 +02:00
Guillaume Wenzek
9211a1770e
Remove BoundedArray.init (#506)
When looking at the new Sharding code I was triggered by the high number
of "try".

I then realized most of the try where traced back to `axes = try
.init(0)` which can never fail.
So I removed BoundedArray.init (aka Andrew mode)

It's generally unsafe because it makes `slice()` return undefined.;
I replaced`try init(0)` by `.empty`
I replaced `try init(n)` by `.fromSlice` or with `.{ .buffer =
undefined, .len = n }` to explicitly show the unsafeness

This leads to widespread changes because it means that most of the
sharding operations can't fail anymore.
2026-04-27 10:31:52 +02:00
Benjamin Rabier
9fa7768808
examples/llm: better single prompt UX + --topk (#438)
- Same UX as the chat
- Add `--topk` argument
2026-03-26 15:58:03 +01:00
Benjamin Rabier
8de2c42fe6
examples/llm: Simplifications (#430) 2026-03-24 18:16:25 +01:00
Benjamin Rabier
81299c67c8
examples/llm: Common model example (#425) 2026-03-24 16:55:29 +01:00