0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

14 commits

Author SHA1 Message Date
Hugo Mano
6a1698f270
zml/ops: add manualComputation2 with explicit partitioning (#1013)
Accept a logical mesh and typed input/output partition specs at manual
computation boundaries.

Will replace `zml.ops.manualComputation` in a future PR.
2026-11-03 18:22:59 +01:00
Guillaume Wenzek
b2f0e03043
zml/Tensor: explicit sharding in Tensor code (#868)
In Buffer code we already have explicit sharding, but not for tensor.

I'm adding a `zml.Sharding` to `.withPartitioning` to make it explicit
which mesh we are working with.
To simplify the migration, `.withPartitioning` in Tensor code can use
`.model` syntax to lookup a sharding by name.
This replace the previous logic of `selectSharding(shape)` which would
look up a sharding that matches the axes of the given partitioning which
could yield surprising behavior.

This also modifies how sharding are stored in zml.Shape.
Before it was storing a list of axis names + a tag (2 * 8 * 64bits), now
I'm storing a zml.Sharding (64 bits) and a compact PartitionArray (32
bits) that references the logical axis of the sharding by offset instead
of by name.

There is an impact on the `TensorRegistry` because it also needs to be
aware of the possible shardings, instead of delegating the matching
partition spec -> sharding to loading. This means error will be detected
sooner.

`store.createTensor` accepts `@enumLiteral()` to lookup the relevant
sharding.

This also simplifies `zml.Buffer` because the full sharding is now
stored in the shape, so it doesn't need to be passed independently. It
also means that it will propagate from the tensor who are graph inputs,
to the buffers that are eg inputs.

`Placement` logic also got a bit simpler.
2026-10-06 15:37:22 +02:00
Benjamin Rabier
af4e079f34
zml/exe: callOpts/zml.testing.autoCall replaced by eval + remove run|callOpts (#897)
Removing callOpts & runOpts to simplify the API and changing `io` to be
required for API clarity. I'm replacing `zml.testing.autoCall` with
`Exe.eval` which solves most needs in a simpler way IMHO
2026-10-05 13:51:36 +02:00
Damien Mehala
e97abbaad4
zml/ops: refactor manualComputation (#757)
Simplify `manualComputation` by merging `inputs` and `body_context` into a single `input` argument.
Internally, collect all and shard all tensors (including `?Tensor`) from `input` before invoking the body function.
2026-09-03 11:02:59 +02:00
Guillaume Wenzek
cd477b858a
zml: refactor module.zig into Compiler.zig (#736)
Migration instructions:

zml.module.CompilationContext -> zml.Compiler
zml.module.CompileError -> zml.Compiler.Error
zml.module.CompilationOptions -> zml.Compiler.Options
zml.module.compile -> zml.Platform.compiler or zml.Compiler.compile

Breaking changes: none, I've kept a small compatibility layer at the top
level, with upgrade instructions.

We can remove it in a second time.
2026-08-27 18:31:50 +02:00
Benjamin Rabier
a06265264f
zml/Sharding: Rename Placement.Shard to Placement (#549)
Keeping `devices_in_canonical_order: []Device` in the `PhysicalMesh` and
inside `Sharding.Data` I've changed `physical: PhysicalMesh` to
`physical: *const PhysicalMesh` to have the platform own it now that
PhysicalMesh has a `deinit()`.
2026-05-26 09:29:49 +02:00
Benjamin Rabier
5d372aa4c0
zml/Sharding: Reduce Placement size by allocating shards on the stack. (#547)
`Placement` is 27KiB big, it led to stack allocation failure with
gigantic struct sizes (~400KiB) in llmd. So changing the shards to be
allocated on the heap with functions to avoid allocations when we don't
need it (`format`, etc.).
2026-05-20 16:34:47 +02:00
elogir
144f3eb0af
platforms/neuron: bump toolchain and stabilize support (#476)
- bump Neuron dependencies in `platforms/neuron/requirements.lock.txt`
- `libneuronxla` `2.2.15515.0+50c26cbd` -> `2.2.16408.0+50c26cbd`
- `neuronx-cc` `2.23.6484.0+3b612583` -> `2.24.8799.0+6f62ff7c`
- make `neuronx-cc` invocation more robust by fixing path resolution,
releasing the Python GIL during compilation, checking the compiler
process exit status, and propagating NEFF wrapping/read
failures
- resolve PJRT devices and memories by PJRT handles instead of assuming
stable numeric ids during platform initialization, including Neuron
cases where memory ids are reused across devices
- avoid unsupported Neuron synchronization/event paths by always
requesting execution events on Neuron, deinitializing them without
awaiting, and removing the explicit await from the sharding
example
- avoid unsupported Neuron runtime paths by:
- falling back to host-backed buffer initialization for
`Buffer.uninitialized`
- using passthrough DMA allocation on Neuron
- skipping `Tensor.print()` custom calls on Neuron
- add NRT-backed Neuron physical mesh topology construction, including
chip/core placement, ring/torus/point-to-point topology inference, and
Neuron local-hardware-id based shard placement
- let the sharding example pick the default partitioner from the current
target
2026-05-12 09:58:32 +02:00
Hugo Mano
13cee57bf2
zml/profiling: unify profiling flow and tracing (#443)
This PR modernizes ZML profiling in four areas:

- replaces the old C++ `xspace_to_perfetto` tool with a native Zig
exporter built on upb
- integrates trace export directly into `zml/profiler.zig`
- introduces a scoped tracing API in `zml.tracer` and wires it through
examples
- adds platform-specific annotation support and Bazel-native profiler
wrappers for CUDA and ROCm

The result is a simpler profiling flow, better large-trace behavior, and
a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
Guillaume Wenzek
0979289ffe
zml/sharding: pass sharding by pointer and simplify replicated usage (#514)
This PR simplifies the common replicated case while making sharding
intent explicit and use sharding by pointer ( sharding struct is quite
big: ~600 bytes).

Using pointers for shardings also avoids unnecessary sharding copies and
makes it easier to share stable sharding instances across compilation,
loading, and runtime paths.

- `Platform` owns the canonical replicated sharding
- `Buffer` APIs now take explicit sharding specs:
    - `.replicated`
    - `.{ .sharded = sharding_ptr }`
- `createTensor` / `maybeCreateTensor `now require explicit partitioning
instead of accepting null
- compilation/loading now use pointer-based sharding lists

This makes simple cases like mnist and benchmark much cleaner: callers
no longer need to manually construct and thread a replicated sharding
just to use the default case, but they still state placement explicitly
at the call site.

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-05-06 12:46:15 +02:00
Guillaume Wenzek
9211a1770e
Remove BoundedArray.init (#506)
When looking at the new Sharding code I was triggered by the high number
of "try".

I then realized most of the try where traced back to `axes = try
.init(0)` which can never fail.
So I removed BoundedArray.init (aka Andrew mode)

It's generally unsafe because it makes `slice()` return undefined.;
I replaced`try init(0)` by `.empty`
I replaced `try init(n)` by `.fromSlice` or with `.{ .buffer =
undefined, .len = n }` to explicitly show the unsafeness

This leads to widespread changes because it means that most of the
sharding operations can't fail anymore.
2026-04-27 10:31:52 +02:00
Corentin Godeau
ac9bc32d89
zml/attention: add Triton Unified Attention backend (#417)
This PR introduces a Triton paged attention backend, unlocking ROCm
support.

It relies on a Python sandbox to generate the TTIR.

The major change is that the KVCache now needs to be per-layer for paged
attention. The `layer_index` is now superfluous and will probably be
removed in the future.
2026-04-13 11:39:08 +02:00
Hugo Mano
619fea8b18
zml: introduce profiler (#413)
Move profiling behind a zml-owned primitive instead of exposing PJRT
details in user code. Add the XLA/TSL Bazel deps needed to convert
collected XSpace protobufs into Perfetto traces with XLA/TLS APIs
instead of having a custom fallible converter.
2026-03-17 18:14:24 +01:00
Hugo Mano
abde0a04dd
zml: sharding (#407)
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-10 11:17:57 +01:00