0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

32 commits

Author SHA1 Message Date
Guillaume Wenzek
b2f0e03043
zml/Tensor: explicit sharding in Tensor code (#868)
In Buffer code we already have explicit sharding, but not for tensor.

I'm adding a `zml.Sharding` to `.withPartitioning` to make it explicit
which mesh we are working with.
To simplify the migration, `.withPartitioning` in Tensor code can use
`.model` syntax to lookup a sharding by name.
This replace the previous logic of `selectSharding(shape)` which would
look up a sharding that matches the axes of the given partitioning which
could yield surprising behavior.

This also modifies how sharding are stored in zml.Shape.
Before it was storing a list of axis names + a tag (2 * 8 * 64bits), now
I'm storing a zml.Sharding (64 bits) and a compact PartitionArray (32
bits) that references the logical axis of the sharding by offset instead
of by name.

There is an impact on the `TensorRegistry` because it also needs to be
aware of the possible shardings, instead of delegating the matching
partition spec -> sharding to loading. This means error will be detected
sooner.

`store.createTensor` accepts `@enumLiteral()` to lookup the relevant
sharding.

This also simplifies `zml.Buffer` because the full sharding is now
stored in the shape, so it doesn't need to be passed independently. It
also means that it will propagate from the tensor who are graph inputs,
to the buffers that are eg inputs.

`Placement` logic also got a bit simpler.
2026-10-06 15:37:22 +02:00
Guillaume Wenzek
f09060e5bf
//tools/xprof: bump to xprof 2.32.2 (#823)
I found a non-nightly version that seems to work well for us.
Previous nightly versions we were using is not hosted anymore on
Pypi.org
2026-09-17 10:02:57 +02:00
Raphael Nahoum
94257d3747
kernels/cute: Zig CuTe DSL and XLA custom call and bump xprof (#810)
Adds a Zig front end for NVIDIA's CuTe DSL, shaped after `cutlass.cute`,
plus the `__gpu$xla.gpu.cute` custom call that compiles and launches it
on CUDA.

- `mlir/dialects/cute_ir`: bindings for the `cute` and `cute_nvgpu`
dialects
- `zml.ops.cute`: emits the custom call with the textual module, `grid`
and `block`.
- `kernels/cute` + `zml.kernel.cute.Kernel`: builder, values, tensors
with coordinate `get`/`set`, thread/block indices, static smem, sync,
`scf`. See `kernels/cute/README.md`.
- bump `xprof-nightly`

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-09-16 14:02:46 +02:00
Hugo Mano
0ebb0c21a9 platforms: bump plugins for CPU, ROCm, CUDA and Metal (#788) 2026-09-10 14:34:40 +00:00
Benjamin Rabier
c6415035db
tools/xprof: Fix xprof after update (#664) 2026-07-23 10:54:22 +02:00
Hugo Mano
3e589c9772
platforms/rocm: bump to rocm-jaxlib-v0.10.2 with TheRock 7.14 (#671)
Linked PR: https://github.com/zml/pjrt-artifacts/pull/96
2026-07-21 16:18:28 +02:00
Kevin Barre
8a89d73fb8
build/tools xprof: refresh xprof-nightly lock (#637)
Regenerate the xprof uv lockfile to move xprof-nightly off the removed
2.23.2a20260505 build and onto 2.23.3a20260707.

Fixes Bazel fetch failures for //tools/xprof:xprof.
2026-07-08 09:33:36 +02:00
Hugo Mano
833f42029d
platforms: bump cpu + cuda plugins and align xla (#626)
Related PR: https://github.com/zml/monorepo/pull/304
2026-07-03 14:22:23 +02:00
Corentin Kerisit
aa90f92832
workspace: bump to bazel 9.1.1 (#601)
A few missing loads
Remove all flipped flags or deperacted flags from .bazelrc

Important:
- Keep autoload rules_cc and rules_python because XLA still has
transitive deps that are missing loads, PR in progress.
2026-06-25 19:37:51 +02:00
Hugo Mano
1fead1c014
platforms/neuron: introduce NKI stack with SDPA kernel (#528)
**Summary**

This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.

The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.

On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
Corentin Kerisit
31887689a9
workspace: add several bazel 9 required changed (#530)
Just a few stuff to get us closer to Bazel 9
XLA is the main blocker and i'll tackle this as soon as other
dependencies merge my PRs.
2026-05-12 10:27:18 +02:00
Hugo Mano
13cee57bf2
zml/profiling: unify profiling flow and tracing (#443)
This PR modernizes ZML profiling in four areas:

- replaces the old C++ `xspace_to_perfetto` tool with a native Zig
exporter built on upb
- integrates trace export directly into `zml/profiler.zig`
- introduces a scoped tracing API in `zml.tracer` and wires it through
examples
- adds platform-specific annotation support and Bazel-native profiler
wrappers for CUDA and ROCm

The result is a simpler profiling flow, better large-trace behavior, and
a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
Guillaume Wenzek
c38179513d
bump xprof-nightly (#523)
the previous nightly we depend on is not on pypi anymore.
I wanted to switch to stable release but 2.23 isn't public yet:
https://pypi.org/project/xprof/#history
2026-05-06 12:26:09 +02:00
Hugo Mano
92b8e34c7b
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522)
This change fixes profile timestamp handling so saved xplane.pb data is
post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the
Zig/C++ FFI used by the profile export bridge.

Before this, we were writing raw profiler output from the PJRT extension
directly to `profiling.xplane.pb`, which bypassed XLA’s single-host
post-processing step done by JAX that we can find here:
efac222ebf/tsl/profiler/lib/profiler_session.cc (L100)

That made ZML behave differently from JAX, including wrong timeline
origins in downstream viewers like XProf.
To fix that, we now run
`tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the
protobuf, and we treat that post-processed xplane.pb as the source for
subsequent Perfetto conversion.
We also register start/stop timestamps in `zml/profiler.zig` so they
match XLA’s expectations for session timestamps with `real` clock.

While touching this path, I tightened the FFI boundary as well. 
Error text returned from the xspace export bridge is now allocated with
the provided Zig allocator instead of malloc, C receives a pointer to
the `zig_allocator wrapper` to avoid lifetime issues from passing
allocator state by value, and error propagation in
`tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly
instead of building std::string error objects just to copy them back
out. This removes mixed allocation ownership between C++ and Zig and
fixes the allocator lifetime bug that could crash across the FFI
boundary.
2026-05-05 14:52:15 +02:00
elogir
57546912f1
bin/zml-smi: Add CSV export and driver version reporting (#452)
- Restructured `bin/zml-smi/`
- Added CSV export
- Added driver version reporting
- Minor fixes
2026-04-07 16:44:52 +02:00
Steeve Morin
96414d29c4 workspace: add stamping and helpers 2026-04-03 12:55:13 +00:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Steeve Morin
78546350af stdx: remove meta.Signature
has proven unstable on Zig nightlies and mostly needed since std.Io
anyway.
2026-03-21 17:33:45 +01:00
Hugo Mano
619fea8b18
zml: introduce profiler (#413)
Move profiling behind a zml-owned primitive instead of exposing PJRT
details in user code. Add the XLA/TSL Bazel deps needed to convert
collected XSpace protobufs into Perfetto traces with XLA/TLS APIs
instead of having a custom fallible converter.
2026-03-17 18:14:24 +01:00
Benjamin Rabier
82c9c25836
examples/lfm: Add LiquidAI LFM2.5 (#376) 2026-02-19 09:27:10 +01:00
Guillaume Wenzek
13aa52b1d7
tools/hf: add hf_transfer implicit dependency (#344)
Fix #339
2025-10-20 12:28:45 +02:00
Corentin Kerisit
36d583dd58
workspace: use upstreamed rules_zig instead of our fork (#340)
This PR uses the new release of rules_zig where we upstreamed all the
work from the zml rules_zig fork.

Not much changes from the POV of the public API except for `copts`
becoming `zigopts`.

Also, the global `c` module is only available to Zig modules that depend
on a non empty `cc_library`.
This is behavior that can probably be modified in `rules_zig` later on.

Finally, ZLS support had to be completely rethought to comply with the
change in shape of ZigModuleInfo providers and subsequent rules.

This PR is blocked until `rules_zig` exposes `ZigModuleInfo` for
`zig_binaries`.
2025-10-13 14:34:33 +02:00
Guillaume Wenzek
e9b42f0ca8
examples: gpt oss (#329)
This implement support for OpenAI "gpt-oss" models. 

It's mostly a take over of #292 branch, but MoE code is kept inside
examples folder cause it's still a bit warm.
In particular the routing logic can break apart when there are undefined
memory in the inputs, where it wasn't an issue for normal MLP.

I only tested with the 20B version.
2025-09-25 14:03:37 +02:00
Steeve Morin
57fd9cda27
workspace: zig 0.15 (#316)
Much of the work was done by @gwenzek

Sadly we drop the torch loader in the process, to be added back later !

---------

Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
Steeve Morin
5f3a2e74c3
workspace: buildifier pass, remove rules_uv, rework tools/hf deps (#315)
Title says it all
2025-08-22 16:37:36 +02:00
Steeve Morin
de4a0e0beb
third_party/xla: de-modularize xla (#309)
This PR intends to remove XLA as a bzlmod and transfer it as a non
bzlmod dep.

This is because this module will never be upstreamed as is, so keep it
private.

Also, we fetch llvm-raw and stablehlo from it, which is fine.

While there, dummify the various local_config XLA symbols to please the
imports, as we don't use those parts in ZML itself.

Closes #305
2025-08-19 23:35:01 +02:00
Guillaume Wenzek
9f2ccf8a54
package HF cli so users don't have to create Python venv (#293) 2025-08-01 16:42:29 +02:00
Steeve Morin
6df48e39de
workspace: make zls runner target explicit (#240)
Easier for future reference
2025-05-29 23:04:20 +02:00
Guillaume Wenzek
a672bb8247
examples: add modernBERT (#192)
Modern bert implementation from Victor: @vctrmn

Tested with https://huggingface.co/answerdotai/ModernBERT-base
I started from his PR #149 and cleaned up a bit the code to make it more
what I consider "idiomatic ZML" even if it's still a bit in flux since
we are so young.

---------

Co-authored-by: vctrmn <victormoin.vm@gmail.com>
Co-authored-by: Rene Schallner <rene@zml.ai>
2025-02-27 18:07:42 +01:00
Steeve Morin
e0041974d2
workspace: rework code completion using a custom build runner (#131)
Following the new release of the VSCode Zig plugin, rework the code
completion logic to leverage a build runner. This is much lighter,
faster and more reliable.
2024-12-16 13:20:27 +01:00
Hugo Mano
0b136ef43a
zml_utils: fix CollectionOver scope (#13)
**Changes**

- zml_utils: fix `AttributeError: type object 'ActivationCollector' has
no attribute 'CollectionOver'`
- zml_utils: remove dead code and comments
- docs(howto_torch2zml): add explanation on why `output` variable can be
`None`
2024-09-19 13:54:21 +02:00
Steeve Morin
13d84615bb zml: initial release 2024-09-17 11:11:18 +02:00