0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

12 commits

Author SHA1 Message Date
Hugo Mano
6cd00a0d00
ci: add supported platforms with bumped and audited deps (#967) 2026-10-20 14:06:49 +02:00
Steeve Morin
df3a43d29f
bazel: add zig_options rule (#820)
Introduce `zig_option` and `zig_options` rules to generate the bazel
equivalent of option modules plugged on bazel flags.

On `string` flags, if the flag has a `allowed_values`, the `string` is
converted into a `enum` in zig code.

Also, port platforms to this and remove the ugly `defines` hack to
comptime test for platforms enablement.

In the process, port `arocc` to the `zig_options`.
2026-09-16 21:50:22 +02:00
Hugo Mano
0ebb0c21a9 platforms: bump plugins for CPU, ROCm, CUDA and Metal (#788) 2026-09-10 14:34:40 +00:00
Hugo Mano
1fead1c014
platforms/neuron: introduce NKI stack with SDPA kernel (#528)
**Summary**

This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.

The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.

On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
elogir
144f3eb0af
platforms/neuron: bump toolchain and stabilize support (#476)
- bump Neuron dependencies in `platforms/neuron/requirements.lock.txt`
- `libneuronxla` `2.2.15515.0+50c26cbd` -> `2.2.16408.0+50c26cbd`
- `neuronx-cc` `2.23.6484.0+3b612583` -> `2.24.8799.0+6f62ff7c`
- make `neuronx-cc` invocation more robust by fixing path resolution,
releasing the Python GIL during compilation, checking the compiler
process exit status, and propagating NEFF wrapping/read
failures
- resolve PJRT devices and memories by PJRT handles instead of assuming
stable numeric ids during platform initialization, including Neuron
cases where memory ids are reused across devices
- avoid unsupported Neuron synchronization/event paths by always
requesting execution events on Neuron, deinitializing them without
awaiting, and removing the explicit await from the sharding
example
- avoid unsupported Neuron runtime paths by:
- falling back to host-backed buffer initialization for
`Buffer.uninitialized`
- using passthrough DMA allocation on Neuron
- skipping `Tensor.print()` custom calls on Neuron
- add NRT-backed Neuron physical mesh topology construction, including
chip/core placement, ring/torus/point-to-point topology inference, and
Neuron local-hardware-id based shard placement
- let the sharding example pick the default partitioner from the current
target
2026-05-12 09:58:32 +02:00
elogir
57546912f1
bin/zml-smi: Add CSV export and driver version reporting (#452)
- Restructured `bin/zml-smi/`
- Added CSV export
- Added driver version reporting
- Minor fixes
2026-04-07 16:44:52 +02:00
elogir
67b4fbdbb8
bin: Introduce zml-smi (#440)
This PR introduces **zml-smi**, a tool to monitor metrics of any
platform supported by ZML, featuring but not limited to:

- real time `--top` mode
- per process tracking
- responsive layout

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-03-31 12:47:07 +02:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Benjamin Rabier
3b76703430
workspace: fixes for build and test //... green
Fixes:
- Added `target_compatible_with = ["@platforms//os:linux"],` for neuron
& rocm
- Updated the zig in neuron
- Updated neuron to use Python 3.12, it's at best a mess to handle two
different python version with `build //...`
- renamed `third_party/flashattn/BUILD.bazel` to
`third_party/flashattn/flashattention.BUILD.bazel` as it's not part of
this module.
- Used HTTPS & fallback URL for swig, sometimes we would end up with an
incorrect archive (wrong sha256)
- Using `--@rules_python//python/config_settings:bootstrap_impl=script`
for hermetic Python build, it relied on the system Python otherwise.
- Used a separate pip extension for neuron to avoid the extra pip args
to leak to other installations.
- Fix `zml/testing` `fn autoCall` function

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-03-23 14:55:48 +01:00
Steeve Morin
ac971b4bc6 workspace: fix upb rule loading 2026-03-18 13:03:04 +01:00
Steeve Morin
1b644887b7 bazel,platforms: centralize runfiles loading
runfiles are process-wide in space and time.
so it makes sense to keep them around

because they are bound to the process, we use the smp_allocator and
except the memory to be kept around during the lifetime of the process.
2026-01-19 18:14:06 +01:00
Steeve Morin
9e2c09066b runtimes: rename to platforms 2026-01-19 17:41:38 +01:00
Renamed from runtimes/neuron/BUILD.bazel (Browse further)