In Buffer code we already have explicit sharding, but not for tensor.
I'm adding a `zml.Sharding` to `.withPartitioning` to make it explicit
which mesh we are working with.
To simplify the migration, `.withPartitioning` in Tensor code can use
`.model` syntax to lookup a sharding by name.
This replace the previous logic of `selectSharding(shape)` which would
look up a sharding that matches the axes of the given partitioning which
could yield surprising behavior.
This also modifies how sharding are stored in zml.Shape.
Before it was storing a list of axis names + a tag (2 * 8 * 64bits), now
I'm storing a zml.Sharding (64 bits) and a compact PartitionArray (32
bits) that references the logical axis of the sharding by offset instead
of by name.
There is an impact on the `TensorRegistry` because it also needs to be
aware of the possible shardings, instead of delegating the matching
partition spec -> sharding to loading. This means error will be detected
sooner.
`store.createTensor` accepts `@enumLiteral()` to lookup the relevant
sharding.
This also simplifies `zml.Buffer` because the full sharding is now
stored in the shape, so it doesn't need to be passed independently. It
also means that it will propagate from the tensor who are graph inputs,
to the buffers that are eg inputs.
`Placement` logic also got a bit simpler.
## Summary
- `zml-smi --top` was stalling under load because each GPU and the
process list hit NVML on their own thread. Those calls serialize on the
driver RM lock, so one slow ioctl froze the whole TUI.
- All NVIDIA queries now run on a single poller.
`nvmlDeviceGetPcieThroughput` (it blocks while sampling) is only taken
on the first tick and every fourth tick after that; link speed/width
stay on the hot path.
- Process-query sizes above 4096 are ignored, and a failed per-process
util sample no longer skips the rest of that GPU.
---------
Co-authored-by: alhussein jamil <alhussein@zml.ai>
Render the zml-smi text logo fallback from the shared ZML logo data,
including the shiny per-glyph colors. Add a lightweight //zml:logo
target so the TUI can depend on the logo art without pulling in //zml.
Verified with:
- bazel test //bin/zml-smi/tui:test
- bazel build //bin/zml-smi:zml-smi
When a DRM fd is duplicated or inherited through fork(), the same oneAPI
DRM client can appear multiple times under /proc/<pid>/fdinfo. Summing
each fdinfo entry double-counts GPU memory and engine time.
Parse drm-client-id and skip entries whose (device_idx, client_id) pair
has already been seen during the collection pass. Include the device in
the key because drm-client-id may be scoped per DRM device.
Add a focused test covering duplicate clients on the same device and
distinct clients across devices or client IDs.
---------
Co-authored-by: Kevin Barre <kevin@zml.ai>
A few missing loads
Remove all flipped flags or deperacted flags from .bazelrc
Important:
- Keep autoload rules_cc and rules_python because XLA still has
transitive deps that are missing loads, PR in progress.
**Summary**
This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.
The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.
On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
Add a oneAPI backend for `zml-smi` that detects Intel DRM render devices
and reports GPU telemetry from sysfs, hwmon, and fdinfo.
The new backend collects utilization, memory usage, power, temperature,
clocks, PCIe link data, and per-process usage for Intel GPUs. It is
wired into device detection, Bazel builds, Prometheus metrics, and the
TUI, including Intel GPU imagery and GPU detail/process table support.
Also clean up related TUI handling for GPU clock/PCIe fields and process
table lifetime management.
Specifying both `rows` and `cols` makes the zml logo in `zml-smi` render
correctly in KDE Konsole. Verified that the fix didn't cause any
regression in a terminal emulator where logo rendering was already
functional (`Ghostty`).
`BUILD.bazel` is also updated to run the `zml-smi` tests.
**Before**
<img width="348" height="238" alt="image"
src="https://github.com/user-attachments/assets/7abc604d-14b9-4c0a-a561-9acde1d5359e"
/>
**With Fix**
<img width="348" height="162" alt="image"
src="https://github.com/user-attachments/assets/d93a7253-c1fc-4ff9-891b-2e0c6ec46a65"
/>
**Test Discovery Fix**
```
master $ bazel query 'kind(zig_test, //bin/zml-smi/...)'
INFO: Empty results
master $ git checkout -
Switched to branch 'fix/zml-smi-logo-size'
fix/zml-smi-logo-size $ bazel query 'kind(zig_test, //bin/zml-smi/...)'
//bin/zml-smi/tui:test
```
This PR fixes several Bazel analysis regressions so both of these
commands work reliably:
- `bazel cquery //...`
- `bazel cquery --platforms=@zml//platforms:linux_amd64 //...`
After recent changes and the rebase, repo-wide cquery was failing for
three separate reasons:
- ZLS toolchains were over-constrained by target platform: updated ZLS
toolchain registration to constrain only the exec platform.
This avoids cross-target analysis failures when host-side completion
tooling is analyzed under a foreign target platform.
- OCI packaging targets were analyzed on host platforms where their base
images do not apply: marked Linux OCI image targets as Linux x86_64
compatible only. Also switched the Debian base image pull from Docker
Hub to the GCR mirror to avoid CI fetch failures when resolving
packaging images manifests due to rate limiting
- Triton Python deps were fetched for the host Python platform instead
of the Linux wheel platform the lockfile was generated for: restored
Linux x86_64 wheel resolution for `triton_py_deps` in `MODULE.bazel`.
This prevents macOS host analysis from attempting to fetch a
`triton==3.6.0` wheel for an unsupported host platform.
Also added a step in ci to prevent regression
Update Zig compiler to 0.16.0-dev.3133+5ec8e45f3 and adjust code
for breaking standard library changes:
- Replace ArrayListUnmanaged/ArrayList init with .empty
- Switch to unmanaged AutoArrayHashMap/StringArrayHashMap
- Use debug.GeneralPurposeAllocator (renamed upstream)
- Replace std.Io.File.ReadStreamingError with error.EndOfStream
- Refactor meta.visit to use structured visitor pattern
- Fix alignment allocation in mem.bufferizeInner
Also update third-party dependencies: flashattn 0.0.6-rc8, arocc,
uucode, libvaxis, and zigimg patches.
Co-authored-by: Kevin Barre <kevin@zml.ai>
This PR introduces:
- Prometheus metrics endpoint through `--prometheus` and optionally
`--prometheus-port`
- JSON output mode with `--json`
- macOS arm64 host information
- An install and release script
This PR introduces **zml-smi**, a tool to monitor metrics of any
platform supported by ZML, featuring but not limited to:
- real time `--top` mode
- per process tracking
- responsive layout
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>