0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

26 commits

Author SHA1 Message Date
Hugo Mano
6cd00a0d00
ci: add supported platforms with bumped and audited deps (#967) 2026-10-20 14:06:49 +02:00
Guillaume Wenzek
00a0af4a55
zig 0.17 (second attempt) (#923)
Bump for zig 0.17

Deps:
- uucode bumped to
1fb73433bb
- zigimg bumped to
734fa919fb
- libvaxis bumped to
14b7684272
- zio bumped to:
417757935e
- rules_zig upgraded to [version
0.17](https://github.com/hermeticbuild/rules_zig/releases/tag/v0.17.0)
- zls is using a commit from master, but they don't have yet a tagged
release so Codex did a bit of packaging. I confirmed it correctly
autocompletes on my machine
2026-10-13 17:11:38 +02:00
Corentin Godeau
bd0886c3e5
zml: re-structure attention kernels (#926)
This PR introduces a restructuration of the kernels in ZML. This is done
to simplify and speedup bringup on new platforms.

From now on each "kernel" will have its own directory under
`zml/kernels`. "Kernel" is in quote because we need to think about it
being more of an "operation in a graph" than an actual kernel. For
example, `moe` will eventually be part of this `kernels` directory
despite being an operation potentially launching multiple actual kernels
on some hardware.

Also it's probable that in the future, this `kernels` directory will be
put outside of zml or, at least, properly separated so that we don't
have circular imports in the codebase. Currently, we have
`@import("../../../zml.zig")` in the kernel code and this is a recipe
for disaster.

On top of the reorganization, it also adds a level of indirection for
attention. Where previously `attention` was a generic term, we now have
`attention` and `flash_attention` because `flash_attention` is an
implementation of `attention`.

Here is a detailed summary of the changes introduced by this PR:
* Kernels reorganization. `flash_attention` and `paged_attention` are
now separated. Each folder has a subfolder for each backend we support.
`flash_attention` is also now a `BackendType` of `zml.attention` with
its own implementations.
* Removed `cuda_fa2` and `cuda_fa3` backend because they were obsolete.
They have been replaced by a `cute` backend that has been generated
automatically following the operation documentation, showing that this
approach of scoping kernels per DSL actually scales.
* Removed `attnd` support for now
2026-10-12 20:31:17 +02:00
Guillaume Wenzek
303ff3e6a4
Revert "zig 0.17 (#693)" (#909)
This reverts commit 5a18757368.

Our internal repos has a few zig patches, I need to also upgrade them
before we ugprade ZML
2026-10-07 11:34:51 +02:00
Guillaume Wenzek
5a18757368
zig 0.17 (#693)
Bump for zig 0.17

Deps:
- uucode bumped to
1fb73433bb
- zio bumped to:
417757935e
- rules_zig upgraded to [version
0.17](https://github.com/hermeticbuild/rules_zig/releases/tag/v0.17.0)

There is one Codex generated patch:
- zigimg: minor fixes to allow zig 0.17 to compile it, until they
officially support
2026-10-07 11:08:06 +02:00
Steeve Morin
9b7d8fb58c bazel: rework zig_options to be config_settting compatible
zig_option can now take a `flag` dependency to expose existing flags or
`bool`, `int or `string` for config_setting flags
2026-09-24 22:21:44 +02:00
Steeve Morin
df3a43d29f
bazel: add zig_options rule (#820)
Introduce `zig_option` and `zig_options` rules to generate the bazel
equivalent of option modules plugged on bazel flags.

On `string` flags, if the flag has a `allowed_values`, the `string` is
converted into a `enum` in zig code.

Also, port platforms to this and remove the ugly `defines` hack to
comptime test for platforms enablement.

In the process, port `arocc` to the `zig_options`.
2026-09-16 21:50:22 +02:00
Hugo Mano
0ebb0c21a9 platforms: bump plugins for CPU, ROCm, CUDA and Metal (#788) 2026-09-10 14:34:40 +00:00
Corentin Kerisit
076417363a
workspace: add missing loads (#606)
nits
2026-06-27 00:51:55 +02:00
Hugo Mano
1fead1c014
platforms/neuron: introduce NKI stack with SDPA kernel (#528)
**Summary**

This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the
LLM attention path.

The main goal is to make Neuron attention run through NKI kernels.
It adds the compiler/runtime plumbing needed to embed NKI kernels as
Neuron custom-native calls, then uses that path for SDPA-style
prefill/decode attention. Paged Attention will come later.

On inf2.8xlarge it runs :
- 25 tok/s for Llama 3.1 8B BF16
- 63.5 tok/s for LFM2.5-1.2B-Instruct BF16
- 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
elogir
144f3eb0af
platforms/neuron: bump toolchain and stabilize support (#476)
- bump Neuron dependencies in `platforms/neuron/requirements.lock.txt`
- `libneuronxla` `2.2.15515.0+50c26cbd` -> `2.2.16408.0+50c26cbd`
- `neuronx-cc` `2.23.6484.0+3b612583` -> `2.24.8799.0+6f62ff7c`
- make `neuronx-cc` invocation more robust by fixing path resolution,
releasing the Python GIL during compilation, checking the compiler
process exit status, and propagating NEFF wrapping/read
failures
- resolve PJRT devices and memories by PJRT handles instead of assuming
stable numeric ids during platform initialization, including Neuron
cases where memory ids are reused across devices
- avoid unsupported Neuron synchronization/event paths by always
requesting execution events on Neuron, deinitializing them without
awaiting, and removing the explicit await from the sharding
example
- avoid unsupported Neuron runtime paths by:
- falling back to host-backed buffer initialization for
`Buffer.uninitialized`
- using passthrough DMA allocation on Neuron
- skipping `Tensor.print()` custom calls on Neuron
- add NRT-backed Neuron physical mesh topology construction, including
chip/core placement, ring/torus/point-to-point topology inference, and
Neuron local-hardware-id based shard placement
- let the sharding example pick the default partitioner from the current
target
2026-05-12 09:58:32 +02:00
Corentin Kerisit
241aaf7d3b
platforms/cuda: add arm64 support (#493)
Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-04-22 12:37:48 +02:00
KB
8edf02d0c3
zml/chore: bump Zig to 0.16.0-dev.3133 and adapt codebase (#462)
Update Zig compiler to 0.16.0-dev.3133+5ec8e45f3 and adjust code
for breaking standard library changes:

- Replace ArrayListUnmanaged/ArrayList init with .empty
- Switch to unmanaged AutoArrayHashMap/StringArrayHashMap
- Use debug.GeneralPurposeAllocator (renamed upstream)
- Replace std.Io.File.ReadStreamingError with error.EndOfStream
- Refactor meta.visit to use structured visitor pattern
- Fix alignment allocation in mem.bufferizeInner

Also update third-party dependencies: flashattn 0.0.6-rc8, arocc,
uucode, libvaxis, and zigimg patches.

Co-authored-by: Kevin Barre <kevin@zml.ai>
2026-04-13 11:56:32 +02:00
elogir
57546912f1
bin/zml-smi: Add CSV export and driver version reporting (#452)
- Restructured `bin/zml-smi/`
- Added CSV export
- Added driver version reporting
- Minor fixes
2026-04-07 16:44:52 +02:00
elogir
67b4fbdbb8
bin: Introduce zml-smi (#440)
This PR introduces **zml-smi**, a tool to monitor metrics of any
platform supported by ZML, featuring but not limited to:

- real time `--top` mode
- per process tracking
- responsive layout

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2026-03-31 12:47:07 +02:00
Steeve Morin
3828a571bc platforms/neuron: minor neuron fixes 2026-03-25 18:05:23 +00:00
Hugo Mano
2f175540bf
ci: rework pipeline + add remote with buildbuddy
Co-authored-by: Benjamin Rabier <benjamin@zml.ai>
2026-03-23 16:07:40 +01:00
Benjamin Rabier
3b76703430
workspace: fixes for build and test //... green
Fixes:
- Added `target_compatible_with = ["@platforms//os:linux"],` for neuron
& rocm
- Updated the zig in neuron
- Updated neuron to use Python 3.12, it's at best a mess to handle two
different python version with `build //...`
- renamed `third_party/flashattn/BUILD.bazel` to
`third_party/flashattn/flashattention.BUILD.bazel` as it's not part of
this module.
- Used HTTPS & fallback URL for swig, sometimes we would end up with an
incorrect archive (wrong sha256)
- Using `--@rules_python//python/config_settings:bootstrap_impl=script`
for hermetic Python build, it relied on the system Python otherwise.
- Used a separate pip extension for neuron to avoid the extra pip args
to leak to other installations.
- Fix `zml/testing` `fn autoCall` function

---------

Co-authored-by: Hugo Mano <hugo@zml.ai>
2026-03-23 14:55:48 +01:00
Steeve Morin
06f13f9a22 platforms: uncompress http_archives using http_bsdtar_archive
Roughly a 2x speedup according to @cerisier. So take it for what it is.
2026-03-20 16:22:26 +00:00
Benjamin Rabier
a011fb0e52
Green CI (#419)
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
2026-03-20 10:31:57 +01:00
Steeve Morin
ac971b4bc6 workspace: fix upb rule loading 2026-03-18 13:03:04 +01:00
Steeve Morin
d511605d5b platforms: port to zig 0.16.0-dev.2611+f996d2866 2026-02-16 13:45:51 +00:00
Steeve Morin
d26386d206 platforms: fix cuda, rocm and neuron sandboxes 2026-01-19 22:16:47 +01:00
Steeve Morin
1b644887b7 bazel,platforms: centralize runfiles loading
runfiles are process-wide in space and time.
so it makes sense to keep them around

because they are bound to the process, we use the smp_allocator and
except the memory to be kept around during the lifetime of the process.
2026-01-19 18:14:06 +01:00
Steeve Morin
21a7c5fec8 platforms: pass allocator to load 2026-01-19 17:41:57 +01:00
Steeve Morin
9e2c09066b runtimes: rename to platforms 2026-01-19 17:41:38 +01:00