0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
zml/AGENTS.md
Corentin Godeau 08391af750
zml/Compiler: remove conditionals from command buffer (#913)
Due to recent changes in XLA (introduced in
3cf18a01c7), the way child are
counted to decide if a conditional is expressed as a graph conditional
or kept outside has been changed.

With the new corrected way, the body of the conditional used in the
paged attention StableHLO backend was recorded as a child graph. The
problem is that when we are sharded, there are collectives inside this
child graph and XLA adds event wait/record around the actual collective
kernel. This is explicitly forbidden because
https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/cuda-graphs.html#conditional-node-body-graph-requirements
states:

> The graph can only contain kernel nodes, empty nodes, memcpy nodes,
memset nodes, child graph nodes, and conditional nodes.

The temporary solution is to remove `CONDITIONAL` from the
`xla_gpu_enable_command_buffer` XLA flag. In practice this reverts to
the old behavior we had when the counting was done differently.

Long term, I don't have a solution yet and. There is an environment
variable that solves it
[NCCL_GRAPH_MIXING_SUPPORT](https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/env.html#nccl-graph-mixing-support)
but I don't know if it's safe to disable it.

Maybe we could also check if the events introduced by NCCL are actually
required.
2026-10-08 12:33:34 +02:00

3.3 KiB

Repository Guidelines

Project Structure & Module Organization

ZML is a Zig project built with Bazel. Core library code lives in zml/, with IO under zml/io/, tokenizers under zml/tokenizer/, attention implementations under zml/attention/, and MoE support under zml/moe/. Shared Zig utilities are in stdx/; FFI and C helpers are in ffi/ and upb/. Bazel rules and workspace helpers live in bazel/ and third_party/. Runnable examples are in examples/, while user-facing documentation is in docs/. Keep tests close to implementation files unless a package already has a separate test file.

Build, Test, and Development Commands

  • bazel build //zml: builds the main ZML library.
  • bazel test //zml:test: runs the core Zig test target.
  • bazel test //stdx:test: runs shared utility tests.
  • bazel test //zml/tokenizer:test: runs tokenizer tests.
  • bazel run //examples/mnist: runs the quick MNIST smoke test.
  • bazel run //examples/llm -- --model=hf://... --prompt="...": runs the LLM example.
  • bazel build //... and bazel test //...: use before broad or shared changes.
  • ./tools/buildifier.sh: formats Bazel/Starlark files.
  • echo "$(bazel info output_base)/$(bazel cquery --output=files "$(bazel cquery "filter('zig_toolchain', deps(//zml))" 2>/dev/null | cut -d' ' -f1 | head -n1)" 2>/dev/null | rg "/lib\$")/std": find the current Zig standard library used by Bazel.

Append platform flags when relevant, for example --@zml//platforms:cuda=true, --@zml//platforms:rocm=true, --@zml//platforms:tpu=true, or --@zml//platforms:cpu=false.

Coding Style & Naming Conventions

Follow the Zig style guide and format Zig code with zig fmt; CI checks zig fmt --check outside third_party/. Use four-space indentation. Prefer const x: Foo = .{ .bar = 1 };, pub fn method(self: Foo), and module imports such as const foo = @import("foo.zig"); foo.bar(). Use PascalCase for types and lowerCamelCase for functions, fields, and locals. Add comments only for non-obvious invariants, ownership, tensor shape semantics, or platform behavior.

Testing Guidelines

Tests are generally inline Zig test "..." {} blocks near the code they cover. Use descriptive test names and narrow Bazel targets first, then broaden when touching shared APIs. For tensor or graph changes, cover shape/tag inference and runtime behavior when practical. CI builds all targets and runs tests on CPU, CUDA, and ROCm where available.

Commit & Pull Request Guidelines

Recent history uses scoped, imperative commit subjects such as zml/tensor: add Tensor.onMemory() or workspace: use latest version of upstreamable rules_zig. Keep subjects concise and mention the affected area first. Pull requests should include a short description, relevant linked issue, platform impact if any, and the exact bazel build or bazel test commands run. Include screenshots only for docs or UI-visible changes.

Quirks

If during build, you encounter an AccessDenied error, the solution is not to use the --repo_env bazel argument to override the zig cache. The solution is to run sudo chmod -R 777on /tmp/zig-cache and /var/tmp/zig-cache. This is due to the fact that all users share the same zig cache directory, and bazel runs as a different user than the one who created the cache.