This website requires JavaScript.
About
Status
₊˚⊹Discord
Explore
Register
Sign in
Reference in a new issue
Blame
Copy permalink
Reference in a new issue
Copy permalink
DKB
/
zml
Star
0
Fork
You've already forked zml
0
mirror of
https://github.com/zml/zml.git
synced
2026-11-16 19:11:09 +00:00
Code
Issues
23
Releases
1
Packages
Activity
raphael/decoder-buffer
zml
/
docs
/
howtos
/
profiling.md
375 lines
11 KiB
Raw
Permalink
Normal View
History
zml/profiling: unify profiling flow and tracing (#443) This PR modernizes ZML profiling in four areas: - replaces the old C++ `xspace_to_perfetto` tool with a native Zig exporter built on upb - integrates trace export directly into `zml/profiler.zig` - introduces a scoped tracing API in `zml.tracer` and wires it through examples - adds platform-specific annotation support and Bazel-native profiler wrappers for CUDA and ROCm The result is a simpler profiling flow, better large-trace behavior, and a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
platforms/neuron: introduce NKI stack with SDPA kernel (#528) **Summary** This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the LLM attention path. The main goal is to make Neuron attention run through NKI kernels. It adds the compiler/runtime plumbing needed to embed NKI kernels as Neuron custom-native calls, then uses that path for SDPA-style prefill/decode attention. Paged Attention will come later. On inf2.8xlarge it runs : - 25 tok/s for Llama 3.1 8B BF16 - 63.5 tok/s for LFM2.5-1.2B-Instruct BF16 - 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
zml/profiling: unify profiling flow and tracing (#443) This PR modernizes ZML profiling in four areas: - replaces the old C++ `xspace_to_perfetto` tool with a native Zig exporter built on upb - integrates trace export directly into `zml/profiler.zig` - introduces a scoped tracing API in `zml.tracer` and wires it through examples - adds platform-specific annotation support and Bazel-native profiler wrappers for CUDA and ROCm The result is a simpler profiling flow, better large-trace behavior, and a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
platforms/neuron: introduce NKI stack with SDPA kernel (#528) **Summary** This PR update to Neuron v2.30, introduces the Neuron NKI stack into ZML and wires it into the LLM attention path. The main goal is to make Neuron attention run through NKI kernels. It adds the compiler/runtime plumbing needed to embed NKI kernels as Neuron custom-native calls, then uses that path for SDPA-style prefill/decode attention. Paged Attention will come later. On inf2.8xlarge it runs : - 25 tok/s for Llama 3.1 8B BF16 - 63.5 tok/s for LFM2.5-1.2B-Instruct BF16 - 19 tok/s for Qwen3.5-9B BF16
2026-06-16 10:57:00 +02:00
zml/profiling: unify profiling flow and tracing (#443) This PR modernizes ZML profiling in four areas: - replaces the old C++ `xspace_to_perfetto` tool with a native Zig exporter built on upb - integrates trace export directly into `zml/profiler.zig` - introduces a scoped tracing API in `zml.tracer` and wires it through examples - adds platform-specific annotation support and Bazel-native profiler wrappers for CUDA and ROCm The result is a simpler profiling flow, better large-trace behavior, and a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00