0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

8 commits

Author SHA1 Message Date
Benjamin Rabier
c58d81a7b3
zml/exe: TypedExe (#697)
in LLMD and //examples/llm, we compile multiple executable, re-use the
args & results. We do all of this manually, in addition we don't have
any type-safety on the arguments/outputs which is a bit too easy to
misuse IMHO.

So adding two abstractions:
- Runner: Packs the args, exe and results together to avoid calling
`args.set`, `exe.run` and `results.fill` with similar arguments. Today
it owns the Exe. It doensn't strictly needs it, but it's simpler and
works for us for now.
- TypedExe: Type-safe version of an Exe, expecting a single input &
ouptut to have keyed arguments to have fully unambiguous arguments &
results mapping.

I've updated //example/llm with this new pattern.

Also adding an explicit `zml.module.CompileError`.
2026-08-18 16:36:38 +02:00
Hugo Mano
13cee57bf2
zml/profiling: unify profiling flow and tracing (#443)
This PR modernizes ZML profiling in four areas:

- replaces the old C++ `xspace_to_perfetto` tool with a native Zig
exporter built on upb
- integrates trace export directly into `zml/profiler.zig`
- introduces a scoped tracing API in `zml.tracer` and wires it through
examples
- adds platform-specific annotation support and Bazel-native profiler
wrappers for CUDA and ROCm

The result is a simpler profiling flow, better large-trace behavior, and
a more consistent experience across Linux CUDA, Linux ROCm, and macOS.
2026-05-11 09:52:11 +02:00
Guillaume Wenzek
039a638aeb
examples/llm: cleanups (#512)
I'm doing a pass on the example code we have.

* llama/model.zig, qwen/model.zig: change layer_index from being a
tensor to being an integer.
It's clearer this way what is happening. Note that lfm doesn't have the
issue because the layer index is passed everywhere.

* tokenizePrompt: the code was doing unnecessary copies of data, I added
some helper to tokenizer.encoder to streamline the code by removing the
extra copies and allocations.

* `.stdout().writer() -> .stdout().writerStreaming()` this one is weird
but apparently using stdout by position isn't always allowed, in
particular when running the exe from within SublimeText.

* runDecode -> show case bake
2026-04-29 10:38:19 +02:00
Hugo Mano
f80f925409
examples/llm: add profiling option (#497)
This allow use to easily trace what going on
2026-04-22 16:23:36 +02:00
Benjamin Rabier
9fa7768808
examples/llm: better single prompt UX + --topk (#438)
- Same UX as the chat
- Add `--topk` argument
2026-03-26 15:58:03 +01:00
Benjamin Rabier
0d0d5020cf
example/llm: Further simplifications (#433) 2026-03-25 10:16:26 +01:00
Benjamin Rabier
8de2c42fe6
examples/llm: Simplifications (#430) 2026-03-24 18:16:25 +01:00
Benjamin Rabier
81299c67c8
examples/llm: Common model example (#425) 2026-03-24 16:55:29 +01:00