This website requires JavaScript.
About
Status
₊˚⊹Discord
Explore
Register
Sign in
Reference in a new issue
Blame
Copy permalink
Reference in a new issue
Copy permalink
DKB
/
zml
Star
0
Fork
You've already forked zml
0
mirror of
https://github.com/zml/zml.git
synced
2026-11-16 19:11:09 +00:00
Code
Issues
23
Releases
1
Packages
Activity
raphael/decoder-buffer
zml
/
ffi
/
zig_allocator.zig
26 lines
1,016 B
Raw
Permalink
Normal View
History
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522) This change fixes profile timestamp handling so saved xplane.pb data is post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the Zig/C++ FFI used by the profile export bridge. Before this, we were writing raw profiler output from the PJRT extension directly to `profiling.xplane.pb`, which bypassed XLA’s single-host post-processing step done by JAX that we can find here: https://github.com/google/tsl/blob/efac222ebfdbd682d3a407ef165865c6b6759e2d/tsl/profiler/lib/profiler_session.cc#L100 That made ZML behave differently from JAX, including wrong timeline origins in downstream viewers like XProf. To fix that, we now run `tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the protobuf, and we treat that post-processed xplane.pb as the source for subsequent Perfetto conversion. We also register start/stop timestamps in `zml/profiler.zig` so they match XLA’s expectations for session timestamps with `real` clock. While touching this path, I tightened the FFI boundary as well. Error text returned from the xspace export bridge is now allocated with the provided Zig allocator instead of malloc, C receives a pointer to the `zig_allocator wrapper` to avoid lifetime issues from passing allocator state by value, and error propagation in `tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly instead of building std::string error objects just to copy them back out. This removes mixed allocation ownership between C++ and Zig and fixes the allocator lifetime bug that could crash across the FFI boundary.
2026-05-05 14:52:15 +02:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
workspace: zig 0.15 (#316) Much of the work was done by @gwenzek Sadly we drop the torch loader in the process, to be added back later ! --------- Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522) This change fixes profile timestamp handling so saved xplane.pb data is post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the Zig/C++ FFI used by the profile export bridge. Before this, we were writing raw profiler output from the PJRT extension directly to `profiling.xplane.pb`, which bypassed XLA’s single-host post-processing step done by JAX that we can find here: https://github.com/google/tsl/blob/efac222ebfdbd682d3a407ef165865c6b6759e2d/tsl/profiler/lib/profiler_session.cc#L100 That made ZML behave differently from JAX, including wrong timeline origins in downstream viewers like XProf. To fix that, we now run `tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the protobuf, and we treat that post-processed xplane.pb as the source for subsequent Perfetto conversion. We also register start/stop timestamps in `zml/profiler.zig` so they match XLA’s expectations for session timestamps with `real` clock. While touching this path, I tightened the FFI boundary as well. Error text returned from the xspace export bridge is now allocated with the provided Zig allocator instead of malloc, C receives a pointer to the `zig_allocator wrapper` to avoid lifetime issues from passing allocator state by value, and error propagation in `tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly instead of building std::string error objects just to copy them back out. This removes mixed allocation ownership between C++ and Zig and fixes the allocator lifetime bug that could crash across the FFI boundary.
2026-05-05 14:52:15 +02:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
workspace: zig 0.15 (#316) Much of the work was done by @gwenzek Sadly we drop the torch loader in the process, to be added back later ! --------- Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522) This change fixes profile timestamp handling so saved xplane.pb data is post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the Zig/C++ FFI used by the profile export bridge. Before this, we were writing raw profiler output from the PJRT extension directly to `profiling.xplane.pb`, which bypassed XLA’s single-host post-processing step done by JAX that we can find here: https://github.com/google/tsl/blob/efac222ebfdbd682d3a407ef165865c6b6759e2d/tsl/profiler/lib/profiler_session.cc#L100 That made ZML behave differently from JAX, including wrong timeline origins in downstream viewers like XProf. To fix that, we now run `tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the protobuf, and we treat that post-processed xplane.pb as the source for subsequent Perfetto conversion. We also register start/stop timestamps in `zml/profiler.zig` so they match XLA’s expectations for session timestamps with `real` clock. While touching this path, I tightened the FFI boundary as well. Error text returned from the xspace export bridge is now allocated with the provided Zig allocator instead of malloc, C receives a pointer to the `zig_allocator wrapper` to avoid lifetime issues from passing allocator state by value, and error propagation in `tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly instead of building std::string error objects just to copy them back out. This removes mixed allocation ownership between C++ and Zig and fixes the allocator lifetime bug that could crash across the FFI boundary.
2026-05-05 14:52:15 +02:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00