This website requires JavaScript.
About
Status
₊˚⊹Discord
Explore
Register
Sign in
Reference in a new issue
Blame
Copy permalink
Reference in a new issue
Copy permalink
DKB
/
zml
Star
0
Fork
You've already forked zml
0
mirror of
https://github.com/zml/zml.git
synced
2026-11-16 19:11:09 +00:00
Code
Issues
23
Releases
1
Packages
Activity
gw/flat_bufferized
zml
/
stdx
/
stdx.zig
36 lines
1.3 KiB
Raw
Permalink
Normal View
History
zml: loadById (#322) We already assign a unique tensor id to each tensor object created by a BufferStore. But we weren't using the id to load find the corresponding buffer in the store, instead relying on the name. This is allow to decouple the shape of the model in the struct vs the shape of the model inside the safetensor file. I also moved the id logic out of Tensor and into BufferStore. I also updated llama/main.zig to use the new logic. Bonus: * fix to `zml.call` tag hashing leading to different MLIR for every recompile. * expose Cpu device count to control the number of CPU "devices"
2025-09-11 10:28:16 +02:00
workspace: zig 0.15 (#316) Much of the work was done by @gwenzek Sadly we drop the torch loader in the process, to be added back later ! --------- Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
async: rework async runtime to use coroutines and cross thread signaling (#100) This mostly reworks the runtime to enable proper cross thread signaling for `callBlocking`. API changes include: - `async_ -> asyncc` - `await_ -> awaitt` - proper type inference for non generic arguments even for generic functions - bump `libxev` - use `epoll` on linux by default (can be comptime overridden through `xev_options.linux_backend` in the root module) Mostly, also, it's pretty fast.
2024-11-22 17:56:22 +01:00
Restore tinyllama models (#173) This is a follow up on #162 * restore homemade tokenizer, integrate it with the new zml/tokenizer library. This will only be used for working with tinyllama, I could also make it work to support GGUF if needed (couple lines of code). * this allows us to use the original tinyllama files from karpathy repo which generates actual sentences. This is important when working on modeling because subtle bugs can degrade the output quality, which is not detectable if the output is crap to begin with. * fix casing of zml/tokenizer api: fromFile, tokenToId. Let's stick to the Zig style guide. Function written in Rust follows rust convention, but the Zig bindings should follow Zig convention. * make tokenizer/main.zig a proper tokenizer test tool reusing llama/test_tokenizer.zig ~~use native_binary instead of cc_binary~~ Reverted because apparently native_binary break the runfiles and therefore the cuda sandbox
2025-02-03 12:06:33 +01:00
stdx.fmt: format slices (#231) this is an improved version from what was implemented in "prettyPrinter" but I had to dig a bit into the fmt internal to achieve it.
2025-05-19 22:39:56 +02:00
runtimes/cuda: sandbox CUDA dependencies (#275) This PR introduces CUDA sandboxing. This essentially means: - Removing all CUDA related dependencies off the leaf binary. - Sandboxing the whole CUDA dependency graph. - Mark dlopen direct dependencies of the PJRT shared library as NEEDED. - Set the RPATH to the sandbox for all libraries that have sandbox dependencies. - Load the PJRT plugin from the sandbox. And in the case of CUDA specifically for now: - Enable usage of weak CUDA symbols without linking against CUDA directly.
2025-07-16 11:33:48 +02:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
zml: add stdx, rework async signature inference (#89) This one is a chonky. This commit adds a new `stdx` module with a few utility functions. It is very far reaching. Forgive the size. Also, clean the logs in the executables.
2024-11-15 17:12:07 +01:00
async: vendor zigcoro and unify APIs (#152) This PR merges zigcoro inside the repo and reworks its internals to play well with `stdx.meta`. It also adds `Channel.try_{send,recv}` methods as well as dynamically sized channels (with a `comptime` capacity). Although not used, it also introduces a `PoolStackAllocator` for allocating coroutine stacks from a pool
2025-01-14 19:33:16 +01:00
zml/tokenizer: use HuggingFace Tokenizers and SentencePiece & Llama improvements (#162) # HF tokenizers & Llama improvements 1. Added HuggingFace tokenizer and SentencePiece bindings, removed old tokenizers 2. Improved Llama example ## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old tokenizers `zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when loading a `tokenizer.json`. To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece` tokenizers. ## 2. examples/llama Changes to the llama example: - config.json support -> less cmdline flags - switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline args parsing - using HuggingFace tokenizer - re-worked and simplified the code - single token prediction - removed esoteric models from BUILD.bazel - default container now uses Llama-3.2-1B-Instruct --------- Co-authored-by: Steeve Morin <steeve@zml.ai> Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-01-24 18:25:57 +01:00
pjrt: add ffi bindings for custom calls (#223) --------- Co-authored-by: Hugo Mano <hugo@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-04-08 16:28:39 +02:00
Bump rules_zig, add `zig_srcs` targets, cleanup build (#267) This is the non-controversial part of #262 It contains: * the new `zig_srcs` helper, * bug fix related to sources in rules_zig, * cleanups of zml/BUILD.bazel * test fixes in async/coro.zig (those tests weren't run in CI but now are) * disabled nemo and yaml models loader (don't compile anymore since 0.14.0 bump)
2025-07-15 14:18:15 +02:00
pjrt: add ffi bindings for custom calls (#223) --------- Co-authored-by: Hugo Mano <hugo@zml.ai> Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-04-08 16:28:39 +02:00
zml: loadById (#322) We already assign a unique tensor id to each tensor object created by a BufferStore. But we weren't using the id to load find the corresponding buffer in the store, instead relying on the name. This is allow to decouple the shape of the model in the struct vs the shape of the model inside the safetensor file. I also moved the id logic out of Tensor and into BufferStore. I also updated llama/main.zig to use the new logic. Bonus: * fix to `zml.call` tag hashing leading to different MLIR for every recompile. * expose Cpu device count to control the number of CPU "devices"
2025-09-11 10:28:16 +02:00
pinToCore
2025-10-06 11:32:44 +02:00