* buffer/hostbuffer utils
* more precise f32 -> bf16 conversion
* visit should not read irrelevant fields
* HostBuffer.data -> HostBuffer.bytes()
* zml.aio loadBuffers: type inference
* store expected input shapes in the executable
* zml.meta.visit support std.BoundedArray
* stdx.json.TaggedUnion
* protection against undefined float hyperparams
* avoid meta.TagPayload which can be quite slow
* improve logging
This PR bumps rules_zig to add command line to the zig compilation in
order to enable debug and release configs from the bazel command line
(and bazelrc).
Most notably, this is useful for passing `-mcpu` flags to Zig
(especially for SIMD support like AVX512).
While we're there, add `stdx.time.Duration.hz()` and
`stdx.math.roundeven()`.
This is a follow up on #162
* restore homemade tokenizer, integrate it with the new zml/tokenizer
library. This will only be used for working with tinyllama, I could also
make it work to support GGUF if needed (couple lines of code).
* this allows us to use the original tinyllama files from karpathy repo
which generates actual sentences. This is important when working on
modeling because subtle bugs can degrade the output quality, which is
not detectable if the output is crap to begin with.
* fix casing of zml/tokenizer api: fromFile, tokenToId. Let's stick to
the Zig style guide. Function written in Rust follows rust convention,
but the Zig bindings should follow Zig convention.
* make tokenizer/main.zig a proper tokenizer test tool reusing
llama/test_tokenizer.zig
~~use native_binary instead of cc_binary~~
Reverted because apparently native_binary break the runfiles and
therefore the cuda sandbox
# HF tokenizers & Llama improvements
1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example
## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers
`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.
To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.
## 2. examples/llama
Changes to the llama example:
- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
- single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
This PR merges zigcoro inside the repo and reworks its internals to play
well with `stdx.meta`.
It also adds `Channel.try_{send,recv}` methods as well as dynamically
sized channels (with a `comptime` capacity).
Although not used, it also introduces a `PoolStackAllocator` for
allocating coroutine stacks from a pool
* add an arena to the CompilationContext
There was already an allocator in the CompilationContext, this PR makes
it into an arena and expose the arena allocator.
This allows ops that need to allocate to do so without abusing
`std.BoundedArray` with arbitrary bounds, and in a safe way.
For now little ops use it, which I believe should stay the case, because
most of the time you want to know how many tensor you're working with.
It'll also simplify the life of Zig new-comers
this include some **breaking changes** to `chunkAllowTrailing` and
`split` that are allocating
* upgrade a few `axis_: i64` into `axis_: anytype` to handle tags
* add a few TODO for the next version of the Tensor API
---------
Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
all code contributed by @steeve
* add intrusive queue
* change the constructor of Channel with default AsyncThread executor
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
* compileInternal now works with function not with "module"
* BaseExe now allocates all the memory needed to call `pjrtExecute`
* `Exe` is the type variant of `BaseExe` and enforce the right Zig arg
are passed to `call`. `exe.prepare(model)` don't need to allocate
anymore, the allocation happened during compilation.
* public entry points like `compile`, `compileModel`, `compileFn` are
now at the top of the file
* the caching logic has been mostly inlined inside `compileInternal`
* `compile`, `compileFn`, `Exe`, `BaseExe` which are the user visible
API have been moved to `exe.zig`
* `CompilationContext` and all the compilation logic stays in
`module.zig`
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.
API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)
Mostly, also, it's pretty fast.
This one is a chonky.
This commit adds a new `stdx` module with a few utility functions. It is
very far reaching. Forgive the size.
Also, clean the logs in the executables.