This is the non-controversial part of #262
It contains:
* the new `zig_srcs` helper,
* bug fix related to sources in rules_zig,
* cleanups of zml/BUILD.bazel
* test fixes in async/coro.zig (those tests weren't run in CI but now
are)
* disabled nemo and yaml models loader (don't compile anymore since
0.14.0 bump)
The first stack of each new coroutines were holding not zeroed and hold
random memory data, which get popped in rbp during first coroutine stack
swap.
Which made unwinding random or even SIGSEGV when _Unwind_Backtrace was
used (during C++ exception for instance).
Zeroing the first part of the stack makes unwinding stop gracefully when
it encounters $rbp=0x00.
currently using async.LogFn inside async runtime itself panics.
So I'm avoiding that by forcing async.zig logging to go through
std.log.defaultLog which is synchronous.
you can reproduce the crash by changing log level to .debug
# HF tokenizers & Llama improvements
1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example
## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers
`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.
To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.
## 2. examples/llama
Changes to the llama example:
- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
- single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
Rework the libxev epoll backend to play nice with zigcoro based
scheduler.
In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.
Should have a slight performance increase, too.
This PR merges zigcoro inside the repo and reworks its internals to play
well with `stdx.meta`.
It also adds `Channel.try_{send,recv}` methods as well as dynamically
sized channels (with a `comptime` capacity).
Although not used, it also introduces a `PoolStackAllocator` for
allocating coroutine stacks from a pool
all code contributed by @steeve
* add intrusive queue
* change the constructor of Channel with default AsyncThread executor
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.
API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)
Mostly, also, it's pretty fast.
This one is a chonky.
This commit adds a new `stdx` module with a few utility functions. It is
very far reaching. Forgive the size.
Also, clean the logs in the executables.
In an async scenario, it is very easy to footgun yourself and block the
reactor, especially with Events.
Reintroduce pjrtx that deals with the specifics.
This implements the `async` module using a thread pool. This is now the
default.
Implementations can be swapped using the
`@zml//async:impl=[threaded,zigcoro]` flag.
Because io_uring is not enabled in the default seccomp profile, some
container implementations will block on io_uring calls. Use the threaded
implementation to work around this, for now.
`asyncc` is arguably nicer to write than `async_`, so use that.
Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).
pjrtx.zig was mean to be a ZML specific wrapper over pjrt.zig.
But pjrt.zig in its current form is pretty usable without wrapper,
the extra layer of indirection is IMHO confusing because it mostly
transfer the calls directly to pjrt.zig but make some call suspending.
I moved the few helpers left in pjrtx.zig where they are used, ie
`pjrtx.Buffer.bufferFromHostBuffer`-> in `buffer.Buffer.from`
`pjrtx.Client.compile` -> in `module.compileModuleToPjrtExecutable`
`pjrtx.Event.await` -> `zml.Platform.awaitEvent()`
This also fix a bug in `pjrtx.Event.await` which was blocking for the
event before suspending for the event.