Much of the work was done by @gwenzek
Sadly we drop the torch loader in the process, to be added back later !
---------
Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
This is the non-controversial part of #262
It contains:
* the new `zig_srcs` helper,
* bug fix related to sources in rules_zig,
* cleanups of zml/BUILD.bazel
* test fixes in async/coro.zig (those tests weren't run in CI but now
are)
* disabled nemo and yaml models loader (don't compile anymore since
0.14.0 bump)
# HF tokenizers & Llama improvements
1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example
## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers
`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.
To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.
## 2. examples/llama
Changes to the llama example:
- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
- single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
Rework the libxev epoll backend to play nice with zigcoro based
scheduler.
In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.
Should have a slight performance increase, too.
This PR merges zigcoro inside the repo and reworks its internals to play
well with `stdx.meta`.
It also adds `Channel.try_{send,recv}` methods as well as dynamically
sized channels (with a `comptime` capacity).
Although not used, it also introduces a `PoolStackAllocator` for
allocating coroutine stacks from a pool
This implements the `async` module using a thread pool. This is now the
default.
Implementations can be swapped using the
`@zml//async:impl=[threaded,zigcoro]` flag.
Because io_uring is not enabled in the default seccomp profile, some
container implementations will block on io_uring calls. Use the threaded
implementation to work around this, for now.
`asyncc` is arguably nicer to write than `async_`, so use that.
Also, add an explicit `Generic` version that can work with `anytype`
functions (but on which type inference will be disabled).
pjrtx.zig was mean to be a ZML specific wrapper over pjrt.zig.
But pjrt.zig in its current form is pretty usable without wrapper,
the extra layer of indirection is IMHO confusing because it mostly
transfer the calls directly to pjrt.zig but make some call suspending.
I moved the few helpers left in pjrtx.zig where they are used, ie
`pjrtx.Buffer.bufferFromHostBuffer`-> in `buffer.Buffer.from`
`pjrtx.Client.compile` -> in `module.compileModuleToPjrtExecutable`
`pjrtx.Event.await` -> `zml.Platform.awaitEvent()`
This also fix a bug in `pjrtx.Event.await` which was blocking for the
event before suspending for the event.