- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`
Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
This enables making sure versions are the same across the board.
Also, import `triton` repo for later use.
---------
Co-authored-by: Hugo Mano <hugo@zml.ai>
Modern bert implementation from Victor: @vctrmn
Tested with https://huggingface.co/answerdotai/ModernBERT-base
I started from his PR #149 and cleaned up a bit the code to make it more
what I consider "idiomatic ZML" even if it's still a bit in flux since
we are so young.
---------
Co-authored-by: vctrmn <victormoin.vm@gmail.com>
Co-authored-by: Rene Schallner <rene@zml.ai>
This is a follow up on #162
* restore homemade tokenizer, integrate it with the new zml/tokenizer
library. This will only be used for working with tinyllama, I could also
make it work to support GGUF if needed (couple lines of code).
* this allows us to use the original tinyllama files from karpathy repo
which generates actual sentences. This is important when working on
modeling because subtle bugs can degrade the output quality, which is
not detectable if the output is crap to begin with.
* fix casing of zml/tokenizer api: fromFile, tokenToId. Let's stick to
the Zig style guide. Function written in Rust follows rust convention,
but the Zig bindings should follow Zig convention.
* make tokenizer/main.zig a proper tokenizer test tool reusing
llama/test_tokenizer.zig
~~use native_binary instead of cc_binary~~
Reverted because apparently native_binary break the runfiles and
therefore the cuda sandbox
# HF tokenizers & Llama improvements
1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example
## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers
`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.
To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.
## 2. examples/llama
Changes to the llama example:
- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
- single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
Rework the libxev epoll backend to play nice with zigcoro based
scheduler.
In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.
Should have a slight performance increase, too.
llama 3.2 1B and 3B do not have lm_head.
I've added transposing over the embed_tokens as a replacement and set
lm_head as optional for backward compatibility with the other models.
I also added both repositories+executions to bazel and updated the
README.
This PR adds support for AWS Neuron / Trainium 1 / Inferentia 2.
It can be enabled via the `--@zml//runtimes:neuron=true` flag.
Everything should work as expected.
---------
Co-authored-by: Corentin Godeau <corentin@zml.ai>
- workspace: new module versions added to Bazel registry:
- deps/xla: 4663f04d26 (20241025.0-4663f04)
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
- deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)
- deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
- deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled