This PR bumps rules_zig to add command line to the zig compilation in
order to enable debug and release configs from the bazel command line
(and bazelrc).
Most notably, this is useful for passing `-mcpu` flags to Zig
(especially for SIMD support like AVX512).
While we're there, add `stdx.time.Duration.hz()` and
`stdx.math.roundeven()`.
This enables making sure versions are the same across the board.
Also, import `triton` repo for later use.
---------
Co-authored-by: Hugo Mano <hugo@zml.ai>
This PR updates the `rules_cc` dependency from `0.1.0` to `0.1.1` and
the Bazel version from `8.0.1` to `8.1.1`.
### Changes:
1. Updated `rules_cc` version in `zml/MODULE.bazel` from 0.1.0 to 0.1.1:
```bazel
bazel_dep(name = "rules_cc", version = "0.1.1")
```
2. Updated the Bazel version in `zml/examples/.bazelversion` from
`8.0.1` to `8.1.1`.
### Context:
1. Initially, running `bazel run -c opt //mnist` failed due to a Bazel
version mismatch (required `8.0.1` but the currently brew-installable
version is `8.1.1`). This was resolved by updating `.bazelversion`.
2. After fixing the Bazel version, the build failed again this time due
to a yanked version of `rules_cc` (`0.1.0`).
> ERROR: Error computing the main repository mapping: Yanked version
detected in your resolved dependency graph: rules_cc@0.1.0, for the
reason: rules_cc 0.1.0 is yanked due to incompatible change (prematurely
removing cc_proto_library from defs.bzl), please upgrade to 0.1.1.
> Yanked versions may contain serious vulnerabilities and should not be
used. To fix this, use a bazel_dep on a newer version of this module. To
continue using this version, allow it using the --allow_yanked_versions
flag or the BZLMOD_ALLOW_YANKED_VERSIONS env variable.
3. Updating `rules_cc` to `0.1.1` resolved this issue.
### Testing:
1. Verified the update by running:
```zsh
git clone https://github.com/jaco-bro/zml
cd zml/examples
bazel run -c opt //mnist
```
2. The build and execution succeeded without errors.
```
info(mnist): ✅ RECOGNIZED DIGIT:
+-------------+
| ### |
| # # |
| # # |
| # # |
| # # |
| # # |
| ### |
+-------------+
```
# HF tokenizers & Llama improvements
1. Added HuggingFace tokenizer and SentencePiece bindings, removed old
tokenizers
2. Improved Llama example
## 1. zml/tokenizer: add HuggingFace tokenizers binding, remove old
tokenizers
`zml.Tokenizer` now uses the new HuggingFace tokenizer bindings when
loading a `tokenizer.json`.
To move fast, we dropped the `tinyllama`, `gguf`, and `sentencepiece`
tokenizers.
## 2. examples/llama
Changes to the llama example:
- config.json support -> less cmdline flags
- switched to [zig-clap](https://github.com/Hejsil/zig-clap) for cmdline
args parsing
- using HuggingFace tokenizer
- re-worked and simplified the code
- single token prediction
- removed esoteric models from BUILD.bazel
- default container now uses Llama-3.2-1B-Instruct
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
Co-authored-by: Corentin Godeau <corentin.godeau@zml.ai>
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
The reason for the upgrade is because the //:lock target of the apt
repository checks for bazel marker files to write the lockfile at the
right relative path from the root module.
In 0.3.8, only WORKSPACE and WORKSPACE.bazel files were checked as
marker files, and since we are full bzlmod with a single MODULE.bazel
marker file, the run target would fail like so:
```bash
> bazel run @neuron_bookworm//:lock
Writing lockfile to runtimes/neuron/packages.lock.json
Error: neither WORKSPACE nor WORKSPACE.bazel file was found
```
0.4.2 brings that support.
# Tests
I only tested `cd examples; bazel run //mnist:image` to validate that it
still works. Let me know what else is needed.
* Bump XLA to 20250122.0-cc075be
* Use nvptx compiler and nvlink with nvjitlink support
* Warn user if cuda path detected in LD_LIBRARY_PATH
* Revert CUDA sandbox fix added here: 6d67fcf and fixed by openxla/xla@a04b262
Rework the libxev epoll backend to play nice with zigcoro based
scheduler.
In the process, load the PJRT plugins using `callBlocking` because it
can a bit slow sometimes.
Should have a slight performance increase, too.
This PR is a collection of fixes related to CUDA and ROCm sandbox
discovery.
In the process, update the epoll libxev patch which caused 100% CPU on
linux.
While fixing CUDA add `xla_gpu_enable_latency_hiding_scheduler` as per
NVIDIA's recommendations.
Finally, bump cuDNN to 9.6.0 while we're here.
---------
Co-authored-by: Corentin Godeau <corentin@zml.ai>
This PR merges zigcoro inside the repo and reworks its internals to play
well with `stdx.meta`.
It also adds `Channel.try_{send,recv}` methods as well as dynamically
sized channels (with a `comptime` capacity).
Although not used, it also introduces a `PoolStackAllocator` for
allocating coroutine stacks from a pool
This PR adds support for AWS Neuron / Trainium 1 / Inferentia 2.
It can be enabled via the `--@zml//runtimes:neuron=true` flag.
Everything should work as expected.
---------
Co-authored-by: Corentin Godeau <corentin@zml.ai>
- workspace: new module versions added to Bazel registry:
- deps/xla: 4663f04d26 (20241025.0-4663f04)
This version introduces an upgraded overlay with Python repository initialization for the TSL extension. The changes were necessitated by this XLA commit: 19b4ecbe98 (diff-df5471accc) , which triggers a repository call. Python's hermetic version is provided via the .bazelrc file.
- deps/stablehlo: f7f8e4e352 (20241021.0-1c0b606)
- deps/llvm: f7f8e4e352 (20241022.0-6c4267f)
This version includes an upgraded module declaration with rules_python dependency added, as LLVM prepares its overlay for Bazel v8. Relevant LLVM changes here: 3903cb4695
- deps/runtimes: use PJRT artifacts v3.0.0 (https://github.com/zml/pjrt-artifacts/releases/tag/v3.0.0)
- workspace: align used versions of rules_zig in the Bazel dependency graph
- pjrt: expose getPluginAttributes and getPluginAttribute private APIs and stablehloCurrentVersion public API
- pjrt: updated customCallRegistry function with the new FFI provided by the PJRT API GPU extension
Minimal support for this feature was added, as this is not the main focus of the current PR. See related changes in XLA here: ef85a7bcc3
- dialects/mlir/stablehlo: expose getCurrentVersion API and fix broken stablehloVersionFromCompatibilityRequirement API
- zml/pjrtx: serialize IR to the PJRT plugin's supported HLO version. If the plugin attribute is unavailable, it falls back to the StableHLO's compatibility requirement version.
- zml/tensor: tests using batching dims for gather and scatter are now re-enabled
This mostly reworks the runtime to enable proper cross thread signaling
for `callBlocking`.
API changes include:
- `async_ -> asyncc`
- `await_ -> awaitt`
- proper type inference for non generic arguments even for generic
functions
- bump `libxev`
- use `epoll` on linux by default (can be comptime overridden through
`xev_options.linux_backend` in the root module)
Mostly, also, it's pretty fast.
Since we were using an "old" version of XLA we where missing the
batching dims stablehlo attribute.
This was preventing the usage of batching in the gather operator.
Since I had already wrote the batching test before hand and was just
ignoring the compilation error,
this PR just bump xla and remove the error suppression code in tests.