Give each concurrent RunsOn job a unique routing label using the
workflow run ID, run attempt, and job name. This prevents runners
provisioned for one matrix job from being assigned to another compatible
job.
- [Runner
stealing](https://runs-on.com/docs/maintenance/troubleshooting/#runner-stealing)
---
Replace the deprecated image-level `disk: large` setting with
`volume=80gb` on each RunsOn job. Storage belongs to the runner
configuration rather than the custom image definition.
- [RunsOn storage labels](https://runs-on.com/docs/runners/labels/#disk)
- [Custom runners and images](https://runs-on.com/docs/runners/custom/)
Use newer versions of several GitHub Actions. Also bump used Zig
version.
`mlugg/setup-zig` is still using Node.js v20 so warning still here but
only for this action. Check
https://codeberg.org/mlugg/setup-zig/pulls/55 in a near future in order
to update the action.
This PR fixes several Bazel analysis regressions so both of these
commands work reliably:
- `bazel cquery //...`
- `bazel cquery --platforms=@zml//platforms:linux_amd64 //...`
After recent changes and the rebase, repo-wide cquery was failing for
three separate reasons:
- ZLS toolchains were over-constrained by target platform: updated ZLS
toolchain registration to constrain only the exec platform.
This avoids cross-target analysis failures when host-side completion
tooling is analyzed under a foreign target platform.
- OCI packaging targets were analyzed on host platforms where their base
images do not apply: marked Linux OCI image targets as Linux x86_64
compatible only. Also switched the Debian base image pull from Docker
Hub to the GCR mirror to avoid CI fetch failures when resolving
packaging images manifests due to rate limiting
- Triton Python deps were fetched for the host Python platform instead
of the Linux wheel platform the lockfile was generated for: restored
Linux x86_64 wheel resolution for `triton_py_deps` in `MODULE.bazel`.
This prevents macOS host analysis from attempting to fetch a
`triton==3.6.0` wheel for an unsupported host platform.
Also added a step in ci to prevent regression
```
4cpus:
total: 2m56s
ci-cpu: 2m14
ci-neuron: 2m07s
ci-tpu: 1m58s
8cpus:
total: 2m26s
ci-cpu: 1m37s
ci-neuron: 1m47s
ci-tpu: 1m36s
16cpus (mi8-flex):
total: 2m07s
ci-cpu: 1m24s
ci-neuron: 1m26s
ci-tpu: 1m19s
16cpus (c8i-flex):
total: 1m58s
ci-cpu: 1m17s
ci-neuron: 1m15s
ci-tpu: 1m17s
32cpus:
total: 1m48s
ci-cpu: 1m05s
ci-neuron: 1m06s
ci-tpu: 1m02s
```
RunsOn tends to prefer m8i instance (memory-optimzed) rather than c8i
ones for some reason, so forced the family with `c8+c7`, maybe we could
just use `c8`. But not sure if it's worth being that selective. RunsOn
selects the best there is.
Fixes:
- Added `target_compatible_with = ["@platforms//os:linux"],` for neuron
& rocm
- Updated the zig in neuron
- Updated neuron to use Python 3.12, it's at best a mess to handle two
different python version with `build //...`
- renamed `third_party/flashattn/BUILD.bazel` to
`third_party/flashattn/flashattention.BUILD.bazel` as it's not part of
this module.
- Used HTTPS & fallback URL for swig, sometimes we would end up with an
incorrect archive (wrong sha256)
- Using `--@rules_python//python/config_settings:bootstrap_impl=script`
for hermetic Python build, it relied on the system Python otherwise.
- Used a separate pip extension for neuron to avoid the extra pip args
to leak to other installations.
- Fix `zml/testing` `fn autoCall` function
---------
Co-authored-by: Hugo Mano <hugo@zml.ai>
- Deduplicate workflows (introduced by
57fd9cda27)
- Fix linking issue on Linux since upb was introduced
- Fix duplicated `_start` symbol for some tests targets
- Temporarily use `-c opt` instead of `--config=debug` to bypass static
archive bug
Fix#316, close#304
- Bump platform specific Zig code that doesn't get built in CI.
- Restore pytorch loader, using `std.fs.File` instead of `async.File`,
allowing to reuse `std.zip`
- Update `zig fmt` in CI to `0.15.1` and ran it on the codebase
(third_party excluded)
- Remove `stdx.io` which was pointing to remove
`std.io.BufferededReader`.
The Neuron target is still broken though:
`bazel build //runtimes/neuron:libneuronxla`
As well as the Cuda plugin in `--config=debug` mode:
`bazel build --config=debug //runtimes/cuda:zmlxcuda`
Much of the work was done by @gwenzek
Sadly we drop the torch loader in the process, to be added back later !
---------
Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`
Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
Ensure that our Bazel setup for the framework and examples work through
queries, builds and tests.
Also fix broken code and deps in the codebase.
As of today only Neuron is broken (cf:
https://github.com/zml/zml/pull/111)
Run `bazel build //llama` in CI to make sure we don't accidentally break
it.
I'm also building all the examples target.
For reference it took 5 minutes to run this CI.