Hugo Mano
4cf4ee3c14
workspace: xla 20250718.0-6319f0d with ROCm 6.4.1 ( #289 )
...
Note: CUDA plugin is not bumped due to an error with libnvptxcompiler. A
specific update will be done in another commit.
2025-07-22 23:50:38 +02:00
Guillaume Wenzek
ba8ff439ba
protobuf, libxev: remove usingnamespace ( #280 )
...
remove usingnamespace from libxev and zig-protobuf
relevant PRs:
* libxev:
7f803181b1
* zig-protobuf: https://github.com/zml/zig-protobuf/pull/1/files
This means we can now build ZML with `-fincremental` (in debug mode only
and linux x86-64)
---------
Signed-off-by: Guillaume Wenzek <wzk@zml.ai>
2025-07-21 18:03:01 +02:00
Guillaume Wenzek
981b6becc3
remove usingnamespace from asyncio ( #205 )
...
Small PR to remove our last usingnamespace.
Also usingnamespace will be out in 0.15.0:
https://github.com/ziglang/zig/pull/24362
2025-07-21 16:21:05 +02:00
Jean-Baptiste Dalido
a1752e8de9
ci: leverage disk cache and leverage runs-on snapshot feature ( #288 )
2025-07-18 16:30:54 +02:00
Steeve Morin
e13acae80b
runtimes/rocm: fetch libdrm from amdgpu repo, add amdgpu.ids layer ( #287 )
...
Properly fetch libdrm related artifacts from the amdgpu driver repo
instead of ubuntu.
While there, add `amdgpu.ids` file to the ROCm layers.
2025-07-17 22:16:27 +02:00
Steeve Morin
81fa2ec44c
workspace: disable glibc constraint, bump rules_rust, re-enable framepointers ( #286 )
...
Since the rework of runtimes sandboxing, explicitely compiling against a certain glibc version isn't needed anymore.
2025-07-17 18:20:07 +02:00
Corentin Kerisit
32216e8c54
runtimes/rocm: use inprocess lld instead of sandboxing it ( #284 )
...
XLA rencetly added the --xla_gpu_use_inprocess_lld bool flag which lets
use ditch the rocm-llvm//:lld from the sandbox :)
See
d01947de6f
2025-07-17 16:39:00 +02:00
Corentin Kerisit
4362fe323e
workspace: bump rules_zig to 20250714.0-b14a4f1 ( #283 )
...
This dummy fixes the presence of the static-lib in the runfiles of the
final target in case of kind != exe.
2025-07-16 18:01:09 +02:00
Hugo Mano
8066d2abdd
ci: use large disk on runs-on ( #282 )
...
With the latests developments the default disk space (40GB) is too small
for our new needs so we use large disks (80GB).
2025-07-16 15:21:37 +02:00
Corentin Kerisit
4c14652fb7
runtimes/rocm: sandbox ROCm dependencies ( #276 )
...
ROCm counterpart of #275
The most important thing with this change is that ROCm dependencies
(along with the PJRT plugin) must be loaded in the main thread.
This is because some ROCm dependencies (amdhip64) use Thread Local
Storage in the destructor of static C++ instances.
Because all dependencies are unloaded at exit of the main program,
destructors are run on whatever thread exited (in general the main
thread), and any access to the TLS will except values from a different
threads....
2025-07-16 14:41:07 +02:00
Corentin Kerisit
d3dda29db5
runtimes/tpu: sandbox TPU PJRT Plugin ( #278 )
...
It has no dependencies so fairly straighforward
2025-07-16 14:13:22 +02:00
Hugo Mano
fc118e058f
deps/hftokenizers: bump dependency ( #281 )
2025-07-16 13:44:19 +02:00
Corentin Kerisit
12293da6af
runtimes/cpu: sandbox CPU PJRT Plugin ( #279 )
...
Same as TPU, simplified because there are no other NEEDED dependencies.
2025-07-16 13:08:55 +02:00
Corentin Kerisit
8274f9c2c0
runtimes/cuda: sandbox CUDA dependencies ( #275 )
...
This PR introduces CUDA sandboxing.
This essentially means:
- Removing all CUDA related dependencies off the leaf binary.
- Sandboxing the whole CUDA dependency graph.
- Mark dlopen direct dependencies of the PJRT shared library as NEEDED.
- Set the RPATH to the sandbox for all libraries that have sandbox
dependencies.
- Load the PJRT plugin from the sandbox.
And in the case of CUDA specifically for now:
- Enable usage of weak CUDA symbols without linking against CUDA
directly.
2025-07-16 11:33:48 +02:00
Corentin Kerisit
8ca8d069f9
pjrt: warn instead of panic if FFI Extension is not found ( #277 )
...
This is the case for TPU where FFI Extension doesn't not exist.
2025-07-15 14:55:15 +02:00
Corentin Kerisit
57abcb5aab
workspace: bump toolchains_llvm_bootstrapped to 0.2.4 ( #274 )
...
This update is important because it orders correctly the CRT which
contains .init and .fini family sections.
2025-07-15 14:54:57 +02:00
Guillaume Wenzek
ee1163a113
Bump rules_zig, add zig_srcs targets, cleanup build ( #267 )
...
This is the non-controversial part of #262
It contains:
* the new `zig_srcs` helper,
* bug fix related to sources in rules_zig,
* cleanups of zml/BUILD.bazel
* test fixes in async/coro.zig (those tests weren't run in CI but now
are)
* disabled nemo and yaml models loader (don't compile anymore since
0.14.0 bump)
2025-07-15 14:18:15 +02:00
Corentin Kerisit
00ebef1a32
async/coro: make coroutines unwindable ( #273 )
...
The first stack of each new coroutines were holding not zeroed and hold
random memory data, which get popped in rbp during first coroutine stack
swap.
Which made unwinding random or even SIGSEGV when _Unwind_Backtrace was
used (during C++ exception for instance).
Zeroing the first part of the stack makes unwinding stop gracefully when
it encounters $rbp=0x00.
2025-07-12 20:48:37 +02:00
Corentin Kerisit
8d0e6e6348
bazel: add rpath manipulation to patchelf ( #272 )
2025-07-12 20:02:46 +02:00
Hugo Mano
b0168d1c63
workspace: bump xla to version 20250710.0-22ea002 ( #271 )
...
Also:
- Bump XLA deps : `com_github_grpc_grpc` and `com_google_protobuf`
- Inject `rules_ml_toolchain`
- Fix `zig_proto_library` rule
2025-07-11 17:45:57 +02:00
Corentin Godeau
67fc7d4b97
runtimes/cuda: downgrade cuda and cudnn ( #269 )
...
This commit reverts part of https://github.com/zml/zml/pull/238/files
This is required because XLA has a strong dependencies on CUDA 12.8 and
upgrading to 12.9 is impossible due to
https://github.com/NVIDIA/cccl/issues/4967
2025-06-23 13:11:56 +02:00
Corentin Godeau
0330e34788
pjrt: Bind PJRT_Client_CreateUninitializedBuffer ( #266 )
2025-06-19 09:11:53 +02:00
Corentin Godeau
3c2d319d20
zml/pjrt: added binding for PJRT_Device_MemoryStats ( #264 )
2025-06-18 18:32:15 +02:00
Corentin Godeau
a6469229d2
async: use stronger memory ordering ( #268 )
...
This was probably causing a segfault on some specific ordering of
things.
2025-06-18 12:55:36 +02:00
Victor Moin
8a6b4a9bdd
Patch aio.zig loadBuffersWithPrefix arguments ( #265 )
...
Corrects the type annotation for init_args parameter to use the same
conditional type as `loadBuffers` function.
@Corendos
2025-06-16 18:33:34 +02:00
Victor Moin
89344e256b
Add model prefix support ( #226 )
...
## Overview
This PR adds the ability to specify a prefix when loading a model from
safetensors file. This is useful when working with models like
[ModernBERT](https://huggingface.co/answerdotai/ModernBERT-large/tree/main?show_file_info=model.safetensors )
; the changes enable me to use `ModernBertModel` to generate embeddings
instead of the full `ModernBertForMaskedLM` model.
## Test
Changes have been tested with my text embeddings server project :
https://github.com/vctrmn/zig-text-embeddings-inference
2025-06-16 17:41:29 +02:00
Corentin Godeau
989d85be2a
workspace: bump xla to version 20250612.0-6e48cbb ( #263 )
2025-06-16 17:37:57 +02:00
Hugo Mano
b4716dba03
ci: revamp commit workflow ( #246 )
...
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`
Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-06-11 13:30:34 +02:00
Guillaume Wenzek
e15d3a9b90
remove usingnamespace from mlir ( #204 )
2025-06-10 15:51:30 +02:00
Guillaume Wenzek
e4705943a6
remove dead imports ( #261 )
2025-06-10 15:36:38 +02:00
Corentin Godeau
fb0a8e4071
zml/ops: wire the has_side_effect field ( #260 )
2025-06-10 15:11:45 +02:00
Guillaume Wenzek
657d4010a3
rename pjrt BufferType to match Zig and ZML conventions ( #259 )
2025-06-10 14:59:08 +02:00
Victor Moin
a38afbbe51
Fix ModernBERT compilation error by removing print calls ( #258 )
...
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.
### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅ Loaded weights in 397ms
info(modernbert): Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201 627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***
error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```
### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`
```logs
info(modernbert): ✅ Loaded weights in 416ms
info(modernbert): Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅ Loaded weights and compiled model in 2968ms
info(modernbert): Input text: [MASK] is the capital of France.
info(modernbert): ⏱️ Model inference in 347ms
info(modernbert): ✅ Top 5 predictions:
info(modernbert): • score: 0.9755 word: 'Paris' token: 36062
info(modernbert): • score: 0.0067 word: 'Nice' token: 29235
info(modernbert): • score: 0.0063 word: 'It' token: 1147
info(modernbert): • score: 0.0010 word: 'Or' token: 3980
info(modernbert): • score: 0.0010 word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00
Corentin Godeau
b49b64df1b
zml/ops: added customCall ( #253 )
2025-06-09 15:11:08 +02:00
Corentin Godeau
0f6eb5076c
zml/meta: fix map tests ( #257 )
2025-06-09 13:33:17 +02:00
Corentin Godeau
b0cfb64a0f
pjrt: added Executable.getCompiledMemoryStats ( #250 )
2025-06-06 19:03:09 +02:00
Corentin Godeau
959936ffa1
zml/meta.zig: support unions in mapping helpers ( #251 )
2025-06-06 18:20:39 +02:00
Corentin Godeau
8bddaffcb8
mlir/dialect/stablehlo: set defaults to operand_layouts and result_layouts ( #252 )
2025-06-06 18:20:25 +02:00
Corentin Godeau
f323979cb2
zml/buffer: various improvements ( #248 )
...
* `Buffer.from` is blocking by default
* `Buffer.from` takes options to control the loading This includes
awaiting for the buffer or to which memory the buffer is copied
2025-06-06 18:12:49 +02:00
Corentin Godeau
7ac7a7f210
examples/llama: set model.norm.eps correctly ( #256 )
2025-06-06 18:12:25 +02:00
Corentin Godeau
88c627a309
Preliminary work for Custom Call ( #247 )
2025-06-06 11:24:21 +02:00
Corentin Godeau
c75e5d1aa5
modernbert: set embeddings layernorm eps default value ( #255 )
...
I'm not sure about the default value. @vctrmn maybe you know better 😄
2025-06-06 11:20:15 +02:00
Corentin Godeau
41e222f39f
workspace: updated default nvim configuration ( #254 )
...
This requires neovim >0.11.0
2025-06-06 11:20:03 +02:00
Steeve Morin
2502c44c0f
workspace: switch to toolchains_llvm_bootstrapped ( #245 )
...
Notably, this will enable using custom linkers or enable LTO.
---------
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-06-05 21:45:15 +02:00
Guillaume Wenzek
56c71d97c0
zml: remove compile time fields from zml.Bufferized ( #237 )
...
this remove the need to pass `undefined` to `exe.call` when the
corresponding `.forward` function expects integer/float arguments that
have been inlined in the executable.
This was not needed and could just create confusion.
Now you may end up having to pass void to `exe.call` if the executable
doesn't have any runtime argument,
but it's still better than passing `undefined`.
The only place that was using the non-buffer parts of a `zml.Bufferized`
object was the infamous `zml.testing.compileAndCall`.
So I had to introduce a `zml.testing.BufferizedWithArgs` which is only
used for this function.
I put it under the `zml.testing` namespace to make it clear it's only
for this testing utility.
Note that because `compileAndCall` is strongly typed, you generally
don't need to explicitly mention `BufferizedWithArgs`
and the can just write the arguments in place and get help from ZLS.
2025-06-05 18:50:20 +02:00
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f ( #238 )
...
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily
XLA diff:
71c67e2a4f...cb67f2f7ce
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00
Steeve Morin
23c7335592
workspace: bump to zig=0.14.1 ( #242 )
...
In the process, add a case where an empty tuple syntax (`&.{}`) was
detected as `*const @TypeOf(.{})`
2025-05-30 16:51:55 +02:00
Steeve Morin
aedcec638c
runtimes: use cloudflare debian snapshots ( #243 )
...
They are more reliable when pinning dependencies.
Replaces and Closes #111
2025-05-29 23:43:43 +02:00
Steeve Morin
42cf435f48
runtimes/neuron: bump runtime ( #241 )
...
In the process, expose the `nrt.h` header to zig.
2025-05-29 23:40:51 +02:00
Steeve Morin
6df48e39de
workspace: make zls runner target explicit ( #240 )
...
Easier for future reference
2025-05-29 23:04:20 +02:00