Corentin Kerisit
340c812fc8
Revert "Update libc constraint to 2.34 since ROCm dependes on that"
...
This reverts commit 532de2b14f .
2025-07-11 18:54:40 +00:00
Corentin Kerisit
36002afd43
Use debian bookwarm source list for cuda
2025-07-11 18:29:47 +00:00
Corentin Kerisit
f9bba29fec
Use jammy as package source
2025-07-11 18:06:46 +00:00
Corentin Kerisit
1aa1cf460d
Fix cuda sandbox for libz
2025-07-11 16:38:29 +00:00
Corentin Kerisit
b18fa87d6f
Load nvToolsExt from the sandbox
2025-07-11 16:38:11 +00:00
Corentin Kerisit
cdcf5ddfd9
wip
2025-07-11 14:56:35 +00:00
Corentin Kerisit
c30d9138a4
Upgrade toolchains_llvm_bootstrapped to 0.2.4
2025-07-11 14:18:02 +00:00
Corentin Kerisit
532de2b14f
Update libc constraint to 2.34 since ROCm dependes on that
2025-07-11 14:17:38 +00:00
Corentin Kerisit
ce6b70241b
Let ROCm extension define its own common packages
2025-07-11 14:00:37 +00:00
Corentin Kerisit
a2db731544
Ensure ORIGIN in all required dependency rpath
2025-07-11 09:56:46 +00:00
Corentin Kerisit
5e085002b2
better readme
2025-07-11 09:55:39 +00:00
Corentin Kerisit
5cecb07c10
Add README
2025-07-11 09:26:05 +00:00
Corentin Kerisit
aec6e78bce
Load pjrt_rocm on the main thread
2025-07-10 17:07:03 +00:00
Corentin Kerisit
3f7659cdaa
Fix initial coroutine stack
2025-07-10 17:06:26 +00:00
Corentin Kerisit
e87fd9f482
Add common libraries to sandbox
2025-07-10 08:32:35 +00:00
Corentin Kerisit
efb132dceb
wip ROCM
2025-07-07 19:40:14 +00:00
Corentin Kerisit
468e6e3425
Remove unused cudart bindings
2025-07-07 13:58:13 +00:00
Corentin Kerisit
5cdbe171df
Weakify symbols from cuda.h and lazy bind them at load
2025-07-07 13:58:13 +00:00
Corentin Kerisit
3a0cdba480
Introduce weakifySymbols and bindWeakSymbols helpers
...
For accessing weak bound symbols of the cuda.so libraries
2025-07-07 13:58:13 +00:00
Corentin Kerisit
7436ce3cb4
Force lazy reloc since the default toolchain is now
2025-07-07 09:00:43 +00:00
Corentin Kerisit
b9dd8e3466
Load pjrt_cuda from sandbox
2025-07-02 11:56:57 +00:00
Corentin Kerisit
497352d4bb
wip
2025-07-01 17:03:51 +02:00
Corentin Kerisit
a7a137f59f
Add rpath manipulation to patchelf
2025-07-01 17:03:36 +02:00
Corentin Godeau
67fc7d4b97
runtimes/cuda: downgrade cuda and cudnn ( #269 )
...
This commit reverts part of https://github.com/zml/zml/pull/238/files
This is required because XLA has a strong dependencies on CUDA 12.8 and
upgrading to 12.9 is impossible due to
https://github.com/NVIDIA/cccl/issues/4967
2025-06-23 13:11:56 +02:00
Corentin Godeau
0330e34788
pjrt: Bind PJRT_Client_CreateUninitializedBuffer ( #266 )
2025-06-19 09:11:53 +02:00
Corentin Godeau
3c2d319d20
zml/pjrt: added binding for PJRT_Device_MemoryStats ( #264 )
2025-06-18 18:32:15 +02:00
Corentin Godeau
a6469229d2
async: use stronger memory ordering ( #268 )
...
This was probably causing a segfault on some specific ordering of
things.
2025-06-18 12:55:36 +02:00
Victor Moin
8a6b4a9bdd
Patch aio.zig loadBuffersWithPrefix arguments ( #265 )
...
Corrects the type annotation for init_args parameter to use the same
conditional type as `loadBuffers` function.
@Corendos
2025-06-16 18:33:34 +02:00
Victor Moin
89344e256b
Add model prefix support ( #226 )
...
## Overview
This PR adds the ability to specify a prefix when loading a model from
safetensors file. This is useful when working with models like
[ModernBERT](https://huggingface.co/answerdotai/ModernBERT-large/tree/main?show_file_info=model.safetensors )
; the changes enable me to use `ModernBertModel` to generate embeddings
instead of the full `ModernBertForMaskedLM` model.
## Test
Changes have been tested with my text embeddings server project :
https://github.com/vctrmn/zig-text-embeddings-inference
2025-06-16 17:41:29 +02:00
Corentin Godeau
989d85be2a
workspace: bump xla to version 20250612.0-6e48cbb ( #263 )
2025-06-16 17:37:57 +02:00
Hugo Mano
b4716dba03
ci: revamp commit workflow ( #246 )
...
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`
Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-06-11 13:30:34 +02:00
Guillaume Wenzek
e15d3a9b90
remove usingnamespace from mlir ( #204 )
2025-06-10 15:51:30 +02:00
Guillaume Wenzek
e4705943a6
remove dead imports ( #261 )
2025-06-10 15:36:38 +02:00
Corentin Godeau
fb0a8e4071
zml/ops: wire the has_side_effect field ( #260 )
2025-06-10 15:11:45 +02:00
Guillaume Wenzek
657d4010a3
rename pjrt BufferType to match Zig and ZML conventions ( #259 )
2025-06-10 14:59:08 +02:00
Victor Moin
a38afbbe51
Fix ModernBERT compilation error by removing print calls ( #258 )
...
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.
### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅ Loaded weights in 397ms
info(modernbert): Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201 627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
GetPjrtApi
async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***
error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```
### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`
```logs
info(modernbert): ✅ Loaded weights in 416ms
info(modernbert): Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅ Loaded weights and compiled model in 2968ms
info(modernbert): Input text: [MASK] is the capital of France.
info(modernbert): ⏱️ Model inference in 347ms
info(modernbert): ✅ Top 5 predictions:
info(modernbert): • score: 0.9755 word: 'Paris' token: 36062
info(modernbert): • score: 0.0067 word: 'Nice' token: 29235
info(modernbert): • score: 0.0063 word: 'It' token: 1147
info(modernbert): • score: 0.0010 word: 'Or' token: 3980
info(modernbert): • score: 0.0010 word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00
Corentin Godeau
b49b64df1b
zml/ops: added customCall ( #253 )
2025-06-09 15:11:08 +02:00
Corentin Godeau
0f6eb5076c
zml/meta: fix map tests ( #257 )
2025-06-09 13:33:17 +02:00
Corentin Godeau
b0cfb64a0f
pjrt: added Executable.getCompiledMemoryStats ( #250 )
2025-06-06 19:03:09 +02:00
Corentin Godeau
959936ffa1
zml/meta.zig: support unions in mapping helpers ( #251 )
2025-06-06 18:20:39 +02:00
Corentin Godeau
8bddaffcb8
mlir/dialect/stablehlo: set defaults to operand_layouts and result_layouts ( #252 )
2025-06-06 18:20:25 +02:00
Corentin Godeau
f323979cb2
zml/buffer: various improvements ( #248 )
...
* `Buffer.from` is blocking by default
* `Buffer.from` takes options to control the loading This includes
awaiting for the buffer or to which memory the buffer is copied
2025-06-06 18:12:49 +02:00
Corentin Godeau
7ac7a7f210
examples/llama: set model.norm.eps correctly ( #256 )
2025-06-06 18:12:25 +02:00
Corentin Godeau
88c627a309
Preliminary work for Custom Call ( #247 )
2025-06-06 11:24:21 +02:00
Corentin Godeau
c75e5d1aa5
modernbert: set embeddings layernorm eps default value ( #255 )
...
I'm not sure about the default value. @vctrmn maybe you know better 😄
2025-06-06 11:20:15 +02:00
Corentin Godeau
41e222f39f
workspace: updated default nvim configuration ( #254 )
...
This requires neovim >0.11.0
2025-06-06 11:20:03 +02:00
Steeve Morin
2502c44c0f
workspace: switch to toolchains_llvm_bootstrapped ( #245 )
...
Notably, this will enable using custom linkers or enable LTO.
---------
Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-06-05 21:45:15 +02:00
Guillaume Wenzek
56c71d97c0
zml: remove compile time fields from zml.Bufferized ( #237 )
...
this remove the need to pass `undefined` to `exe.call` when the
corresponding `.forward` function expects integer/float arguments that
have been inlined in the executable.
This was not needed and could just create confusion.
Now you may end up having to pass void to `exe.call` if the executable
doesn't have any runtime argument,
but it's still better than passing `undefined`.
The only place that was using the non-buffer parts of a `zml.Bufferized`
object was the infamous `zml.testing.compileAndCall`.
So I had to introduce a `zml.testing.BufferizedWithArgs` which is only
used for this function.
I put it under the `zml.testing` namespace to make it clear it's only
for this testing utility.
Note that because `compileAndCall` is strongly typed, you generally
don't need to explicitly mention `BufferizedWithArgs`
and the can just write the arguments in place and get help from ZLS.
2025-06-05 18:50:20 +02:00
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f ( #238 )
...
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily
XLA diff:
71c67e2a4f...cb67f2f7ce
---------
Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00
Steeve Morin
23c7335592
workspace: bump to zig=0.14.1 ( #242 )
...
In the process, add a case where an empty tuple syntax (`&.{}`) was
detected as `*const @TypeOf(.{})`
2025-05-30 16:51:55 +02:00