0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

231 commits

Author SHA1 Message Date
Corentin Kerisit
340c812fc8 Revert "Update libc constraint to 2.34 since ROCm dependes on that"
This reverts commit 532de2b14f.
2025-07-11 18:54:40 +00:00
Corentin Kerisit
36002afd43 Use debian bookwarm source list for cuda 2025-07-11 18:29:47 +00:00
Corentin Kerisit
f9bba29fec Use jammy as package source 2025-07-11 18:06:46 +00:00
Corentin Kerisit
1aa1cf460d Fix cuda sandbox for libz 2025-07-11 16:38:29 +00:00
Corentin Kerisit
b18fa87d6f Load nvToolsExt from the sandbox 2025-07-11 16:38:11 +00:00
Corentin Kerisit
cdcf5ddfd9 wip 2025-07-11 14:56:35 +00:00
Corentin Kerisit
c30d9138a4 Upgrade toolchains_llvm_bootstrapped to 0.2.4 2025-07-11 14:18:02 +00:00
Corentin Kerisit
532de2b14f Update libc constraint to 2.34 since ROCm dependes on that 2025-07-11 14:17:38 +00:00
Corentin Kerisit
ce6b70241b Let ROCm extension define its own common packages 2025-07-11 14:00:37 +00:00
Corentin Kerisit
a2db731544 Ensure ORIGIN in all required dependency rpath 2025-07-11 09:56:46 +00:00
Corentin Kerisit
5e085002b2 better readme 2025-07-11 09:55:39 +00:00
Corentin Kerisit
5cecb07c10 Add README 2025-07-11 09:26:05 +00:00
Corentin Kerisit
aec6e78bce Load pjrt_rocm on the main thread 2025-07-10 17:07:03 +00:00
Corentin Kerisit
3f7659cdaa Fix initial coroutine stack 2025-07-10 17:06:26 +00:00
Corentin Kerisit
e87fd9f482 Add common libraries to sandbox 2025-07-10 08:32:35 +00:00
Corentin Kerisit
efb132dceb wip ROCM 2025-07-07 19:40:14 +00:00
Corentin Kerisit
468e6e3425 Remove unused cudart bindings 2025-07-07 13:58:13 +00:00
Corentin Kerisit
5cdbe171df Weakify symbols from cuda.h and lazy bind them at load 2025-07-07 13:58:13 +00:00
Corentin Kerisit
3a0cdba480 Introduce weakifySymbols and bindWeakSymbols helpers
For accessing weak bound symbols of the cuda.so libraries
2025-07-07 13:58:13 +00:00
Corentin Kerisit
7436ce3cb4 Force lazy reloc since the default toolchain is now 2025-07-07 09:00:43 +00:00
Corentin Kerisit
b9dd8e3466 Load pjrt_cuda from sandbox 2025-07-02 11:56:57 +00:00
Corentin Kerisit
497352d4bb wip 2025-07-01 17:03:51 +02:00
Corentin Kerisit
a7a137f59f Add rpath manipulation to patchelf 2025-07-01 17:03:36 +02:00
Corentin Godeau
67fc7d4b97
runtimes/cuda: downgrade cuda and cudnn (#269)
This commit reverts part of https://github.com/zml/zml/pull/238/files
This is required because XLA has a strong dependencies on CUDA 12.8 and
upgrading to 12.9 is impossible due to
https://github.com/NVIDIA/cccl/issues/4967
2025-06-23 13:11:56 +02:00
Corentin Godeau
0330e34788
pjrt: Bind PJRT_Client_CreateUninitializedBuffer (#266) 2025-06-19 09:11:53 +02:00
Corentin Godeau
3c2d319d20
zml/pjrt: added binding for PJRT_Device_MemoryStats (#264) 2025-06-18 18:32:15 +02:00
Corentin Godeau
a6469229d2
async: use stronger memory ordering (#268)
This was probably causing a segfault on some specific ordering of
things.
2025-06-18 12:55:36 +02:00
Victor Moin
8a6b4a9bdd
Patch aio.zig loadBuffersWithPrefix arguments (#265)
Corrects the type annotation for init_args parameter to use the same 
conditional type as `loadBuffers` function. 

@Corendos
2025-06-16 18:33:34 +02:00
Victor Moin
89344e256b
Add model prefix support (#226)
## Overview

This PR adds the ability to specify a prefix when loading a model from
safetensors file. This is useful when working with models like
[ModernBERT](https://huggingface.co/answerdotai/ModernBERT-large/tree/main?show_file_info=model.safetensors)
; the changes enable me to use `ModernBertModel` to generate embeddings
instead of the full `ModernBertForMaskedLM` model.

## Test

Changes have been tested with my text embeddings server project :
https://github.com/vctrmn/zig-text-embeddings-inference
2025-06-16 17:41:29 +02:00
Corentin Godeau
989d85be2a
workspace: bump xla to version 20250612.0-6e48cbb (#263) 2025-06-16 17:37:57 +02:00
Hugo Mano
b4716dba03
ci: revamp commit workflow (#246)
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags 
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`

Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-06-11 13:30:34 +02:00
Guillaume Wenzek
e15d3a9b90
remove usingnamespace from mlir (#204) 2025-06-10 15:51:30 +02:00
Guillaume Wenzek
e4705943a6
remove dead imports (#261) 2025-06-10 15:36:38 +02:00
Corentin Godeau
fb0a8e4071
zml/ops: wire the has_side_effect field (#260) 2025-06-10 15:11:45 +02:00
Guillaume Wenzek
657d4010a3
rename pjrt BufferType to match Zig and ZML conventions (#259) 2025-06-10 14:59:08 +02:00
Victor Moin
a38afbbe51
Fix ModernBERT compilation error by removing print calls (#258)
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.

### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅	Loaded weights in 397ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201  627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***

error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```

### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`

```logs
info(modernbert): ✅	Loaded weights in 416ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅	Loaded weights and compiled model in 2968ms
info(modernbert): 	Input text: [MASK] is the capital of France.
info(modernbert): ⏱️	Model inference in  347ms
info(modernbert): ✅	Top 5 predictions:
info(modernbert): 	  • score: 0.9755  word: 'Paris' token: 36062
info(modernbert): 	  • score: 0.0067  word: 'Nice' token: 29235
info(modernbert): 	  • score: 0.0063  word: 'It' token: 1147
info(modernbert): 	  • score: 0.0010  word: 'Or' token: 3980
info(modernbert): 	  • score: 0.0010  word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00
Corentin Godeau
b49b64df1b
zml/ops: added customCall (#253) 2025-06-09 15:11:08 +02:00
Corentin Godeau
0f6eb5076c
zml/meta: fix map tests (#257) 2025-06-09 13:33:17 +02:00
Corentin Godeau
b0cfb64a0f
pjrt: added Executable.getCompiledMemoryStats (#250) 2025-06-06 19:03:09 +02:00
Corentin Godeau
959936ffa1
zml/meta.zig: support unions in mapping helpers (#251) 2025-06-06 18:20:39 +02:00
Corentin Godeau
8bddaffcb8
mlir/dialect/stablehlo: set defaults to operand_layouts and result_layouts (#252) 2025-06-06 18:20:25 +02:00
Corentin Godeau
f323979cb2
zml/buffer: various improvements (#248)
* `Buffer.from` is blocking by default
* `Buffer.from` takes options to control the loading This includes
awaiting for the buffer or to which memory the buffer is copied
2025-06-06 18:12:49 +02:00
Corentin Godeau
7ac7a7f210
examples/llama: set model.norm.eps correctly (#256) 2025-06-06 18:12:25 +02:00
Corentin Godeau
88c627a309
Preliminary work for Custom Call (#247) 2025-06-06 11:24:21 +02:00
Corentin Godeau
c75e5d1aa5
modernbert: set embeddings layernorm eps default value (#255)
I'm not sure about the default value. @vctrmn maybe you know better 😄
2025-06-06 11:20:15 +02:00
Corentin Godeau
41e222f39f
workspace: updated default nvim configuration (#254)
This requires neovim >0.11.0
2025-06-06 11:20:03 +02:00
Steeve Morin
2502c44c0f
workspace: switch to toolchains_llvm_bootstrapped (#245)
Notably, this will enable using custom linkers or enable LTO.

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-06-05 21:45:15 +02:00
Guillaume Wenzek
56c71d97c0
zml: remove compile time fields from zml.Bufferized (#237)
this remove the need to pass `undefined` to `exe.call` when the
corresponding `.forward` function expects integer/float arguments that
have been inlined in the executable.

This was not needed and could just create confusion.
Now you may end up having to pass void to `exe.call` if the executable
doesn't have any runtime argument,
but it's still better than passing `undefined`.

The only place that was using the non-buffer parts of a `zml.Bufferized`
object was the infamous `zml.testing.compileAndCall`.
So I had to introduce a `zml.testing.BufferizedWithArgs` which is only
used for this function.
I put it under the `zml.testing` namespace to make it clear it's only
for this testing utility.
Note that because `compileAndCall` is strongly typed, you generally
don't need to explicitly mention `BufferizedWithArgs`
and the can just write the arguments in place and get help from ZLS.
2025-06-05 18:50:20 +02:00
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f (#238)
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily

XLA diff:
71c67e2a4f...cb67f2f7ce

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00
Steeve Morin
23c7335592
workspace: bump to zig=0.14.1 (#242)
In the process, add a case where an empty tuple syntax (`&.{}`) was
detected as `*const @TypeOf(.{})`
2025-05-30 16:51:55 +02:00