0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

232 commits

Author SHA1 Message Date
Steeve Morin
3c19df1341 WIP 2025-08-14 15:55:29 +02:00
Corentin Godeau
baca71ec3b zml/tools: temporarily disable CudaTracer for universal binaries 2025-07-17 22:16:52 +02:00
Corentin Godeau
68e9995373 zml/nn: correctly parse RopeOpts.Scaling from JSON 2025-07-17 22:16:52 +02:00
Corentin Godeau
b39b92829c stdx/math: added cdiv function 2025-07-17 22:16:52 +02:00
Corentin Godeau
8c889d48a3 zml/tools/tracer.zig: make FakeTracer api compatible with CudaTracer 2025-07-17 22:16:52 +02:00
Corentin Godeau
497e4b8d3d zml/module.zig: disable gemm autotuning on ROCm
This is taking way too much time for llmd
2025-07-17 22:16:52 +02:00
Corentin Godeau
637e67a07b zml/pjrtx: remove async blocking call to execute 2025-07-17 22:16:52 +02:00
Corentin Godeau
2be964ba9a zml/exe: don't wait for event
Don't forget to delete them
2025-07-17 22:16:52 +02:00
Steeve Morin
e13acae80b
runtimes/rocm: fetch libdrm from amdgpu repo, add amdgpu.ids layer (#287)
Properly fetch libdrm related artifacts from the amdgpu driver repo
instead of ubuntu.

While there, add `amdgpu.ids` file to the ROCm layers.
2025-07-17 22:16:27 +02:00
Steeve Morin
81fa2ec44c
workspace: disable glibc constraint, bump rules_rust, re-enable framepointers (#286)
Since the rework of runtimes sandboxing, explicitely compiling against a certain glibc version isn't needed anymore.
2025-07-17 18:20:07 +02:00
Corentin Kerisit
32216e8c54
runtimes/rocm: use inprocess lld instead of sandboxing it (#284)
XLA rencetly added the --xla_gpu_use_inprocess_lld bool flag which lets
use ditch the rocm-llvm//:lld from the sandbox :)

See
d01947de6f
2025-07-17 16:39:00 +02:00
Corentin Kerisit
4362fe323e
workspace: bump rules_zig to 20250714.0-b14a4f1 (#283)
This dummy fixes the presence of the static-lib in the runfiles of the
final target in case of kind != exe.
2025-07-16 18:01:09 +02:00
Hugo Mano
8066d2abdd
ci: use large disk on runs-on (#282)
With the latests developments the default disk space (40GB) is too small
for our new needs so we use large disks (80GB).
2025-07-16 15:21:37 +02:00
Corentin Kerisit
4c14652fb7
runtimes/rocm: sandbox ROCm dependencies (#276)
ROCm counterpart of #275 

The most important thing with this change is that ROCm dependencies
(along with the PJRT plugin) must be loaded in the main thread.

This is because some ROCm dependencies (amdhip64) use Thread Local
Storage in the destructor of static C++ instances.
Because all dependencies are unloaded at exit of the main program,
destructors are run on whatever thread exited (in general the main
thread), and any access to the TLS will except values from a different
threads....
2025-07-16 14:41:07 +02:00
Corentin Kerisit
d3dda29db5
runtimes/tpu: sandbox TPU PJRT Plugin (#278)
It has no dependencies so fairly straighforward
2025-07-16 14:13:22 +02:00
Hugo Mano
fc118e058f
deps/hftokenizers: bump dependency (#281) 2025-07-16 13:44:19 +02:00
Corentin Kerisit
12293da6af
runtimes/cpu: sandbox CPU PJRT Plugin (#279)
Same as TPU, simplified because there are no other NEEDED dependencies.
2025-07-16 13:08:55 +02:00
Corentin Kerisit
8274f9c2c0
runtimes/cuda: sandbox CUDA dependencies (#275)
This PR introduces CUDA sandboxing.

This essentially means:
- Removing all CUDA related dependencies off the leaf binary.
- Sandboxing the whole CUDA dependency graph.
- Mark dlopen direct dependencies of the PJRT shared library as NEEDED.
- Set the RPATH to the sandbox for all libraries that have sandbox
dependencies.
- Load the PJRT plugin from the sandbox.

And in the case of CUDA specifically for now:
- Enable usage of weak CUDA symbols without linking against CUDA
directly.
2025-07-16 11:33:48 +02:00
Corentin Kerisit
8ca8d069f9
pjrt: warn instead of panic if FFI Extension is not found (#277)
This is the case for TPU where FFI Extension doesn't not exist.
2025-07-15 14:55:15 +02:00
Corentin Kerisit
57abcb5aab
workspace: bump toolchains_llvm_bootstrapped to 0.2.4 (#274)
This update is important because it orders correctly the CRT which
contains .init and .fini family sections.
2025-07-15 14:54:57 +02:00
Guillaume Wenzek
ee1163a113
Bump rules_zig, add zig_srcs targets, cleanup build (#267)
This is the non-controversial part of #262 

It contains:
* the new `zig_srcs` helper,
* bug fix related to sources in rules_zig,
* cleanups of zml/BUILD.bazel
* test fixes in async/coro.zig (those tests weren't run in CI but now
are)
* disabled nemo and yaml models loader (don't compile anymore since
0.14.0 bump)
2025-07-15 14:18:15 +02:00
Corentin Kerisit
00ebef1a32
async/coro: make coroutines unwindable (#273)
The first stack of each new coroutines were holding not zeroed and hold
random memory data, which get popped in rbp during first coroutine stack
swap.

Which made unwinding random or even SIGSEGV when _Unwind_Backtrace was
used (during C++ exception for instance).

Zeroing the first part of the stack makes unwinding stop gracefully when
it encounters $rbp=0x00.
2025-07-12 20:48:37 +02:00
Corentin Kerisit
8d0e6e6348
bazel: add rpath manipulation to patchelf (#272) 2025-07-12 20:02:46 +02:00
Hugo Mano
b0168d1c63
workspace: bump xla to version 20250710.0-22ea002 (#271)
Also:
- Bump XLA deps : `com_github_grpc_grpc` and `com_google_protobuf`
- Inject `rules_ml_toolchain`
- Fix `zig_proto_library` rule
2025-07-11 17:45:57 +02:00
Corentin Godeau
67fc7d4b97
runtimes/cuda: downgrade cuda and cudnn (#269)
This commit reverts part of https://github.com/zml/zml/pull/238/files
This is required because XLA has a strong dependencies on CUDA 12.8 and
upgrading to 12.9 is impossible due to
https://github.com/NVIDIA/cccl/issues/4967
2025-06-23 13:11:56 +02:00
Corentin Godeau
0330e34788
pjrt: Bind PJRT_Client_CreateUninitializedBuffer (#266) 2025-06-19 09:11:53 +02:00
Corentin Godeau
3c2d319d20
zml/pjrt: added binding for PJRT_Device_MemoryStats (#264) 2025-06-18 18:32:15 +02:00
Corentin Godeau
a6469229d2
async: use stronger memory ordering (#268)
This was probably causing a segfault on some specific ordering of
things.
2025-06-18 12:55:36 +02:00
Victor Moin
8a6b4a9bdd
Patch aio.zig loadBuffersWithPrefix arguments (#265)
Corrects the type annotation for init_args parameter to use the same 
conditional type as `loadBuffers` function. 

@Corendos
2025-06-16 18:33:34 +02:00
Victor Moin
89344e256b
Add model prefix support (#226)
## Overview

This PR adds the ability to specify a prefix when loading a model from
safetensors file. This is useful when working with models like
[ModernBERT](https://huggingface.co/answerdotai/ModernBERT-large/tree/main?show_file_info=model.safetensors)
; the changes enable me to use `ModernBertModel` to generate embeddings
instead of the full `ModernBertForMaskedLM` model.

## Test

Changes have been tested with my text embeddings server project :
https://github.com/vctrmn/zig-text-embeddings-inference
2025-06-16 17:41:29 +02:00
Corentin Godeau
989d85be2a
workspace: bump xla to version 20250612.0-6e48cbb (#263) 2025-06-16 17:37:57 +02:00
Hugo Mano
b4716dba03
ci: revamp commit workflow (#246)
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags 
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`

Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-06-11 13:30:34 +02:00
Guillaume Wenzek
e15d3a9b90
remove usingnamespace from mlir (#204) 2025-06-10 15:51:30 +02:00
Guillaume Wenzek
e4705943a6
remove dead imports (#261) 2025-06-10 15:36:38 +02:00
Corentin Godeau
fb0a8e4071
zml/ops: wire the has_side_effect field (#260) 2025-06-10 15:11:45 +02:00
Guillaume Wenzek
657d4010a3
rename pjrt BufferType to match Zig and ZML conventions (#259) 2025-06-10 14:59:08 +02:00
Victor Moin
a38afbbe51
Fix ModernBERT compilation error by removing print calls (#258)
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.

### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅	Loaded weights in 397ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201  627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***

error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```

### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`

```logs
info(modernbert): ✅	Loaded weights in 416ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅	Loaded weights and compiled model in 2968ms
info(modernbert): 	Input text: [MASK] is the capital of France.
info(modernbert): ⏱️	Model inference in  347ms
info(modernbert): ✅	Top 5 predictions:
info(modernbert): 	  • score: 0.9755  word: 'Paris' token: 36062
info(modernbert): 	  • score: 0.0067  word: 'Nice' token: 29235
info(modernbert): 	  • score: 0.0063  word: 'It' token: 1147
info(modernbert): 	  • score: 0.0010  word: 'Or' token: 3980
info(modernbert): 	  • score: 0.0010  word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00
Corentin Godeau
b49b64df1b
zml/ops: added customCall (#253) 2025-06-09 15:11:08 +02:00
Corentin Godeau
0f6eb5076c
zml/meta: fix map tests (#257) 2025-06-09 13:33:17 +02:00
Corentin Godeau
b0cfb64a0f
pjrt: added Executable.getCompiledMemoryStats (#250) 2025-06-06 19:03:09 +02:00
Corentin Godeau
959936ffa1
zml/meta.zig: support unions in mapping helpers (#251) 2025-06-06 18:20:39 +02:00
Corentin Godeau
8bddaffcb8
mlir/dialect/stablehlo: set defaults to operand_layouts and result_layouts (#252) 2025-06-06 18:20:25 +02:00
Corentin Godeau
f323979cb2
zml/buffer: various improvements (#248)
* `Buffer.from` is blocking by default
* `Buffer.from` takes options to control the loading This includes
awaiting for the buffer or to which memory the buffer is copied
2025-06-06 18:12:49 +02:00
Corentin Godeau
7ac7a7f210
examples/llama: set model.norm.eps correctly (#256) 2025-06-06 18:12:25 +02:00
Corentin Godeau
88c627a309
Preliminary work for Custom Call (#247) 2025-06-06 11:24:21 +02:00
Corentin Godeau
c75e5d1aa5
modernbert: set embeddings layernorm eps default value (#255)
I'm not sure about the default value. @vctrmn maybe you know better 😄
2025-06-06 11:20:15 +02:00
Corentin Godeau
41e222f39f
workspace: updated default nvim configuration (#254)
This requires neovim >0.11.0
2025-06-06 11:20:03 +02:00
Steeve Morin
2502c44c0f
workspace: switch to toolchains_llvm_bootstrapped (#245)
Notably, this will enable using custom linkers or enable LTO.

---------

Co-authored-by: Corentin Kerisit <corentin.kerisit@gmail.com>
2025-06-05 21:45:15 +02:00
Guillaume Wenzek
56c71d97c0
zml: remove compile time fields from zml.Bufferized (#237)
this remove the need to pass `undefined` to `exe.call` when the
corresponding `.forward` function expects integer/float arguments that
have been inlined in the executable.

This was not needed and could just create confusion.
Now you may end up having to pass void to `exe.call` if the executable
doesn't have any runtime argument,
but it's still better than passing `undefined`.

The only place that was using the non-buffer parts of a `zml.Bufferized`
object was the infamous `zml.testing.compileAndCall`.
So I had to introduce a `zml.testing.BufferizedWithArgs` which is only
used for this function.
I put it under the `zml.testing` namespace to make it clear it's only
for this testing utility.
Note that because `compileAndCall` is strongly typed, you generally
don't need to explicitly mention `BufferizedWithArgs`
and the can just write the arguments in place and get help from ZLS.
2025-06-05 18:50:20 +02:00
Hugo Mano
bde41243bc
workspace: bump xla to version 20250527.0-cb67f2f (#238)
third_party/modules/xla: refactor XLA module extensions
ci: disable failing cache temporarily

XLA diff:
71c67e2a4f...cb67f2f7ce

---------

Co-authored-by: Steeve Morin <steeve@zml.ai>
2025-06-02 21:44:39 +02:00