0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
Commit graph

12 commits

Author SHA1 Message Date
Guillaume Wenzek
0312d6f678
zml: gather (#330)
Revamp the `gather` api to match `scatter`

(prerequesite of #329 )

## Gather new api

Now the syntax is to give a named list of indices. Each name must
correspond to one axis in the gathered tensor.
This match `scatter`, and avoid the user to have to concatenate indices
in the same order than the input axes,
which is error prone cause any screw up will compile but index at the
wrong offsets.

For 1 axis it doesn't change much: 

```
// before
self.weight.withTags(.{ .voc, .d }).gather(.{.voc}, .{ idx }, .{});
// after
self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx }, .{});
```

But for two it helps a lot:

```
// before
self.weight.withTags(.{ .voc, .d }).gather(.{.voc, .d}, .{ .concatenate(&.{idx_voc, idx_d}, .last)}, .{});
// after
self.weight.withTags(.{ .voc, .d }).gather(.{ .voc = idx_voc, .d = idx_d }, .{});
```

Also adds a `gather_` that offers the same apis but with axis position
instead of names.
This one is more suitable inside library functions when dealing with
unknown names, making it complicated to create the "indices" struct.
I think we could generalize this pattern of offering "non-tagged" api
with an underscore suffix, simplifying the job of LSP.

## topk new API

I realized that most of the code calling topk was renaming the output
axis.
The idea is that logprob.topk(.voc) doesn't return you a tensor with voc
entry but only top_voc entries.
For the values it doesn't make much a difference but for the indices
it's embarrassing and it break usages of gather/scatter.

Now the user of `topK` can either use the unnamed api and pass a number,
of if using the name api must give a new name for the indexed axis.

## Yarn

Support Yarn rope embeddings. It's equivalent to the "llama3" embedding
except they do the interpolation in time space instead of freq space.
2025-09-25 12:29:46 +02:00
Guillaume Wenzek
dda7f6c3f0
zml: remove old writer interface APIs (#324)
We still had a few old reader/writer left over.
I think we are good now
2025-09-18 11:36:23 +02:00
Guillaume Wenzek
df77b33d8e
async: remove asynk, asyncc, awaitt, await_ (#325) 2025-09-18 10:05:54 +02:00
Steeve Morin
57fd9cda27
workspace: zig 0.15 (#316)
Much of the work was done by @gwenzek

Sadly we drop the torch loader in the process, to be added back later !

---------

Co-authored-by: Guillaume Wenzek <wzk@zml.ai>
2025-08-27 14:11:22 +02:00
Steeve Morin
24575c2232
workspace: switch from zig_cc_binary to zig_binary (#311)
Following the rules_zig bump that adds the `linkmode` setting, we can
use the C linker directly.

We can now remove the `zig_cc_binary` hack.
2025-08-21 13:12:15 +02:00
Steeve Morin
3b81c5630e
zml: remove examples workspace (#308)
Closes #302
2025-08-19 19:42:50 +02:00
Hugo Mano
b4716dba03
ci: revamp commit workflow (#246)
- Use runs-on s3 cache
- Use mlugg/setup-zig@v2 with ZML mirror instead of deprecated and unreachable goto-bus-stop/setup-zig@v2
- Setup Bazel with bazel-contrib/setup-bazel@0.15.0, leverage disk cache and bazelrc flags 
- Leverage Bazel `manual` tag to filter out unneeded targets for `//...`

Co-authored-by: Jean-Baptiste Dalido <jb@zml.ai>
2025-06-11 13:30:34 +02:00
Victor Moin
a38afbbe51
Fix ModernBERT compilation error by removing print calls (#258)
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.

### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅	Loaded weights in 397ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201  627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***

error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```

### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`

```logs
info(modernbert): ✅	Loaded weights in 416ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅	Loaded weights and compiled model in 2968ms
info(modernbert): 	Input text: [MASK] is the capital of France.
info(modernbert): ⏱️	Model inference in  347ms
info(modernbert): ✅	Top 5 predictions:
info(modernbert): 	  • score: 0.9755  word: 'Paris' token: 36062
info(modernbert): 	  • score: 0.0067  word: 'Nice' token: 29235
info(modernbert): 	  • score: 0.0063  word: 'It' token: 1147
info(modernbert): 	  • score: 0.0010  word: 'Or' token: 3980
info(modernbert): 	  • score: 0.0010  word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00
Corentin Godeau
c75e5d1aa5
modernbert: set embeddings layernorm eps default value (#255)
I'm not sure about the default value. @vctrmn maybe you know better 😄
2025-06-06 11:20:15 +02:00
Guillaume Wenzek
bf79263919
Fix llama3 rope scaling (#230) 2025-05-09 14:32:15 +02:00
Guillaume Wenzek
a85b53fba0
zig 0.14.0 (#189)
Co-authored-by: Hugo Mano <hugo@zml.ai>
2025-03-14 09:49:23 +01:00
Guillaume Wenzek
a672bb8247
examples: add modernBERT (#192)
Modern bert implementation from Victor: @vctrmn

Tested with https://huggingface.co/answerdotai/ModernBERT-base
I started from his PR #149 and cleaned up a bit the code to make it more
what I consider "idiomatic ZML" even if it's still a bit in flux since
we are so young.

---------

Co-authored-by: vctrmn <victormoin.vm@gmail.com>
Co-authored-by: Rene Schallner <rene@zml.ai>
2025-02-27 18:07:42 +01:00