0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
zml/examples/modernbert/modernbert.zig
Victor Moin a38afbbe51
Fix ModernBERT compilation error by removing print calls (#258)
Fixes compilation error in ModernBERT examples on my intel-based macbook
pro.

### Problem
ModernBERT failed to compile with sharding error:
```logs
info(modernbert): ✅	Loaded weights in 397ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749552133.985201  627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
*** Begin stack trace ***
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	GetPjrtApi
	async.callBlocking__anon_34323.TaskT.run
*** End stack trace ***

error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64}
error(zml/module): pjrt-cpu failed to compile: error.Internal
error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir
error: Internal
```

### Solution
Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()`
functions in `examples/modernbert/modernbert.zig`

```logs
info(modernbert): ✅	Loaded weights in 416ms
info(modernbert): 	Compiling ModernBERT model...
info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) }
info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir
2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations
2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback.
info(zml/module): Compilation took 2.548s
info(modernbert): ✅	Loaded weights and compiled model in 2968ms
info(modernbert): 	Input text: [MASK] is the capital of France.
info(modernbert): ⏱️	Model inference in  347ms
info(modernbert): ✅	Top 5 predictions:
info(modernbert): 	  • score: 0.9755  word: 'Paris' token: 36062
info(modernbert): 	  • score: 0.0067  word: 'Nice' token: 29235
info(modernbert): 	  • score: 0.0063  word: 'It' token: 1147
info(modernbert): 	  • score: 0.0010  word: 'Or' token: 3980
info(modernbert): 	  • score: 0.0010  word: 'This' token: 1552
```
2025-06-10 13:49:18 +02:00

269 lines
10 KiB