mirror of
https://github.com/zml/zml.git
synced 2026-11-16 19:11:09 +00:00
Fixes compilation error in ModernBERT examples on my intel-based macbook pro. ### Problem ModernBERT failed to compile with sharding error: ```logs info(modernbert): ✅ Loaded weights in 397ms info(modernbert): Compiling ModernBERT model... info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) } info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir 2025-06-10 12:42:13.905711: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations 2025-06-10 12:42:13.981006: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback. WARNING: All log messages before absl::InitializeLog() is called are written to STDERR E0000 00:00:1749552133.985201 627264 status_macros.cc:57] INTERNAL: RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64} *** Begin stack trace *** GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi GetPjrtApi async.callBlocking__anon_34323.TaskT.run *** End stack trace *** error(pjrt): [PJRT_Client_Compile] RET_CHECK failure (xla/service/spmd/spmd_partitioner.cc:5555) hlo->has_sharding() Side-effect HLO must have sharding: %custom-call.0 = f32[1,64,64]{2,1,0} custom-call(%broadcast.2177), custom_call_target="zmlHostBufferCallback", operand_layout_constraints={f32[1,64,64]{2,1,0}}, custom_call_has_side_effect=true, output_to_operand_aliasing={{}: (0, {})}, api_version=API_VERSION_TYPED_FFI, metadata={source_file="ops.zig" source_line=1378}, backend_config={callback = 4553681824 : ui64, pjrt_api = 4795162728 : ui64, pjrt_client = 105553131915920 : ui64, user_context = 0 : ui64} error(zml/module): pjrt-cpu failed to compile: error.Internal error(zml/module): mlir can be found at /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_192023e1d2d0c331/module.mlir error: Internal ``` ### Solution Removed `.print()` calls from `globalAttnMask()` and `localAttnMask()` functions in `examples/modernbert/modernbert.zig` ```logs info(modernbert): ✅ Loaded weights in 416ms info(modernbert): Compiling ModernBERT model... info(zml/exe): Compiling modernbert.ModernBertForMaskedLM.forward with { Shape({b=1,s=64,u32}) } info(zml/module): Wrote MLIR to /private/tmp/zml/modernbert/cpu/modernbert.ModernBertForMaskedLM.forward_10a8f1a59c873a8f/module.mlir 2025-06-10 12:53:54.150319: I xla/service/dump.cc:562] HloModule dump enabled with path prefix: , suffix: before_optimizations 2025-06-10 12:53:54.227584: I xla/hlo/utils/hlo_sharding_util.cc:3079] There is no registered layout_canonicalization_callback. info(zml/module): Compilation took 2.548s info(modernbert): ✅ Loaded weights and compiled model in 2968ms info(modernbert): Input text: [MASK] is the capital of France. info(modernbert): ⏱️ Model inference in 347ms info(modernbert): ✅ Top 5 predictions: info(modernbert): • score: 0.9755 word: 'Paris' token: 36062 info(modernbert): • score: 0.0067 word: 'Nice' token: 29235 info(modernbert): • score: 0.0063 word: 'It' token: 1147 info(modernbert): • score: 0.0010 word: 'Or' token: 3980 info(modernbert): • score: 0.0010 word: 'This' token: 1552 ```
269 lines
10 KiB
269 lines
10 KiB