0
Fork 0
mirror of https://github.com/zml/zml.git synced 2026-11-16 19:11:09 +00:00
zml/ffi/zig_allocator.zig

26 lines
1,016 B

zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522) This change fixes profile timestamp handling so saved xplane.pb data is post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the Zig/C++ FFI used by the profile export bridge. Before this, we were writing raw profiler output from the PJRT extension directly to `profiling.xplane.pb`, which bypassed XLA’s single-host post-processing step done by JAX that we can find here: https://github.com/google/tsl/blob/efac222ebfdbd682d3a407ef165865c6b6759e2d/tsl/profiler/lib/profiler_session.cc#L100 That made ZML behave differently from JAX, including wrong timeline origins in downstream viewers like XProf. To fix that, we now run `tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the protobuf, and we treat that post-processed xplane.pb as the source for subsequent Perfetto conversion. We also register start/stop timestamps in `zml/profiler.zig` so they match XLA’s expectations for session timestamps with `real` clock. While touching this path, I tightened the FFI boundary as well. Error text returned from the xspace export bridge is now allocated with the provided Zig allocator instead of malloc, C receives a pointer to the `zig_allocator wrapper` to avoid lifetime issues from passing allocator state by value, and error propagation in `tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly instead of building std::string error objects just to copy them back out. This removes mixed allocation ownership between C++ and Zig and fixes the allocator lifetime bug that could crash across the FFI boundary.
2026-05-05 14:52:15 +02:00
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522) This change fixes profile timestamp handling so saved xplane.pb data is post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the Zig/C++ FFI used by the profile export bridge. Before this, we were writing raw profiler output from the PJRT extension directly to `profiling.xplane.pb`, which bypassed XLA’s single-host post-processing step done by JAX that we can find here: https://github.com/google/tsl/blob/efac222ebfdbd682d3a407ef165865c6b6759e2d/tsl/profiler/lib/profiler_session.cc#L100 That made ZML behave differently from JAX, including wrong timeline origins in downstream viewers like XProf. To fix that, we now run `tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the protobuf, and we treat that post-processed xplane.pb as the source for subsequent Perfetto conversion. We also register start/stop timestamps in `zml/profiler.zig` so they match XLA’s expectations for session timestamps with `real` clock. While touching this path, I tightened the FFI boundary as well. Error text returned from the xspace export bridge is now allocated with the provided Zig allocator instead of malloc, C receives a pointer to the `zig_allocator wrapper` to avoid lifetime issues from passing allocator state by value, and error propagation in `tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly instead of building std::string error objects just to copy them back out. This removes mixed allocation ownership between C++ and Zig and fixes the allocator lifetime bug that could crash across the FFI boundary.
2026-05-05 14:52:15 +02:00
zml/profiler: post-process saved xplane profiles and clean up profiler FFI (#522) This change fixes profile timestamp handling so saved xplane.pb data is post-processed like JAX/XLA’s `ProfilerSession`, and also cleans up the Zig/C++ FFI used by the profile export bridge. Before this, we were writing raw profiler output from the PJRT extension directly to `profiling.xplane.pb`, which bypassed XLA’s single-host post-processing step done by JAX that we can find here: https://github.com/google/tsl/blob/efac222ebfdbd682d3a407ef165865c6b6759e2d/tsl/profiler/lib/profiler_session.cc#L100 That made ZML behave differently from JAX, including wrong timeline origins in downstream viewers like XProf. To fix that, we now run `tsl::profiler::PostProcessSingleHostXSpace(...)` before writing the protobuf, and we treat that post-processed xplane.pb as the source for subsequent Perfetto conversion. We also register start/stop timestamps in `zml/profiler.zig` so they match XLA’s expectations for session timestamps with `real` clock. While touching this path, I tightened the FFI boundary as well. Error text returned from the xspace export bridge is now allocated with the provided Zig allocator instead of malloc, C receives a pointer to the `zig_allocator wrapper` to avoid lifetime issues from passing allocator state by value, and error propagation in `tools/xspace_to_perfetto_lib.cc `now uses `zig_slice` outputs directly instead of building std::string error objects just to copy them back out. This removes mixed allocation ownership between C++ and Zig and fixes the allocator lifetime bug that could crash across the FFI boundary.
2026-05-05 14:52:15 +02:00