mirror of
https://github.com/TanStack/router.git
synced 2026-11-16 19:38:07 +00:00
4 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
a3e24c35a5 |
bench(memory): prefix client and server benchmark names (#7743) | ||
|
|
d644dbb5c2 |
bench(memory): settle-until-quiescent GC barrier between iterations (#7739)
* bench(memory): settle-until-quiescent GC barrier between iterations The fixed-turn settle before the pinned collection was hardware-fragile: teardown needing one more event-loop turn on a given runner leaked a whole payload of garbage past the collection point and flipped the measured peak bimodally (serialization-payload solid/react resurfacing the pre-#7730 6.2MB attractor on some CI runners). Replace it with an adaptive barrier that hops and collects until the post-GC heap size stops moving, with a minimum-turn floor matching the fixed count it replaces (React's stream teardown spans several turns and races a shorter floor). Also pin collections in request-churn and the aborted-requests drain: their peak floated on GC scheduling even with bit-identical work (18% spread on CI for solid churn). Reachable accumulation stays visible to the metric - a forced collection cannot reclaim leaked or cached objects, it only removes floating garbage, whose collection timing was the dominant cross-run noise source. Verified with a synthetic reachable leak: +4.7% peak over a 0.6% A/A noise floor, where the old 18% noise would have swallowed it. Local A/A spreads (3x codspeed run per suite): solid <=1%, react <=2.2% except request-churn (rare +130KB mode, framework teardown variance), vue <=2.3% except aborted-requests (~8%, intra-iteration work variance, down from 21% on CI). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): fixed-count settle barrier, drop adaptive quiescence exit The adaptive exit (collect until post-GC heap size stops moving) made the collection points land at data-dependent turns on CI: heap readings never fully stabilize there, so the exit point - and every subsequent GC point - shifted between identical runs. The first CI A/A pair showed it flipping serialization-payload (vue) to a 6MB mode (67%), inflating peak-large-page (vue) 3.4x, and adding 7-8% swings to the react/vue abort scenarios, while fixing the solid suite. Every step is now a fixed count: 16 settle turns (enough for React/Vue multi-turn teardown that the original 2-turn barrier raced), one pinned collection, one turn, one more collection for finalizer fallout. Fixed counts make the GC points identical across runs by construction, which is the property the barrier exists to provide. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): size iteration counts to the 2-10s measured-run window On CI (eBPF instrument + --no-opt) one pinned iteration costs ~0.1-0.6s, so the previous counts put request-churn at ~36s measured and the error -paths/server-fn benches at 0.1-0.5s - outside the useful 2-10s range in both directions. Retune: request-churn/aborted-requests/error-paths/ server-fn-churn to 40 iterations, serialization-payload to 12; peak-large-page and streaming-peak already sit in range at 20. The floor is raised by pinning error-paths and server-fn-churn rather than inflating their iteration counts: thousands of unpinned iterations would cross V8's own GC scheduling and reintroduce the timing noise the barrier removes, while the pin's fixed per-iteration cost lifts them into range at 40 iterations and makes their collection points deterministic. Note: fewer churn iterations proportionally shrink the accumulation window, so peak-based leak sensitivity drops accordingly; leaks remain visible in total-allocated (forced compactions re-copy the growing live set every iteration). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): disable V8 bytecode flushing in memory bench workers The retuned (shorter) runs exposed a recurring V8 internal event - a +7-13MB allocation burst with the signature of bytecode flush + lazy recompile - that lands inside or outside the measured window at random, flipping peaks by whole MBs between identical CI runs (request-churn vue 785KB vs 3.2MB). The pinned collections age code fast enough for V8 to flush unused bytecode mid-run; --no-flush-bytecode keeps compiled code alive for the process lifetime so the measured window sees no recompile bursts. Vite's mergeConfig concatenates the scenario's test.execArgv with the flags the CodSpeed plugin injects, so both apply to the fork workers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): pre-size V8 heap geometry in memory bench workers --no-flush-bytecode alone did not stop the run-to-run flips: the recurring event is heap-space growth, which allocates several MB at a run-dependent moment inside the shortened measured windows (error-paths unmatched vue 674KB vs 2MB, serialization-payload solid resurfacing its 6.4MB attractor on one run in three). Pre-sizing old space and pinning the semi-space size removes mid-measurement growth while leaving the per-iteration page map/unmap signal intact - local runs confirm peaks still track payload sizes exactly (serialization 3.5MB, streaming 30.3MB, levels unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): size tiny benches just above the 2s measured-run floor error-paths and server-fn-churn sat at 5-8s measured on CI; fewer iterations bring them to ~2.3-3.4s, keeping the 2s floor while shrinking the window a stray allocator event can land in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): verify the pinned collection against the heap floor The residual CI flips all share one signature: exactly one iteration's payload added to the measured peak (serialization-payload vue 3.5 vs 6.9MB, peak-large-page vue 780 vs 1349KB, error-paths redirect vue 300 vs 885KB). On some runners the response teardown holds the payload past the fixed settle window - released only by a later internal timer - so it survives the pinned collection and bleeds into the next iteration. After the fixed 16-turn settle and collection, compare the post-GC heap against the smallest post-collection size seen this run (the inter- iteration floor): while it has not returned to the floor, keep hopping and collecting, bounded at 64 turns. The barrier stays fixed-count in the common case and extends only in exactly the failure case. Workloads that genuinely accumulate reachable memory raise the floor as they go, so accumulation still measures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): restrict floor verification to peak-shape scenarios The floor-verified barrier fixed its target - the serialization family went to <=0.45% across three CI runs, attractor gone - but destabilized churn/abort scenarios (request-churn solid 420KB vs 3.4MB): their inter- iteration floor legitimately drifts, so the verification capped out chronically and the ~48 extra collections per iteration eventually hit a compaction that double-copies the live set. Split by scenario shape: serialization-payload, peak-large-page and streaming-peak keep the verified barrier (flat floor, proven fix); churn and abort scenarios go back to the plain fixed-count barrier that held them at <=1.7% in the two previous CI rounds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a6337fb98e |
bench(memory): stabilize CodSpeed memory benchmarks (#7730)
* bench(memory): stabilize CodSpeed memory benchmarks Memory flags on PRs were ~85% noise, from two causes: 1. Stale baselines (dominant): the Benchmarks workflow only ran on main pushes touching packages/** or benchmarks/**. Main commits outside those paths had no CodSpeed run, so PRs were compared against an old base run ([^unexpected-base] on every recent codspeed-hq comment; six late-June PRs all compared to the same June-23 run). One outlier base measurement then replays into every PR until the next eligible push. Fix: drop the paths filter for the push trigger + weekly cron backstop. 2. Wall-clock timers racing the single measured run: aborted-requests (solid/vue) resolved deferred loader data on 500/750ms timeouts raced against abort, and drained cancellation with at most one setTimeout(0); streaming-peak staggered deferred sections by milliseconds. Under variable runner load and instrumentation slowdown the interleaving — and therefore the allocation sequence — differed run to run. Fix: count 0ms timers-phase hops (event-loop turns) instead of milliseconds everywhere async ordering must be staged, and drain aborts with a fixed 8-turn settlement barrier. Local before/after evidence (8 fresh-process runs per config, CodSpeed V8 determinism flags, idle + CPU-stressed; kernel peak RSS spread): aborted-requests/solid idle 11.2MB -> 1.8MB (GC counts bit-identical) aborted-requests/solid stress 7.7MB -> 2.5MB streaming-peak/solid idle 6.2MB -> 2.5MB request-churn/solid (untouched control): unchanged, confirming harness streaming-peak/vue keeps ~2% proxy-level variance at any tick count; its PR flags were the identical -3.2% in all three occurrences, i.e. the stale-base artifact rather than per-run jitter. Bench names and workload shapes are unchanged; absolute values will step once on the CodSpeed dashboard from the removed cross-iteration overlap (e.g. aborted-requests/solid retained 15.6MB -> 2.5MB per invocation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): exclude docs/examples/e2e from main benchmark runs, drop cron paths-ignore instead of no filter: docs/examples/e2e-only pushes cannot affect benchmark results, and falling back across such a commit is safe because the previous run's benchmark-relevant code is identical. The weekly cron backstop is unnecessary for the same reason. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench(memory): pin GC between iterations in peak-shape scenarios A/A runs of the real memory instrument showed the two big-payload peak scenarios were still unstable with no timers involved: whether V8 collects iteration i's garbage before iteration i+1 allocates its payload is not reproducible run to run, so the measured peak flipped by a whole payload (serialization-payload solid: 6.8-9.2 MB across identical runs; peak-large-page: 3.5-3.7 MB). Peak scenarios measure the footprint of a single request, not accumulation, so pin the collection points: settle two event-loop turns (so trailing renderer/stream teardown is included) and force a GC after each iteration. Peak then deterministically measures max(single-request footprint). Churn scenarios are unchanged — their signal is accumulation, which a forced GC would mask; the README rule now spells out both sides. A/A validation (3 identical instrumented runs, solid suite): serialization-payload 6.8-9.2 MB -> 3.5-3.6 MB peak-large-page 3.5-3.7 MB -> 754-757 KB (0.45% spread) streaming-peak 38.5-39.5 -> 30.3/30.3/30.3 MB churn scenarios unchanged (aborted-requests 1.4 MB x3) Absolute values step because cross-iteration garbage no longer inflates the peak; the new values are the per-request footprint the scenarios were designed to track. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
854ac95b0d |
benchmark: memory (codspeed) (#7623) |