* bench(memory): settle-until-quiescent GC barrier between iterations
The fixed-turn settle before the pinned collection was hardware-fragile:
teardown needing one more event-loop turn on a given runner leaked a whole
payload of garbage past the collection point and flipped the measured peak
bimodally (serialization-payload solid/react resurfacing the pre-#7730
6.2MB attractor on some CI runners). Replace it with an adaptive barrier
that hops and collects until the post-GC heap size stops moving, with a
minimum-turn floor matching the fixed count it replaces (React's stream
teardown spans several turns and races a shorter floor).
Also pin collections in request-churn and the aborted-requests drain:
their peak floated on GC scheduling even with bit-identical work (18%
spread on CI for solid churn). Reachable accumulation stays visible to
the metric - a forced collection cannot reclaim leaked or cached objects,
it only removes floating garbage, whose collection timing was the
dominant cross-run noise source. Verified with a synthetic reachable
leak: +4.7% peak over a 0.6% A/A noise floor, where the old 18% noise
would have swallowed it.
Local A/A spreads (3x codspeed run per suite): solid <=1%, react <=2.2%
except request-churn (rare +130KB mode, framework teardown variance),
vue <=2.3% except aborted-requests (~8%, intra-iteration work variance,
down from 21% on CI).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): fixed-count settle barrier, drop adaptive quiescence exit
The adaptive exit (collect until post-GC heap size stops moving) made the
collection points land at data-dependent turns on CI: heap readings never
fully stabilize there, so the exit point - and every subsequent GC point -
shifted between identical runs. The first CI A/A pair showed it flipping
serialization-payload (vue) to a 6MB mode (67%), inflating peak-large-page
(vue) 3.4x, and adding 7-8% swings to the react/vue abort scenarios, while
fixing the solid suite.
Every step is now a fixed count: 16 settle turns (enough for React/Vue
multi-turn teardown that the original 2-turn barrier raced), one pinned
collection, one turn, one more collection for finalizer fallout. Fixed
counts make the GC points identical across runs by construction, which is
the property the barrier exists to provide.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): size iteration counts to the 2-10s measured-run window
On CI (eBPF instrument + --no-opt) one pinned iteration costs ~0.1-0.6s,
so the previous counts put request-churn at ~36s measured and the error
-paths/server-fn benches at 0.1-0.5s - outside the useful 2-10s range in
both directions. Retune: request-churn/aborted-requests/error-paths/
server-fn-churn to 40 iterations, serialization-payload to 12;
peak-large-page and streaming-peak already sit in range at 20.
The floor is raised by pinning error-paths and server-fn-churn rather
than inflating their iteration counts: thousands of unpinned iterations
would cross V8's own GC scheduling and reintroduce the timing noise the
barrier removes, while the pin's fixed per-iteration cost lifts them into
range at 40 iterations and makes their collection points deterministic.
Note: fewer churn iterations proportionally shrink the accumulation
window, so peak-based leak sensitivity drops accordingly; leaks remain
visible in total-allocated (forced compactions re-copy the growing live
set every iteration).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): disable V8 bytecode flushing in memory bench workers
The retuned (shorter) runs exposed a recurring V8 internal event - a
+7-13MB allocation burst with the signature of bytecode flush + lazy
recompile - that lands inside or outside the measured window at random,
flipping peaks by whole MBs between identical CI runs (request-churn vue
785KB vs 3.2MB). The pinned collections age code fast enough for V8 to
flush unused bytecode mid-run; --no-flush-bytecode keeps compiled code
alive for the process lifetime so the measured window sees no recompile
bursts. Vite's mergeConfig concatenates the scenario's test.execArgv with
the flags the CodSpeed plugin injects, so both apply to the fork workers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): pre-size V8 heap geometry in memory bench workers
--no-flush-bytecode alone did not stop the run-to-run flips: the
recurring event is heap-space growth, which allocates several MB at a
run-dependent moment inside the shortened measured windows (error-paths
unmatched vue 674KB vs 2MB, serialization-payload solid resurfacing its
6.4MB attractor on one run in three). Pre-sizing old space and pinning
the semi-space size removes mid-measurement growth while leaving the
per-iteration page map/unmap signal intact - local runs confirm peaks
still track payload sizes exactly (serialization 3.5MB, streaming
30.3MB, levels unchanged).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): size tiny benches just above the 2s measured-run floor
error-paths and server-fn-churn sat at 5-8s measured on CI; fewer
iterations bring them to ~2.3-3.4s, keeping the 2s floor while shrinking
the window a stray allocator event can land in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): verify the pinned collection against the heap floor
The residual CI flips all share one signature: exactly one iteration's
payload added to the measured peak (serialization-payload vue 3.5 vs
6.9MB, peak-large-page vue 780 vs 1349KB, error-paths redirect vue 300 vs
885KB). On some runners the response teardown holds the payload past the
fixed settle window - released only by a later internal timer - so it
survives the pinned collection and bleeds into the next iteration.
After the fixed 16-turn settle and collection, compare the post-GC heap
against the smallest post-collection size seen this run (the inter-
iteration floor): while it has not returned to the floor, keep hopping
and collecting, bounded at 64 turns. The barrier stays fixed-count in the
common case and extends only in exactly the failure case. Workloads that
genuinely accumulate reachable memory raise the floor as they go, so
accumulation still measures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): restrict floor verification to peak-shape scenarios
The floor-verified barrier fixed its target - the serialization family
went to <=0.45% across three CI runs, attractor gone - but destabilized
churn/abort scenarios (request-churn solid 420KB vs 3.4MB): their inter-
iteration floor legitimately drifts, so the verification capped out
chronically and the ~48 extra collections per iteration eventually hit a
compaction that double-copies the live set. Split by scenario shape:
serialization-payload, peak-large-page and streaming-peak keep the
verified barrier (flat floor, proven fix); churn and abort scenarios go
back to the plain fixed-count barrier that held them at <=1.7% in the
two previous CI rounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): stabilize CodSpeed memory benchmarks
Memory flags on PRs were ~85% noise, from two causes:
1. Stale baselines (dominant): the Benchmarks workflow only ran on main
pushes touching packages/** or benchmarks/**. Main commits outside
those paths had no CodSpeed run, so PRs were compared against an old
base run ([^unexpected-base] on every recent codspeed-hq comment; six
late-June PRs all compared to the same June-23 run). One outlier base
measurement then replays into every PR until the next eligible push.
Fix: drop the paths filter for the push trigger + weekly cron backstop.
2. Wall-clock timers racing the single measured run: aborted-requests
(solid/vue) resolved deferred loader data on 500/750ms timeouts raced
against abort, and drained cancellation with at most one setTimeout(0);
streaming-peak staggered deferred sections by milliseconds. Under
variable runner load and instrumentation slowdown the interleaving —
and therefore the allocation sequence — differed run to run. Fix:
count 0ms timers-phase hops (event-loop turns) instead of milliseconds
everywhere async ordering must be staged, and drain aborts with a
fixed 8-turn settlement barrier.
Local before/after evidence (8 fresh-process runs per config, CodSpeed V8
determinism flags, idle + CPU-stressed; kernel peak RSS spread):
aborted-requests/solid idle 11.2MB -> 1.8MB (GC counts bit-identical)
aborted-requests/solid stress 7.7MB -> 2.5MB
streaming-peak/solid idle 6.2MB -> 2.5MB
request-churn/solid (untouched control): unchanged, confirming harness
streaming-peak/vue keeps ~2% proxy-level variance at any tick count; its
PR flags were the identical -3.2% in all three occurrences, i.e. the
stale-base artifact rather than per-run jitter.
Bench names and workload shapes are unchanged; absolute values will step
once on the CodSpeed dashboard from the removed cross-iteration overlap
(e.g. aborted-requests/solid retained 15.6MB -> 2.5MB per invocation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): exclude docs/examples/e2e from main benchmark runs, drop cron
paths-ignore instead of no filter: docs/examples/e2e-only pushes cannot
affect benchmark results, and falling back across such a commit is safe
because the previous run's benchmark-relevant code is identical. The
weekly cron backstop is unnecessary for the same reason.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): pin GC between iterations in peak-shape scenarios
A/A runs of the real memory instrument showed the two big-payload peak
scenarios were still unstable with no timers involved: whether V8
collects iteration i's garbage before iteration i+1 allocates its
payload is not reproducible run to run, so the measured peak flipped by
a whole payload (serialization-payload solid: 6.8-9.2 MB across
identical runs; peak-large-page: 3.5-3.7 MB).
Peak scenarios measure the footprint of a single request, not
accumulation, so pin the collection points: settle two event-loop turns
(so trailing renderer/stream teardown is included) and force a GC after
each iteration. Peak then deterministically measures
max(single-request footprint). Churn scenarios are unchanged — their
signal is accumulation, which a forced GC would mask; the README rule
now spells out both sides.
A/A validation (3 identical instrumented runs, solid suite):
serialization-payload 6.8-9.2 MB -> 3.5-3.6 MB
peak-large-page 3.5-3.7 MB -> 754-757 KB (0.45% spread)
streaming-peak 38.5-39.5 -> 30.3/30.3/30.3 MB
churn scenarios unchanged (aborted-requests 1.4 MB x3)
Absolute values step because cross-iteration garbage no longer inflates
the peak; the new values are the per-request footprint the scenarios
were designed to track.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>