* feat(benchmarks): add client-side CPU benchmark scenario suite
Add 11 client-side CPU benchmark scenarios to benchmarks/client-nav, each
implemented identically across react/solid/vue and tracked by CodSpeed
through the existing Benchmarks workflow (no CI changes needed):
async-pipeline, control-flow, head, history, links, loaders, mount,
nested-params, preload, route-tree-scale, search-params
Each scenario is an isolated file-based-routing app (route trees generated
by @tanstack/router-plugin) built for production and driven in jsdom by a
shared harness (scenarios/harness.ts): circular step sequences of real
<Link> clicks (plus hover/preload/history-traversal/invalidate steps),
each synchronized on the router's onRendered event, with a warm-up lap
that asserts every step's observable output. Determinism rules: no
wall-clock timers (counted 0ms hops only), staleTime/gcTime pinned to 0 or
1e9, deterministic preload staleness, stationary DOM and history depth.
The existing baseline apps and bench names are untouched for CodSpeed
continuity; baseline vite configs only gain an explicit root/setupFiles so
they resolve identically under the new per-framework aggregate configs.
Notable constraints encoded in the scenarios:
- Component-level Await/Suspense is excluded from async-pipeline: React 19
throttles every Suspense reveal by ~300ms wall-clock, which is
inherently non-deterministic to benchmark.
- control-flow/react silences React 19's onCaughtError reporting, which
otherwise dominates the measured loop with console I/O.
- Vue route components must be defined before their createFileRoute call
(bundled var hoisting silently yields component: undefined otherwise).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: apply automated fixes
* chore(benchmarks): prefix client-nav bench names with "client-"
All benches in @benchmarks/client-nav now start with "client" so they are
easy to identify on the CodSpeed dashboard (the baseline already did).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(benchmarks): address review findings on client-nav scenario stability
Fixes from an adversarial review of the scenario suite:
- mount: create the browser history explicitly and destroy() it on unmount.
The default per-router createBrowserHistory() monkey-patches
window.history.pushState/replaceState, chaining one wrapper per router —
per-mount cost degraded 1.8ms -> 6ms over 3000 mounts (O(N^2) drift in
the cold-start bench). Verified flat (~1.86ms) after the fix.
- links: replace the five <MatchRoute> components with setup-scoped
useMatchRoute probes in all three apps. vue-router's MatchRoute calls
useMatchRoute() inside its render function, leaking one undisposed
watcher per render — the Vue bench accumulated ~5 dead subscribers per
navigation. Verified flat per-tick cost over 2000 ticks after the fix.
- all scenarios: bound the scroll-restoration cache with
getScrollRestorationKey: (location) => location.pathname. The default
key is a fresh random per-entry location key, so the module-level cache
grew one entry per push navigation for the whole run.
- harness hover step: dispatch a single mouseover instead of
mouseover+mouseenter. Solid/Vue attach intent-preload handlers to BOTH
events, so the old double dispatch ran the preload pipeline twice for
them but once for React. Verified all three adapters preload exactly
once per hover via cached-matches probes.
- README: correct the loaders row (the invalidate step means cached routes
re-run once per lap), the links row, and document the session-history
growth characteristics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(benchmarks): cover client rewrites, lazy route chunks, encoded params
Closes the three coverage gaps identified in the scenario-suite analysis:
- new `rewrites` scenario (react/solid/vue): router basepath '/app' composed
with a locale input/output rewrite pair — the client analog of the SSR
rewrites scenario. Every href build runs the output rewrite and every
committed location runs the input rewrite; assertions check both the
internal router pathname and the external window pathname. The harness
gained an `initialUrl` option since a basepath app cannot start at '/'.
- route-tree-scale now builds with `autoCodeSplitting: true`, so its
navigations also exercise lazy route-chunk resolution — the default shape
of real file-based apps, previously uncovered.
- nested-params and route-tree-scale param sets now include characters that
need percent-encoding (spaces, `&`, `%`, `+`, unicode) so the segment
encode/decode paths run on every navigation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(benchmarks): reduce syscall time in measured loops, lengthen mount bench
CodSpeed excludes syscall time from simulation measures, and past a
threshold it warns ("cannot be consistently instrumented") and skips the
benchmark; the exclusion is also inconsistent run-to-run, which was the
root cause of the two same-code CI swings (route-tree-scale react 13%,
async-pipeline react 3.6%) and of skipped runs on hop-heavy benches like
async-pipeline solid (579 calls / 12.1ms).
- Replace all counted setTimeout(0) hops with setImmediate hops (harness
timerHop + async-pipeline hopDelay). Both are deterministic event-loop
turns that yield to timers and the React Node scheduler, but a timer
costs ~3-4 syscalls (timerfd + epoll) per hop vs ~1 for an immediate.
Same-machine instrumented before/after: preload solid 39.8ms -> 1.2ms of
syscall time, preload vue 30.1 -> 1.1, async-pipeline solid 23.5 -> 1.1,
async-pipeline vue 17.9 -> 0.9. head/route-tree-scale are unchanged
(their residual syscalls are GC/scheduler, and their measures were
already stable); history's syscalls come from jsdom's internal
timer-queued traversal and are not reachable from bench code (its
measured values are stable regardless).
- mount: 6 ticks per iteration instead of 2. A single mount simulates to
only ~8ms and very short measures amplify allocator/GC quantization;
the bench now measures ~50ms+ of simulated CPU.
All 39 benches pass; full nx typecheck green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(benchmarks): stop history benches sleeping in jsdom's traversal timers
The history benches were skip-warned by CodSpeed for excessive syscall
time (5-26ms, highly variable across runs). Root cause: jsdom delivers
history traversals through two nested window.setTimeout(0) tasks, and Node
clamps zero timers to 1ms of wall time — so while awaiting onRendered
after back/forward/go, the event loop blocked in epoll_wait until each
timer expired. That blocked wall-time is recorded as syscall time, and its
duration depends on host timer behavior, hence the variability.
The harness now reroutes zero-delay window.setTimeout calls onto
setImmediate for the duration of a benchmark (non-zero delays pass
through; clearTimeout handles both). Zero-delay timeouts carry no ordering
semantics a check-phase immediate doesn't satisfy, and the suite's
conventions already forbid real-delay timers in measured code.
Instrumented A/A verification (2 runs per framework): history syscall time
react 23-26ms -> 2.5ms, solid 14-23ms -> 1.1ms, vue 11-12ms -> 1.2ms, with
bit-identical syscall counts across runs and measured values stable to
0.06%. The benches also stopped measuring sleep: locally an iteration
dropped from ~42-49ms to ~5-11ms of actual CPU work. All 39 benches pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
11 KiB
Client Navigation Benchmarks
Cross-framework client-side CPU benchmarks for:
@tanstack/react-router@tanstack/solid-router@tanstack/vue-router
The benchmarks run in jsdom against production builds of real apps, and are tracked in CI by CodSpeed (simulation mode).
Scope: these benchmarks cover the standalone client router only. The client side of TanStack Start (hydration of a server-rendered document, streamed payload consumption, server-function calls from the client, ...) is not covered here; the server side of Start is covered by
benchmarks/ssr.
Layout
react/,solid/,vue/- baseline benchmark (mixed navigation loop) + Vitest configvitest.react.config.ts,vitest.solid.config.ts,vitest.vue.config.ts- per-framework aggregate configs that run the baseline first, then scenario projectsscenarios/harness.ts- shared scenario runner (mount, link-click steps,onRenderedsynchronization)scenarios/<scenario>/shared.ts- framework-agnostic scenario definition (workload data, step sequence, assertions, bench options)scenarios/<scenario>/<framework>/- isolated scenario apps
Scenario app layout:
scenarios/<scenario>/<framework>/
vite.config.ts
speed.bench.ts
speed.flame.ts
setup.ts
project.json
tsconfig.json
src/
main.tsx
routeTree.gen.ts
routes/
Scenario apps use file-based routing (@tanstack/router-plugin) with a
generated routeTree.gen.ts, like a regular user app. Each scenario uses one
app per framework instead of sharing routes in the baseline app. This keeps
route-tree size and router options isolated so one scenario cannot shift
another scenario's numbers. The existing baseline apps and bench names stay
stable for CodSpeed continuity.
Scenario Responsibilities
Each scenario isolates one client-side responsibility so benchmark changes can be attributed to a specific feature area.
| Scenario | Client-side responsibility |
|---|---|
react/, solid/, vue/ baseline apps |
Mixed navigation loop: path params, search params, route context, and useParams/useSearch/useLoaderData selector subscriptions. |
async-pipeline |
The router's async pipeline via counted 0ms timer hops: async loaders (transition-held navigation), async beforeLoad context, and parallel nested async loaders. Component-level Await/Suspense is excluded: React 19 throttles Suspense reveals by ~300ms wall-clock, which is inherently non-deterministic to benchmark. |
control-flow |
Loader-thrown redirect (including a 2-hop chain), notFound() with notFoundComponent, loader errors with errorComponent, and boundary reset on recovery navigation. |
head |
HeadContent per-navigation work: nested route head() evaluation, title/meta/link dedupe across matches, and head tag DOM updates during navigation. |
history |
History push/replace/back/forward traversal, location masking, registered-but-never-blocking useBlocker, and useCanGoBack/useLocation subscriptions. |
links |
Per-navigation cost of ~200 mounted <Link>s: link prop building, active-state recompute across activeOptions variants, activeProps swaps, and useMatchRoute probes (the MatchRoute component is avoided: vue-router's implementation leaks one subscription per render). |
loaders |
Client loader dispatch: always-stale re-runs (staleTime: 0), cached revisits (re-run once per lap by the invalidate step), loaderDeps-keyed caching, router.invalidate(), and useLoaderData selectors. |
mount |
Cold start: createRouter (route-tree processing) + first render + initial router.load() + unmount, with a fresh router per mount. The only scenario measuring router creation. |
nested-params |
Deep nesting (8 dynamic levels): per-level params.parse/stringify, beforeLoad context accumulation across matches, and per-level useParams/useRouteContext subscriptions. Param values include characters requiring percent-encoding (as do route-tree-scale's), so segment encode/decode paths run on every navigation. |
preload |
Intent preloading from hover events, programmatic router.preloadRoute, deterministic preload cache behavior (defaultPreloadStaleTime: 0), and commit-time cache maintenance. |
rewrites |
Composed client-side location rewrites: router basepath plus a locale input/output rewrite pair, running on every href build and location parse (the client analog of the SSR rewrites scenario). |
route-tree-scale |
Route matching and link-target resolution on a wide (~40 route) tree mixing static, dynamic, prefixed-param, splat, pathless-layout, and route-group paths, with autoCodeSplitting enabled so navigations also resolve lazy route chunks. |
search-params |
validateSearch execution, search middlewares (retainSearchParams/stripSearchParams), functional search updaters, structural sharing, and useSearch selector subscriptions. |
Conventions
- Apps are built with
NODE_ENV=production(minify: false) intodist/app.js; benches import the built bundle, so production package builds and production JSX output are measured, not dev transforms. - Scenarios behave like a real user app: navigation happens through
<Link>clicks dispatched on real anchor elements (unless a scenario specifically measures the imperative API), the router uses the default browser history, andscrollRestorationis enabled. - Each benchmark iteration advances a fixed, circular sequence of steps; every step awaits the router's
onRenderedevent, so render work is included and steps cannot overlap. No two consecutive steps may target the same location, and the sequence ends back on the initial route. - Setup runs one warm-up lap through the sequence and asserts each step's observable output (e.g.
document.title), so a scenario that silently stops doing its work fails instead of reporting a fast time. - Determinism: no wall-clock timers (async work is resolved promises or counted
setTimeout(0)hops),staleTime/gcTimeonly ever0or effectively-infinite, noMath.random/Date.nowin the measured loop. Scroll restoration usesgetScrollRestorationKey: (location) => location.pathnameso its cache stays bounded (the default random per-entry key grows one cache entry per push navigation). Themountscenario opts out ofscrollRestorationentirely and destroys its history on unmount — both register page-lifetime globals that would leak across its mount/unmount loop. - Push-only step laps grow jsdom's session-history entry list over a run (jsdom never truncates entries behind the current index). Per-push cost stays O(1) so timings are unaffected; only the
historyscenario needs — and has — a lap that is depth-stationary. - Calibration: pick per-iteration step counts so a bench's
vitest benchrun stays roughly between 8 and 30 seconds (long enough to average out, short enough for CI).
Run
Run all benchmarks through Nx so dependency builds are part of the graph:
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf --outputStyle=stream --skipRemoteCache
Run framework-specific benchmarks (baseline + all scenarios):
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:react --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:solid --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:vue --outputStyle=stream --skipRemoteCache
Run a single scenario app manually (after building it through Nx):
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-<scenario>-<framework>:build:client --outputStyle=stream --skipRemoteCache
cd benchmarks/client-nav && NODE_ENV=production vitest bench --config ./scenarios/<scenario>/<framework>/vite.config.ts
Run framework-specific flame benchmarks (10 second loop, profiled with @platformatic/flame, forced to NODE_ENV=production):
# baseline
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:flame:react --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:flame:solid --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:flame:vue --outputStyle=stream --skipRemoteCache
# scenarios
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-<scenario>-<framework>:test:flame --outputStyle=stream --skipRemoteCache
Typecheck benchmark sources (baseline + scenarios):
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:types --outputStyle=stream --skipRemoteCache