0
Fork 0
mirror of https://github.com/TanStack/router.git synced 2026-11-16 19:38:07 +00:00
router/benchmarks/client-nav
Flo 208100b7c0
feat(benchmarks): client-side CPU benchmark scenario suite (#7732)
* feat(benchmarks): add client-side CPU benchmark scenario suite

Add 11 client-side CPU benchmark scenarios to benchmarks/client-nav, each
implemented identically across react/solid/vue and tracked by CodSpeed
through the existing Benchmarks workflow (no CI changes needed):

async-pipeline, control-flow, head, history, links, loaders, mount,
nested-params, preload, route-tree-scale, search-params

Each scenario is an isolated file-based-routing app (route trees generated
by @tanstack/router-plugin) built for production and driven in jsdom by a
shared harness (scenarios/harness.ts): circular step sequences of real
<Link> clicks (plus hover/preload/history-traversal/invalidate steps),
each synchronized on the router's onRendered event, with a warm-up lap
that asserts every step's observable output. Determinism rules: no
wall-clock timers (counted 0ms hops only), staleTime/gcTime pinned to 0 or
1e9, deterministic preload staleness, stationary DOM and history depth.

The existing baseline apps and bench names are untouched for CodSpeed
continuity; baseline vite configs only gain an explicit root/setupFiles so
they resolve identically under the new per-framework aggregate configs.

Notable constraints encoded in the scenarios:
- Component-level Await/Suspense is excluded from async-pipeline: React 19
  throttles every Suspense reveal by ~300ms wall-clock, which is
  inherently non-deterministic to benchmark.
- control-flow/react silences React 19's onCaughtError reporting, which
  otherwise dominates the measured loop with console I/O.
- Vue route components must be defined before their createFileRoute call
  (bundled var hoisting silently yields component: undefined otherwise).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: apply automated fixes

* chore(benchmarks): prefix client-nav bench names with "client-"

All benches in @benchmarks/client-nav now start with "client" so they are
easy to identify on the CodSpeed dashboard (the baseline already did).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(benchmarks): address review findings on client-nav scenario stability

Fixes from an adversarial review of the scenario suite:

- mount: create the browser history explicitly and destroy() it on unmount.
  The default per-router createBrowserHistory() monkey-patches
  window.history.pushState/replaceState, chaining one wrapper per router —
  per-mount cost degraded 1.8ms -> 6ms over 3000 mounts (O(N^2) drift in
  the cold-start bench). Verified flat (~1.86ms) after the fix.
- links: replace the five <MatchRoute> components with setup-scoped
  useMatchRoute probes in all three apps. vue-router's MatchRoute calls
  useMatchRoute() inside its render function, leaking one undisposed
  watcher per render — the Vue bench accumulated ~5 dead subscribers per
  navigation. Verified flat per-tick cost over 2000 ticks after the fix.
- all scenarios: bound the scroll-restoration cache with
  getScrollRestorationKey: (location) => location.pathname. The default
  key is a fresh random per-entry location key, so the module-level cache
  grew one entry per push navigation for the whole run.
- harness hover step: dispatch a single mouseover instead of
  mouseover+mouseenter. Solid/Vue attach intent-preload handlers to BOTH
  events, so the old double dispatch ran the preload pipeline twice for
  them but once for React. Verified all three adapters preload exactly
  once per hover via cached-matches probes.
- README: correct the loaders row (the invalidate step means cached routes
  re-run once per lap), the links row, and document the session-history
  growth characteristics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(benchmarks): cover client rewrites, lazy route chunks, encoded params

Closes the three coverage gaps identified in the scenario-suite analysis:

- new `rewrites` scenario (react/solid/vue): router basepath '/app' composed
  with a locale input/output rewrite pair — the client analog of the SSR
  rewrites scenario. Every href build runs the output rewrite and every
  committed location runs the input rewrite; assertions check both the
  internal router pathname and the external window pathname. The harness
  gained an `initialUrl` option since a basepath app cannot start at '/'.
- route-tree-scale now builds with `autoCodeSplitting: true`, so its
  navigations also exercise lazy route-chunk resolution — the default shape
  of real file-based apps, previously uncovered.
- nested-params and route-tree-scale param sets now include characters that
  need percent-encoding (spaces, `&`, `%`, `+`, unicode) so the segment
  encode/decode paths run on every navigation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(benchmarks): reduce syscall time in measured loops, lengthen mount bench

CodSpeed excludes syscall time from simulation measures, and past a
threshold it warns ("cannot be consistently instrumented") and skips the
benchmark; the exclusion is also inconsistent run-to-run, which was the
root cause of the two same-code CI swings (route-tree-scale react 13%,
async-pipeline react 3.6%) and of skipped runs on hop-heavy benches like
async-pipeline solid (579 calls / 12.1ms).

- Replace all counted setTimeout(0) hops with setImmediate hops (harness
  timerHop + async-pipeline hopDelay). Both are deterministic event-loop
  turns that yield to timers and the React Node scheduler, but a timer
  costs ~3-4 syscalls (timerfd + epoll) per hop vs ~1 for an immediate.
  Same-machine instrumented before/after: preload solid 39.8ms -> 1.2ms of
  syscall time, preload vue 30.1 -> 1.1, async-pipeline solid 23.5 -> 1.1,
  async-pipeline vue 17.9 -> 0.9. head/route-tree-scale are unchanged
  (their residual syscalls are GC/scheduler, and their measures were
  already stable); history's syscalls come from jsdom's internal
  timer-queued traversal and are not reachable from bench code (its
  measured values are stable regardless).

- mount: 6 ticks per iteration instead of 2. A single mount simulates to
  only ~8ms and very short measures amplify allocator/GC quantization;
  the bench now measures ~50ms+ of simulated CPU.

All 39 benches pass; full nx typecheck green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(benchmarks): stop history benches sleeping in jsdom's traversal timers

The history benches were skip-warned by CodSpeed for excessive syscall
time (5-26ms, highly variable across runs). Root cause: jsdom delivers
history traversals through two nested window.setTimeout(0) tasks, and Node
clamps zero timers to 1ms of wall time — so while awaiting onRendered
after back/forward/go, the event loop blocked in epoll_wait until each
timer expired. That blocked wall-time is recorded as syscall time, and its
duration depends on host timer behavior, hence the variability.

The harness now reroutes zero-delay window.setTimeout calls onto
setImmediate for the duration of a benchmark (non-zero delays pass
through; clearTimeout handles both). Zero-delay timeouts carry no ordering
semantics a check-phase immediate doesn't satisfy, and the suite's
conventions already forbid real-delay timers in measured code.

Instrumented A/A verification (2 runs per framework): history syscall time
react 23-26ms -> 2.5ms, solid 14-23ms -> 1.1ms, vue 11-12ms -> 1.2ms, with
bit-identical syscall counts across runs and measured values stable to
0.06%. The benches also stopped measuring sleep: locally an iteration
dropped from ~42-49ms to ~5-11ms of actual CPU work. All 39 benches pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
History 2026-07-03 08:18:08 +02:00
..
react feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
scenarios feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
solid feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
vue feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
jsdom.ts feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
package.json feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
README.md feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
setup-helpers.ts test: broaden client-nav benchmark coverage (#7010) 2026-03-22 13:10:11 +01:00
tsconfig.json test: add cross-framework client navigation benchmarks (#6800) 2026-03-02 21:23:53 +01:00
vitest.config.ts feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
vitest.react.config.ts feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
vitest.setup.ts test: add cross-framework client navigation benchmarks (#6800) 2026-03-02 21:23:53 +01:00
vitest.solid.config.ts feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00
vitest.vue.config.ts feat(benchmarks): client-side CPU benchmark scenario suite (#7732) 2026-07-03 08:18:08 +02:00

Client Navigation Benchmarks

Cross-framework client-side CPU benchmarks for:

  • @tanstack/react-router
  • @tanstack/solid-router
  • @tanstack/vue-router

The benchmarks run in jsdom against production builds of real apps, and are tracked in CI by CodSpeed (simulation mode).

Scope: these benchmarks cover the standalone client router only. The client side of TanStack Start (hydration of a server-rendered document, streamed payload consumption, server-function calls from the client, ...) is not covered here; the server side of Start is covered by benchmarks/ssr.

Layout

  • react/, solid/, vue/ - baseline benchmark (mixed navigation loop) + Vitest config
  • vitest.react.config.ts, vitest.solid.config.ts, vitest.vue.config.ts - per-framework aggregate configs that run the baseline first, then scenario projects
  • scenarios/harness.ts - shared scenario runner (mount, link-click steps, onRendered synchronization)
  • scenarios/<scenario>/shared.ts - framework-agnostic scenario definition (workload data, step sequence, assertions, bench options)
  • scenarios/<scenario>/<framework>/ - isolated scenario apps

Scenario app layout:

scenarios/<scenario>/<framework>/
  vite.config.ts
  speed.bench.ts
  speed.flame.ts
  setup.ts
  project.json
  tsconfig.json
  src/
    main.tsx
    routeTree.gen.ts
    routes/

Scenario apps use file-based routing (@tanstack/router-plugin) with a generated routeTree.gen.ts, like a regular user app. Each scenario uses one app per framework instead of sharing routes in the baseline app. This keeps route-tree size and router options isolated so one scenario cannot shift another scenario's numbers. The existing baseline apps and bench names stay stable for CodSpeed continuity.

Scenario Responsibilities

Each scenario isolates one client-side responsibility so benchmark changes can be attributed to a specific feature area.

Scenario Client-side responsibility
react/, solid/, vue/ baseline apps Mixed navigation loop: path params, search params, route context, and useParams/useSearch/useLoaderData selector subscriptions.
async-pipeline The router's async pipeline via counted 0ms timer hops: async loaders (transition-held navigation), async beforeLoad context, and parallel nested async loaders. Component-level Await/Suspense is excluded: React 19 throttles Suspense reveals by ~300ms wall-clock, which is inherently non-deterministic to benchmark.
control-flow Loader-thrown redirect (including a 2-hop chain), notFound() with notFoundComponent, loader errors with errorComponent, and boundary reset on recovery navigation.
head HeadContent per-navigation work: nested route head() evaluation, title/meta/link dedupe across matches, and head tag DOM updates during navigation.
history History push/replace/back/forward traversal, location masking, registered-but-never-blocking useBlocker, and useCanGoBack/useLocation subscriptions.
links Per-navigation cost of ~200 mounted <Link>s: link prop building, active-state recompute across activeOptions variants, activeProps swaps, and useMatchRoute probes (the MatchRoute component is avoided: vue-router's implementation leaks one subscription per render).
loaders Client loader dispatch: always-stale re-runs (staleTime: 0), cached revisits (re-run once per lap by the invalidate step), loaderDeps-keyed caching, router.invalidate(), and useLoaderData selectors.
mount Cold start: createRouter (route-tree processing) + first render + initial router.load() + unmount, with a fresh router per mount. The only scenario measuring router creation.
nested-params Deep nesting (8 dynamic levels): per-level params.parse/stringify, beforeLoad context accumulation across matches, and per-level useParams/useRouteContext subscriptions. Param values include characters requiring percent-encoding (as do route-tree-scale's), so segment encode/decode paths run on every navigation.
preload Intent preloading from hover events, programmatic router.preloadRoute, deterministic preload cache behavior (defaultPreloadStaleTime: 0), and commit-time cache maintenance.
rewrites Composed client-side location rewrites: router basepath plus a locale input/output rewrite pair, running on every href build and location parse (the client analog of the SSR rewrites scenario).
route-tree-scale Route matching and link-target resolution on a wide (~40 route) tree mixing static, dynamic, prefixed-param, splat, pathless-layout, and route-group paths, with autoCodeSplitting enabled so navigations also resolve lazy route chunks.
search-params validateSearch execution, search middlewares (retainSearchParams/stripSearchParams), functional search updaters, structural sharing, and useSearch selector subscriptions.

Conventions

  • Apps are built with NODE_ENV=production (minify: false) into dist/app.js; benches import the built bundle, so production package builds and production JSX output are measured, not dev transforms.
  • Scenarios behave like a real user app: navigation happens through <Link> clicks dispatched on real anchor elements (unless a scenario specifically measures the imperative API), the router uses the default browser history, and scrollRestoration is enabled.
  • Each benchmark iteration advances a fixed, circular sequence of steps; every step awaits the router's onRendered event, so render work is included and steps cannot overlap. No two consecutive steps may target the same location, and the sequence ends back on the initial route.
  • Setup runs one warm-up lap through the sequence and asserts each step's observable output (e.g. document.title), so a scenario that silently stops doing its work fails instead of reporting a fast time.
  • Determinism: no wall-clock timers (async work is resolved promises or counted setTimeout(0) hops), staleTime/gcTime only ever 0 or effectively-infinite, no Math.random/Date.now in the measured loop. Scroll restoration uses getScrollRestorationKey: (location) => location.pathname so its cache stays bounded (the default random per-entry key grows one cache entry per push navigation). The mount scenario opts out of scrollRestoration entirely and destroys its history on unmount — both register page-lifetime globals that would leak across its mount/unmount loop.
  • Push-only step laps grow jsdom's session-history entry list over a run (jsdom never truncates entries behind the current index). Per-push cost stays O(1) so timings are unaffected; only the history scenario needs — and has — a lap that is depth-stationary.
  • Calibration: pick per-iteration step counts so a bench's vitest bench run stays roughly between 8 and 30 seconds (long enough to average out, short enough for CI).

Run

Run all benchmarks through Nx so dependency builds are part of the graph:

CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf --outputStyle=stream --skipRemoteCache

Run framework-specific benchmarks (baseline + all scenarios):

CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:react --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:solid --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:perf:vue --outputStyle=stream --skipRemoteCache

Run a single scenario app manually (after building it through Nx):

CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-<scenario>-<framework>:build:client --outputStyle=stream --skipRemoteCache
cd benchmarks/client-nav && NODE_ENV=production vitest bench --config ./scenarios/<scenario>/<framework>/vite.config.ts

Run framework-specific flame benchmarks (10 second loop, profiled with @platformatic/flame, forced to NODE_ENV=production):

# baseline
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:flame:react --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:flame:solid --outputStyle=stream --skipRemoteCache
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:flame:vue --outputStyle=stream --skipRemoteCache
# scenarios
CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav-<scenario>-<framework>:test:flame --outputStyle=stream --skipRemoteCache

Typecheck benchmark sources (baseline + scenarios):

CI=1 NX_DAEMON=false pnpm nx run @benchmarks/client-nav:test:types --outputStyle=stream --skipRemoteCache