diff --git a/.github/workflows/client-nav-benchmarks.yml b/.github/workflows/client-nav-benchmarks.yml index 7eedc222ae..a74af25d29 100644 --- a/.github/workflows/client-nav-benchmarks.yml +++ b/.github/workflows/client-nav-benchmarks.yml @@ -20,9 +20,14 @@ on: - 'packages/**' - 'benchmarks/**' workflow_dispatch: + inputs: + memory-only: + description: Run only the memory benchmarks + type: boolean + default: false concurrency: - group: client-nav-benchmarks-${{ github.ref }} + group: client-nav-benchmarks-${{ github.event_name == 'workflow_dispatch' && github.run_id || github.ref }} cancel-in-progress: false queue: single @@ -45,20 +50,7 @@ jobs: - react - solid - vue - benchmark: - - client-nav - - ssr - - memory-server - - memory-client - include: - - benchmark: client-nav - mode: simulation - - benchmark: ssr - mode: simulation - - benchmark: memory-server - mode: memory - - benchmark: memory-client - mode: memory + benchmark: ${{ fromJSON(inputs.memory-only && '["memory-server", "memory-client"]' || '["client-nav", "ssr", "memory-server", "memory-client"]') }} runs-on: ubuntu-latest steps: - name: Checkout @@ -80,7 +72,7 @@ jobs: uses: CodSpeedHQ/action@4296e51e7041e24dadb86d1d6e8b9320d223dbe8 # v5.0.3 timeout-minutes: 20 with: - mode: ${{ matrix.mode }} + mode: ${{ startsWith(matrix.benchmark, 'memory-') && 'memory' || 'simulation' }} run: >- WITH_INSTRUMENTATION=1 pnpm nx run diff --git a/benchmarks/memory/README.md b/benchmarks/memory/README.md index cf871f3439..e60c0c68d8 100644 --- a/benchmarks/memory/README.md +++ b/benchmarks/memory/README.md @@ -43,40 +43,51 @@ same workload through the Flame profiler. after a forced GC. Under plain `vitest bench` the suites only smoke-test: timing output is meaningless; real numbers come from CodSpeed. - Under CodSpeed the bench fn runs several warmup invocations plus the - measured one **on the same mount**, so bench fns must be idempotent and - module-level counters/LCGs are used where ids must never repeat across - invocations. -- Plain `vitest bench` never runs suite hooks (`beforeAll`/`afterAll`) and + measured one. Client suite hooks mount a fresh app before each invocation + and tear it down afterward, outside the measurement window. This prevents + warmup navigation history and caches from changing the starting state. + Module-level counters/LCGs keep generated URLs unique across invocations. +- Plain `vitest bench` never runs suite hooks (`beforeEach`/`afterEach`) and only honors tinybench's `setup`/`teardown` options; the CodSpeed runner does the exact opposite. Client benches therefore register **both** — in any given mode exactly one pair runs. -- The process runs with V8 determinism flags (predictable GC schedule, - `--no-opt`). Never call `global.gc()` manually in **churn** scenarios — - their signal is accumulation across iterations, which a forced collection - masks. **Peak** scenarios do the opposite: they set - `pinGcBetweenIterations` on the request loop so a collection runs between - iterations. Their signal is the footprint of a single request, and without - pinned GC points the measured peak flips by a whole payload depending on - whether iteration i's garbage is collected before iteration i+1 allocates. - Because of `--no-opt`, allocation counts overstate production; numbers are - for regression tracking, not absolute claims. +- Worker flags live in `runtime.ts` and apply only to the CodSpeed memory + instrument. They supplement the integration's predictable execution flags. + Disabling machine-code generation and bytecode flushing keeps compiler allocations + out of the signal. GC follows allocation thresholds without incremental + marking or minor-GC tasks; a fixed initial old-generation budget reduces + sensitivity to startup heap policy. +- Before each CodSpeed invocation, a setup hook yields to the event loop, + outside the measurement marker and before CodSpeed forces GC. Promise-only + warmup chains can keep WeakRef targets alive through a collection in the same + job. Client app setup also runs before the forced GC. +- Do not force GC inside a measured request loop or poll `heapUsed` to choose + how much work to do. Both introduce allocator activity from the harness into + the measurement. Footprint scenarios run exactly one large request per + invocation; churn scenarios retain their fixed sequential request counts. +- Error-path cases use separate isolated workers so one case's module and + allocator history cannot contaminate the next case. - Keep each bench under **~1.5M allocations** (instrument overhead grows past 2M); this is the main constraint when tuning iteration counts. ## Bench shapes and signals -- **Churn (leak detector):** N sequential iterations at steady state. If one - iteration leaks L bytes, peak grows by ~N·L; healthy builds show a flat - timeline floor independent of N. Tuning check: doubling N must leave peak - roughly unchanged. -- **Peak (footprint):** one (or very few) large operations; peak memory - scaling with the workload is the signal. +- **Churn:** a fixed number of sequential operations, measuring native + allocation activity and peak outstanding native allocations. +- **Peak (footprint):** one large operation, measuring its native footprint. + +CodSpeed reports native allocator activity, not individual JavaScript object +allocations or retained JS heap size. V8 can reuse heap pages without a visible +native allocation. These benchmarks can detect native buffering and allocator +regressions, but a flat graph does not establish that route data or JS objects +are collectable. Absolute numbers include Node/framework/jsdom activity under +benchmark-specific V8 settings; compare identical harness versions. ## Scenarios ### Server -| Scenario | Shape | Guards against | +| Scenario | Shape | Exercises | | ----------------------- | ----- | --------------------------------------------------------- | | `request-churn` | churn | cross-request retention in document SSR (unique URLs) | | `server-fn-churn` | churn | retention in the server-function RPC path | @@ -88,7 +99,7 @@ same workload through the Flame profiler. ### Client -| Scenario | Shape | Guards against | +| Scenario | Shape | Exercises | | ------------------------- | ----- | -------------------------------------------------------- | | `navigation-churn` | churn | per-navigation retention at steady state | | `unique-location-churn` | churn | unbounded href/search-keyed caches (never-repeated URLs) | @@ -103,14 +114,10 @@ same workload through the Flame profiler. server response is fully consumed before the next request. Pairing a single navigation with its render signal via `Promise.all([navigate, rendered])` is fine — never overlap distinct work items. -- Randomness only via the seeded LCG in `bench-utils.ts`; no `Math.random`, - `Date.now`, or timers with a nonzero delay — anywhere async ordering must be - staged (`streaming-peak`'s deferred sections, `aborted-requests`' deferred - loader data and post-abort drain), chain **0ms `setTimeout` hops** and count - event-loop turns instead. A wall-clock delay races renderer work differently - depending on runner load and instrumentation overhead, which makes the - single measured run non-reproducible; 0ms hops fire in registration order in - the timers phase, so the interleaving is a pure function of the schedule. +- Randomness only via the seeded LCG in `bench-utils.ts`. Deferred streaming + uses counted zero-delay timer hops to stage work. Timers still depend on the + event loop, so validate repeatability on CI instead of assuming deterministic + ordering from a zero delay alone. - Sanity assertions run once at module load and throw on wrong status/markers, so a bench can never silently measure the wrong thing. - Server requests follow `benchmarks/ssr` conventions: document GETs send @@ -184,7 +191,7 @@ pnpm --filter @benchmarks/memory-client clean:profiles ``` Client memory benches are useful for regression tracking of router/React/jsdom -integration behavior, especially retained route/cache data. They are not pure +integration behavior and native allocation activity. They are not pure browser-memory measurements, and local Flame attribution can include jsdom, React DOM, and profiler shutdown frames. @@ -200,3 +207,16 @@ WITH_INSTRUMENTATION=1 codspeed run --mode memory -- pnpm nx run @benchmarks/mem WITH_INSTRUMENTATION=1 codspeed run --mode memory -- pnpm nx run @benchmarks/memory-client:test:perf:solid WITH_INSTRUMENTATION=1 codspeed run --mode memory -- pnpm nx run @benchmarks/memory-client:test:perf:vue ``` + +Run only memory jobs on an experiment branch: + +```bash +gh workflow run client-nav-benchmarks.yml --ref -f memory-only=true +``` + +Repeat at the same commit, using distinct branch names if runs should overlap +(the workflow serializes runs on the same ref). Compare all 48 fresh memory +results: 18 client and 30 server. Exclude inherited CodSpeed results and failed +or incomplete repetitions. For each workload, calculate +`100 * (max / min - 1)` across runs for peak bytes, allocated bytes, and +allocation counts; aggregate averages hide unstable cases. diff --git a/benchmarks/memory/client/package.json b/benchmarks/memory/client/package.json index 5bc11e053b..af1770ba0d 100644 --- a/benchmarks/memory/client/package.json +++ b/benchmarks/memory/client/package.json @@ -231,7 +231,7 @@ } }, "test:types": { - "executor": "nx:noop", + "executor": "nx:run-commands", "dependsOn": [ { "projects": [ @@ -256,6 +256,17 @@ ], "target": "test:types:client" } + ], + "options": { + "command": "tsc -p ./tsconfig.json --noEmit", + "cwd": "benchmarks/memory/client" + }, + "inputs": [ + "default", + "^production", + "{workspaceRoot}/eslint.config.js", + "dependentTaskOutputs", + "{workspaceRoot}/benchmarks/memory/*.ts" ] } } diff --git a/benchmarks/memory/client/scenarios/interrupted-navigations/react/memory.bench.ts b/benchmarks/memory/client/scenarios/interrupted-navigations/react/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/interrupted-navigations/react/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/interrupted-navigations/react/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/interrupted-navigations/react/vite.config.ts b/benchmarks/memory/client/scenarios/interrupted-navigations/react/vite.config.ts index 048a9cc833..0f05cedf40 100644 --- a/benchmarks/memory/client/scenarios/interrupted-navigations/react/vite.config.ts +++ b/benchmarks/memory/client/scenarios/interrupted-navigations/react/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client interrupted-navigations (react)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/interrupted-navigations/solid/memory.bench.ts b/benchmarks/memory/client/scenarios/interrupted-navigations/solid/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/interrupted-navigations/solid/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/interrupted-navigations/solid/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/interrupted-navigations/solid/vite.config.ts b/benchmarks/memory/client/scenarios/interrupted-navigations/solid/vite.config.ts index e710e0fd12..26ff0b5c8c 100644 --- a/benchmarks/memory/client/scenarios/interrupted-navigations/solid/vite.config.ts +++ b/benchmarks/memory/client/scenarios/interrupted-navigations/solid/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client interrupted-navigations (solid)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/interrupted-navigations/vue/memory.bench.ts b/benchmarks/memory/client/scenarios/interrupted-navigations/vue/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/interrupted-navigations/vue/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/interrupted-navigations/vue/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/interrupted-navigations/vue/vite.config.ts b/benchmarks/memory/client/scenarios/interrupted-navigations/vue/vite.config.ts index b2912ce9d1..02574908d2 100644 --- a/benchmarks/memory/client/scenarios/interrupted-navigations/vue/vite.config.ts +++ b/benchmarks/memory/client/scenarios/interrupted-navigations/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import vue from '@vitejs/plugin-vue' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -28,9 +29,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client interrupted-navigations (vue)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/loader-data-retention/react/memory.bench.ts b/benchmarks/memory/client/scenarios/loader-data-retention/react/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/loader-data-retention/react/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/loader-data-retention/react/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/loader-data-retention/react/vite.config.ts b/benchmarks/memory/client/scenarios/loader-data-retention/react/vite.config.ts index 555e5f5612..8630417936 100644 --- a/benchmarks/memory/client/scenarios/loader-data-retention/react/vite.config.ts +++ b/benchmarks/memory/client/scenarios/loader-data-retention/react/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client loader-data-retention (react)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/loader-data-retention/solid/memory.bench.ts b/benchmarks/memory/client/scenarios/loader-data-retention/solid/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/loader-data-retention/solid/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/loader-data-retention/solid/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/loader-data-retention/solid/vite.config.ts b/benchmarks/memory/client/scenarios/loader-data-retention/solid/vite.config.ts index 58d6b77acc..5dfd5b53fe 100644 --- a/benchmarks/memory/client/scenarios/loader-data-retention/solid/vite.config.ts +++ b/benchmarks/memory/client/scenarios/loader-data-retention/solid/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client loader-data-retention (solid)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/loader-data-retention/vue/memory.bench.ts b/benchmarks/memory/client/scenarios/loader-data-retention/vue/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/loader-data-retention/vue/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/loader-data-retention/vue/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/loader-data-retention/vue/vite.config.ts b/benchmarks/memory/client/scenarios/loader-data-retention/vue/vite.config.ts index bbf2fd02d8..006636eaa0 100644 --- a/benchmarks/memory/client/scenarios/loader-data-retention/vue/vite.config.ts +++ b/benchmarks/memory/client/scenarios/loader-data-retention/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import vue from '@vitejs/plugin-vue' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -28,9 +29,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client loader-data-retention (vue)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/mount-unmount/react/memory.bench.ts b/benchmarks/memory/client/scenarios/mount-unmount/react/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/mount-unmount/react/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/mount-unmount/react/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/mount-unmount/react/vite.config.ts b/benchmarks/memory/client/scenarios/mount-unmount/react/vite.config.ts index bb0f3c2e0a..a08352a837 100644 --- a/benchmarks/memory/client/scenarios/mount-unmount/react/vite.config.ts +++ b/benchmarks/memory/client/scenarios/mount-unmount/react/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client mount-unmount (react)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/mount-unmount/solid/memory.bench.ts b/benchmarks/memory/client/scenarios/mount-unmount/solid/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/mount-unmount/solid/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/mount-unmount/solid/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/mount-unmount/solid/vite.config.ts b/benchmarks/memory/client/scenarios/mount-unmount/solid/vite.config.ts index 5f0ab26252..33699639de 100644 --- a/benchmarks/memory/client/scenarios/mount-unmount/solid/vite.config.ts +++ b/benchmarks/memory/client/scenarios/mount-unmount/solid/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client mount-unmount (solid)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/mount-unmount/vue/memory.bench.ts b/benchmarks/memory/client/scenarios/mount-unmount/vue/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/mount-unmount/vue/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/mount-unmount/vue/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/mount-unmount/vue/vite.config.ts b/benchmarks/memory/client/scenarios/mount-unmount/vue/vite.config.ts index 0f7573a408..a3f0eeaa8c 100644 --- a/benchmarks/memory/client/scenarios/mount-unmount/vue/vite.config.ts +++ b/benchmarks/memory/client/scenarios/mount-unmount/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import vue from '@vitejs/plugin-vue' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -28,9 +29,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client mount-unmount (vue)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/navigation-churn/react/memory.bench.ts b/benchmarks/memory/client/scenarios/navigation-churn/react/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/navigation-churn/react/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/navigation-churn/react/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/navigation-churn/react/vite.config.ts b/benchmarks/memory/client/scenarios/navigation-churn/react/vite.config.ts index b05b999297..eef60a658e 100644 --- a/benchmarks/memory/client/scenarios/navigation-churn/react/vite.config.ts +++ b/benchmarks/memory/client/scenarios/navigation-churn/react/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client navigation-churn (react)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/navigation-churn/solid/memory.bench.ts b/benchmarks/memory/client/scenarios/navigation-churn/solid/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/navigation-churn/solid/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/navigation-churn/solid/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/navigation-churn/solid/vite.config.ts b/benchmarks/memory/client/scenarios/navigation-churn/solid/vite.config.ts index 6b6735891a..7dfb186b2e 100644 --- a/benchmarks/memory/client/scenarios/navigation-churn/solid/vite.config.ts +++ b/benchmarks/memory/client/scenarios/navigation-churn/solid/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client navigation-churn (solid)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/navigation-churn/vue/memory.bench.ts b/benchmarks/memory/client/scenarios/navigation-churn/vue/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/navigation-churn/vue/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/navigation-churn/vue/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/navigation-churn/vue/vite.config.ts b/benchmarks/memory/client/scenarios/navigation-churn/vue/vite.config.ts index d26ce23d07..b3bdd39dd2 100644 --- a/benchmarks/memory/client/scenarios/navigation-churn/vue/vite.config.ts +++ b/benchmarks/memory/client/scenarios/navigation-churn/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import vue from '@vitejs/plugin-vue' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -28,9 +29,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client navigation-churn (vue)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/preload-churn/react/memory.bench.ts b/benchmarks/memory/client/scenarios/preload-churn/react/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/react/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/react/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/preload-churn/react/vite.config.ts b/benchmarks/memory/client/scenarios/preload-churn/react/vite.config.ts index f9c426656b..c12995f68e 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/react/vite.config.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/react/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client preload-churn (react)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/preload-churn/shared.ts b/benchmarks/memory/client/scenarios/preload-churn/shared.ts index 8cea4aca81..68fdfedd3f 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/shared.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/shared.ts @@ -35,18 +35,14 @@ type PreloadRouter = { subscribe: (event: 'onRendered', listener: () => void) => () => void } -// Fixed id for the eviction navigations interleaved into the bench loop; its -// payload is a constant-size steady-state resident, never part of the signal. +// Reuse one navigation target while evicting the distinct preloaded routes. const evictionItemId = 'nav-evict' const preloadChurnIterations = 200 // A navigation commit is what triggers the router's clearExpiredCache -- // preloaded matches (defaultPreloadGcTime: 0) are only evicted then, never -// during a preload-only loop. Interleaving a navigation every few preloads is -// what makes the flat floor assert "eviction releases preloaded payloads". +// during a preload-only loop. Interleave navigation to exercise eviction. const preloadsPerEvictionNavigation = 10 -// Module-level so ids stay unique across runner invocations on one mount; a -// per-invocation LCG would replay identical ids, and every preload after the -// first invocation would dedupe against cachedMatches instead of doing work. +// Keep generated URLs unique across warmup and measured invocations. const benchmarkRandom = createDeterministicRandom(0x706c6f61) let preloadCounter = 0 diff --git a/benchmarks/memory/client/scenarios/preload-churn/solid/memory.bench.ts b/benchmarks/memory/client/scenarios/preload-churn/solid/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/solid/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/solid/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/preload-churn/solid/vite.config.ts b/benchmarks/memory/client/scenarios/preload-churn/solid/vite.config.ts index c86a6fcbb3..efc75d2b93 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/solid/vite.config.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/solid/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client preload-churn (solid)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/preload-churn/vue/memory.bench.ts b/benchmarks/memory/client/scenarios/preload-churn/vue/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/vue/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/vue/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/preload-churn/vue/vite.config.ts b/benchmarks/memory/client/scenarios/preload-churn/vue/vite.config.ts index ea7ed7b695..3d56089ba7 100644 --- a/benchmarks/memory/client/scenarios/preload-churn/vue/vite.config.ts +++ b/benchmarks/memory/client/scenarios/preload-churn/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import vue from '@vitejs/plugin-vue' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -28,9 +29,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client preload-churn (vue)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/unique-location-churn/react/memory.bench.ts b/benchmarks/memory/client/scenarios/unique-location-churn/react/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/unique-location-churn/react/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/unique-location-churn/react/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/unique-location-churn/react/vite.config.ts b/benchmarks/memory/client/scenarios/unique-location-churn/react/vite.config.ts index f424c57cef..7ee67bfe30 100644 --- a/benchmarks/memory/client/scenarios/unique-location-churn/react/vite.config.ts +++ b/benchmarks/memory/client/scenarios/unique-location-churn/react/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client unique-location-churn (react)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/unique-location-churn/solid/memory.bench.ts b/benchmarks/memory/client/scenarios/unique-location-churn/solid/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/unique-location-churn/solid/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/unique-location-churn/solid/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/unique-location-churn/solid/vite.config.ts b/benchmarks/memory/client/scenarios/unique-location-churn/solid/vite.config.ts index d4acf0019a..4db90da2d2 100644 --- a/benchmarks/memory/client/scenarios/unique-location-churn/solid/vite.config.ts +++ b/benchmarks/memory/client/scenarios/unique-location-churn/solid/vite.config.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -26,9 +27,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client unique-location-churn (solid)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/scenarios/unique-location-churn/vue/memory.bench.ts b/benchmarks/memory/client/scenarios/unique-location-churn/vue/memory.bench.ts index e645ab38f1..85a1f11ee6 100644 --- a/benchmarks/memory/client/scenarios/unique-location-churn/vue/memory.bench.ts +++ b/benchmarks/memory/client/scenarios/unique-location-churn/vue/memory.bench.ts @@ -1,4 +1,4 @@ -import { afterAll, beforeAll, bench, describe } from 'vitest' +import { afterEach, beforeEach, bench, describe } from 'vitest' import { memoryBenchOptions } from '#memory-client/bench-utils' import { workload } from './setup' @@ -6,8 +6,8 @@ await workload.sanity() describe('memory', () => { if (workload.before && workload.after) { - beforeAll(workload.before) - afterAll(workload.after) + beforeEach(workload.before) + afterEach(workload.after) bench(workload.name, workload.run, { ...memoryBenchOptions, diff --git a/benchmarks/memory/client/scenarios/unique-location-churn/vue/vite.config.ts b/benchmarks/memory/client/scenarios/unique-location-churn/vue/vite.config.ts index 423126019b..d074d571b9 100644 --- a/benchmarks/memory/client/scenarios/unique-location-churn/vue/vite.config.ts +++ b/benchmarks/memory/client/scenarios/unique-location-churn/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import vue from '@vitejs/plugin-vue' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -28,9 +29,9 @@ export default defineConfig({ }, }, test: { + ...memoryConfig('client'), name: '@benchmarks/memory-client unique-location-churn (vue)', watch: false, environment: 'jsdom', - setupFiles: ['../../../vitest.setup.ts'], }, }) diff --git a/benchmarks/memory/client/tsconfig.json b/benchmarks/memory/client/tsconfig.json index 9280d5c42f..eefc5c581e 100644 --- a/benchmarks/memory/client/tsconfig.json +++ b/benchmarks/memory/client/tsconfig.json @@ -3,5 +3,10 @@ "compilerOptions": { "types": ["node", "vite/client", "vitest/globals"] }, - "include": ["bench-utils.ts", "lifecycle.ts", "vitest.setup.ts"] + "include": [ + "bench-utils.ts", + "lifecycle.ts", + "vitest.setup.ts", + "scenarios/*/shared.ts" + ] } diff --git a/benchmarks/memory/client/vitest.setup.ts b/benchmarks/memory/client/vitest.setup.ts index ce3127275a..5f78f1a7d1 100644 --- a/benchmarks/memory/client/vitest.setup.ts +++ b/benchmarks/memory/client/vitest.setup.ts @@ -1,3 +1,6 @@ +import { beforeEach } from 'vitest' +import { endMemoryTurn, isMemoryInstrumented } from '../turn' + // @ts-expect-error global.IS_REACT_ACT_ENVIRONMENT = true @@ -7,3 +10,7 @@ window.scrollTo = scrollTo globalThis.scrollTo = scrollTo export {} + +if (isMemoryInstrumented()) { + beforeEach(endMemoryTurn) +} diff --git a/benchmarks/memory/runtime.ts b/benchmarks/memory/runtime.ts new file mode 100644 index 0000000000..af03b73400 --- /dev/null +++ b/benchmarks/memory/runtime.ts @@ -0,0 +1,24 @@ +import { fileURLToPath } from 'node:url' +import { isMemoryInstrumented } from './turn' + +/** Worker settings for the CodSpeed memory instrument. */ +export function memoryExecArgv() { + if (!isMemoryInstrumented()) { + return [] + } + + // Keep machine-code generation out of native allocation measurements and GC + // tied to allocations instead of event-loop tasks and marking time budgets. + return ['--no-incremental-marking'] +} + +export function memoryConfig(side: 'client' | 'server') { + const execArgv = memoryExecArgv() + return { + execArgv, + setupFiles: + side === 'client' || isMemoryInstrumented() + ? [fileURLToPath(new URL(`./${side}/vitest.setup.ts`, import.meta.url))] + : [], + } +} diff --git a/benchmarks/memory/server/bench-utils.test.ts b/benchmarks/memory/server/bench-utils.test.ts new file mode 100644 index 0000000000..6ea717ae24 --- /dev/null +++ b/benchmarks/memory/server/bench-utils.test.ts @@ -0,0 +1,62 @@ +import { afterEach, expect, it, vi } from 'vitest' +import { runSequentialRequestLoop } from './bench-utils' + +afterEach(() => { + vi.useRealTimers() + vi.unstubAllGlobals() +}) + +it('drains each response before starting the next request', async () => { + vi.useFakeTimers() + const events: Array = [] + let index = 0 + const run = runSequentialRequestLoop( + { + fetch() { + const current = index++ + events.push(`fetch ${current}`) + return new Response( + new ReadableStream({ + start(controller) { + setTimeout(() => { + events.push(`body ${current}`) + controller.enqueue(new Uint8Array([1])) + controller.close() + }, 0) + }, + }), + ) + }, + }, + { + seed: 1, + iterations: 2, + buildRequest: () => new Request('http://localhost/'), + }, + ) + await vi.runAllTimersAsync() + await run + expect(events).toEqual(['fetch 0', 'body 0', 'fetch 1', 'body 1']) +}) + +it('releases a failed response reader and does not start another request', async () => { + const failure = new Error('stream failed') + const body = new ReadableStream({ + start(controller) { + controller.error(failure) + }, + }) + const fetch = vi.fn(() => new Response(body)) + await expect( + runSequentialRequestLoop( + { fetch }, + { + seed: 1, + iterations: 2, + buildRequest: () => new Request('http://localhost/'), + }, + ), + ).rejects.toBe(failure) + expect(body.locked).toBe(false) + expect(fetch).toHaveBeenCalledTimes(1) +}) diff --git a/benchmarks/memory/server/bench-utils.ts b/benchmarks/memory/server/bench-utils.ts index b8a09fdd8e..8893d5be4b 100644 --- a/benchmarks/memory/server/bench-utils.ts +++ b/benchmarks/memory/server/bench-utils.ts @@ -17,24 +17,6 @@ export type RunSequentialRequestLoopOptions = iterations?: number buildRequest: (random: () => number, index: number) => Request validateResponse?: (response: Response, request: Request) => void - // Whether V8 collects iteration i's garbage before iteration i+1 - // allocates its payload is not reproducible run to run — the measured - // peak flips by a whole payload depending on GC timing, which shifts - // with runner hardware. Forcing a collection between iterations pins - // the GC points so peak deterministically measures the largest - // single-iteration footprint plus any reachable accumulation. - // Accumulation signals stay visible: leaked or cached objects are - // still referenced, so a forced collection cannot reclaim them — it - // only removes floating garbage, whose collection timing is the - // dominant cross-run noise source. - pinGcBetweenIterations?: boolean - // For peak-shape scenarios ONLY (flat inter-iteration heap floor): - // verify each pinned collection actually returned the heap to the - // floor, extending the barrier when a runner's late teardown holds the - // payload past the fixed window. Churn/abort scenarios must not set - // this: their floor legitimately drifts, so the verification caps out - // chronically and the extra collections it fires destabilize the peak. - verifyGcFloor?: boolean } export const memoryBenchOptions = { @@ -66,7 +48,7 @@ export async function drainResponse(response: Response) { } try { - while (true) { + for (;;) { const result = await reader.read() if (result.done) { @@ -82,13 +64,7 @@ export async function runSequentialRequestLoop( handler: StartRequestHandler, options: RunSequentialRequestLoopOptions, ) { - const { - iterations = 10, - buildRequest, - validateResponse, - pinGcBetweenIterations = false, - verifyGcFloor = false, - } = options + const { iterations = 10, buildRequest, validateResponse } = options const random = options.seed !== undefined ? createDeterministicRandom(options.seed) @@ -103,8 +79,6 @@ export async function runSequentialRequestLoop( } }) - const gcPinState = verifyGcFloor ? createGcPinState() : undefined - for (let index = 0; index < iterations; index++) { const request = buildRequest(random, index) const response = await handler.fetch(request) @@ -112,89 +86,12 @@ export async function runSequentialRequestLoop( validate(response, request) await drainResponse(response) - - if (pinGcBetweenIterations) { - await settleAndPinGc(gcPinState) - } } } -// Renderer/stream teardown spans several event-loop turns (React and Vue -// need more than the two turns the original barrier allowed for), and a -// too-short settle window leaks a whole payload of garbage past the -// collection point, flipping the measured peak bimodally between runs. -// The settle length is a fixed count on purpose: an adaptive exit (collect -// until the post-GC heap size stops moving) makes the collection points -// land at data-dependent turns, which re-introduces exactly the run-to-run -// variance this barrier exists to remove — heap-size readings never fully -// stabilize under the instrumented CI environment, so the exit point (and -// with it every subsequent GC point) shifted between identical runs. -const settleTurnsBeforeGc = 16 -const maxSettleTurns = 64 -// Below every scenario's payload size, above normal inter-iteration jitter. -const settledHeapSlackBytes = 256 * 1024 - -// Tracks the smallest post-collection heap size seen in a workload run: -// the inter-iteration floor the heap must return to before the next -// iteration may start. -export interface GcPinState { - floorHeapUsed: number -} - -export function createGcPinState(): GcPinState { - return { floorHeapUsed: Infinity } -} - -// Settle trailing renderer/stream teardown with a fixed number of 0ms timer -// hops (one full event-loop turn each, microtasks flushing between hops), -// then pin a collection point. The second collection one turn later picks -// up whatever the first collection's finalizers released. gc() is present -// under CodSpeed (--expose-gc); in plain smoke runs there is nothing to -// pin, so only the settle hops run. -// -// The collection is then verified against the inter-iteration heap floor: -// on some runners the response teardown holds the payload past the fixed -// settle window (released only by a later internal timer), and that one -// still-reachable payload survives the collection and bleeds into the next -// iteration's measured peak. When the post-collection heap has not returned -// to the floor, keep hopping and collecting — bounded — until the payload -// is actually released, so the barrier self-heals in exactly the case where -// the fixed-count barrier fails. A workload that genuinely accumulates -// reachable memory raises the floor as it goes (the floor is the minimum -// seen), so accumulation still measures; it only pays the turn cap. -export async function settleAndPinGc(state?: GcPinState) { - for (let turn = 0; turn < settleTurnsBeforeGc; turn++) { +// Cancellation can leave deferred loaders and renderer teardown queued. +export async function settle() { + for (let turn = 0; turn < 16; turn++) { await new Promise((resolve) => setTimeout(resolve, 0)) } - - const gc = (globalThis as { gc?: () => void }).gc - - if (!gc) { - return - } - - gc() - await new Promise((resolve) => setTimeout(resolve, 0)) - gc() - - if (!state) { - return - } - - let heapUsed = process.memoryUsage().heapUsed - - for ( - let turn = settleTurnsBeforeGc; - turn < maxSettleTurns && - heapUsed > state.floorHeapUsed + settledHeapSlackBytes; - turn++ - ) { - await new Promise((resolve) => setTimeout(resolve, 0)) - gc() - heapUsed = process.memoryUsage().heapUsed - } - - if (heapUsed < state.floorHeapUsed) { - state.floorHeapUsed = heapUsed - } } diff --git a/benchmarks/memory/server/client-lifecycle.test.ts b/benchmarks/memory/server/client-lifecycle.test.ts new file mode 100644 index 0000000000..451708ef26 --- /dev/null +++ b/benchmarks/memory/server/client-lifecycle.test.ts @@ -0,0 +1,69 @@ +import { globSync } from 'node:fs' +import { dirname, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' +import { expect, it, vi } from 'vitest' + +const clientRoot = fileURLToPath(new URL('../client/', import.meta.url)) +const entrypoints = globSync('scenarios/*/*/memory.bench.ts', { + cwd: clientRoot, +}) + +it.each(entrypoints)( + '%s resets the app between CodSpeed invocations', + async (file) => { + const events: Array = [] + const beforeHooks: Array<() => unknown> = [] + const afterHooks: Array<() => unknown> = [] + const workload = { + name: 'lifecycle control', + sanity: vi.fn(), + before: () => events.push('mount'), + run: () => events.push('navigate'), + after: () => events.push('unmount'), + } + let run: (() => unknown) | undefined + let options: { setup?: unknown; teardown?: unknown } | undefined + const entrypoint = resolve(clientRoot, file) + const setup = resolve(dirname(entrypoint), 'setup.ts') + + vi.doMock('vitest', () => ({ + describe: (_name: string, register: () => void) => register(), + beforeEach: (hook: () => unknown) => beforeHooks.push(hook), + afterEach: (hook: () => unknown) => afterHooks.push(hook), + bench: ( + _name: string, + fn: () => unknown, + benchOptions: typeof options, + ) => { + run = fn + options = benchOptions + }, + })) + vi.doMock(setup, () => ({ workload })) + + try { + await import(/* @vite-ignore */ entrypoint) + expect(workload.sanity).toHaveBeenCalledOnce() + expect(run).toBe(workload.run) + // Tinybench uses its own hooks; CodSpeed uses the suite hooks below. + expect(options?.setup).toBe(workload.before) + expect(options?.teardown).toBe(workload.after) + for (let invocation = 0; invocation < 3; invocation++) { + for (const before of beforeHooks) { + await before() + } + await run!() + for (const after of afterHooks) { + await after() + } + } + expect(events).toEqual( + Array(3).fill(['mount', 'navigate', 'unmount']).flat(), + ) + } finally { + vi.doUnmock(setup) + vi.doUnmock('vitest') + vi.resetModules() + } + }, +) diff --git a/benchmarks/memory/server/deferred-records.test.ts b/benchmarks/memory/server/deferred-records.test.ts new file mode 100644 index 0000000000..1ad9130545 --- /dev/null +++ b/benchmarks/memory/server/deferred-records.test.ts @@ -0,0 +1,45 @@ +import { afterEach, expect, it, vi } from 'vitest' +import { makeDeferredRecords } from './scenarios/aborted-requests/deferred-records' + +afterEach(() => { + vi.useRealTimers() +}) + +it('keeps abort probes pending without timers until the request is cancelled', async () => { + vi.useFakeTimers() + const controller = new AbortController() + const resolved = vi.fn() + const records = makeDeferredRecords('abort-1', 'alpha', controller.signal) + void records.then(resolved) + await vi.runAllTimersAsync() + expect(resolved).not.toHaveBeenCalled() + expect(vi.getTimerCount()).toBe(0) + controller.abort() + await expect(records).resolves.toEqual([]) +}) + +it('handles an already cancelled request without scheduling work', async () => { + vi.useFakeTimers() + await expect( + makeDeferredRecords('abort-1', 'beta', AbortSignal.abort()), + ).resolves.toEqual([]) + expect(vi.getTimerCount()).toBe(0) +}) + +it('cancels timers for an interrupted full response', async () => { + vi.useFakeTimers() + const controller = new AbortController() + const records = makeDeferredRecords('sanity-full', 'alpha', controller.signal) + controller.abort() + await expect(records).resolves.toEqual([]) + expect(vi.getTimerCount()).toBe(0) +}) + +it('still produces full deferred payloads for an uninterrupted response', async () => { + vi.useFakeTimers() + const controller = new AbortController() + const records = makeDeferredRecords('sanity-full', 'beta', controller.signal) + await vi.runAllTimersAsync() + await expect(records).resolves.toHaveLength(20) + expect(vi.getTimerCount()).toBe(0) +}) diff --git a/benchmarks/memory/server/package.json b/benchmarks/memory/server/package.json index 423dde0eb3..d3812ffc3c 100644 --- a/benchmarks/memory/server/package.json +++ b/benchmarks/memory/server/package.json @@ -237,7 +237,7 @@ } }, "test:types": { - "executor": "nx:noop", + "executor": "nx:run-commands", "dependsOn": [ { "projects": [ @@ -265,6 +265,32 @@ ], "target": "test:types:ssr" } + ], + "options": { + "command": "tsc -p ./tsconfig.json --noEmit", + "cwd": "benchmarks/memory/server" + }, + "inputs": [ + "default", + "^production", + "{workspaceRoot}/eslint.config.js", + "dependentTaskOutputs", + "{workspaceRoot}/benchmarks/memory/*.ts" + ] + }, + "test:unit": { + "executor": "nx:run-commands", + "options": { + "command": "vitest run --config ./vitest.unit.config.ts", + "cwd": "benchmarks/memory/server" + }, + "inputs": [ + "default", + "^production", + "{workspaceRoot}/eslint.config.js", + "dependentTaskOutputs", + "{workspaceRoot}/benchmarks/memory/*.ts", + "{workspaceRoot}/benchmarks/memory/client/scenarios/*/*/memory.bench.ts" ] } } diff --git a/benchmarks/memory/server/runtime.test.ts b/benchmarks/memory/server/runtime.test.ts new file mode 100644 index 0000000000..4b348a9ce6 --- /dev/null +++ b/benchmarks/memory/server/runtime.test.ts @@ -0,0 +1,99 @@ +import { execFileSync } from 'node:child_process' +import { readFileSync } from 'node:fs' +import { stripTypeScriptTypes } from 'node:module' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { resolveConfig } from 'vite' +import codspeedPlugin from '@codspeed/vitest-plugin' +import { memoryConfig, memoryExecArgv } from '../runtime' + +afterEach(() => { + vi.unstubAllEnvs() +}) + +describe('memory worker configuration', () => { + it('preserves CodSpeed instrumentation flags in a runnable worker', async () => { + vi.stubEnv('CODSPEED_ENV', 'test') + vi.stubEnv('CODSPEED_RUNNER_MODE', 'memory') + + const config = await resolveConfig( + { + configFile: false, + plugins: [codspeedPlugin()], + test: memoryConfig('server'), + }, + 'serve', + 'benchmark', + ) + const flags = config.test!.execArgv! + expect(flags).toContain('--expose-gc') + expect(flags).toContain('--predictable') + expect(flags).toContain('--hash-seed=1') + expect(config.test!.setupFiles).toEqual([ + expect.stringMatching(/\/memory\/server\/vitest\.setup\.ts$/), + ]) + + const output = execFileSync( + process.execPath, + [ + ...flags, + '--trace-opt', + '--eval', + ` + if (typeof global.gc !== 'function') { + throw new Error('Missing GC instrumentation') + } + function hot(value) { return value + 1 } + let total = 0 + for (let index = 0; index < 100000; index++) { + total += hot(index) + } + console.log(total) + `, + ], + { encoding: 'utf8' }, + ) + expect(output).toContain('5000050000') + }) + + it.each([undefined, 'simulation', 'instrumentation', 'walltime'])( + 'does not change %s workers', + (mode) => { + vi.stubEnv('CODSPEED_ENV', 'test') + vi.stubEnv('CODSPEED_RUNNER_MODE', mode) + expect(memoryExecArgv()).toEqual([]) + expect(memoryConfig('server')).toEqual({ execArgv: [], setupFiles: [] }) + }, + ) + + it('allows prior-job WeakRef targets to be collected before measurement', () => { + vi.stubEnv('CODSPEED_ENV', 'test') + vi.stubEnv('CODSPEED_RUNNER_MODE', 'memory') + // Node's TS loader needs Wasm, which --jitless disables. Strip types in + // the parent, as Vite does for actual benchmark workers. + const source = stripTypeScriptTypes( + readFileSync(new URL('../turn.ts', import.meta.url), 'utf8'), + ) + const runtimeUrl = `data:text/javascript,${encodeURIComponent(source)}` + const output = execFileSync( + process.execPath, + [ + ...memoryExecArgv(), + '--expose-gc', + '--input-type=module', + '--eval', + ` + const { endMemoryTurn } = await import(${JSON.stringify(runtimeUrl)}) + const ref = (() => new WeakRef({ value: 1 }))() + await Promise.resolve() + global.gc() + console.log(String(ref.deref() !== undefined)) + await endMemoryTurn() + global.gc() + console.log(String(ref.deref() === undefined)) + `, + ], + { encoding: 'utf8' }, + ) + expect(output.trim()).toBe('true\ntrue') + }) +}) diff --git a/benchmarks/memory/server/scenarios/aborted-requests/deferred-records.ts b/benchmarks/memory/server/scenarios/aborted-requests/deferred-records.ts index 4886e3ee80..7a1cd6d254 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/deferred-records.ts +++ b/benchmarks/memory/server/scenarios/aborted-requests/deferred-records.ts @@ -16,3 +16,42 @@ export function makeAbortedRequestRecords( label: `deferred-${group}-${id}-${index}`, })) } + +// Abort probes keep their deferred payload pending until the request is +// cancelled. Timer chains can outlive cancellation and spill into the next +// measured request; route-loader signals stop observing the request after load. +export function makeDeferredRecords( + id: string, + group: RecordGroup, + signal: AbortSignal, +): Promise> { + return new Promise((resolve) => { + if (signal.aborted) { + resolve([]) + return + } + + let timeoutId: ReturnType | undefined + const onAbort = () => { + clearTimeout(timeoutId) + resolve([]) + } + signal.addEventListener('abort', onAbort, { once: true }) + + if (id === 'sanity-mid-stream' || id.startsWith('abort-')) { + return + } + + let remaining = group === 'alpha' ? 4 : 6 + const step = () => { + remaining -= 1 + if (remaining === 0) { + signal.removeEventListener('abort', onAbort) + resolve(makeAbortedRequestRecords(id, group)) + return + } + timeoutId = setTimeout(step, 0) + } + timeoutId = setTimeout(step, 0) + }) +} diff --git a/benchmarks/memory/server/scenarios/aborted-requests/react/vite.config.ts b/benchmarks/memory/server/scenarios/aborted-requests/react/vite.config.ts index 2ce925e406..eff0d0e043 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/aborted-requests/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server aborted-requests (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/aborted-requests/shared.ts b/benchmarks/memory/server/scenarios/aborted-requests/shared.ts index c98f462538..b5a61594ba 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/shared.ts +++ b/benchmarks/memory/server/scenarios/aborted-requests/shared.ts @@ -1,4 +1,4 @@ -import { settleAndPinGc } from '#memory-server/bench-utils' +import { settle } from '#memory-server/bench-utils' import type { StartRequestHandler } from '#memory-server/bench-utils' export type { StartRequestHandler } @@ -6,13 +6,6 @@ export type { StartRequestHandler } type Framework = 'react' | 'solid' | 'vue' type AbortedRequestReadMode = 'first-chunk' | 'shell-before-deferred' -type AbortedRequestCancelMode = 'plain' | 'swallow-abort-error' - -type AbortedRequestMode = { - readMode: AbortedRequestReadMode - cancelMode: AbortedRequestCancelMode -} - const abortedRequestIterations = 40 let abortedRequestCounter = 0 const eagerMarker = 'data-bench="aborted-requests-eager"' @@ -24,19 +17,10 @@ const betaFirstRecord = (id: string) => `deferred-beta-${id}-0` const betaLastRecord = (id: string) => `deferred-beta-${id}-19` const textDecoder = new TextDecoder() -const abortedRequestModes: Record = { - react: { - readMode: 'first-chunk', - cancelMode: 'swallow-abort-error', - }, - solid: { - readMode: 'first-chunk', - cancelMode: 'swallow-abort-error', - }, - vue: { - readMode: 'shell-before-deferred', - cancelMode: 'swallow-abort-error', - }, +const abortedRequestModes: Record = { + react: 'first-chunk', + solid: 'first-chunk', + vue: 'shell-before-deferred', } const documentRequestInit = { @@ -96,7 +80,7 @@ async function readShellBeforeDeferred( const reader = response.body.getReader() let text = '' - while (true) { + for (;;) { const result = await reader.read() const value = result.value @@ -137,10 +121,10 @@ async function readShellBeforeDeferred( async function readSanityStream( response: Response, request: Request, - mode: AbortedRequestMode, + mode: AbortedRequestReadMode, id: string, ) { - if (mode.readMode === 'shell-before-deferred') { + if (mode === 'shell-before-deferred') { return readShellBeforeDeferred(response, request, id) } @@ -165,36 +149,19 @@ function readLoopStream( return readFirstChunk(response, request) } -async function cancelReader( - reader: ReadableStreamDefaultReader, - mode: AbortedRequestCancelMode, -) { - if (mode === 'swallow-abort-error') { - try { - await reader.cancel() - } catch (error) { - if (!(error instanceof DOMException && error.name === 'AbortError')) { - throw error - } +async function cancelReader(reader: ReadableStreamDefaultReader) { + try { + await reader.cancel() + } catch (error) { + if (!(error instanceof DOMException && error.name === 'AbortError')) { + throw error } - - return } - - await reader.cancel() -} - -// Post-abort settlement barrier. Settles renderer teardown across a fixed -// number of event-loop turns, then (under CodSpeed) pins a collection point -// after every aborted request, so the measured peak cannot drift with how -// much floating garbage the previous iterations happened to leave behind. -async function drainCancellation() { - await settleAndPinGc() } async function assertAbortedRequestsSanity( handler: StartRequestHandler, - mode: AbortedRequestMode, + mode: AbortedRequestReadMode, ) { const fullId = 'sanity-full' const fullRequest = buildStreamRequest(fullId) @@ -227,13 +194,13 @@ async function assertAbortedRequestsSanity( // reader.cancel() is the response-stream cancellation path if the handler // does not observe Request.signal for this in-process request. controller.abort() - await cancelReader(reader, mode.cancelMode) - await drainCancellation() + await cancelReader(reader) + await settle() } async function runAbortedRequestLoop( handler: StartRequestHandler, - mode: AbortedRequestMode, + mode: AbortedRequestReadMode, ) { for (let index = 0; index < abortedRequestIterations; index++) { const controller = new AbortController() @@ -242,15 +209,10 @@ async function runAbortedRequestLoop( const response = await handler.fetch(request) validateDocumentResponse(response, request) - const { reader } = await readLoopStream( - mode.readMode, - response, - request, - id, - ) + const { reader } = await readLoopStream(mode, response, request, id) controller.abort() - await cancelReader(reader, mode.cancelMode) - await drainCancellation() + await cancelReader(reader) + await settle() } } diff --git a/benchmarks/memory/server/scenarios/aborted-requests/solid/src/routes/stream.$id.tsx b/benchmarks/memory/server/scenarios/aborted-requests/solid/src/routes/stream.$id.tsx index b47f25a405..bf6af17f31 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/solid/src/routes/stream.$id.tsx +++ b/benchmarks/memory/server/scenarios/aborted-requests/solid/src/routes/stream.$id.tsx @@ -1,90 +1,17 @@ +import { createServerOnlyFn } from '@tanstack/solid-start' +import { getRequest } from '@tanstack/solid-start/server' import { createFileRoute } from '@tanstack/solid-router' import { Show, Suspense, createResource } from 'solid-js' -import { - makeAbortedRequestRecords, - type DeferredRecord, - type RecordGroup, -} from '../../../deferred-records' +import { makeDeferredRecords } from '../../../deferred-records' +import type { DeferredRecord } from '../../../deferred-records' -const alphaResolveTicks = 4 -const betaResolveTicks = 6 -const abortProbeAlphaResolveTicks = 40 -const abortProbeBetaResolveTicks = 60 - -function isAbortProbeId(id: string) { - return id === 'sanity-mid-stream' || id.startsWith('abort-') -} - -function getResolveTicks(id: string, group: RecordGroup) { - if (isAbortProbeId(id)) { - return group === 'alpha' - ? abortProbeAlphaResolveTicks - : abortProbeBetaResolveTicks - } - - return group === 'alpha' ? alphaResolveTicks : betaResolveTicks -} - -// One tick = one 0ms timers-phase hop. Counting event-loop turns instead of -// milliseconds keeps the resolve/abort interleaving a pure function of the -// event-loop schedule: a wall-clock delay races the abort differently -// depending on runner load and instrumentation overhead, which made this -// benchmark's single measured run swing between runs. -function resolveAfterTicks( - ticks: number, - signal: AbortSignal, - value: () => T, - abortedValue: () => T, -) { - return new Promise((resolve) => { - if (signal.aborted) { - resolve(abortedValue()) - return - } - - let remaining = ticks - let timeoutId: ReturnType - - const onAbort = () => { - clearTimeout(timeoutId) - resolve(abortedValue()) - } - - const step = () => { - remaining -= 1 - - if (remaining <= 0) { - signal.removeEventListener('abort', onAbort) - resolve(value()) - return - } - - timeoutId = setTimeout(step, 0) - } - - signal.addEventListener('abort', onAbort, { once: true }) - timeoutId = setTimeout(step, 0) - }) -} - -function makeDeferredRecords( - id: string, - group: RecordGroup, - signal: AbortSignal, -) { - return resolveAfterTicks( - getResolveTicks(id, group), - signal, - () => makeAbortedRequestRecords(id, group), - () => [], - ) -} +const getRequestSignal = createServerOnlyFn(() => getRequest().signal) export const Route = createFileRoute('/stream/$id')({ - loader: ({ params, abortController }) => ({ + loader: ({ params }) => ({ eager: `eager-${params.id}`, - alpha: makeDeferredRecords(params.id, 'alpha', abortController.signal), - beta: makeDeferredRecords(params.id, 'beta', abortController.signal), + alpha: makeDeferredRecords(params.id, 'alpha', getRequestSignal()), + beta: makeDeferredRecords(params.id, 'beta', getRequestSignal()), }), component: StreamComponent, }) diff --git a/benchmarks/memory/server/scenarios/aborted-requests/solid/vite.config.ts b/benchmarks/memory/server/scenarios/aborted-requests/solid/vite.config.ts index 5ee4e152af..d7fd3459ee 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/aborted-requests/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server aborted-requests (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/aborted-requests/vue/src/routes/stream.$id.tsx b/benchmarks/memory/server/scenarios/aborted-requests/vue/src/routes/stream.$id.tsx index b723c7f69e..448355b122 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/vue/src/routes/stream.$id.tsx +++ b/benchmarks/memory/server/scenarios/aborted-requests/vue/src/routes/stream.$id.tsx @@ -1,90 +1,17 @@ +import { createServerOnlyFn } from '@tanstack/vue-start' +import { getRequest } from '@tanstack/vue-start/server' import { Await, createFileRoute } from '@tanstack/vue-router' import { Suspense } from 'vue' -import { - makeAbortedRequestRecords, - type DeferredRecord, - type RecordGroup, -} from '../../../deferred-records' +import { makeDeferredRecords } from '../../../deferred-records' +import type { DeferredRecord } from '../../../deferred-records' -const alphaResolveTicks = 4 -const betaResolveTicks = 6 -const abortProbeAlphaResolveTicks = 40 -const abortProbeBetaResolveTicks = 60 - -function isAbortProbeId(id: string) { - return id === 'sanity-mid-stream' || id.startsWith('abort-') -} - -function getResolveTicks(id: string, group: RecordGroup) { - if (isAbortProbeId(id)) { - return group === 'alpha' - ? abortProbeAlphaResolveTicks - : abortProbeBetaResolveTicks - } - - return group === 'alpha' ? alphaResolveTicks : betaResolveTicks -} - -// One tick = one 0ms timers-phase hop. Counting event-loop turns instead of -// milliseconds keeps the resolve/abort interleaving a pure function of the -// event-loop schedule: a wall-clock delay races the abort differently -// depending on runner load and instrumentation overhead, which made this -// benchmark's single measured run swing between runs. -function resolveAfterTicks( - ticks: number, - signal: AbortSignal, - value: () => T, - abortedValue: () => T, -) { - return new Promise((resolve) => { - if (signal.aborted) { - resolve(abortedValue()) - return - } - - let remaining = ticks - let timeoutId: ReturnType - - const onAbort = () => { - clearTimeout(timeoutId) - resolve(abortedValue()) - } - - const step = () => { - remaining -= 1 - - if (remaining <= 0) { - signal.removeEventListener('abort', onAbort) - resolve(value()) - return - } - - timeoutId = setTimeout(step, 0) - } - - signal.addEventListener('abort', onAbort, { once: true }) - timeoutId = setTimeout(step, 0) - }) -} - -function makeDeferredRecords( - id: string, - group: RecordGroup, - signal: AbortSignal, -) { - return resolveAfterTicks( - getResolveTicks(id, group), - signal, - () => makeAbortedRequestRecords(id, group), - () => [], - ) -} +const getRequestSignal = createServerOnlyFn(() => getRequest().signal) export const Route = createFileRoute('/stream/$id')({ - loader: ({ params, abortController }) => ({ + loader: ({ params }) => ({ eager: `eager-${params.id}`, - alpha: makeDeferredRecords(params.id, 'alpha', abortController.signal), - beta: makeDeferredRecords(params.id, 'beta', abortController.signal), + alpha: makeDeferredRecords(params.id, 'alpha', getRequestSignal()), + beta: makeDeferredRecords(params.id, 'beta', getRequestSignal()), }), component: StreamComponent, }) diff --git a/benchmarks/memory/server/scenarios/aborted-requests/vue/vite.config.ts b/benchmarks/memory/server/scenarios/aborted-requests/vue/vite.config.ts index 5ccc9f5d1d..e4e1eb66d4 100644 --- a/benchmarks/memory/server/scenarios/aborted-requests/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/aborted-requests/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server aborted-requests (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/error-paths/react/memory.bench.ts b/benchmarks/memory/server/scenarios/error-paths/react/memory.bench.ts index 9a5c211fee..2780830cb2 100644 --- a/benchmarks/memory/server/scenarios/error-paths/react/memory.bench.ts +++ b/benchmarks/memory/server/scenarios/error-paths/react/memory.bench.ts @@ -4,8 +4,8 @@ import { workloadGroup } from './setup' await workloadGroup.sanity() +const workload = workloadGroup.workloads[0]! + describe('memory', () => { - for (const workload of workloadGroup.workloads) { - bench(workload.name, workload.run, memoryBenchOptions) - } + bench(workload.name, workload.run, memoryBenchOptions) }) diff --git a/benchmarks/memory/server/scenarios/error-paths/react/memory.error.bench.ts b/benchmarks/memory/server/scenarios/error-paths/react/memory.error.bench.ts new file mode 100644 index 0000000000..d41877e6ce --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/react/memory.error.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[2]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/react/memory.not-found.bench.ts b/benchmarks/memory/server/scenarios/error-paths/react/memory.not-found.bench.ts new file mode 100644 index 0000000000..42d67b1164 --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/react/memory.not-found.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[1]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/react/memory.unmatched.bench.ts b/benchmarks/memory/server/scenarios/error-paths/react/memory.unmatched.bench.ts new file mode 100644 index 0000000000..6179e469dd --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/react/memory.unmatched.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[3]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/react/tsconfig.json b/benchmarks/memory/server/scenarios/error-paths/react/tsconfig.json index 11ddcce4ea..2dba47fc99 100644 --- a/benchmarks/memory/server/scenarios/error-paths/react/tsconfig.json +++ b/benchmarks/memory/server/scenarios/error-paths/react/tsconfig.json @@ -7,7 +7,7 @@ "types": ["node", "vite/client", "vitest/globals"] }, "include": [ - "memory.bench.ts", + "memory*.bench.ts", "memory.flame.ts", "setup.ts", "vite.config.ts", diff --git a/benchmarks/memory/server/scenarios/error-paths/react/vite.config.ts b/benchmarks/memory/server/scenarios/error-paths/react/vite.config.ts index bc67b1cfc7..c00de7727a 100644 --- a/benchmarks/memory/server/scenarios/error-paths/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/error-paths/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,8 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + isolate: true, + ...memoryConfig('server'), name: '@benchmarks/memory-server error-paths (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/error-paths/shared.ts b/benchmarks/memory/server/scenarios/error-paths/shared.ts index 57e229b7e3..966505f3a0 100644 --- a/benchmarks/memory/server/scenarios/error-paths/shared.ts +++ b/benchmarks/memory/server/scenarios/error-paths/shared.ts @@ -9,8 +9,6 @@ export type { StartRequestHandler } type Framework = 'react' | 'solid' | 'vue' -// Sized to sit just above the 2s measured-run floor on CI (per-iteration -// cost with the pinned collection is ~0.13-0.19s across frameworks). const errorPathsIterations = 18 const redirectSeed = 0xdecafbad const notFoundSeed = 0xdecafb0d @@ -144,7 +142,6 @@ export function createWorkloadGroup( iterations: errorPathsIterations, buildRequest: buildRedirectRequest, validateResponse: validateRedirectResponse, - pinGcBetweenIterations: true, }) const runNotFound = () => @@ -153,7 +150,6 @@ export function createWorkloadGroup( iterations: errorPathsIterations, buildRequest: buildNotFoundRequest, validateResponse: validateNotFoundResponse, - pinGcBetweenIterations: true, }) const runError = () => @@ -162,7 +158,6 @@ export function createWorkloadGroup( iterations: errorPathsIterations, buildRequest: buildErrorRequest, validateResponse: validateErrorResponse, - pinGcBetweenIterations: true, }) const runUnmatched = () => @@ -171,7 +166,6 @@ export function createWorkloadGroup( iterations: errorPathsIterations, buildRequest: buildUnmatchedRequest, validateResponse: validateNotFoundResponse, - pinGcBetweenIterations: true, }) return { diff --git a/benchmarks/memory/server/scenarios/error-paths/solid/memory.bench.ts b/benchmarks/memory/server/scenarios/error-paths/solid/memory.bench.ts index 9a5c211fee..2780830cb2 100644 --- a/benchmarks/memory/server/scenarios/error-paths/solid/memory.bench.ts +++ b/benchmarks/memory/server/scenarios/error-paths/solid/memory.bench.ts @@ -4,8 +4,8 @@ import { workloadGroup } from './setup' await workloadGroup.sanity() +const workload = workloadGroup.workloads[0]! + describe('memory', () => { - for (const workload of workloadGroup.workloads) { - bench(workload.name, workload.run, memoryBenchOptions) - } + bench(workload.name, workload.run, memoryBenchOptions) }) diff --git a/benchmarks/memory/server/scenarios/error-paths/solid/memory.error.bench.ts b/benchmarks/memory/server/scenarios/error-paths/solid/memory.error.bench.ts new file mode 100644 index 0000000000..d41877e6ce --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/solid/memory.error.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[2]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/solid/memory.not-found.bench.ts b/benchmarks/memory/server/scenarios/error-paths/solid/memory.not-found.bench.ts new file mode 100644 index 0000000000..42d67b1164 --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/solid/memory.not-found.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[1]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/solid/memory.unmatched.bench.ts b/benchmarks/memory/server/scenarios/error-paths/solid/memory.unmatched.bench.ts new file mode 100644 index 0000000000..6179e469dd --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/solid/memory.unmatched.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[3]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/solid/tsconfig.json b/benchmarks/memory/server/scenarios/error-paths/solid/tsconfig.json index 4b61264e11..1e8103127a 100644 --- a/benchmarks/memory/server/scenarios/error-paths/solid/tsconfig.json +++ b/benchmarks/memory/server/scenarios/error-paths/solid/tsconfig.json @@ -7,7 +7,7 @@ "types": ["node", "vite/client", "vitest/globals"] }, "include": [ - "memory.bench.ts", + "memory*.bench.ts", "memory.flame.ts", "setup.ts", "vite.config.ts", diff --git a/benchmarks/memory/server/scenarios/error-paths/solid/vite.config.ts b/benchmarks/memory/server/scenarios/error-paths/solid/vite.config.ts index 401d1e5f25..d7a0bb51b0 100644 --- a/benchmarks/memory/server/scenarios/error-paths/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/error-paths/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,8 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + isolate: true, + ...memoryConfig('server'), name: '@benchmarks/memory-server error-paths (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/error-paths/vue/memory.bench.ts b/benchmarks/memory/server/scenarios/error-paths/vue/memory.bench.ts index 9a5c211fee..2780830cb2 100644 --- a/benchmarks/memory/server/scenarios/error-paths/vue/memory.bench.ts +++ b/benchmarks/memory/server/scenarios/error-paths/vue/memory.bench.ts @@ -4,8 +4,8 @@ import { workloadGroup } from './setup' await workloadGroup.sanity() +const workload = workloadGroup.workloads[0]! + describe('memory', () => { - for (const workload of workloadGroup.workloads) { - bench(workload.name, workload.run, memoryBenchOptions) - } + bench(workload.name, workload.run, memoryBenchOptions) }) diff --git a/benchmarks/memory/server/scenarios/error-paths/vue/memory.error.bench.ts b/benchmarks/memory/server/scenarios/error-paths/vue/memory.error.bench.ts new file mode 100644 index 0000000000..d41877e6ce --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/vue/memory.error.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[2]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/vue/memory.not-found.bench.ts b/benchmarks/memory/server/scenarios/error-paths/vue/memory.not-found.bench.ts new file mode 100644 index 0000000000..42d67b1164 --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/vue/memory.not-found.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[1]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/vue/memory.unmatched.bench.ts b/benchmarks/memory/server/scenarios/error-paths/vue/memory.unmatched.bench.ts new file mode 100644 index 0000000000..6179e469dd --- /dev/null +++ b/benchmarks/memory/server/scenarios/error-paths/vue/memory.unmatched.bench.ts @@ -0,0 +1,11 @@ +import { bench, describe } from 'vitest' +import { memoryBenchOptions } from '#memory-server/bench-utils' +import { workloadGroup } from './setup' + +await workloadGroup.sanity() + +const workload = workloadGroup.workloads[3]! + +describe('memory', () => { + bench(workload.name, workload.run, memoryBenchOptions) +}) diff --git a/benchmarks/memory/server/scenarios/error-paths/vue/tsconfig.json b/benchmarks/memory/server/scenarios/error-paths/vue/tsconfig.json index 9ad6481342..af8b83005c 100644 --- a/benchmarks/memory/server/scenarios/error-paths/vue/tsconfig.json +++ b/benchmarks/memory/server/scenarios/error-paths/vue/tsconfig.json @@ -7,7 +7,7 @@ "types": ["node", "vite/client", "vitest/globals"] }, "include": [ - "memory.bench.ts", + "memory*.bench.ts", "memory.flame.ts", "setup.ts", "vite.config.ts", diff --git a/benchmarks/memory/server/scenarios/error-paths/vue/vite.config.ts b/benchmarks/memory/server/scenarios/error-paths/vue/vite.config.ts index 9a0ca28a25..3dc0762aa0 100644 --- a/benchmarks/memory/server/scenarios/error-paths/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/error-paths/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,8 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + isolate: true, + ...memoryConfig('server'), name: '@benchmarks/memory-server error-paths (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/peak-large-page/react/vite.config.ts b/benchmarks/memory/server/scenarios/peak-large-page/react/vite.config.ts index adaacda86e..dcfcf1f92b 100644 --- a/benchmarks/memory/server/scenarios/peak-large-page/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/peak-large-page/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server peak-large-page (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/peak-large-page/shared.ts b/benchmarks/memory/server/scenarios/peak-large-page/shared.ts index 578fc9c5a2..486c025f5a 100644 --- a/benchmarks/memory/server/scenarios/peak-large-page/shared.ts +++ b/benchmarks/memory/server/scenarios/peak-large-page/shared.ts @@ -6,7 +6,7 @@ export type { StartRequestHandler } type Framework = 'react' | 'solid' | 'vue' const benchmarkSeed = 0x5eed_0005 -const peakLargePageIterations = 20 +const peakLargePageIterations = 1 const peakLargePageUrl = 'http://localhost/l1/l2/l3/l4/l5/l6/l7/l8' const levelEightMarker = 'data-bench="peak-large-page-level-8"' @@ -54,8 +54,6 @@ export function createWorkloadGroup( iterations: peakLargePageIterations, buildRequest: buildPeakLargePageRequest, validateResponse: validatePeakLargePageResponse, - pinGcBetweenIterations: true, - verifyGcFloor: true, }) return { diff --git a/benchmarks/memory/server/scenarios/peak-large-page/solid/vite.config.ts b/benchmarks/memory/server/scenarios/peak-large-page/solid/vite.config.ts index 27d4dd65f7..15b8c3a571 100644 --- a/benchmarks/memory/server/scenarios/peak-large-page/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/peak-large-page/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server peak-large-page (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/peak-large-page/vue/vite.config.ts b/benchmarks/memory/server/scenarios/peak-large-page/vue/vite.config.ts index 0f238bd7f7..3fab2aff9e 100644 --- a/benchmarks/memory/server/scenarios/peak-large-page/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/peak-large-page/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server peak-large-page (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/request-churn/react/vite.config.ts b/benchmarks/memory/server/scenarios/request-churn/react/vite.config.ts index 1e956958ec..35adf6a249 100644 --- a/benchmarks/memory/server/scenarios/request-churn/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/request-churn/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server request-churn (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/request-churn/shared.ts b/benchmarks/memory/server/scenarios/request-churn/shared.ts index a196eec8b5..32125148cd 100644 --- a/benchmarks/memory/server/scenarios/request-churn/shared.ts +++ b/benchmarks/memory/server/scenarios/request-churn/shared.ts @@ -68,7 +68,6 @@ export function createWorkloadGroup( iterations: requestChurnIterations, buildRequest: buildItemRequest, validateResponse: validateItemResponse, - pinGcBetweenIterations: true, }) return { diff --git a/benchmarks/memory/server/scenarios/request-churn/solid/vite.config.ts b/benchmarks/memory/server/scenarios/request-churn/solid/vite.config.ts index fbf8df9491..6601309a74 100644 --- a/benchmarks/memory/server/scenarios/request-churn/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/request-churn/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server request-churn (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/request-churn/vue/vite.config.ts b/benchmarks/memory/server/scenarios/request-churn/vue/vite.config.ts index a551a96e2f..7f15c5c7af 100644 --- a/benchmarks/memory/server/scenarios/request-churn/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/request-churn/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server request-churn (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/serialization-payload/react/vite.config.ts b/benchmarks/memory/server/scenarios/serialization-payload/react/vite.config.ts index de5549de3b..1ed3bcc335 100644 --- a/benchmarks/memory/server/scenarios/serialization-payload/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/serialization-payload/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server serialization-payload (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/serialization-payload/shared.ts b/benchmarks/memory/server/scenarios/serialization-payload/shared.ts index ce095b5e10..7f40aeb9d2 100644 --- a/benchmarks/memory/server/scenarios/serialization-payload/shared.ts +++ b/benchmarks/memory/server/scenarios/serialization-payload/shared.ts @@ -9,7 +9,7 @@ export type { StartRequestHandler } type Framework = 'react' | 'solid' | 'vue' const benchmarkSeed = 0x51eaa11 -const serializationPayloadIterations = 12 +const serializationPayloadIterations = 1 const payloadPageMarker = 'data-bench="serialization-payload"' const requestInit = { @@ -61,8 +61,6 @@ export function createWorkloadGroup( iterations: serializationPayloadIterations, buildRequest: buildPayloadRequest, validateResponse: validatePayloadResponse, - pinGcBetweenIterations: true, - verifyGcFloor: true, }) return { diff --git a/benchmarks/memory/server/scenarios/serialization-payload/solid/vite.config.ts b/benchmarks/memory/server/scenarios/serialization-payload/solid/vite.config.ts index 926646ee12..a2d04b986c 100644 --- a/benchmarks/memory/server/scenarios/serialization-payload/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/serialization-payload/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server serialization-payload (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/serialization-payload/vue/vite.config.ts b/benchmarks/memory/server/scenarios/serialization-payload/vue/vite.config.ts index 41a5596adf..b63a561228 100644 --- a/benchmarks/memory/server/scenarios/serialization-payload/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/serialization-payload/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server serialization-payload (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/server-fn-churn/react/vite.config.ts b/benchmarks/memory/server/scenarios/server-fn-churn/react/vite.config.ts index 68d9ed839b..f72cc53d6f 100644 --- a/benchmarks/memory/server/scenarios/server-fn-churn/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/server-fn-churn/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server server-fn-churn (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/server-fn-churn/shared.ts b/benchmarks/memory/server/scenarios/server-fn-churn/shared.ts index 6e2959e757..5fd1e22632 100644 --- a/benchmarks/memory/server/scenarios/server-fn-churn/shared.ts +++ b/benchmarks/memory/server/scenarios/server-fn-churn/shared.ts @@ -38,8 +38,6 @@ type SerovalNode = const benchmarkSeed = 0xdecafbad const payloadSeed = 0x51f0cafe const fixtureCount = 16 -// Sized to sit just above the 2s measured-run floor on CI (per-iteration -// cost with the pinned collection is ~0.08-0.11s across frameworks). const serverFnChurnIterations = 30 const origin = 'http://localhost' const tssContentTypeFramed = 'application/x-tss-framed' @@ -208,7 +206,6 @@ export async function createWorkloadGroup( runSequentialRequestLoop(handler, { seed: benchmarkSeed, iterations: serverFnChurnIterations, - pinGcBetweenIterations: true, buildRequest: (_random, index) => { const fixtureIndex = Math.floor(index / 2) % fixtureCount diff --git a/benchmarks/memory/server/scenarios/server-fn-churn/solid/vite.config.ts b/benchmarks/memory/server/scenarios/server-fn-churn/solid/vite.config.ts index 8d6d226e97..4fba196bd2 100644 --- a/benchmarks/memory/server/scenarios/server-fn-churn/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/server-fn-churn/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server server-fn-churn (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/server-fn-churn/vue/vite.config.ts b/benchmarks/memory/server/scenarios/server-fn-churn/vue/vite.config.ts index 0e249b9f3b..5ab5ec37e3 100644 --- a/benchmarks/memory/server/scenarios/server-fn-churn/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/server-fn-churn/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server server-fn-churn (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/streaming-peak/react/vite.config.ts b/benchmarks/memory/server/scenarios/streaming-peak/react/vite.config.ts index 5af0f0d138..5d555441b6 100644 --- a/benchmarks/memory/server/scenarios/streaming-peak/react/vite.config.ts +++ b/benchmarks/memory/server/scenarios/streaming-peak/react/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/react-start/plugin/vite' import react from '@vitejs/plugin-react' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server streaming-peak (react)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/streaming-peak/shared.ts b/benchmarks/memory/server/scenarios/streaming-peak/shared.ts index 6b45470329..01917c4743 100644 --- a/benchmarks/memory/server/scenarios/streaming-peak/shared.ts +++ b/benchmarks/memory/server/scenarios/streaming-peak/shared.ts @@ -9,7 +9,7 @@ export type { StartRequestHandler } type Framework = 'react' | 'solid' | 'vue' const benchmarkSeed = 0xdecafbad -const streamingPeakIterations = 20 +const streamingPeakIterations = 1 const fallbackMarker = 'streaming-peak-fallback-0' const requestInit = { @@ -50,7 +50,7 @@ async function readStreamingBody(response: Response) { let body = '' let chunkCount = 0 - while (true) { + for (;;) { const result = await reader.read() if (result.done) { @@ -98,12 +98,15 @@ export function createWorkloadGroup( iterations: streamingPeakIterations, buildRequest: buildStreamingRequest, validateResponse: validateStreamingResponse, - pinGcBetweenIterations: true, - verifyGcFloor: true, }) return { - sanity: () => assertStreamingPeakSanity(handler), + sanity: async () => { + // Import the route chunks before checking streaming. Cold disk I/O can + // otherwise outlast the deferred payload and collapse the first response. + await run() + await assertStreamingPeakSanity(handler) + }, workloads: [ { name: `mem server streaming-peak chunked (${framework})`, diff --git a/benchmarks/memory/server/scenarios/streaming-peak/solid/vite.config.ts b/benchmarks/memory/server/scenarios/streaming-peak/solid/vite.config.ts index 18bf5b2d04..5fbdb48c23 100644 --- a/benchmarks/memory/server/scenarios/streaming-peak/solid/vite.config.ts +++ b/benchmarks/memory/server/scenarios/streaming-peak/solid/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/solid-start/plugin/vite' import solid from 'vite-plugin-solid' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server streaming-peak (solid)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/scenarios/streaming-peak/vue/vite.config.ts b/benchmarks/memory/server/scenarios/streaming-peak/vue/vite.config.ts index 7cd481da0b..cd848584a5 100644 --- a/benchmarks/memory/server/scenarios/streaming-peak/vue/vite.config.ts +++ b/benchmarks/memory/server/scenarios/streaming-peak/vue/vite.config.ts @@ -3,6 +3,7 @@ import { defineConfig } from 'vitest/config' import codspeedPlugin from '@codspeed/vitest-plugin' import { tanstackStart } from '@tanstack/vue-start/plugin/vite' import vueJsx from '@vitejs/plugin-vue-jsx' +import { memoryConfig } from '../../../../runtime' const rootDir = fileURLToPath(new URL('.', import.meta.url)) @@ -22,19 +23,7 @@ export default defineConfig({ minify: false, }, test: { - // Keep lazily-compiled code alive for the whole run: the pinned - // collections age code fast enough for V8 to flush unused bytecode, - // and the mid-measurement recompile injects a multi-MB allocation - // burst at a run-dependent time. - execArgv: [ - '--no-flush-bytecode', - // Pre-size the V8 heap so no space has to grow mid-measurement: - // heap-growth events allocate several MB at a run-dependent moment, - // which flips the measured peak bimodally between identical runs. - '--initial-old-space-size=64', - '--min-semi-space-size=16', - '--max-semi-space-size=16', - ], + ...memoryConfig('server'), name: '@benchmarks/memory-server streaming-peak (vue)', watch: false, environment: 'node', diff --git a/benchmarks/memory/server/streaming-sanity.test.ts b/benchmarks/memory/server/streaming-sanity.test.ts new file mode 100644 index 0000000000..d4b37aa6ab --- /dev/null +++ b/benchmarks/memory/server/streaming-sanity.test.ts @@ -0,0 +1,35 @@ +import { expect, it, vi } from 'vitest' +import { createWorkloadGroup } from './scenarios/streaming-peak/shared' + +function response(chunks: number) { + return new Response( + new ReadableStream({ + start(controller) { + for (let index = 0; index < chunks; index++) { + controller.enqueue( + new TextEncoder().encode('streaming-peak-fallback-0'), + ) + } + controller.close() + }, + }), + ) +} + +it('checks streaming after the route has loaded', async () => { + const fetch = vi + .fn() + .mockImplementationOnce(() => response(1)) + .mockImplementationOnce(() => response(2)) + await createWorkloadGroup('solid', { fetch }).sanity() + expect(fetch).toHaveBeenCalledTimes(2) +}) + +it('still rejects a buffered response after warmup', async () => { + const fetch = () => response(1) + await expect( + createWorkloadGroup('solid', { fetch }).sanity(), + ).rejects.toThrow( + 'Expected chunked sanity response to produce multiple chunks', + ) +}) diff --git a/benchmarks/memory/server/tsconfig.json b/benchmarks/memory/server/tsconfig.json index 5d5dcbf825..ffb2a11caa 100644 --- a/benchmarks/memory/server/tsconfig.json +++ b/benchmarks/memory/server/tsconfig.json @@ -3,5 +3,12 @@ "compilerOptions": { "types": ["node", "vite/client", "vitest/globals"] }, - "include": ["bench-utils.ts"] + "include": [ + "bench-utils.ts", + "scenarios/*/shared.ts", + "../runtime.ts", + "*.test.ts", + "vitest.unit.config.ts", + "vitest.setup.ts" + ] } diff --git a/benchmarks/memory/server/vitest.setup.ts b/benchmarks/memory/server/vitest.setup.ts new file mode 100644 index 0000000000..be96c0067a --- /dev/null +++ b/benchmarks/memory/server/vitest.setup.ts @@ -0,0 +1,4 @@ +import { beforeEach } from 'vitest' +import { endMemoryTurn } from '../turn' + +beforeEach(endMemoryTurn) diff --git a/benchmarks/memory/server/vitest.unit.config.ts b/benchmarks/memory/server/vitest.unit.config.ts new file mode 100644 index 0000000000..19405a6f92 --- /dev/null +++ b/benchmarks/memory/server/vitest.unit.config.ts @@ -0,0 +1,9 @@ +import { defineConfig } from 'vitest/config' + +export default defineConfig({ + test: { + watch: false, + environment: 'node', + include: ['*.test.ts'], + }, +}) diff --git a/benchmarks/memory/tsconfig.json b/benchmarks/memory/tsconfig.json new file mode 100644 index 0000000000..5a325f3b6c --- /dev/null +++ b/benchmarks/memory/tsconfig.json @@ -0,0 +1,7 @@ +{ + "extends": "../../tsconfig.json", + "compilerOptions": { + "types": ["node"] + }, + "include": ["runtime.ts"] +} diff --git a/benchmarks/memory/turn.ts b/benchmarks/memory/turn.ts new file mode 100644 index 0000000000..3ee6d53833 --- /dev/null +++ b/benchmarks/memory/turn.ts @@ -0,0 +1,12 @@ +export function isMemoryInstrumented() { + return ( + process.env.CODSPEED_ENV !== undefined && + process.env.CODSPEED_RUNNER_MODE === 'memory' + ) +} + +// CodSpeed calls beforeEach before its forced GC and measurement marker. End +// the previous warmup's job so WeakRef targets can be released by that GC. +export function endMemoryTurn() { + return new Promise((resolve) => setImmediate(resolve)) +}