0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
mastra/packages/memory/scripts/simulate/reconstruct.test.ts
Jamie Nichols eda942cb76
feat(memory): add an observational memory replay simulator (#22058)
## Summary

Add local conversation simulation tooling to `@mastra/memory` for
comparing experimental Subconscious capture and curation prompt changes
against reconstructed real observation cycles.

## Root cause

Evaluating prompt changes through deployments mixes prompt behavior with
newly derived cycle boundaries and requires days of organic usage.
Re-observing raw messages would also let the tested prompt move
segmentation boundaries, making A/B results look meaningful while
changing more than one variable.

The simulator tests are colocated with the implementation they cover
under `packages/memory/scripts/simulate`, which sits outside the
package's original `src/**/*.test.ts` Vitest include. The package
include is widened to `['src/**/*.test.ts', 'scripts/**/*.test.ts']` so
they are discovered and run as part of the normal unit suite.

## What changed

- Add local extract, replay, and A/B commands under
`packages/memory/scripts/simulate`.
- Reconstruct persisted observation cycles while excluding reflection
heads from later generations.
- Reject duplicate `generationCount` records as ambiguous instead of
replaying them in unstable order.
- Preserve deterministic ordering and report missing-generation
warnings.
- Replay through the real capture and curation path with optional
production knowledge-resource scoping.
- Isolate every arm in a fresh local database and refuse remote write
targets.
- Add an A/A control to expose model sampling noise.
- Keep the simulator suites beside the implementation under
`packages/memory/scripts/simulate`, named after the module each covers
(`diff.test.ts`, `extract.test.ts`, `replay.test.ts`,
`reconstruct.test.ts`, `drive.test.ts`, `drive.isolation.test.ts`), and
widen the package Vitest include so they are still collected by the unit
run.
- Add simulator TypeScript checking to package verification.
- Export the existing observation-boundary regex for simulator reuse
without changing its pattern or runtime behavior.

## Design notes

This is a bounded simulator, not a general PostgreSQL cloning tool.
Generated target DDL does not preserve every PostgreSQL type,
constraint, default, or precision detail. Worklist counting is capped,
missing generations produce incomplete comparisons with warnings, and
the content differ remains aggregate and positional in several cases.

The replay command dynamically imports the built PostgreSQL adapter, so
linked workspace packages must be built before running the simulator
from a fresh checkout. The README documents the local PostgreSQL,
pgvector, credentials, and build prerequisites.

The widened Vitest include makes the simulator suites part of the
default `@mastra/memory` unit run while leaving them colocated with the
tooling. Normal production TypeScript scope remains unchanged because
package TypeScript excludes `**/*.test.ts`; simulator checking remains
owned by `tsconfig.scripts.json`.

## Safety

- Source access is read-only.
- Replay targets must use a literal IPv4 or IPv6 loopback address;
hostnames and routing query parameters such as `host`, `hostaddr`, and
`service` are rejected.
- Extraction, standalone replay, and database recreation attest the live
PostgreSQL server address as loopback before target initialization,
writes, or destructive operations.
- Extraction compares connected database identity and port, treating
IPv4 and IPv6 loopback as the same local server, and refuses to
overwrite the source database.
- Each arm uses a separate fresh database.
- No conversation or knowledge payload is committed.

Tracking: COR-1167

## Test plan

From `packages/memory`:

```bash
pnpm vitest run scripts/simulate --reporter=dot
pnpm test:unit
pnpm lint
git diff --check
```

Results on the reviewed local head:

- Simulator selection: 78 passed across 6 files.
- Full memory unit suite: 1,602 passed and 1 todo across 68 files.
- Lint: 0 warnings and 0 errors across 207 files.
- Diff check: passed.
- Simulator suites are discovered at their colocated path by the widened
include; the full unit run collects 69 files with the suites in place.

The production-source blast radius is limited to exporting the existing
observation-boundary regex. Its pattern and runtime behavior are
unchanged. No changeset is included: this adds repo-local tooling under
`scripts/`, not a shipped package feature.

---------

Co-authored-by: Jamie Nichols <CodingButter@users.noreply.github.com>
Co-authored-by: Jamie Nichols <10267569+CodingButter@users.noreply.github.com>
Co-authored-by: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
2026-08-31 14:32:38 -04:00

182 lines
6.6 KiB