0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
Commit graph

2 commits

Author SHA1 Message Date
Jamie Nichols
b40fa91496
refactor(memory): curate observations directly in Subconscious (#22738)
## Summary

Simplify experimental Subconscious knowledge ingestion around two
built-ins, `remind` and `curate`, both configured in the observation
agent list:

- keep `remind` unchanged for selective retrieval of relevant durable
knowledge;
- make `curate` the sole experimental Subconscious ingestion and
reconciliation agent, receiving each successfully committed observation
directly;
- remove the separate `capture` and `learn` agents, reflection-time
Subconscious execution, and Subconscious-owned worklist/cursor/cadence
orchestration.

This is intentionally deletion-heavy to reduce two ingestion prompts and
quality surfaces to one: 40 files changed, with 1,047 additions and
4,032 deletions overall; Subconscious production source has 80 additions
and 718 deletions.

This is plan-driven work with no linked GitHub issue.

## How direct curation works

Observation strategies preserve the raw per-cycle observation delta
separately from cumulative observational memory. After persistence and
observation end markers succeed, the shared strategy lifecycle awaits
one failure-isolated curator invocation per nonblank source-thread delta
before normal reflection, so curation model latency is now part of every
observation-cycle path that produces a delta. Resource-scoped cycles
retain per-thread attribution and additionally run one curator call per
contributing thread delta, serially.

The curator receives the current observation delta, searches relevant
existing knowledge through constrained read tools, and creates or
reconciles durable knowledge through code-governed write tools.
Provenance, timestamps, source identity, scope ceilings, versioning,
activity, mention indexing, and semantic indexing remain code-owned
rather than model-supplied.

Reminder execution remains independent: reminder failure or no-signal
cannot suppress curator delivery. Curator failure is reported and
isolated after observation commit; it does not fail the committed
observation or prevent normal reflection.

## Deliberate reliability boundaries

Direct curator delivery is best-effort in this PR. Removing the durable
worklist/cursor frontier means a failed or interrupted curator run is
reported but is not automatically replayed. This PR does not introduce a
replacement queue, scheduler, retry, cadence, or backoff mechanism.

Core storage worklist/cursor APIs, including `knowledgeBySource`, remain
for other consumers such as Factory's knowledge route; the Subconscious
runtime stops using them for curation orchestration.

The curator continues to require `organizationId` in request context,
matching the prior capture path; a missing value is reported as a
curator error after observation commit.

The new `knowledge_create` tool preserves the existing node-first
storage sequence: if the initial record append fails after node
creation, the node may remain. No new cross-adapter transaction contract
is introduced.

## Compatibility and cross-package cleanup

- `@mastra/memory`: removes `capture`, `learn`, reflection-time
Subconscious execution, worklist/cursor/cadence orchestration, and the
public `Memory.runCuration()` method. There is no replacement manual
curation entry point; Subconscious curation now runs only from
successfully committed observations.
- `@mastra/code-sdk`: removes Factory-only capture-pinning and cadence
configuration.
- `@mastra/factory`: removes phase-exit curation triggers that depended
on `Memory.runCuration()`.
- Simulator: removes capture arms, cadence/tail flushing, cursor
accounting, and A/B comparison in favor of one direct-curator replay
path.

A minimal patch changeset is included for `@mastra/memory`,
`@mastra/code-sdk`, and `@mastra/factory`.

## Verification

- Focused Subconscious, lifecycle, replay, reminder, scope, activity,
read-tool, and knowledge-write coverage: 11 files, 59 tests passed.
- Full `@mastra/memory` unit suite: 63 files, 1,497 tests passed, 1
todo.
- Memory simulator suite: 3 files, 46 tests passed.
- `pnpm build:memory`: 15/15 tasks passed.
- Memory typecheck and lint passed; lint reported 0 warnings and 0
errors.
- `@mastra/factory` build, typecheck, full tests, and lint passed: 103
files, 1,980 tests passed, 6 skipped.
- `@mastra/code-sdk` build, typecheck, and lint passed; the changed
memory-construction suite passed 15/15 tests.
- Red/green lifecycle coverage and a deterministic two-cycle replay over
real in-memory storage verify direct delivery, reconciliation to one
active fact, later cross-thread reminder retrieval, and no Subconscious
worklist/cursor calls in the exercised paths. No live hosted-model run
is claimed.
- Removal greps, `git diff --check`, and clean-status checks passed.

Two broader gates could not complete cleanly; details follow.

## Verification limitations

The isolated `packages/memory/integration-tests` bootstrap could not
install dependencies in this environment: the frozen install reports an
override mismatch, and the required non-frozen retry cannot fetch
private `@internal/ai-v6` from the npm registry. The integration tests
therefore did not execute.

The full Code SDK suite completed with 105 files and 1,524 tests
passing, while four tests failed from startup timeout, `spawn E2BIG`,
callback-port `EADDRINUSE`, and an MCP OAuth callback assertion. These
failures were not used as evidence for this change; the changed SDK
memory-construction suite passes independently.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

The PR makes `curate` the single agent that turns new observations into
knowledge. It removes older capture, learn, and automatic scheduling
paths so curation runs independently after observations are saved.

## Changes

- Refactor Subconscious ingestion around `remind` and `curate`.
- Dispatch committed observation deltas directly to
`SubconsciousCurateExtractor`.
- Make curation asynchronous and failure-isolated.
- Remove reflection-time execution, capture and learn agents, curation
cadence, worklist orchestration, and cursor management.
- Preserve reminder execution, provenance, scope enforcement, and
code-owned indexing.
- Add `knowledge_create` with scoped writes and provenance metadata.
- Validate RFC 3339 timestamps for knowledge writes.
- Harden observation framing against untrusted input.
- Remove related public APIs and types from Memory, Code SDK, Factory,
and simulator integrations.
- Replace the simulator A/B workflow with direct observation replay.
- Add tests for replay isolation, curation behavior, scope enforcement,
provenance, failure handling, and input validation.
- Add a patch changeset for `@mastra/memory`, `@mastra/code-sdk`, and
`@mastra/factory`.

## Validation

Focused tests, memory and simulator suites, builds, typechecks, linting,
and relevant Factory and Code SDK checks passed. Memory integration
tests were blocked by dependency installation issues. Four unrelated
failures remained in the full Code SDK suite; changed SDK tests passed
independently.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Jamie Nichols <CodingButter@users.noreply.github.com>
Co-authored-by: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
Co-authored-by: Jamie Nichols <10267569+CodingButter@users.noreply.github.com>
Co-authored-by: Tyler Barnes <tylerdbarnes@gmail.com>
2026-09-01 23:03:56 -07:00
Jamie Nichols
eda942cb76
feat(memory): add an observational memory replay simulator (#22058)
## Summary

Add local conversation simulation tooling to `@mastra/memory` for
comparing experimental Subconscious capture and curation prompt changes
against reconstructed real observation cycles.

## Root cause

Evaluating prompt changes through deployments mixes prompt behavior with
newly derived cycle boundaries and requires days of organic usage.
Re-observing raw messages would also let the tested prompt move
segmentation boundaries, making A/B results look meaningful while
changing more than one variable.

The simulator tests are colocated with the implementation they cover
under `packages/memory/scripts/simulate`, which sits outside the
package's original `src/**/*.test.ts` Vitest include. The package
include is widened to `['src/**/*.test.ts', 'scripts/**/*.test.ts']` so
they are discovered and run as part of the normal unit suite.

## What changed

- Add local extract, replay, and A/B commands under
`packages/memory/scripts/simulate`.
- Reconstruct persisted observation cycles while excluding reflection
heads from later generations.
- Reject duplicate `generationCount` records as ambiguous instead of
replaying them in unstable order.
- Preserve deterministic ordering and report missing-generation
warnings.
- Replay through the real capture and curation path with optional
production knowledge-resource scoping.
- Isolate every arm in a fresh local database and refuse remote write
targets.
- Add an A/A control to expose model sampling noise.
- Keep the simulator suites beside the implementation under
`packages/memory/scripts/simulate`, named after the module each covers
(`diff.test.ts`, `extract.test.ts`, `replay.test.ts`,
`reconstruct.test.ts`, `drive.test.ts`, `drive.isolation.test.ts`), and
widen the package Vitest include so they are still collected by the unit
run.
- Add simulator TypeScript checking to package verification.
- Export the existing observation-boundary regex for simulator reuse
without changing its pattern or runtime behavior.

## Design notes

This is a bounded simulator, not a general PostgreSQL cloning tool.
Generated target DDL does not preserve every PostgreSQL type,
constraint, default, or precision detail. Worklist counting is capped,
missing generations produce incomplete comparisons with warnings, and
the content differ remains aggregate and positional in several cases.

The replay command dynamically imports the built PostgreSQL adapter, so
linked workspace packages must be built before running the simulator
from a fresh checkout. The README documents the local PostgreSQL,
pgvector, credentials, and build prerequisites.

The widened Vitest include makes the simulator suites part of the
default `@mastra/memory` unit run while leaving them colocated with the
tooling. Normal production TypeScript scope remains unchanged because
package TypeScript excludes `**/*.test.ts`; simulator checking remains
owned by `tsconfig.scripts.json`.

## Safety

- Source access is read-only.
- Replay targets must use a literal IPv4 or IPv6 loopback address;
hostnames and routing query parameters such as `host`, `hostaddr`, and
`service` are rejected.
- Extraction, standalone replay, and database recreation attest the live
PostgreSQL server address as loopback before target initialization,
writes, or destructive operations.
- Extraction compares connected database identity and port, treating
IPv4 and IPv6 loopback as the same local server, and refuses to
overwrite the source database.
- Each arm uses a separate fresh database.
- No conversation or knowledge payload is committed.

Tracking: COR-1167

## Test plan

From `packages/memory`:

```bash
pnpm vitest run scripts/simulate --reporter=dot
pnpm test:unit
pnpm lint
git diff --check
```

Results on the reviewed local head:

- Simulator selection: 78 passed across 6 files.
- Full memory unit suite: 1,602 passed and 1 todo across 68 files.
- Lint: 0 warnings and 0 errors across 207 files.
- Diff check: passed.
- Simulator suites are discovered at their colocated path by the widened
include; the full unit run collects 69 files with the suites in place.

The production-source blast radius is limited to exporting the existing
observation-boundary regex. Its pattern and runtime behavior are
unchanged. No changeset is included: this adds repo-local tooling under
`scripts/`, not a shipped package feature.

---------

Co-authored-by: Jamie Nichols <CodingButter@users.noreply.github.com>
Co-authored-by: Jamie Nichols <10267569+CodingButter@users.noreply.github.com>
Co-authored-by: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai>
2026-08-31 14:32:38 -04:00