mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
5 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
43f77e77a0 |
ci: delete the Changed Test Gate (#29099)
## What this does
Deletes the Changed Test Gate workflow, its labeler workflow (which
labels PRs from the gate's result) and `scripts/pr-test-gate/`. The
gate's deployer rule is kept: it moves into Prebuild as a small
`Deployer E2E coverage` job.
## What the gate enforced, and what goes
The gate checked out the base branch, restored the PR's changed test
files on top of it, and ran them. Removing it drops two rules:
1. **Fail-before-fix.** If a PR changed tests, those tests had to fail
against the base branch ("Changed tests passed against the base branch.
They should fail before the PR code is applied."). This rule goes.
2. **External contributors must add tests.** A PR from outside the org
(`AUTHOR_ASSOCIATION` not OWNER/MEMBER/COLLABORATOR) with no changed
test files failed. This rule goes.
A third rule stays: **a PR that changes `packages/deployer/` must also
change `e2e-tests/monorepo/`** (`packages/deployer/AGENTS.md` and
`CHANGELOG.md` exempt; the same-repo Changesets release PR skipped).
It's now `Deployer E2E coverage` in `prebuild.yml`: a `git diff
--name-only` against the merge base, no install or build. It's its own
job rather than a step in `changes`, because a failing `changes` job
would skip every test job. `packages/deployer/AGENTS.md` now names that
job.
## Why dropping 1 and 2 is acceptable
- Neither rule was enforced at merge time: the gate isn't a required
check. The required checks are `Lint` and `Unit and E2E Tests / Merge
Test Reports`, and this PR touches neither.
- Of the gate's last 1000 runs, 8 failed. All 8 were the fail-before-fix
rule firing on tests that passed without the fix; each PR resolved it by
rewriting its tests. Across 812 gate logs from Aug–Oct, every run that
executed tests took the passing path (changed tests failed against base,
as required). So it shaped how a few tests were written, but no run
showed it stopping broken code from merging.
- It costs a runner, a checkout and a test run on every non-draft PR, on
top of Prebuild running the same changed tests against the PR head.
- External contributions still go through maintainer review, which can
ask for tests.
## Test plan
- [x] `rg` for remaining references: only two "former" comments and an
unrelated fixture name
- [x] `pnpm install --frozen-lockfile` works after removing
`scripts/pr-test-gate` (192 projects)
- [x] Deployer check shell logic run locally under `bash -eo pipefail`
with a stubbed diff: deployer-only change fails with the annotation;
deployer + `e2e-tests/monorepo/` passes; `AGENTS.md`/`CHANGELOG.md`-only
passes; no deployer change passes
- [ ] CI on this PR: no Changed Test Gate check appears; `Deployer E2E
coverage` passes
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
This change removes the old system that checked whether pull requests
added and tested tests. It keeps one focused rule: changes to the
deployer package must include a monorepo E2E test change, with specified
documentation files exempt.
## Changes
- Deleted the Changed Test Gate workflow, its labeler workflow, and the
`scripts/pr-test-gate/` package.
- Added a separate `Deployer E2E coverage` job to Prebuild. The job
checks the merge-base diff and fails when a non-exempt file under
`packages/deployer/` changes without a file under `e2e-tests/monorepo/`.
- Exempted `packages/deployer/AGENTS.md` and
`packages/deployer/CHANGELOG.md` from the coverage check.
- Updated `packages/deployer/AGENTS.md` to refer to the Prebuild job.
- Updated comments in the memory simulation config and two test
link-provider files.
## Validation
The PR description reports that `pnpm install --frozen-lockfile`
succeeded across 192 projects and that the deployer-check shell logic
passed the listed local cases. CI validation is not confirmed: the
description says it is not yet known whether the old check disappears
and the new job passes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: remy-builder[bot] <324360501+remy-builder[bot]@users.noreply.github.com>
|
||
|
|
0a553f0f6f |
Knowledge alpha: standalone Knowledge for agents, Subconscious removed (#28353)
## What this is
**Knowledge** is Mastra's store for durable knowledge that agents share
across conversations. Until now its agent-facing parts lived in
`@mastra/memory`, inside an experimental feature called
**Subconscious**, a background curator that wrote to Knowledge after
observational-memory reflections.
This PR makes Knowledge stand on its own as an experimental alpha,
rebuilt around pages and declared scopes.
- **Subconscious is removed** from `@mastra/memory` and Mastra Code.
Setting `experimental_subconscious` or the Memory `knowledge` option
throws, saying to attach Knowledge to the agent instead. Old
Subconscious data is ignored.
- **Knowledge attaches to an agent directly:** `new Agent({ knowledge,
scopes })`. It brings its own tools, instructions and pinned context.
Tools work without Memory; pinned context needs Memory.
- **Knowledge is a tree of pages.** Each page lives in one scope and
holds a body plus short notes. Notes can mention other pages with
`[[wikilinks]]`. A page can have a key, unique among its siblings, and
is addressed by key path: `<scope>#<key>/<child key>`. Pages declared in
a scope's config are created when missing and can be edited but not
moved, deleted or merged away by tools.
- **Access comes from declared scopes.** Agent `scopes` are plain
`"<type>:<value>"` strings that say who the run is, set by the host and
never by the model. Knowledge's required `scopes` option declares which
scope types exist and which roles (viewer, contributor, editor, owner)
holders get on which scopes. Rules reach one hop with no inheritance. A
run holding an undeclared scope fails. `knowledgeMemoryScopes()`
declares Memory's `thread:`/`resource:` scopes and an `org:` scope.
- **Five tools:** `knowledge_search`, `knowledge_read`, `knowledge_page`
(write, move, merge, delete), `knowledge_note` (add, restamp, remove)
and `knowledge_pin` (pin, unpin). Tools are off by default: `tools:
true` enables read and write, `tools: { read: true }` read only, and
`pinnedContext` adds pins. Deleting a page needs owner on the page's
scope and is refused for pages with child pages, pages declared in a
scope's config, and scope pages. On a scope page, write accepts the body
only: its name and description come from config.
- **Limits are in tokens:** about 250 per note and 100 per page
description. Pinned context defaults to 500 tokens, with a maximum of
2,000.
- **Search:** keyword search by default: every word of the query must
appear in the page (or note), and name matches rank first, the same in
every store. With a `vector` store and an `embedder`, search also
returns semantic matches. Every semantic hit is re-checked against what
the run can read before ranking, so hidden content never takes a result
slot. `knowledge.reindex()` re-embeds everything.
- **Host APIs:** `knowledge.readScopeFor(principals)` returns what
holders of those scopes can read. `knowledge.explainAccess(runScopes,
target)` says why a run can or can't reach a scope or page.
`explainAccess` is host-only.
- **Renamed scope declarations:** if stored scopes use a declaration
name the config no longer declares, Knowledge fails loudly on first use,
naming each orphaned name and how many scopes use it.
`knowledge.renameScopeDeclaration(from, to)` renames them. It is
host-only and atomic, except on MongoDB without a replica set.
- **Per-message scopes:** `session.sendMessage({ content, scopes })`,
plus queue, steer and notifications. A message whose scopes differ from
the running run's waits for that run to end and then starts its own run.
Scopes are set in-process only.
- **Storage:** LibSQL, PostgreSQL, MySQL and MongoDB store Knowledge as
schema 4 (pages, notes, mentions, pins). On startup, the tables of the
earlier experimental Knowledge node store (shipped in 1.79.0) are
dropped with any rows in them, and the new tables are created. If an
older process recreates them during a rolling deploy, the next startup
drops them again. Schema-4 tables and non-Knowledge tables are never
touched. On MongoDB, Knowledge collections found without a schema
version are adopted as schema 4 rather than refused. Don't run
`dangerouslyReset()` from an older version against a database the new
version already uses. Development databases created from earlier commits
of this branch also need a reset, since sibling key uniqueness changed
within schema 4. `storage.stores?.knowledge?.dangerouslyReset()` clears
Knowledge rows only, never Memory. Adapter peer floors are unchanged: on
an older `@mastra/core` the adapters still load and the other storage
domains keep working.
- **Mastra Code:** Knowledge is opt-in through **Experimental
Knowledge** in `/settings`. The `/knowledge` browser and the SDK
inspector read pages. Factory's Knowledge routes are disabled until the
Studio page readers land.
## Test plan
- Shared schema-4 storage contract suite on real stores (as of
|
||
|
|
eb7090b2d1 |
feat(storage): add LibSQL and PostgreSQL Knowledge v2 schema (#26018)
> Supersedes #22507, closed during the Knowledge pause. GitHub cannot reopen it because the branch was rebased onto current `main`; its review history stays on #22507. Wave: 1 of 4 Segment: W1.2 of 8 Purpose: Add normalized Knowledge v2 physical storage and explicit reset-safe schema negotiation for LibSQL and PostgreSQL while preserving MySQL and MongoDB v1 compatibility. Previous PR: https://github.com/mastra-ai/mastra/pull/26017 Next PR: https://github.com/mastra-ai/mastra/pull/26019 Previous wave: None Next wave: feat/knowledge-v2-w2-contract (planned) ## Summary - creates and indexes all 15 Knowledge-owned v2 tables in LibSQL and PostgreSQL - requires every expected table/column plus an adapter-owned completion marker before treating a schema as compatible - keeps normalized node/record scope projections, activity linkage, CAS versions, mentions, and semantic outbox writes transactionally coherent - integrates PostgreSQL schema inspection with the shared init snapshot and keeps explicit reset limited to Knowledge-owned objects - registers normalized tables in Core and InMemory without changing MySQL/MongoDB v1 schemas ## Test plan - `pnpm build:core` - `pnpm --filter ./packages/core check` - `pnpm --dir packages/core exec vitest run src/storage/domains/knowledge/__tests__/base.test.ts src/storage/domains/knowledge/__tests__/scope.test.ts --reporter=dot --bail 1` - `pnpm --filter ./stores/libsql test -- --reporter=dot --bail 1` (997 passed, 19 skipped) - `pnpm --dir stores/libsql exec vitest run src/storage/domains/knowledge/index.test.ts --reporter=dot --bail 1` (29 passed) - `pnpm --dir stores/pg exec vitest run src/storage/domains/knowledge/index.test.ts src/storage/db/schema-snapshot.test.ts src/storage/db/schema-snapshot.stress.test.ts --reporter=dot --bail 1` (46 passed) - MySQL v1 Knowledge focused suite (27 passed) - Turbo builds for LibSQL, PostgreSQL, MySQL, and MongoDB - Prettier, Oxlint, and `git diff --check` ## Known unrelated baseline evidence - PostgreSQL full suite passed 2,038 tests with 121 skipped; one workflow planner-index assertion selected a different valid index. - MongoDB focused v1 Knowledge testing exposed the shipped unified-collection unique-index collision for record documents with null node-only keys; this PR does not modify MongoDB. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## ELI5 This update gives Knowledge a more organized way to store its data in LibSQL and PostgreSQL. It creates the new tables when Knowledge is first used, and it stops if it finds a database layout it cannot safely recognize. ## Changes - Added Knowledge v2 schemas and compatibility checks. Initialization verifies all required tables and columns, plus a schema-version marker. - Added normalized storage for scopes, grants, addresses, imports, proposals, activity, and other Knowledge data. LibSQL and PostgreSQL maintain scope links and record versions during Knowledge updates. - Made Knowledge initialization lazy. `storage.init()` skips Knowledge. When automatic initialization is enabled, `storage.getStore('knowledge')` activates it. Call `storage.initKnowledge()` to activate it explicitly. - Added guarded handling for recognized published v1 schemas. The adapter drops those tables without migrating their data. Unrecognized layouts remain unchanged and require an explicit reset. - Limited `dangerouslyReset()` to Knowledge-owned data. PostgreSQL reset fails if dependencies prevent the operation. - Removed the `maxScope` ceiling from Knowledge storage and `Subconscious` configuration. Knowledge writes can use any scope visible to the conversation. - Preserved MySQL and MongoDB Knowledge schemas. ## Validation The PR description reports passing build, type-check, lint, formatting, and focused storage tests. Reported results include 997 LibSQL tests, 46 PostgreSQL Knowledge and schema-snapshot tests, and 27 MySQL v1 Knowledge tests. The PostgreSQL full suite reportedly passed 2,038 tests, with 121 skipped. The description also reports a MongoDB focused-test unique-index collision involving record documents with null node-only keys. This PR does not change MongoDB’s schema. These test results are author-reported and were not independently verified here. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: remy-builder[bot] <324360501+remy-builder[bot]@users.noreply.github.com> Co-authored-by: Jamie Lee Nichols <10267569+CodingButter@users.noreply.github.com> |
||
|
|
b40fa91496 |
refactor(memory): curate observations directly in Subconscious (#22738)
## Summary Simplify experimental Subconscious knowledge ingestion around two built-ins, `remind` and `curate`, both configured in the observation agent list: - keep `remind` unchanged for selective retrieval of relevant durable knowledge; - make `curate` the sole experimental Subconscious ingestion and reconciliation agent, receiving each successfully committed observation directly; - remove the separate `capture` and `learn` agents, reflection-time Subconscious execution, and Subconscious-owned worklist/cursor/cadence orchestration. This is intentionally deletion-heavy to reduce two ingestion prompts and quality surfaces to one: 40 files changed, with 1,047 additions and 4,032 deletions overall; Subconscious production source has 80 additions and 718 deletions. This is plan-driven work with no linked GitHub issue. ## How direct curation works Observation strategies preserve the raw per-cycle observation delta separately from cumulative observational memory. After persistence and observation end markers succeed, the shared strategy lifecycle awaits one failure-isolated curator invocation per nonblank source-thread delta before normal reflection, so curation model latency is now part of every observation-cycle path that produces a delta. Resource-scoped cycles retain per-thread attribution and additionally run one curator call per contributing thread delta, serially. The curator receives the current observation delta, searches relevant existing knowledge through constrained read tools, and creates or reconciles durable knowledge through code-governed write tools. Provenance, timestamps, source identity, scope ceilings, versioning, activity, mention indexing, and semantic indexing remain code-owned rather than model-supplied. Reminder execution remains independent: reminder failure or no-signal cannot suppress curator delivery. Curator failure is reported and isolated after observation commit; it does not fail the committed observation or prevent normal reflection. ## Deliberate reliability boundaries Direct curator delivery is best-effort in this PR. Removing the durable worklist/cursor frontier means a failed or interrupted curator run is reported but is not automatically replayed. This PR does not introduce a replacement queue, scheduler, retry, cadence, or backoff mechanism. Core storage worklist/cursor APIs, including `knowledgeBySource`, remain for other consumers such as Factory's knowledge route; the Subconscious runtime stops using them for curation orchestration. The curator continues to require `organizationId` in request context, matching the prior capture path; a missing value is reported as a curator error after observation commit. The new `knowledge_create` tool preserves the existing node-first storage sequence: if the initial record append fails after node creation, the node may remain. No new cross-adapter transaction contract is introduced. ## Compatibility and cross-package cleanup - `@mastra/memory`: removes `capture`, `learn`, reflection-time Subconscious execution, worklist/cursor/cadence orchestration, and the public `Memory.runCuration()` method. There is no replacement manual curation entry point; Subconscious curation now runs only from successfully committed observations. - `@mastra/code-sdk`: removes Factory-only capture-pinning and cadence configuration. - `@mastra/factory`: removes phase-exit curation triggers that depended on `Memory.runCuration()`. - Simulator: removes capture arms, cadence/tail flushing, cursor accounting, and A/B comparison in favor of one direct-curator replay path. A minimal patch changeset is included for `@mastra/memory`, `@mastra/code-sdk`, and `@mastra/factory`. ## Verification - Focused Subconscious, lifecycle, replay, reminder, scope, activity, read-tool, and knowledge-write coverage: 11 files, 59 tests passed. - Full `@mastra/memory` unit suite: 63 files, 1,497 tests passed, 1 todo. - Memory simulator suite: 3 files, 46 tests passed. - `pnpm build:memory`: 15/15 tasks passed. - Memory typecheck and lint passed; lint reported 0 warnings and 0 errors. - `@mastra/factory` build, typecheck, full tests, and lint passed: 103 files, 1,980 tests passed, 6 skipped. - `@mastra/code-sdk` build, typecheck, and lint passed; the changed memory-construction suite passed 15/15 tests. - Red/green lifecycle coverage and a deterministic two-cycle replay over real in-memory storage verify direct delivery, reconciliation to one active fact, later cross-thread reminder retrieval, and no Subconscious worklist/cursor calls in the exercised paths. No live hosted-model run is claimed. - Removal greps, `git diff --check`, and clean-status checks passed. Two broader gates could not complete cleanly; details follow. ## Verification limitations The isolated `packages/memory/integration-tests` bootstrap could not install dependencies in this environment: the frozen install reports an override mismatch, and the required non-frozen retry cannot fetch private `@internal/ai-v6` from the npm registry. The integration tests therefore did not execute. The full Code SDK suite completed with 105 files and 1,524 tests passing, while four tests failed from startup timeout, `spawn E2BIG`, callback-port `EADDRINUSE`, and an MCP OAuth callback assertion. These failures were not used as evidence for this change; the changed SDK memory-construction suite passes independently. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## ELI5 The PR makes `curate` the single agent that turns new observations into knowledge. It removes older capture, learn, and automatic scheduling paths so curation runs independently after observations are saved. ## Changes - Refactor Subconscious ingestion around `remind` and `curate`. - Dispatch committed observation deltas directly to `SubconsciousCurateExtractor`. - Make curation asynchronous and failure-isolated. - Remove reflection-time execution, capture and learn agents, curation cadence, worklist orchestration, and cursor management. - Preserve reminder execution, provenance, scope enforcement, and code-owned indexing. - Add `knowledge_create` with scoped writes and provenance metadata. - Validate RFC 3339 timestamps for knowledge writes. - Harden observation framing against untrusted input. - Remove related public APIs and types from Memory, Code SDK, Factory, and simulator integrations. - Replace the simulator A/B workflow with direct observation replay. - Add tests for replay isolation, curation behavior, scope enforcement, provenance, failure handling, and input validation. - Add a patch changeset for `@mastra/memory`, `@mastra/code-sdk`, and `@mastra/factory`. ## Validation Focused tests, memory and simulator suites, builds, typechecks, linting, and relevant Factory and Code SDK checks passed. Memory integration tests were blocked by dependency installation issues. Four unrelated failures remained in the full Code SDK suite; changed SDK tests passed independently. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Jamie Nichols <CodingButter@users.noreply.github.com> Co-authored-by: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai> Co-authored-by: Jamie Nichols <10267569+CodingButter@users.noreply.github.com> Co-authored-by: Tyler Barnes <tylerdbarnes@gmail.com> |
||
|
|
eda942cb76 |
feat(memory): add an observational memory replay simulator (#22058)
## Summary Add local conversation simulation tooling to `@mastra/memory` for comparing experimental Subconscious capture and curation prompt changes against reconstructed real observation cycles. ## Root cause Evaluating prompt changes through deployments mixes prompt behavior with newly derived cycle boundaries and requires days of organic usage. Re-observing raw messages would also let the tested prompt move segmentation boundaries, making A/B results look meaningful while changing more than one variable. The simulator tests are colocated with the implementation they cover under `packages/memory/scripts/simulate`, which sits outside the package's original `src/**/*.test.ts` Vitest include. The package include is widened to `['src/**/*.test.ts', 'scripts/**/*.test.ts']` so they are discovered and run as part of the normal unit suite. ## What changed - Add local extract, replay, and A/B commands under `packages/memory/scripts/simulate`. - Reconstruct persisted observation cycles while excluding reflection heads from later generations. - Reject duplicate `generationCount` records as ambiguous instead of replaying them in unstable order. - Preserve deterministic ordering and report missing-generation warnings. - Replay through the real capture and curation path with optional production knowledge-resource scoping. - Isolate every arm in a fresh local database and refuse remote write targets. - Add an A/A control to expose model sampling noise. - Keep the simulator suites beside the implementation under `packages/memory/scripts/simulate`, named after the module each covers (`diff.test.ts`, `extract.test.ts`, `replay.test.ts`, `reconstruct.test.ts`, `drive.test.ts`, `drive.isolation.test.ts`), and widen the package Vitest include so they are still collected by the unit run. - Add simulator TypeScript checking to package verification. - Export the existing observation-boundary regex for simulator reuse without changing its pattern or runtime behavior. ## Design notes This is a bounded simulator, not a general PostgreSQL cloning tool. Generated target DDL does not preserve every PostgreSQL type, constraint, default, or precision detail. Worklist counting is capped, missing generations produce incomplete comparisons with warnings, and the content differ remains aggregate and positional in several cases. The replay command dynamically imports the built PostgreSQL adapter, so linked workspace packages must be built before running the simulator from a fresh checkout. The README documents the local PostgreSQL, pgvector, credentials, and build prerequisites. The widened Vitest include makes the simulator suites part of the default `@mastra/memory` unit run while leaving them colocated with the tooling. Normal production TypeScript scope remains unchanged because package TypeScript excludes `**/*.test.ts`; simulator checking remains owned by `tsconfig.scripts.json`. ## Safety - Source access is read-only. - Replay targets must use a literal IPv4 or IPv6 loopback address; hostnames and routing query parameters such as `host`, `hostaddr`, and `service` are rejected. - Extraction, standalone replay, and database recreation attest the live PostgreSQL server address as loopback before target initialization, writes, or destructive operations. - Extraction compares connected database identity and port, treating IPv4 and IPv6 loopback as the same local server, and refuses to overwrite the source database. - Each arm uses a separate fresh database. - No conversation or knowledge payload is committed. Tracking: COR-1167 ## Test plan From `packages/memory`: ```bash pnpm vitest run scripts/simulate --reporter=dot pnpm test:unit pnpm lint git diff --check ``` Results on the reviewed local head: - Simulator selection: 78 passed across 6 files. - Full memory unit suite: 1,602 passed and 1 todo across 68 files. - Lint: 0 warnings and 0 errors across 207 files. - Diff check: passed. - Simulator suites are discovered at their colocated path by the widened include; the full unit run collects 69 files with the suites in place. The production-source blast radius is limited to exporting the existing observation-boundary regex. Its pattern and runtime behavior are unchanged. No changeset is included: this adds repo-local tooling under `scripts/`, not a shipped package feature. --------- Co-authored-by: Jamie Nichols <CodingButter@users.noreply.github.com> Co-authored-by: Jamie Nichols <10267569+CodingButter@users.noreply.github.com> Co-authored-by: Mastra Code (openai/gpt-5.6-sol) <noreply@mastra.ai> |