mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
feat(core,stores,server): Knowledge page link index and alpha review follow-ups (#29260)
## Summary Follow-ups to the Knowledge alpha (#29159, #29207, #29218, #29256), in one PR: **Page link index** - Knowledge keeps an index of `[[links]]` in page bodies. `countPageLinks` / `listPageLinks` on the Knowledge store return in and out links for a set of pages, gated by the reader's scopes like every other read; note `[[mentions]]` show up as `note` links. - Links resolve when a page is written through the tools, `upsertPage` or `removePage` (id, `scope#key/key` path, scope address, then page name in the page's own scope, then store-wide). Dangling links resolve once their target is created and follow renames, moves, merges and deletes. `knowledge.reindexLinks()` backfills pages written before this release. - Implemented in the in-memory, LibSQL, PostgreSQL, MySQL and MongoDB stores (additive table/collection, no schema bump). Refs over 512 characters stay plain text; the per-request link cap comes from core through the store loader. - Server routes `GET /knowledge/:knowledgeKey/link-counts` and `GET /knowledge/:knowledgeKey/pages/:pageId/links`, client-js methods, and FGA coverage (`knowledge:read` on the knowledge key, denied caller gets 403). **Review follow-ups** - Fenced code and headings parse like CommonMark (closing fence must match the opener; `## Use C#` keeps its `#`); reading the last part of a page that ends in a newline works. - Chunk embedding runs 4 at a time and semantic search stops at its candidate cap. - PostgreSQL and MongoDB refuse to remove a note that's already removed, like the other stores. - Search queries use at most 16 words. - Changeset for the `reindex()` removal scoped correctly. ## Child scopes filtered by reader in the store - `listChildScopes` takes an optional `read` scope and leaves out child scopes the reader can't see before applying the limit and cursor. New `countChildScopes` counts readable child scopes for many parents in one call. All five stores implement both. - `knowledge_list` uses them: it used to page through every child scope under a scope and drop the unreadable ones afterwards, and list each scope's child scopes just to count them. Under a scope with many private child scopes that meant loading all of them. - This changes cost only, not access: `knowledge_list` already removed unreadable scopes before returning anything, so agents see the same entries and counts. Without `read`, the host's full list (Studio, link resolution, `knowledge_read`'s restricted entries) is unchanged. ## Test plan - Core Knowledge + storage: 274 tests pass; tsc 0; oxlint 0/0; oxfmt clean. - Real stores (Docker): LibSQL 77, PostgreSQL 87, MySQL 78, MongoDB 81 pass. - Server Knowledge handlers + adapter FGA: 143 pass. client-js Knowledge: 6 pass. - New tests fail without their fixes (link hooks, host-write links, ref length, fences/headings, candidate cap, embed concurrency, double note removal, FGA route mapping, child scopes filtered before the limit, knowledge_list never receiving unreadable scopes). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 Knowledge pages can now keep track of links to other pages, including when those pages are added, renamed, moved, or removed. Readers can see only links they are allowed to access, and page listings avoid loading child scopes they cannot read. ## What changed - Added a page-link index for `[[links]]` across in-memory, LibSQL, PostgreSQL, MySQL, and MongoDB stores. `knowledge.reindexLinks()` can backfill the index for existing pages. - Added scope-filtered link counts and link-list APIs, server routes, and client-js methods. - Added child-scope filtering before pagination and child-scope counts for readable scopes. - Improved Markdown fence and heading parsing, page-range handling for trailing newlines, embedding concurrency, and semantic-search candidate limits. - Made repeated removal of an already-removed note return `KnowledgeNotFoundError` in PostgreSQL and MongoDB. Updated the `reindex()` changeset. ## Validation Author-reported test and tooling results are available in the PR objectives; they were not independently verified here. Review finding counts are unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: remy-builder[bot] <324360501+remy-builder[bot]@users.noreply.github.com>
This commit is contained in:
parent
ead727266d
commit
01524b22db
56 changed files with 2698 additions and 85 deletions
Loading…
Add table
Add a link
Reference in a new issue