mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
## Description
Adds `summary: true` listings that return only each run's status,
timestamp and parent, and `WorkflowsStorage.getRunOwnerships()` to read
many runs' ownership at once. Nothing in core calls them yet, so no
workflow behavior changes.
### Summary listings
`listWorkflowRuns({ summary: true })` now reduces each run's snapshot
to:
```ts
{ status?, timestamp?, parentWorkflow?: { workflowId, runId } }
```
`parentWorkflow` is present only when the stored parent has both IDs,
and is never `null`. A field the run doesn't have is left out. `context`
and every other snapshot field are never returned. Run fields outside
the snapshot (`runId`, `workflowName`, `resourceId`, `createdAt`,
`updatedAt`) are unchanged, and so is `total`.
| Where the snapshot is reduced | Stores |
|---|---|
| In the query (only the summary fields leave the database) | PostgreSQL
and LibSQL (already projected `{ status, timestamp }`, now add the
parent), MySQL, MSSQL, OracleDB, Spanner, DSQL, Cloudflare D1 and
Durable Objects, MongoDB (`project()`) |
| In the app, after reading the whole snapshot | DynamoDB, Redis,
Valkey, Upstash, Convex, in-memory |
This PR doesn't change the status filter. Among the SQL stores,
PostgreSQL (expression index on `snapshot ->> 'status'`), Spanner
(generated status column) and OracleDB (function index on `JSON_VALUE`)
filter on an index, and LibSQL storage v2 runs have an indexed `status`
column. MySQL, MSSQL, DSQL and Cloudflare D1 and Durable Objects extract
the status from each row's JSON.
MSSQL now caches whether the snapshot table has a `resourceId` column.
The non-summary `resourceId` filter uses the same cache, with the same
result as before.
### Batch ownership reads
`getRunOwnerships({ runIds })` returns a `Map` from run ID to the record
`getRunOwnership()` would return for that ID; runs that were never
claimed are left out. The default reads each run with
`getRunOwnership()`, at most 8 at a time, and stops starting new reads
once one fails. Overrides:
| Store | Round trips |
|---|---|
| PostgreSQL | One query for all IDs (one array parameter) |
| LibSQL | One query per 500 IDs |
| MySQL, MSSQL | One query per 500 IDs, joining the requested IDs so the
column's collation matches them as the single-run read does |
| MongoDB | One `find` per 1,000 IDs |
| DynamoDB | One strongly consistent `BatchGetItem` per 100 IDs,
unprocessed keys retried with backoff; IDs that differ only in case name
one item, as in the single-run read |
| Upstash | One pipeline request per 100 IDs |
| Redis, Valkey | One single-key script call per ID (works across
cluster hash slots), up to 500 in flight |
| In-memory | Reads every record in one pass |
| Convex, DSQL, Spanner, OracleDB, Cloudflare D1 and Durable Objects |
Base default |
### In-memory `updatedAt`
The in-memory store now moves a run's `updatedAt` on
`updateWorkflowResults()`, `updateWorkflowState()` and
`updateWorkflowRunHeader()`, as the database adapters already did. A
write a status guard rejects leaves it alone.
### Conformance
In `@internal/storage-test-utils`, run on every store with run fencing:
the summary listing shape for v1 and storage v2 runs, including a null
parent, a parent with one ID and a missing timestamp; `getRunOwnerships`
over 1,001 IDs with live, released, expired and unclaimed runs, each
record equal to the single-run read, plus IDs that differ from a claimed
one only by a trailing space or case (MySQL, MSSQL and DynamoDB match
those in the single-run read). The `updatedAt` cases run through
`createWorkflowsTests`. Valkey and Cloudflare Durable Objects don't run
that suite: Valkey writes through the same path as Redis, and Durable
Objects don't implement partial writes.
Checks run locally: `pnpm build:core`, core `check`, lint and `tsc` of
every touched store, and the conformance suite against in-memory,
LibSQL, PostgreSQL, MySQL, MSSQL, MongoDB (replica set), Redis, Valkey,
DynamoDB Local and Upstash (serverless-redis-http). MSSQL has 15
failures outside this PR's cases that fail the same way on `main`
locally. DSQL, Convex, Spanner, OracleDB and Cloudflare were not run
locally.
Stack 1 of 4; the recovery pass in the later PRs uses these reads.
## Related issue(s)
Part of #28712
## Type of change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [x] Performance improvement
- [x] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
**ELI5:** Workflow runs can now be listed in a smaller form that holds
only the status, time, and parent workflow. The PR also adds a way to
look up the owners of many runs in one call. Storage adapters return
only the fields that callers need.
## Overview
This PR adds two storage capabilities for workflow runs:
- `listWorkflowRuns({ summary: true })` returns a reduced snapshot for
each run. The snapshot contains `status` and `timestamp`. It adds
`parentWorkflow: { workflowId, runId }` only when both parent IDs are
non-empty strings.
- `WorkflowsStorage.getRunOwnerships({ runIds })` returns ownership
records for many runs. It returns a `Map` keyed by run ID and omits
unclaimed runs.
The in-memory store also changes. Successful writes refresh `updatedAt`.
A write rejected by a status guard leaves `updatedAt` unchanged.
The PR description states that core does not call these APIs yet.
Workflow behavior does not change in this PR.
## Changes by area
**Core (`@mastra/core`)**
- `storage/domains/workflows/base.ts`: Adds a default `getRunOwnerships`
implementation. It deduplicates run IDs and reads at most 8 records
concurrently through `getRunOwnership`. It omits unclaimed runs. A
failed read rejects the batch and stops further reads from starting.
- `storage/domains/workflows/inmemory.ts`: Adds `summary` support to
`listWorkflowRuns`. Summary reduction runs after filtering and
pagination. `updateWorkflowResults`, `updateWorkflowState`, and
`updateWorkflowRunHeader` refresh `updatedAt` after successful writes.
- `storage/types.ts`: Documents the fuller reduced-snapshot shape for
`summary`. It changes no runtime behavior.
**Store adapters**
Summary support (`summary: true`) is added in: Cloudflare D1, Cloudflare
Durable Objects, Convex, DSQL, DynamoDB, LibSQL, MongoDB, MSSQL, MySQL,
OracleDB, PostgreSQL, Redis, Spanner, Upstash, and Valkey.
Reduction happens in the database for these stores:
- SQL stores: PostgreSQL, LibSQL, MySQL, MSSQL, OracleDB, Spanner, DSQL
- Cloudflare D1 and Durable Objects
- MongoDB (projection)
Reduction happens in application code for these stores:
- Convex, DynamoDB, Redis, Upstash, Valkey
`getRunOwnerships` is added in: PostgreSQL, LibSQL, MySQL, MSSQL,
MongoDB, DynamoDB, Redis, Upstash, and Valkey. The batch strategy
differs by store:
- PostgreSQL: one query.
- LibSQL, MySQL, MSSQL: chunks of up to 500 IDs.
- MongoDB: batches of 1,000 IDs.
- DynamoDB: strongly consistent batch reads of up to 100 keys.
Unprocessed keys retry with exponential backoff.
- Upstash: pipeline batches of 100.
- Redis and Valkey: one single-key ownership script call per run. Calls
run in batches of up to 500.
Cloudflare D1 and Durable Objects include `resourceId` in summary
queries only when the table has that column. The column check is cached.
**Conformance test utilities (`stores/_test-utils`)**
- `createSummaryListingTests`: Covers summary shapes, parent metadata
handling, and comparison with full listings. It skips stores without
run-fencing support.
- `createUpdatedAtTests`: Covers `updatedAt` on partial writes. For
storage v2 run journals, it checks header and state updates, including
guard-rejected writes.
- Run-fencing tests: Add batch ownership read cases for live, released,
expired, and unclaimed runs. They also cover inputs larger than
MongoDB's 1,000-item chunk size, collation lookalike IDs, and empty
input. The `untilLapsed` helper polls for up to 5 seconds, checking
every 20 ms.
**Changesets**
Patch changesets cover the affected store packages. A minor changeset
covers `@mastra/core`.
## Scope
- Files changed: 30
- Lines added: 1,555
- Lines removed: 52
## Verification
The author reports these local results. They have not been checked
against the current head:
- `pnpm build:core`, core checks, lint, and TypeScript checks for
touched stores were run.
- Conformance runs were run against the listed stores. The author
reports 15 MSSQL failures outside this PR's test cases. The author
reports the same failures on `main`.
- Conformance tests were not run locally for DSQL, Convex, Spanner,
OracleDB, or Cloudflare.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
History
|
|
|---|---|---|
| .. | ||
| storage | ||
| cache.ts | ||
| index.test.ts | ||
| index.ts | ||
| integration.test.ts | ||