mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
694 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
a5859ee2b9 |
chore: version packages (alpha) (#30430)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
cd2a326988 |
fix(libsql): stop background task writes stalling shutdown, and fix backend timing flakes (#30504)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
88b4a97042 |
refactor(factory): skip legacy migration tables on new databases (#30488)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
6c647e6fae |
refactor(factory): stop creating columns and tables nothing reads (#30495)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
1c52fdd034 |
chore: version packages (alpha) (#30261)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. ⚠️⚠️⚠️⚠️⚠️⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`. ⚠️⚠️⚠️⚠️⚠️⚠️ # Releases > The changelog information of each package has been omitted from this message, as the content exceeds the size limit. ## @mastra/factory@0.29.0-alpha.1 ## @mastra/core@1.84.0-alpha.1 ## @mastra/playground-ui@69.1.0-alpha.1 ## @mastra/client-js@1.60.1-alpha.1 ## @mastra/react@1.15.1-alpha.1 ## @mastra/deployer-cloud@1.84.0-alpha.1 ## @mastra/longmemeval@1.1.43-alpha.1 ## @mastra/opencode@0.1.43-alpha.1 ## create-factory@0.4.4-alpha.1 ## @mastra/code-sdk@1.19.1-alpha.1 ## mastracode@0.50.1-alpha.1 ## @mastra/arize@1.3.31-alpha.0 ## @mastra/arthur@0.4.31-alpha.0 ## @mastra/braintrust@1.3.28-alpha.0 ## @mastra/datadog@1.4.23-alpha.0 ## @mastra/deepeval@0.1.25-alpha.0 ## @mastra/laminar@1.3.33-alpha.0 ## @mastra/langfuse@1.6.7-alpha.0 ## @mastra/langsmith@1.4.4-alpha.0 ## @mastra/observability@1.24.1-alpha.0 ## @mastra/otel-bridge@1.5.23-alpha.0 ## @mastra/otel-exporter@1.5.3-alpha.0 ## @mastra/posthog@1.4.8-alpha.0 ## @mastra/sentry@1.2.33-alpha.0 ## @mastra/agent-builder@1.2.2-alpha.0 ## mastra@1.39.1-alpha.1 ## @mastra/connect@1.4.1-alpha.0 ## @mastra/deployer@1.84.0-alpha.1 ## @mastra/editor@0.17.2-alpha.0 ## @mastra/mcp-docs-server@1.3.14-alpha.1 ## @mastra/memory@1.41.2-alpha.0 ## @mastra/server@1.84.0-alpha.1 ## @mastra/google-cloud-pubsub@1.1.8-alpha.0 ## @mastra/redis-streams@0.8.2-alpha.0 ## @mastra/valkey-streams@0.6.3-alpha.0 ## @mastra/elysia@0.1.24-alpha.1 ## @mastra/express@1.5.28-alpha.1 ## @mastra/fastify@1.5.28-alpha.1 ## @mastra/hono@1.7.26-alpha.1 ## @mastra/koa@1.7.28-alpha.1 ## @mastra/nestjs@0.3.1-alpha.1 ## @mastra/next@0.2.42-alpha.1 ## @mastra/tanstack-start@0.2.42-alpha.1 ## @mastra/clickhouse@1.30.1-alpha.0 ## @mastra/cloudflare@1.8.1-alpha.0 ## @mastra/cloudflare-d1@1.5.1-alpha.0 ## @mastra/convex@1.9.1-alpha.0 ## @mastra/dsql@1.5.8-alpha.0 ## @mastra/dynamodb@1.5.2-alpha.0 ## @mastra/libsql@1.30.1-alpha.0 ## @mastra/mongodb@1.27.1-alpha.0 ## @mastra/mssql@1.11.2-alpha.0 ## @mastra/mysql@0.17.1-alpha.0 ## @mastra/oracledb@0.5.7-alpha.0 ## @mastra/pg@1.36.1-alpha.0 ## @mastra/redis@1.4.14-alpha.0 ## @mastra/spanner@1.12.1-alpha.0 ## @mastra/turso@0.1.22-alpha.0 ## @mastra/upstash@1.6.2-alpha.0 ## @mastra/valkey@0.2.10-alpha.0 ## @mastra/temporal@0.6.1-alpha.1 ## @mastra/daytona@0.11.6-alpha.0 ## create-mastra@1.39.1-alpha.1 ## @internal/playground@1.39.1-alpha.1 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
2788a6d959 |
feat(core): recover stopped workflow runs from a loop with one leader (#30281)
## Description
Restarts a default-engine workflow run whose instance crashed without
waiting for another instance to boot. `startWorkers()` now starts a
recovery loop on an instance that runs the orchestration worker, and one
instance among those that share a store runs its passes.
### The loop
Each pass applies the rule `restartAllActiveWorkflowRuns()` already uses
for default-engine runs: restart a `running` or `waiting` run once no
instance holds its run lease and `recovery.workflowGraceMs` passed since
the run or its lease last changed. The loop replaces the per-run recheck
timers the previous PR kept (`#scheduleWorkflowRecoveryRecheck`,
`#recheckWorkflowRun`); the durable-agent rechecks are unchanged.
The loop never touches evented runs. A redelivering broker recovers
them, and on other pubsubs the explicit `restartAllActiveWorkflowRuns()`
restarts them in the caller, as before. That call no longer joins a loop
pass for its evented part, so `startWorkers()` followed by the explicit
call still restarts evented runs on `EventEmitterPubSub`
(`evented/sleep-restart.test.ts` passes).
### Leader
One row, `::recovery-leader`, in the run-owners table, which gates the
loop's ticks only. Each tick of an instance tries a non-forced claim
until it gets the row, then keeps it: the lease helper renews it every
renew interval until `stopWorkers()` or `shutdown()` releases it. Only
the holder's loop lists runs; another instance's tick costs one refused
claim. When the leader dies, the row lapses after the lease TTL and the
next tick elsewhere takes it.
The leader saves listing cost only. Two restarts of one run are still
prevented by the run lease (`restart()` claims non-forced and throws
`WORKFLOW_RESTART_RUN_OWNED`).
An explicit `restartAllActiveWorkflowRuns()` never looks at the row. As
in the previous PR, each booting instance restarts its own stopped
default-engine runs (joining its own pass in flight), so another app's
workflows on the same store are recovered at its boot, and runs
recovered after a redeploy spread over the booting instances. Without
run fencing there is no leader and no loop, and the call restarts every
active run as before.
A failed probe, `getStore` or claim skips that tick and the loop keeps
going. A probe that answers "no fencing" stops the loop for good.
### Timings
- Interval: `recovery.workflowScanIntervalMs`, 60 s by default, each
wait up to 20 % shorter or longer, capped at 2^31 - 1 ms.
- A pass that saw a run in its grace period (and not live) moves the
next pass to that run's due time, no sooner than 1 s later and up to 20
% past it.
- A crashed instance's run is restarted about
`MASTRA_AGENT_THREAD_LEASE_TTL_MS` (15 s) plus `workflowGraceMs` (2 s)
after the crash, plus up to one interval for the next pass. When the
crashed instance was the leader, add up to one more interval for a
follower to take the row.
- `shutdown()` stops the loop when it starts, waits within
`drainTimeout` for the pass in flight and the restarts it handed out,
and releases the row in `stopWorkers()`.
### Config
```ts
const mastra = new Mastra({
recovery: { workflowScanIntervalMs: 30_000 },
});
```
`recovery.workflows: 'off'` turns the loop off. `workers: false`, or a
`MASTRA_WORKERS` filter without `orchestration`, runs no loop. An
instance that registers no default-engine workflow to recover never
claims the leader row.
### LibSQL thread state writes
`@mastra/libsql` thread state writes (`setState`, `deleteState`) now
take the client write lock the store's other writes take. Without it, on
a local database file, a thread state write that ran while the same
store held a write transaction open (a lease claim or renewal) failed
with `SQLITE_BUSY`; the leader's renewals made that much more likely.
### Known limits
- A process that calls `restartAllActiveWorkflowRuns()` but never
`startWorkers()` runs no loop and doesn't recheck the runs it left
alone. A run its boot call found live or in its grace period is left to
the loop of another instance, if any.
- A leader whose renewals fail for longer than the lease TTL can run
passes alongside the new leader for up to one renew interval after the
store answers again. The run lease still prevents a run from being
restarted twice.
- Between boots, the loop recovers only the workflows the leader
registers. Instances that register different workflows on one store get
loop recovery for the leader's workflows only, and the others' runs at
their next boot.
- A run the loop restarts executes on the leader.
- A `mastra worker` process calls `stopWorkers()` on `SIGTERM` and exits
without `shutdown()`, so its default-engine runs wait as long as a
crashed instance's.
- A run that another instance won between the pass's lease read and
`restart()` is looked at again only at the next pass, up to one interval
later.
- Rolling deploys are out of scope for this stack.
### Tests
New `recovery-loop.test.ts` (in-memory store, 600 ms lease TTL):
- a crashed leader's run is restarted by the follower with no boot,
within TTL + 2 intervals + 1.2 s
- instances started together run passes on one of them, and the leader
keeps the row while passes are further apart than the lease TTL
- the leader's `stopWorkers()` releases the row and the follower takes
over
- a tick that `stopWorkers()` overtook (on the first tick's store read,
or on a later tick's probe) claims no row, so another instance can lead
- a stopped run of a workflow only another app registers is restarted by
that app's boot call while a different instance leads, and the leader's
loop never lists that workflow
- next-pass timing table (no run, live run, run in grace, grace ending
within 1 s), timers unref'd
- no listing after `stopWorkers()`; `shutdown()` waits for a pass in
flight
- a loop on `startWorkers('orchestration')`
- the loop keeps going after a failed probe, leader claim, listing or
restart
- no listing and no claim with `workflows: 'off'` or without run fencing
The recheck cases of `restart-active-runs-ownership.test.ts` and
`restart-active-runs-rule.test.ts` are gone with the rechecks: the crash
case covers a crashed owner, the failure table covers transient errors,
and the existing shutdown and limiter cases cover draining and
`workflowConcurrency`. The LibSQL cross-process cases and the MCP
survivor cases boot like a server (`startWorkers()`, then the explicit
call) and recover through the loop.
Checks: core `src/workflows src/mastra src/agent/durable src/events`
(only the two `scheduler.test.ts` retry cases that fail outside UTC),
core typecheck, LibSQL cross-process, Postgres and LibSQL
`recovery-pass.test.ts`, MCP `server-tasks-background`.
## Related issue(s)
Part of #28712. Stack 4 of 4, on top of the recovery pass PR.
## Type of change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [x] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
ELI5: Sometimes a workflow run stops when its server crashes. This
change adds a background check. One server regularly looks for stopped
runs and restarts them, so the work can finish without a manual restart.
## Overview
This change adds a workflow recovery loop to `@mastra/core`.
`startWorkers()` starts the loop. Among instances that share a store,
only the leader runs default-engine passes. A pass restarts
default-engine runs in `running` or `waiting` status when:
- no instance holds the run lease, and
- the configured grace period has elapsed.
The loop excludes evented runs. Explicit
`restartAllActiveWorkflowRuns()` calls handle evented runs and are not
leader-gated.
The change also updates LibSQL thread-state writes. `setState` and
`deleteState` now run through `withClientWriteLock`.
## Key changes
- `packages/core/src/mastra/index.ts`
- Adds `workflowScanIntervalMs` to `MastraRecoveryConfig`. The default
is 60000 ms. Values are capped at `2 ** 31 - 1`. A value of `-1`
resolves to the default.
- `startWorkers()` starts the loop when the selected workers handle
workflow events and recovery is enabled.
- When workflows use run fencing, the loop elects a leader through a
renewable lease. If fencing is unsupported, the loop stops.
- A pass records the earliest due time for runs still in their grace
period. The loop can then scan sooner.
- Failed lease reads leave runs for a later pass. Restart failures are
tracked per engine.
- Per-run recheck, retry, and backoff logic is removed.
- `stopWorkers()` stops the loop and releases the leader lease. Shutdown
waits for an in-flight pass.
- Explicit `restartAllActiveWorkflowRuns()` calls share an in-flight
default-engine pass. Broker-redelivered evented runs stay broker-owned,
except process-local scheduled runs.
- `packages/core/src/workflows/run-ownership.ts`
- `repeatAfterSettled` accepts a function for the delay and an optional
`immediately` flag.
- Signature changes from `(tick, intervalMs: number)` to `(tick,
intervalMs: number | (() => number), options?)`.
- `stores/libsql/src/storage/domains/thread-state/index.ts`
- `setState` and `deleteState` use `withClientWriteLock`. SQL,
arguments, and error handling are unchanged.
- Tests
- Adds `packages/core/src/mastra/recovery-loop.test.ts`. It covers
leadership, scheduling, shutdown, and failure handling. The file is
added to `vitest.isolated-tests.ts`.
- Removes three tests from `restart-active-runs-ownership.test.ts`.
- In `restart-active-runs-rule.test.ts`, removes one test and rewrites
the lease-read failure test to use the loop.
- Updates the recovery-defaults test, the MCP background server test,
and two LibSQL cross-process tests for the new scan interval.
- Adds a LibSQL thread-state test. It holds a write transaction open
during `setState` and `deleteState`.
- Documentation and changesets
- Updates the workers, server-adapter, workflow overview, and
`mastra-class` reference pages. Adds a "Recovery loop" section to the
workflow overview.
- Corrects the crash-recovery text in the workers page.
- Adds changesets for `@mastra/core` and `@mastra/libsql`.
## Public API changes
- Adds `workflowScanIntervalMs?: number` to `MastraRecoveryConfig`.
- Changes the signature of `repeatAfterSettled` in `run-ownership.ts`.
## Scope
- 19 files changed. 1028 lines added and 379 removed.
## Verification status
- Review findings: not available. No current review findings were
supplied.
- Author-reported checks: the PR objectives list core tests, core
typecheck, LibSQL cross-process tests, Postgres and LibSQL
`recovery-pass.test.ts`, and MCP `server-tasks-background`. These are
author reports. Results on the current head were not available for this
summary.
- Known failures: the author reports that two `scheduler.test.ts` retry
cases fail outside UTC.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
99ec3bd582 |
fix(core): leave evented workflow runs to a redelivering broker and restart only stopped runs (#30279)
## Description This PR stops `restartAllActiveWorkflowRuns()` from restarting evented runs that a redelivering broker (Redis Streams, Valkey Streams, Google Cloud Pub/Sub) already recovers. It also restarts a default-engine run only once no instance holds its run lease and a grace period (2 s by default) has passed. Before, every booting instance restarted every active evented run, so a step interrupted by a crash ran once per booting pod (#28712). The broker already recovers those runs: Redis and Valkey Streams reclaim an unacked message once it has been idle for `reclaimIdleMs`, and Google Cloud Pub/Sub redelivers once the ack deadline passes. ### Evented runs on a broker `PubSub` gets a `redeliversUnacked` getter (default `false`). Redis and Valkey Streams return `true` while the reclaim loop runs (`reclaimIntervalMs > 0`), Google Cloud Pub/Sub returns `true`, `CachingPubSub` forwards to its inner pubsub, and the in-process `EventEmitterPubSub` and the Unix socket pubsub return `false`. When the pubsub (read through `__rawBus()`) redelivers, recovery never lists, reads or restarts an evented run. An evented workflow without declarative schedules costs no listing at all. Evented runs on any other pubsub (`EventEmitterPubSub`, Unix socket, or a third-party pubsub that doesn't override the getter) are restarted at once by the explicit call, as before. ### Scheduled runs The `mastra.pubsub` proxy publishes every workflow event of a scheduled run (`sched_wf_*`, plus the notification dispatcher's row) `localOnly`, so the broker never holds it. Recovery still restarts those runs: an evented workflow with declarative schedules gets one summary listing per status, and only its process-local scheduled runs are kept. The proxy and recovery share one predicate, `isProcessLocalScheduledRunId`. ### Default engine One recovery rule: a `running` or `waiting` run is restarted when its run lease isn't live and `recovery.workflowGraceMs` (new, 2,000 ms by default for every pubsub) has passed since the later of the lease's last expiry and the run's last write. The pass lists summaries (`listWorkflowRuns({ status, perPage: false, summary: true })` per status), reads leases in batches of 100 with `getRunOwnerships`, re-reads a run's lease once its restart gets a slot, and runs every restart on one instance-wide limiter (`recovery.workflowConcurrency`). A run left live or in its grace period gets a per-run recheck timer, as on main. Overlapping calls share one pass. `shutdown()` waits, within `drainTimeout`, for a pass still reading the store and for the restarts it handed out. Without run fencing every active run is restarted, as before. ### Nested resume A default-engine nested run resumed directly now holds its parent's run lease while it runs and hands it to the parent's resume, so recovery doesn't restart the parent meanwhile. On a store without compare-and-set, where the parent stays `suspended`, the nested run holds its own lease instead. Also: `renewWorkflowLease` waits for each renewal to settle before arming the next one (a self-rearming `setTimeout` instead of `setInterval`), and `EventEmitterPubSub`'s JSDoc and the docs say it's in-process only. ### Repro Postgres + Valkey Streams with default reclaim settings (`reclaimIdleMs` 60 s, `reclaimIntervalMs` 30 s). Instance A starts a run whose step records an execution and then hangs; A is killed with `SIGKILL` mid-step. Then 2 or 4 fresh instances boot and all call `restartAllActiveWorkflowRuns()` at once. "At boot" counts step executions caused by those calls; "total" counts every execution after the kill until the run succeeds; "rerun after" is seconds from the kill to the first new execution. 3 trials per row, identical results within each row except for timing. | Build | Engine | Booting pods | Executions at boot | Executions total | Rerun after | |---|---|---|---|---|---| | Base (this PR's base branch) | evented | 2 | 2 | 2 | 0.1 s | | Base | evented | 4 | 4 | 4 | 0.1 s | | Base | default | 2 | 0 | 1 | 15.1 to 16.6 s | | Base | default | 4 | 0 | 1 | 15.0 to 16.9 s | | This PR | evented | 2 | 0 | 1 | 60.1 to 60.2 s | | This PR | evented | 4 | 0 | 1 | 60.1 to 60.2 s | | This PR | default | 2 | 0 | 1 | 17.0 to 19.1 s | | This PR | default | 4 | 0 | 1 | 17.2 to 21.0 s | On this PR the evented rerun is the broker's reclaim: one execution on one pod, about 60 s after the kill. The default-engine rerun comes once A's lease (15 s TTL) lapsed plus the 2 s grace and up to 5 s of recheck jitter. ### Benchmark 3,000 active default-engine runs that another instance holds, among 30,000 finished runs, two workflows. "Before" is a copy of main's sweep run in the same process (full listing, then `createRun()` + `restart()` per run, 5 at a time); "after" is `restartAllActiveWorkflowRuns()` on this PR. Median of 5 calls. | Store | Before | After | Listing payload before | Listing payload after | |---|---|---|---|---| | Postgres | 1,773 ms | 30 ms | 1.12 MB | 0.60 MB | | LibSQL | 627 ms | 44 ms | 1.12 MB | 0.60 MB | The benchmark's snapshots are nearly empty, so the payload gap here is small; with real step results it grows with snapshot size. ### Known limits On a redelivering broker these evented runs stay `running` until someone calls `restart()` on them. Main's boot restart recovered them; stack 2 takes them over. - A sleeping evented run whose instance exits, after a crash or during a normal deploy. The sleep acks its delivery and keeps a `setTimeout` in the process that started it, so any exit of that process loses the timer. - A step whose message Redis or Valkey Streams trimmed. Both trim with `MAXLEN ~ maxStreamLength` (10,000 by default), which also drops entries that are still pending. A step that is in flight while that many newer events are published loses its message if its instance then exits. Deployments with long steps and a high event volume should raise `maxStreamLength`. - A message the broker dropped after `maxDeliveryAttempts` or at `inFlightTimeoutMs`. - A failure publish that gave up after its retries (the worker acks the message). - A run whose `running` snapshot was saved but whose start event never reached the broker, because its process stopped between the two. - A Redis or Valkey instance that restarted without persistence. - A scheduled evented run that was in flight when a deploy stopped its process and removed the workflow's schedule from code. Recovery doesn't list it, so it stays `running` until `restart()`. #28712 is fixed for evented runs on a redelivering broker. These are still restarted by every booting instance, as before: - scheduled evented runs (`sched_wf_*`), whose events never leave their process - evented runs on the in-process `EventEmitterPubSub` or the Unix socket pubsub. Several instances sharing one storage with these pubsubs aren't supported for evented workflows; the docs now say a multi-instance evented deployment needs a broker. With Redis or Valkey Streams, `reclaimIdleMs` must stay at 30 s or more: a message reclaimed while the dead worker's 30 s step fence is still held is dropped as a duplicate. The getter's JSDoc, the adapter docs and the changeset state it. Rolling deploys are out of scope. Stack 3 of 4: storage projections and batch ownership reads, then the finished-step check, then this PR, then the recovery loop with a leader. ## Related issue(s) Fixes #28712 ## Type of change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Code refactoring - [x] Performance improvement - [x] Test update ## Checklist - [x] I have linked the related issue(s) in the description above - [x] I have made corresponding changes to the documentation (if applicable) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have addressed all Coderabbit comments on this PR <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ELI5: When a server crashes in the middle of a workflow, Mastra can restart that workflow. Before this change, some workflows ran their steps again on every new server that started, so a step could run many times. This change lets the message broker handle the restart for those workflows, so each step runs only once. Other workflows now wait for a short grace period before they restart. ## Summary This pull request changes how workflow recovery restarts runs after a crash or a boot. It targets duplicate step execution in two cases: - **Evented runs on brokers that redeliver unacknowledged messages.** Recovery skips these runs. The broker redelivers the unacknowledged step events instead. - **Default-engine runs.** Recovery restarts a run only after no live lease remains and a grace period has passed. ### Behavior changes - Google Cloud Pub/Sub, Redis Streams, and Valkey Streams report `redeliversUnacked` as `true` when their reclaim or redelivery path is active. Redis Streams and Valkey Streams report `true` only when `reclaimIntervalMs` is above 0. - `PubSub` adds a `redeliversUnacked` getter that defaults to `false`. `CachingPubSub`, `EventEmitterPubSub`, and `UnixSocketPubSub` expose or declare this capability. - Recovery leaves evented runs to the broker when `redeliversUnacked` is `true`. Scheduled evented runs restart on boot. Other evented runs restart immediately when the broker does not redeliver. - Default-engine recovery applies `recovery.workflowGraceMs`. The default is 2,000 ms. Negative or non-finite values use the default. A run restarts only after its lease is no longer live and the grace period has elapsed since the later of its last update and lease expiry. Runs that are not yet eligible are checked again later. If the storage adapter does not support run fencing, active runs restart without the grace rule. - Recovery lists active run summaries, reads ownership in batches, and rechecks ownership before each restart. - Restarts use a shared concurrency limiter (`createConcurrencyLimiter`). A restarted run holds its slot until it finishes. - Overlapping recovery passes are deduplicated. Shutdown waits for an in-flight pass. - Recovery skips nested runs whose active parent drives them. - Lease renewal runs sequentially (`renewWorkflowLease`, `repeatAfterSettled`). A new renewal starts only after the previous one settles. - Nested resume coordinates parent and child leases. A failed lease claim rejects the resume and leaves the runs suspended. - Run-claim operations return whether a status transition occurred. They return `false` when storage is absent, running snapshots are not persisted, or compare-and-set is not supported. ### Public API changes - `MastraRecoveryConfig.workflowGraceMs?: number` (default 2,000 ms). - `PubSub.redeliversUnacked` getter (default `false`). - `redeliversUnacked` getters on `GoogleCloudPubSub`, `RedisStreamsPubSub`, `ValkeyStreamsPubSub`, and `UnixSocketPubSub`. - `ConcurrencyLimiter` interface and `createConcurrencyLimiter(limit)` function in `run-with-concurrency.ts`. - `renewWorkflowLease`, `repeatAfterSettled`, and `WORKFLOW_RUN_LEASE_RENEW_INTERVAL_MS` in `run-ownership.ts`. - `Run._resume(params)` accepts an optional `takeHeldOwnership` callback. ### Documentation - Crash-recovery and worker docs describe broker reassignment and multi-instance limits. - `EventEmitterPubSub` documents that delivery is process-local. - Redis Streams and Valkey Streams docs recommend `reclaimIdleMs` of at least 30 seconds. Reclaiming earlier can cause duplicate-dropped step events. - The recovery reference documents `workflowGraceMs`. ### Tests and test utilities - New recovery tests cover exit cases (crash and shutdown), recovery reads, rules, ownership during nested resumes, and lease renewal. - Shared test helpers add `sharedBroker`, `killableProcess`, `interceptStarts`, `persistActiveRun`, and `describeRecoveryExitCases`. - Existing tests set `workflowGraceMs: 0` where they need immediate recovery. - Redis Streams and Valkey Streams tests check `redeliversUnacked` against `reclaimIntervalMs`. - LibSQL and Postgres recovery-pass suites run the shared exit cases. ### Scope and limits - The change addresses issue `#28712` for evented runs on redelivering brokers. Evented runs on in-process or Unix socket pubsubs still restart at boot. The PR states that multi-instance evented deployments need a broker. - The PR lists unrecovered cases: sleeping evented runs, trimmed or dropped messages, failed publishes, missing start events, non-persistent Redis or Valkey restarts, and scheduled runs whose schedules were removed. Rolling deploys are out of scope. - Reported benchmark results (median recovery time on 3,000 active and 30,000 finished runs: 1,773 ms to 30 ms on Postgres, 627 ms to 44 ms on LibSQL) and repro results are author-reported. They were not independently verified. - CI and test results: not available. - Review findings and severity counts: not available. ### Size - 41 files changed, 2,594 lines added, 209 lines removed (from the pull request diff). <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
8559e74767 |
feat(storage): workflow run summary listings and batched run ownership reads (#30274)
## Description
Adds `summary: true` listings that return only each run's status,
timestamp and parent, and `WorkflowsStorage.getRunOwnerships()` to read
many runs' ownership at once. Nothing in core calls them yet, so no
workflow behavior changes.
### Summary listings
`listWorkflowRuns({ summary: true })` now reduces each run's snapshot
to:
```ts
{ status?, timestamp?, parentWorkflow?: { workflowId, runId } }
```
`parentWorkflow` is present only when the stored parent has both IDs,
and is never `null`. A field the run doesn't have is left out. `context`
and every other snapshot field are never returned. Run fields outside
the snapshot (`runId`, `workflowName`, `resourceId`, `createdAt`,
`updatedAt`) are unchanged, and so is `total`.
| Where the snapshot is reduced | Stores |
|---|---|
| In the query (only the summary fields leave the database) | PostgreSQL
and LibSQL (already projected `{ status, timestamp }`, now add the
parent), MySQL, MSSQL, OracleDB, Spanner, DSQL, Cloudflare D1 and
Durable Objects, MongoDB (`project()`) |
| In the app, after reading the whole snapshot | DynamoDB, Redis,
Valkey, Upstash, Convex, in-memory |
This PR doesn't change the status filter. Among the SQL stores,
PostgreSQL (expression index on `snapshot ->> 'status'`), Spanner
(generated status column) and OracleDB (function index on `JSON_VALUE`)
filter on an index, and LibSQL storage v2 runs have an indexed `status`
column. MySQL, MSSQL, DSQL and Cloudflare D1 and Durable Objects extract
the status from each row's JSON.
MSSQL now caches whether the snapshot table has a `resourceId` column.
The non-summary `resourceId` filter uses the same cache, with the same
result as before.
### Batch ownership reads
`getRunOwnerships({ runIds })` returns a `Map` from run ID to the record
`getRunOwnership()` would return for that ID; runs that were never
claimed are left out. The default reads each run with
`getRunOwnership()`, at most 8 at a time, and stops starting new reads
once one fails. Overrides:
| Store | Round trips |
|---|---|
| PostgreSQL | One query for all IDs (one array parameter) |
| LibSQL | One query per 500 IDs |
| MySQL, MSSQL | One query per 500 IDs, joining the requested IDs so the
column's collation matches them as the single-run read does |
| MongoDB | One `find` per 1,000 IDs |
| DynamoDB | One strongly consistent `BatchGetItem` per 100 IDs,
unprocessed keys retried with backoff; IDs that differ only in case name
one item, as in the single-run read |
| Upstash | One pipeline request per 100 IDs |
| Redis, Valkey | One single-key script call per ID (works across
cluster hash slots), up to 500 in flight |
| In-memory | Reads every record in one pass |
| Convex, DSQL, Spanner, OracleDB, Cloudflare D1 and Durable Objects |
Base default |
### In-memory `updatedAt`
The in-memory store now moves a run's `updatedAt` on
`updateWorkflowResults()`, `updateWorkflowState()` and
`updateWorkflowRunHeader()`, as the database adapters already did. A
write a status guard rejects leaves it alone.
### Conformance
In `@internal/storage-test-utils`, run on every store with run fencing:
the summary listing shape for v1 and storage v2 runs, including a null
parent, a parent with one ID and a missing timestamp; `getRunOwnerships`
over 1,001 IDs with live, released, expired and unclaimed runs, each
record equal to the single-run read, plus IDs that differ from a claimed
one only by a trailing space or case (MySQL, MSSQL and DynamoDB match
those in the single-run read). The `updatedAt` cases run through
`createWorkflowsTests`. Valkey and Cloudflare Durable Objects don't run
that suite: Valkey writes through the same path as Redis, and Durable
Objects don't implement partial writes.
Checks run locally: `pnpm build:core`, core `check`, lint and `tsc` of
every touched store, and the conformance suite against in-memory,
LibSQL, PostgreSQL, MySQL, MSSQL, MongoDB (replica set), Redis, Valkey,
DynamoDB Local and Upstash (serverless-redis-http). MSSQL has 15
failures outside this PR's cases that fail the same way on `main`
locally. DSQL, Convex, Spanner, OracleDB and Cloudflare were not run
locally.
Stack 1 of 4; the recovery pass in the later PRs uses these reads.
## Related issue(s)
Part of #28712
## Type of change
- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [x] Performance improvement
- [x] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
**ELI5:** Workflow runs can now be listed in a smaller form that holds
only the status, time, and parent workflow. The PR also adds a way to
look up the owners of many runs in one call. Storage adapters return
only the fields that callers need.
## Overview
This PR adds two storage capabilities for workflow runs:
- `listWorkflowRuns({ summary: true })` returns a reduced snapshot for
each run. The snapshot contains `status` and `timestamp`. It adds
`parentWorkflow: { workflowId, runId }` only when both parent IDs are
non-empty strings.
- `WorkflowsStorage.getRunOwnerships({ runIds })` returns ownership
records for many runs. It returns a `Map` keyed by run ID and omits
unclaimed runs.
The in-memory store also changes. Successful writes refresh `updatedAt`.
A write rejected by a status guard leaves `updatedAt` unchanged.
The PR description states that core does not call these APIs yet.
Workflow behavior does not change in this PR.
## Changes by area
**Core (`@mastra/core`)**
- `storage/domains/workflows/base.ts`: Adds a default `getRunOwnerships`
implementation. It deduplicates run IDs and reads at most 8 records
concurrently through `getRunOwnership`. It omits unclaimed runs. A
failed read rejects the batch and stops further reads from starting.
- `storage/domains/workflows/inmemory.ts`: Adds `summary` support to
`listWorkflowRuns`. Summary reduction runs after filtering and
pagination. `updateWorkflowResults`, `updateWorkflowState`, and
`updateWorkflowRunHeader` refresh `updatedAt` after successful writes.
- `storage/types.ts`: Documents the fuller reduced-snapshot shape for
`summary`. It changes no runtime behavior.
**Store adapters**
Summary support (`summary: true`) is added in: Cloudflare D1, Cloudflare
Durable Objects, Convex, DSQL, DynamoDB, LibSQL, MongoDB, MSSQL, MySQL,
OracleDB, PostgreSQL, Redis, Spanner, Upstash, and Valkey.
Reduction happens in the database for these stores:
- SQL stores: PostgreSQL, LibSQL, MySQL, MSSQL, OracleDB, Spanner, DSQL
- Cloudflare D1 and Durable Objects
- MongoDB (projection)
Reduction happens in application code for these stores:
- Convex, DynamoDB, Redis, Upstash, Valkey
`getRunOwnerships` is added in: PostgreSQL, LibSQL, MySQL, MSSQL,
MongoDB, DynamoDB, Redis, Upstash, and Valkey. The batch strategy
differs by store:
- PostgreSQL: one query.
- LibSQL, MySQL, MSSQL: chunks of up to 500 IDs.
- MongoDB: batches of 1,000 IDs.
- DynamoDB: strongly consistent batch reads of up to 100 keys.
Unprocessed keys retry with exponential backoff.
- Upstash: pipeline batches of 100.
- Redis and Valkey: one single-key ownership script call per run. Calls
run in batches of up to 500.
Cloudflare D1 and Durable Objects include `resourceId` in summary
queries only when the table has that column. The column check is cached.
**Conformance test utilities (`stores/_test-utils`)**
- `createSummaryListingTests`: Covers summary shapes, parent metadata
handling, and comparison with full listings. It skips stores without
run-fencing support.
- `createUpdatedAtTests`: Covers `updatedAt` on partial writes. For
storage v2 run journals, it checks header and state updates, including
guard-rejected writes.
- Run-fencing tests: Add batch ownership read cases for live, released,
expired, and unclaimed runs. They also cover inputs larger than
MongoDB's 1,000-item chunk size, collation lookalike IDs, and empty
input. The `untilLapsed` helper polls for up to 5 seconds, checking
every 20 ms.
**Changesets**
Patch changesets cover the affected store packages. A minor changeset
covers `@mastra/core`.
## Scope
- Files changed: 30
- Lines added: 1,555
- Lines removed: 52
## Verification
The author reports these local results. They have not been checked
against the current head:
- `pnpm build:core`, core checks, lint, and TypeScript checks for
touched stores were run.
- Conformance runs were run against the listed stores. The author
reports 15 MSSQL failures outside this PR's test cases. The author
reports the same failures on `main`.
- Conformance tests were not run locally for DSQL, Convex, Spanner,
OracleDB, or Cloudflare.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
956143f6bd |
test(libsql): guard live-run boot recovery (#30264)
Adds a two-process regression test for boot recovery with both durable
and evented agents. A peer blocks in its first tool while a second
Mastra instance opens the same LibSQL database, starts workers,
discovers the running agent, and calls `recoverAllDurableAgents()`. The
test checks that recovery skips the healthy owner, makes zero model
calls in the booting replica, and lets the original peer commit all
three steps. Reuses the T81 peer and IPC gates with no added sleeps or
skipped engine cells.
Run locally with `pnpm build:core`, then `pnpm --filter @mastra/libsql
exec vitest run
src/__tests__/cross-process/xproc-boot-recovery.test.ts`.
Three consecutive local runs passed, followed by another passing run
after restoring the red control:
```text
durable evented result
Green run 1 1901ms 1632ms 2 passed (2)
Green run 2 3186ms 1605ms 2 passed (2)
Green run 3 1399ms 2423ms 2 passed (2)
Restored green 1451ms 1798ms 2 passed (2)
```
For the red control, temporarily changed the recovery branch in
`packages/core/src/agent/durable/execution-fence.ts` from
`backend.tryRecover(executionId, args.untrackedRun)` to
`backend.tryTakeover(executionId)`, rebuilt core, and ran the same test.
Both engines failed on the intended assertions:
```text
Test Files 1 failed (1)
Tests 2 failed (2)
Both durable and evented:
Expected: { agents: 1, recovered: 0, succeeded: 0, failed: 0, skipped: 1 }
Received: { agents: 1, recovered: 1, succeeded: 1, failed: 0, skipped: 0 }
boot recovery must not call the model: expected 4 to be +0
Original peer commits: expected [1] to deeply equal [1, 2, 3]
the peer finishes without work from the booting replica: expected 4 to be +0
```
The bypass was reverted and core rebuilt before the final green run.
There are no production-code changes and no changeset. Formatting,
type-aware lint, and `git diff --check` pass. `pnpm --filter
@mastra/libsql exec tsc --noEmit` is blocked by five existing errors in
unrelated fixtures: one in `dead-claim-pg-agent.ts:10` and four in
`foreach-billing-workflow.ts`.
Fixes COR-1492
Ref COR-1365
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
ELI5: The computer runs a task on one program. A second copy of the
program starts and must not repeat that task. This PR adds a test that
checks this rule. The second copy must skip the task, and the first copy
must finish it.
## Summary
This PR adds one new test file:
`stores/libsql/src/__tests__/cross-process/xproc-boot-recovery.test.ts`
(113 lines added, 0 removed). It contains no production-code changes and
no changeset.
The file defines a cross-process regression test for boot recovery. It
runs once for `durable` and once for `evented` agents. Each run has the
same flow:
1. **Setup.** `beforeEach` sets `MASTRA_AGENT_THREAD_LEASE_TTL_MS` to
`1500`. It then creates the test environment with workers enabled.
2. **Start the peer.** The test spawns a peer process from
`fixtures/t81-peer.ts`. It waits until the peer reports `blocked` at
step 1.
3. **Boot a replica.** The test opens the same LibSQL database from a
second `Mastra` instance. The replica registers a step agent with three
steps. It uses `MockMemory` and workers. Recovery is set to
`durableAgents: 'auto'`.
4. **Discover the run.** The test calls `listActiveRuns()`. It expects
exactly one active run with the peer's `runId`.
5. **Run recovery.** The test calls `recoverAllDurableAgents()`. It
expects `{ agents: 1, recovered: 0, succeeded: 0, failed: 0, skipped: 1
}`. It uses soft assertions for this result.
6. **Check model calls.** The test expects the replica's model call
count to be `0`.
7. **Release the peer.** The test sends `release` and reads the peer
result. It expects no errors. It expects `commit` events for steps `1,
2, 3`. It expects the peer to exit with code `0` and no signal.
8. **Teardown.** `afterEach` releases and kills the peer. It then shuts
down Mastra, closes storage, cleans up the environment, and restores
stubbed environment variables.
The test links to COR-1492 and COR-1436 in a code comment. The comment
says a running snapshot can belong to a healthy replica. Recovery must
find that run without running its tools again.
## Verification
The PR author reports:
- Four passing local runs of `pnpm --filter @mastra/libsql exec vitest
run src/__tests__/cross-process/xproc-boot-recovery.test.ts`, after
`pnpm build:core`.
- A temporary red control caused both engine tests to fail on the
intended assertions. The control was then reverted.
- Formatting, type-aware lint, and `git diff --check` pass.
- `pnpm --filter @mastra/libsql exec tsc --noEmit` reports five errors
in unrelated fixtures.
These results come from the author's report. This review did not run the
tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: review <review@local>
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
|
||
|
|
d60ba71be0 | chore: version - exit prerelease mode | ||
|
|
9e82af0237 |
chore: version packages (alpha) (#30074)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
b0ef22e7bf |
feat(schedules): Add bounded one-off schedules (#25131)
> Opened by @abhiaiyer91 Refs #24610 ## Summary Schedules can now fire once or stop after a given time, so you no longer need to pin a cron expression to a date and clean it up yourself. - **New `completed` status**: the scheduler and timing reconciliation set it when no automatic fires remain. It's written in the same atomic `nextFireAt` check-and-set used to claim a run, so a schedule still fires only once across instances. A cron with no future occurrence left also moves to `completed` instead of erroring. `pause`/`resume` on a completed schedule return 409. Changing `runAt`/`endAt` reactivates it only when the resulting timing can still fire. `list()` hides completed schedules unless you pass `status: 'completed'`. - **Declarative workflow schedules**: `createWorkflow({ schedule: { runAt } })` and `{ cron, endAt }` are supported. Past times are accepted so redeploys don't fail, and a completed schedule isn't re-armed unless its timing changes. - **Trigger kind**: one-off fires report `trigger.kind: 'once'`. - `run(id)` doesn't complete a one-off schedule. ## Surfaces - **Core**: types, validation, scheduler claim logic, create/update/pause/resume/list, declarative sync. - **Storage**: new nullable `run_at`/`end_at` columns for libsql, pg, mysql, mongodb, spanner, convex and in-memory. Existing columns are untouched: one-off schedules store `cron` as `''`. `updateScheduleNextFire` takes an optional `newStatus`. - **Server/client-js/CLI**: request bodies accept `runAt`/`endAt` (one-of rule checked in the schema). Responses include the new fields and `completed`. Route types and CLI route metadata regenerated. - **Studio**: shows the run time for one-off schedules, shows the `endAt` bound, adds a `completed` badge and hides pause/resume for completed schedules. - **Docs + changeset.** ## Not in this PR (remaining #24610 work) - ISO 8601 string inputs for `runAt`/`endAt` (numbers and `Date` are accepted today). - The template `schedule_once` tool. ## Test plan - [x] core: schedules, scheduler, cron, schedule-config and scheduler integration tests (260 passing), clean `tsc` - [x] Shared adapter suite gained tests for `runAt`/`endAt` round-trip and atomic status CAS. Passing on libsql, pg, mysql and mongodb (Docker), plus convex server tests. **Not run on Spanner (no emulator available); typecheck only.** - [x] server schedules handler tests (51 passing, including one-off/bounded creation and body validation) - [x] client-js and playground typecheck clean; playground schedules tests passing <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 Schedules can now run once at a chosen time or repeat until a chosen end time. They complete when their planned runs are over, and changing their timing can make them active again. ## Summary - Added `runAt` for one-off agent and workflow schedules. API-created schedules require a future `runAt`; declarative workflow schedules can register past times as `completed`. - Added `endAt` for cron schedules. A schedule becomes `completed` when its next run would occur after `endAt`, or when no future run remains. - Completed schedules cannot be paused or resumed. Timing changes can reactivate a completed schedule when a future run remains. `list()` hides completed schedules unless callers filter with `status: 'completed'`. - Resuming a paused one-off schedule after its `runAt` has passed allows it to fire on the next scheduler tick. Calling `run(id)` manually does not complete a one-off schedule. - Added the `once` trigger kind for scheduled one-off agent fires. Manual runs remain `manual`. - Declarative workflow schedules preserve completed status across redeploys when timing is unchanged. Timing changes can re-arm them. - Updated schedule APIs, storage adapters, Studio, and documentation. Storage persists `runAt` and `endAt` and supports atomic status updates. ## Checks Current-head CI, build, and test results: not available. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Abhi Aiyer <abhiaiyer91@gmail.com> Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com> Co-authored-by: NikAiyer <nick.aiyer@gmail.com> Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com> Co-authored-by: Ehindero Israel <iehindero419@stu.ui.edu.ng> |
||
|
|
779d2875b1 |
chore: version packages (alpha) (#29912)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. ⚠️⚠️⚠️⚠️⚠️⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`. ⚠️⚠️⚠️⚠️⚠️⚠️ # Releases > The changelog information of each package has been omitted from this message, as the content exceeds the size limit. ## @mastra/auth-studio@1.4.0-alpha.1 ## @mastra/factory@0.28.0-alpha.5 ## @mastra/mcp-apps-components@0.1.0-alpha.5 ## @mastra/playground-ui@69.0.0-alpha.6 ## @mastra/client-js@1.60.0-alpha.5 ## @mastra/react@1.15.0-alpha.5 ## @mastra/deployer-cloud@1.83.0-alpha.5 ## @mastra/longmemeval@1.1.42-alpha.5 ## @mastra/opencode@0.1.42-alpha.5 ## create-factory@0.4.3-alpha.5 ## @mastra/code-sdk@1.19.0-alpha.5 ## mastracode@0.50.0-alpha.5 ## @mastra/agent-builder@1.2.1-alpha.2 ## mastra@1.39.0-alpha.5 ## @mastra/core@1.83.0-alpha.5 ## @mastra/deployer@1.83.0-alpha.5 ## @mastra/editor@0.17.1-alpha.3 ## @mastra/mcp@2.9.0-alpha.2 ## @mastra/mcp-docs-server@1.3.13-alpha.8 ## @mastra/memory@1.41.1-alpha.2 ## @mastra/rag@2.7.3-alpha.1 ## @mastra/server@1.83.0-alpha.5 ## @mastra/elysia@0.1.23-alpha.5 ## @mastra/express@1.5.27-alpha.5 ## @mastra/fastify@1.5.27-alpha.5 ## @mastra/hono@1.7.25-alpha.5 ## @mastra/koa@1.7.27-alpha.5 ## @mastra/nestjs@0.3.0-alpha.5 ## @mastra/next@0.2.41-alpha.5 ## @mastra/server-adapters-test-suite@0.3.0-alpha.2 ## @mastra/tanstack-start@0.2.41-alpha.5 ## @mastra/libsql@1.30.0-alpha.2 ## @mastra/turso@0.1.21-alpha.2 ## @mastra/inngest@1.11.2-alpha.3 ## @mastra/temporal@0.6.0-alpha.5 ## create-mastra@1.39.0-alpha.5 ## @internal/playground@1.39.0-alpha.5 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
6563e7abfc |
fix(libsql): keep vector indexes writable after truncateIndex (#26802)
## What broke `LibSQLVector.truncateIndex()` ran `DELETE FROM <table>` with no `WHERE` clause. On a table with a `libsql_vector_idx` index, that statement empties the table but leaves the old rows in the vector index's shadow table. The next insert then fails: ``` SQLITE_ERROR: vector index(insert): failed to insert shadow row ``` So once you truncated an index, you couldn't write to it again. I reproduced this on both file-backed and `:memory:` databases with `@libsql/client` 0.18: after the bare delete the shadow table still has 1 row and the reinsert fails. The same cycle with `WHERE rowid IS NOT NULL` leaves 0 shadow rows and the reinsert succeeds. Upstream tracking: tursodatabase/libsql#2289. ## The fix `truncateIndex()` now runs `DELETE FROM <table> WHERE rowid IS NOT NULL`. It still deletes every row, and in the repro it also removes the shadow rows (0 left after the delete). The source change is one line in `stores/libsql/src/vector/index.ts`, and it runs for every connection type. ## Tests New tests in `stores/libsql/src/vector/index.test.ts`. They check behaviour only, using public methods: - file database: insert, truncate, insert again, then query and get only the new vector - file database: truncate an empty index, then run repeated truncate/insert cycles - file database: single-row and filter deletes still work, and the index still accepts inserts afterwards - `:memory:` database: insert, truncate, insert again, then query I ran these tests against the old `truncateIndex`. Both file truncate tests and the `:memory:` test fail with the shadow-row error. The partial-delete test passes there, because it guards existing behaviour. With the fix, the whole file passes: 146 passed, 1 skipped (`npx vitest run src/vector/index.test.ts`). oxlint and oxfmt are clean on the changed files. ## Not tested - Remote Turso/sqld URLs and embedded replicas (`syncUrl`). They run the same statement, but I didn't test them against a server. - Cost on large tables. I haven't measured the qualified delete's performance. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 Truncating a local LibSQL vector index could leave behind data that stopped new vectors from being added. This change removes the vector rows while keeping the index ready for new inserts. ## Changes - Updated `LibSQLVector` truncation to delete rows with `rowid IS NOT NULL`. - Added tests for file-backed and in-memory databases. They cover reinserting and querying vectors after truncation, repeated truncation, and partial deletes. - Added a patch changeset for `@mastra/libsql`. ## Validation Test and lint results: not available. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Jamie Lee Nichols <10267569+CodingButter@users.noreply.github.com> Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
7c4dd88400 |
chore: version packages (alpha) (#29889)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
832fb190f7 |
feat(observability): allow opting into remote storage export on Platform (#29737)
When `MASTRA_DEPLOYMENT_ID` is set, storage exporters are suppressed
even when a customer intentionally exports to both their own database
and Platform. Keep that default and add a per-config
`retainStorageExporterOnPlatform: true` opt-in for
`MastraStorageExporter`, including runtime registration.
```ts
new Observability({
configs: {
production: {
serviceName: 'my-service',
retainStorageExporterOnPlatform: true,
exporters: [new MastraStorageExporter(), new MastraPlatformExporter()],
},
},
});
```
Deprecated `DefaultExporter` remains suppressed on Platform even with
the opt-in. Its implementation is unchanged.
The opt-in does not permit known local observability storage.
MastraStorageExporter warns during initialization and disables
persistence when the resolved observability domain reports local
storage. This blocks InMemoryStore (including Mastra's automatic
fallback when no storage is configured), local libSQL files, in-memory
libSQL, embedded replicas, and DuckDB while allowing remote libSQL/Turso
and PostgreSQL. Other storage domains and exporters are unaffected, and
local development retains local storage support.
Adds the optional persistenceLocation capability to observability
storage, with in-memory, libSQL, and DuckDB implementations. These
adapters must be upgraded alongside observability for the guard to
apply; older/custom adapters without location metadata retain the
existing opt-in behavior. No provider names or private connection fields
are inspected.
Constructor skip warnings are deferred until the application logger is
installed and emitted once. Docs explain the local-storage restriction
before the example, include a supported/blocked provider table, and
distinguish the implementation-specific option from the core interface.
The restriction remains explicit even for users who cannot see the
warning. Includes one minor changeset per affected package:
observability, core, libSQL, and DuckDB. Preserves the default policy
introduced in #23555.
Validation: 451 targeted observability tests pass, including regression
coverage that reproduced unwanted in-memory persistence before the fix
for both explicit InMemoryStore and the automatic fallback. The tests
also verify initialization warnings and continued local-development and
remote-storage behavior. All 756 MDX files pass formatting using the
CI-matching dependencies; focused type-aware lint, Remark, and Vale
pass. The earlier 9 adapter tests also passed. No live Platform or
PostgreSQL endpoint was exercised. Full typechecking remains limited by
workflow-helper errors previously reproduced on the base and
installed-dependency type mismatches encountered while regenerating core
declarations.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
On Mastra Platform, storage exporters are normally turned off to prevent
observability data from going to local storage. This change lets each
observability configuration opt in to remote storage export, while still
blocking storage known to be local.
## Changes
- Added `retainStorageExporterOnPlatform` to observability
configuration. It defaults to `false` and applies to exporters
registered during setup or later.
- When enabled on Platform, the option retains `MastraStorageExporter`
but does not enable the deprecated `DefaultExporter`.
- Added `persistenceLocation` to observability storage. The base
implementation returns `undefined`; in-memory storage and DuckDB report
`local`; libSQL reports `local`, `remote`, or `undefined` based on its
client configuration.
- On Platform, `MastraStorageExporter` skips storage that reports
`local` and warns during initialization. Adapters without location
metadata retain the opt-in behavior.
- Added documentation and changesets for observability, core, libSQL,
and DuckDB.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
|
675caf0613 |
chore: version packages (alpha) (#29336)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. ⚠️⚠️⚠️⚠️⚠️⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`. ⚠️⚠️⚠️⚠️⚠️⚠️ # Releases > The changelog information of each package has been omitted from this message, as the content exceeds the size limit. ## @mastra/client-js@1.60.0-alpha.0 ## @mastra/react@1.15.0-alpha.0 ## @mastra/factory@0.28.0-alpha.0 ## @mastra/code-sdk@1.19.0-alpha.0 ## mastra@1.39.0-alpha.0 ## @mastra/core@1.83.0-alpha.0 ## @mastra/mcp@2.9.0-alpha.0 ## @mastra/mcp-apps-components@0.1.0-alpha.0 ## @mastra/playground-ui@69.0.0-alpha.0 ## @mastra/server@1.83.0-alpha.0 ## @mastra/nestjs@0.3.0-alpha.0 ## @mastra/clickhouse@1.30.0-alpha.0 ## @mastra/duckdb@1.19.0-alpha.0 ## @mastra/libsql@1.30.0-alpha.0 ## @mastra/pg@1.36.0-alpha.0 ## @mastra/temporal@0.6.0-alpha.0 ## @mastra/auth-better-auth@1.1.11-alpha.0 ## @mastra/deployer-cloud@1.83.0-alpha.0 ## @mastra/deployer-cloudflare@1.2.42-alpha.0 ## @mastra/deployer-netlify@1.2.42-alpha.0 ## @mastra/deployer-sandbox@0.3.27-alpha.0 ## @mastra/deployer-vercel@1.2.42-alpha.0 ## @mastra/longmemeval@1.1.42-alpha.0 ## @mastra/opencode@0.1.42-alpha.0 ## create-factory@0.4.3-alpha.0 ## mastracode@0.49.1-alpha.0 ## @mastra/arize@1.3.30-alpha.0 ## @mastra/arthur@0.4.30-alpha.0 ## @mastra/braintrust@1.3.27-alpha.0 ## @mastra/datadog@1.4.22-alpha.0 ## @mastra/deepeval@0.1.24-alpha.0 ## @mastra/laminar@1.3.32-alpha.0 ## @mastra/langfuse@1.6.6-alpha.0 ## @mastra/langsmith@1.4.3-alpha.0 ## @mastra/observability@1.23.2-alpha.0 ## @mastra/otel-bridge@1.5.22-alpha.0 ## @mastra/otel-exporter@1.5.2-alpha.0 ## @mastra/posthog@1.4.7-alpha.0 ## @mastra/sentry@1.2.32-alpha.0 ## @mastra/agent-builder@1.2.1-alpha.0 ## @mastra/connect@1.3.2-alpha.0 ## create-mastra@1.39.0-alpha.0 ## @mastra/deployer@1.83.0-alpha.0 ## @mastra/editor@0.17.1-alpha.0 ## @mastra/mcp-docs-server@1.3.13-alpha.0 ## @mastra/memory@1.41.1-alpha.0 ## @mastra/rag@2.7.3-alpha.0 ## @mastra/redis-streams@0.8.1-alpha.0 ## @mastra/elysia@0.1.23-alpha.0 ## @mastra/express@1.5.27-alpha.0 ## @mastra/fastify@1.5.27-alpha.0 ## @mastra/hono@1.7.25-alpha.0 ## @mastra/koa@1.7.27-alpha.0 ## @mastra/next@0.2.41-alpha.0 ## @mastra/tanstack-start@0.2.41-alpha.0 ## @mastra/spanner@1.11.2-alpha.0 ## @mastra/turso@0.1.21-alpha.0 ## @mastra/inngest@1.11.2-alpha.0 ## @mastra/gcs@0.3.6-alpha.0 ## @internal/playground@1.39.0-alpha.0 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
4afeceaf2d |
feat(pg, libsql): add experimental storage v2 for workflow runs (Postgres and libSQL) (#28299)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> Co-authored-by: review <review@local> |
||
|
|
3757223067 |
fix(libsql): stop JSON-parsing text columns on read (#29357) | ||
|
|
6e8dfcb83f | chore: version - exit prerelease mode | ||
|
|
4bc3aa672f |
chore: version packages (alpha) (#29255)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. ⚠️⚠️⚠️⚠️⚠️⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`. ⚠️⚠️⚠️⚠️⚠️⚠️ # Releases > The changelog information of each package has been omitted from this message, as the content exceeds the size limit. ## @mastra/factory@0.27.0-alpha.3 ## @mastra/agent-builder@1.2.0-alpha.3 ## @mastra/core@1.82.0-alpha.3 ## @mastra/mcp@2.8.0-alpha.2 ## @mastra/memory@1.41.0-alpha.3 ## @mastra/playground-ui@68.0.0-alpha.3 ## @mastra/ring-signals@0.1.0-alpha.0 ## @mastra/libsql@1.29.0-alpha.2 ## @mastra/mongodb@1.26.0-alpha.2 ## @mastra/mysql@0.16.0-alpha.3 ## @mastra/pg@1.35.0-alpha.3 ## @mastra/discord@1.3.3-alpha.0 ## @mastra/slack@1.8.1-alpha.0 ## @mastra/teams@0.1.4-alpha.0 ## @mastra/telegram@0.2.5-alpha.0 ## @mastra/client-js@1.59.0-alpha.3 ## @mastra/react@1.14.1-alpha.3 ## @mastra/deployer-cloud@1.82.0-alpha.3 ## @mastra/longmemeval@1.1.41-alpha.3 ## @mastra/opencode@0.1.41-alpha.3 ## create-factory@0.4.2-alpha.3 ## @mastra/code-sdk@1.18.0-alpha.3 ## mastracode@0.49.0-alpha.3 ## @mastra/arize@1.3.29-alpha.2 ## @mastra/arthur@0.4.29-alpha.2 ## @mastra/laminar@1.3.31-alpha.2 ## @mastra/langfuse@1.6.5-alpha.2 ## @mastra/otel-bridge@1.5.21-alpha.2 ## @mastra/otel-exporter@1.5.1-alpha.2 ## @mastra/sentry@1.2.31-alpha.3 ## mastra@1.38.0-alpha.3 ## @mastra/connect@1.3.1-alpha.1 ## @mastra/deployer@1.82.0-alpha.3 ## @mastra/editor@0.17.0-alpha.3 ## @mastra/evals@1.10.11-alpha.0 ## @mastra/mcp-docs-server@1.3.12-alpha.4 ## @mastra/server@1.82.0-alpha.3 ## @mastra/elysia@0.1.22-alpha.3 ## @mastra/express@1.5.26-alpha.3 ## @mastra/fastify@1.5.26-alpha.3 ## @mastra/hono@1.7.24-alpha.3 ## @mastra/koa@1.7.26-alpha.3 ## @mastra/nestjs@0.2.41-alpha.3 ## @mastra/next@0.2.40-alpha.3 ## @mastra/tanstack-start@0.2.40-alpha.3 ## @mastra/turso@0.1.20-alpha.2 ## @mastra/inngest@1.11.1-alpha.2 ## @mastra/temporal@0.5.0-alpha.3 ## create-mastra@1.38.0-alpha.3 ## @internal/playground@1.38.0-alpha.3 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
234d7a3ee6 |
feat(core,stores): pin listings of the Knowledge page tree from config or knowledge_pin, and #29260 review follow-ups (#29263)
Pin part of the Knowledge page tree into an agent's context, from config
or from the agent itself with `knowledge_pin`, plus follow-ups to #29260
(Knowledge page link index and alpha review follow-ups).
> **Migration step on first start.** Pins now have a kind (`page` or
`list`). Each storage adapter upgrades its pins table once, the first
time it starts with this version. LibSQL rebuilds the table (SQLite
can't drop the old one-pin-per-page constraint in place); Postgres and
MySQL alter it in place; MongoDB backfills `kind: 'page'` and swaps the
unique index. Existing pins become page pins; no reset is needed.
## Pin a listing of the page tree
`pinnedContext` on a scope declaration now takes a listing as well as a
page:
```ts
scopes: {
'docs:$orgId': { belongsTo: 'org:$orgId', access: [{ principal: 'org:$orgId', role: 'reader' }] },
'org:$orgId': {
pinnedContext: [
'docs:$orgId#getting-started', // a page, as before
{ list: 'docs:$orgId#guides', depth: 2 }, // the pages under guides, two levels down
],
},
},
```
Every run holding `org:<id>` then gets the same slim lines
`knowledge_list` returns (ref, name, child count) for the readable pages
under that point, on every turn, without having to search or list first.
It updates when pages are added, renamed or removed below it.
- A listing is checked at startup like a page pin: the scope type must
be declared and every `$param` bound. `depth` is 1 to 3, defaulting to
1.
- Only pages the run can read are listed, and a run that can't read the
listing's root gets nothing.
- A listing takes one `maxPins` slot and shares
`pinnedContext.maxTokens` with pinned pages. Its size is set by that
token share, not an entry count: it fills a level at a time (every
first-level page while room lasts, then second-level, up to `depth`) and
prints the result as an indented tree, so every listed page's parent is
listed too. When it doesn't fit, it's cut at a whole line and ends with
`…and N more; knowledge_list from: <ref>` (or `…and more` when the count
isn't known). At the default 500 tokens a wide root may show only its
first level; raise `pinnedContext.maxTokens` to see deeper levels.
- Page pins work as before.
To share the tree walk, it moves out of `knowledge_list` into one
function used by both. `knowledge_list` output doesn't change.
## Agents pin listings with `knowledge_pin`
`knowledge_pin` takes a `kind`: `'page'` (the default, as before) or
`'list'`, with an optional `depth` (1 to 3, default 1):
```json
{ "op": "pin", "kind": "list", "depth": 2, "page": "docs:acme#guides", "audience": ["org:acme"] }
```
- A page can hold a page pin and a listing pin at once; each is pinned
and unpinned on its own. Pinning a listing again sets its new depth and
adds audiences.
- Stored listings follow the config-listing rules: only readable pages,
one `maxPins` slot, shared `maxTokens`, ranked with page pins by
`pinnedContextPriority` then recency. A stored listing that repeats a
config listing is shown once.
- Pin rows store `kind` and `depth`, unique per page and kind, in
LibSQL, Postgres, MySQL, MongoDB and in-memory. `listPins` takes a
`kind` filter; Studio's pinned badge still counts page pins only.
- Adapters report the new `supportsPinKinds` capability. With an adapter
from before this release, page pins keep working and pinning a listing
is refused before anything is stored, with a message to upgrade the
storage package.
## Review follow-ups to #29260
- **`knowledge_list` checks child-scope access in core again.** The
store still filters out unreadable child scopes. Core now checks each
scope it gets back, so a custom store that ignores `read` can't expose
private scopes.
- **Fences follow CommonMark.** A fence indented four or more spaces is
indented code, so it no longer hides the headings after it from
sections, outlines and embedding chunks.
- **Chunk embedding respects the model.** A long page's chunks are
embedded one at a time when the embedding model doesn't support parallel
calls, instead of up to 4 at once.
- **Changesets** now lead with the outcome, say "16 distinct words" for
the search limit, and include usage examples for the link count and page
link client methods, server routes and core readers.
## Test plan
- Pin kinds: core tests for listing pins through the tool (8, including
an old-adapter case that stores nothing) fail without the change. Each
adapter has an upgrade test that seeds a pre-kind pins table, starts the
store, and checks the pin survives as a page pin next to a new listing
pin on the same page. Real stores: LibSQL 79, Postgres 89, MySQL 80,
MongoDB 83; shared contract suite 935 on in-memory; server Knowledge 48.
Core Knowledge and storage 303/303.
- New tests fail without each fix: fence indentation, a store that
ignores `read` on child scopes, sequential embedding when parallel calls
aren't supported, and the 11 pinned-listing tests, and a level-at-a-time
test (a root with 45 children and 3 grandchildren each shows all 45
children, then grandchildren as they fit; on the old 50-entry cap it
showed 5).
- `@mastra/core` Knowledge and storage tests: 295/295 on main
|
||
|
|
01524b22db |
feat(core,stores,server): Knowledge page link index and alpha review follow-ups (#29260)
## Summary Follow-ups to the Knowledge alpha (#29159, #29207, #29218, #29256), in one PR: **Page link index** - Knowledge keeps an index of `[[links]]` in page bodies. `countPageLinks` / `listPageLinks` on the Knowledge store return in and out links for a set of pages, gated by the reader's scopes like every other read; note `[[mentions]]` show up as `note` links. - Links resolve when a page is written through the tools, `upsertPage` or `removePage` (id, `scope#key/key` path, scope address, then page name in the page's own scope, then store-wide). Dangling links resolve once their target is created and follow renames, moves, merges and deletes. `knowledge.reindexLinks()` backfills pages written before this release. - Implemented in the in-memory, LibSQL, PostgreSQL, MySQL and MongoDB stores (additive table/collection, no schema bump). Refs over 512 characters stay plain text; the per-request link cap comes from core through the store loader. - Server routes `GET /knowledge/:knowledgeKey/link-counts` and `GET /knowledge/:knowledgeKey/pages/:pageId/links`, client-js methods, and FGA coverage (`knowledge:read` on the knowledge key, denied caller gets 403). **Review follow-ups** - Fenced code and headings parse like CommonMark (closing fence must match the opener; `## Use C#` keeps its `#`); reading the last part of a page that ends in a newline works. - Chunk embedding runs 4 at a time and semantic search stops at its candidate cap. - PostgreSQL and MongoDB refuse to remove a note that's already removed, like the other stores. - Search queries use at most 16 words. - Changeset for the `reindex()` removal scoped correctly. ## Child scopes filtered by reader in the store - `listChildScopes` takes an optional `read` scope and leaves out child scopes the reader can't see before applying the limit and cursor. New `countChildScopes` counts readable child scopes for many parents in one call. All five stores implement both. - `knowledge_list` uses them: it used to page through every child scope under a scope and drop the unreadable ones afterwards, and list each scope's child scopes just to count them. Under a scope with many private child scopes that meant loading all of them. - This changes cost only, not access: `knowledge_list` already removed unreadable scopes before returning anything, so agents see the same entries and counts. Without `read`, the host's full list (Studio, link resolution, `knowledge_read`'s restricted entries) is unchanged. ## Test plan - Core Knowledge + storage: 274 tests pass; tsc 0; oxlint 0/0; oxfmt clean. - Real stores (Docker): LibSQL 77, PostgreSQL 87, MySQL 78, MongoDB 81 pass. - Server Knowledge handlers + adapter FGA: 143 pass. client-js Knowledge: 6 pass. - New tests fail without their fixes (link hooks, host-write links, ref length, fences/headings, candidate cap, embed concurrency, double note removal, FGA route mapping, child scopes filtered before the limit, knowledge_list never receiving unreadable scopes). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 Knowledge pages can now keep track of links to other pages, including when those pages are added, renamed, moved, or removed. Readers can see only links they are allowed to access, and page listings avoid loading child scopes they cannot read. ## What changed - Added a page-link index for `[[links]]` across in-memory, LibSQL, PostgreSQL, MySQL, and MongoDB stores. `knowledge.reindexLinks()` can backfill the index for existing pages. - Added scope-filtered link counts and link-list APIs, server routes, and client-js methods. - Added child-scope filtering before pagination and child-scope counts for readable scopes. - Improved Markdown fence and heading parsing, page-range handling for trailing newlines, embedding concurrency, and semantic-search candidate limits. - Made repeated removal of an already-removed note return `KnowledgeNotFoundError` in PostgreSQL and MongoDB. Updated the `reindex()` changeset. ## Validation Author-reported test and tooling results are available in the PR objectives; they were not independently verified here. Review finding counts are unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: remy-builder[bot] <324360501+remy-builder[bot]@users.noreply.github.com> |
||
|
|
886679eb10 |
chore: version packages (alpha) (#29205)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. ⚠️⚠️⚠️⚠️⚠️⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`. ⚠️⚠️⚠️⚠️⚠️⚠️ # Releases > The changelog information of each package has been omitted from this message, as the content exceeds the size limit. ## @mastra/client-js@1.59.0-alpha.2 ## @mastra/factory@0.27.0-alpha.2 ## @mastra/code-sdk@1.18.0-alpha.2 ## mastra@1.38.0-alpha.2 ## @mastra/core@1.82.0-alpha.2 ## @mastra/editor@0.17.0-alpha.2 ## @mastra/server@1.82.0-alpha.2 ## @mastra/clickhouse@1.29.0-alpha.1 ## @mastra/duckdb@1.18.0-alpha.1 ## @mastra/pg@1.35.0-alpha.2 ## @mastra/platform-workspace@1.12.0-alpha.1 ## @mastra/react@1.14.1-alpha.2 ## @mastra/deployer-cloud@1.82.0-alpha.2 ## @mastra/deployer-sandbox@0.3.26-alpha.1 ## @mastra/longmemeval@1.1.41-alpha.2 ## @mastra/opencode@0.1.41-alpha.2 ## @mastra/parallel@0.1.5-alpha.0 ## create-factory@0.4.2-alpha.2 ## mastracode@0.49.0-alpha.2 ## @mastra/arize@1.3.29-alpha.1 ## @mastra/arthur@0.4.29-alpha.1 ## @mastra/braintrust@1.3.26-alpha.1 ## @mastra/datadog@1.4.21-alpha.1 ## @mastra/deepeval@0.1.23-alpha.1 ## @mastra/laminar@1.3.31-alpha.1 ## @mastra/langfuse@1.6.5-alpha.1 ## @mastra/langsmith@1.4.2-alpha.1 ## @mastra/observability@1.23.1-alpha.1 ## @mastra/otel-bridge@1.5.21-alpha.1 ## @mastra/otel-exporter@1.5.1-alpha.1 ## @mastra/posthog@1.4.6-alpha.1 ## @mastra/sentry@1.2.31-alpha.2 ## @mastra/agent-builder@1.1.33-alpha.2 ## @mastra/deployer@1.82.0-alpha.2 ## @mastra/mcp@2.8.0-alpha.1 ## @mastra/mcp-docs-server@1.3.12-alpha.3 ## @mastra/memory@1.41.0-alpha.2 ## @mastra/playground-ui@68.0.0-alpha.2 ## @mastra/rag@2.7.2-alpha.0 ## @mastra/elysia@0.1.22-alpha.2 ## @mastra/express@1.5.26-alpha.2 ## @mastra/fastify@1.5.26-alpha.2 ## @mastra/hono@1.7.24-alpha.2 ## @mastra/koa@1.7.26-alpha.2 ## @mastra/nestjs@0.2.41-alpha.2 ## @mastra/next@0.2.40-alpha.2 ## @mastra/tanstack-start@0.2.40-alpha.2 ## @mastra/dsql@1.5.6-alpha.0 ## @mastra/libsql@1.29.0-alpha.1 ## @mastra/mongodb@1.26.0-alpha.1 ## @mastra/mssql@1.11.1-alpha.0 ## @mastra/mysql@0.16.0-alpha.2 ## @mastra/oracledb@0.5.6-alpha.0 ## @mastra/spanner@1.11.1-alpha.0 ## @mastra/turso@0.1.20-alpha.1 ## @mastra/voice-google@0.14.5-alpha.0 ## @mastra/inngest@1.11.1-alpha.1 ## @mastra/temporal@0.5.0-alpha.2 ## @mastra/e2b@0.16.1-alpha.1 ## create-mastra@1.38.0-alpha.2 ## @internal/playground@1.38.0-alpha.2 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
f21823d204 |
feat(core): browse Knowledge with knowledge_list and read long pages in parts (#29207)
Agents using Knowledge could only find pages by searching. Results named nested pages by their last path segment, which `knowledge_read` couldn't resolve, and long pages came back in one piece, sometimes larger than the useful answer. This PR lets agents browse the page tree, read every page a result names, and read long pages a part at a time. Every result says what it left out and how to get it. Stacked on #29159 (Knowledge host writes for pages and notes). Review that first; this PR's diff is against its branch. ## What changes **Full page refs in results.** Search, read, list and page-write results carry `ref`, the page's full readable ref: `site:mastra#models/providers/anthropic`, a scope address, or the page id when the page can't be named by path. The old last-segment `key` field is gone. A read that misses names up to 3 close matches by ref. **`knowledge_list`**, a new read tool: ```ts knowledge_list({ from?: string, depth?: 1-3 /* default 1 */, limit?: 1-100 /* default 50 */, descriptions?: boolean, cursor?: string }) // → { from?: { ref, name }, // entries: [{ ref, name, description?, depth, children, scope?: true }], // description only with descriptions: true // nextCursor?, more?: string[] } ``` Without `from` it lists the scopes the run can read. A result stops at `limit` entries or about 1,000 tokens, whichever comes first. When it stops early, `more` says how many entries it left out and gives a `cursor` that continues exactly where it stopped, at any depth. Unreadable scopes and pages are left out, so the list doesn't reveal they exist. `children` is a count, from a new `countChildPages` store method implemented in all five stores (in-memory, LibSQL, Postgres, MySQL, MongoDB) with a shared contract test. **Path slashes.** Leading, trailing and doubled slashes in a page path are dropped wherever a path is parsed: `upsertPage`, `removePage`, the note methods, `[[links]]`, tool refs and `pinnedContext`. `docs/` and `docs` are the same page. **Reading long pages by line.** A body that fits in one read (1,000 tokens by default) comes back whole as `body`. A longer one comes back as numbered `lines` (`12│text`) with the `range` they cover and `lineCount`, ending at the last whole line that fits; `more` says where to go on with `from: { line }`. The first read of a long page also lists the headings below what it shows, with their line numbers and sizes, so an agent can go straight to the section it needs. A line too long for one read, such as a wide table row, is read by column (`from: { line, col }`). Also: - `outline`: the body's headings with their line numbers and sizes - `section`: one section by heading, as numbered lines - `match`: one or more terms, like `grep -C`: only the lines that contain them, with `context` lines around each hit (default 2) These part reads return only that part, not the notes and children again. **Notes and backlinks only when small.** A whole-page read keeps the page's notes, and the notes on other pages that mention it, while each list stays under 1,000 tokens. A longer list is left out, counted in `more`, and read on its own with `notes: true` or `mentionedIn: true`. A read with `cursor` or `mentionedInCursor` returns only those notes and doesn't repeat the body. **Editing long pages by line.** `knowledge_page` takes a `range` from a read, with `expectedVersion`, and replaces only those lines; an empty `body` removes them. A range may be the one a read returned or smaller, but never wider than one read from its first line shows, so a writer can't overwrite text it never saw. A body write without a range is refused on a page too long for one read, and so is a body that still carries the read's line numbers. **Configurable read size.** `tools: { read: { maxTokens: 3000 }, write: true }` sets how many estimated tokens one `knowledge_read` body part or match result, and one `knowledge_list` result, returns (default 1,000, at least 100). Range writes follow the same size. **Search that points somewhere.** `knowledge_search` matches quoted words (`"api" "responses"`) instead of treating the quote marks as text. An empty result says no page has every word and suggests fewer key words or `knowledge_list`. Passing a page ref as `scope` says it is a page, not a scope, and how to list under it. **Saying what was cut.** Read, search and list results list everything they cut short in `more`, with the exact call that gets the rest. The default instructions mention `knowledge_list` and part reads, and tell agents to follow `more` only when what they need isn't already in what they have read. They don't tell agents to always list first. ## Proof Model runs on a copy of mastra.ai loaded into Knowledge (1,707 pages, scope `site:mastra`, keyword search only): DeepSeek Flash, temperature 0, 12-step cap, 3 fresh runs per question, at `c4495284d46` (this head minus the configurable cap, which leaves the default unchanged). | Question | Result | Calls / input tokens per run | |---|---|---| | A fact in the "Select the OpenAI Responses API" section of the 589-line models page | 2 of 3 answered (the third ran out of steps searching) | 18/160k, 13/128k, 7/35k | | A fact 27,700 chars into the 38,820-char podcast page | 3 of 3 | 6/54k, 4/36k, 4/18k | | Name the observability guides (6) | 6 of 6 on all 3 runs | 20/208k, 10/61k, 16/139k | | Anthropic provider options (17 facts), at an earlier commit | 17/17 each run | 5/8/5 calls, 37–70k chars of tool results (102–221k before) | Direct tool calls on the same data (no model): | Read | Before | After | |---|---|---| | `match` "Steinberger" on the 38k-char podcast page | 20,800 chars | 3,457 | | First read of `site:mastra#models` | 52,561 chars | 6,979 (lines 1–109 and the headings below, notes and backlinks counted) | | Section "Select the OpenAI Responses API" | — | 1,368 | ## Test plan - [x] `pnpm test:core` Knowledge and storage suites: 236 pass. New tests fail on the base for refs that read back, slash paths, `knowledge_list` paging (by count and by tokens, each entry once at depth 2) and access, line reads and grep match, range writes (refusing whole-body, one-line-too-wide, shifted and copied-line-number writes), leaving large notes out, the heading list on a first long read, search hints, and `tools.read.maxTokens` (reads, lists and the write guard follow it; invalid values refused). - [x] Shared store contract (including `countChildPages`) on real stores: LibSQL 66, Postgres 76, MySQL 67, MongoDB 70 - [x] Type check, oxlint and oxfmt clean <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 Knowledge pages are easier to find, browse, and read in useful pieces. The update also makes page references clearer and helps prevent partial edits from changing text that was not read. ## Summary - Adds `knowledge_list` to browse readable scopes and pages. It supports depth limits, descriptions, child counts, and cursor-based pagination. - Uses full page `ref` values in search, read, list, and write results. Failed reads can suggest close matches. - Adds token-bounded reads with numbered line ranges, outlines, sections, and matching lines with context. Range edits require the read range and version; whole-body writes are blocked when the page exceeds the read limit. - Reports omitted notes, backlinks, or page content and explains how to retrieve more. - Normalizes leading, trailing, and repeated slashes in page paths. - Updates search to match any query word and rank results by phrase match, matched-word count, field weight, and recency. - Adds `tools.read.maxTokens` to configure read and list result size, and updates Knowledge instructions to guide browsing and partial reads. - Adds `countChildPages` support to the in-memory, LibSQL, Postgres, MySQL, and MongoDB stores. ## Validation The PR description reports passing Knowledge and storage tests, clean type checking, oxlint, and oxfmt, plus model runs and direct tool-call comparisons. Review finding counts were not provided. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: remy-builder[bot] <324360501+remy-builder[bot]@users.noreply.github.com> |
||
|
|
c7d27f650b |
feat(editor): make model fallback lists editable and persistent (#27287)
Adds opt-in Editor ownership for static model fallback lists on
code-defined agents.
```ts
new Agent({
id: 'support-agent',
model: [
{ id: 'primary', model: 'openai/gpt-5.4-mini' },
{ id: 'fallback', model: 'anthropic/claude-sonnet-4-5' },
],
editor: { model: true },
})
```
Studio can reorder models, change provider/model selections, and enable
or disable entries. These changes follow the configured Editor
persistence workflow:
- Database-backed Editor storage creates versioned drafts that can be
published.
- Code-backed Editor storage saves overrides to the filesystem or
through the configured pull-request workflow.
Persisted entries include their order, provider, model ID, enabled
state, and `maxRetries`. Immutable model IDs reconnect stored values to
code-defined entries while preserving non-serializable code
configuration.
Function-valued model configurations remain code-owned and read-only in
Studio.
This also adds storage support, model-list persistence through the
existing server mutation routes, generated client types, regression
coverage, documentation, and a changeset.
Closes #27089.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
This lets teams edit and save an agent’s model fallback list in Studio
without changing code. The feature is opt-in, so existing agents keep
their current behavior by default.
## Summary
- Added `editor: { model: true }` support for code-defined agents.
- Added persistent `modelList` storage with entry IDs, provider, model
ID, enabled state, and `maxRetries`.
- Applied stored overrides by matching immutable entry IDs while
preserving code-owned configuration.
- Persisted reorder and update actions through Editor drafts and
publishing workflows.
- Added filesystem, LibSQL, server route, export, versioning, and client
cache support.
- Kept function-valued model configurations code-owned and read-only in
Studio.
- Updated ownership logic, API schemas, client behavior, and
documentation.
- Added regression coverage for runtime overrides, persistence, exports,
ownership, draft activation, storage compatibility, and query
invalidation.
- Added changesets for the affected packages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
|
||
|
|
eed9cc214e | chore: version packages | ||
|
|
0a553f0f6f |
Knowledge alpha: standalone Knowledge for agents, Subconscious removed (#28353)
## What this is
**Knowledge** is Mastra's store for durable knowledge that agents share
across conversations. Until now its agent-facing parts lived in
`@mastra/memory`, inside an experimental feature called
**Subconscious**, a background curator that wrote to Knowledge after
observational-memory reflections.
This PR makes Knowledge stand on its own as an experimental alpha,
rebuilt around pages and declared scopes.
- **Subconscious is removed** from `@mastra/memory` and Mastra Code.
Setting `experimental_subconscious` or the Memory `knowledge` option
throws, saying to attach Knowledge to the agent instead. Old
Subconscious data is ignored.
- **Knowledge attaches to an agent directly:** `new Agent({ knowledge,
scopes })`. It brings its own tools, instructions and pinned context.
Tools work without Memory; pinned context needs Memory.
- **Knowledge is a tree of pages.** Each page lives in one scope and
holds a body plus short notes. Notes can mention other pages with
`[[wikilinks]]`. A page can have a key, unique among its siblings, and
is addressed by key path: `<scope>#<key>/<child key>`. Pages declared in
a scope's config are created when missing and can be edited but not
moved, deleted or merged away by tools.
- **Access comes from declared scopes.** Agent `scopes` are plain
`"<type>:<value>"` strings that say who the run is, set by the host and
never by the model. Knowledge's required `scopes` option declares which
scope types exist and which roles (viewer, contributor, editor, owner)
holders get on which scopes. Rules reach one hop with no inheritance. A
run holding an undeclared scope fails. `knowledgeMemoryScopes()`
declares Memory's `thread:`/`resource:` scopes and an `org:` scope.
- **Five tools:** `knowledge_search`, `knowledge_read`, `knowledge_page`
(write, move, merge, delete), `knowledge_note` (add, restamp, remove)
and `knowledge_pin` (pin, unpin). Tools are off by default: `tools:
true` enables read and write, `tools: { read: true }` read only, and
`pinnedContext` adds pins. Deleting a page needs owner on the page's
scope and is refused for pages with child pages, pages declared in a
scope's config, and scope pages. On a scope page, write accepts the body
only: its name and description come from config.
- **Limits are in tokens:** about 250 per note and 100 per page
description. Pinned context defaults to 500 tokens, with a maximum of
2,000.
- **Search:** keyword search by default: every word of the query must
appear in the page (or note), and name matches rank first, the same in
every store. With a `vector` store and an `embedder`, search also
returns semantic matches. Every semantic hit is re-checked against what
the run can read before ranking, so hidden content never takes a result
slot. `knowledge.reindex()` re-embeds everything.
- **Host APIs:** `knowledge.readScopeFor(principals)` returns what
holders of those scopes can read. `knowledge.explainAccess(runScopes,
target)` says why a run can or can't reach a scope or page.
`explainAccess` is host-only.
- **Renamed scope declarations:** if stored scopes use a declaration
name the config no longer declares, Knowledge fails loudly on first use,
naming each orphaned name and how many scopes use it.
`knowledge.renameScopeDeclaration(from, to)` renames them. It is
host-only and atomic, except on MongoDB without a replica set.
- **Per-message scopes:** `session.sendMessage({ content, scopes })`,
plus queue, steer and notifications. A message whose scopes differ from
the running run's waits for that run to end and then starts its own run.
Scopes are set in-process only.
- **Storage:** LibSQL, PostgreSQL, MySQL and MongoDB store Knowledge as
schema 4 (pages, notes, mentions, pins). On startup, the tables of the
earlier experimental Knowledge node store (shipped in 1.79.0) are
dropped with any rows in them, and the new tables are created. If an
older process recreates them during a rolling deploy, the next startup
drops them again. Schema-4 tables and non-Knowledge tables are never
touched. On MongoDB, Knowledge collections found without a schema
version are adopted as schema 4 rather than refused. Don't run
`dangerouslyReset()` from an older version against a database the new
version already uses. Development databases created from earlier commits
of this branch also need a reset, since sibling key uniqueness changed
within schema 4. `storage.stores?.knowledge?.dangerouslyReset()` clears
Knowledge rows only, never Memory. Adapter peer floors are unchanged: on
an older `@mastra/core` the adapters still load and the other storage
domains keep working.
- **Mastra Code:** Knowledge is opt-in through **Experimental
Knowledge** in `/settings`. The `/knowledge` browser and the SDK
inspector read pages. Factory's Knowledge routes are disabled until the
Studio page readers land.
## Test plan
- Shared schema-4 storage contract suite on real stores (as of
|
||
|
|
77076a1012 | chore: version - exit prerelease mode | ||
|
|
582ef17a2b |
chore: version packages (alpha) (#28888)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
e02c876685 |
docs: point store READMEs at the new integrations vector pages (#28970)
Co-authored-by: LekoArts <lekoarts@gmail.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: LekoArts <16143594+LekoArts@users.noreply.github.com> |
||
|
|
b79e61a92d |
feat(core): add expectedSuspendedAt write fence for background tasks (#28590)
> Stacked on #28425 → #28507. This diff is only the change on top of
`daniel/mcp-tasks-3b2-pinned-pending`.
## Why
MCP tasks (#28521) keep a suspended task's open questions and answers in
the background-task row's `metadata`. `tasks/update` reads the row, adds
answers, and writes the whole metadata back fenced on `{ expectedStatus:
'suspended' }`.
If a tool suspends twice (question K1, then K2), a late `tasks/update`
for K1 that lands after the tool resumed and suspended again on K2 still
passes that fence: the status is `suspended` again, and owner and lease
are cleared on suspend, so they don't tell the two suspensions apart.
The stale write replaces K2's questions, and the tool resumes with K1's
answer as the answer to K2.
`suspendedAt` is the field that differs between two suspensions, and the
check has to happen inside the storage write.
## What
- `UpdateBackgroundTaskOptions` gains `expectedSuspendedAt?: Date |
null`, with the same semantics as `expectedLeaseExpiresAt` (`null` means
the row has no suspension time).
- The in-memory store and all 12 adapters implement it by mirroring
their existing lease fence, at the same atomicity level (single
conditional `UPDATE`, Lua CAS, Convex mutation, or DynamoDB condition).
Adapter-specific notes:
- **pg**: fences on the timezone-aware `suspendedAtZ` column, for the
same reason as the lease fence.
- **lance**: an unset `suspendedAt` is stored as `0`, so `null` matches
`IS NULL OR = 0`.
- **dynamodb**: `null` uses `notExists`, and the new condition counts
toward conditional-check failures returning `false`.
- **upstash**: adds an extra ARGV pair to the Lua script.
- The manager now uses the fence when it takes a resume event
(
|
||
|
|
292dd6cd97 |
feat(core): pin pending background tasks to the enqueuing worker (#28507)
## Summary
Background tasks can now be pinned to the worker that enqueued them, so
a queued task whose work is already running in one process isn't started
again by another process.
**Problem.** An MCP `auto` task adopts a tool call that is already
running inside the request on instance A. A enqueues a `pending` row for
it, but a `pending` row has no owner or lease until A claims it. If A is
at capacity the row waits locally, and instance B's stale-task recovery
(or a shared-topic dispatch) can claim it and run the tool a second time
with its static executor.
**Change.** `backgroundTaskManager.enqueue(payload, context, {
pinToWorker: true })`:
- writes the `pending` row with this worker as owner and an execution
lease, renewed by the existing lease heartbeat while the row waits for a
slot;
- other workers skip the row in recovery and shared-topic dispatch while
the lease is live;
- the owning worker claims its own pinned row fenced on owner only, so a
concurrent heartbeat renewal can't strand it;
- if the owner dies (lease lapses) or shuts down, the row is treated as
a **lost attempt**, like a `running` row whose worker died: it consumes
a retry and is re-dispatched if `retryCount < maxRetries`, otherwise it
fails with `Worker process terminated before task completed`. A task
with `maxRetries: 0` never runs on a peer.
**Graceful shutdown.** On shutdown a worker expires the lease on its
pinned pending rows (fenced on owner) but keeps itself as owner, rather
than clearing the pin. Peers recover the row immediately instead of
waiting for the lease to lapse, and still treat it as a lost attempt:
the adopted work died with the process, so a `maxRetries: 0` task fails
rather than reruns. Clearing the owner would make the row look like
fresh work and let a peer run it for free.
Pinning is opt-in. Every agent background task has a task context, so
pinning by default would change how agent pending rows are stored and
recovered; agent behaviour is unchanged. `pinToWorker` without a task
context throws.
No storage adapter changes: every adapter already stores
`ownerId`/`leaseExpiresAt` on create and supports status/owner/lease
fences. The shared adapter suite now proves this for pinned pending
rows.
Based on #28425. Consumed by the MCP tasks background-store PR (3c).
## Test plan
- `packages/core/src/background-tasks/manager-owners.test.ts`: two
managers on shared storage and pubsub — a pinned row on a busy worker is
never run by a peer and runs once on its owner; owner crash with
`maxRetries: 1` runs it once on the peer (`retryCount` 1); owner crash
and graceful shutdown with `maxRetries: 0` fail it without running
anywhere; default enqueue stays unpinned; `pinToWorker` without context
throws. 5 of the new tests fail without the fix.
- `stores/libsql/src/storage/background-tasks-pinned.test.ts`: the same
A/B scenarios across two Mastra instances sharing one libsql file. All 3
fail against core built without the fix.
- Shared storage suite (`execution ownership`) passes on libsql, pg,
mysql, mssql, mongodb, dynamodb, spanner, upstash, lance, cloudflare-d1,
cloudflare DO (convex not run: needs a live deployment).
- Core background-tasks and agent tool-call-step tests: 311 passed.
`@mastra/core` typecheck and oxlint clean.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
A task can wait until its worker has room to run. This change lets
callers keep that waiting task with the worker that created it, so
another worker does not start the same work while the first worker is
still active. If the first worker stops renewing its lease, the task
counts that as a lost attempt and retries or fails.
## Summary
- Add the optional `pinToWorker` setting to
`BackgroundTaskManager.enqueue()`. Pinning requires a task context.
- While a pinned task waits for capacity, its worker renews a lease.
Other workers skip the task while the lease is active.
- When a pin expires, the system consumes a retry if one is available
and a concurrency slot is free. Otherwise, the expired pin remains
pending until a slot is available. If no retries remain, the task fails
with `Worker process terminated before task completed`.
- Periodically scan for expired pins. Queue recovered tasks locally for
another dispatch attempt if publishing fails.
- Document the behavior and add manager, storage ownership, and LibSQL
integration tests.
- Pinning is opt-in. Unpinned tasks retain their existing behavior.
## Validation
The PR objectives report 311 passing core background-task and agent
tool-call-step tests, plus clean `@mastra/core` typecheck and oxlint
runs. Convex was not tested because it requires a live deployment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
|
||
|
|
e77a7d910b |
feat(core): background tasks for non-agent owners (#28425)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
6a191d46dc | chore: version - exit prerelease mode | ||
|
|
976fde7416 |
refactor(lint): enable no-else-return at root (#28601)
Enables `no-else-return: error` in the root Oxlint config and removes 186 redundant `else` branches across 111 source/test files. Existing rule defaults are retained. Reviewed every code diff for behavior preservation. An independent TypeScript AST comparison confirms that changes are limited to removing `else` after returning branches, with the same expressions, statement order, return values, and `try`/`catch`/`finally` boundaries. All 181,158 identifier occurrences retain their bindings. The recorder replay path keeps an explicit block: flattening it would shadow the earlier request `body` with the response `body` and introduce a temporal-dead-zone error. Extends the existing monorepo optimized-dependency test to assert that a raw TypeScript workspace package, including its enum and transitive dependency, still executes in the production bundle. Validation: the root rule-only scan passes on 15,273 files with zero diagnostics; formatting and `git diff --check` pass. Schema compatibility tests: 1,060 passed, 9 skipped. Recorder runtime tests: 21 passed. The core selection initially passed 3,513 tests; rerunning the suites requiring local server/cache access passed 185 tests, with one remaining cache-file assertion also reproduced on unchanged `main`. Remaining validation limits: four asset-download cases cannot reach `placehold.co`; the changed-file lint run reports six import-order errors reproduced on unchanged files from `main`. Deployer unit tests and the monorepo test could not run because this checkout lacks required package dependencies (`fs-extra`/workspace links and `get-port`, respectively). These limits are left visible for CI. No changeset is included because this is a behavior-preserving lint cleanup. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 This PR makes the code easier to read by removing `else` branches that are not needed after a return. It also enables a rule that catches those branches and adds a test for loading a raw TypeScript workspace package in a production bundle. ## Summary - Enables Oxlint’s `no-else-return` rule at error severity and removes 186 redundant `else` branches across 111 files. - Extends the monorepo optimized-dependencies test to check a raw TypeScript workspace package, its enum, and a transitive dependency. - Updates the deployer’s transpile-package path pattern. - In workflow event processing, a truthy branch index with no matching child now falls through to publish a skipped-step end event. Previously, this case published no event. - Keeps the recorder replay block explicit to avoid request/response `body` shadowing and a temporal-dead-zone error. ## Validation Reported checks passed: the rule-only scan across 15,273 files, formatting, `git diff --check`, schema compatibility tests (1,060 passed, 9 skipped), recorder runtime tests (21 passed), and an initial core test selection (3,513 passed). Some test limits remain. Four asset-download cases could not reach `placehold.co`. Changed-file lint reported six import-order errors that also occur on unchanged files from `main`. Deployer unit tests and the monorepo test could not run because dependencies were unavailable. On rerun, suites requiring local server/cache access had 185 passes; one cache-file assertion also reproduced on unchanged `main`. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
31431019dd |
fix(core): distinguish unavailable durable recovery checkpoints (#28611)
Fixes COR-1493. An interrupted durable run can leave a valid initial
pending loop row without restartable input when running-checkpoint
persistence is disabled. `recover()` now reports
`DURABLE_AGENT_RECOVER_CHECKPOINT_NOT_AVAILABLE` with actionable
persistence guidance instead of incorrectly reporting snapshot
corruption. The outer loop remains the restart authority;
malformed-input, workflow-kind, agent-identity, and ownership checks are
preserved. Claim-race refusals release ownership without publishing a
terminal error.
Documents the checkpoint source and replay semantics, adds a core patch
changeset, and restores T4 recovery coverage with an offline aimock
scenario rather than provider recordings. Enable `recovery: {
durableAgents: "auto" }` before starting recoverable runs; enabling it
after a crash cannot reconstruct a missing checkpoint.
Validation: 76 focused core tests passed, including malformed-context
and claim-race controls; the real LibSQL SIGKILL recovery test passed
(also 20/20 earlier repetitions); both offline T4 cells passed in five
repeated runs. The regression was proved red against the unfixed
implementation, producing INVALID_SNAPSHOT in the recovery-off cell.
Core and memory typechecks, focused scenario TypeScript compilation,
core lint, LibSQL test lint, and diff checks passed. Three independent
inline reviews found no remaining correctness blockers; their separate
sandboxes could not execute this checkout.
https://linear.app/kepler-crm/issue/COR-1493
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
An interrupted agent run may have a saved record but still lack the
information needed to restart. This change reports that case clearly and
explains that checkpoint saving must be enabled before the run starts.
## Summary
- `DurableAgent.recover()` reports
`DURABLE_AGENT_RECOVER_CHECKPOINT_NOT_AVAILABLE` when an initial pending
loop snapshot has no workflow input.
- Recovery retains its malformed-input, workflow-kind, agent-identity,
and ownership checks. If claim-time revalidation finds the checkpoint
missing, recovery releases the claim without publishing a terminal
error.
- The recovery route returns HTTP 500 for an unavailable checkpoint and
HTTP 409 for an inactive run.
- Documentation explains checkpoint requirements, replay behavior, and
how to enable running-checkpoint persistence before starting a
recoverable run.
- Tests cover recovery refusals, HTTP responses, offline aimock
recovery, and cross-process LibSQL recovery.
## Reported validation
The author reports that 76 focused core tests passed. The author also
reports that the real LibSQL SIGKILL recovery test passed and both
offline T4 cells passed in five repeated runs. Core and memory
typechecks, focused scenario TypeScript compilation, core lint, LibSQL
test lint, and diff checks also reportedly passed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
|
||
|
|
ef92d71d94 |
chore: version packages (alpha) (#28499)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
d580530ff3 |
fix(core): stop background tasks cancelled from another process from running to completion (#28523)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
2447071b11 |
fix(core): report uncertain scheduler publication outcomes (#28544)
> Opened by @user_01KPS2202RYQR4JAMPVXBB9A1H Refs COR-1363. This ships the narrowed, no-storage-change approach, not the superseded durable-dispatch plan. The scheduler commits its claim before publishing, so rejection or process death can leave an occurrence unstarted. Replaying an ambiguous publication could instead start it twice. ## Changes - Log successful claims and accepted publications with the schedule ID, run ID, and scheduled fire time. A claim without a later success or final failure log identifies an unconfirmed publication, subject to log-flush and claim-to-log crash windows. - After all three immediate publication attempts fail, report `publicationOutcome: 'unknown'` and `automaticReplay: false`. Existing retries use 100 ms and 200 ms backoff and can duplicate accepted events when acknowledgement is lost. - Document best-effort delivery for workflow and agent schedules. After retries are exhausted, that occurrence is not replayed; later recurring fires continue. Exhausted one-shot cron schedules remain completed, which describes cadence exhaustion rather than successful execution. - No replay protocol, public API, schema, migration, or future-fire blocking. Net production diff against integrated main: +14 / -0 lines. ## Verification Integrated origin/main at |
||
|
|
ab1727d5ca |
fix(observability): end spans started by another exporter instance (#28147)
## Summary Under the `batch-with-updates` and `realtime` tracing strategies, a span started by one exporter instance couldn't be ended by another. Durable-agent crash recovery hits this: the recovering process rebuilds the crashed run's agent and model spans and ends them as interrupted, but its `MastraStorageExporter` never saw `SPAN_STARTED`, so it deferred the end until retries ran out and dropped it. The stored span stayed `RUNNING`. `insert-only` and `event-sourced` aren't affected, since `SPAN_ENDED` is a create there. Storage now has an optional conditional update, `batchUpdateUnendedSpans()`. It changes a row only if the row exists and `endedAt` is null, never inserts, and returns the spans it changed. The exporter defers an update only while its own create for that span is unconfirmed, and sends everything else straight to storage: ```diff for each span update or end: - if this exporter flushed the span's create: batchUpdateSpans - else: defer until the create shows up, drop when retries run out + if the store has no batchUpdateUnendedSpans: + same as before + else if this exporter saw SPAN_STARTED but the create isn't stored yet: + defer + else: + batchUpdateUnendedSpans + updates that didn't apply -> drop event, reason span-not-updatable ``` - Ended spans stay immutable. Ending one again doesn't change the row and is reported as a `span-not-updatable` drop. - No reads before writes. The `getSpan` lookups from the earlier version of this PR are gone. - Implemented for the in-memory store, `@mastra/pg` (one `UPDATE ... AND "endedAt" IS NULL RETURNING 1` per record in a transaction) and `@mastra/libsql` (same condition, using `rowsAffected`). `batchUpdateSpans` is unchanged. - The exporter checks for the method at runtime, so older core or store versions keep today's behavior. No peer dependency changes. - The deprecated `DefaultExporter` isn't changed. Known limit, out of scope: if the original process crashes before its create is flushed, there's no row to end. ## Evidence | Before | After | |---|---| |  |  | - **Before:** in `MastraStorageExporter spans from another process` (`observability/mastra/src/exporters/mastra-storage.test.ts`), with the exporter fix reverted, 6 of 10 cases fail for both strategies. Instance A starts an agent span and flushes without ending it, instance B rebuilds and ends it as interrupted, and the stored span keeps `endedAt: null`. **After:** all 10 pass. The stored span ends with the interrupted output. A second end leaves the stored span unchanged and emits one `span-not-updatable` drop. An end for a span that was never stored creates nothing. - Mutation check: sending updates while this exporter's own create is still being written makes `holds back an end until its own create is stored` fail under `batch-with-updates`. - `vitest run src/exporters src/bus` in `@mastra/observability`: 500 passed. - New `batchUpdateUnendedSpans` cases in the shared storage suite (`stores/_test-utils`), which run for every adapter that implements the method: pass on LibSQL and on PostgreSQL 16 in Docker. The full PG storage file (`src/storage/index.test.ts` plus `src/storage/db`) passed 2316 tests after the `executeUpdate` refactor. - Typecheck and oxlint pass for the changed packages. Not tested: other storage adapters (MySQL, MSSQL, MongoDB, ClickHouse and the rest). They don't implement the method, so they keep the previous behavior. ## Merge Danger **Door:** two-way. The new storage method is optional and additive, and the exporter falls back to the old path when it's missing. **Blast radius:** span updates and ends from `MastraStorageExporter` under `batch-with-updates` and `realtime` on in-memory, PG and LibSQL storage. If the conditional update were wrong, span ends could be dropped and spans would stay `RUNNING` in Studio. Fixes #27479 Created with Claude Opus 5.5 in Claude Code. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 One exporter can save a span, and another can finish it later. This change lets supported storage providers update that saved span without creating a duplicate or changing one that has already ended. ## Summary - Added the optional `batchUpdateUnendedSpans` storage method. In-memory, LibSQL, and PostgreSQL storage implement it to update only existing spans that have not ended and report which spans changed. - Updated `MastraStorageExporter` to use conditional updates when a provider supports them. It reports updates that storage did not apply as `span-not-updatable`. Providers without the method retain the previous behavior. - Added tests for spans restored across exporter instances, duplicate ends, missing stored spans, creates still in progress, and providers without conditional updates. - Documented the new drop reason and added patch changesets for the affected packages. ## Validation The supplied PR objectives report 500 passing tests in `@mastra/observability`, passing shared storage-suite cases for LibSQL and PostgreSQL 16, and 2,316 passing tests in the PostgreSQL storage file after the `executeUpdate` refactor. They also report passing typechecks and oxlint for changed packages. Other storage adapters were not tested and retain the previous behavior. A crash before the original span create is flushed remains out of scope. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: 大补 <38815121+AuYuHui@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4fc17c65e8 |
fix(core): preserve remote workflow cancellation (#27904)
> Opened by @abhiaiyer91
Fixes #27805. Supersedes #28174, which was closed as a duplicate. Its
cross-instance test is included here as `cancel-remote-run.test.ts`.
## Summary
You can cancel a workflow run on the default engine from another
process, meaning another `Mastra` instance that shares the same storage.
Before this PR, the run kept going after that cancel and then overwrote
the stored `canceled` status with `success`. The engine only knew about
cancellations made in its own process.
This PR makes the engine respect a cancellation it finds in storage:
- **Atomic guard on snapshot writes.** `persistWorkflowSnapshot` takes
an optional `expectedStatus` compare-and-set guard. The engine passes
every status except `canceled`, so a write is rejected only when storage
already holds `canceled`. The in-memory, LibSQL, PostgreSQL, MySQL,
MSSQL, DSQL and Cloudflare D1 stores check the guard in the same
statement as the write. Other stores read the status right before
writing.
- **Rejected writes stop the run.** A rejected write is recorded as a
cancellation. Final, paused and waiting writes report `canceled` instead
of their own status. A rejected step-start write aborts the run
controller before the step body runs. So `dountil` and `dowhile` stop at
the next iteration's start. Since #26214 on `main`, `foreach` items skip
their own start write, and the foreach checkpoints finished items
without holding the queue. This PR keeps that, so a `foreach`
(concurrency 1) still finishes the item that started while the first
checkpoint was in flight, then stops before the next one.
- **Status-only reads.** `loadWorkflowRunStatus()` reads only the status
column. The engine uses it for the check between entries and as the
read-before-write fallback, so it doesn't load the full snapshot on
every step.
- **One status list, internal to core.** The status list lives in
`storage/domains/workflows/run-statuses.ts`, typed as
`Record<WorkflowRunStatus, true>`, so adding a status to the union
without listing it is a `tsc` error. The engine derives the guard's
`expectedStatus` from it. It is not exported, and the SQL stores return
the stored status directly, as the base fallback and the in-memory store
already do. So the stores import no new core value and their
`@mastra/core` peer ranges are unchanged from `main`.
- **Time travel and runId reuse.** Replaying a run that is already
canceled still works the way it does on `main`. A fresh `start()` on a
runId whose earlier attempt was canceled also runs its steps, as on
`main`: the stored `canceled` belongs to the old attempt, so the new
attempt's first write may replace it. After that first write, a remote
cancel of the new attempt is honored again.
## Proof
- Cross-instance probe on `main` + this PR: a second `Mastra` instance
cancels during the first iteration or step. Each shape stores `canceled`
and skips the later step. `dountil` and `foreach` run the body once. The
shapes are sequential, `dountil`, `foreach`, sleep, and resumed
parallel. On `main`, all five keep running and store `success`.
- `cancel-sleep.test.ts`: 21/21. The loop tests (`dountil`/`foreach` ×
guarded/read-before-write) assert 1 `dountil` iteration and 2 `foreach`
items. With the `handlers/step.ts` cancel check reverted, they fail 4/4
(the loop runs all 3). The write-race tests (final, per-step pause,
executor-abort waiting) now also run with
`supportsGuardedSnapshotWrites()` forced to `false`. With the fallback
status reads disabled in `entry.ts` and `default.ts`, 10/21 fail.
- `cancel-remote-run.test.ts`: 6/6. This covers the sequential
cross-instance cancel from #28174, plus fresh `start()` on a canceled
runId (guarded and read-before-write, same and other instance) and a
remote cancel of that new attempt. With 7776a636's `default.ts` change
reverted, the 5 runId-reuse cases fail and
`background-tasks/manager.test.ts` fails 31 tests.
- `src/background-tasks`: 7/7 files, 171/171 tests (`manager.test.ts`
included).
- Status list: dropping `skipped` from `run-statuses.ts` makes core
`tsc` fail with `TS2741: Property 'skipped' is missing`.
- Head
|
||
|
|
7dbc6c1d18 |
refactor(lint): simplify unnecessary template expressions (#28447)
Enable `typescript/no-unnecessary-template-expression` in the shared
TypeScript configuration and simplify 17 redundant interpolations.
Preserve 11 intentional coercions at runtime boundaries and in mock
response text with local explanations. Existing lint scope and package
manifests are unchanged.
Reviewed every edit and compared emitted JavaScript after the approved
string simplifications, including interpolation order and tagged
templates. All 916 function return types and 1,833 variable binding
types in the 21 affected source files are unchanged. CI lint and build
pass; the local 57-task build, CLI typecheck, formatting, and whitespace
checks pass.
Validation covers 193 focused processor, CLI, Couchbase, and GraphRAG
tests, plus 902 deployer tests across the full run and an isolated retry
of one timeout. That protocol-import case passed alone in 14.55 seconds;
unchanged main took 14.61 seconds against its 15-second timeout. Both
selected published-package monorepo E2E cases pass, including the
extended HTTP check for external module resolution. The broad rule audit
found zero rule violations across 15,106 files, alongside four
TypeScript configuration diagnostics in unchanged files; the targeted
scan passes cleanly.
The DuckDB CI job also fails two existing capability-list assertions
because their expected arrays omit `span-aggregate`. Both failures were
reproduced on unchanged base `
|
||
|
|
2c646b53d0 |
chore: version packages (alpha) (#28399)
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. ⚠️⚠️⚠️⚠️⚠️⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`. ⚠️⚠️⚠️⚠️⚠️⚠️ # Releases > The changelog information of each package has been omitted from this message, as the content exceeds the size limit. ## @mastra/factory@0.25.0-alpha.5 ## @mastra/langsmith@1.4.0-alpha.2 ## @mastra/connect@1.2.0-alpha.2 ## @mastra/core@1.80.0-alpha.4 ## @mastra/playground-ui@66.0.0-alpha.4 ## @mastra/acp@0.4.4-alpha.0 ## @mastra/claude@0.3.4-alpha.0 ## @mastra/cursor@0.3.4-alpha.0 ## @mastra/openai@1.1.4-alpha.0 ## @mastra/auth-auth0@1.2.6-alpha.0 ## @mastra/auth-better-auth@1.1.9-alpha.0 ## @mastra/auth-clerk@1.3.2-alpha.0 ## @mastra/auth-cloud@1.2.8-alpha.0 ## @mastra/auth-firebase@1.1.5-alpha.0 ## @mastra/auth-google@0.1.7-alpha.0 ## @mastra/auth-neon@0.3.5-alpha.0 ## @mastra/auth-okta@0.2.6-alpha.0 ## @mastra/auth-studio@1.3.11-alpha.1 ## @mastra/auth-supabase@1.1.6-alpha.0 ## @mastra/auth-workos@1.6.8-alpha.0 ## @mastra/agent-browser@0.5.5-alpha.0 ## @mastra/browser-viewer@0.2.6-alpha.0 ## @mastra/browser-firecrawl@0.2.5-alpha.1 ## @mastra/stagehand@0.3.7-alpha.1 ## @mastra/discord@1.3.2-alpha.0 ## @mastra/slack@1.8.0-alpha.1 ## @mastra/teams@0.1.3-alpha.0 ## @mastra/telegram@0.2.4-alpha.0 ## @mastra/client-js@1.57.0-alpha.4 ## @mastra/react@1.13.0-alpha.4 ## @mastra/isolated-vm@0.1.4-alpha.0 ## @mastra/quickjs@0.1.5-alpha.0 ## @mastra/deployer-cloud@1.80.0-alpha.4 ## @mastra/deployer-cloudflare@1.2.39-alpha.1 ## @mastra/deployer-netlify@1.2.39-alpha.1 ## @mastra/deployer-sandbox@0.3.24-alpha.1 ## @mastra/deployer-vercel@1.2.39-alpha.1 ## @mastra/voyageai@0.4.3-alpha.0 ## @mastra/longmemeval@1.1.39-alpha.4 ## @mastra/livekit@0.5.2-alpha.0 ## @mastra/opencode@0.1.39-alpha.4 ## @mastra/parallel@0.1.4-alpha.0 ## create-factory@0.3.1-alpha.4 ## @mastra/code-sdk@1.15.1-alpha.5 ## mastracode@0.48.1-alpha.4 ## @mastra/arize@1.3.27-alpha.2 ## @mastra/arthur@0.4.27-alpha.2 ## @mastra/braintrust@1.3.24-alpha.2 ## @mastra/datadog@1.4.19-alpha.2 ## @mastra/deepeval@0.1.21-alpha.2 ## @mastra/laminar@1.3.29-alpha.2 ## @mastra/langfuse@1.6.3-alpha.2 ## @mastra/observability@1.21.1-alpha.2 ## @mastra/otel-bridge@1.5.19-alpha.2 ## @mastra/otel-exporter@1.4.10-alpha.2 ## @mastra/posthog@1.4.4-alpha.2 ## @mastra/sentry@1.2.29-alpha.2 ## @mastra/agent-builder@1.1.31-alpha.2 ## @mastra/auth@1.1.5-alpha.0 ## mastra@1.36.0-alpha.4 ## @mastra/codemod@1.1.8-alpha.0 ## create-mastra@1.36.0-alpha.4 ## @mastra/deployer@1.80.0-alpha.4 ## @mastra/editor@0.16.5-alpha.2 ## @mastra/evals@1.10.9-alpha.0 ## @mastra/fastembed@1.3.5-alpha.2 ## @mastra/loggers@1.3.10-alpha.1 ## @mastra/mcp@2.6.0-alpha.1 ## @mastra/mcp-docs-server@1.3.10-alpha.7 ## @mastra/mcp-registry-registry@1.1.7-alpha.0 ## @mastra/memory@1.39.1-alpha.2 ## @mastra/rag@2.7.1-alpha.0 ## @mastra/schema-compat@1.3.17-alpha.0 ## @mastra/server@1.80.0-alpha.4 ## @mastra/google-cloud-pubsub@1.1.5-alpha.1 ## @mastra/redis-streams@0.6.5-alpha.1 ## @mastra/valkey-streams@0.5.7-alpha.1 ## @mastra/elysia@0.1.20-alpha.4 ## @mastra/express@1.5.24-alpha.4 ## @mastra/fastify@1.5.24-alpha.4 ## @mastra/hono@1.7.22-alpha.4 ## @mastra/koa@1.7.24-alpha.4 ## @mastra/nestjs@0.2.39-alpha.4 ## @mastra/next@0.2.38-alpha.4 ## @mastra/server-adapters-test-suite@0.1.4-alpha.2 ## @mastra/tanstack-start@0.2.38-alpha.4 ## @mastra/astra@1.1.3-alpha.0 ## @mastra/azure-ai-search@0.1.2-alpha.1 ## @mastra/chroma@1.1.5-alpha.0 ## @mastra/clickhouse@1.27.0-alpha.1 ## @mastra/cloudflare@1.7.6-alpha.0 ## @mastra/cloudflare-d1@1.4.5-alpha.1 ## @mastra/convex@1.7.6-alpha.0 ## @mastra/couchbase@1.1.4-alpha.1 ## @mastra/dsql@1.5.5-alpha.0 ## @mastra/duckdb@1.16.0-alpha.1 ## @mastra/dynamodb@1.4.5-alpha.1 ## @mastra/elasticsearch@1.4.6-alpha.1 ## @mastra/lance@1.5.2-alpha.1 ## @mastra/libsql@1.27.1-alpha.0 ## @mastra/mongodb@1.24.1-alpha.1 ## @mastra/mssql@1.10.6-alpha.0 ## @mastra/mysql@0.14.1-alpha.0 ## @mastra/opensearch@1.1.4-alpha.1 ## @mastra/oracledb@0.5.5-alpha.0 ## @mastra/pg@1.33.0-alpha.2 ## @mastra/pinecone@1.1.3-alpha.0 ## @mastra/qdrant@1.1.6-alpha.0 ## @mastra/redis@1.4.12-alpha.1 ## @mastra/s3vectors@1.1.5-alpha.0 ## @mastra/spanner@1.10.1-alpha.1 ## @mastra/turbopuffer@1.2.3-alpha.0 ## @mastra/turso@0.1.18-alpha.0 ## @mastra/upstash@1.5.5-alpha.1 ## @mastra/valkey@0.2.8-alpha.1 ## @mastra/vectorize@1.1.4-alpha.0 ## @mastra/weaviate@0.1.4-alpha.0 ## @mastra/voice-aws-nova-sonic@0.2.6-alpha.0 ## @mastra/voice-azure@0.12.3-alpha.0 ## @mastra/voice-cloudflare@0.13.4-alpha.0 ## @mastra/voice-deepgram@0.13.3-alpha.0 ## @mastra/voice-elevenlabs@0.13.3-alpha.0 ## @mastra/voice-gladia@0.13.3-alpha.0 ## @mastra/voice-google@0.14.4-alpha.0 ## @mastra/voice-google-gemini-live@0.14.18-alpha.0 ## @mastra/voice-inworld@0.4.6-alpha.0 ## @mastra/voice-mistral@0.1.3-alpha.0 ## @mastra/voice-modelslab@0.2.3-alpha.0 ## @mastra/voice-murf@0.13.3-alpha.0 ## @mastra/voice-openai@0.13.4-alpha.0 ## @mastra/voice-openai-realtime@0.14.8-alpha.0 ## @mastra/voice-playai@0.13.3-alpha.0 ## @mastra/voice-sarvam@1.1.4-alpha.0 ## @mastra/voice-speechify@0.14.3-alpha.0 ## @mastra/voice-xai-realtime@0.2.17-alpha.0 ## @mastra/inngest@1.10.8-alpha.3 ## @mastra/temporal@0.4.18-alpha.4 ## @mastra/agentcore@0.5.3-alpha.0 ## @mastra/agentfs@0.2.3-alpha.0 ## @mastra/apple-container@0.5.2-alpha.0 ## @mastra/archil@0.3.2-alpha.0 ## @mastra/azure@0.3.3-alpha.1 ## @mastra/blaxel@0.10.2-alpha.0 ## @mastra/cloudflare-sandbox@0.5.4-alpha.0 ## @mastra/daytona@0.11.4-alpha.0 ## @mastra/docker@0.12.1-alpha.0 ## @mastra/e2b@0.15.1-alpha.1 ## @mastra/e2b-desktop@0.1.10-alpha.1 ## @mastra/files-sdk@0.3.3-alpha.0 ## @mastra/gcs@0.3.5-alpha.1 ## @mastra/google-drive@0.2.4-alpha.0 ## @mastra/mesa@0.3.3-alpha.0 ## @mastra/modal@0.7.3-alpha.0 ## @mastra/platform-workspace@1.10.0-alpha.1 ## @mastra/railway@0.10.2-alpha.1 ## @mastra/s3@0.6.5-alpha.1 ## @mastra/sprites@0.1.2-alpha.0 ## @mastra/vercel@1.6.2-alpha.0 ## @internal/playground@1.36.0-alpha.4 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
|
f02d3b711e |
refactor(lint): remove redundant returns while preserving exits (#28448)
Enable `no-useless-return` in the shared configuration. Remove 28 terminal bare returns, use 31 explicit switch breaks, and retain 12 value-returning contracts with explicit `undefined`. Preserve 13 intentional guards with local explanations, including index-skipping property reads and the platform availability probe. Reviewed every edit and checked that removed returns and new switch breaks have no effectful continuation before function exit, including cleanup through `finally`. All other source and emitted JavaScript syntax is preserved apart from formatting, and all 2,020 function return types and 3,280 variable binding types in the 59 affected source files are unchanged. The new rule reports zero violations across 15,106 files. CI lint and build, the local 57-task build, Factory build, core typecheck, formatting, and whitespace checks pass. All 756 focused tests pass across stream conversions, idle-loop cleanup, workflow validation, Factory, Factory UI, Playground UI, Datadog, Sentry, Cloudflare, and E2B. Both selected published-package monorepo E2E cases pass. The extended build case covers a missing application `.npmrc` and verifies successful builds with dependency installation both skipped and enabled. The DuckDB CI job also fails two existing capability-list assertions because their expected arrays omit `span-aggregate`. Both failures were reproduced on unchanged base ` |
||
|
|
358c71145d |
fix(core): don't re-run finished parallel arms or foreach items when restarting after a crash (#27935)
> Opened by @user_01KNA8J74D4RF84RXSDRX26BS9 Closes #26214 On the default engine, if a run crashes partway through a `.parallel()` block or a `.foreach()` and you call `run.restart()`, the steps that had already finished run again. That can repeat side effects such as charges. Durable agents hit the same bug: they run parallel tool calls through `.foreach()`, so a recovered durable agent re-ran tool calls that had already finished. ## Root cause - **Parallel:** a finished arm isn't checkpointed until the whole block ends. A crash snapshot still lists the finished arm as active, so restart runs it again. - **Foreach:** per-item results are only persisted when the foreach suspends, fails or completes. If it crashes in between, the snapshot has no record of which items finished, so restart runs every item again. ## Fix - **Parallel** (`handlers/control-flow.ts`): each arm that succeeds is checkpointed (phase `parallel-arm-end.<i>`), so restart skips it. - **Foreach** (`handlers/control-flow.ts`): finished items are recorded on the running foreach as `metadata.foreachProgress = {[index]: {output, nestedRunId?}}`. On restart, only items that are not in that record get queued. The record is removed when the foreach ends and never reaches a final step result or the workflow result. - Progress is written by **one coalescing chain per foreach**: one write in flight at a time, and items that finish during a write are folded into the next one, which starts as soon as the previous write lands. There is no timer. A crash can only lose items that finished while a write was in flight, and the write rate is capped by how fast storage completes writes. When the foreach ends, only an in-flight write is awaited. - **Foreach items don't write their own start snapshot** while progress is checkpointed. Each one used to rewrite the whole snapshot with all progress so far. Instead the foreach records itself as running once, before its first item starts, and the chain carries progress from there. - If a progress checkpoint fails, the chain stops writing, cleanup still runs, and the foreach fails with that error. Before this change it would have failed at its next write too. - If a stored snapshot has malformed progress (an entry that isn't an object with `output`, or a non-string `nestedRunId`), restart ignores the progress and runs every item again, the same as a crash before any item finished. - **Write ordering** (`default.ts`, `handlers/entry.ts`): snapshot writes for a run now go through a per-run queue (`serializeSnapshotWrite`), so they land in the order they were made. Before, concurrent full-snapshot writes could land out of order and an older snapshot could overwrite a newer one. The suspended/paused overwrite guard runs inside the queued write, so it is evaluated in order too. - **`handlers/step.ts`:** foreach items skip their start write on engines that checkpoint progress; Inngest keeps it. - **`handlers/foreach-progress.ts`** (new, internal, not re-exported from `@mastra/core/workflows`): `ForeachProgress`, `getForeachProgress`, `omitForeachProgress`. `omitForeachProgress` keeps progress out of per-item step info. - **Engines:** mid-block checkpoints and the write queue only apply to engines that don't replay memoized durable operations (`!requiresDurableContextSerialization()`), so default and the durable agent. Inngest doesn't support `restart()` and records every write as its own durable step, so it writes exactly what it wrote before. The evented engine already wrote per-item foreach results and re-ran only unfinished items; its code is unchanged. ## Proof All tests below are new. Each fix test fails with the fix reverted (`default.ts`, `handlers/{control-flow,entry,step,foreach-progress}.ts` checked out from `origin/main` / before the fix) and passes with the fix: | Test | Fix reverted | With fix | | --- | --- | --- | | `restart-mid-block.test.ts` › parallel arm finished before the crash | ✗ finished arm called again | ✓ | | › 3-arm parallel, unfinished arm between finished ones | ✗ | ✓ | | › foreach, concurrency 1 | ✗ re-runs `[0,1,2,3,4]` instead of `[2,3,4]` | ✓ | | › foreach, concurrency 3 | ✗ | ✓ | | › concurrency 5, items finish after every item has started | ✗ | ✓ | | › billing: crash one macrotask after customer 1 is charged (concurrency 1 and 3) → every customer charged exactly once | ✗ at ` |
||
|
|
0f82b38ed4 |
refactor(lint): flatten lone if statements in else branches (#28431)
Enable `no-lonely-if` in the shared Oxlint config and flatten 38 lone
`if` statements in `else` blocks across 35 files. For example, `else {
if (condition) { ... } }` becomes `else if (condition) { ... }`.
Conditions, evaluation order, statements, comments, and local bindings
are preserved. No package filters or lint exceptions are added.
Strengthen the existing monorepo test to execute another named export
from an optimized dependency. Comparisons of TypeScript syntax, emitted
JavaScript, raw template text, comments, and local bindings pass for all
35 migrated files after normalizing only the redundant else blocks. The
rule clears all 15,094 scanned files.
[CI lint and formatting
pass](https://github.com/mastra-ai/mastra/actions/runs/40344286057/job/121647174331)
for `
|
||
|
|
ce5d4731ee |
chore(deps): update dependencies across the monorepo (#28417)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com> |
||
|
|
0fa30cfbea |
refactor(lint): remove redundant boolean casts and rethrow catches (#27985)
**─────── TLDR ───────**
> The shared Oxlint config now errors on `!!x` inside a condition and on
a `catch` block that only rethrows.
A `catch (err) { throw err; }` reads like something is handled. Nothing
is. `!!x` inside an `if` reads like a deliberate coercion the `if`
already performs. Ten casts and twelve catches across 14 files go.
Nothing reaches a caller differently.
an error thrown inside one of the twelve removed `try` blocks
└─ **unchanged** — same object, same stack, same caller
receives it
| | | |
|---|---|---|
| **+** | `no-useless-catch` | a catch that only rethrows is dead code |
| **+** | `no-extra-boolean-cast` | `if` and `?:` already coerce |
### Decisions
- **no changeset** — no consumer can observe a difference, so no
release; `changeset add --empty` if the red status check bothers you
- **two monorepo e2e assertions for a no-op** — the gate requires
monorepo coverage for any `packages/deployer/` change; they pin the two
reads touched
### Not verified
- [ ] MSSQL, Spanner, Railway, Pinecone and Vectorize paths — their
suites need live services
---
> ██████████████████ **source** `+122 −169` — net −47
> █████ **tests** `+36 −11`
> █ **lint config** `+2`
|
||
|
|
40a522c142 |
chore(deps): update build tools (#27997) |