This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.
## @mastra/auth-auth0@1.3.0-alpha.0
## @mastra/auth-clerk@1.4.0-alpha.0
## @mastra/auth-okta@0.3.0-alpha.0
## @mastra/auth-supabase@1.2.0-alpha.0
## @mastra/client-js@1.60.0-alpha.13
## @mastra/auth@1.2.0-alpha.0
## @mastra/core@1.83.0-alpha.13
## @mastra/playground-ui@69.0.0-alpha.15
## @mastra/rag@2.8.0-alpha.3
## @mastra/server@1.83.0-alpha.13
## @mastra/server-adapters-test-suite@0.3.0-alpha.5
## @mastra/auth-better-auth@1.1.11-alpha.2
## @mastra/auth-cloud@1.2.9-alpha.0
## @mastra/auth-neon@0.3.6-alpha.1
## @mastra/auth-workos@1.6.9-alpha.0
## @mastra/discord@1.3.4-alpha.0
## @mastra/react@1.15.0-alpha.13
## @mastra/deployer-cloud@1.83.0-alpha.13
## @mastra/deployer-cloudflare@1.2.42-alpha.3
## @mastra/deployer-netlify@1.2.42-alpha.1
## @mastra/deployer-sandbox@0.3.27-alpha.1
## @mastra/deployer-vercel@1.2.42-alpha.1
## @mastra/longmemeval@1.1.42-alpha.13
## @mastra/opencode@0.1.42-alpha.13
## @mastra/factory@0.28.0-alpha.15
## create-factory@0.4.3-alpha.15
## @mastra/code-sdk@1.19.0-alpha.15
## mastracode@0.50.0-alpha.14
## @mastra/agent-builder@1.2.1-alpha.5
## mastra@1.39.0-alpha.15
## @mastra/connect@1.4.0-alpha.6
## @mastra/deployer@1.83.0-alpha.13
## @mastra/editor@0.17.1-alpha.7
## @mastra/mcp@2.9.0-alpha.5
## @mastra/mcp-docs-server@1.3.13-alpha.17
## @mastra/memory@1.41.1-alpha.5
## @mastra/schema-compat@1.4.0-alpha.2
## @mastra/elysia@0.1.23-alpha.13
## @mastra/express@1.5.27-alpha.13
## @mastra/fastify@1.5.27-alpha.13
## @mastra/hono@1.7.25-alpha.13
## @mastra/koa@1.7.27-alpha.13
## @mastra/nestjs@0.3.0-alpha.13
## @mastra/next@0.2.41-alpha.13
## @mastra/tanstack-start@0.2.41-alpha.13
## @mastra/voice-google-gemini-live@0.14.19-alpha.2
## @mastra/voice-openai-realtime@0.14.9-alpha.3
## @mastra/voice-xai-realtime@0.2.18-alpha.2
## @mastra/inngest@1.11.2-alpha.9
## @mastra/temporal@0.6.0-alpha.13
## create-mastra@1.39.0-alpha.15
## @internal/playground@1.39.0-alpha.15
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Checks on `b2d9fdf436` — sibling stops and awaited background
cancellation
- A timeout/abort marker on any sibling tool result takes precedence
over a mapping failure, regardless of result order. Mapping failures
still throw when the run has not stopped.
- Cancellation while awaiting background work returns the stopped tool
result with `deadlineTimeout` or `runAborted`. Only rejection with the
signal's actual abort reason is converted; unrelated wait failures still
propagate.
- Regression tests use the real background task manager and cover
deadline/abort before dispatch, during execution, healthy execution and
tool failure in foreground/background modes, across both storage
versions. Focused suites: **58 passed**.
- Revert proofs: restoring per-result mapping checks fails **8 sibling
cases**; removing the background cancellation catch fails **4
during-wait cases**. Both fixes restored and all 58 tests rerun green.
- Broader core checks, including native outcomes across core engines and
complete timeout/message-reference recovery suites: **320 passed / 6
skipped**.
- Real Inngest dev server, retries disabled: two-worker suite **5
passed**; native-outcomes, parity-error-surfaces and
memory-save-on-terminal-failure combined **75 passed / 10 existing
expected failures / 4 skipped**. No expected-failure markers changed.
- Core/Inngest `tsc --noEmit`, oxlint, oxfmt and whitespace checks pass.
Changeset now uses short, outcome-focused paragraphs while retaining the
abort-signal/grace-window and non-cooperative-tool limitations.
- Production multi-pod placement, non-harness pubsub transports and host
clock skew remain unverified.
## Main merge and checks on `0b0e117d96`
## Checks on 579e3ef14d — queued tool dispatch after stop
- Added a composed-signal guard before foreground/background dispatch: a
step whose carried deadline already expired does not invoke the tool or
enqueue a background task, and preserves the timeout/abort marker.
- Regression matrix: foreground/background × expired carried
deadline/plain abort/healthy control, real BackgroundTaskManager, both
storage versions: 16 passed. Removing only the guard fails 8 cases on
actual invocations/task creation. Healthy controls also verify the
returned result and completed background task.
- Core focused recovery/deadline checks: 12 project/file runs, 156 tests
passed; the suspected message-ref recovery regression does not reproduce
with this guard on the merged head.
- Real Inngest two-worker suite: 5 passed. Separately run Inngest
parity-error-surfaces: 20 passed / 2 skipped;
memory-save-on-terminal-failure: 17 passed / 9 expected failures / 2
skipped. All retries off; existing expected-failure markers unchanged.
- Core and Inngest tsc, oxlint, oxfmt, and diff whitespace checks pass.
Core changeset now describes the abort grace window and non-cooperative
tool limitation accurately.
- Production multi-pod placement, non-harness pubsub transports, and
host clock skew remain unverified.
Merged main after #29036 in `ce95d269ba` (merge, not rebase). Resolved
the two scenario files by preserving main's durable/evented
deadline-save behavior and this PR's Inngest finish-path assertions.
Four newly passing Inngest tool-result rows use `itStoppedStep`; the
model-output variant finishing after the deadline remains expected to
fail.
- Core scenario files plus total-timeout suite: 240 passed, 20 skipped.
After marker changes, core memory-save scenario: 112 passed.
- Inngest two-worker deadline suite: three runs, 5 passed each, retries
off.
- Inngest parity-error-surfaces: 20 passed, 2 skipped.
- Inngest memory-save-on-terminal-failure: two consecutive passing runs,
each 17 passed, 9 expected fail, 2 skipped.
- Core and Inngest TypeScript checks passed after the merge; core
TypeScript rerun passed after marker changes. Formatting passed; oxlint
found no applicable files for the final scenario-only change.
- The pre-aborted-tool dispatch guard is now included in `579e3ef14d`.
The earlier suspected message-reference recovery regression did not
reproduce on the merged head; full recovery tests and guard revert
proofs pass (see checks above).
## Summary
When `modelSettings.timeout.totalMs` expired on an Inngest durable
agent, only the process that started the run stopped. A tool already
running on another Inngest worker kept going, and the loop could call
the model again after the caller had reported the timeout.
This PR makes the deadline stop the run on every worker:
- **Deadline carried with the run** (`abort-transport.ts`,
`create-inngest-agent.ts`): the caller puts `{ at, timeoutMs }` on
request context (`RUN_DEADLINE_KEY`) for trigger and resume events. A
worker's abort listener arms a timer for it, or aborts right away if the
deadline has already passed.
- **Deadline arms a timer on every worker**: a worker that receives the
carried deadline aborts the run controller with a `MastraTimeoutError`
when it passes, and the tool's `abortSignal` sees that reason. The
caller no longer publishes the deadline as an abort request (d0d4e64c):
the worker timers already cover it, so the timeout-reason plumbing in
`streams/abort-transport.ts` and `streams/publisher.ts` is back to main.
- **Tool steps listen for aborts** (`tool-call.ts`): tool steps now
install the cross-process abort listener and retry the subscribe if an
earlier step's subscribe failed. Retry is tracked with
`remoteAbortListenerInstalled`, not with whether a signal exists.
- **Stop carried on tool output** (`deadlineTimeout` / `runAborted` on
tool output → llm-mapping → iteration state): the loop predicate ends
the run even on a worker that never heard the abort. This also covers a
call whose `toModelOutput` throws after the deadline stopped it: the
mapping-failure result now carries the stop flags, and llm-mapping skips
the mapping-error throw when they are set, so the run ends as the
timeout/abort instead of a mapping failure that never saves the turn
(80921249).
- **Final map** (`create-inngest-agentic-workflow.ts`): a run past its
carried deadline ends as the timeout (`finishReason: 'error'`, error
event). An aborted run ends as `'abort'`.
- **One error for the caller** (`stream-adapter.ts`): if the caller and
a worker both report the same deadline error, the stream keeps one copy.
- **Deadline timer is independent of the subscription**
(`abort-transport.ts`): the carried deadline is armed before
subscribing, so a failed subscribe still stops the step. A warm worker
re-arms it when a resumed segment carries a new deadline, and drops it
when the resumed segment carries none, so the old segment's timer cannot
stop the resumed run. The timer is bound to the controller it was armed
for, so after a warm resume swaps in a fresh controller, the old timer
cannot abort it even before the resumed segment's first step re-syncs
the deadline. Tool steps call the listener on every step for this.
- **Deadline shows in the final result and tracing**: a deadline-stopped
run ends with step reason `'error'` (tripwire still wins), and its model
and agent spans close as `limit` / `timeout` via
`executionFailureTermination`, not as a normal finish. The final map
checks the deadline again once the finish side effects and thread title
generation are done, so a deadline that passes during the memory save,
output processors or title generation also ends the run as the timeout.
A run the caller aborted first stays an abort: the carried deadline is
not used to turn it into a timeout.
- **Caller accepts its own worker's timeout**
(`create-inngest-agent.ts`): in one process the worker's
carried-deadline timer and the caller's timer abort the same controller.
If the worker's fired first, the caller used to skip emitting the
`MastraTimeoutError`. It now treats any `MastraTimeoutError` on the
signal as its own timeout.
Split from #28950. This PR does not include the caller-side fallback
save at the deadline or the recording of late tool results; those come
in a later PR.
- **Fresh worker controller on warm resume** (2af87c6a): if the previous
segment's total deadline stopped a warm worker's own controller while
the run was suspended, the resumed segment gets a fresh controller
instead of starting already stopped. Owner controllers and other aborts
are kept. The test "gives a warm worker's resumed segment a fresh
controller when the previous deadline passed while suspended" fails with
the fix reverted.
- **Narrower error dedupe** (075ebd8c): the durable stream adapter folds
only a repeated `MastraTimeoutError`. Other errors with a matching name
and message are always delivered. The test "still reports a run error
whose name and message match a streamed non-timeout error" fails with
the fix reverted.
## Proof
- New `durable-deadline-cross-worker.test.ts`: three real worker
processes behind a round-robin proxy and a real Inngest dev server. It
covers five cases: tool stopped on another worker at the deadline;
`abort()` still stops it; a step that first reaches a worker after the
deadline stops; a last step on a fresh worker ends as the timeout; an
aborted run on a fresh worker that never heard the abort makes no more
model calls. Each case asserts exactly one `MastraTimeoutError` and no
further model calls. **3/3 runs passed** with retries off.
- Revert proofs. With each fix reverted on its own, a test fails:
- carried deadline → 2 cross-worker cases fail
- loop predicate check → cross-worker suite fails
- final-map deadline check → cross-worker suite fails
- stop flag on tool output → "fresh worker that never heard the abort"
fails (`finishReason` `'stop'` instead of `'abort'`)
- tool-step listener retry (keyed on signal instead of flag) →
`tool-call-abort-listener.test.ts` fails
- stream dedupe → new `stream-adapter-timeout-dedupe.test.ts` fails (2
errors instead of 1). The cross-worker suite alone did not catch this
because the stream usually closes on the first error.
- Review fixes, each with a revert proof:
- `durable-agent-cross-process-abort.test.ts` adds "arms the carried
deadline even when the subscription fails", "re-arms a warm worker's
deadline when a resumed segment carries a new one" "does not stop a warm
resume that carries no deadline at the previous segment's deadline" and
"does not stop a warm resume at the previous segment's deadline before
its first step starts" (fake timers). Each fails with its
`abort-transport.ts` fix reverted.
- `native-outcomes.scenario.test.ts` adds "reports a deadline hit during
a tool call as a limit". It passes on all 4 core engines and on the
Inngest lane. With the Inngest deadline span termination removed, it
fails (`outcome` `'error'` instead of `'limit'`).
- `native-outcomes.scenario.test.ts` adds "reports a deadline that
expires while the run is finishing as a limit" (an output processor
sleeps past `totalMs`). It passes on Inngest, durable and evented; it is
skipped on the plain loop engines, whose stream has already closed by
then. With the post-side-effects deadline re-check removed, Inngest ends
the run as `success`.
- `native-outcomes.scenario.test.ts` adds "keeps a caller abort when the
deadline passes while the aborted run finishes" and "reports a deadline
that expires while the thread title is generated as a limit" (title
model wrapped with a 2.5s delay). Without the `runAborted` guard the
abort ends as `limit/timeout`; with the re-check before the title wait
the title case ends as `success`. All 5 Inngest deadline cases pass;
core engines pass 14 (both new cases skipped on the plain loop engines).
- After the review fixes, the two-worker suite passed 3/3 again (5/5
each, retries off).
- Core durable suites: 188 files / 1910 tests pass. Core
parity-error-surfaces for all non-Inngest engines: 68 pass.
- Inngest aimock lane, run serially (`INNGEST_TEST_PORT_OFFSET=2000`):
parity-error-surfaces 20/20 pass. The 4 files that fail
(`stream-until-idle-detach`, `observe-lifecycle`, `observe-reattach`,
`stream-until-idle-parked-cancel`; 32 tests) fail the same way on
`origin/main` (0db65eb9a1). One `output-stream-processor` case failed
once, then passed 3/3 when rerun alone.
- `tsc --noEmit` is clean for core and inngest. oxlint and oxfmt are
clean.
## Behaviour change in existing tests
On Inngest, a tool that ignores its `abortSignal` used to keep its step
hanging forever, so nothing after the deadline was saved. The tool step
now ends once the abort grace period passes. The worker then finishes
the turn through its normal finish path, the same path that already
saves a tool that returns after the deadline. The step is saved with the
aborted-tool notice. Two parity tests change their Inngest expectations
to match:
- "closes a stuck tool run on time": the save is released, and the test
checks that the abort notice is saved.
- "only steps that ended reach memory at the deadline": on Inngest, the
completed step and the stopped step are both saved. readOnly still saves
nothing.
- `memory-save-on-terminal-failure` (from #29029/#29047 on main):
Inngest stays in its "deadline save pending" list, because rows that
need the caller-side fallback save (e.g. a slow model reply with no step
finished) still fail. Rows that now pass on Inngest use a separate
`itStoppedStep` marker. Two of them ("saves the text of a step whose
call was cut off once" and "saves a step whose tool call failed
alongside one cut off") fail with this PR's source reverted to main; the
other five also pass on main's source, where main's `it.fails` pin only
held because the test read memory 300ms after the error. Inngest saves
land after the caller reports the timeout, so those rows poll memory
instead of a fixed 300ms wait. On Inngest the "before any step finishes"
row now also saves the streamed assistant text. 3/3 Inngest runs: 12
pass, 8 expected-fail, 1 skipped; core engines: 52 pass, 32
expected-fail.
Other engines' expectations are unchanged.
### Checks on 075ebd8c
- Core: `durable-agent-cross-process-abort` and
`stream-adapter-timeout-dedupe` 28/28 passing; core and Inngest tsc,
oxlint and oxfmt clean.
- Inngest two-worker deadline suite: 3/3 runs, 5/5 each, retries off.
- Inngest aimock `native-outcomes` and
`memory-save-on-terminal-failure`: both exited 0.
- Inngest aimock `parity-error-surfaces`: 20 passed, 2 skipped (run on
its own, retries off). An earlier run had 4 timeouts in the tool-timeout
block because it was the last of six suites run back to back and started
cold (first row took 45s).
### Checks on d0d4e64c (beb23d77 only rewords the core changeset)
- Mapping-failure fix: llm-mapping tests "ends the run as the total
deadline / abort() instead of the mapping failure" fail with only the
`llm-mapping.ts` change reverted; tool-call step test "marks a call
whose toModelOutput throws after the deadline stopped it" fails with
only the `tool-call.ts` change reverted. New native-outcomes row
"reports a deadline hit during a tool call whose toModelOutput throws as
a limit" passes on normal, fs, durable, evented and Inngest.
- Touched core tests: 48 files / 373 tests pass (durable steps,
cross-process abort, abort ack, stream dedupe, streams). Core and
Inngest tsc, oxlint, oxfmt clean.
- Inngest two-worker deadline suite: 3/3 runs, 5/5 each, retries off.
Inngest parity-error-surfaces: 20 passed, 2 skipped. Inngest
native-outcomes deadline rows: 6 passed.
## Unverified
- Real multi-pod production deployments. Proof comes from local
multi-process workers behind a proxy.
- Pubsub transports other than the one in the Inngest test harness.
- After the review fixes, Inngest parity-error-surfaces passed in 2 of 3
full-file runs. In one run, "closes a stuck tool run on time (save
blocked: true)" hit its 15s timeout; it passed when run alone and in
both later full-file runs. Not investigated further.
- Rebased onto main after #29047 (head `4e125e2c`). Rerun after the
rebase: two-worker suite 3/3 (5/5 each, retries off); Inngest
parity-error-surfaces 20 pass / 2 skipped; Inngest native-outcomes 37
pass / 1 expected-fail; core durable suite 3975 tests pass; core and
inngest tsc, oxlint, oxfmt clean. Revert proof for the caller fix: the
tripwire memory-save row failed 1 of 3 runs without it, 3/3 pass with
it.
- Core native-outcomes "keeps a generation ongoing while a provider
attempt is retried" failed in the full file because the new
title-deadline test returned while its delayed title request was still
pending; that request then hit the shared aimock server during the retry
test. Fixed in c58deaf6: the test now waits for the title request. The
full file passes 2/2 (146 passed, 6 skipped); the retry and title tests
run together fail with the old test and pass with the fix.
- `durable-agent-crash-recovery.test.ts` runs out of worker heap in this
sandbox, on pristine `origin/main` too; with `--max-old-space-size=6144`
it passes 10/10 on this branch.
- Warm-resume timer fix (7d85cf14): the new test fails on c58deaf6 and
with only the `abort-transport.ts` change reverted (2/2 projects),
passes with it (18/18). Rerun: two-worker suite 3/3 (5/5 each, retries
off); Inngest parity-error-surfaces 20 pass / 2 skipped; Inngest
memory-save 12 pass / 8 expected-fail / 1 skipped; Inngest
native-outcomes 37 pass / 1 expected-fail; core native-outcomes 146 pass
/ 6 skipped; core and inngest tsc, oxlint, oxfmt clean.
## Still open
- History-off / custom-history: the finish path saves the stuck-tool
turn after the deadline (see Risk). This moves to #28886.
## Risk: medium
- On Inngest, the normal finish path saves a stuck-tool turn after the
deadline. That path flushes the save queue without checking
`historyOff`, a custom history processor, or history turned off at
runtime. So these configurations now save the turn where they used to
save nothing (the tool hung and the save never ran). The later
fallback-save PR adds the deadline save that respects those processors.
- Changesets: `huge-rivers-dance` (`@mastra/core`) and
`polite-snakes-teach` (`@mastra/inngest`) describe the new behaviour.
`late-tool-results-saved.md` was removed on main, so this PR no longer
edits it.
## Assumptions
- The final map still runs normal finish side effects (memory save) on a
deadline run, then emits the error event instead of finish. Skipping the
save would have broken the existing Inngest "tool that finishes after
the deadline still saves its step" behaviour.
- The two-worker fallback-save row from #28950 was dropped. The
output-pass/memory assertions were trimmed to stop + one-error checks,
with an output-pass `finishReason` probe for the final-map case.
- Changesets were generated with `pnpm changeset`, one per package
(`@mastra/core`, `@mastra/inngest`).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
When a run reaches its time limit, each worker can stop the tools it
runs. The run then reports one timeout, and a caller abort stays a
separate outcome.
## Summary
- Carries the absolute run deadline in initial and resumed Inngest
events.
- Arms a deadline timer on each worker. The timer runs even when
abort-listener subscription fails.
- Subscribes tool steps to remote abort requests from other processes. A
remote abort reaches a running tool.
- Keeps the abort reason. A remote abort no longer becomes a generic
`Error('Aborted')`.
- Refreshes deadline state on warm resumes. Replaces a stale worker
controller and clears stale timers.
- Passes deadline and run-abort markers through tool output, mapping
output, and iteration state.
- Stops the agent loop after a deadline or run abort. Emits a
non-continuing step finish.
- Does not invoke a tool whose step starts after the run stopped.
- Returns an aborted-tool notice for a tool that ignores its signal.
Saves that notice in the step.
- Rechecks the deadline after finish side effects and title generation.
- Resolves the final error in this order: total-timeout abort reason,
serialized deadline state, then an expired request-context deadline.
Abort outcomes stay separate from timeout outcomes.
- Reports the deadline as a `MastraTimeoutError`. Deadline-stopped runs
use failure termination data in both spans, unless a tripwire is
present.
- Deduplicates a repeated matching `MastraTimeoutError` in the durable
stream adapter. Non-timeout errors are not deduplicated.
- Runs deadline-stopped finish side effects, including memory saves.
- Adds `readRunDeadline`, `RUN_DEADLINE_KEY`, and the `RunDeadline`
type. Adds an optional `deadline` parameter to
`ensureRemoteAbortListener`.
- Adds `executionFailureTermination` to
`workflows/inngest/src/observability-compat.ts`.
- Adds a cross-worker Inngest test suite with a fixture agent, a worker
process, and a dev-server helper.
- Updates abort, mapping, memory-save, outcome, and tracing tests. Adds
changesets for `@mastra/inngest` and the durable agent.
Scope: 26 files changed, +1711 and -142 lines, per the pull request
diff.
## Validation and limitations
Current-head CI, build, and test results are not available. The supplied
author-reported results are:
- Three runs of the two-worker Inngest deadline suite. Five tests passed
in each run, with retries disabled.
- In Inngest `parity-error-surfaces`, 20 tests passed and 2 were
skipped.
- Core and Inngest TypeScript checks, oxlint, and oxfmt passed.
- One `parity-error-surfaces` run timed out on "closes a stuck tool run
on time (save blocked: true)". The test passed when run alone and in two
later full-file runs.
These results are author-reported and were not independently verified
here.
Open risks:
- The Inngest finish path can save a stuck-tool turn after the deadline.
This can happen after the abort grace period.
- That save does not respect `historyOff` or custom history processors.
This handling is deferred to `#28886`.
- Real multi-pod deployments and pubsub transports outside the Inngest
test harness are not verified.
- The caller-side fallback save and recording of late tool results are
excluded. The author plans them for a later PR.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: NikAiyer <nick.aiyer@gmail.com>
Dynamic agent config callbacks now get the authenticated end user as an
optional `user` argument, so you can personalize an agent without
digging auth keys out of `requestContext` yourself. It works in any
`DynamicArgument` callback: instructions, model, tools, memory,
workspace, processors, scorers, and so on.
```ts
import { Agent } from '@mastra/core/agent';
const supportAgent = new Agent({
id: 'support-agent',
instructions: ({ user }) =>
user ? `You are helping ${user.name ?? user.id}.` : 'You are a helpful assistant.',
model: ({ user }) => (user?.role === 'admin' ? 'openai/gpt-5' : 'openai/gpt-5-mini'),
tools: ({ user }) => (user ? { accountTool, ...(user.role === 'admin' ? { adminTool } : {}) } : {}),
memory: ({ user }) => (user ? userMemory : undefined),
});
```
`user` always has the same `MastraUser` shape, no matter which auth
provider you use, and it's `undefined` for unauthenticated requests:
```ts
import type { MastraUser } from '@mastra/core/types';
// { id: string; name?: string; email?: string; avatarUrl?: string; role?: string }
```
The server auth middleware builds it through a new overridable
`toMastraUser()` on `MastraAuthProvider`. The default handles common
shapes (`id`/`sub`/`userId`, `name`, `email`, `avatarUrl`/`picture`,
`role`), and JWT, Cloud, Supabase, Neon and Better Auth ship overrides
for their user objects. Custom providers can map their own shape:
```ts
import { MastraAuthProvider } from '@mastra/core/server';
import type { MastraUser } from '@mastra/core/types';
class MyAuth extends MastraAuthProvider<MyUser> {
async authenticateToken(token: string) { /* ... */ }
async authorizeUser(user: MyUser) { return true; }
toMastraUser(user: MyUser): MastraUser {
return { id: user.accountId, name: user.profile.displayName, role: user.permissions.primaryRole };
}
}
```
Inline auth configs can pass `toMastraUser` too. If the provider user
has no role and an RBAC provider is configured, the primary RBAC role
fills it in, so `user.role` also works with WorkOS, Okta, Google and
Neon. The raw provider user is still available when you need
provider-specific fields, and the new context key is reserved, so
clients can't spoof it through the request body:
```ts
instructions: ({ requestContext }) => {
const rawUser = requestContext.get('mastra__user'); // provider-specific object, unchanged
// ...
},
```
Dynamic `vectorStore` resolvers on RAG tools get `user` as well:
```ts
createVectorQueryTool({
indexName: 'docs',
model: embedder,
vectorStore: ({ user }) => (user ? tenantStores.get(user.id)! : sharedStore),
});
```
Inside core, every place that invoked a `DynamicArgument` callback
inline now goes through a single `resolveDynamicArgument` helper, which
reads the user from `requestContext` at call time. This is non-breaking:
callbacks get an extra optional field, providers get a method with a
working default. Requests served through an `MCPServer` don't get `user`
yet, since MCP builds its own request context; that's documented.
Tested with unit tests for the helper, normalization and each provider
override, middleware tests (inline config, provider role, RBAC role,
CompositeAuth ordering, unauthenticated), regression tests for the
migrated call sites, and an end-to-end test where an agent behind a fake
auth provider gets personalized instructions. Core, server, rag and the
touched auth packages typecheck clean, and their suites pass.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
When a signed-in user makes a request, configuration callbacks can now
use a standard set of details about that user. When a workflow resumes,
it uses the new request’s user details instead of restoring them from
the earlier run.
## Summary
- Added optional `user` data to `DynamicArgument` callbacks and RAG
vector-store resolvers. Callbacks can also use `getRequestUser()` to
read the normalized user from request context.
- Added the `MastraUser` type and `toMastraUser()` normalization hooks
for auth providers and inline auth configs. The server stores the
normalized user in a reserved request-context key and uses the primary
RBAC role when the user has no role.
- Updated core dynamic configuration paths to use the shared resolver.
Added user support to agent scopes and migrated A2A memory resolution.
- Added `withoutLiveRequestKeys()` to remove live authentication and
authorization data from saved context during workflow resume, restart,
and time travel. Saved non-authentication context, such as tenant data,
remains available.
- Added documentation and changesets. Raised relevant `@mastra/core`
peer dependency minimums to 1.81.0. MCPServer requests do not receive
the normalized `user`.
## Validation
The author reports passing typechecks and test suites across core,
server, RAG, and several auth packages. The author also reports three
failing Inngest tests and earlier failures in evented workflow and
scheduler tests, which reportedly fail the same way on comparison
sources. These results are author-reported and have not been
independently verified.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
## @mastra/client-js@1.60.0-alpha.12
### Minor Changes
- Allow `querySpans()` to sort by `model` or `provider`.
([#30096](https://github.com/mastra-ai/mastra/pull/30096))
```ts
orderBy: [{ field: 'model', direction: 'asc' }];
```
### Patch Changes
- The generated route types for scores now include the `SCORER` and
`PROCESSOR` entity types, matching the entity types experiments can
emit. ([#30145](https://github.com/mastra-ai/mastra/pull/30145))
- Add span-local error counts.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/factory@0.28.0-alpha.14
### Minor Changes
- A card's sandbox now stops as soon as the card is done with that
context, instead of idling until the provider times it out.
([#29981](https://github.com/mastra-ai/mastra/pull/29981))
- When a card moves out of the last phase of a context into a phase that
runs in another one, Factory stops that context's sandbox. Triage →
Planning stops the triage sandbox; Planning → Building keeps the shared
`work` sandbox running. A run still in flight on the session keeps its
sandbox until it ends.
- When a card finishes, Factory stops every sandbox it holds, from any
Factory process, by identity through the configured sandbox provider.
Before, only the process that had opened a sandbox stopped it.
- A card that comes back to a context resumes the same sandbox with its
files.
Stopping never fails the move. Sandboxes are stopped, not destroyed;
deleting a session still destroys its sandbox.
- Added two read routes that answer what the new session page shows
without sending a project's cards.
([#30097](https://github.com/mastra-ai/mastra/pull/30097))
`GET /web/factory/projects/:id/work-items/starters` counts the cards
behind each starter on the project's default board, such as the cards
whose run failed, the pull requests awaiting review and the cards not
yet triaged.
`GET /web/factory/projects/:id/work-items/activity?since=<instant>`
returns how many cards the project has, and when cards were created and
completed since that instant. `since` is required and must be an ISO
date-time such as `2026-06-01T00:00:00.000Z`. The answer carries an
`ETag`, so asking again for the same `since` with `If-None-Match`
returns a 304 with no body when nothing changed.
### Patch Changes
- Fixed environment templates that could stop rebuilding for good. When
a build start was never recorded and the sandbox provider can't list its
builds (Mastra Platform), every later Build now, schedule and push run
used to skip with `unresolved` forever. Factory now treats that start as
failed 30 minutes after its lease lapsed and the next trigger builds.
([#30148](https://github.com/mastra-ai/mastra/pull/30148))
Saving a change to a template input (setup commands, the working
directory, settings, environment repositories, source-control
connections or Build variables) now rebuilds the template once the
project has one, so new sandboxes stop rerunning setup on every boot.
Saves that change nothing don't build, and a change saved during any
build, scheduled ones included, is built when that build ends.
- Factory sessions and MCP clients can now read the Factory's audit
trail with `factory_list_audit_events`: who did what to which work item,
run, branch or setting, and when. Filter by actor (`me`, a user id, or
an agent, optionally with the agent runs a person started), namespace or
exact actions, work item and time range, and page back with a cursor.
The tool only returns this Factory's events and never returns request
context (IP address, user agent), actor emails or event metadata beyond
who started an agent run. It leaves out events from or about other
people's private sessions.
([#30165](https://github.com/mastra-ai/mastra/pull/30165))
- Added a "Join the issue's card instead" switch to intake routes in the
Factory UI. A route for pull requests that close an issue can now put
the pull request on that issue's card instead of giving it a card of its
own; until now only the route Factory seeds could do this. The switch
shows on new and saved routes whose conditions include "Source kind is
Pull request" and "Pull request closes an issue is Yes".
([#30197](https://github.com/mastra-ai/mastra/pull/30197))
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/code-sdk@1.19.0-alpha.14
- @mastra/mcp@2.9.0-alpha.4
## @mastra/observability@1.24.0-alpha.7
### Minor Changes
- Add span-local error counts.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/core@1.83.0-alpha.12
### Minor Changes
- Add model and provider as span query sort fields.
([#30096](https://github.com/mastra-ai/mastra/pull/30096))
```ts
orderBy: [{ field: 'model', direction: 'asc' }];
```
- Add span-local error counts.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))
### Patch Changes
- Name the agent in the missing-storage error thrown by `Memory`. When
an agent's memory never received storage because the agent was never
attached to a Mastra instance, accessing `memory.storage` now fails with
`[Agent:<name>] Memory requires a storage provider to function. ...`, so
it is clear which agent is misconfigured. The registration and
standalone-storage guidance added in #29972 is unchanged, and agents
that inherit storage from their Mastra or configure it directly on
`Memory` are unaffected.
([#30164](https://github.com/mastra-ai/mastra/pull/30164))
- Experiments with a scorer or processor target now save their scores,
and workflow step scorers in experiments now save their step scores.
Previously these scores were silently discarded.
([#30145](https://github.com/mastra-ai/mastra/pull/30145))
## @mastra/mcp-apps-components@0.1.0-alpha.7
### Minor Changes
- Dashboards authored in chat now accept the same widgets as the
dashboard editor: KPIs that combine values (ratio, difference, max,
fallback), read trace, span or score aggregates (for example p95 span
duration) and use duration, percent or score formats; line charts of
scores; six more chart colors (blue, orange, pink, yellow, red, purple);
and optional KPI icon and detail, series color and table empty message.
([#30192](https://github.com/mastra-ai/mastra/pull/30192))
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/playground-ui@69.0.0-alpha.14
### Minor Changes
- Widened what dashboard widgets can show:
([#30192](https://github.com/mastra-ai/mastra/pull/30192))
- KPIs can combine values with `ratio`, `minus`, `max` and `coalesce`,
read trace and span aggregates (for example `duration.p95`) and score
aggregates, and are formatted as `duration`, `percent` or `score`.
Combined values are compared with the previous window.
- Charts accept six more colors: `blue`, `orange`, `pink`, `yellow`,
`red` and `purple`.
- KPI `icon` and `detail`, series `color` and table `emptyMessage` are
optional. Series without a color pick one from a palette, and tables
without an empty message show "No data".
### Patch Changes
- Fixed hover highlights on cards and other surfaces stopping short of
the edge. The highlight now covers the whole surface and follows its
rounded corners exactly, including the sidebar credits card and
clickable cards. Hover highlights on these surfaces now switch instantly
instead of fading.
([#30047](https://github.com/mastra-ai/mastra/pull/30047))
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/client-js@1.60.0-alpha.12
- @mastra/react@1.15.0-alpha.12
## @mastra/server@1.83.0-alpha.12
### Minor Changes
- Support model and provider sorting in span queries.
([#30096](https://github.com/mastra-ai/mastra/pull/30096))
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/clickhouse@1.30.0-alpha.4
### Minor Changes
- Sort span queries by model or provider in ascending or descending
order. ([#30096](https://github.com/mastra-ai/mastra/pull/30096))
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/duckdb@1.19.0-alpha.5
### Minor Changes
- Sort span queries by model or provider in ascending or descending
order. ([#30096](https://github.com/mastra-ai/mastra/pull/30096))
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/pg@1.36.0-alpha.6
### Minor Changes
- Sort span queries by model or provider in ascending or descending
order. ([#30096](https://github.com/mastra-ai/mastra/pull/30096))
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/react@1.15.0-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/client-js@1.60.0-alpha.12
## @mastra/deployer-cloud@1.83.0-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/deployer@1.83.0-alpha.12
## @mastra/longmemeval@1.1.42-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/libsql@1.30.0-alpha.3
## @mastra/opencode@0.1.42-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/libsql@1.30.0-alpha.3
## create-factory@0.4.3-alpha.14
### Patch Changes
- Updated dependencies
[[`1fbc69a`](1fbc69aa9a),
[`4e378c3`](4e378c3e26),
[`1fbc69a`](1fbc69aa9a)]:
- mastra@1.39.0-alpha.14
## @mastra/code-sdk@1.19.0-alpha.14
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
- @mastra/duckdb@1.19.0-alpha.5
- @mastra/client-js@1.60.0-alpha.12
- @mastra/pg@1.36.0-alpha.6
- @mastra/mcp@2.9.0-alpha.4
- @mastra/libsql@1.30.0-alpha.3
## mastracode@0.50.0-alpha.13
### Patch Changes
- Fix `/settings` crashing when mouse preferences are missing from saved
settings by defaulting to Off.
([#30184](https://github.com/mastra-ai/mastra/pull/30184))
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
- @mastra/duckdb@1.19.0-alpha.5
- @mastra/pg@1.36.0-alpha.6
- @mastra/code-sdk@1.19.0-alpha.14
- @mastra/mcp@2.9.0-alpha.4
- @mastra/libsql@1.30.0-alpha.3
## @mastra/arize@1.3.30-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/arthur@0.4.30-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/braintrust@1.3.27-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/datadog@1.4.22-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/deepeval@0.1.24-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/laminar@1.3.32-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/langfuse@1.6.6-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
- @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/langsmith@1.4.3-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/otel-bridge@1.5.22-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
- @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/otel-exporter@1.5.2-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/posthog@1.4.7-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
## @mastra/sentry@1.2.32-alpha.7
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/observability@1.24.0-alpha.7
- @mastra/otel-exporter@1.5.2-alpha.7
## mastra@1.39.0-alpha.14
### Patch Changes
- Added `mastra api factory work-item starters` and `mastra api factory
work-item activity`. The first counts the cards behind each new-session
starter on a project's default board. The second lists when a project's
cards were created and completed since an instant:
([#30097](https://github.com/mastra-ai/mastra/pull/30097))
```bash
mastra api factory work-item starters <project-id>
mastra api factory work-item activity <project-id>
'{"since":"2026-06-01T00:00:00.000Z"}'
```
- Added a "Join the issue's card instead" switch to intake routes in the
Factory UI. A route for pull requests that close an issue can now put
the pull request on that issue's card instead of giving it a card of its
own; until now only the route Factory seeds could do this. The switch
shows on new and saved routes whose conditions include "Source kind is
Pull request" and "Pull request closes an issue is Yes".
([#30197](https://github.com/mastra-ai/mastra/pull/30197))
- Improved the Factory new session page on large boards. Its starters
and its activity band now load the counts and dates they show instead of
downloading every card of the project. They show the same content as
before. ([#30097](https://github.com/mastra-ai/mastra/pull/30097))
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/deployer@1.83.0-alpha.12
## @mastra/deployer@1.83.0-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/mcp-docs-server@1.3.13-alpha.16
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/elysia@0.1.23-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/express@1.5.27-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/fastify@1.5.27-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/hono@1.7.25-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/koa@1.7.27-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/nestjs@0.3.0-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
## @mastra/next@0.2.41-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
- @mastra/hono@1.7.25-alpha.12
## @mastra/tanstack-start@0.2.41-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/server@1.83.0-alpha.12
- @mastra/hono@1.7.25-alpha.12
## @mastra/voice-openai-realtime@0.14.9-alpha.2
### Patch Changes
- Use gpt-realtime-2.1-mini by default for OpenAI Realtime voice
connections instead of the retired
gpt-4o-mini-realtime-preview-2024-12-17 model.
([#30183](https://github.com/mastra-ai/mastra/pull/30183))
## @mastra/inngest@1.11.2-alpha.8
### Patch Changes
- Support span-local error counts in Inngest workflows.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
## @mastra/temporal@0.6.0-alpha.12
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/deployer@1.83.0-alpha.12
## create-mastra@1.39.0-alpha.14
## @internal/playground@1.39.0-alpha.14
### Patch Changes
- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`f9088d8`](f9088d8510),
[`987566e`](987566e882),
[`f071044`](f0710446e0),
[`43d1e9c`](43d1e9c16d)]:
- @mastra/core@1.83.0-alpha.12
- @mastra/client-js@1.60.0-alpha.12
- @mastra/playground-ui@69.0.0-alpha.14
- @mastra/react@1.15.0-alpha.12
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Description
Export `errorCount` as failures originating directly on each span. Child
failures forwarded through a model step, workflow or agent add no parent
count; independent parent failures do. Failed retries remain counted
after recovery, and replay does not create new occurrences. No
parent/child totals or suspend/resume rollup are maintained.
Attribution stays in a WeakMap without modifying user errors. Explicit
internal serializers and Inngest memoized operations preserve provenance
across boundaries; public workflow results and stored workflow state
omit transport metadata. Failures before child execution count on their
actual originating span. Excluded spans preserve attribution so their
failures cannot be misassigned to visible parents, but excluded failures
are absent from exported totals.
Retry context stays in the server-only context module, while shared
origin utilities use global crypto. Guarded cross-package calls preserve
mixed-version startup compatibility. Unsupported accounting and older
snapshots remain unknown. Error-free spans do not allocate occurrence
sets.
Validation: 117 focused observability/workflow/compatibility tests, 23
real Inngest error/retry scenarios and 37 Inngest engine/nested-result
checks pass. New regressions reproduced resolver undercounting,
excluded-child misattribution and public transport-field leakage before
the fixes. Real Inngest tests cover an empty execution graph,
unavailable remote child, instrumented child errors, nested propagation,
retries, public responses and stored state. Core/observability build,
Inngest typecheck, changed-file type-aware lint and CI formatting checks
pass. The preceding merge also passed 167 core checks and redaction
integration coverage.
Playground production builds were verified on Node 24. Docs checks with
Vale report no errors (three warnings on unchanged API headings); the CI
Vale configuration passes. Standalone observability `tsc` previously
reported generic union-call errors in untouched
`workflow-control-flow-suite.ts`; package builds and focused tests pass.
No production rollout or query-table changes. CI reruns on the new head.
## Related issue(s)
Fixes https://linear.app/kepler-crm/issue/OBS-901
Follow-ups: https://linear.app/kepler-crm/issue/OBS-902 and
https://linear.app/kepler-crm/issue/OBS-903
Nonblocking review follow-up:
https://linear.app/kepler-crm/issue/OBS-905 tracks explicit propagated
classification on previously untagged errors and reuse of one Error
object across separate foreach iterations. Changesets intentionally
remain single sentences per maintainer direction.
## Type of change
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
- [x] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
This change counts each error on the span where it first happened. It
keeps the count correct when errors are wrapped, retried, or rebuilt, so
one failure is not counted twice.
## Summary
- Adds span-local `errorCount` to exported spans. The count covers
failures that start directly on that span.
- Adds `claimSpanError`, `BaseSpan.recordError`, and
`BaseSpan.restoreErrorCount`. Each claim records one occurrence ID on
the span.
- Ignores propagated failures on the receiving span. Propagated failures
count only on their origin span.
- Keeps error origin through cause chains, durable tool serialization,
and event codecs.
- Stores origin metadata in a `WeakMap` for live errors and formatted
results. Omits it from public workflow JSON.
- Stores occurrence IDs in span snapshots. Rebuilt spans restore them,
so replayed attempts are not counted again.
- Records failed attempts for default, evented, and Inngest workflow
retries. Each attempt uses an attempt-scoped occurrence ID.
- Reports `errorCount` as unknown when the runtime lacks the claim API
or when an older snapshot has no accounting history.
- Updates tracing types, the span record schema description, the span
reference documentation, and changesets for `@mastra/core`,
`@mastra/observability`, `@mastra/client-js`, and `@mastra/inngest`.
- Adds tests for repeated reports, retry scopes, concurrent async spans,
excluded children, cause-linked errors, tool and agent spans, rebuilt
and legacy snapshots, codecs, durable serialization, workflow retries,
and ESM/CJS compatibility with an older core build.
## Scope
- Parent and child totals are not computed.
- Suspend and resume rollups are not computed.
- Query-time summation, display, filtering, sorting, and aggregate-table
integration are not included. These are tracked in
[OBS-902](https://linear.app/kepler-crm/issue/OBS-902) and
[OBS-903](https://linear.app/kepler-crm/issue/OBS-903).
- Counts reflect observed exported failures only. Filtering, sampling,
or loss before export can leave counts incomplete.
## Change size
- 32 files changed
- 862 lines added
- 161 lines removed
## Validation
- Completed checks on the current head: not available from the supplied
evidence.
- Author-reported results, not verified here:
- Focused observability tests, codec and durable-serialization checks,
and engine and Inngest retry scenarios.
- Node 24 builds, 167 core checks, 125 observability and redaction
tests, and 19 real Inngest scenarios.
- Type-aware lint and `lint:format`.
- Known limitations:
- A prior monorepo E2E attempt did not run application assertions.
Fixture dependency installation failed with a pnpm store ENOENT error.
- Earlier runtime tests ran on Node 23.11.0, which is outside the
supported engine range.
- Standalone observability `tsc` reports errors in an untouched test
utility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Opened by @wardpeet
## Result: Approved — c2acf6499e
The three duplicated `fullStream` detach wrappers are now one shared
`detachFullStreamOnEnd` helper. The extraction is complete,
behavior-preserving, and covered by the helper's own tests plus the
COR-1525 end-to-end regression.
### Checks on this head
| Check | Result | Tests |
| --- | --- | --- |
| Diff against base `9efd1ea8` | ✅ Passed | 1 focused commit, 8 files,
+186/−90 |
| Only one copy of the wrapper remains | ✅ Passed | helper only;
`agent.ts:12426`, `durable-agent.ts:5942`,
`create-inngest-agent.ts:2303` all call the helper |
| Removed inline wrapper matches helper body | ✅ Passed |
byte-identical, modulo `this` typing |
| `@mastra/core` typecheck (`pnpm check`) | ✅ Passed | 0 errors |
| `@mastra/inngest` typecheck (`tsc --noEmit -p tsconfig.json`) | ✅
Passed | 0 errors |
| Lint (`oxlint --type-aware`, changed files) | ✅ Passed | 0 warnings, 0
errors |
| New unit test `observe-full-stream.test.ts` | ✅ Passed | 4 tests × 2
vitest projects |
| `@mastra/core` durable suite (`src/agent/durable/`) | ✅ Passed | 3989
passed, 4 skipped; crash-recovery 20/20 with a raised heap (see note) |
| Inngest observe e2e (COR-1525 regression) | ✅ Passed | replays a
finished run through `observe(runId).fullStream` |
| Inngest unit `create-inngest-agent.test.ts` | ✅ Passed | 117 passed |
Note: the heavy `durable-agent-crash-recovery.test.ts` aborts one worker
on the sandbox's default Node heap limit; the same abort reproduces at
the base commit, and the file passes 20/20 with
`NODE_OPTIONS=--max-old-space-size=6144`. Environmental, not caused by
this change.
### Follow-ups
- `packages/core/src/agent/durable/observe-full-stream.test.ts`: the
end-path test counts `detach` calls but does not assert it is not called
before the source ends; an earlier assertion would tighten the contract.
- `packages/core/src/agent/durable/observe-full-stream.test.ts`: the
structural `FakeObserverOutput` stands in for the real
`MastraModelOutput`; real-path coverage is the COR-1525 Inngest e2e, so
this is proportionate.
### Adversarial check
Strongest case for request changes: the extraction silently altered
behavior (the wrapper's `this` receiver or the `ReadableStream`
constructor). It fails: the removed inline code is byte-identical to the
helper body, both instantiate the global `ReadableStream` (the
`node:stream/web` import is type-only), and the destructurable-proxy
path is exercised green by the COR-1525 e2e on this head.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
When an observer watches an agent's output stream, it must stop watching
at the right time. This PR moves that stop logic into one shared helper.
Both the core agent and the durable agent now use it. The Inngest
package keeps its own copy, because it can run with older core versions
that lack the helper.
## Overview
The PR extracts the `fullStream` detach wrapper from `Agent.observe()`
and `DurableAgent.observe()` into a shared helper,
`detachFullStreamOnEnd`, in `@mastra/core`. The helper is exported from
the durable-agent module. The `@mastra/inngest` package keeps its
wrapper inline, with a comment that explains why.
Diff scope: 7 files changed, 196 lines added, 60 lines removed.
## Changes
- **`packages/core/src/agent/durable/observe-full-stream.ts`** (+58):
Adds `detachFullStreamOnEnd(output, detach)`. It wraps the output's
inherited `fullStream` getter in a lazy stream. The wrapper calls the
original getter with the output as receiver and forwards chunks. It
calls `detach` when the reader reaches the end, when a read errors, or
when the consumer cancels. Cancellation forwards the cancel reason to
the underlying reader.
- **`packages/core/src/agent/durable/observe-full-stream.test.ts`**
(+109): Adds tests for chunk forwarding and detach on source end. It
also covers read-error propagation, cancellation reason forwarding, and
an inherited getter called through a proxy.
- **`packages/core/src/agent/durable/index.ts`** (+4): Re-exports
`detachFullStreamOnEnd` with a comment about use by integrations outside
core.
- **`packages/core/src/agent/agent.ts`** (+2/-30): `Agent.observe()`
calls the helper. The inline `fullStream` wrapper is removed.
- **`packages/core/src/agent/durable/durable-agent.ts`** (+2/-30):
`DurableAgent.observe()` calls the helper. Its custom `fullStream`
wrapper is removed.
- **`workflows/inngest/src/durable-agent/create-inngest-agent.ts`**
(+5): Adds a comment explaining why the `fullStream` detach logic stays
inline. The declared `@mastra/core` peer range includes versions that do
not export `detachFullStreamOnEnd`. No runtime behavior changes.
- **`.changeset/observe-full-stream-helper-core.md`** (+16): Adds a
patch changeset for `@mastra/core`. It documents the helper and states
that existing `observe()` behavior is unchanged.
## Public API
- Added: `detachFullStreamOnEnd<TOutput>(output:
FullStreamOutput<TOutput>, detach: () => void): void`, exported from the
durable-agent module in `@mastra/core`.
- No other exported declarations change.
## Testing
Results below come from the PR author. This summary did not rerun them.
- Typechecks pass for `@mastra/core` and `@mastra/inngest`.
- Lint passes on changed files.
- Four helper tests pass in each of two Vitest projects.
- The durable suite reports 3,989 passing tests and 4 skipped.
- `create-inngest-agent.test.ts` reports 117 passing tests.
- The crash-recovery test passes 20/20 with
`NODE_OPTIONS=--max-old-space-size=6144`. Under the sandbox's default
heap limit, one worker aborts. The same abort occurs at the base commit.
- The COR-1525 end-to-end regression replays a finished run through
`observe(runId).fullStream`.
## Known gaps
- The helper's end-path test does not assert that `detach` is not called
before the source ends.
- The test's structural `FakeObserverOutput` does not match the real
`MastraModelOutput`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: NikAiyer <nick.aiyer@gmail.com>
## Summary
A `DurableAgent` run started with `stream()` stopped streaming its
sub-agents after a resume that passed a `requestContext`. On a warm
resume with a request context, the run's tools are rebuilt, and the
rebuild picked `generate` whenever `closeOnSuspend` was set.
`closeOnSuspend` is also a `stream()` option, and `resumeStream()` (so
`approveToolCall()` too) always sets it, so `agent-<name>` delegation
tools ran the sub-agent through `generate` and its chunks never reached
the stream as `tool-output`.
The fix records the method that started the run on the run registry
entry and rebuilds for that method. The same line in the Inngest durable
agent gets the same change.
The same hunk restores a second outcome: tool calls that a delegated
sub-agent makes on durable and evented runs report `subagent_tool_end`
again. The agent controller emits
`subagent_tool_start`/`subagent_tool_end` only while it reads the
sub-agent's stream (`agent-controller/tools.ts`) and uses
`subagent_tool_end` to set the tracked call's status
(`agent-controller/session.ts`), so while the delegation tool was built
for `generate` those rows never got a status or result. That is why the
`it.fails` pins on the durable and evented `delegated` cases in
`controller-tool-errors.scenario.test.ts` are removed; the changeset
says so too.
```diff
// prepareForDurableExecution
registryEntry = {
tools,
requestTools: tools,
+ methodType, // 'stream' or 'generate', from the call that started the run
...
}
// warm resume with requestContext / delegation / hooks
getToolsForExecution({
...
- methodType: resolvedOptions.closeOnSuspend ? 'generate' : 'stream',
+ methodType: entry.methodType ?? 'stream',
})
```
## Evidence
New shared aimock scenario
`packages/core/src/loop/test-utils/aimock/scenarios/subagent-resume-streaming.scenario.test.ts`.
A lead agent with a suspending `ask_user` tool and a sub-agent `helper`
(with its own tool) talk to a real OpenAI provider pointed at the aimock
HTTP server. The lead suspends, is resumed, then delegates to `helper`.
Each case checks that `helper`'s `tool-call`, `tool-result` and
`text-delta` chunks arrive on the resumed stream as `tool-output`, and
that every `helper` model request was a streaming request. Cases:
- `resumeStream()` with a `requestContext` (the #29436 path)
- `resumeStream()` without one (control)
- durable only: `resume()` with `requestContext` and `closeOnSuspend:
true`, as in the issue's repro
- durable only: a `generate()` run resumed with `resumeGenerate()` still
runs the sub-agent through `generate` (non-streaming requests)
- **Before (origin/main):** normal, durable, evented: `Tests 4 failed |
6 passed | 4 skipped (16)`. The 4 failures are the request-context cases
on durable and evented: `expected [] to deeply equal ArrayContaining
["tool-call","tool-result","text-delta"]`.
**After:** `Tests 10 passed | 4 skipped (16)`.
- **Inngest engine** (real dev server,
`AIMOCK_SCENARIOS='subagent-resume-streaming' pnpm --dir
workflows/inngest test:aimock`): with the Inngest line reverted, `2
failed | 2 passed (4)`, the same two request-context cases. With the
fix: `4 passed (4)`.
- Neighbouring tests, unchanged and green (`Tests 130 passed | 3
expected fail | 11 skipped`): `durable-agent-resume-request-context`,
`durable-agent-request-context`, `durable-agent-delegation`,
`durable-agent-generate`, and the `request-context-resume`,
`supervisor-suspension`, `supervisor-delegation-generate`,
`parity-sub-agent-suspend`, `parity-sub-agent-delegation`,
`subagent-auto-resume`, `resume-stream` scenarios.
- `tsc --noEmit -p tsconfig.build.json` for core, `tsc --noEmit` for
`@mastra/inngest`, oxlint on the touched files and prettier on the
changed files all pass.
- `controller-tool-errors.scenario.test.ts` pinned its durable and
evented `delegated` cases with `it.fails` because those delegations
emitted no `subagent_tool_end`. This fix makes them pass (CI reported
`Expect test to fail` on all 14), so the marker is removed: `Tests 84
passed (84)`.
Not covered: a cold resume (registry entry gone, rebuilt from storage)
still calls `prepare()` with the default `stream`, so a `generate()` run
resumed in another process gets stream-built tools. That was already the
case before this PR, and the tools still work under `resumeGenerate()`,
so it is left as-is.
## Merge Danger
**Door:** two-way. A one-field registry change plus two one-line
call-site changes, all easy to revert.
**Blast radius:** warm `DurableAgent`, `EventedAgent` and Inngest
durable resumes that pass a `requestContext`, `delegation` or `hooks`.
One behavior change to know about: `resumeGenerate()` on a run started
with `stream()` now rebuilds its tools for `stream`, matching the first
segment and the no-context resume. Other open PRs touch
`durable-agent.ts` / `types.ts` (#28283, #28530, #29036, #29282,
#27522), but none changes these lines.
Closes#29436
## Council review
Rounds: 2
| Seat | Round 1 | Round 2 |
| --- | --- | --- |
| Claude Opus 5.5 | not run | 9/10 (Correctness 9, Tests 9, Scope 10,
Code quality 9) |
| GPT-6.1 Sol | 9/10 (Correctness 9, Tests 9, Scope 10, Code quality 9)
| 9/10 (Correctness 9, Tests 9, Scope 10, Code quality 9) |
| Grok 4.7 | empty seat | empty seat |
Created with Claude Opus 5.5 in T3 Code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
When a paused run continues, it should keep working the way it started.
This change makes sub-agents keep sending progress updates after a
streamed run resumes. This holds even when the resume includes a
`requestContext` or `closeOnSuspend`.
## Summary
- Store the original method type (`stream` or `generate`) on each run
registry entry when the run is prepared.
- On warm resume, rebuild request-scoped tools with the stored method
type. If no method type is stored, use `stream`. Stop inferring the
method from `closeOnSuspend`.
- Apply the same change to the Inngest durable agent resume path.
- Add an AIMock scenario test for sub-agent streaming after resume, with
and without a request context. The test also checks that resumed
`generate()` runs make non-streaming helper requests.
- Remove the `it.fails` markers from the delegated-tool tests for the
durable and evented engines. These tests now run as ordinary tests.
- Add patch changesets for `@mastra/core` and `@mastra/inngest`. The
changesets reference issue `#29436`.
Scope:
- Cold resumes, which rebuild the run from storage, are not changed.
They still default to `stream`, including for a run that started with
`generate()`.
Test and CI results: not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
## Description
Scores from a `createInngestAgent` agent pointed at a span that is never
stored, so the agent run in a trace showed no scores. The score event
also lost the agent and the `runId`. The score now points at the run's
`agent_run` span, the same as for a plain `Agent`.
- The `execute-scorers` step passed its own span to the scorers. On
Inngest that span is internal and has no live parent, so the score
resolved to the internal workflow span.
- The step now passes the run's `agent_run` span, resolved the same way
as in the `map-final-output` step.
- Added one test to the shared scorer scenario. It runs on every engine
and failed on Inngest before the fix.
## Related Issue(s)
Fixes#29795
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update
## Checklist
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
Inngest now attaches agent scores to the saved agent-run span. Scores no
longer point to a span missing from the run trace.
## Changes
- Reuses or rebuilds the `agent_run` span before executing scorers.
- Uses the existing tracing context only when no agent span is
available.
- Adds all-engine test coverage for matching score and stored span IDs.
- Adds a patch changeset for `@mastra/inngest`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Opened by @roaminro
Fixes#29785
## Summary
On agents created with `createInngestAgent`, `streamUntilIdle()` (and
`stream(..., { untilIdle: true })`) closed right after
`background-task-completed`. The follow-up answer to a deferred
background task never reached the stream. Instead, a `background-task`
notification signal woke the thread as a separate run, and the answer
only reached memory. This was a regression from #28998.
**Root cause:** #28998 added `Agent.__notifyBackgroundTaskFinished()`,
which skips the wake when `__untilIdleHandlesBackgroundTask()` reports
an open until-idle stream. Core's `DurableAgent` overrides that check to
use its own registry. `createInngestAgent` keeps its own
`activeStreamUntilIdle` map but had no override. Its Proxy forwarded
both hooks to the wrapped `Agent` and bound them to it, and that agent's
registry is always empty, so every finished deferred task woke the
thread.
## Changes
- `@mastra/core`: export the existing `untilIdleHandlesBackgroundTask`
helper from `@mastra/core/agent/durable`.
- `@mastra/inngest`: define `__untilIdleHandlesBackgroundTask` and
`__notifyBackgroundTaskFinished` on the Inngest agent so both check its
own `activeStreamUntilIdle` registry. Both overrides are needed: because
of the proxy binding, an override of the check alone would never be
reached from the wrapped agent's notify.
- Regression tests in `create-inngest-agent.test.ts`: no wake while an
until-idle stream is open, and the thread still wakes when no stream is
open.
- Changeset (patch for both packages).
## Self-review
Approved at `29c10035db`.
| Check | Result | Tests |
|---|---|---|
| New #29785 regression tests (`create-inngest-agent.test.ts -t 29785`)
| ✅ Passed | 2/2 |
| Full `create-inngest-agent.test.ts` | ✅ Passed | 115/115 |
| `tsc --noEmit` for `workflows/inngest` | ✅ Passed | — |
| `scripts/validate-peerdeps.mjs` | ✅ Passed | — |
| Core `build:lib` (incl. d.ts) | ✅ Passed | — |
**Regression proof:**
- With the base `create-inngest-agent.ts` restored, the open-stream test
fails. The idle-path test still passes.
- Keeping only the `__untilIdleHandlesBackgroundTask` override and
dropping the notify override also fails (`sendSignal` called 1 time).
The tests catch each override on its own.
**Not run:**
- Inngest aimock until-idle scenarios: the Inngest dev server exited
with code 1 in the sandbox. This is an environment failure, not a pass.
- The live repro from the issue: it needs `OPENAI_API_KEY`.
**Note:** the peer range `>=1.83.0-0` also admits the already-published
1.83.0-alpha.0–2, which don't export the helper. This is the same
accepted alpha-window gap as #29635, and the peer validator passes.
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
## Description
Stacked on #29975. The Inngest durable agent's `step-finish` carried the
run's whole response message list on every step
(`_durableResponseMessages`), so its `agent.stream.<runId>` log grew
with the square of the step count, like core's DurableAgent did before
#29975.
It now publishes `_durableResponseMessagesDiff`, the same format as the
plain Agent and DurableAgent, and the shared stream adapter rebuilds the
full list for callers and observers:
- The diff is built inside the memoized emit step, which returns the
cursor, so Inngest replays reuse it instead of recomputing it; the
cursor travels on the iteration state like core's.
- The producer includes Inngest's function run id, so the first run and
each resume (or a rerun of the same Mastra run) start with the whole
list.
- Resume and `observe()` streams at an offset rebuild the skipped list
from the run's log first (`replayResponseMessagesBeforeOffset`).
- `diffResponseMessages` is exported from `@mastra/core/agent/durable`
for this; the `@mastra/core` peer floor stays `>=1.83.0-0`, as the stack
is meant to land before 1.83.0 stable.
Same upgrade note as #29975: a reader on an older `@mastra/core` reports
no response messages for steps published by this version.
## Related issue(s)
Completes COR-1532 (plain #29971, DurableAgent/EventedAgent #29975,
Inngest here). Related: #27475.
## Type of change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [x] Performance improvement
- [x] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
Tests: core's `agent-stream-log` scenario file runs on Inngest through a
real dev server (`AIMOCK_SCENARIOS='agent-stream-log'` with the
`e2e:workflows/inngest:aimock` project): 7 passed, 1 expected fail
(delegated-agent chunks, #29035). With the Inngest change reverted, the
early tool result is stored twice as often as the last one (14 vs 7).
`durable-resume-iteration-2-serve.test.ts` now checks that a stream
resumed after a suspension in iteration 2 and a full replay report the
same response messages.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
Inngest was saving the full list of response messages after every step,
even when most messages had not changed. This change saves only the
changed messages, while callers and observers still receive the full
list.
## Changes
- Inngest step-finish events now carry response-message diffs instead of
the full list.
- Diff generation runs inside a memoized Inngest step. Replays reuse the
diff, and the cursor travels with iteration state.
- Each Inngest function run starts a new diff sequence, including when a
run resumes.
- Resumed streams and `observe()` rebuild response messages from earlier
events when reading from an offset.
- Pubsub wrappers forward topic history replay decisions. When a bus
does not keep history, steps send whole lists.
- `@mastra/core/agent/durable` now exports `diffResponseMessages` and
the `ResponseMessagesCursor` type.
- Tests check reduced duplication, replay routing, and that resumed
response messages match observer output.
- The changeset warns that older `@mastra/core` readers may not report
response messages from these events. Upgrade agent processes and readers
together.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Description
Evented workflows rejected valid input. `start()` and `startAsync()`
replaced an omitted or `null` `inputData` with `{}` before validating it
against the workflow `inputSchema`, so a top-level schema default never
applied and a nullable schema rejected an explicit `null`. `stream()`
calls `start()`, so it had the same bug. Both start paths in
`packages/core/src/workflows/evented/workflow.ts` now pass `inputData`
through unchanged, which is what the default engine already does.
Follow-up fixes on the same input path (from review):
- The evented step executor stored a step's `payload` as `input ?? {}`.
Resume feeds that payload back into the step, so a first step started
with `null` or no input got `{}` on resume and a nullable or optional
schema rejected it. It now stores the input unchanged, as the default
engine does. Nested resume uses the stored payload whenever the step
record exists, instead of `payload ?? parentInput`.
- Time travel to the first step built that step's payload as `stored
payload ?? {}`, so a run started with `null` or no input replayed `{}`
on the evented engine and a nullable or optional schema rejected it. The
evented engine also ignored new `inputData` passed to `timeTravel()` for
the first step. `createTimeTravelExecutionParams` (shared by all
engines) now gives a first-entry target the same input the default
engine gives it: new `inputData` if passed (an explicit `null`
included), else the stored workflow input, kept as `null` or
`undefined`. A nested workflow in the first entry replays the stored
input too, and when the evented engine finds no child snapshot it seeds
the child's input from what the parent passes in. Other steps keep their
existing payload fallback.
- With `null` input, `context.input` is `null`. Two scans read `.status`
on it: the evented suspended-step list, and
`hydrateSerializedStepErrors` (shared with Inngest). A suspend crashed
`start()`, and a step failure left it waiting forever. Both now skip the
`null` entry.
```diff
// evented start() and startAsync()
-const inputDataToUse = await this._validateInput(inputData ?? ({} as TInput));
+const inputDataToUse = await this._validateInput(inputData);
```
```ts
const input = z.object({ count: z.number() }).default({ count: 5 });
// evented engine, before: throws "Invalid input data: count: expected number, received undefined"
// evented engine, after: success, the first step and getInitData() both see { count: 5 }
await run.start({});
```
### Evidence
New cases, added to the shared workflow suite
(`workflows/_test-utils/src/domains/schema-validation.ts`), run on the
default, evented and Inngest engines. They cover an omitted input with
an object default, an explicit value winning over the default, a scalar
default, an explicit `null` with a nullable schema, `getInitData()`
returning the defaulted input, omitted input with `validateInputs:
false` reaching the step as `undefined`, and missing required input
still being rejected without the step running. A focused evented file
(`start-input-validation.test.ts`) covers `startAsync()`, which the
shared harness doesn't expose, and asserts on the persisted run
snapshot. It also covers suspend and resume with `null` and omitted
input, a step failure with `null` input, and a nested step resumed with
its stored `null` input.
- **Review follow-ups, before** (commit 80b7fd6851, same tests): 4
failed. `Cannot read properties of null (reading 'status')`, `expected
{} to be undefined`, a 15s timeout on the failing step, and the nested
step resumed with `'seed'` instead of `null`. **After:** 9 passed in
that file.
- **Before** (origin/main evented `workflow.ts`, same tests): `vitest
run start-input-validation.test.ts evented-workflow.test.ts
workflow.test.ts -t 29424` gave **10 failed | 9 passed**. All the
failures were on the evented engine, for example `Invalid input data: -
: Invalid input: expected number, received object`.
**After:** **19 passed**.
- **Time travel, before** (head 765d1f564f): the shared time-travel
cases for `null`, omitted, new `inputData` and nested `{}`/`null` replay
failed on the evented engine with `Step input validation failed`, or
`expected 'failed' to be 'success'`. **After:** all 9 pass on the
default engine, the default engine with storage v2, and the evented
engine.
- The Inngest harness had no `timeTravel` helper, so every shared
time-travel case skipped there. With the helper added, all 27 shared
time-travel cases pass on Inngest. Three of Inngest's older time-travel
tests (suspend and resume a time-travelled run, time travel a suspended
run, conditional chains) fail on this branch both with and without the
time-travel change; their diffs are about `resumePayload: undefined`
keys and a conditional sibling marked `skipped`, which this PR doesn't
touch.
- Inngest, whole Schema Validation domain: 13 passed, 1 expected failure
(see below).
- Regression check: every test in `src/workflows/evented/`, every core
test file that calls `timeTravel`, plus `workflow.test.ts`,
`storage-v2-suite.test.ts` and `utils.test.ts` passes (57 files, 1691
passed, 14 skipped). `pnpm --filter ./packages/core check` is clean.
`workflows/_test-utils` typecheck reports no errors in the touched
files; it has 14 existing errors elsewhere, the same on main. oxlint and
oxfmt are clean on the changed files.
- Built with a hill climb: the failing tests were locked first, and the
score went from 10 to 0 in one source step.
Inngest already passed input through unchanged, but the new shared cases
found two Inngest bugs:
- `getInitData()` returns `undefined` instead of the defaulted input
when `inputData` is omitted. Cause not yet investigated. Pinned with
`it.fails` (`schemaInitDataDefaultFails`).
- An explicit `null` input crashed the status poll in
`hydrateSerializedStepErrors`. Fixed by the shared `null` check above,
so that case now passes without a pin.
### Merge Danger
**Door:** two-way. Reverting the commit restores the old fallback.
**Blast radius:** callers of evented `start()`, `startAsync()` and
`stream()` that omit `inputData`. A workflow whose bare object schema
(no default) passed only because of the evented-only `{}` fallback now
fails validation, as it already does on the default engine. With no
`inputSchema` or with `validateInputs: false`, the first step now
receives `undefined` instead of `{}`. Evented step results now record a
`null` or `undefined` `payload` as is, instead of `{}`, as the default
engine does; code that reads `steps[id].payload` on the evented engine
can see those values. The changeset tells those callers to pass
`inputData: {}` or add a schema default. Scheduled runs are unchanged,
because the scheduler already sends `inputData ?? {}`. Nested workflows
don't go through `start()`. Time travel to the first step now replays a
stored `null` or `undefined` instead of `{}`, and on the evented engine
uses new `inputData` passed to `timeTravel()`, as the default engine
already did.
### Follow-ups (not in this PR)
- The Inngest `getInitData()` bug above needs its own issue.
- On every engine, time travel to a later step whose previous step
returned `null` or `undefined` still replays `{}`, and a falsy
`inputData` passed to `timeTravel()` for a step after the first one is
ignored (the `inputData &&` check in `createTimeTravelExecutionParams`).
Both predate this PR. For the first step, an explicit `null` `inputData`
now replaces the stored input.
## Related issue(s)
Closes#29424
## Type of change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable). Not applicable: the docs already say validation applies
schema defaults and `inputData` is optional.
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
## Council review
Rounds: 4
| Seat | Round 1 | Round 2 | Round 3 | Round 4 |
| --- | --- | --- | --- | --- |
| Claude Opus 5.5 | not run | 9/10 | 9/10 | 9/10 |
| GPT-6.1 Sol | 9.5/10 | 9.5/10 | 9/10 | 9/10 |
| Grok 4.7 | empty seat | empty seat | empty seat | empty seat |
Every seat scored at least 9 in each category (correctness, tests,
scope, code quality) in rounds 2 to 4. Findings applied: clear the
`startAsync` test's timeout timer, describe the no-schema and
`validateInputs: false` change in the changeset, and add a shared case
for that change. Findings not applied:
- Add an Inngest branch to the required-input rejection case. That case
follows the same `schemaValidationThrows` skip as the existing
invalid-input cases.
- Persist an explicit `null` in the evented initial snapshot. The final
persisted run already stores `context.input = null`, and that assertion
passed without a code change.
Time-travel follow-up review (GPT-6.1 Sol, Correctness / Tests / Scope /
Code quality): round 1 8/8/10/9 (added the Inngest `timeTravel` helper),
round 2 7/8/10/9 (fixed a nested time-travel regression it found), round
3 9/9/10/9 with no findings.
Created with Claude Opus 5.5 in T3 Code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
Workflows now pass the input you provide—or leave it missing—through
validation instead of changing it to an empty object first. This lets
schemas apply defaults or accept `null`, and helps workflows keep the
same input when they resume or replay.
## Changes
- Evented `start()` and `startAsync()` pass input directly to schema
validation. `stream()` uses `start()`.
- Step payloads preserve `null` and `undefined`. Resume paths use stored
payloads, including for nested steps.
- Time travel to a workflow’s first step preserves its input. Supplied
`inputData` can replace the stored input. For a nested workflow without
a saved snapshot, replay uses the parent step’s successful output as the
child input.
- Null-safe status checks handle missing step results and context
entries. A step failure after a `null` start returns a failed result
instead of leaving `start()` waiting.
- Added evented and shared tests for defaults, nullable input, invalid
input, resume, time travel, `getInitData()`, and validation-disabled
behavior.
- Added a patch changeset that documents the behavior and compatibility
impact.
When no input schema is configured or validation is disabled, omitted
input reaches the first step as `undefined`. Callers that need `{}` can
pass it explicitly or add a schema default. The shared suite marks the
Inngest `getInitData()` default-input case as an expected failure.
**Test and check results:** not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.
## @mastra/factory@0.28.0-alpha.6
## @mastra/code-sdk@1.19.0-alpha.6
## @mastra/playground-ui@69.0.0-alpha.7
## @mastra/client-js@1.60.0-alpha.6
## @mastra/react@1.15.0-alpha.6
## @mastra/deployer-cloud@1.83.0-alpha.6
## @mastra/longmemeval@1.1.42-alpha.6
## @mastra/opencode@0.1.42-alpha.6
## create-factory@0.4.3-alpha.6
## mastracode@0.50.0-alpha.6
## @mastra/agent-builder@1.2.1-alpha.3
## mastra@1.39.0-alpha.6
## @mastra/core@1.83.0-alpha.6
## @mastra/deployer@1.83.0-alpha.6
## @mastra/editor@0.17.1-alpha.4
## @mastra/mcp@2.9.0-alpha.3
## @mastra/mcp-docs-server@1.3.13-alpha.9
## @mastra/memory@1.41.1-alpha.3
## @mastra/rag@2.7.3-alpha.2
## @mastra/schema-compat@1.3.18-alpha.0
## @mastra/server@1.83.0-alpha.6
## @mastra/elysia@0.1.23-alpha.6
## @mastra/express@1.5.27-alpha.6
## @mastra/fastify@1.5.27-alpha.6
## @mastra/hono@1.7.25-alpha.6
## @mastra/koa@1.7.27-alpha.6
## @mastra/nestjs@0.3.0-alpha.6
## @mastra/next@0.2.41-alpha.6
## @mastra/server-adapters-test-suite@0.3.0-alpha.3
## @mastra/tanstack-start@0.2.41-alpha.6
## @mastra/upstash@1.6.1-alpha.0
## @mastra/vectorize@1.1.5-alpha.0
## @mastra/voice-google-gemini-live@0.14.19-alpha.0
## @mastra/voice-openai-realtime@0.14.9-alpha.0
## @mastra/voice-xai-realtime@0.2.18-alpha.0
## @mastra/inngest@1.11.2-alpha.4
## @mastra/temporal@0.6.0-alpha.6
## create-mastra@1.39.0-alpha.6
## @internal/playground@1.39.0-alpha.6
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Description
`run.timeTravel()` treated `false`, `0`, `''` and `null` resume data as
missing. The suspended step saw `undefined` and suspended again, even
when the value matched its `resumeSchema`. `run.resume()` already
delivered these values. This PR replaces three truthiness checks in
`@mastra/core` with presence checks:
```diff
// packages/core/src/workflows/utils.ts — validateStepResumeData
- if (!resumeData) {
+ if (resumeData === undefined) {
return { resumeData: undefined, validationError: undefined };
// packages/core/src/workflows/utils.ts — createTimeTravelExecutionParams
// (stopped collecting the suspended step's result when time travel starts at an earlier step)
- if (currentExecPathLength > 0 && !resumeData) {
+ if (currentExecPathLength > 0 && resumeData === undefined) {
// packages/core/src/workflows/evented/workflow-event-processor/index.ts
// (the default engine already uses !== undefined here, handlers/step.ts)
- if (timeTravelResumeData && !timeTravelResumeValidationError) {
+ if (timeTravelResumeData !== undefined && !timeTravelResumeValidationError) {
```
Omitted `resumeData` still leaves the step suspended. A falsy value that
fails the `resumeSchema` is still logged and not delivered. `null` now
reaches a step whose schema allows it, as it does with `run.resume()`.
### Evidence
New cases in the shared time-travel suite
(`workflows/_test-utils/src/domains/time-travel.ts`) start a run with a
fixed `runId`, wait for it to suspend, then time-travel that run with
`false`, `0`, `''` and `null`. Each value is tried at the suspended step
and from the step before it, plus controls for omitted and
schema-invalid values. They run on the default engine
(`workflow.test.ts`), the evented engine with real workers and
`EventEmitterPubSub` (`evented/evented-workflow.test.ts`), and
`storage-v2-suite.test.ts`, all on in-memory storage. A separate file,
`timetravel-nested-resume-data.test.ts`, covers steps inside a nested
workflow.
- **Before** (fix reverted): `vitest run workflow.test.ts
evented-workflow.test.ts storage-v2-suite.test.ts
timetravel-nested-resume-data.test.ts -t 29390` gives 28 failed, 9
passed, 5 expected fail. Every failure is `expected [ undefined ] to
deeply equal [ false | 0 | '' | null ]`: 24 flat cases plus 4 nested
ones on the default engine.
**After:** 37 passed, 5 expected fail, 0 failed.
- Existing time-travel and workflow suites (`utils`,
`timetravel-divergence`, `timetravel-foreach`, both
`concurrent-time-travel` files, `default`, `workflow`,
`evented-workflow`, `storage-v2-suite`): 1177 passed, 5 expected fail, 0
failed. `tsc --noEmit` for core and a strict typecheck of the new test
file both pass, and so do `oxlint` and `oxfmt` on the changed files.
- Not covered: Inngest calls the same core
`createTimeTravelExecutionParams` and default step handler, so it gets
the fix, but its suite sets `timeTravel: false` and skips the shared
time-travel domain. Nested time travel on the evented engine never
delivers resume data to the inner step, truthy or falsy, before or after
this change. That's a separate, pre-existing bug, not yet investigated,
and the 5 evented nested cases are pinned with `it.fails`.
- Added after CodeRabbit review: `evented/step-executor.ts` used `if
(params.resumeData)` when it records `resumePayload` and `resumedAt` on
the step result, so a falsy resume value was left off the stored result.
It now checks `!== undefined`. New `step-executor.test.ts` cases for
`false`, `0`, `''` and `null` fail 4/4 before and pass after; the whole
`src/workflows/evented` directory passes (43 files, 695 tests). This
doesn't add crash recovery for a time-travelled step: time travel still
writes no running record before the step runs, which is a separate,
pre-existing gap.
### Merge Danger
**Door:** two-way. Reverting the commit restores the old checks, and
nothing persisted changes shape.
**Blast radius:** `run.timeTravel()` with `resumeData` on the default,
evented and Inngest engines. A caller who passed `resumeData: null` to
mean "no data" now has `null` reach a step whose schema accepts it.
## Related issue(s)
Closes#29390
## Type of change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [x] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable). No docs change is needed: the `timeTravel` reference
already says it accepts resume data without restricting it to truthy
values.
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
## Council review
Scores out of 10: Correctness / Tests / Scope / Code quality.
| Round | GPT-6.1 Sol | Claude Opus 5.5 | Grok 4.7 |
|---|---|---|---|
| 1 | 9 / 9 / 10 / 7 | not seated yet | empty seat |
| 2 | 10 / 9 / 10 / 10 | not seated yet | empty seat |
| 3 | 10 / 9 / 10 / 10 | 9 / 8 / 10 / 9 | empty seat |
| 4 | 10 / 8 / 10 / 9 | 9 / 9 / 10 / 9 | empty seat |
| 5 | 10 / 9 / 10 / 10 | 9 / 9 / 10 / 9 | empty seat |
Created with Claude Opus 5.5 in T3 Code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
`run.timeTravel()` treated some valid answers as if the user had not
answered. This PR lets suspended steps receive valid values such as
`false`, `0`, an empty string, and `null`.
## What changed
- Updated time-travel validation and graph traversal to treat only
`undefined` as missing resume data.
- Updated the evented step executor to record defined resume data in
`resumePayload` and set `resumedAt`.
- Added regression tests for falsy resume values from the suspended step
and a preceding step.
- Added a patch changeset for `@mastra/core`.
## Known limitation
Evented workflows still do not deliver time-travel resume data to steps
inside nested workflows. Tests for this behavior remain marked as
expected failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
## Summary
On durable, evented and Inngest agents, a custom `stopWhen` received
each step's `toolResults` as Mastra tool-call payloads (`{ toolCallId,
toolName, args, result }`, no `type`). `stopWhen` is typed as an AI SDK
stop condition, so a predicate that read `toolResults[i].output` or
checked `type === 'tool-result'` never matched and the run kept going.
Those engines now pass the same AI SDK tool-result parts the default
engine does.
The new `durableStopConditionSteps` helper in `@internal/core/durable`
builds `toolResults`, `staticToolResults` and `dynamicToolResults` from
each step's tool-result content parts, after restoring any outputs that
compaction dropped. Only the view handed to `stopWhen` changes. Stored
step records keep their payload-shaped `toolResults`, because snapshot
compaction reads them.
```diff
// durable-loop-builder.ts (durable + evented) and create-inngest-agentic-workflow.ts (Inngest)
-steps: expandDurableSteps(state.accumulatedSteps)
+steps: durableStopConditionSteps(state.accumulatedSteps)
// expandDurableSteps(steps), then per step with content:
// toolResults = content.filter(part => part.type === 'tool-result')
// staticToolResults = toolResults.filter(part => part.dynamic === false)
// dynamicToolResults = toolResults.filter(part => part.dynamic === true)
```
That mirrors how the default loop builds its steps
(`packages/core/src/loop/loop-builder.ts`). Step records from snapshots
written before steps carried `content` are returned as stored, as the
issue allows.
## Evidence
Test changes in `stopwhen-step-results.scenario.test.ts`, which runs on
every engine against a real HTTP mock LLM:
- The existing `it.fails` case for #28519 is now a plain `it`.
- New case: a predicate that matches on `toolResults[i].output` stops
the run after step 1.
- New case: `toolResults` equal each step's tool-result content parts,
including `input` and the static/dynamic views.
- New case: a throwing tool reaches the predicate as a tool-result part
whose output carries the error.
- New case: an output processor drops tool-result chunks from the stream
(durable records them as `resultHidden`), and the predicate still sees
those results.
There is also a unit test for the helper in
`packages/_internal-core/src/durable/index.test.ts`. It covers restored
outputs, unchanged stored records, records without content, and the
static/dynamic split.
- **Before:** with only the call site reverted to `expandDurableSteps`,
the core scenario run has `Tests 10 failed | 38 passed (48)`. All 5
cases above fail on `durable` and on `evented`. On Inngest, the scenario
run against a real Inngest dev server had `Tests 2 failed | 7 passed
(9)` before the Inngest call site changed.
- **After:**
- Core scenario (normal, durable, evented, fs): `Tests 48 passed (48)`
- Inngest scenario (real dev server): `Tests 12 passed (12)`
- Helper unit test: `Tests 4 passed (4)`
- **Guards, all on the pushed head:**
- Related stopWhen, maxSteps and onFinish scenarios plus all of
`packages/core/src/agent/durable`: `Tests 2117 passed | 18 skipped`
- Inngest workflow and stopWhen serve tests: `27 passed`
- Inngest runs of the related stopWhen scenarios: `22 passed | 4
skipped`
- `tsc --noEmit` for `@mastra/core`, `@internal/core` and
`@mastra/inngest`, and `oxlint --type-aware` on the touched files: all
clean.
Not covered: a processor that holds a tool result and releases it in a
later step. The `stopWhen` view comes from step `content`, and releasing
a held result only clears `resultHidden` on the stored records, so it
does not change what the predicate sees.
## Merge Danger
**Door:** two-way. This changes only the object passed to user
`stopWhen` predicates. Stored snapshots and other step consumers
(`onFinish`, processors, `stream.steps`) are untouched.
**Blast radius:** custom `stopWhen` predicates on durable, evented and
Inngest agents. A predicate on those engines that read the old internal
fields (`args`, `result`) on `toolResults` will no longer find them.
Those fields were never part of the documented AI SDK shape, and the
default engine never had them.
Closes#28519
## Council review
| Round | Sol (gpt-6.1-sol) | Opus (claude-opus-5-5) | Grok 4.7 |
|---|---|---|---|
| 1 | Correctness 9, Tests 8, Scope 10, Quality 9 | not run | empty seat
|
| 2 | Correctness 10, Tests 9, Scope 10, Quality 10 | not run | empty
seat |
| 3 | Correctness 9, Tests 9, Scope 10, Quality 9 | Correctness 9, Tests
9, Scope 10, Quality 9 | empty seat |
Round 1 findings were applied: cross-engine cases for tool errors,
processor-hidden results, `input` and the static/dynamic views, and a
typed helper return. The round 2 nit with mixed `dynamic` flags was
applied. Not applied:
- **Delayed-release scenario:** it doesn't change the content-derived
view.
- **Mapping legacy no-content records to parts:** the issue allows
keeping their current shape.
- **AI SDK-style `dynamic !== true` static split:** it would break
parity with the default engine, so it belongs in a separate change to
both engines.
Created with Claude Opus 5.5 in T3 Code.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
Some agents gave stop rules tool results in a format those rules could
not read. This change gives the rules the format they expect, so they
can use a tool’s output to decide when to stop.
## Changes
- Added `durableStopConditionSteps` to build `toolResults` from each
step’s content in the AI SDK format.
- The helper also builds `staticToolResults` and `dynamicToolResults`.
When content is present, it restores outputs removed by compaction. When
content is absent, it returns the stored step unchanged.
- Durable, evented, and Inngest engines now use the helper for
stop-condition callbacks. Stored step records remain unchanged.
- Updated tests to cover result shapes, output-based conditions, tool
errors, static and dynamic results, processor-hidden results, and legacy
records. Delayed release of a held tool result is not covered.
## Compatibility
Custom predicates that rely on the old `args` or `result` fields in
`toolResults` will no longer find those fields in the `stopWhen` view.
Use `input` and `output` instead.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.
## @mastra/auth-studio@1.4.0-alpha.1
## @mastra/factory@0.28.0-alpha.5
## @mastra/mcp-apps-components@0.1.0-alpha.5
## @mastra/playground-ui@69.0.0-alpha.6
## @mastra/client-js@1.60.0-alpha.5
## @mastra/react@1.15.0-alpha.5
## @mastra/deployer-cloud@1.83.0-alpha.5
## @mastra/longmemeval@1.1.42-alpha.5
## @mastra/opencode@0.1.42-alpha.5
## create-factory@0.4.3-alpha.5
## @mastra/code-sdk@1.19.0-alpha.5
## mastracode@0.50.0-alpha.5
## @mastra/agent-builder@1.2.1-alpha.2
## mastra@1.39.0-alpha.5
## @mastra/core@1.83.0-alpha.5
## @mastra/deployer@1.83.0-alpha.5
## @mastra/editor@0.17.1-alpha.3
## @mastra/mcp@2.9.0-alpha.2
## @mastra/mcp-docs-server@1.3.13-alpha.8
## @mastra/memory@1.41.1-alpha.2
## @mastra/rag@2.7.3-alpha.1
## @mastra/server@1.83.0-alpha.5
## @mastra/elysia@0.1.23-alpha.5
## @mastra/express@1.5.27-alpha.5
## @mastra/fastify@1.5.27-alpha.5
## @mastra/hono@1.7.25-alpha.5
## @mastra/koa@1.7.27-alpha.5
## @mastra/nestjs@0.3.0-alpha.5
## @mastra/next@0.2.41-alpha.5
## @mastra/server-adapters-test-suite@0.3.0-alpha.2
## @mastra/tanstack-start@0.2.41-alpha.5
## @mastra/libsql@1.30.0-alpha.2
## @mastra/turso@0.1.21-alpha.2
## @mastra/inngest@1.11.2-alpha.3
## @mastra/temporal@0.6.0-alpha.5
## create-mastra@1.39.0-alpha.5
## @internal/playground@1.39.0-alpha.5
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Summary
Agents created with `createInngestAgent` could not stream an
observation. The first read of `observe(runId).fullStream` threw
`TypeError: Cannot read private member #options from an object whose
class did not declare it`.
`observe()` replaces `fullStream` with a getter that detaches the
observer when the stream ends. That getter called the base `fullStream`
getter with `output` as the receiver. `output` is the destructurable
proxy, and a proxy can't reach the stream's private `#options` field.
The getter now types its receiver (`get(this: typeof output)`) and calls
`baseFullStream.call(this)`. `this` is the real stream instance, because
the proxy reads getters with its target as receiver. Core's
`DurableAgent.observe()` already does it this way.
Linear: COR-1525
## Changes
- `workflows/inngest/src/durable-agent/create-inngest-agent.ts`: one
getter hunk. The base `fullStream` getter is now called on the real
instance instead of the proxy.
- `workflows/inngest/src/__tests__/inngest-agent-observe.test.ts`: new
regression test. It runs a real Inngest dev server (v1.49.0), and AIMock
is the only mock, standing in for the model over HTTP. The test runs a
stream to completion, replays it through `observe(runId).fullStream`,
and checks the replayed text and that the last chunk is `finish`.
- `.changeset/inngest-observe-fullstream-receiver.md`: `@mastra/inngest`
patch.
No dependency, peer-range, checker, workflow, lifecycle or output-path
changes.
## Proof
**Author (session build, base `66c620c0`, all runs with retries off):**
- The new test fails on `main` with the `#options` TypeError. With the
fix it passed 4 of 4 runs. Reverting only the production hunk makes it
fail again with the same error.
- Existing core aimock `observe-*` scenarios on the Inngest engine:
- `main`: 13 failed, 1 passed, 1 expected fail, 6 skipped.
- This fix: 14 passed, 1 expected fail, 6 skipped. I ran it 3 times and
got the same result each time.
- The expected fail is the existing Inngest `it.fails` for run-log TTL,
which has a different cause.
- `create-inngest-agent.test.ts`: 113/113 passed.
- `tsc --noEmit` (@mastra/inngest and the new test): passed. oxlint
`--type-aware` on the changed files: 0 warnings and 0 errors.
**Independent review (separate session, exact head `3bcdda3f`, runtime
freshly rebuilt):**
- New regression test, red → green → restore: it passes 1/1 on head and
fails with the exact `#options` TypeError when only the production hunk
is reverted. Restoring the hunk makes it pass again.
- Core aimock `observe-*`, `observer-outcomes`,
`parity-until-idle-replay` and `resume-stream` through Inngest: 27
passed, 1 expected fail, 8 skipped. On base, the observe, replay and
approval-resume scenarios had 13 failures (`observe-approval-resume` 1,
`observe-lifecycle` 8, `observe-reattach` 4). On head they had none.
## Unverified
- All tests ran the workspace source with a freshly built core `dist`.
None ran against packed npm artifacts.
- I haven't checked whether the HTTP `/observe` server route reaches
this getter.
- I make no claim about which earlier change introduced the bug.
- I make no claim about cancel or abort runtime behavior beyond what the
existing observe scenarios cover.
- The full `@mastra/inngest` e2e suite and the full aimock scenario
matrix were not run.
## Risk
Low. The change only affects which object the existing wrapper calls the
base getter on. Detach-on-error and the rest of the stream wrapping are
unchanged.
Unrelated: `create-inngest-agent.ts:2275` already fails Prettier on
`main`, outside this hunk.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
The PR fixes replaying a finished run from an Inngest agent. It makes
the stream use the correct object when it opens, so reading the replay
no longer fails.
## Changes
- Updated `observe()` so the `fullStream` getter uses its actual
receiver when opening the stream.
- Added an integration regression test that runs an agent and replays
its completed stream. The test checks the reply text and final `finish`
chunk.
- Added a patch changeset for `@mastra/inngest`.
The PR description reports that the regression test and other listed
checks passed. Full Inngest end-to-end tests and the full aimock
scenario matrix were not run.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.
## @mastra/slack@1.9.0-alpha.0
## @mastra/factory@0.28.0-alpha.3
## @mastra/code-sdk@1.19.0-alpha.3
## mastracode@0.50.0-alpha.3
## @mastra/observability@1.24.0-alpha.2
## @mastra/core@1.83.0-alpha.3
## @mastra/mcp-apps-components@0.1.0-alpha.3
## @mastra/playground-ui@69.0.0-alpha.4
## @mastra/server@1.83.0-alpha.3
## @mastra/server-adapters-test-suite@0.3.0-alpha.1
## @mastra/platform-workspace@1.13.0-alpha.0
## @mastra/auth-better-auth@1.1.11-alpha.1
## @mastra/client-js@1.60.0-alpha.3
## @mastra/react@1.15.0-alpha.3
## @mastra/deployer-cloud@1.83.0-alpha.3
## @mastra/longmemeval@1.1.42-alpha.3
## @mastra/opencode@0.1.42-alpha.3
## create-factory@0.4.3-alpha.3
## @mastra/arize@1.3.30-alpha.2
## @mastra/arthur@0.4.30-alpha.2
## @mastra/braintrust@1.3.27-alpha.2
## @mastra/datadog@1.4.22-alpha.2
## @mastra/deepeval@0.1.24-alpha.2
## @mastra/laminar@1.3.32-alpha.2
## @mastra/langfuse@1.6.6-alpha.2
## @mastra/langsmith@1.4.3-alpha.2
## @mastra/otel-bridge@1.5.22-alpha.2
## @mastra/otel-exporter@1.5.2-alpha.2
## @mastra/posthog@1.4.7-alpha.2
## @mastra/sentry@1.2.32-alpha.2
## @mastra/agent-builder@1.2.1-alpha.1
## mastra@1.39.0-alpha.3
## @mastra/connect@1.3.2-alpha.2
## @mastra/deployer@1.83.0-alpha.3
## @mastra/editor@0.17.1-alpha.2
## @mastra/fastembed@1.3.7-alpha.0
## @mastra/mcp-docs-server@1.3.13-alpha.6
## @mastra/memory@1.41.1-alpha.1
## @mastra/elysia@0.1.23-alpha.3
## @mastra/express@1.5.27-alpha.3
## @mastra/fastify@1.5.27-alpha.3
## @mastra/hono@1.7.25-alpha.3
## @mastra/koa@1.7.27-alpha.3
## @mastra/nestjs@0.3.0-alpha.3
## @mastra/next@0.2.41-alpha.3
## @mastra/tanstack-start@0.2.41-alpha.3
## @mastra/convex@1.8.1-alpha.0
## @mastra/pg@1.36.0-alpha.2
## @mastra/inngest@1.11.2-alpha.2
## @mastra/temporal@0.6.0-alpha.3
## create-mastra@1.39.0-alpha.3
## @internal/playground@1.39.0-alpha.3
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Description
zod v3 schemas work at runtime but some public types rejected them or
inferred `unknown`, and `UIMessage[]` from `ai@7.0.163` stopped
type-checking against core's vendored v7 types.
```ts
import { z } from 'zod/v3';
const schema = z.object({ answer: z.string() });
createStep(agent, { structuredOutput: { schema } }); // before: No overload matches this call
new Agent({ ...config, defaultOptions: { structuredOutput: { schema } } }); // before: TS2322
(await agent.generateLegacy('hi', { output: schema })).object; // before: unknown, now { answer: string }
await client.getAgent('a').generate('hi', { structuredOutput: { schema } }); // before: TS2590
import type { UIMessage } from 'ai'; // 7.0.163, has SteeringMessageUIPart
await agent.generate(uiMessages as UIMessage[]); // before: 'SteeringMessageUIPart' is not assignable
```
The zod v3 fixes point the user-facing option types at `PublicSchema` /
`PublicAgentExecutionOptions` (the runtime already converts with
`toStandardSchema`). `inferOutput` gets a zod v3 branch that reads
`_output` structurally, like `InferPublicSchema` in schema-compat, so a
v3 schema from a different zod install than core's is inferred too.
Client-js `StreamParamsBase` is built on `AgentExecutionOptionsBase`,
which avoids the TS2590.
For the v7 messages, `MessageInput` also accepts a v7 `UIMessage` whose
parts only need a string `type`. Core can't know the shape of part types
released after its vendored copy, so those parts are deliberately loose.
Inline object literals are still checked against the known part types,
because excess property checks rule out the loose member for them.
Mastra doesn't process the new part types yet.
`AIV6Adapter.fromUIMessage` paired parts with its v5 bridge output by
index, so a part the bridge drops (an unknown type, or an empty text
part) shifted every later part. It now drops those before pairing.
The PR also fixes two tests that failed CI on `main`: the stored board
runtime test still imported `factoryMemorySettingsUserId` after #29366
renamed it, and the redactor leak check matched `4111` inside a random
message UUID.
Type tests cover each case in the core and inngest type-test files, and
they fail with the fix reverted. The e2e client-js fixture imported
`zod/v3` and `zod/v4`, which don't resolve there, so its schemas were
`any`. This PR fixes those imports. The e2e type-check run still ignores
its exit code (`e2e-tests/type-check/test.ts:45`), which is left for a
separate PR.
## Related issue(s)
Fixes#29574Fixes#29575
## Type of change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update
## Checklist
- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
This PR lets TypeScript accept Zod v3 schemas and newer AI SDK v7
messages in Mastra’s public APIs. It also fixes message-part matching so
ignored parts do not change how later parts are handled.
## Summary
- Updated agent and workflow option types to accept public schemas,
including Zod v3 schemas. Added Zod v3 output inference for
`generateLegacy`.
- Added forward-compatible input types for newer AI SDK v7 UI message
parts. Mastra accepts these messages, but does not yet process unknown
part types.
- Updated the AI v6 adapter to filter unsupported parts and empty text
parts before matching converted message parts.
- Updated client-js stream parameter types to avoid a complex TypeScript
union. Corrected Zod imports in a type-check fixture and added
regression type tests.
- Updated `getLastMessage` to use the last part only when it contains
nonempty string text.
- Added patch changesets for `@mastra/core`, `@mastra/client-js`, and
`@mastra/inngest`.
**Diff:** 19 files changed; 191 additions and 34 deletions.
**CI and test results:** not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## Summary
Two related problems in the durable agent stream surfaced while fixing
stale `@mastra/inngest` tests.
**A publish failure stalls a durable run for ten seconds.**
`CachingPubSub.publish` writes each frame to the replay log before it
publishes to the inner transport. When that publish rejects, the frame
stays in the log with its index — late subscribers can still replay it —
but the live subscription will never deliver it. `waitForLoggedEvents`
waited for that index until the pending-frame deadline
(`LOGGED_EVENT_DELIVERY_TIMEOUT_MS`, 10s in
`packages/core/src/agent/durable/stream-adapter.ts`), and `fail()` calls
`waitForEventDelivery()` before it delivers the error chunk. So a broken
transport delayed a run's error by the full watchdog: with a `publish`
that always throws, `stream`'s first chunk arrived at ~10.0s and
`generate` rejected at ~10.0s.
`CachingPubSub` now records the log indices whose live publish threw and
exposes them via `getUndeliveredIndices(topic)` (dropped in
`clearTopic`); `waitForLoggedEvents` skips those indices when computing
its wait target. The frame stays in the log for replay — only the wait
changes.
**Four `@mastra/inngest` tests assert durable-run behavior that has
since changed**, so `workflows/inngest` fails on `main` even though the
product is fine otherwise:
- **Publish-failure termination** (`create-inngest-agent.test.ts`,
stream + generate): previously raised the `Promise.race` budget to 15s
so the 10s watchdog could elapse. With the core fix the original 1s
budget bounds a genuine hang again, and the test fails if the stall
returns.
- **Resumed-chunk parity** (`resume() starts after cached history`,
`resumeGenerate() returns the resumed result`): durable delivery stamps
each chunk with its pubsub log index so late subscribers can order
frames, so exact chunk arrays are compared with the index stripped. The
two negative assertions had the same problem and could never have
failed; they now compare index-stripped chunks too.
- **Persisted request context** (`resume-runtime-rehydration.test.ts`):
the durable snapshot always records the run's scope set —
`mastra__scopes: []` included — so a cold resume can tell "held no
scopes" apart from a pre-upgrade snapshot
(`packages/core/src/agent/durable/preparation.ts`). The expectation
includes the marker.
- **Mocked suspended snapshot** (same file): the fixture predates the
resume guard that requires `status === 'suspended'`, so `resume()`
rejected it as "not suspended (no snapshot found)". The fixture now sets
a status.
The suite does not run in CI: the affected job runs only `--project
'unit:*' --project 'typecheck:*'` (`.github/workflows/test-suite.yml`),
and this package declares its suites as `e2e:workflows/inngest*`, so
only typechecking covers it. That is deliberate — those suites spawn
Inngest dev servers and are too slow for the affected-test job. Locally,
the package's plain `pnpm test` also aborts on a vitest project-config
conflict, so run the files directly.
## Test plan
```bash
pnpm exec vitest run --no-isolate --retry=1 --project e2e:workflows/inngest <file>
```
Run per file: this package's suites spawn dev servers on a shared port
and are unreliable as one serial invocation.
Branch rebased on `main` at `ed1cecba5c` (post-#29635):
- `src/__tests__/create-inngest-agent.test.ts` — 113 passed; the two
publish-failure tests now complete in ~15ms each instead of waiting out
the 10s deadline
- `src/resume-runtime-rehydration.test.ts`, `src/pubsub.test.ts` — 11
passed
- `packages/core`: `src/events` + `src/agent/durable/__tests__` — 324
files, 3356 passed / 4 skipped, no type errors
- `tsc -p workflows/inngest/tsconfig.json` clean; oxlint clean on the
changed files
- `pnpm turbo build --filter=@mastra/core --filter=@mastra/inngest` —
31/31
Before the core fix, the same two tests fail on the original 1s budget
and pass only when it is raised past the 10s watchdog.
Co-Authored-By: mastracode
<284800079+mastra-platform[bot]@users.noreply.github.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
When the event transport rejects a frame, a durable run no longer waits
up to ten seconds for a frame that was never delivered. The frame stays
in history for replay, and the tests now account for saved resume data
and changing log indices.
## Changes
- `CachingPubSub` records the indices of frames that were cached but
rejected by the live transport. `waitForLoggedEvents` skips those
indices while waiting for delivery.
- Added tests for rejected publishes, retained history, index tracking,
clearing tracked indices, and prompt durable-stream error handling.
- Resume comparisons ignore log indices. The rehydration test expects
`mastra__scopes: []` and a `suspended` workflow snapshot.
## Validation
- The supplied PR report says core event and durable-agent tests passed:
3,358 passed and 4 skipped.
- The report also says the Inngest tests, type checks, lint, and build
passed. The Inngest suites are not run by the affected-test CI job.
- Current-head CI results: not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: review <review@local>
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
## Summary
Extract reusable run-abort and cleanup helpers for durable agents, and
make Inngest reuse core's `AgentStreamBus` instead of maintaining its
own replay-cache/bus initialization.
This is the behavior-neutral shared-session lane (roadmap PR 1 /
COR-1509). It is independent of
[#29600](https://github.com/mastra-ai/mastra/pull/29600), which changes
server authorization organization. Neither branch includes the other.
- Replace four repeated DurableAgent abort-controller setups and two
Inngest setups with `createRunAbortController`, preserving
external-abort callbacks and remote-abort publication. The plain
publisher now uses the same forwarding helper and detaches it during
teardown, retaining its local no-reason abort default.
- Extract `createRunCleanup` for DurableAgent's stream, resume, recover,
generate and observe paths, retaining their different release policies,
ordering, timer behavior and retry-on-throw behavior.
- Resolve Inngest's bus through `AgentStreamBus`, preserving
`InngestPubSub`, its router, lazy cache resolution, already-cached
inner-bus reuse and fork bus identity.
- Replace the internal `super({ durable: false })` publisher-suppression
workaround with a protected publisher-ownership hook. DurableAgent still
does not start a plain publisher; its historically observable `durable
=== false` getter is preserved.
### Intentionally unchanged
Plain Agent continues to deliver directly to its original caller and
publish an observation copy. DurableAgent continues to publish and read
back. Public result shapes, event envelopes/topics, execution fencing
and path-specific cleanup policies remain distinct. Plain Agent retains
its existing publisher/abort/ownership policy rather than being forced
through durable-only lifecycle helpers.
No server authorization changes, topic-namespace redesign, output-path
convergence, durable-by-default execution, worker dispatch, checker
changes or propagated peer bumps. The separate streamed-delegation bug
fix shipped in [#29592](https://github.com/mastra-ai/mastra/pull/29592).
## Peer compatibility — unresolved review decision
Opening the exact reviewed candidate as requested; no additional
dependency or checker edits have been made for publication.
The candidate currently changes only Inngest's core peer range to
`>=1.83.0-0 <2.0.0-0`. Inngest imports the newly exported
`AgentStreamBus` and `createRunAbortController`, but published core
`1.83.0-alpha.0` satisfies that range and lacks those exports. The range
therefore does not yet establish compatibility with every version it
accepts.
- Stock import checking in the unprepared alpha checkout failed because
stable core `1.83.0` could not be downloaded at verification time.
- With authentic Changesets CI preparation, the unmodified checker
packed matching local core `1.83.0`: **18/18 import paths and 114 named
value imports passed**.
- Peer validation passed **128 valid / 0 invalid**; that is not proof
that published alpha.0 exposes the new APIs.
Decide the appropriate peer range during review. The suggested
stable-only range `>=1.83.0 <2.0.0-0` has **not** been applied. There is
no publish-first prerequisite or proposed checker workaround.
## Proof
Original reviewed candidate: `262c34adec`,
based on `f82c222119`. Twelve changed
files; no PR 2/server files or checker changes.
- Core TypeScript, changed-file repository formatting/type-aware lint,
peer validation and whitespace checks passed.
- Independent bounded core review: **24/24 test executions** across
standard and alternate-storage projects. Inspection covered abort
callback/reason ordering, cleanup retry/order, generate's lighter timer
release, publisher suppression and getter preservation.
- Strict retained before/after harness: **15/15 identical ordered frame
sequences** across plain, durable and evented engines for text,
tool-then-text, approval suspension, provider error and
abort-inside-tool. The comparison retains duplicate and
lifecycle/control frames. The scratch harness is not committed; this is
not generate, Inngest or remote-transport parity proof.
- Core suites reported **6,335 passing / 71 skipped**, with two parallel
worker processes terminating with SIGABRT in crash-recovery testing. The
crash-recovery file passed independently **20/20**. The broad parallel
run is not represented as clean.
- Earlier Inngest bus-integration-stage evidence: real dev-server abort
suites **9/9** and durable-agent suites **247/247**. Two create-agent
failures reproduced on main.
### Human-review follow-up — 541c618118
Two files changed: the plain publisher now reuses
`createRunAbortController(callerSignal)` and calls `detach()` on
teardown; the patch changeset explicitly documents `AgentStreamBus`,
`createRunAbortController`, and type `RunAbortController` with their
import path. Remote abort keeps its local `new Error('Aborted')`
default. No peer, checker, output-path, server, or cleanup-policy
change.
- Final focused stream/abort/cleanup suites: **66/66** across 11 test
files, including AIMock event parity and remote abort.
- Six disposable publisher-contract probes passed: live/already-aborted
caller signals with exact explicit/default reason identity, detachment
after teardown, and the local remote-abort default. Omitting
caller-signal forwarding made exactly **4 fail / 2 pass**; restoring it
returned **6/6**. Probe files are scratch evidence, not committed tests.
- Fresh `pnpm build:core`: **15/15 tasks passed**. Core `tsc --noEmit`
passed after rebuilding stale declarations. Changed-file type-aware
Oxlint, Oxfmt, Changesets status, whitespace checks, and the normal
commit hook passed.
- Broader cancellation/liveness/lease run: **300 passed / 2 failed / 24
skipped**. The lease case passed on isolated retry (12 matched cases
passed). The GC-child timeout reproduced with the publisher restored
byte-for-byte to original PR head `262c34ad`; this attributes it as
pre-existing to this two-file follow-up, **not** as proven main-baseline
behavior. The broader run remains non-clean.
- Fresh GitHub CI is pending. The disclosed Inngest peer-floor decision
remains unresolved and unchanged. No new hosted-service or two-process
verification was performed for this follow-up; earlier evidence below
retains its original provenance.
### Independent combined compatibility check
A disposable checkout applied only PR 2's three-file patch at
`71d868c384` to this exact PR 1 candidate.
Neither published branch was merged into the other, and the changed-file
sets have zero overlap.
Fresh core/server builds and both TypeScript checks passed. Executable
ESM/CJS source maps were checked against candidate sources, and fixture
imports resolved the freshly built distributions.
- Server handler tests: **281/281**.
- Real-TCP Hono access tests: **4/4**.
- Real Redis / two actual HTTP child processes: **8/8**, including
caching on/off, approval/finish observation, acknowledged remote abort,
unconfirmed abort false, tenant/anonymous/unknown/expired denials and
cleanup.
- Bounded Inngest cache/bus identity tests: **4/4**, plus the exported
`AgentStreamBus` check. This is wiring coverage, not a full Inngest
runtime run.
The subsequent PR 2 review additions are test-only and are not included
in that historical combined-run claim.
## Unverified
- Fresh GitHub CI and current-main integration after opening this PR.
- A compatible published minimum core version for all newly imported
APIs.
- A clean full-monorepo/full-parallel-core run; the parallel worker
SIGABRTs remain disclosed.
- Final-head full Inngest lifecycle/parity, generate parity, hosted
Redis, GCP Pub/Sub emulator, real-remote latency, browser
Stop/reconnect, load/failover and process-kill recovery.
## Risk
Lifecycle extraction can change abort-listener ordering, double
publication, cleanup timing or registry release even without intentional
behavior changes. Regression tests cover single control publication, the
absence of a second plain run record, one replay-cache wrapper, cache
identity, cleanup ordering and retry behavior. Explicit hooks preserve
path-specific policy rather than asserting identical output/result
contracts.
The unresolved peer range is a real package-compatibility issue and
remains a review/release gate, not a claim of runtime equivalence with
old core packages.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
This change gives durable and Inngest runs shared tools to handle
cancellation and cleanup. It also makes Inngest use core’s stream-bus
setup while keeping its own run behavior.
## Changes
- Added and exported helpers for forwarding abort signals and
coordinating run cleanup.
- Updated `DurableAgent` run paths to use the shared helpers. Added a
hook to prevent durable runs from using the plain Agent stream
publisher. Kept the `durable` getter set to `false`.
- Routed Inngest stream pubsub setup through `AgentStreamBus`. The setup
reuses an existing caching pubsub and resolves Mastra’s cache when
needed.
- Raised the Inngest package’s `@mastra/core` peer dependency minimum to
`1.83.0-0`.
- Added tests for abort forwarding, cleanup behavior, durable stream
publishing, and Inngest cache and bus setup.
## Validation and compatibility
The supplied PR notes report that focused stream, abort, and cleanup
suites passed 66/66 across 11 files. They also report 24/24 bounded core
test executions, 15/15 identical ordered frame sequences in a
before-and-after harness, six publisher-contract probes passed, and a
fresh `pnpm build:core` passed 15/15 tasks.
The broad core test run reportedly had 6,335 passing tests and 71
skipped tests, but two workers terminated with `SIGABRT`. Crash-recovery
reportedly passed 20/20 when run independently. A broader cancellation,
liveness, and lease run was not clean: 300 passed, 2 failed, and 24 were
skipped.
Fresh GitHub CI results and several runtime and recovery scenarios
remain unverified.
The PR notes identify a peer compatibility concern: published
`@mastra/core` version `1.83.0-alpha.0` lacks exports imported by
Inngest. The suggested stable-only peer range was not applied, so
compatibility with the published minimum remains unresolved.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
## Description
A durable agent run that finishes on a different worker from the one
that started it now records its real span outcome. Before, the facts
that decide the outcome lived only in the starting process's in-memory
run registry, so another worker ended `AGENT_RUN` and `MODEL_GENERATION`
with `unknown` or the wrong reason.
- Carry the stop cause, termination and delegation failure in the
durable loop's saved state, so the final step can read them on any
worker. Runs already in flight fall back to the registry, as they do
today.
- Keep `blocked / guardrail` and its `processorId` by passing the saved
step tripwire to the outcome classifier.
- Log a warning when a run reaches the final step with no recorded stop
cause.
- Inngest: end the caller's spans with the error when a run fails before
it reaches a worker. Nothing else ends them.
Real-run check with `gpt-4o-mini`, separate OS processes, main against
this PR:
| Case | main | This PR |
| --- | --- | --- |
| Observer failure on Inngest | `blocked / guardrail` in 6 of 9 runs |
`interrupted / observer_failed` in 9 of 9 |
| Delegation bail decided on one process, run ends on another |
`unknown` in 4 of 4 | Correct in 3 of 3 |
Stop conditions use the same field, but no real run crossed workers on
that path.
Unit tests run both final steps with the saved facts and no registry
entry, and they fail on `main`. The Inngest pool test stays as a smoke
test.
The rest of OBS-754 is in #29274 (span end that reaches storage first),
#29282 (resumed and recovered spans) and #29280 (Inngest deadlines and
workflow resume traces).
## Related Issue(s)
[OBS-754](https://linear.app/kepler-crm/issue/OBS-754/preserve-tracing-outcomes-and-resumed-span-identity-across-workers)
(part 1)
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update
## Checklist
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
When a durable agent run finishes on a different worker, that worker now
has the saved facts it needs to describe how the run ended. This helps
the run’s tracing record show the real result instead of an unknown one.
If an Inngest start request fails before a worker receives it, the open
tracing records are also ended with an error.
## Summary
- Save stop causes, termination details, and delegation failures in
durable iteration state. Finalization reads those saved values and falls
back to the in-process run registry when needed.
- Classify final span outcomes from the saved state, including
completion, limits, guardrails, observer failures, and delegation
failures. Log a warning when a final step lacks a stop cause.
- End the caller’s agent and model spans with an error when the initial
Inngest trigger chain fails.
- Add tests for saved-state outcomes without a registry entry, state
propagation, and start-event dispatch failure. Add an Inngest
integration test for outcomes across workers.
- Add changesets for `@mastra/core` and `@mastra/inngest`. Remove
documentation describing the previous cross-worker tracing limitation.
**Diff:** 21 files changed; 679 lines added and 57 removed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#29502
## Problem
On Temporal workflows, `run.start()` and `run.resume()` stopped
returning the workflow input under `result.steps.input`. A local run
always returns it. Top-level `result.input` was not affected.
## Root cause
`run.ts` built `steps` with `normalizeStepResults(...)` on the success,
failed/tripwire and suspended paths. That function only converts the
worker's step records and never adds `input`. The worker's `steps`
object has no `input` key either, so `input` was missing for every
status.
A local run always puts the start input first in `steps` (`input: <start
input>`) for success, failed, tripwire, suspended and after resume. With
no start input it is `input: undefined`.
## Fix
- New helper `toStepResults(input, steps)` in
`workflows/temporal/src/run.ts` returns `{ input,
...normalizeStepResults(steps) }`. It is used on the success,
failed/tripwire and suspended paths.
- On success the start input is used. On resume no input is passed, so
it falls back to the worker's `output.input`.
- The suspended path uses `status.input`. `isSuspendedWorkflowResult`'s
type now includes `input?: unknown`.
- Patch changeset for `@mastra/temporal`.
## Tests
- `run.test.ts` has a test that runs the same workflows locally and
through the package's Temporal path, using the generated worker modules.
It now:
- includes a successful workflow;
- asserts that `steps.input` matches the local run's (`{ value: 5 }`);
- no longer skips the `input` key.
- The mocked suspend/resume tests and lifecycle-hook tests asserted the
old shape without `input`, so they are updated to expect it.
- Without the fix, 6 of 32 tests in `run.test.ts` fail. With it, 32/32
pass in `run.test.ts` and 119/119 across the package.
- `tsc --noEmit` and `oxlint` (`pnpm lint`) are clean.
## Not verified
- Nothing was run against a real Temporal server. The tests use the
generated workflow and activity code with a mocked client.
- Suspend and resume are covered only by mocked tests.
- A run that fails after `resume()` has no `steps.input`, because the
worker's failure details carry no input. Top-level `input` is already
undefined there today.
- Canceled runs still return `steps: {}`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
Temporal workflow results did not always include the original input
alongside their step results. This change adds it back so callers can
read `result.steps.input`, as they can with local runs.
## Summary
- Add `toStepResults` to normalize step records and include the workflow
input under `result.steps.input`.
- Include the input in successful, failed, tripwire, and suspended
results. For successful results, use `output.input` when the supplied
input is unavailable.
- Test local and Temporal results for success, failure, and tripwire
workflows. Add checks for suspended results and dispatch failures.
- Add a patch changeset for `@mastra/temporal`.
Failures after `resume()` may still lack `steps.input` when worker
failure details do not include the input. Canceled runs still return
`steps: {}`. The tests mock Temporal behavior and do not use a real
Temporal server.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.
## @mastra/client-js@1.60.0-alpha.0
## @mastra/react@1.15.0-alpha.0
## @mastra/factory@0.28.0-alpha.0
## @mastra/code-sdk@1.19.0-alpha.0
## mastra@1.39.0-alpha.0
## @mastra/core@1.83.0-alpha.0
## @mastra/mcp@2.9.0-alpha.0
## @mastra/mcp-apps-components@0.1.0-alpha.0
## @mastra/playground-ui@69.0.0-alpha.0
## @mastra/server@1.83.0-alpha.0
## @mastra/nestjs@0.3.0-alpha.0
## @mastra/clickhouse@1.30.0-alpha.0
## @mastra/duckdb@1.19.0-alpha.0
## @mastra/libsql@1.30.0-alpha.0
## @mastra/pg@1.36.0-alpha.0
## @mastra/temporal@0.6.0-alpha.0
## @mastra/auth-better-auth@1.1.11-alpha.0
## @mastra/deployer-cloud@1.83.0-alpha.0
## @mastra/deployer-cloudflare@1.2.42-alpha.0
## @mastra/deployer-netlify@1.2.42-alpha.0
## @mastra/deployer-sandbox@0.3.27-alpha.0
## @mastra/deployer-vercel@1.2.42-alpha.0
## @mastra/longmemeval@1.1.42-alpha.0
## @mastra/opencode@0.1.42-alpha.0
## create-factory@0.4.3-alpha.0
## mastracode@0.49.1-alpha.0
## @mastra/arize@1.3.30-alpha.0
## @mastra/arthur@0.4.30-alpha.0
## @mastra/braintrust@1.3.27-alpha.0
## @mastra/datadog@1.4.22-alpha.0
## @mastra/deepeval@0.1.24-alpha.0
## @mastra/laminar@1.3.32-alpha.0
## @mastra/langfuse@1.6.6-alpha.0
## @mastra/langsmith@1.4.3-alpha.0
## @mastra/observability@1.23.2-alpha.0
## @mastra/otel-bridge@1.5.22-alpha.0
## @mastra/otel-exporter@1.5.2-alpha.0
## @mastra/posthog@1.4.7-alpha.0
## @mastra/sentry@1.2.32-alpha.0
## @mastra/agent-builder@1.2.1-alpha.0
## @mastra/connect@1.3.2-alpha.0
## create-mastra@1.39.0-alpha.0
## @mastra/deployer@1.83.0-alpha.0
## @mastra/editor@0.17.1-alpha.0
## @mastra/mcp-docs-server@1.3.13-alpha.0
## @mastra/memory@1.41.1-alpha.0
## @mastra/rag@2.7.3-alpha.0
## @mastra/redis-streams@0.8.1-alpha.0
## @mastra/elysia@0.1.23-alpha.0
## @mastra/express@1.5.27-alpha.0
## @mastra/fastify@1.5.27-alpha.0
## @mastra/hono@1.7.25-alpha.0
## @mastra/koa@1.7.27-alpha.0
## @mastra/next@0.2.41-alpha.0
## @mastra/tanstack-start@0.2.41-alpha.0
## @mastra/spanner@1.11.2-alpha.0
## @mastra/turso@0.1.21-alpha.0
## @mastra/inngest@1.11.2-alpha.0
## @mastra/gcs@0.3.6-alpha.0
## @internal/playground@1.39.0-alpha.0
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
PR #27020 accidentally removed the cancellation-specific result handling
while resolving its merge with lifecycle-hook changes. Temporal
cancellations were therefore returned as failed workflow results.
This restores canceled result classification for both `start()` and
background `startAsync()` settlement. Canceled runs invoke `onFinish`
with `status: canceled` and do not invoke `onError`.
Test plan: `pnpm test`, `pnpm typecheck`, `pnpm lint`, `pnpm build`,
formatting, and whitespace checks in `workflows/temporal`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->
## ELI5
Canceled Temporal workflows now show as canceled instead of failed. This
lets `onFinish` report the right status and keeps `onError` from
treating cancellation as an error.
## Changes
- `resolveResult` maps a `WorkflowFailedError` caused by
`CancelledFailure` to a canceled result for `start()` and background
`startAsync()` handling.
- Duplicate workflow-start errors do not trigger terminal hooks.
`start()` returns a failed result, while `startAsync()` rejects with the
original error.
- Workflow dispatch now passes `validateInputs` to Temporal.
- Added regression tests and patch changesets for cancellation handling,
duplicate starts, and input validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
## Summary
Adds Temporal-backed suspend/resume support for Mastra workflows.
This lets Temporal-backed workflow steps call `suspend(...)`, wait
durably inside the Temporal workflow runtime, and continue when
`TemporalRun.resume(...)` or `resumeAsync(...)` sends resume data
through a Temporal signal.
Fixes#24024.
## What changed
- Adds `suspend(...)` to generated Temporal step activity contexts.
- Converts suspended step activity execution into an internal Temporal
suspension marker.
- Adds a `mastra.resume` signal handler to the generated Temporal
workflow runtime.
- Uses Temporal `condition(...)` to durably wait until the matching step
or label is resumed.
- Implements `TemporalRun.resume(...)`.
- Implements `TemporalRun.resumeAsync(...)`.
- Validates `resumeData` against the suspended step’s `resumeSchema`
where available.
- Preserves Mastra-facing step metadata after resume:
- `suspendPayload`
- `resumeData`
- original step payload
- final step output
- Updates generated workflow/activity fixtures and regression tests.
- Adds a changeset for `@mastra/temporal`.
## Notes
Temporal does not expose Mastra-style `suspended` as a separate platform
execution status for this flow. A suspended Mastra step is represented
as an open Temporal workflow waiting for the `mastra.resume` signal, so
Temporal UI may show the workflow as running/waiting until it is
resumed.
This PR does not implement full Studio live streaming parity, time
travel, or complete nested/foreach/parallel resume parity.
## Test plan
- `pnpm --filter @mastra/temporal test`
- `pnpm --filter @mastra/temporal typecheck`
- `pnpm --filter @mastra/temporal lint`
- `pnpm exec oxfmt --check workflows/temporal/src/run.ts
workflows/temporal/src/transforms/activities.test.ts
workflows/temporal/src/transforms/temporal-workflow-runtime.mjs`
- `pnpm --filter @mastra/temporal build`
- `git diff --check`
Local Temporal verification:
- Temporal server: `temporalio/auto-setup:1.22.0` (`temporal-server`
reported `1.22.0`); TypeScript SDK/worker: `1.22.0`; Temporal UI:
`2.17.0`.
- Started a workflow that suspended on `approval-step`.
- Confirmed Temporal UI/history showed the workflow open and waiting.
- Resumed it through `TemporalRun.resume(...)`.
- Confirmed `WorkflowExecutionSignaled` appeared before completion.
- Confirmed the resumed result included `suspendPayload` and
`resumeData`.
- Ran unsupported foreach suspension as workflow ID
`foreach-suspend-failure-1792067367063` (Temporal run ID
`c3a17543-56f9-461e-8edc-ddd098d4f23f`).
- Confirmed the execution closed as `Failed` with history length 11 and
exactly one `WorkflowExecutionFailed` event.
- Confirmed the failure was `ApplicationFailure` with `nonRetryable:
true`, message `Temporal workflows do not support suspending inside
foreach steps yet.`, and no repeated activity or workflow failure
attempts. History counts were one activity scheduled/started/completed
and one workflow execution failure; the two workflow-task cycles were
the normal activity scheduling and post-activity completion tasks.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5
A workflow step can now pause and wait for someone to send it
information. Temporal keeps the workflow open while it waits, then runs
the step again with that information so the workflow can continue.
## Summary
- Generated step activities now provide `suspend(...)`. When a step
suspends, the activity returns a suspension marker instead of treating
suspension as a failure.
- The Temporal workflow runtime records suspended step state and waits
for a `mastra.resume` signal that matches the step ID or label. It
reruns the step with the supplied `resumeData`.
- A `mastra.status` query exposes the current workflow result.
`TemporalRun.start()` polls this query and can return a suspended result
while the Temporal workflow remains open.
- `TemporalRun.resume()` and `resumeAsync()` now signal the existing
workflow. Resume data is validated against the step’s resume schema when
available.
- Step results retain structured status, payload, output, timestamps,
and suspension or resume data where available.
- The change does not add full Studio live streaming, time travel, or
complete resume support for nested workflows, `foreach`, and parallel
paths.
## Verification
The PR description reports passing the Temporal package tests,
typecheck, lint, build, formatting check, and `git diff --check`. It
also reports a local Temporal verification in which a suspended workflow
resumed through `TemporalRun.resume()` and returned suspension and
resume data.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.
⚠️⚠️⚠️⚠️⚠️⚠️
`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.
⚠️⚠️⚠️⚠️⚠️⚠️
# Releases
> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.
## @mastra/factory@0.27.0-alpha.3
## @mastra/agent-builder@1.2.0-alpha.3
## @mastra/core@1.82.0-alpha.3
## @mastra/mcp@2.8.0-alpha.2
## @mastra/memory@1.41.0-alpha.3
## @mastra/playground-ui@68.0.0-alpha.3
## @mastra/ring-signals@0.1.0-alpha.0
## @mastra/libsql@1.29.0-alpha.2
## @mastra/mongodb@1.26.0-alpha.2
## @mastra/mysql@0.16.0-alpha.3
## @mastra/pg@1.35.0-alpha.3
## @mastra/discord@1.3.3-alpha.0
## @mastra/slack@1.8.1-alpha.0
## @mastra/teams@0.1.4-alpha.0
## @mastra/telegram@0.2.5-alpha.0
## @mastra/client-js@1.59.0-alpha.3
## @mastra/react@1.14.1-alpha.3
## @mastra/deployer-cloud@1.82.0-alpha.3
## @mastra/longmemeval@1.1.41-alpha.3
## @mastra/opencode@0.1.41-alpha.3
## create-factory@0.4.2-alpha.3
## @mastra/code-sdk@1.18.0-alpha.3
## mastracode@0.49.0-alpha.3
## @mastra/arize@1.3.29-alpha.2
## @mastra/arthur@0.4.29-alpha.2
## @mastra/laminar@1.3.31-alpha.2
## @mastra/langfuse@1.6.5-alpha.2
## @mastra/otel-bridge@1.5.21-alpha.2
## @mastra/otel-exporter@1.5.1-alpha.2
## @mastra/sentry@1.2.31-alpha.3
## mastra@1.38.0-alpha.3
## @mastra/connect@1.3.1-alpha.1
## @mastra/deployer@1.82.0-alpha.3
## @mastra/editor@0.17.0-alpha.3
## @mastra/evals@1.10.11-alpha.0
## @mastra/mcp-docs-server@1.3.12-alpha.4
## @mastra/server@1.82.0-alpha.3
## @mastra/elysia@0.1.22-alpha.3
## @mastra/express@1.5.26-alpha.3
## @mastra/fastify@1.5.26-alpha.3
## @mastra/hono@1.7.24-alpha.3
## @mastra/koa@1.7.26-alpha.3
## @mastra/nestjs@0.2.41-alpha.3
## @mastra/next@0.2.40-alpha.3
## @mastra/tanstack-start@0.2.40-alpha.3
## @mastra/turso@0.1.20-alpha.2
## @mastra/inngest@1.11.1-alpha.2
## @mastra/temporal@0.5.0-alpha.3
## create-mastra@1.38.0-alpha.3
## @internal/playground@1.38.0-alpha.3
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>