0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
Commit graph

1,086 commits

Author SHA1 Message Date
NikAiyer
d929736435
fix(core): use one stream bus for durable agents (#30229)
Durable agents now use the wrapped Agent's stream bus instead of
constructing a second one. An Agent-level pubsub carries both run frames
and thread events, including after registration and forks; evented
execution still uses Mastra's shared transport. Standalone durable
agents remain observable, and plain-run records and abort checks stay
separate from durable execution.

Previously, observing across replicas required repeating the pubsub on
the durable wrapper. This now works with the Agent configuration alone:

```ts
const agent = createDurableAgent({
  agent: new Agent({ id: 'assistant', name: 'Assistant', instructions, model, pubsub }),
});
```

Inngest explicitly exposes its detached-streaming capability without
migrating its bus. The server retains the `isDurableAgentLike` fallback
in `hasResumableStreams` for compatibility with older Inngest wrappers:
their proxy can inherit `false` from a plain Agent or lack the method
entirely. Server and Inngest versions are not coupled, so removing the
fallback would silently turn HTTP disconnects into run cancellation on
supported mixed-version installs. This is a deliberate, user-approved
exception to the ticket's shortcut-removal bullet; no peer-dependency
bump is needed while the fallback remains. Regression coverage exercises
capability `true`, `false`, and missing; the latter two fail without the
fallback.

Server branch audit (`packages/server/src/server/handlers/agents.ts`):
`validateDurableToolCallAccess` (line 298) remains because approvals
need durable workflow snapshots; `authorizeRunAccess` (2676) remains
because durable ownership uses workflow rows/local and replay owners
rather than plain run records (COR-1542). The two abort-route branches
(2724 and 2742) retain durable cancellation support and its
missing/finished-run acknowledgement semantics, not separate transports.

The observe-route branch (3141) remains for generation-fenced
`__subscribeToRunStream` and the Inngest adapter's `pubsub` fallback;
consolidating observation is COR-1541, and migrating Inngest's bus is
COR-1539. The recover-route guard (3746) remains because only durable
execution has recoverable workflow snapshots. None of these remaining
branches requires a second stream bus on core DurableAgent.

The Mastra-fallback regression also runs through a fork created after
registration, both with and without resolving the bus before
registration. Another replica must receive replay plus live tail through
finish, and the terminal frame must be published on Mastra's transport.
The latest targeted rerun passed 33 tests, including cache-policy
warnings and plain-publisher isolation across both durable storage
variants; core typecheck, targeted lint, formatting and diff checks
passed.

Verified with 1,643 core durable/stream tests, 78 AIMock parity tests,
292 server handler tests (including the mixed-version capability cases),
119 Inngest tests, and six real Redis cross-server cases covering live
observation, thread events and HTTP Stop. Typechecks, lint and
formatting pass. The Agent-pubsub regression failed before the fix; a
built-package red/green demo also confirms the transport change. An
earlier approval-resume timing failure passed in isolation and on the
final full rerun.

Fixes COR-1540




## Transport precedence and upgrade scope

Default-engine durable run transport precedence is now: explicit
`createDurableAgent` pubsub → wrapped Agent pubsub → Mastra pubsub.
Evented runs retain Mastra's shared transport. Mastra-only
configurations are unchanged.

If the wrapped Agent already specifies a different transport from Mastra
and the wrapper has no override, this fix changes the effective
run-frame and remote Stop transport. To preserve routing across an
upgrade, explicitly pass the previous transport to `createDurableAgent`
or align the Agent and Mastra transports. The wrapper configures the
shared stream bus, so its override also affects resumable streams from
direct calls on the wrapped Agent. The core changeset includes this
upgrade note.

A local investigation used separately built pre-change main
(`049b31dd2c`) and this branch with real Redis in both owner/observer
version directions. Shared effective transport passed live completion,
replay, and remote Stop. Distinct selected transports lost live
completion and remote Stop acknowledgement; shared-cache replay still
worked. This is a configuration-dependent routing cutover, not a general
rolling-deploy incompatibility. The investigation-only second-checkout
harness and expected failures are not included in this PR; existing
same-version Redis coverage remains.

Permanent coverage now executes wrapper and fork runs with three
distinct wrapper/Agent/Mastra buses, verifies run frames only reach the
wrapper bus, and verifies replayed output. Dropping the wrapper override
makes this test fail on the intended publication assertion. Latest
focused verification: 38 tests passed; core typecheck, type-aware lint,
formatting, and diff checks passed.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

ELI5: Durable agents now send live updates through the same stream
channel as the normal agent they wrap. Before, they built a second
channel. Users can now set up cross-server viewing by configuring only
the wrapped agent.

## Summary

This PR changes durable agents to use the wrapped `Agent`'s stream bus.
They no longer build a second bus. Agent-level pubsub carries run frames
and thread events, including after registration and forks. Evented
execution continues to use Mastra's shared transport.

### Core changes (`packages/core`)

- `Agent` stores a fallback stream pubsub and a shared-run-log
capability callback.
- `__getStreamBus()` initializes a bus when wrapper stream configuration
exists or streams are enabled. It prefers wrapper configuration over
agent durable settings.
- `__getStreamBus()` uses the configured pubsub, or the agent pubsub. If
neither exists, it uses a reusable fallback `EventEmitterPubSub`.
- The shared-run-log check uses the wrapper callback when configured.
Otherwise, it calls `__usesSharedRunLog()`.
- `__ownsPlainStreamPublisher()` still selects the plain publisher. It
now gets its bus through `__getStreamBus()`.
- Plain publishing still requires enabled agent streams.
- `__configureDurableStreams(config, usesSharedRunLog)` overlays defined
wrapper values on the agent durable settings and any existing wrapper
settings. It stores the callback and resets the bus.
- Agent forks copy the durable stream configuration and share the
fallback pubsub.
- `DurableAgent` delegates stream-bus ownership and access to the
wrapped agent. Its cache, pubsub, cleanup scheduling, and replay-owner
operations use that bus.
- The `DurableAgent` constructor configures durable streams on the
wrapped agent. `__getStreamBus()` returns that bus.
- `DurableAgent` no longer copies a private bus configuration when
forked. It no longer adopts the inner pubsub. Mastra registration no
longer registers a separate bus.
- `DurableAgent` adds `__setLogger(logger: IMastraLogger)`. It forwards
logger updates to the wrapped agent.
- The `AgentStreamBus` constructor accepts an optional `defaultPubsub`.
The bus uses it when `config.pubsub` is absent, before the new
`EventEmitterPubSub` fallback.
- `AgentStreamBus.forkConfig` and `adoptInnerPubsub` are removed.
- Direct `stream` and `generate` calls on an agent with `durable: false`
stay unpublished. This holds after durable wrapping configures the
shared bus. The wrapper can still publish through that bus.
- Plain-run records and abort checks stay separate from durable
execution.

### Inngest changes (`workflows/inngest`)

- `InngestAgent` adds the public method `hasResumableStreams():
boolean`. It always returns `true`.
- The internal `Pick` type includes this method.
- Inngest exposes detached streaming. It does not migrate its bus.
- A new test checks that `createInngestAgent` reports resumable streams
when its wrapped `Agent` has `durable: false`.

### Server changes (`packages/server`)

- `hasResumableStreams` keeps the `isDurableAgentLike` fallback for
older Inngest wrappers in mixed-version installs.
- The observe-stream handler keeps the same fallback.
- The fallback applies when a wrapper reports `false` or does not report
the capability.
- Removing the fallback would cause the reported capability-`false` and
missing-method regression cases to fail.
- This PR does not require a peer-dependency bump while the fallback
remains.
- Other server branches handle durable tool-call access, run
authorization, abort, observation, and recovery. The PR states these
address separate durable-workflow behavior or deferred work. They do not
need a second stream bus on core `DurableAgent`.
- Comments in `hasResumableStreams` and the observe-stream handler were
revised. No executable behavior changed.
- The durable-wrapper stream-detachment test runs with
`hasResumableStreams` set to `true`, `false`, and absent. The stream
observer detaches in each case.

### Redis pubsub (`pubsub/redis-streams`)

- The cross-server abort test runs for the durable and evented engines
with the Mastra bus. It runs for the durable engine with the agent bus.
- Each child server receives the selected bus location.
- The successful-abort case subscribes the other server to the run's
thread. That server verifies a tool-call chunk. It then verifies
completion and a finish chunk after the abort.
- The remote abort response remains `{ aborted: true }`. The owner's run
ends aborted.
- The abort fixture adds `BUS_LOCATION=agent`. With this value, the
agent uses Redis Streams pubsub. The Mastra-level bus uses
`EventEmitterPubSub`.
- Without `BUS_LOCATION=agent`, the agent has no explicit pubsub. With
`ENGINE=evented`, the Mastra-level bus uses `CachingPubSub` over Redis
Streams. Otherwise, it uses Redis Streams directly.
- The `stream` command accepts optional `threadId` and `resourceId`. It
passes them as memory configuration when `threadId` is present.
- The fixture adds `observe` and `subscribe` commands. `observe` streams
run observations. `subscribe` streams thread subscription chunks.

### Tests

- The shared-bus tests add a standalone mode. They cover replay and
live-tail observation with and without forks. They also cover bus
resolution before registration.
- The tests verify finish-event publication.
- Replayed start and tool-call frames are asserted only for registered
replicas.
- A prior logger-warning assertion was removed.
- Thread-coordination tests abort across processes in every
`sharedFrames` case. They observe the aborted run's frames. They expect
an aborted finish reason and one model request.
- New tests check run-frame routing for plain, durable, and evented
agents. They cover configurations with both agent-level and Mastra
buses. They also check replay from an unregistered durable agent whose
wrapped agent has `durable: false`.
- The explicit durable-execution test expects the original agent and its
durable wrapper to share the same stream bus.
- A direct-execution test checks that `stream` and `generate` runs stay
unpublished when a `durable: false` agent is wrapped and registered.
Runs through the durable wrapper publish a finish event.

### Changesets

- Patch changesets cover `@mastra/core`, `@mastra/server`, and
`@mastra/inngest`.
- The `@mastra/core` changeset includes an upgrade note. Mastra-only
configurations keep their behavior. Users who need the previous
transport can keep it by configuring it explicitly.

### Verification

The PR description reports these results. This summary does not
independently verify them.

- 1,643 core durable and stream tests passed.
- 78 AIMock parity tests passed.
- 292 server handler tests passed.
- 119 Inngest tests passed.
- Six real Redis cross-server cases passed.
- A latest targeted rerun passed 33 tests. It covered cache-policy
warnings and plain-publisher isolation across both durable storage
variants.
- Typechecks, targeted lint, formatting, and diff checks passed.
- The Agent-pubsub regression test failed before the fix.
- An earlier approval-resume timing test failed. It passed in isolation
and on the final full rerun.

### Diff size

- 14 files changed.
- 341 lines added.
- 118 lines removed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: review <review@local>
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-16 07:21:54 -08:00
mastra-platform[bot]
11c2d1dcea
test(inngest): verify OM resuspension on PostgreSQL with a separate worker (#30492) 2026-11-16 15:13:13 +01:00
dane-ai-mastra[bot]
a5859ee2b9
chore: version packages (alpha) (#30430)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-16 14:09:31 +01:00
dane-ai-mastra[bot]
15dc07c51d
chore: version packages (alpha) (#30394)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @mastra/core@1.84.0-alpha.2

### Minor Changes

- Added run metadata to background process callbacks.
([#30161](https://github.com/mastra-ai/mastra/pull/30161))

`backgroundProcesses` callbacks on `execute_command` now receive, when
available, the `agentId`, `threadId`, `resourceId`, and `runId` of the
call that started the process, so hosts can route exit notices to the
right conversation. `onExit` also receives `killedByAbort`, which is
`true` when the abort signal fired (the run's signal by default) and
killed the process.

  ```ts
  backgroundProcesses: {
    onExit: ({ pid, exitCode, threadId, killedByAbort }) => {
if (!killedByAbort) notifyThread(threadId, `Process ${pid} exited with
code ${exitCode}`);
    },
  },
  ```

`truncateOutput` and `DEFAULT_TAIL_LINES` are now exported from
`@mastra/core/workspace`, so hosts can trim process output the same way
`execute_command` does.

### Patch Changes

- Fixed retries in `dountil` and `dowhile` loops on the evented engine.
Each iteration of a loop body now starts at `retryCount` 0 and gets the
step's full `retries`, as on the default engine. Before, the count
carried over from earlier iterations, so a body that failed once in a
later iteration could run out of retries and fail the run.
([#30383](https://github.com/mastra-ai/mastra/pull/30383))

If your step code reads `retryCount` inside a loop body, it now gets the
attempt within the current iteration. The loop condition receives the
iteration number as `iterationCount`. To use it inside the body, pass it
through the step's output or workflow state.

- Fixed durable agent runs failing with
`DURABLE_AGENT_EXECUTION_UNVERIFIED` when an ownership check took longer
than the lease TTL (15 seconds by default) while the same process still
held the run. These runs used to fail and lose their saved snapshots.
They now keep running, and a step stops only when an ownership check
fails, the claim can't be renewed before it expires, or another
execution takes over the run.
([#30386](https://github.com/mastra-ai/mastra/pull/30386))

- Fixed workflow recovery restarting a run that is still executing,
which ran its steps twice. `startAsync()` now takes the run's lease
before it saves the first snapshot, so it resolves after the lease is
taken and rejects if taking it fails. Runs of workflows whose
`shouldPersistSnapshot` saves `waiting` but not `running` now hold a
lease too, so recovery no longer restarts them while they execute or
sleep. ([#30384](https://github.com/mastra-ai/mastra/pull/30384))

- `processToolResult` now sees and can rewrite a tool's `toModelOutput`
result. Previously, when a tool mapped its result with `toModelOutput`
(for example a file viewer returning images), output processors only saw
and changed the raw result, and the mapped output, images included,
still reached the next model call and stored history unchanged.
([#30401](https://github.com/mastra-ai/mastra/pull/30401))

The hook now receives `modelOutput`. To replace it, commit a
`result`-state part that carries the new value in
`providerMetadata.mastra.modelOutput`:

  ```ts
processToolResult({ messageList, toolCallId, toolName, args, result,
modelOutput }) {
    messageList.updateToolInvocation({
      type: 'tool-invocation',
toolInvocation: { state: 'result', toolCallId, toolName, args, result },
providerMetadata: { mastra: { modelOutput: shrinkImages(modelOutput) }
},
    });
  }
  ```

A processor that replaces only `result`, or edits it in place, now has
the new value mapped through `toModelOutput` again, so the model reads
the mapping of the processed result rather than of the original. With
several processors, each one receives the `modelOutput` the processor
before it committed, so a later processor can't bring back an image an
earlier one removed. This applies to both regular and durable agents. In
durable agents, a `processToolResult` that throws now also keeps the
unprocessed mapped output from the model, not only the raw result.
## @mastra/client-js@1.60.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
## @mastra/react@1.15.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/client-js@1.60.1-alpha.2
## @mastra/deployer-cloud@1.84.0-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/deployer@1.84.0-alpha.2
## @mastra/longmemeval@1.1.43-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
## @mastra/opencode@0.1.43-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
## @mastra/factory@0.29.0-alpha.2

### Patch Changes

- Recognize Detail findings as linked work items so intake cards retain
their external source and rule identity.
([#30410](https://github.com/mastra-ai/mastra/pull/30410))

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/code-sdk@1.19.1-alpha.2
## create-factory@0.4.4-alpha.2

### Patch Changes

- Updated dependencies:
  - mastra@1.39.1-alpha.2
## @mastra/code-sdk@1.19.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
  - @mastra/client-js@1.60.1-alpha.2
## mastracode@0.50.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
  - @mastra/code-sdk@1.19.1-alpha.2
## @mastra/agent-builder@1.2.2-alpha.1

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
## mastra@1.39.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/deployer@1.84.0-alpha.2
## @mastra/deployer@1.84.0-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/editor@0.17.2-alpha.1

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
## @mastra/mcp-docs-server@1.3.14-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
## @mastra/memory@1.41.2-alpha.1

### Patch Changes

- Observational Memory does half as much work per step in long tool
runs. Token counting no longer serializes every plain JSON tool result a
second time just to check it for images, so each step serializes the
earlier tool results twice instead of four times. Token counts are
unchanged. ([#30400](https://github.com/mastra-ai/mastra/pull/30400))

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
## @mastra/playground-ui@69.1.0-alpha.2

### Patch Changes

- Connect providers that use OAuth client credentials (`OAUTH2_CC`) or
no authentication (`NONE`) without leaving Studio. Client-credential
providers ask for a client ID and secret in the connect dialog;
providers with no authentication connect immediately instead of opening
an OAuth popup.
([#30336](https://github.com/mastra-ai/mastra/pull/30336))

Providers whose catalog lists only connection settings (for example a
subdomain) now also ask for their API key, username and password, or
client ID and secret. Previously the dialog showed only the settings,
and the connection failed.

The headless connect helpers in `@mastra/playground-ui/domains/connect`
now return a fully resolved `HeadlessAuthField` and support conditional
fields (`visibleWhen`), so other apps can import them instead of keeping
their own copy.

  ```ts
  import {
    headlessAuthFieldKey,
    headlessAuthFields,
    isHeadlessAuthFieldVisible,
    splitHeadlessAuthValues,
    startHeadlessAuth,
  } from '@mastra/playground-ui/domains/connect';

// Every field comes back fully resolved: label, target, required, and
so on are always set.
  const fields = headlessAuthFields(provider);

// Render only the fields whose `visibleWhen` condition matches the
current values.
const visible = fields.filter(field =>
isHeadlessAuthFieldVisible(fields, field, values));
const missingRequired = visible.some(field => field.required &&
!values[headlessAuthFieldKey(field)]);

  // On submit, values for hidden fields are dropped.
const { credentials, params } = splitHeadlessAuthValues(fields, values);
  await startHeadlessAuth({
    sessionToken,
    integrationId: provider.id,
    authType: provider.authType,
    credentials,
    params,
  }).result;
  ```

- Updated dependencies
[[`7277766`](7277766d6c),
[`9eae968`](9eae9680d2),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/memory@1.41.2-alpha.1
  - @mastra/client-js@1.60.1-alpha.2
  - @mastra/ai-sdk@1.12.0
  - @mastra/react@1.15.1-alpha.2
## @mastra/server@1.84.0-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
## @mastra/elysia@0.1.24-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/express@1.5.28-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/fastify@1.5.28-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/hono@1.7.26-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/koa@1.7.28-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/nestjs@0.3.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/next@0.2.42-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/hono@1.7.26-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/tanstack-start@0.2.42-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/hono@1.7.26-alpha.2
  - @mastra/server@1.84.0-alpha.2
## @mastra/temporal@0.6.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/deployer@1.84.0-alpha.2
## create-mastra@1.39.1-alpha.2


## @internal/playground@1.39.1-alpha.2

### Patch Changes

- Updated dependencies
[[`7277766`](7277766d6c),
[`2f4a9ed`](2f4a9ed771),
[`951a18b`](951a18b085),
[`f4458ee`](f4458eea7f),
[`a3b4ef9`](a3b4ef9ecd),
[`547600d`](547600d004)]:
  - @mastra/core@1.84.0-alpha.2
  - @mastra/playground-ui@69.1.0-alpha.2
  - @mastra/client-js@1.60.1-alpha.2
  - @mastra/react@1.15.1-alpha.2

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-14 10:23:58 -08:00
Arthur Fiorette
7277766d6c
fix(core): reset retryCount on each evented loop iteration (#30383)
## Description

On the evented engine, each iteration of a `dountil`/`dowhile` loop
whose body is a step now starts with `retryCount` 0 and gets the step's
full `retries`, as on the default engine.

### Why

The evented engine carried `retryCount` from one iteration into the
next, so iteration N started at N-1 plus every retry earlier iterations
had used. A body with `retries` ran out of budget on a later iteration:
with `retries: 1`, a single failure in iteration 3 failed the run
instead of being retried. Step code reading `retryCount` also saw that
inflated number. The default engine has counted retries per iteration
since #23593.

```ts
const body = createStep({ id: 'body', retries: 1, execute: async ({ retryCount }) => { /* ... */ } });
workflow.dountil(body, async ({ iterationCount }) => iterationCount >= 4).commit();

// retryCount seen by the body when iteration 3 fails once
// before: 0, 1, 2 and the run fails
// after:  0, 0, 0, 1, 0 and the run succeeds
```

### Behavior change

- Step code that reads `retryCount` inside an evented loop body now gets
the attempt within the current iteration. The iteration number is
available to the loop condition as `iterationCount`.
- The loop condition's `retryCount` is now the body's attempt in the
current iteration on every path. One path used to pass attempt + 1. The
default engine passes -1 to the condition; that difference is left as
is.

### Not affected

`foreach` loops, and loops whose body is a nested workflow (those
already started each iteration at 0).

### Tests

New case in the shared workflow suite
(`workflows/_test-utils/src/domains/run-count.ts`): a `dountil` body
with `retries: 1` that throws once in iteration 3. It runs on the
default and evented engines and is skipped on Inngest with the other
`retryCount` cases. Without the fix it fails on the evented engine with
`expected [ 0, 1, 2 ] to deeply equal [ 0, 0, 0, 1, 0 ]`.

## Related issue(s)

Linear PLTFRM-3202. No GitHub issue.

## Type of change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

ELI5: When a loop step fails and retries, the retry count from earlier
loop passes was carried into later passes. This could use up the retry
limit early. This change resets the count so each loop pass gets its
full retry limit.

## Summary

This pull request changes how the evented workflow engine tracks
`retryCount` for `dountil` and `dowhile` loops whose body is a step.

## Changes

-
`packages/core/src/workflows/evented/workflow-event-processor/loop.ts`:
When a loop iteration runs again, `processWorkflowLoop` passes
`retryCount: 0` instead of the current retry count. Each iteration
starts with the body's full retry budget.
-
`packages/core/src/workflows/evented/workflow-event-processor/index.ts`:
In `runLeafStep`, the loop-condition path passes `retryCount` to
`#processLoop` without adding 1.
- `.changeset/bright-terms-crash.md`: Adds a patch changeset for
`@mastra/core`. It documents that `dountil` and `dowhile` body retries
start at 0 on each iteration. It also states that `retryCount` is scoped
to the current iteration.
- `workflows/_test-utils/src/domains/run-count.ts`: Adds
`createRunCountWorkflows` and `createRunCountTests`. The new workflow is
a loop whose body allows one retry and throws once on the third
iteration. The test asserts that the run succeeds with recorded retry
counts `[0, 0, 0, 1, 0]` and final output `4`.

Diff size: 4 files, 63 lines added, 2 lines removed.

## Not changed

- The default engine's behavior is unchanged.
- `foreach` loops and loops with nested workflow bodies are not affected
by the diff shown.

## Verification

- The shared test case runs on the default and evented engines. It is
skipped on Inngest with the other `retryCount` cases.
- The author reports that, without the fix, the evented case fails with
`expected [ 0, 1, 2 ] to deeply equal [ 0, 0, 0, 1, 0 ]`. The author
reports this result. This review did not run the tests.
- CI results: not available.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-14 01:01:42 -03:00
dane-ai-mastra[bot]
1c52fdd034
chore: version packages (alpha) (#30261)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases

> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.

## @mastra/factory@0.29.0-alpha.1


## @mastra/core@1.84.0-alpha.1


## @mastra/playground-ui@69.1.0-alpha.1


## @mastra/client-js@1.60.1-alpha.1


## @mastra/react@1.15.1-alpha.1


## @mastra/deployer-cloud@1.84.0-alpha.1


## @mastra/longmemeval@1.1.43-alpha.1


## @mastra/opencode@0.1.43-alpha.1


## create-factory@0.4.4-alpha.1


## @mastra/code-sdk@1.19.1-alpha.1


## mastracode@0.50.1-alpha.1


## @mastra/arize@1.3.31-alpha.0


## @mastra/arthur@0.4.31-alpha.0


## @mastra/braintrust@1.3.28-alpha.0


## @mastra/datadog@1.4.23-alpha.0


## @mastra/deepeval@0.1.25-alpha.0


## @mastra/laminar@1.3.33-alpha.0


## @mastra/langfuse@1.6.7-alpha.0


## @mastra/langsmith@1.4.4-alpha.0


## @mastra/observability@1.24.1-alpha.0


## @mastra/otel-bridge@1.5.23-alpha.0


## @mastra/otel-exporter@1.5.3-alpha.0


## @mastra/posthog@1.4.8-alpha.0


## @mastra/sentry@1.2.33-alpha.0


## @mastra/agent-builder@1.2.2-alpha.0


## mastra@1.39.1-alpha.1


## @mastra/connect@1.4.1-alpha.0


## @mastra/deployer@1.84.0-alpha.1


## @mastra/editor@0.17.2-alpha.0


## @mastra/mcp-docs-server@1.3.14-alpha.1


## @mastra/memory@1.41.2-alpha.0


## @mastra/server@1.84.0-alpha.1


## @mastra/google-cloud-pubsub@1.1.8-alpha.0


## @mastra/redis-streams@0.8.2-alpha.0


## @mastra/valkey-streams@0.6.3-alpha.0


## @mastra/elysia@0.1.24-alpha.1


## @mastra/express@1.5.28-alpha.1


## @mastra/fastify@1.5.28-alpha.1


## @mastra/hono@1.7.26-alpha.1


## @mastra/koa@1.7.28-alpha.1


## @mastra/nestjs@0.3.1-alpha.1


## @mastra/next@0.2.42-alpha.1


## @mastra/tanstack-start@0.2.42-alpha.1


## @mastra/clickhouse@1.30.1-alpha.0


## @mastra/cloudflare@1.8.1-alpha.0


## @mastra/cloudflare-d1@1.5.1-alpha.0


## @mastra/convex@1.9.1-alpha.0


## @mastra/dsql@1.5.8-alpha.0


## @mastra/dynamodb@1.5.2-alpha.0


## @mastra/libsql@1.30.1-alpha.0


## @mastra/mongodb@1.27.1-alpha.0


## @mastra/mssql@1.11.2-alpha.0


## @mastra/mysql@0.17.1-alpha.0


## @mastra/oracledb@0.5.7-alpha.0


## @mastra/pg@1.36.1-alpha.0


## @mastra/redis@1.4.14-alpha.0


## @mastra/spanner@1.12.1-alpha.0


## @mastra/turso@0.1.22-alpha.0


## @mastra/upstash@1.6.2-alpha.0


## @mastra/valkey@0.2.10-alpha.0


## @mastra/temporal@0.6.1-alpha.1


## @mastra/daytona@0.11.6-alpha.0


## create-mastra@1.39.1-alpha.1


## @internal/playground@1.39.1-alpha.1

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-13 19:58:20 -08:00
Tyler Barnes
cdebd2eff0
test: stop rebuilding workflow test mocks before every test (#29144)
## Summary

The core unit lane spends tens of seconds in `vi.clearAllMocks()` /
`vi.resetAllMocks()`, and almost all of it is caused by the workflow
test factory.

- **Mock registry built mocks on every reset.**
`workflows/_test-utils/src/factory.ts` resets every registry entry
before each test. Each domain shares one `MockRegistry`, and `reset()`
rebuilt every factory in it, so one run of `evented-workflow.test.ts`
created **722,585** `vi.fn()`s. `@vitest/spy` keeps every mock in a
module-level set for the life of the worker, so later `clearAllMocks`
calls in the same worker took about 440 ms each. `register()` and
`reset()` now only drop the cached mock, and `get()` builds it on first
use. Every read already goes through `get()`, so tests see the same
mocks. The same file now creates 579 mocks.
- **`evented-workflow.test.ts` Engine Specific block** used
`vi.resetAllMocks()` before each test. Every spy in that block is
created inside a test body and the file has no `vi.mock`, so it now uses
`vi.restoreAllMocks()`.
- **`durable-agent-background-tasks.test.ts`**: six fixed 500–1500 ms
sleeps are now `vi.waitFor` on the state the test asserts next. The
readOnly test keeps its sleep because it proves something does *not*
happen. No assertions changed.

## Numbers

Full `@mastra/core` unit lane, 5-CPU container, shared module cache
(stacked on #29134):

| | Base | This PR |
|---|---|---|
| Wall | 378.4 s | 339.7 s |
| Test time (worker-s) | 1,366.6 | 1,176.8 |
| `clearAllMocks` total | 37.6 s | 2.1 s |
| Result | 25,247 passed, 0 failed | 25,247 passed, 0 failed |

`workflow.test.ts` alone: 79 s → 24 s. The Inngest engine suite, which
uses the same factory, gives an identical result on the base and on this
branch: 222 passed, and the same 5 failures on both.

## Test plan

- [x] Full core unit lane: 25,247 passed, 0 failed, on both the base and
this branch.
- [x] `workflows/_test-utils`: 581/581.
- [x] Inngest engine suite: same result on base and branch.
- [x] oxlint and prettier clean.

Stacked on #29134 (core unit test wait knobs). I'll retarget it to main
once that merges.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

This change makes tests faster and less fragile. Tests now wait for the
result they need instead of sleeping for a fixed time. Workflow test
mocks are also created only when a test first uses them.

## Changes

- `workflows/_test-utils/src/mock-registry.ts`: `register()` removes any
existing mock for a key. `reset()` clears all mocks. `get()` creates a
mock on first use, so registered factories no longer run during reset.
The Vitest imports are split into value and type imports.
- `packages/core/src/workflows/evented/evented-workflow.test.ts`: The
Engine Specific setup calls `vi.restoreAllMocks()` instead of
`vi.resetAllMocks()`.
-
`packages/core/src/agent/durable/__tests__/durable-agent-background-tasks.test.ts`:
Fixed sleeps are replaced with `vi.waitFor` checks for dispatch, tool
results, persisted results, and task status. The suspension test also
waits for `completed` after resuming. The waits use a 10-second timeout
and a 10 ms interval. The readOnly test keeps its sleep.
- `mastracode/factory/src/tools/messaging-tools.test.ts`: The
queued-message ordering test and the missing-thread test use
`vi.waitFor` at 20 ms intervals, with a timeout of twice
`QUEUE_WAIT_MS`. The missing-thread test waits for the dropped-delivery
audit record before it asserts that no message was sent.
- `mastracode/tui/src/tui/handlers/tool.test.ts`: `flushParser` advances
Vitest fake timers past the render-coalescing interval. With real
timers, it keeps the existing timeout wait. The compact-shell streaming
test uses fake timers and restores real timers in cleanup.
- `packages/core/src/agent/durable/__tests__/restart-agreement.test.ts`:
The T26 `echoContext` post-park delay drops from 200 ms to 20 ms. The
T55 `loggedStep` delay drops from 100 ms to 10 ms.
- `workflows/_test-utils/src/domains/abort.ts`: The immediate-abort
setup wait drops from 1 s to 50 ms. The in-step and nested-workflow
waits drop from 5 s to 50 ms. Comments now state that the in-step test
does not trigger an abort.

Diff size: 7 files, 85 lines added, 38 lines removed. No exported
declarations changed.

## Validation

Results below are reported by the PR author. This review did not run
them.

- Core unit lane: 25,247 passed and 0 failed on both the base and this
branch. Wall time fell from 378.4 s to 339.7 s. `clearAllMocks` time
fell from 37.6 s to 2.1 s.
- `workflow.test.ts`: runtime fell from 79 s to 24 s.
- `evented-workflow.test.ts`: mock creation fell from 722,585 to 579.
- Inngest engine suite: 222 passed and the same 5 failures on both the
base and this branch.
- `workflows/_test-utils`: 581 of 581 passed.
- oxlint and prettier: reported clean.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: remy-builder[bot] <324360501+remy-builder[bot]@users.noreply.github.com>
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: yoko-reviewer[bot] <324360165+yoko-reviewer[bot]@users.noreply.github.com>
2026-11-12 21:05:07 -07:00
dane-ai-mastra[bot]
34dbbd363e
chore: version packages (alpha) (#30241)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @mastra/factory@0.29.0-alpha.0

### Minor Changes

- Intake rules can now set a description on the cards they create, so
cards from integrations show the item's details instead of an empty
description. ([#30253](https://github.com/mastra-ai/mastra/pull/30253))

  ```ts
  return {
    type: 'upsertLinkedWorkItem',
    board: context.intake.board,
    source: 'github-dependabot-alert',
    sourceKey: alert.id,
    title: alert.title,
    description: alert.description,
    url: alert.url,
    stage: context.intake.initialPhase,
    // ...
  };
  ```

- Added Vanta vulnerabilities as a Factory intake source through
MastraConnect. Once you connect Vanta and enable it in intake settings,
each active vulnerability becomes a card labeled `security` with its
severity, package, and remediation date in the description.
([#30255](https://github.com/mastra-ai/mastra/pull/30255))

- Intake integrations are now off until you turn them on. A newly
connected integration no longer starts pulling items onto your boards
until you enable it in the intake settings.
([#30251](https://github.com/mastra-ai/mastra/pull/30251))

- Added two reads that serve a Factory board without sending every card.
`GET /web/factory/projects/:id/boards/:boardKey/phases` returns, for
each phase of the board, how many cards it holds and the first of them
in the order asked, after the search and filters given. `GET
/web/factory/projects/:id/work-items/filter-options` returns the labels
and people the board can be filtered by. `item=<card id>` also answers
for one card and its place in each phase, however deep it lies, without
listing the phase down to it. Each card is named with a `version` of its
body, which `?workItemId=` also sends, so a reader keeps the bodies it
holds and reads only the ones that changed.
([#30177](https://github.com/mastra-ai/mastra/pull/30177))

  ```ts
const res = await
fetch(`/web/factory/projects/${projectId}/boards/work/phases?label=bug&limit=10&cards=true`);
const { phases, matchingTotal, boardTotal, workItems } = await
res.json();
// phases: [{ phase: "intake", total: 42, cards: [{ id, version }, …] },
…]
  ```

- Added Dependabot alerts as a Factory intake source. Turn on **Sync
alerts** under GitHub in intake settings and pick repositories, or `PUT
/web/intake/config` with `{ "dependabot": { "enabled": true,
"sourceIds": ["owner/repo"] } }`. Open alerts in those repositories
become cards labeled `security` with the advisory details as the
description, and they close automatically when the alert is fixed or
dismissed. ([#30254](https://github.com/mastra-ai/mastra/pull/30254))

- Changing an intake route now warns you when cards already on your
boards match a different board, and offers to move those untouched cards
to the start of their new board.
([#30252](https://github.com/mastra-ai/mastra/pull/30252))

### Patch Changes

- Fixed who owns the sessions a Factory card runs in, and who can
approve what they wait on.
([#30146](https://github.com/mastra-ai/mastra/pull/30146))

  **Who owns a card's phase session**

- In a Factory that runs card work on the organization's model keys (the
`factory` and `hybrid` credential settings), every new phase session now
belongs to the Factory itself and uses the organization's keys and the
Factory's default model. That holds whoever moved the card: a person, an
agent, a webhook or a rule. Before, it ran as whoever connected the
repository, or as the person who approved the move.
- The person who moved or approved the card stays on record as the one
who did it. When they move a card from their own session, that session
follows the card, so it hears when the phase waits on someone and can
answer from there.
- If the organization has neither a model key nor a provider of its own,
the run stops with a clear "add an organization key or provider" message
and is not tried again on its own. Add one, then press Retry on the
card.
- In a Factory on personal credentials there are no organization keys,
so a phase runs as the person who moved or approved the card (or whose
session's agent moved it), and otherwise as the person who created the
Factory.

  Sessions that already exist keep their owner.

  **Approving a waiting tool call**

- The person who asked for the work, and anyone whose session follows
the card, can approve or decline any tool call the phase session is
waiting on, a plan or any other tool that asks first. They can do it
from their own session in the sidebar, with `factory_answer_approval`,
or over HTTP:

    ```http
    POST /web/user-sessions/:sessionId/parked-approval
    { "decision": "approve" }
    ```

Add `toolCallId` when the session waits on more than one approval.
Questions that need a typed answer are still answered in the session.

- A session following the card is told when the phase session waits on a
yes or no, for a plan or any other tool, with the call to answer it. In
the chat that notice now has Approve and Decline buttons and a link to
open the waiting session; a notice about a question that needs a typed
answer has the link only.
- Nobody is asked to approve what a phase approves itself: when a
phase's `exitTool` has `autoApprove` and the card matches
`autoApproveWhen`, the run carries on without a notice. This now works
for any exit tool that pauses for a yes or no, not only `submit_plan`.
- When two people answer the same call at once, one answer is used and
the other is told the call was already answered.
- The run then carries on as the phase session's owner. Before, it
carried on as whoever answered, and could fail with "the request has no
authenticated user".
- Following moves with a session: after a hand-over, the session's new
owner can answer and the old one cannot.

  **Sidebar**

A card's phase sessions now show under the session that asked for the
work when child sessions are revealed, marked "run by Factory" when the
Factory owns them, and that session shows "waiting on you", with Approve
and Decline in its menu, while one of them waits on the viewer.

- Mastra Code's Knowledge scopes now use canonical addresses, matching
`@mastra/core`'s one-segment scope values. Factory sessions hold their
org and project ids encoded (`org:acme%3Aeu` for org `acme:eu`), and the
`/knowledge` inspector and Factory Knowledge routes read the same
addresses the code agent writes, instead of encoding host scopes a
second time. ([#30019](https://github.com/mastra-ai/mastra/pull/30019))

- Sandboxes in a Factory with more than ten repositories can now reach
all of them with `git` and `gh`. The Factory mints each sandbox token
for at most ten repositories, so the sandbox now gets one token per
group of ten (`GH_TOKEN`, `GH_TOKEN_2`, …). Each checkout's git
credential helper uses the token that covers it, `github_refresh_token`
refreshes every group, and the environment note tells the agent which
variable to use for `gh` in each repository, for example
`GH_TOKEN=$GH_TOKEN_2 gh pr list -R owner/repo`. Before, a Factory over
ten repositories fell back to a token for the session repository only.
([#30121](https://github.com/mastra-ai/mastra/pull/30121))

- Fixed GitHub pull request cards losing their labels, so intake routes
that match on labels (for example `security`) now route pull requests
too. ([#30250](https://github.com/mastra-ai/mastra/pull/30250))

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`175b4b5`](175b4b5194),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/code-sdk@1.19.1-alpha.0
## @mastra/core@1.84.0-alpha.0

### Minor Changes

- **Breaking (Knowledge alpha):** a `$param` in a Knowledge scope
pattern now matches exactly one segment, so scope values can no longer
contain a raw `:`. Before, the last param took the rest of the address,
so `org:acme:viewer` counted as an `org:$orgId` scope and its holder got
that pattern's `self` rights there.
([#30019](https://github.com/mastra-ai/mastra/pull/30019))

- Agent `scopes` are canonical addresses. An id containing `:`, `/`, `#`
or `%` must be encoded with the new `encodeKnowledgeScopeId`. The raw
form (e.g. `org🅰️b` against `org:$orgId`) fails with "does not match a
declared scope type", and `%` is always read as an escape, so each scope
has one spelling.
- Memory ids are bound encoded: resource `slack:U123` becomes
`resource:slack%3AU123`.
- Patterns of one type with different segment counts no longer overlap,
so `org:$orgId` and `org:$orgId:viewer` can be declared side by side.
- `decodeKnowledgeScopeId` reads an id back from one segment of an
address.

  ```ts
  import { encodeKnowledgeScopeId } from '@mastra/core/knowledge';

const scope = `org:${encodeKnowledgeScopeId('auth0|a:b')}`; //
'org:auth0|a%3Ab', pass this in Agent scopes
  ```

  No data is rewritten:

- **Ids with a colon in the last segment get new addresses.** Before,
the last segment kept a raw `:` (`resource:slack:U123`); now it is
encoded (`resource:slack%3AU123`), so a run reaches a new, empty scope,
and pages, pins and notes stored under the old address stay there
unreached. Nothing moves them.
- **Every other id keeps its address if you pass it through
`encodeKnowledgeScopeId`.** `/`, `#` and `%` were already encoded the
same way, so `org:${encodeKnowledgeScopeId('a/b')}` reaches rows stored
from the raw `org:a/b`. Passing the old raw string unchanged does not:
`org:a/b` is refused, and a raw scope that happened to contain an escape
(`org:a%3Ab`, once stored as `org:a%253Ab`) now means the id `a:b`.
- **Stale rows get nothing from their new match.** A stored scope row
whose address now matches a different declaration than the one it was
created under (e.g. `org:acme:viewer` created under `org:$orgId`, once
`org:$orgId:viewer` is declared) gets none of the new declaration's
access. It is never a `belongsTo` parent, and using it as a scope of the
new declaration fails with an error. Prefix grants of the declaration it
was created under still let their holders read it, as before, without a
role on it.

### Patch Changes

- Fixed web search on models served through the Mastra gateway. An agent
using `webSearchTool` with a model such as `mastra/openai/gpt-6.1-sol`
failed every turn with `unknown tool type "openai:web_search"`, and on
Anthropic and Google models the tool was dropped without an error. These
models now use the gateway's web search.
([#30239](https://github.com/mastra-ai/mastra/pull/30239))

Other provider-native tools on gateway models now fail with a clear
error. Previously the gateway rejected them with a 400 or ignored them,
leaving the model without the tool.

- Fixed a run waiting on a tool approval losing its saved state when the
thread lease expired. This happened when the lease backend (such as
Redis) was down for too long. Before, the run's snapshot was deleted, so
the approval could not be resumed. Now the run resumes normally once the
backend is back. If another run took the thread first, a resume in the
same process is refused and the run keeps waiting.
([#27734](https://github.com/mastra-ai/mastra/pull/27734))
## @mastra/client-js@1.60.1-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
## @mastra/react@1.15.1-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/client-js@1.60.1-alpha.0
## @mastra/deployer-cloud@1.84.0-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## @mastra/deployer-cloudflare@1.2.43-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## @mastra/deployer-netlify@1.2.43-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## @mastra/deployer-sandbox@0.3.28-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## @mastra/deployer-vercel@1.2.43-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## @mastra/longmemeval@1.1.43-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
## @mastra/opencode@0.1.43-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
## create-factory@0.4.4-alpha.0

### Patch Changes

- `create-factory` no longer asks where your Factory project should run.
New projects are created in the US region by default. To create the
project in the EU instead, pass `--region eu`:
([#30249](https://github.com/mastra-ai/mastra/pull/30249))

  ```bash
  npx create-factory@latest my-mastra-factory --region eu
  ```

- Updated dependencies
[[`4a3920c`](4a3920c7bf),
[`f75eb1a`](f75eb1a197),
[`338ec03`](338ec03925),
[`28e243f`](28e243f638)]:
  - mastra@1.39.1-alpha.0
## @mastra/code-sdk@1.19.1-alpha.0

### Patch Changes

- Mastra Code's Knowledge scopes now use canonical addresses, matching
`@mastra/core`'s one-segment scope values. Factory sessions hold their
org and project ids encoded (`org:acme%3Aeu` for org `acme:eu`), and the
`/knowledge` inspector and Factory Knowledge routes read the same
addresses the code agent writes, instead of encoding host scopes a
second time. ([#30019](https://github.com/mastra-ai/mastra/pull/30019))

- Updated dependencies
[[`be818ec`](be818ecb53),
[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/github-signals@0.5.3-alpha.0
  - @mastra/core@1.84.0-alpha.0
  - @mastra/client-js@1.60.1-alpha.0
## mastracode@0.50.1-alpha.0

### Patch Changes

- Updated dependencies
[[`be818ec`](be818ecb53),
[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`175b4b5`](175b4b5194),
[`9abbec6`](9abbec671c)]:
  - @mastra/github-signals@0.5.3-alpha.0
  - @mastra/core@1.84.0-alpha.0
  - @mastra/code-sdk@1.19.1-alpha.0
## mastra@1.39.1-alpha.0

### Patch Changes

- Added `factory intake-route reroute-preview` and `factory intake-route
reroute` to `mastra api`, for listing and moving untouched cards whose
intake route now points at a different board.
([#30252](https://github.com/mastra-ai/mastra/pull/30252))

  ```sh
  mastra api factory intake-route reroute-preview <factoryProjectId>
  mastra api factory intake-route reroute <factoryProjectId>
  ```

- Improved the Factory board on large projects. The board now loads the
first cards of each phase, then more as you scroll. It no longer
downloads every card of the project.
([#30191](https://github.com/mastra-ai/mastra/pull/30191))

Search, filters and sorting now run on the server. Search matches cards
that contain every word typed, in any order. It also finds a card by its
number or source id.

- Fixed the checkmark on a finished Factory onboarding step sitting 12px
right of the step numbers below it.
([#30248](https://github.com/mastra-ai/mastra/pull/30248))

- Added `mastra api factory board phases` and `mastra api factory
work-item filter-options`, which read the first cards under each phase
of a Factory board and the labels and people it can be filtered by.
([#30177](https://github.com/mastra-ai/mastra/pull/30177))

  ```bash
mastra api factory board phases <project-id> work
'{"label":["bug"],"limit":10}'
  mastra api factory work-item filter-options <project-id>
  ```

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## @mastra/deployer@1.84.0-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/mcp-docs-server@1.3.14-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
## @mastra/playground-ui@69.0.1-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/client-js@1.60.1-alpha.0
  - @mastra/react@1.15.1-alpha.0
## @mastra/server@1.84.0-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
## @mastra/elysia@0.1.24-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/express@1.5.28-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/fastify@1.5.28-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/hono@1.7.26-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/koa@1.7.28-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/nestjs@0.3.1-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/next@0.2.42-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/hono@1.7.26-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/tanstack-start@0.2.42-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/hono@1.7.26-alpha.0
  - @mastra/server@1.84.0-alpha.0
## @mastra/github-signals@0.5.3-alpha.0

### Patch Changes

- Fixed delivery of authorized review comments and inline findings even
when bot summaries change or check status is unknown. Limit comment
bursts to ten findings and a counted overflow notice per poll.
([#30246](https://github.com/mastra-ai/mastra/pull/30246))

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
## @mastra/temporal@0.6.1-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/deployer@1.84.0-alpha.0
## create-mastra@1.39.1-alpha.0


## @internal/playground@1.39.1-alpha.0

### Patch Changes

- Updated dependencies
[[`175b4b5`](175b4b5194),
[`5266060`](5266060ccd),
[`9abbec6`](9abbec671c)]:
  - @mastra/core@1.84.0-alpha.0
  - @mastra/client-js@1.60.1-alpha.0
  - @mastra/react@1.15.1-alpha.0
  - @mastra/playground-ui@69.0.1-alpha.0

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-12 16:45:14 -08:00
dane-ai-mastra[bot]
d60ba71be0 chore: version - exit prerelease mode 2026-11-12 21:12:29 +00:00
dane-ai-mastra[bot]
5c87442ed7
chore: version packages (alpha) (#30203)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases

> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.

## @mastra/auth-auth0@1.3.0-alpha.0


## @mastra/auth-clerk@1.4.0-alpha.0


## @mastra/auth-okta@0.3.0-alpha.0


## @mastra/auth-supabase@1.2.0-alpha.0


## @mastra/client-js@1.60.0-alpha.13


## @mastra/auth@1.2.0-alpha.0


## @mastra/core@1.83.0-alpha.13


## @mastra/playground-ui@69.0.0-alpha.15


## @mastra/rag@2.8.0-alpha.3


## @mastra/server@1.83.0-alpha.13


## @mastra/server-adapters-test-suite@0.3.0-alpha.5


## @mastra/auth-better-auth@1.1.11-alpha.2


## @mastra/auth-cloud@1.2.9-alpha.0


## @mastra/auth-neon@0.3.6-alpha.1


## @mastra/auth-workos@1.6.9-alpha.0


## @mastra/discord@1.3.4-alpha.0


## @mastra/react@1.15.0-alpha.13


## @mastra/deployer-cloud@1.83.0-alpha.13


## @mastra/deployer-cloudflare@1.2.42-alpha.3


## @mastra/deployer-netlify@1.2.42-alpha.1


## @mastra/deployer-sandbox@0.3.27-alpha.1


## @mastra/deployer-vercel@1.2.42-alpha.1


## @mastra/longmemeval@1.1.42-alpha.13


## @mastra/opencode@0.1.42-alpha.13


## @mastra/factory@0.28.0-alpha.15


## create-factory@0.4.3-alpha.15


## @mastra/code-sdk@1.19.0-alpha.15


## mastracode@0.50.0-alpha.14


## @mastra/agent-builder@1.2.1-alpha.5


## mastra@1.39.0-alpha.15


## @mastra/connect@1.4.0-alpha.6


## @mastra/deployer@1.83.0-alpha.13


## @mastra/editor@0.17.1-alpha.7


## @mastra/mcp@2.9.0-alpha.5


## @mastra/mcp-docs-server@1.3.13-alpha.17


## @mastra/memory@1.41.1-alpha.5


## @mastra/schema-compat@1.4.0-alpha.2


## @mastra/elysia@0.1.23-alpha.13


## @mastra/express@1.5.27-alpha.13


## @mastra/fastify@1.5.27-alpha.13


## @mastra/hono@1.7.25-alpha.13


## @mastra/koa@1.7.27-alpha.13


## @mastra/nestjs@0.3.0-alpha.13


## @mastra/next@0.2.41-alpha.13


## @mastra/tanstack-start@0.2.41-alpha.13


## @mastra/voice-google-gemini-live@0.14.19-alpha.2


## @mastra/voice-openai-realtime@0.14.9-alpha.3


## @mastra/voice-xai-realtime@0.2.18-alpha.2


## @mastra/inngest@1.11.2-alpha.9


## @mastra/temporal@0.6.0-alpha.13


## create-mastra@1.39.0-alpha.15


## @internal/playground@1.39.0-alpha.15

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-12 12:07:03 -08:00
mastra-platform[bot]
5ff479a34b
fix(inngest): stop steps on every worker at the total deadline (#29002)
## Checks on `b2d9fdf436` — sibling stops and awaited background
cancellation

- A timeout/abort marker on any sibling tool result takes precedence
over a mapping failure, regardless of result order. Mapping failures
still throw when the run has not stopped.
- Cancellation while awaiting background work returns the stopped tool
result with `deadlineTimeout` or `runAborted`. Only rejection with the
signal's actual abort reason is converted; unrelated wait failures still
propagate.
- Regression tests use the real background task manager and cover
deadline/abort before dispatch, during execution, healthy execution and
tool failure in foreground/background modes, across both storage
versions. Focused suites: **58 passed**.
- Revert proofs: restoring per-result mapping checks fails **8 sibling
cases**; removing the background cancellation catch fails **4
during-wait cases**. Both fixes restored and all 58 tests rerun green.
- Broader core checks, including native outcomes across core engines and
complete timeout/message-reference recovery suites: **320 passed / 6
skipped**.
- Real Inngest dev server, retries disabled: two-worker suite **5
passed**; native-outcomes, parity-error-surfaces and
memory-save-on-terminal-failure combined **75 passed / 10 existing
expected failures / 4 skipped**. No expected-failure markers changed.
- Core/Inngest `tsc --noEmit`, oxlint, oxfmt and whitespace checks pass.
Changeset now uses short, outcome-focused paragraphs while retaining the
abort-signal/grace-window and non-cooperative-tool limitations.
- Production multi-pod placement, non-harness pubsub transports and host
clock skew remain unverified.

## Main merge and checks on `0b0e117d96`

## Checks on 579e3ef14d — queued tool dispatch after stop

- Added a composed-signal guard before foreground/background dispatch: a
step whose carried deadline already expired does not invoke the tool or
enqueue a background task, and preserves the timeout/abort marker.
- Regression matrix: foreground/background × expired carried
deadline/plain abort/healthy control, real BackgroundTaskManager, both
storage versions: 16 passed. Removing only the guard fails 8 cases on
actual invocations/task creation. Healthy controls also verify the
returned result and completed background task.
- Core focused recovery/deadline checks: 12 project/file runs, 156 tests
passed; the suspected message-ref recovery regression does not reproduce
with this guard on the merged head.
- Real Inngest two-worker suite: 5 passed. Separately run Inngest
parity-error-surfaces: 20 passed / 2 skipped;
memory-save-on-terminal-failure: 17 passed / 9 expected failures / 2
skipped. All retries off; existing expected-failure markers unchanged.
- Core and Inngest tsc, oxlint, oxfmt, and diff whitespace checks pass.
Core changeset now describes the abort grace window and non-cooperative
tool limitation accurately.
- Production multi-pod placement, non-harness pubsub transports, and
host clock skew remain unverified.


Merged main after #29036 in `ce95d269ba` (merge, not rebase). Resolved
the two scenario files by preserving main's durable/evented
deadline-save behavior and this PR's Inngest finish-path assertions.
Four newly passing Inngest tool-result rows use `itStoppedStep`; the
model-output variant finishing after the deadline remains expected to
fail.

- Core scenario files plus total-timeout suite: 240 passed, 20 skipped.
After marker changes, core memory-save scenario: 112 passed.
- Inngest two-worker deadline suite: three runs, 5 passed each, retries
off.
- Inngest parity-error-surfaces: 20 passed, 2 skipped.
- Inngest memory-save-on-terminal-failure: two consecutive passing runs,
each 17 passed, 9 expected fail, 2 skipped.
- Core and Inngest TypeScript checks passed after the merge; core
TypeScript rerun passed after marker changes. Formatting passed; oxlint
found no applicable files for the final scenario-only change.
- The pre-aborted-tool dispatch guard is now included in `579e3ef14d`.
The earlier suspected message-reference recovery regression did not
reproduce on the merged head; full recovery tests and guard revert
proofs pass (see checks above).

## Summary

When `modelSettings.timeout.totalMs` expired on an Inngest durable
agent, only the process that started the run stopped. A tool already
running on another Inngest worker kept going, and the loop could call
the model again after the caller had reported the timeout.

This PR makes the deadline stop the run on every worker:

- **Deadline carried with the run** (`abort-transport.ts`,
`create-inngest-agent.ts`): the caller puts `{ at, timeoutMs }` on
request context (`RUN_DEADLINE_KEY`) for trigger and resume events. A
worker's abort listener arms a timer for it, or aborts right away if the
deadline has already passed.
- **Deadline arms a timer on every worker**: a worker that receives the
carried deadline aborts the run controller with a `MastraTimeoutError`
when it passes, and the tool's `abortSignal` sees that reason. The
caller no longer publishes the deadline as an abort request (d0d4e64c):
the worker timers already cover it, so the timeout-reason plumbing in
`streams/abort-transport.ts` and `streams/publisher.ts` is back to main.
- **Tool steps listen for aborts** (`tool-call.ts`): tool steps now
install the cross-process abort listener and retry the subscribe if an
earlier step's subscribe failed. Retry is tracked with
`remoteAbortListenerInstalled`, not with whether a signal exists.
- **Stop carried on tool output** (`deadlineTimeout` / `runAborted` on
tool output → llm-mapping → iteration state): the loop predicate ends
the run even on a worker that never heard the abort. This also covers a
call whose `toModelOutput` throws after the deadline stopped it: the
mapping-failure result now carries the stop flags, and llm-mapping skips
the mapping-error throw when they are set, so the run ends as the
timeout/abort instead of a mapping failure that never saves the turn
(80921249).
- **Final map** (`create-inngest-agentic-workflow.ts`): a run past its
carried deadline ends as the timeout (`finishReason: 'error'`, error
event). An aborted run ends as `'abort'`.
- **One error for the caller** (`stream-adapter.ts`): if the caller and
a worker both report the same deadline error, the stream keeps one copy.
- **Deadline timer is independent of the subscription**
(`abort-transport.ts`): the carried deadline is armed before
subscribing, so a failed subscribe still stops the step. A warm worker
re-arms it when a resumed segment carries a new deadline, and drops it
when the resumed segment carries none, so the old segment's timer cannot
stop the resumed run. The timer is bound to the controller it was armed
for, so after a warm resume swaps in a fresh controller, the old timer
cannot abort it even before the resumed segment's first step re-syncs
the deadline. Tool steps call the listener on every step for this.
- **Deadline shows in the final result and tracing**: a deadline-stopped
run ends with step reason `'error'` (tripwire still wins), and its model
and agent spans close as `limit` / `timeout` via
`executionFailureTermination`, not as a normal finish. The final map
checks the deadline again once the finish side effects and thread title
generation are done, so a deadline that passes during the memory save,
output processors or title generation also ends the run as the timeout.
A run the caller aborted first stays an abort: the carried deadline is
not used to turn it into a timeout.

- **Caller accepts its own worker's timeout**
(`create-inngest-agent.ts`): in one process the worker's
carried-deadline timer and the caller's timer abort the same controller.
If the worker's fired first, the caller used to skip emitting the
`MastraTimeoutError`. It now treats any `MastraTimeoutError` on the
signal as its own timeout.

Split from #28950. This PR does not include the caller-side fallback
save at the deadline or the recording of late tool results; those come
in a later PR.
- **Fresh worker controller on warm resume** (2af87c6a): if the previous
segment's total deadline stopped a warm worker's own controller while
the run was suspended, the resumed segment gets a fresh controller
instead of starting already stopped. Owner controllers and other aborts
are kept. The test "gives a warm worker's resumed segment a fresh
controller when the previous deadline passed while suspended" fails with
the fix reverted.
- **Narrower error dedupe** (075ebd8c): the durable stream adapter folds
only a repeated `MastraTimeoutError`. Other errors with a matching name
and message are always delivered. The test "still reports a run error
whose name and message match a streamed non-timeout error" fails with
the fix reverted.

## Proof

- New `durable-deadline-cross-worker.test.ts`: three real worker
processes behind a round-robin proxy and a real Inngest dev server. It
covers five cases: tool stopped on another worker at the deadline;
`abort()` still stops it; a step that first reaches a worker after the
deadline stops; a last step on a fresh worker ends as the timeout; an
aborted run on a fresh worker that never heard the abort makes no more
model calls. Each case asserts exactly one `MastraTimeoutError` and no
further model calls. **3/3 runs passed** with retries off.
- Revert proofs. With each fix reverted on its own, a test fails:
  - carried deadline → 2 cross-worker cases fail
  - loop predicate check → cross-worker suite fails
  - final-map deadline check → cross-worker suite fails
- stop flag on tool output → "fresh worker that never heard the abort"
fails (`finishReason` `'stop'` instead of `'abort'`)
- tool-step listener retry (keyed on signal instead of flag) →
`tool-call-abort-listener.test.ts` fails
- stream dedupe → new `stream-adapter-timeout-dedupe.test.ts` fails (2
errors instead of 1). The cross-worker suite alone did not catch this
because the stream usually closes on the first error.
- Review fixes, each with a revert proof:
- `durable-agent-cross-process-abort.test.ts` adds "arms the carried
deadline even when the subscription fails", "re-arms a warm worker's
deadline when a resumed segment carries a new one" "does not stop a warm
resume that carries no deadline at the previous segment's deadline" and
"does not stop a warm resume at the previous segment's deadline before
its first step starts" (fake timers). Each fails with its
`abort-transport.ts` fix reverted.
- `native-outcomes.scenario.test.ts` adds "reports a deadline hit during
a tool call as a limit". It passes on all 4 core engines and on the
Inngest lane. With the Inngest deadline span termination removed, it
fails (`outcome` `'error'` instead of `'limit'`).
- `native-outcomes.scenario.test.ts` adds "reports a deadline that
expires while the run is finishing as a limit" (an output processor
sleeps past `totalMs`). It passes on Inngest, durable and evented; it is
skipped on the plain loop engines, whose stream has already closed by
then. With the post-side-effects deadline re-check removed, Inngest ends
the run as `success`.
- `native-outcomes.scenario.test.ts` adds "keeps a caller abort when the
deadline passes while the aborted run finishes" and "reports a deadline
that expires while the thread title is generated as a limit" (title
model wrapped with a 2.5s delay). Without the `runAborted` guard the
abort ends as `limit/timeout`; with the re-check before the title wait
the title case ends as `success`. All 5 Inngest deadline cases pass;
core engines pass 14 (both new cases skipped on the plain loop engines).
- After the review fixes, the two-worker suite passed 3/3 again (5/5
each, retries off).
- Core durable suites: 188 files / 1910 tests pass. Core
parity-error-surfaces for all non-Inngest engines: 68 pass.
- Inngest aimock lane, run serially (`INNGEST_TEST_PORT_OFFSET=2000`):
parity-error-surfaces 20/20 pass. The 4 files that fail
(`stream-until-idle-detach`, `observe-lifecycle`, `observe-reattach`,
`stream-until-idle-parked-cancel`; 32 tests) fail the same way on
`origin/main` (0db65eb9a1). One `output-stream-processor` case failed
once, then passed 3/3 when rerun alone.
- `tsc --noEmit` is clean for core and inngest. oxlint and oxfmt are
clean.

## Behaviour change in existing tests

On Inngest, a tool that ignores its `abortSignal` used to keep its step
hanging forever, so nothing after the deadline was saved. The tool step
now ends once the abort grace period passes. The worker then finishes
the turn through its normal finish path, the same path that already
saves a tool that returns after the deadline. The step is saved with the
aborted-tool notice. Two parity tests change their Inngest expectations
to match:

- "closes a stuck tool run on time": the save is released, and the test
checks that the abort notice is saved.
- "only steps that ended reach memory at the deadline": on Inngest, the
completed step and the stopped step are both saved. readOnly still saves
nothing.
- `memory-save-on-terminal-failure` (from #29029/#29047 on main):
Inngest stays in its "deadline save pending" list, because rows that
need the caller-side fallback save (e.g. a slow model reply with no step
finished) still fail. Rows that now pass on Inngest use a separate
`itStoppedStep` marker. Two of them ("saves the text of a step whose
call was cut off once" and "saves a step whose tool call failed
alongside one cut off") fail with this PR's source reverted to main; the
other five also pass on main's source, where main's `it.fails` pin only
held because the test read memory 300ms after the error. Inngest saves
land after the caller reports the timeout, so those rows poll memory
instead of a fixed 300ms wait. On Inngest the "before any step finishes"
row now also saves the streamed assistant text. 3/3 Inngest runs: 12
pass, 8 expected-fail, 1 skipped; core engines: 52 pass, 32
expected-fail.

Other engines' expectations are unchanged.

### Checks on 075ebd8c
- Core: `durable-agent-cross-process-abort` and
`stream-adapter-timeout-dedupe` 28/28 passing; core and Inngest tsc,
oxlint and oxfmt clean.
- Inngest two-worker deadline suite: 3/3 runs, 5/5 each, retries off.
- Inngest aimock `native-outcomes` and
`memory-save-on-terminal-failure`: both exited 0.
- Inngest aimock `parity-error-surfaces`: 20 passed, 2 skipped (run on
its own, retries off). An earlier run had 4 timeouts in the tool-timeout
block because it was the last of six suites run back to back and started
cold (first row took 45s).

### Checks on d0d4e64c (beb23d77 only rewords the core changeset)
- Mapping-failure fix: llm-mapping tests "ends the run as the total
deadline / abort() instead of the mapping failure" fail with only the
`llm-mapping.ts` change reverted; tool-call step test "marks a call
whose toModelOutput throws after the deadline stopped it" fails with
only the `tool-call.ts` change reverted. New native-outcomes row
"reports a deadline hit during a tool call whose toModelOutput throws as
a limit" passes on normal, fs, durable, evented and Inngest.
- Touched core tests: 48 files / 373 tests pass (durable steps,
cross-process abort, abort ack, stream dedupe, streams). Core and
Inngest tsc, oxlint, oxfmt clean.
- Inngest two-worker deadline suite: 3/3 runs, 5/5 each, retries off.
Inngest parity-error-surfaces: 20 passed, 2 skipped. Inngest
native-outcomes deadline rows: 6 passed.

## Unverified

- Real multi-pod production deployments. Proof comes from local
multi-process workers behind a proxy.
- Pubsub transports other than the one in the Inngest test harness.
- After the review fixes, Inngest parity-error-surfaces passed in 2 of 3
full-file runs. In one run, "closes a stuck tool run on time (save
blocked: true)" hit its 15s timeout; it passed when run alone and in
both later full-file runs. Not investigated further.
- Rebased onto main after #29047 (head `4e125e2c`). Rerun after the
rebase: two-worker suite 3/3 (5/5 each, retries off); Inngest
parity-error-surfaces 20 pass / 2 skipped; Inngest native-outcomes 37
pass / 1 expected-fail; core durable suite 3975 tests pass; core and
inngest tsc, oxlint, oxfmt clean. Revert proof for the caller fix: the
tripwire memory-save row failed 1 of 3 runs without it, 3/3 pass with
it.
- Core native-outcomes "keeps a generation ongoing while a provider
attempt is retried" failed in the full file because the new
title-deadline test returned while its delayed title request was still
pending; that request then hit the shared aimock server during the retry
test. Fixed in c58deaf6: the test now waits for the title request. The
full file passes 2/2 (146 passed, 6 skipped); the retry and title tests
run together fail with the old test and pass with the fix.
- `durable-agent-crash-recovery.test.ts` runs out of worker heap in this
sandbox, on pristine `origin/main` too; with `--max-old-space-size=6144`
it passes 10/10 on this branch.
- Warm-resume timer fix (7d85cf14): the new test fails on c58deaf6 and
with only the `abort-transport.ts` change reverted (2/2 projects),
passes with it (18/18). Rerun: two-worker suite 3/3 (5/5 each, retries
off); Inngest parity-error-surfaces 20 pass / 2 skipped; Inngest
memory-save 12 pass / 8 expected-fail / 1 skipped; Inngest
native-outcomes 37 pass / 1 expected-fail; core native-outcomes 146 pass
/ 6 skipped; core and inngest tsc, oxlint, oxfmt clean.

## Still open

- History-off / custom-history: the finish path saves the stuck-tool
turn after the deadline (see Risk). This moves to #28886.

## Risk: medium

- On Inngest, the normal finish path saves a stuck-tool turn after the
deadline. That path flushes the save queue without checking
`historyOff`, a custom history processor, or history turned off at
runtime. So these configurations now save the turn where they used to
save nothing (the tool hung and the save never ran). The later
fallback-save PR adds the deadline save that respects those processors.
- Changesets: `huge-rivers-dance` (`@mastra/core`) and
`polite-snakes-teach` (`@mastra/inngest`) describe the new behaviour.
`late-tool-results-saved.md` was removed on main, so this PR no longer
edits it.

## Assumptions

- The final map still runs normal finish side effects (memory save) on a
deadline run, then emits the error event instead of finish. Skipping the
save would have broken the existing Inngest "tool that finishes after
the deadline still saves its step" behaviour.
- The two-worker fallback-save row from #28950 was dropped. The
output-pass/memory assertions were trimmed to stop + one-error checks,
with an output-pass `finishReason` probe for the final-map case.
- Changesets were generated with `pnpm changeset`, one per package
(`@mastra/core`, `@mastra/inngest`).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

When a run reaches its time limit, each worker can stop the tools it
runs. The run then reports one timeout, and a caller abort stays a
separate outcome.

## Summary

- Carries the absolute run deadline in initial and resumed Inngest
events.
- Arms a deadline timer on each worker. The timer runs even when
abort-listener subscription fails.
- Subscribes tool steps to remote abort requests from other processes. A
remote abort reaches a running tool.
- Keeps the abort reason. A remote abort no longer becomes a generic
`Error('Aborted')`.
- Refreshes deadline state on warm resumes. Replaces a stale worker
controller and clears stale timers.
- Passes deadline and run-abort markers through tool output, mapping
output, and iteration state.
- Stops the agent loop after a deadline or run abort. Emits a
non-continuing step finish.
- Does not invoke a tool whose step starts after the run stopped.
- Returns an aborted-tool notice for a tool that ignores its signal.
Saves that notice in the step.
- Rechecks the deadline after finish side effects and title generation.
- Resolves the final error in this order: total-timeout abort reason,
serialized deadline state, then an expired request-context deadline.
Abort outcomes stay separate from timeout outcomes.
- Reports the deadline as a `MastraTimeoutError`. Deadline-stopped runs
use failure termination data in both spans, unless a tripwire is
present.
- Deduplicates a repeated matching `MastraTimeoutError` in the durable
stream adapter. Non-timeout errors are not deduplicated.
- Runs deadline-stopped finish side effects, including memory saves.
- Adds `readRunDeadline`, `RUN_DEADLINE_KEY`, and the `RunDeadline`
type. Adds an optional `deadline` parameter to
`ensureRemoteAbortListener`.
- Adds `executionFailureTermination` to
`workflows/inngest/src/observability-compat.ts`.
- Adds a cross-worker Inngest test suite with a fixture agent, a worker
process, and a dev-server helper.
- Updates abort, mapping, memory-save, outcome, and tracing tests. Adds
changesets for `@mastra/inngest` and the durable agent.

Scope: 26 files changed, +1711 and -142 lines, per the pull request
diff.

## Validation and limitations

Current-head CI, build, and test results are not available. The supplied
author-reported results are:

- Three runs of the two-worker Inngest deadline suite. Five tests passed
in each run, with retries disabled.
- In Inngest `parity-error-surfaces`, 20 tests passed and 2 were
skipped.
- Core and Inngest TypeScript checks, oxlint, and oxfmt passed.
- One `parity-error-surfaces` run timed out on "closes a stuck tool run
on time (save blocked: true)". The test passed when run alone and in two
later full-file runs.

These results are author-reported and were not independently verified
here.

Open risks:

- The Inngest finish path can save a stuck-tool turn after the deadline.
This can happen after the abort grace period.
- That save does not respect `historyOff` or custom history processors.
This handling is deferred to `#28886`.
- Real multi-pod deployments and pubsub transports outside the Inngest
test harness are not verified.
- The caller-side fallback save and recording of late tool results are
excluded. The author plans them for a later PR.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: NikAiyer <nick.aiyer@gmail.com>
2026-11-12 08:08:03 -08:00
arctic-char
c0d3bfdc43
feat: pass the authenticated user to DynamicArgument callbacks (#28612)
Dynamic agent config callbacks now get the authenticated end user as an
optional `user` argument, so you can personalize an agent without
digging auth keys out of `requestContext` yourself. It works in any
`DynamicArgument` callback: instructions, model, tools, memory,
workspace, processors, scorers, and so on.

```ts
import { Agent } from '@mastra/core/agent';

const supportAgent = new Agent({
  id: 'support-agent',
  instructions: ({ user }) =>
    user ? `You are helping ${user.name ?? user.id}.` : 'You are a helpful assistant.',
  model: ({ user }) => (user?.role === 'admin' ? 'openai/gpt-5' : 'openai/gpt-5-mini'),
  tools: ({ user }) => (user ? { accountTool, ...(user.role === 'admin' ? { adminTool } : {}) } : {}),
  memory: ({ user }) => (user ? userMemory : undefined),
});
```

`user` always has the same `MastraUser` shape, no matter which auth
provider you use, and it's `undefined` for unauthenticated requests:

```ts
import type { MastraUser } from '@mastra/core/types';

// { id: string; name?: string; email?: string; avatarUrl?: string; role?: string }
```

The server auth middleware builds it through a new overridable
`toMastraUser()` on `MastraAuthProvider`. The default handles common
shapes (`id`/`sub`/`userId`, `name`, `email`, `avatarUrl`/`picture`,
`role`), and JWT, Cloud, Supabase, Neon and Better Auth ship overrides
for their user objects. Custom providers can map their own shape:

```ts
import { MastraAuthProvider } from '@mastra/core/server';
import type { MastraUser } from '@mastra/core/types';

class MyAuth extends MastraAuthProvider<MyUser> {
  async authenticateToken(token: string) { /* ... */ }
  async authorizeUser(user: MyUser) { return true; }

  toMastraUser(user: MyUser): MastraUser {
    return { id: user.accountId, name: user.profile.displayName, role: user.permissions.primaryRole };
  }
}
```

Inline auth configs can pass `toMastraUser` too. If the provider user
has no role and an RBAC provider is configured, the primary RBAC role
fills it in, so `user.role` also works with WorkOS, Okta, Google and
Neon. The raw provider user is still available when you need
provider-specific fields, and the new context key is reserved, so
clients can't spoof it through the request body:

```ts
instructions: ({ requestContext }) => {
  const rawUser = requestContext.get('mastra__user'); // provider-specific object, unchanged
  // ...
},
```

Dynamic `vectorStore` resolvers on RAG tools get `user` as well:

```ts
createVectorQueryTool({
  indexName: 'docs',
  model: embedder,
  vectorStore: ({ user }) => (user ? tenantStores.get(user.id)! : sharedStore),
});
```

Inside core, every place that invoked a `DynamicArgument` callback
inline now goes through a single `resolveDynamicArgument` helper, which
reads the user from `requestContext` at call time. This is non-breaking:
callbacks get an extra optional field, providers get a method with a
working default. Requests served through an `MCPServer` don't get `user`
yet, since MCP builds its own request context; that's documented.

Tested with unit tests for the helper, normalization and each provider
override, middleware tests (inline config, provider role, RBAC role,
CompositeAuth ordering, unauthenticated), regression tests for the
migrated call sites, and an end-to-end test where an agent behind a fake
auth provider gets personalized instructions. Core, server, rag and the
touched auth packages typecheck clean, and their suites pass.



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5
When a signed-in user makes a request, configuration callbacks can now
use a standard set of details about that user. When a workflow resumes,
it uses the new request’s user details instead of restoring them from
the earlier run.

## Summary
- Added optional `user` data to `DynamicArgument` callbacks and RAG
vector-store resolvers. Callbacks can also use `getRequestUser()` to
read the normalized user from request context.
- Added the `MastraUser` type and `toMastraUser()` normalization hooks
for auth providers and inline auth configs. The server stores the
normalized user in a reserved request-context key and uses the primary
RBAC role when the user has no role.
- Updated core dynamic configuration paths to use the shared resolver.
Added user support to agent scopes and migrated A2A memory resolution.
- Added `withoutLiveRequestKeys()` to remove live authentication and
authorization data from saved context during workflow resume, restart,
and time travel. Saved non-authentication context, such as tenant data,
remains available.
- Added documentation and changesets. Raised relevant `@mastra/core`
peer dependency minimums to 1.81.0. MCPServer requests do not receive
the normalized `user`.

## Validation
The author reports passing typechecks and test suites across core,
server, RAG, and several auth packages. The author also reports three
failing Inngest tests and earlier failures in evented workflow and
scheduler tests, which reportedly fail the same way on comparison
sources. These results are author-reported and have not been
independently verified.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-12 10:55:12 -05:00
dane-ai-mastra[bot]
05a6931471
chore: version packages (alpha) (#30172)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases
## @mastra/client-js@1.60.0-alpha.12

### Minor Changes

- Allow `querySpans()` to sort by `model` or `provider`.
([#30096](https://github.com/mastra-ai/mastra/pull/30096))

  ```ts
  orderBy: [{ field: 'model', direction: 'asc' }];
  ```

### Patch Changes

- The generated route types for scores now include the `SCORER` and
`PROCESSOR` entity types, matching the entity types experiments can
emit. ([#30145](https://github.com/mastra-ai/mastra/pull/30145))

- Add span-local error counts.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/factory@0.28.0-alpha.14

### Minor Changes

- A card's sandbox now stops as soon as the card is done with that
context, instead of idling until the provider times it out.
([#29981](https://github.com/mastra-ai/mastra/pull/29981))

- When a card moves out of the last phase of a context into a phase that
runs in another one, Factory stops that context's sandbox. Triage →
Planning stops the triage sandbox; Planning → Building keeps the shared
`work` sandbox running. A run still in flight on the session keeps its
sandbox until it ends.
- When a card finishes, Factory stops every sandbox it holds, from any
Factory process, by identity through the configured sandbox provider.
Before, only the process that had opened a sandbox stopped it.
- A card that comes back to a context resumes the same sandbox with its
files.

Stopping never fails the move. Sandboxes are stopped, not destroyed;
deleting a session still destroys its sandbox.

- Added two read routes that answer what the new session page shows
without sending a project's cards.
([#30097](https://github.com/mastra-ai/mastra/pull/30097))

`GET /web/factory/projects/:id/work-items/starters` counts the cards
behind each starter on the project's default board, such as the cards
whose run failed, the pull requests awaiting review and the cards not
yet triaged.

`GET /web/factory/projects/:id/work-items/activity?since=<instant>`
returns how many cards the project has, and when cards were created and
completed since that instant. `since` is required and must be an ISO
date-time such as `2026-06-01T00:00:00.000Z`. The answer carries an
`ETag`, so asking again for the same `since` with `If-None-Match`
returns a 304 with no body when nothing changed.

### Patch Changes

- Fixed environment templates that could stop rebuilding for good. When
a build start was never recorded and the sandbox provider can't list its
builds (Mastra Platform), every later Build now, schedule and push run
used to skip with `unresolved` forever. Factory now treats that start as
failed 30 minutes after its lease lapsed and the next trigger builds.
([#30148](https://github.com/mastra-ai/mastra/pull/30148))

Saving a change to a template input (setup commands, the working
directory, settings, environment repositories, source-control
connections or Build variables) now rebuilds the template once the
project has one, so new sandboxes stop rerunning setup on every boot.
Saves that change nothing don't build, and a change saved during any
build, scheduled ones included, is built when that build ends.

- Factory sessions and MCP clients can now read the Factory's audit
trail with `factory_list_audit_events`: who did what to which work item,
run, branch or setting, and when. Filter by actor (`me`, a user id, or
an agent, optionally with the agent runs a person started), namespace or
exact actions, work item and time range, and page back with a cursor.
The tool only returns this Factory's events and never returns request
context (IP address, user agent), actor emails or event metadata beyond
who started an agent run. It leaves out events from or about other
people's private sessions.
([#30165](https://github.com/mastra-ai/mastra/pull/30165))

- Added a "Join the issue's card instead" switch to intake routes in the
Factory UI. A route for pull requests that close an issue can now put
the pull request on that issue's card instead of giving it a card of its
own; until now only the route Factory seeds could do this. The switch
shows on new and saved routes whose conditions include "Source kind is
Pull request" and "Pull request closes an issue is Yes".
([#30197](https://github.com/mastra-ai/mastra/pull/30197))

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/code-sdk@1.19.0-alpha.14
  - @mastra/mcp@2.9.0-alpha.4
## @mastra/observability@1.24.0-alpha.7

### Minor Changes

- Add span-local error counts.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/core@1.83.0-alpha.12

### Minor Changes

- Add model and provider as span query sort fields.
([#30096](https://github.com/mastra-ai/mastra/pull/30096))

  ```ts
  orderBy: [{ field: 'model', direction: 'asc' }];
  ```

- Add span-local error counts.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))

### Patch Changes

- Name the agent in the missing-storage error thrown by `Memory`. When
an agent's memory never received storage because the agent was never
attached to a Mastra instance, accessing `memory.storage` now fails with
`[Agent:<name>] Memory requires a storage provider to function. ...`, so
it is clear which agent is misconfigured. The registration and
standalone-storage guidance added in #29972 is unchanged, and agents
that inherit storage from their Mastra or configure it directly on
`Memory` are unaffected.
([#30164](https://github.com/mastra-ai/mastra/pull/30164))

- Experiments with a scorer or processor target now save their scores,
and workflow step scorers in experiments now save their step scores.
Previously these scores were silently discarded.
([#30145](https://github.com/mastra-ai/mastra/pull/30145))
## @mastra/mcp-apps-components@0.1.0-alpha.7

### Minor Changes

- Dashboards authored in chat now accept the same widgets as the
dashboard editor: KPIs that combine values (ratio, difference, max,
fallback), read trace, span or score aggregates (for example p95 span
duration) and use duration, percent or score formats; line charts of
scores; six more chart colors (blue, orange, pink, yellow, red, purple);
and optional KPI icon and detail, series color and table empty message.
([#30192](https://github.com/mastra-ai/mastra/pull/30192))

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/playground-ui@69.0.0-alpha.14

### Minor Changes

- Widened what dashboard widgets can show:
([#30192](https://github.com/mastra-ai/mastra/pull/30192))

- KPIs can combine values with `ratio`, `minus`, `max` and `coalesce`,
read trace and span aggregates (for example `duration.p95`) and score
aggregates, and are formatted as `duration`, `percent` or `score`.
Combined values are compared with the previous window.
- Charts accept six more colors: `blue`, `orange`, `pink`, `yellow`,
`red` and `purple`.
- KPI `icon` and `detail`, series `color` and table `emptyMessage` are
optional. Series without a color pick one from a palette, and tables
without an empty message show "No data".

### Patch Changes

- Fixed hover highlights on cards and other surfaces stopping short of
the edge. The highlight now covers the whole surface and follows its
rounded corners exactly, including the sidebar credits card and
clickable cards. Hover highlights on these surfaces now switch instantly
instead of fading.
([#30047](https://github.com/mastra-ai/mastra/pull/30047))

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/client-js@1.60.0-alpha.12
  - @mastra/react@1.15.0-alpha.12
## @mastra/server@1.83.0-alpha.12

### Minor Changes

- Support model and provider sorting in span queries.
([#30096](https://github.com/mastra-ai/mastra/pull/30096))

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/clickhouse@1.30.0-alpha.4

### Minor Changes

- Sort span queries by model or provider in ascending or descending
order. ([#30096](https://github.com/mastra-ai/mastra/pull/30096))

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/duckdb@1.19.0-alpha.5

### Minor Changes

- Sort span queries by model or provider in ascending or descending
order. ([#30096](https://github.com/mastra-ai/mastra/pull/30096))

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/pg@1.36.0-alpha.6

### Minor Changes

- Sort span queries by model or provider in ascending or descending
order. ([#30096](https://github.com/mastra-ai/mastra/pull/30096))

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/react@1.15.0-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/client-js@1.60.0-alpha.12
## @mastra/deployer-cloud@1.83.0-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/deployer@1.83.0-alpha.12
## @mastra/longmemeval@1.1.42-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/libsql@1.30.0-alpha.3
## @mastra/opencode@0.1.42-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/libsql@1.30.0-alpha.3
## create-factory@0.4.3-alpha.14

### Patch Changes

- Updated dependencies
[[`1fbc69a`](1fbc69aa9a),
[`4e378c3`](4e378c3e26),
[`1fbc69a`](1fbc69aa9a)]:
  - mastra@1.39.0-alpha.14
## @mastra/code-sdk@1.19.0-alpha.14

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
  - @mastra/duckdb@1.19.0-alpha.5
  - @mastra/client-js@1.60.0-alpha.12
  - @mastra/pg@1.36.0-alpha.6
  - @mastra/mcp@2.9.0-alpha.4
  - @mastra/libsql@1.30.0-alpha.3
## mastracode@0.50.0-alpha.13

### Patch Changes

- Fix `/settings` crashing when mouse preferences are missing from saved
settings by defaulting to Off.
([#30184](https://github.com/mastra-ai/mastra/pull/30184))

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882),
[`f071044`](f0710446e0)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
  - @mastra/duckdb@1.19.0-alpha.5
  - @mastra/pg@1.36.0-alpha.6
  - @mastra/code-sdk@1.19.0-alpha.14
  - @mastra/mcp@2.9.0-alpha.4
  - @mastra/libsql@1.30.0-alpha.3
## @mastra/arize@1.3.30-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/arthur@0.4.30-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/braintrust@1.3.27-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/datadog@1.4.22-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/deepeval@0.1.24-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/laminar@1.3.32-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/langfuse@1.6.6-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
  - @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/langsmith@1.4.3-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/otel-bridge@1.5.22-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
  - @mastra/otel-exporter@1.5.2-alpha.7
## @mastra/otel-exporter@1.5.2-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/posthog@1.4.7-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
## @mastra/sentry@1.2.32-alpha.7

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`72abae6`](72abae68eb),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/observability@1.24.0-alpha.7
  - @mastra/otel-exporter@1.5.2-alpha.7
## mastra@1.39.0-alpha.14

### Patch Changes

- Added `mastra api factory work-item starters` and `mastra api factory
work-item activity`. The first counts the cards behind each new-session
starter on a project's default board. The second lists when a project's
cards were created and completed since an instant:
([#30097](https://github.com/mastra-ai/mastra/pull/30097))

  ```bash
  mastra api factory work-item starters <project-id>
mastra api factory work-item activity <project-id>
'{"since":"2026-06-01T00:00:00.000Z"}'
  ```

- Added a "Join the issue's card instead" switch to intake routes in the
Factory UI. A route for pull requests that close an issue can now put
the pull request on that issue's card instead of giving it a card of its
own; until now only the route Factory seeds could do this. The switch
shows on new and saved routes whose conditions include "Source kind is
Pull request" and "Pull request closes an issue is Yes".
([#30197](https://github.com/mastra-ai/mastra/pull/30197))

- Improved the Factory new session page on large boards. Its starters
and its activity band now load the counts and dates they show instead of
downloading every card of the project. They show the same content as
before. ([#30097](https://github.com/mastra-ai/mastra/pull/30097))

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/deployer@1.83.0-alpha.12
## @mastra/deployer@1.83.0-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/mcp-docs-server@1.3.13-alpha.16

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/elysia@0.1.23-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/express@1.5.27-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/fastify@1.5.27-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/hono@1.7.25-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/koa@1.7.27-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/nestjs@0.3.0-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
## @mastra/next@0.2.41-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
  - @mastra/hono@1.7.25-alpha.12
## @mastra/tanstack-start@0.2.41-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/server@1.83.0-alpha.12
  - @mastra/hono@1.7.25-alpha.12
## @mastra/voice-openai-realtime@0.14.9-alpha.2

### Patch Changes

- Use gpt-realtime-2.1-mini by default for OpenAI Realtime voice
connections instead of the retired
gpt-4o-mini-realtime-preview-2024-12-17 model.
([#30183](https://github.com/mastra-ai/mastra/pull/30183))
## @mastra/inngest@1.11.2-alpha.8

### Patch Changes

- Support span-local error counts in Inngest workflows.
([#30088](https://github.com/mastra-ai/mastra/pull/30088))

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
## @mastra/temporal@0.6.0-alpha.12

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`72abae6`](72abae68eb),
[`987566e`](987566e882)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/deployer@1.83.0-alpha.12
## create-mastra@1.39.0-alpha.14


## @internal/playground@1.39.0-alpha.14

### Patch Changes

- Updated dependencies
[[`56f654f`](56f654fc0d),
[`f071044`](f0710446e0),
[`987566e`](987566e882),
[`72abae6`](72abae68eb),
[`f9088d8`](f9088d8510),
[`987566e`](987566e882),
[`f071044`](f0710446e0),
[`43d1e9c`](43d1e9c16d)]:
  - @mastra/core@1.83.0-alpha.12
  - @mastra/client-js@1.60.0-alpha.12
  - @mastra/playground-ui@69.0.0-alpha.14
  - @mastra/react@1.15.0-alpha.12

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-12 14:42:02 +01:00
Eric Pinzur
72abae68eb
feat(observability): export span-local originating error counts (#30088)
## Description

Export `errorCount` as failures originating directly on each span. Child
failures forwarded through a model step, workflow or agent add no parent
count; independent parent failures do. Failed retries remain counted
after recovery, and replay does not create new occurrences. No
parent/child totals or suspend/resume rollup are maintained.

Attribution stays in a WeakMap without modifying user errors. Explicit
internal serializers and Inngest memoized operations preserve provenance
across boundaries; public workflow results and stored workflow state
omit transport metadata. Failures before child execution count on their
actual originating span. Excluded spans preserve attribution so their
failures cannot be misassigned to visible parents, but excluded failures
are absent from exported totals.

Retry context stays in the server-only context module, while shared
origin utilities use global crypto. Guarded cross-package calls preserve
mixed-version startup compatibility. Unsupported accounting and older
snapshots remain unknown. Error-free spans do not allocate occurrence
sets.

Validation: 117 focused observability/workflow/compatibility tests, 23
real Inngest error/retry scenarios and 37 Inngest engine/nested-result
checks pass. New regressions reproduced resolver undercounting,
excluded-child misattribution and public transport-field leakage before
the fixes. Real Inngest tests cover an empty execution graph,
unavailable remote child, instrumented child errors, nested propagation,
retries, public responses and stored state. Core/observability build,
Inngest typecheck, changed-file type-aware lint and CI formatting checks
pass. The preceding merge also passed 167 core checks and redaction
integration coverage.

Playground production builds were verified on Node 24. Docs checks with
Vale report no errors (three warnings on unchanged API headings); the CI
Vale configuration passes. Standalone observability `tsc` previously
reported generic union-call errors in untouched
`workflow-control-flow-suite.ts`; package builds and focused tests pass.
No production rollout or query-table changes. CI reruns on the new head.

## Related issue(s)

Fixes https://linear.app/kepler-crm/issue/OBS-901

Follow-ups: https://linear.app/kepler-crm/issue/OBS-902 and
https://linear.app/kepler-crm/issue/OBS-903

Nonblocking review follow-up:
https://linear.app/kepler-crm/issue/OBS-905 tracks explicit propagated
classification on previously untagged errors and reuse of one Error
object across separate foreach iterations. Changesets intentionally
remain single sentences per maintainer direction.

## Type of change

- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update
- [x] Test update

## Checklist

- [x] I have linked the related issue(s) in the description above
- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

This change counts each error on the span where it first happened. It
keeps the count correct when errors are wrapped, retried, or rebuilt, so
one failure is not counted twice.

## Summary

- Adds span-local `errorCount` to exported spans. The count covers
failures that start directly on that span.
- Adds `claimSpanError`, `BaseSpan.recordError`, and
`BaseSpan.restoreErrorCount`. Each claim records one occurrence ID on
the span.
- Ignores propagated failures on the receiving span. Propagated failures
count only on their origin span.
- Keeps error origin through cause chains, durable tool serialization,
and event codecs.
- Stores origin metadata in a `WeakMap` for live errors and formatted
results. Omits it from public workflow JSON.
- Stores occurrence IDs in span snapshots. Rebuilt spans restore them,
so replayed attempts are not counted again.
- Records failed attempts for default, evented, and Inngest workflow
retries. Each attempt uses an attempt-scoped occurrence ID.
- Reports `errorCount` as unknown when the runtime lacks the claim API
or when an older snapshot has no accounting history.
- Updates tracing types, the span record schema description, the span
reference documentation, and changesets for `@mastra/core`,
`@mastra/observability`, `@mastra/client-js`, and `@mastra/inngest`.
- Adds tests for repeated reports, retry scopes, concurrent async spans,
excluded children, cause-linked errors, tool and agent spans, rebuilt
and legacy snapshots, codecs, durable serialization, workflow retries,
and ESM/CJS compatibility with an older core build.

## Scope

- Parent and child totals are not computed.
- Suspend and resume rollups are not computed.
- Query-time summation, display, filtering, sorting, and aggregate-table
integration are not included. These are tracked in
[OBS-902](https://linear.app/kepler-crm/issue/OBS-902) and
[OBS-903](https://linear.app/kepler-crm/issue/OBS-903).
- Counts reflect observed exported failures only. Filtering, sampling,
or loss before export can leave counts incomplete.

## Change size

- 32 files changed
- 862 lines added
- 161 lines removed

## Validation

- Completed checks on the current head: not available from the supplied
evidence.
- Author-reported results, not verified here:
- Focused observability tests, codec and durable-serialization checks,
and engine and Inngest retry scenarios.
- Node 24 builds, 167 core checks, 125 observability and redaction
tests, and 19 real Inngest scenarios.
  - Type-aware lint and `lint:format`.
- Known limitations:
- A prior monorepo E2E attempt did not run application assertions.
Fixture dependency installation failed with a pnpm store ENOENT error.
- Earlier runtime tests ran on Node 23.11.0, which is outside the
supported engine range.
- Standalone observability `tsc` reports errors in an untouched test
utility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-12 14:21:22 +01:00
dane-ai-mastra[bot]
a4258a989c
chore: version packages (alpha) (#30127)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-11 21:12:19 -08:00
dane-ai-mastra[bot]
055a8c3b82 chore: version packages 2026-11-12 04:17:40 +00:00
mastra-platform[bot]
a0cd52cdc1
refactor(core): extract shared observe() fullStream detach wrapper (#30111)
> Opened by @wardpeet

## Result: Approved — c2acf6499e

The three duplicated `fullStream` detach wrappers are now one shared
`detachFullStreamOnEnd` helper. The extraction is complete,
behavior-preserving, and covered by the helper's own tests plus the
COR-1525 end-to-end regression.

### Checks on this head

| Check | Result | Tests |
| --- | --- | --- |
| Diff against base `9efd1ea8` | ✅ Passed | 1 focused commit, 8 files,
+186/−90 |
| Only one copy of the wrapper remains | ✅ Passed | helper only;
`agent.ts:12426`, `durable-agent.ts:5942`,
`create-inngest-agent.ts:2303` all call the helper |
| Removed inline wrapper matches helper body | ✅ Passed |
byte-identical, modulo `this` typing |
| `@mastra/core` typecheck (`pnpm check`) | ✅ Passed | 0 errors |
| `@mastra/inngest` typecheck (`tsc --noEmit -p tsconfig.json`) | ✅
Passed | 0 errors |
| Lint (`oxlint --type-aware`, changed files) | ✅ Passed | 0 warnings, 0
errors |
| New unit test `observe-full-stream.test.ts` | ✅ Passed | 4 tests × 2
vitest projects |
| `@mastra/core` durable suite (`src/agent/durable/`) | ✅ Passed | 3989
passed, 4 skipped; crash-recovery 20/20 with a raised heap (see note) |
| Inngest observe e2e (COR-1525 regression) | ✅ Passed | replays a
finished run through `observe(runId).fullStream` |
| Inngest unit `create-inngest-agent.test.ts` | ✅ Passed | 117 passed |

Note: the heavy `durable-agent-crash-recovery.test.ts` aborts one worker
on the sandbox's default Node heap limit; the same abort reproduces at
the base commit, and the file passes 20/20 with
`NODE_OPTIONS=--max-old-space-size=6144`. Environmental, not caused by
this change.

### Follow-ups

- `packages/core/src/agent/durable/observe-full-stream.test.ts`: the
end-path test counts `detach` calls but does not assert it is not called
before the source ends; an earlier assertion would tighten the contract.
- `packages/core/src/agent/durable/observe-full-stream.test.ts`: the
structural `FakeObserverOutput` stands in for the real
`MastraModelOutput`; real-path coverage is the COR-1525 Inngest e2e, so
this is proportionate.

### Adversarial check

Strongest case for request changes: the extraction silently altered
behavior (the wrapper's `this` receiver or the `ReadableStream`
constructor). It fails: the removed inline code is byte-identical to the
helper body, both instantiate the global `ReadableStream` (the
`node:stream/web` import is type-only), and the destructurable-proxy
path is exercised green by the COR-1525 e2e on this head.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

When an observer watches an agent's output stream, it must stop watching
at the right time. This PR moves that stop logic into one shared helper.
Both the core agent and the durable agent now use it. The Inngest
package keeps its own copy, because it can run with older core versions
that lack the helper.

## Overview

The PR extracts the `fullStream` detach wrapper from `Agent.observe()`
and `DurableAgent.observe()` into a shared helper,
`detachFullStreamOnEnd`, in `@mastra/core`. The helper is exported from
the durable-agent module. The `@mastra/inngest` package keeps its
wrapper inline, with a comment that explains why.

Diff scope: 7 files changed, 196 lines added, 60 lines removed.

## Changes

- **`packages/core/src/agent/durable/observe-full-stream.ts`** (+58):
Adds `detachFullStreamOnEnd(output, detach)`. It wraps the output's
inherited `fullStream` getter in a lazy stream. The wrapper calls the
original getter with the output as receiver and forwards chunks. It
calls `detach` when the reader reaches the end, when a read errors, or
when the consumer cancels. Cancellation forwards the cancel reason to
the underlying reader.
- **`packages/core/src/agent/durable/observe-full-stream.test.ts`**
(+109): Adds tests for chunk forwarding and detach on source end. It
also covers read-error propagation, cancellation reason forwarding, and
an inherited getter called through a proxy.
- **`packages/core/src/agent/durable/index.ts`** (+4): Re-exports
`detachFullStreamOnEnd` with a comment about use by integrations outside
core.
- **`packages/core/src/agent/agent.ts`** (+2/-30): `Agent.observe()`
calls the helper. The inline `fullStream` wrapper is removed.
- **`packages/core/src/agent/durable/durable-agent.ts`** (+2/-30):
`DurableAgent.observe()` calls the helper. Its custom `fullStream`
wrapper is removed.
- **`workflows/inngest/src/durable-agent/create-inngest-agent.ts`**
(+5): Adds a comment explaining why the `fullStream` detach logic stays
inline. The declared `@mastra/core` peer range includes versions that do
not export `detachFullStreamOnEnd`. No runtime behavior changes.
- **`.changeset/observe-full-stream-helper-core.md`** (+16): Adds a
patch changeset for `@mastra/core`. It documents the helper and states
that existing `observe()` behavior is unchanged.

## Public API

- Added: `detachFullStreamOnEnd<TOutput>(output:
FullStreamOutput<TOutput>, detach: () => void): void`, exported from the
durable-agent module in `@mastra/core`.
- No other exported declarations change.

## Testing

Results below come from the PR author. This summary did not rerun them.

- Typechecks pass for `@mastra/core` and `@mastra/inngest`.
- Lint passes on changed files.
- Four helper tests pass in each of two Vitest projects.
- The durable suite reports 3,989 passing tests and 4 skipped.
- `create-inngest-agent.test.ts` reports 117 passing tests.
- The crash-recovery test passes 20/20 with
`NODE_OPTIONS=--max-old-space-size=6144`. Under the sandbox's default
heap limit, one worker aborts. The same abort occurs at the base commit.
- The COR-1525 end-to-end regression replays a finished run through
`observe(runId).fullStream`.

## Known gaps

- The helper's end-path test does not assert that `detach` is not called
before the source ends.
- The test's structural `FakeObserverOutput` does not match the real
`MastraModelOutput`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: NikAiyer <nick.aiyer@gmail.com>
2026-11-11 20:11:53 -08:00
Leo
d7fa57619b
fix(core): keep sub-agents streaming after a durable resume with a request context (#29616)
## Summary

A `DurableAgent` run started with `stream()` stopped streaming its
sub-agents after a resume that passed a `requestContext`. On a warm
resume with a request context, the run's tools are rebuilt, and the
rebuild picked `generate` whenever `closeOnSuspend` was set.
`closeOnSuspend` is also a `stream()` option, and `resumeStream()` (so
`approveToolCall()` too) always sets it, so `agent-<name>` delegation
tools ran the sub-agent through `generate` and its chunks never reached
the stream as `tool-output`.

The fix records the method that started the run on the run registry
entry and rebuilds for that method. The same line in the Inngest durable
agent gets the same change.

The same hunk restores a second outcome: tool calls that a delegated
sub-agent makes on durable and evented runs report `subagent_tool_end`
again. The agent controller emits
`subagent_tool_start`/`subagent_tool_end` only while it reads the
sub-agent's stream (`agent-controller/tools.ts`) and uses
`subagent_tool_end` to set the tracked call's status
(`agent-controller/session.ts`), so while the delegation tool was built
for `generate` those rows never got a status or result. That is why the
`it.fails` pins on the durable and evented `delegated` cases in
`controller-tool-errors.scenario.test.ts` are removed; the changeset
says so too.

```diff
 // prepareForDurableExecution
 registryEntry = {
   tools,
   requestTools: tools,
+  methodType,            // 'stream' or 'generate', from the call that started the run
   ...
 }

 // warm resume with requestContext / delegation / hooks
 getToolsForExecution({
   ...
-  methodType: resolvedOptions.closeOnSuspend ? 'generate' : 'stream',
+  methodType: entry.methodType ?? 'stream',
 })
```

## Evidence

New shared aimock scenario
`packages/core/src/loop/test-utils/aimock/scenarios/subagent-resume-streaming.scenario.test.ts`.
A lead agent with a suspending `ask_user` tool and a sub-agent `helper`
(with its own tool) talk to a real OpenAI provider pointed at the aimock
HTTP server. The lead suspends, is resumed, then delegates to `helper`.
Each case checks that `helper`'s `tool-call`, `tool-result` and
`text-delta` chunks arrive on the resumed stream as `tool-output`, and
that every `helper` model request was a streaming request. Cases:

- `resumeStream()` with a `requestContext` (the #29436 path)
- `resumeStream()` without one (control)
- durable only: `resume()` with `requestContext` and `closeOnSuspend:
true`, as in the issue's repro
- durable only: a `generate()` run resumed with `resumeGenerate()` still
runs the sub-agent through `generate` (non-streaming requests)

- **Before (origin/main):** normal, durable, evented: `Tests 4 failed |
6 passed | 4 skipped (16)`. The 4 failures are the request-context cases
on durable and evented: `expected [] to deeply equal ArrayContaining
["tool-call","tool-result","text-delta"]`.
  **After:** `Tests  10 passed | 4 skipped (16)`.
- **Inngest engine** (real dev server,
`AIMOCK_SCENARIOS='subagent-resume-streaming' pnpm --dir
workflows/inngest test:aimock`): with the Inngest line reverted, `2
failed | 2 passed (4)`, the same two request-context cases. With the
fix: `4 passed (4)`.
- Neighbouring tests, unchanged and green (`Tests 130 passed | 3
expected fail | 11 skipped`): `durable-agent-resume-request-context`,
`durable-agent-request-context`, `durable-agent-delegation`,
`durable-agent-generate`, and the `request-context-resume`,
`supervisor-suspension`, `supervisor-delegation-generate`,
`parity-sub-agent-suspend`, `parity-sub-agent-delegation`,
`subagent-auto-resume`, `resume-stream` scenarios.
- `tsc --noEmit -p tsconfig.build.json` for core, `tsc --noEmit` for
`@mastra/inngest`, oxlint on the touched files and prettier on the
changed files all pass.
- `controller-tool-errors.scenario.test.ts` pinned its durable and
evented `delegated` cases with `it.fails` because those delegations
emitted no `subagent_tool_end`. This fix makes them pass (CI reported
`Expect test to fail` on all 14), so the marker is removed: `Tests 84
passed (84)`.

Not covered: a cold resume (registry entry gone, rebuilt from storage)
still calls `prepare()` with the default `stream`, so a `generate()` run
resumed in another process gets stream-built tools. That was already the
case before this PR, and the tools still work under `resumeGenerate()`,
so it is left as-is.

## Merge Danger

**Door:** two-way. A one-field registry change plus two one-line
call-site changes, all easy to revert.

**Blast radius:** warm `DurableAgent`, `EventedAgent` and Inngest
durable resumes that pass a `requestContext`, `delegation` or `hooks`.
One behavior change to know about: `resumeGenerate()` on a run started
with `stream()` now rebuilds its tools for `stream`, matching the first
segment and the no-context resume. Other open PRs touch
`durable-agent.ts` / `types.ts` (#28283, #28530, #29036, #29282,
#27522), but none changes these lines.

Closes #29436

## Council review

Rounds: 2

| Seat | Round 1 | Round 2 |
| --- | --- | --- |
| Claude Opus 5.5 | not run | 9/10 (Correctness 9, Tests 9, Scope 10,
Code quality 9) |
| GPT-6.1 Sol | 9/10 (Correctness 9, Tests 9, Scope 10, Code quality 9)
| 9/10 (Correctness 9, Tests 9, Scope 10, Code quality 9) |
| Grok 4.7 | empty seat | empty seat |

Created with Claude Opus 5.5 in T3 Code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5
When a paused run continues, it should keep working the way it started.
This change makes sub-agents keep sending progress updates after a
streamed run resumes. This holds even when the resume includes a
`requestContext` or `closeOnSuspend`.

## Summary
- Store the original method type (`stream` or `generate`) on each run
registry entry when the run is prepared.
- On warm resume, rebuild request-scoped tools with the stored method
type. If no method type is stored, use `stream`. Stop inferring the
method from `closeOnSuspend`.
- Apply the same change to the Inngest durable agent resume path.
- Add an AIMock scenario test for sub-agent streaming after resume, with
and without a request context. The test also checks that resumed
`generate()` runs make non-streaming helper requests.
- Remove the `it.fails` markers from the delegated-tool tests for the
durable and evented engines. These tests now run as ordinary tests.
- Add patch changesets for `@mastra/core` and `@mastra/inngest`. The
changesets reference issue `#29436`.

Scope:
- Cold resumes, which rebuild the run from storage, are not changed.
They still default to `stream`, including for a run that started with
`generate()`.

Test and CI results: not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:23:12 -05:00
dane-ai-mastra[bot]
9e82af0237
chore: version packages (alpha) (#30074)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-11 14:21:14 -08:00
Anurag Ojha
decc490d78
fix(inngest): save agent scores on the stored agent run span (#30016)
## Description

Scores from a `createInngestAgent` agent pointed at a span that is never
stored, so the agent run in a trace showed no scores. The score event
also lost the agent and the `runId`. The score now points at the run's
`agent_run` span, the same as for a plain `Agent`.

- The `execute-scorers` step passed its own span to the scorers. On
Inngest that span is internal and has no live parent, so the score
resolved to the internal workflow span.
- The step now passes the run's `agent_run` span, resolved the same way
as in the `map-final-output` step.
- Added one test to the shared scorer scenario. It runs on every engine
and failed on Inngest before the fix.

## Related Issue(s)

Fixes #29795

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

Inngest now attaches agent scores to the saved agent-run span. Scores no
longer point to a span missing from the run trace.

## Changes

- Reuses or rebuilds the `agent_run` span before executing scorers.
- Uses the existing tracing context only when no agent span is
available.
- Adds all-engine test coverage for matching score and stored span IDs.
- Adds a patch changeset for `@mastra/inngest`.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 19:08:00 +01:00
dane-ai-mastra[bot]
3a6ca42696
chore: version packages (alpha) (#30026)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-11 08:16:57 -08:00
dane-ai-mastra[bot]
8bce7717eb
chore: version packages (alpha) (#29986)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-10 20:51:40 -08:00
mastra-platform[bot]
69f5d83147
fix(inngest): keep background-task follow-ups on the open streamUntilIdle stream (#29877)
> Opened by @roaminro

Fixes #29785

## Summary
On agents created with `createInngestAgent`, `streamUntilIdle()` (and
`stream(..., { untilIdle: true })`) closed right after
`background-task-completed`. The follow-up answer to a deferred
background task never reached the stream. Instead, a `background-task`
notification signal woke the thread as a separate run, and the answer
only reached memory. This was a regression from #28998.

**Root cause:** #28998 added `Agent.__notifyBackgroundTaskFinished()`,
which skips the wake when `__untilIdleHandlesBackgroundTask()` reports
an open until-idle stream. Core's `DurableAgent` overrides that check to
use its own registry. `createInngestAgent` keeps its own
`activeStreamUntilIdle` map but had no override. Its Proxy forwarded
both hooks to the wrapped `Agent` and bound them to it, and that agent's
registry is always empty, so every finished deferred task woke the
thread.

## Changes
- `@mastra/core`: export the existing `untilIdleHandlesBackgroundTask`
helper from `@mastra/core/agent/durable`.
- `@mastra/inngest`: define `__untilIdleHandlesBackgroundTask` and
`__notifyBackgroundTaskFinished` on the Inngest agent so both check its
own `activeStreamUntilIdle` registry. Both overrides are needed: because
of the proxy binding, an override of the check alone would never be
reached from the wrapped agent's notify.
- Regression tests in `create-inngest-agent.test.ts`: no wake while an
until-idle stream is open, and the thread still wakes when no stream is
open.
- Changeset (patch for both packages).

## Self-review
Approved at `29c10035db`.

| Check | Result | Tests |
|---|---|---|
| New #29785 regression tests (`create-inngest-agent.test.ts -t 29785`)
| ✅ Passed | 2/2 |
| Full `create-inngest-agent.test.ts` | ✅ Passed | 115/115 |
| `tsc --noEmit` for `workflows/inngest` | ✅ Passed | — |
| `scripts/validate-peerdeps.mjs` | ✅ Passed | — |
| Core `build:lib` (incl. d.ts) | ✅ Passed | — |

**Regression proof:**
- With the base `create-inngest-agent.ts` restored, the open-stream test
fails. The idle-path test still passes.
- Keeping only the `__untilIdleHandlesBackgroundTask` override and
dropping the notify override also fails (`sendSignal` called 1 time).
The tests catch each override on its own.

**Not run:**
- Inngest aimock until-idle scenarios: the Inngest dev server exited
with code 1 in the sandbox. This is an environment failure, not a pass.
- The live repro from the issue: it needs `OPENAI_API_KEY`.

**Note:** the peer range `>=1.83.0-0` also admits the already-published
1.83.0-alpha.0–2, which don't export the helper. This is the same
accepted alpha-window gap as #29635, and the peer validator passes.

Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-10 23:08:25 -05:00
Arthur Fiorette
a30ee08ccd
fix(inngest): send only the response messages that changed on durable stream events (#29976)
## Description

Stacked on #29975. The Inngest durable agent's `step-finish` carried the
run's whole response message list on every step
(`_durableResponseMessages`), so its `agent.stream.<runId>` log grew
with the square of the step count, like core's DurableAgent did before
#29975.

It now publishes `_durableResponseMessagesDiff`, the same format as the
plain Agent and DurableAgent, and the shared stream adapter rebuilds the
full list for callers and observers:

- The diff is built inside the memoized emit step, which returns the
cursor, so Inngest replays reuse it instead of recomputing it; the
cursor travels on the iteration state like core's.
- The producer includes Inngest's function run id, so the first run and
each resume (or a rerun of the same Mastra run) start with the whole
list.
- Resume and `observe()` streams at an offset rebuild the skipped list
from the run's log first (`replayResponseMessagesBeforeOffset`).
- `diffResponseMessages` is exported from `@mastra/core/agent/durable`
for this; the `@mastra/core` peer floor stays `>=1.83.0-0`, as the stack
is meant to land before 1.83.0 stable.

Same upgrade note as #29975: a reader on an older `@mastra/core` reports
no response messages for steps published by this version.

## Related issue(s)

Completes COR-1532 (plain #29971, DurableAgent/EventedAgent #29975,
Inngest here). Related: #27475.

## Type of change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [x] Performance improvement
- [x] Test update

## Checklist

- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

Tests: core's `agent-stream-log` scenario file runs on Inngest through a
real dev server (`AIMOCK_SCENARIOS='agent-stream-log'` with the
`e2e:workflows/inngest:aimock` project): 7 passed, 1 expected fail
(delegated-agent chunks, #29035). With the Inngest change reverted, the
early tool result is stored twice as often as the last one (14 vs 7).
`durable-resume-iteration-2-serve.test.ts` now checks that a stream
resumed after a suspension in iteration 2 and a full replay report the
same response messages.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

Inngest was saving the full list of response messages after every step,
even when most messages had not changed. This change saves only the
changed messages, while callers and observers still receive the full
list.

## Changes

- Inngest step-finish events now carry response-message diffs instead of
the full list.
- Diff generation runs inside a memoized Inngest step. Replays reuse the
diff, and the cursor travels with iteration state.
- Each Inngest function run starts a new diff sequence, including when a
run resumes.
- Resumed streams and `observe()` rebuild response messages from earlier
events when reading from an offset.
- Pubsub wrappers forward topic history replay decisions. When a bus
does not keep history, steps send whole lists.
- `@mastra/core/agent/durable` now exports `diffResponseMessages` and
the `ResponseMessagesCursor` type.
- Tests check reduced duplication, replay routing, and that resumed
response messages match observer output.
- The changeset warns that older `@mastra/core` readers may not report
response messages from these events. Upgrade agent processes and readers
together.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 23:51:02 -03:00
Leo
42bc816c07
fix(core): pass omitted and null input to evented inputSchema validation (#29542)
## Description

Evented workflows rejected valid input. `start()` and `startAsync()`
replaced an omitted or `null` `inputData` with `{}` before validating it
against the workflow `inputSchema`, so a top-level schema default never
applied and a nullable schema rejected an explicit `null`. `stream()`
calls `start()`, so it had the same bug. Both start paths in
`packages/core/src/workflows/evented/workflow.ts` now pass `inputData`
through unchanged, which is what the default engine already does.

Follow-up fixes on the same input path (from review):
- The evented step executor stored a step's `payload` as `input ?? {}`.
Resume feeds that payload back into the step, so a first step started
with `null` or no input got `{}` on resume and a nullable or optional
schema rejected it. It now stores the input unchanged, as the default
engine does. Nested resume uses the stored payload whenever the step
record exists, instead of `payload ?? parentInput`.
- Time travel to the first step built that step's payload as `stored
payload ?? {}`, so a run started with `null` or no input replayed `{}`
on the evented engine and a nullable or optional schema rejected it. The
evented engine also ignored new `inputData` passed to `timeTravel()` for
the first step. `createTimeTravelExecutionParams` (shared by all
engines) now gives a first-entry target the same input the default
engine gives it: new `inputData` if passed (an explicit `null`
included), else the stored workflow input, kept as `null` or
`undefined`. A nested workflow in the first entry replays the stored
input too, and when the evented engine finds no child snapshot it seeds
the child's input from what the parent passes in. Other steps keep their
existing payload fallback.
- With `null` input, `context.input` is `null`. Two scans read `.status`
on it: the evented suspended-step list, and
`hydrateSerializedStepErrors` (shared with Inngest). A suspend crashed
`start()`, and a step failure left it waiting forever. Both now skip the
`null` entry.

```diff
 // evented start() and startAsync()
-const inputDataToUse = await this._validateInput(inputData ?? ({} as TInput));
+const inputDataToUse = await this._validateInput(inputData);
```

```ts
const input = z.object({ count: z.number() }).default({ count: 5 });
// evented engine, before: throws "Invalid input data: count: expected number, received undefined"
// evented engine, after:  success, the first step and getInitData() both see { count: 5 }
await run.start({});
```

### Evidence

New cases, added to the shared workflow suite
(`workflows/_test-utils/src/domains/schema-validation.ts`), run on the
default, evented and Inngest engines. They cover an omitted input with
an object default, an explicit value winning over the default, a scalar
default, an explicit `null` with a nullable schema, `getInitData()`
returning the defaulted input, omitted input with `validateInputs:
false` reaching the step as `undefined`, and missing required input
still being rejected without the step running. A focused evented file
(`start-input-validation.test.ts`) covers `startAsync()`, which the
shared harness doesn't expose, and asserts on the persisted run
snapshot. It also covers suspend and resume with `null` and omitted
input, a step failure with `null` input, and a nested step resumed with
its stored `null` input.
- **Review follow-ups, before** (commit 80b7fd6851, same tests): 4
failed. `Cannot read properties of null (reading 'status')`, `expected
{} to be undefined`, a 15s timeout on the failing step, and the nested
step resumed with `'seed'` instead of `null`. **After:** 9 passed in
that file.

- **Before** (origin/main evented `workflow.ts`, same tests): `vitest
run start-input-validation.test.ts evented-workflow.test.ts
workflow.test.ts -t 29424` gave **10 failed | 9 passed**. All the
failures were on the evented engine, for example `Invalid input data: -
: Invalid input: expected number, received object`.
  **After:** **19 passed**.
- **Time travel, before** (head 765d1f564f): the shared time-travel
cases for `null`, omitted, new `inputData` and nested `{}`/`null` replay
failed on the evented engine with `Step input validation failed`, or
`expected 'failed' to be 'success'`. **After:** all 9 pass on the
default engine, the default engine with storage v2, and the evented
engine.
- The Inngest harness had no `timeTravel` helper, so every shared
time-travel case skipped there. With the helper added, all 27 shared
time-travel cases pass on Inngest. Three of Inngest's older time-travel
tests (suspend and resume a time-travelled run, time travel a suspended
run, conditional chains) fail on this branch both with and without the
time-travel change; their diffs are about `resumePayload: undefined`
keys and a conditional sibling marked `skipped`, which this PR doesn't
touch.
- Inngest, whole Schema Validation domain: 13 passed, 1 expected failure
(see below).
- Regression check: every test in `src/workflows/evented/`, every core
test file that calls `timeTravel`, plus `workflow.test.ts`,
`storage-v2-suite.test.ts` and `utils.test.ts` passes (57 files, 1691
passed, 14 skipped). `pnpm --filter ./packages/core check` is clean.
`workflows/_test-utils` typecheck reports no errors in the touched
files; it has 14 existing errors elsewhere, the same on main. oxlint and
oxfmt are clean on the changed files.
- Built with a hill climb: the failing tests were locked first, and the
score went from 10 to 0 in one source step.

Inngest already passed input through unchanged, but the new shared cases
found two Inngest bugs:
- `getInitData()` returns `undefined` instead of the defaulted input
when `inputData` is omitted. Cause not yet investigated. Pinned with
`it.fails` (`schemaInitDataDefaultFails`).
- An explicit `null` input crashed the status poll in
`hydrateSerializedStepErrors`. Fixed by the shared `null` check above,
so that case now passes without a pin.

### Merge Danger

**Door:** two-way. Reverting the commit restores the old fallback.

**Blast radius:** callers of evented `start()`, `startAsync()` and
`stream()` that omit `inputData`. A workflow whose bare object schema
(no default) passed only because of the evented-only `{}` fallback now
fails validation, as it already does on the default engine. With no
`inputSchema` or with `validateInputs: false`, the first step now
receives `undefined` instead of `{}`. Evented step results now record a
`null` or `undefined` `payload` as is, instead of `{}`, as the default
engine does; code that reads `steps[id].payload` on the evented engine
can see those values. The changeset tells those callers to pass
`inputData: {}` or add a schema default. Scheduled runs are unchanged,
because the scheduler already sends `inputData ?? {}`. Nested workflows
don't go through `start()`. Time travel to the first step now replays a
stored `null` or `undefined` instead of `{}`, and on the evented engine
uses new `inputData` passed to `timeTravel()`, as the default engine
already did.

### Follow-ups (not in this PR)

- The Inngest `getInitData()` bug above needs its own issue.
- On every engine, time travel to a later step whose previous step
returned `null` or `undefined` still replays `{}`, and a falsy
`inputData` passed to `timeTravel()` for a step after the first one is
ignored (the `inputData &&` check in `createTimeTravelExecutionParams`).
Both predate this PR. For the first step, an explicit `null` `inputData`
now replaces the stored input.

## Related issue(s)

Closes #29424

## Type of change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable). Not applicable: the docs already say validation applies
schema defaults and `inputData` is optional.
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

## Council review

Rounds: 4

| Seat | Round 1 | Round 2 | Round 3 | Round 4 |
| --- | --- | --- | --- | --- |
| Claude Opus 5.5 | not run | 9/10 | 9/10 | 9/10 |
| GPT-6.1 Sol | 9.5/10 | 9.5/10 | 9/10 | 9/10 |
| Grok 4.7 | empty seat | empty seat | empty seat | empty seat |

Every seat scored at least 9 in each category (correctness, tests,
scope, code quality) in rounds 2 to 4. Findings applied: clear the
`startAsync` test's timeout timer, describe the no-schema and
`validateInputs: false` change in the changeset, and add a shared case
for that change. Findings not applied:
- Add an Inngest branch to the required-input rejection case. That case
follows the same `schemaValidationThrows` skip as the existing
invalid-input cases.
- Persist an explicit `null` in the evented initial snapshot. The final
persisted run already stores `context.input = null`, and that assertion
passed without a code change.

Time-travel follow-up review (GPT-6.1 Sol, Correctness / Tests / Scope /
Code quality): round 1 8/8/10/9 (added the Inngest `timeTravel` helper),
round 2 7/8/10/9 (fixed a nested time-travel regression it found), round
3 9/9/10/9 with no findings.

Created with Claude Opus 5.5 in T3 Code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5
Workflows now pass the input you provide—or leave it missing—through
validation instead of changing it to an empty object first. This lets
schemas apply defaults or accept `null`, and helps workflows keep the
same input when they resume or replay.

## Changes
- Evented `start()` and `startAsync()` pass input directly to schema
validation. `stream()` uses `start()`.
- Step payloads preserve `null` and `undefined`. Resume paths use stored
payloads, including for nested steps.
- Time travel to a workflow’s first step preserves its input. Supplied
`inputData` can replace the stored input. For a nested workflow without
a saved snapshot, replay uses the parent step’s successful output as the
child input.
- Null-safe status checks handle missing step results and context
entries. A step failure after a `null` start returns a failed result
instead of leaving `start()` waiting.
- Added evented and shared tests for defaults, nullable input, invalid
input, resume, time travel, `getInitData()`, and validation-disabled
behavior.
- Added a patch changeset that documents the behavior and compatibility
impact.

When no input schema is configured or validation is disabled, omitted
input reaches the first step as `undefined`. Callers that need `{}` can
pass it explicitly or add a schema default. The shared suite marks the
Inngest `getInitData()` default-input case as an expected failure.

**Test and check results:** not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 18:58:43 -05:00
dane-ai-mastra[bot]
23cc34a9c5
chore: version packages (alpha) (#29938)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases

> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.

## @mastra/factory@0.28.0-alpha.6


## @mastra/code-sdk@1.19.0-alpha.6


## @mastra/playground-ui@69.0.0-alpha.7


## @mastra/client-js@1.60.0-alpha.6


## @mastra/react@1.15.0-alpha.6


## @mastra/deployer-cloud@1.83.0-alpha.6


## @mastra/longmemeval@1.1.42-alpha.6


## @mastra/opencode@0.1.42-alpha.6


## create-factory@0.4.3-alpha.6


## mastracode@0.50.0-alpha.6


## @mastra/agent-builder@1.2.1-alpha.3


## mastra@1.39.0-alpha.6


## @mastra/core@1.83.0-alpha.6


## @mastra/deployer@1.83.0-alpha.6


## @mastra/editor@0.17.1-alpha.4


## @mastra/mcp@2.9.0-alpha.3


## @mastra/mcp-docs-server@1.3.13-alpha.9


## @mastra/memory@1.41.1-alpha.3


## @mastra/rag@2.7.3-alpha.2


## @mastra/schema-compat@1.3.18-alpha.0


## @mastra/server@1.83.0-alpha.6


## @mastra/elysia@0.1.23-alpha.6


## @mastra/express@1.5.27-alpha.6


## @mastra/fastify@1.5.27-alpha.6


## @mastra/hono@1.7.25-alpha.6


## @mastra/koa@1.7.27-alpha.6


## @mastra/nestjs@0.3.0-alpha.6


## @mastra/next@0.2.41-alpha.6


## @mastra/server-adapters-test-suite@0.3.0-alpha.3


## @mastra/tanstack-start@0.2.41-alpha.6


## @mastra/upstash@1.6.1-alpha.0


## @mastra/vectorize@1.1.5-alpha.0


## @mastra/voice-google-gemini-live@0.14.19-alpha.0


## @mastra/voice-openai-realtime@0.14.9-alpha.0


## @mastra/voice-xai-realtime@0.2.18-alpha.0


## @mastra/inngest@1.11.2-alpha.4


## @mastra/temporal@0.6.0-alpha.6


## create-mastra@1.39.0-alpha.6


## @internal/playground@1.39.0-alpha.6

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-10 15:51:29 -08:00
Leo
df4b70bd41
fix(core): deliver falsy resumeData through timeTravel (#29425)
## Description

`run.timeTravel()` treated `false`, `0`, `''` and `null` resume data as
missing. The suspended step saw `undefined` and suspended again, even
when the value matched its `resumeSchema`. `run.resume()` already
delivered these values. This PR replaces three truthiness checks in
`@mastra/core` with presence checks:

```diff
 // packages/core/src/workflows/utils.ts — validateStepResumeData
-  if (!resumeData) {
+  if (resumeData === undefined) {
     return { resumeData: undefined, validationError: undefined };

 // packages/core/src/workflows/utils.ts — createTimeTravelExecutionParams
 // (stopped collecting the suspended step's result when time travel starts at an earlier step)
-    if (currentExecPathLength > 0 && !resumeData) {
+    if (currentExecPathLength > 0 && resumeData === undefined) {

 // packages/core/src/workflows/evented/workflow-event-processor/index.ts
 // (the default engine already uses !== undefined here, handlers/step.ts)
-    if (timeTravelResumeData && !timeTravelResumeValidationError) {
+    if (timeTravelResumeData !== undefined && !timeTravelResumeValidationError) {
```

Omitted `resumeData` still leaves the step suspended. A falsy value that
fails the `resumeSchema` is still logged and not delivered. `null` now
reaches a step whose schema allows it, as it does with `run.resume()`.

### Evidence

New cases in the shared time-travel suite
(`workflows/_test-utils/src/domains/time-travel.ts`) start a run with a
fixed `runId`, wait for it to suspend, then time-travel that run with
`false`, `0`, `''` and `null`. Each value is tried at the suspended step
and from the step before it, plus controls for omitted and
schema-invalid values. They run on the default engine
(`workflow.test.ts`), the evented engine with real workers and
`EventEmitterPubSub` (`evented/evented-workflow.test.ts`), and
`storage-v2-suite.test.ts`, all on in-memory storage. A separate file,
`timetravel-nested-resume-data.test.ts`, covers steps inside a nested
workflow.

- **Before** (fix reverted): `vitest run workflow.test.ts
evented-workflow.test.ts storage-v2-suite.test.ts
timetravel-nested-resume-data.test.ts -t 29390` gives 28 failed, 9
passed, 5 expected fail. Every failure is `expected [ undefined ] to
deeply equal [ false | 0 | '' | null ]`: 24 flat cases plus 4 nested
ones on the default engine.
  **After:** 37 passed, 5 expected fail, 0 failed.
- Existing time-travel and workflow suites (`utils`,
`timetravel-divergence`, `timetravel-foreach`, both
`concurrent-time-travel` files, `default`, `workflow`,
`evented-workflow`, `storage-v2-suite`): 1177 passed, 5 expected fail, 0
failed. `tsc --noEmit` for core and a strict typecheck of the new test
file both pass, and so do `oxlint` and `oxfmt` on the changed files.
- Not covered: Inngest calls the same core
`createTimeTravelExecutionParams` and default step handler, so it gets
the fix, but its suite sets `timeTravel: false` and skips the shared
time-travel domain. Nested time travel on the evented engine never
delivers resume data to the inner step, truthy or falsy, before or after
this change. That's a separate, pre-existing bug, not yet investigated,
and the 5 evented nested cases are pinned with `it.fails`.
- Added after CodeRabbit review: `evented/step-executor.ts` used `if
(params.resumeData)` when it records `resumePayload` and `resumedAt` on
the step result, so a falsy resume value was left off the stored result.
It now checks `!== undefined`. New `step-executor.test.ts` cases for
`false`, `0`, `''` and `null` fail 4/4 before and pass after; the whole
`src/workflows/evented` directory passes (43 files, 695 tests). This
doesn't add crash recovery for a time-travelled step: time travel still
writes no running record before the step runs, which is a separate,
pre-existing gap.

### Merge Danger

**Door:** two-way. Reverting the commit restores the old checks, and
nothing persisted changes shape.

**Blast radius:** `run.timeTravel()` with `resumeData` on the default,
evented and Inngest engines. A caller who passed `resumeData: null` to
mean "no data" now has `null` reach a step whose schema accepts it.

## Related issue(s)

Closes #29390

## Type of change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [x] Test update

## Checklist

- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable). No docs change is needed: the `timeTravel` reference
already says it accepts resume data without restricting it to truthy
values.
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

## Council review

Scores out of 10: Correctness / Tests / Scope / Code quality.

| Round | GPT-6.1 Sol | Claude Opus 5.5 | Grok 4.7 |
|---|---|---|---|
| 1 | 9 / 9 / 10 / 7 | not seated yet | empty seat |
| 2 | 10 / 9 / 10 / 10 | not seated yet | empty seat |
| 3 | 10 / 9 / 10 / 10 | 9 / 8 / 10 / 9 | empty seat |
| 4 | 10 / 8 / 10 / 9 | 9 / 9 / 10 / 9 | empty seat |
| 5 | 10 / 9 / 10 / 10 | 9 / 9 / 10 / 9 | empty seat |

Created with Claude Opus 5.5 in T3 Code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5
`run.timeTravel()` treated some valid answers as if the user had not
answered. This PR lets suspended steps receive valid values such as
`false`, `0`, an empty string, and `null`.

## What changed
- Updated time-travel validation and graph traversal to treat only
`undefined` as missing resume data.
- Updated the evented step executor to record defined resume data in
`resumePayload` and set `resumedAt`.
- Added regression tests for falsy resume values from the suspended step
and a preceding step.
- Added a patch changeset for `@mastra/core`.

## Known limitation
Evented workflows still do not deliver time-travel resume data to steps
inside nested workflows. Tests for this behavior remain marked as
expected failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 18:21:04 -05:00
Leo
6ef07410a4
fix(core): give stopWhen AI SDK tool-result parts on durable engines (#29426)
## Summary

On durable, evented and Inngest agents, a custom `stopWhen` received
each step's `toolResults` as Mastra tool-call payloads (`{ toolCallId,
toolName, args, result }`, no `type`). `stopWhen` is typed as an AI SDK
stop condition, so a predicate that read `toolResults[i].output` or
checked `type === 'tool-result'` never matched and the run kept going.
Those engines now pass the same AI SDK tool-result parts the default
engine does.

The new `durableStopConditionSteps` helper in `@internal/core/durable`
builds `toolResults`, `staticToolResults` and `dynamicToolResults` from
each step's tool-result content parts, after restoring any outputs that
compaction dropped. Only the view handed to `stopWhen` changes. Stored
step records keep their payload-shaped `toolResults`, because snapshot
compaction reads them.

```diff
 // durable-loop-builder.ts (durable + evented) and create-inngest-agentic-workflow.ts (Inngest)
-steps: expandDurableSteps(state.accumulatedSteps)
+steps: durableStopConditionSteps(state.accumulatedSteps)
   // expandDurableSteps(steps), then per step with content:
   //   toolResults        = content.filter(part => part.type === 'tool-result')
   //   staticToolResults  = toolResults.filter(part => part.dynamic === false)
   //   dynamicToolResults = toolResults.filter(part => part.dynamic === true)
```

That mirrors how the default loop builds its steps
(`packages/core/src/loop/loop-builder.ts`). Step records from snapshots
written before steps carried `content` are returned as stored, as the
issue allows.

## Evidence

Test changes in `stopwhen-step-results.scenario.test.ts`, which runs on
every engine against a real HTTP mock LLM:

- The existing `it.fails` case for #28519 is now a plain `it`.
- New case: a predicate that matches on `toolResults[i].output` stops
the run after step 1.
- New case: `toolResults` equal each step's tool-result content parts,
including `input` and the static/dynamic views.
- New case: a throwing tool reaches the predicate as a tool-result part
whose output carries the error.
- New case: an output processor drops tool-result chunks from the stream
(durable records them as `resultHidden`), and the predicate still sees
those results.

There is also a unit test for the helper in
`packages/_internal-core/src/durable/index.test.ts`. It covers restored
outputs, unchanged stored records, records without content, and the
static/dynamic split.

- **Before:** with only the call site reverted to `expandDurableSteps`,
the core scenario run has `Tests 10 failed | 38 passed (48)`. All 5
cases above fail on `durable` and on `evented`. On Inngest, the scenario
run against a real Inngest dev server had `Tests 2 failed | 7 passed
(9)` before the Inngest call site changed.
- **After:**
- Core scenario (normal, durable, evented, fs): `Tests 48 passed (48)`
  - Inngest scenario (real dev server): `Tests  12 passed (12)`
  - Helper unit test: `Tests  4 passed (4)`
- **Guards, all on the pushed head:**
- Related stopWhen, maxSteps and onFinish scenarios plus all of
`packages/core/src/agent/durable`: `Tests 2117 passed | 18 skipped`
  - Inngest workflow and stopWhen serve tests: `27 passed`
- Inngest runs of the related stopWhen scenarios: `22 passed | 4
skipped`
- `tsc --noEmit` for `@mastra/core`, `@internal/core` and
`@mastra/inngest`, and `oxlint --type-aware` on the touched files: all
clean.

Not covered: a processor that holds a tool result and releases it in a
later step. The `stopWhen` view comes from step `content`, and releasing
a held result only clears `resultHidden` on the stored records, so it
does not change what the predicate sees.

## Merge Danger

**Door:** two-way. This changes only the object passed to user
`stopWhen` predicates. Stored snapshots and other step consumers
(`onFinish`, processors, `stream.steps`) are untouched.

**Blast radius:** custom `stopWhen` predicates on durable, evented and
Inngest agents. A predicate on those engines that read the old internal
fields (`args`, `result`) on `toolResults` will no longer find them.
Those fields were never part of the documented AI SDK shape, and the
default engine never had them.

Closes #28519

## Council review

| Round | Sol (gpt-6.1-sol) | Opus (claude-opus-5-5) | Grok 4.7 |
|---|---|---|---|
| 1 | Correctness 9, Tests 8, Scope 10, Quality 9 | not run | empty seat
|
| 2 | Correctness 10, Tests 9, Scope 10, Quality 10 | not run | empty
seat |
| 3 | Correctness 9, Tests 9, Scope 10, Quality 9 | Correctness 9, Tests
9, Scope 10, Quality 9 | empty seat |

Round 1 findings were applied: cross-engine cases for tool errors,
processor-hidden results, `input` and the static/dynamic views, and a
typed helper return. The round 2 nit with mixed `dynamic` flags was
applied. Not applied:

- **Delayed-release scenario:** it doesn't change the content-derived
view.
- **Mapping legacy no-content records to parts:** the issue allows
keeping their current shape.
- **AI SDK-style `dynamic !== true` static split:** it would break
parity with the default engine, so it belongs in a separate change to
both engines.

Created with Claude Opus 5.5 in T3 Code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

Some agents gave stop rules tool results in a format those rules could
not read. This change gives the rules the format they expect, so they
can use a tool’s output to decide when to stop.

## Changes

- Added `durableStopConditionSteps` to build `toolResults` from each
step’s content in the AI SDK format.
- The helper also builds `staticToolResults` and `dynamicToolResults`.
When content is present, it restores outputs removed by compaction. When
content is absent, it returns the stored step unchanged.
- Durable, evented, and Inngest engines now use the helper for
stop-condition callbacks. Stored step records remain unchanged.
- Updated tests to cover result shapes, output-based conditions, tool
errors, static and dynamic results, processor-hidden results, and legacy
records. Delayed release of a held tool result is not covered.

## Compatibility

Custom predicates that rely on the old `args` or `result` fields in
`toolResults` will no longer find those fields in the `stopWhen` view.
Use `input` and `output` instead.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 18:11:53 -05:00
Arthur Fiorette
01b2b958af
fix(core): fail the run when a loop callback throws, instead of retrying it or leaving it running (#29850) 2026-11-10 12:53:52 -08:00
dane-ai-mastra[bot]
779d2875b1
chore: version packages (alpha) (#29912)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases

> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.

## @mastra/auth-studio@1.4.0-alpha.1


## @mastra/factory@0.28.0-alpha.5


## @mastra/mcp-apps-components@0.1.0-alpha.5


## @mastra/playground-ui@69.0.0-alpha.6


## @mastra/client-js@1.60.0-alpha.5


## @mastra/react@1.15.0-alpha.5


## @mastra/deployer-cloud@1.83.0-alpha.5


## @mastra/longmemeval@1.1.42-alpha.5


## @mastra/opencode@0.1.42-alpha.5


## create-factory@0.4.3-alpha.5


## @mastra/code-sdk@1.19.0-alpha.5


## mastracode@0.50.0-alpha.5


## @mastra/agent-builder@1.2.1-alpha.2


## mastra@1.39.0-alpha.5


## @mastra/core@1.83.0-alpha.5


## @mastra/deployer@1.83.0-alpha.5


## @mastra/editor@0.17.1-alpha.3


## @mastra/mcp@2.9.0-alpha.2


## @mastra/mcp-docs-server@1.3.13-alpha.8


## @mastra/memory@1.41.1-alpha.2


## @mastra/rag@2.7.3-alpha.1


## @mastra/server@1.83.0-alpha.5


## @mastra/elysia@0.1.23-alpha.5


## @mastra/express@1.5.27-alpha.5


## @mastra/fastify@1.5.27-alpha.5


## @mastra/hono@1.7.25-alpha.5


## @mastra/koa@1.7.27-alpha.5


## @mastra/nestjs@0.3.0-alpha.5


## @mastra/next@0.2.41-alpha.5


## @mastra/server-adapters-test-suite@0.3.0-alpha.2


## @mastra/tanstack-start@0.2.41-alpha.5


## @mastra/libsql@1.30.0-alpha.2


## @mastra/turso@0.1.21-alpha.2


## @mastra/inngest@1.11.2-alpha.3


## @mastra/temporal@0.6.0-alpha.5


## create-mastra@1.39.0-alpha.5


## @internal/playground@1.39.0-alpha.5

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-10 10:50:08 -08:00
mastra-platform[bot]
e74225f4b7
fix(inngest): observe(runId).fullStream throws on Inngest agents (#29917)
## Summary
Agents created with `createInngestAgent` could not stream an
observation. The first read of `observe(runId).fullStream` threw
`TypeError: Cannot read private member #options from an object whose
class did not declare it`.

`observe()` replaces `fullStream` with a getter that detaches the
observer when the stream ends. That getter called the base `fullStream`
getter with `output` as the receiver. `output` is the destructurable
proxy, and a proxy can't reach the stream's private `#options` field.
The getter now types its receiver (`get(this: typeof output)`) and calls
`baseFullStream.call(this)`. `this` is the real stream instance, because
the proxy reads getters with its target as receiver. Core's
`DurableAgent.observe()` already does it this way.

Linear: COR-1525

## Changes
- `workflows/inngest/src/durable-agent/create-inngest-agent.ts`: one
getter hunk. The base `fullStream` getter is now called on the real
instance instead of the proxy.
- `workflows/inngest/src/__tests__/inngest-agent-observe.test.ts`: new
regression test. It runs a real Inngest dev server (v1.49.0), and AIMock
is the only mock, standing in for the model over HTTP. The test runs a
stream to completion, replays it through `observe(runId).fullStream`,
and checks the replayed text and that the last chunk is `finish`.
- `.changeset/inngest-observe-fullstream-receiver.md`: `@mastra/inngest`
patch.

No dependency, peer-range, checker, workflow, lifecycle or output-path
changes.

## Proof

**Author (session build, base `66c620c0`, all runs with retries off):**
- The new test fails on `main` with the `#options` TypeError. With the
fix it passed 4 of 4 runs. Reverting only the production hunk makes it
fail again with the same error.
- Existing core aimock `observe-*` scenarios on the Inngest engine:
  - `main`: 13 failed, 1 passed, 1 expected fail, 6 skipped.
- This fix: 14 passed, 1 expected fail, 6 skipped. I ran it 3 times and
got the same result each time.
- The expected fail is the existing Inngest `it.fails` for run-log TTL,
which has a different cause.
- `create-inngest-agent.test.ts`: 113/113 passed.
- `tsc --noEmit` (@mastra/inngest and the new test): passed. oxlint
`--type-aware` on the changed files: 0 warnings and 0 errors.

**Independent review (separate session, exact head `3bcdda3f`, runtime
freshly rebuilt):**
- New regression test, red → green → restore: it passes 1/1 on head and
fails with the exact `#options` TypeError when only the production hunk
is reverted. Restoring the hunk makes it pass again.
- Core aimock `observe-*`, `observer-outcomes`,
`parity-until-idle-replay` and `resume-stream` through Inngest: 27
passed, 1 expected fail, 8 skipped. On base, the observe, replay and
approval-resume scenarios had 13 failures (`observe-approval-resume` 1,
`observe-lifecycle` 8, `observe-reattach` 4). On head they had none.

## Unverified
- All tests ran the workspace source with a freshly built core `dist`.
None ran against packed npm artifacts.
- I haven't checked whether the HTTP `/observe` server route reaches
this getter.
- I make no claim about which earlier change introduced the bug.
- I make no claim about cancel or abort runtime behavior beyond what the
existing observe scenarios cover.
- The full `@mastra/inngest` e2e suite and the full aimock scenario
matrix were not run.

## Risk
Low. The change only affects which object the existing wrapper calls the
base getter on. Detach-on-error and the rest of the stream wrapping are
unchanged.

Unrelated: `create-inngest-agent.ts:2275` already fails Prettier on
`main`, outside this hunk.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## ELI5

The PR fixes replaying a finished run from an Inngest agent. It makes
the stream use the correct object when it opens, so reading the replay
no longer fails.

## Changes

- Updated `observe()` so the `fullStream` getter uses its actual
receiver when opening the stream.
- Added an integration regression test that runs an agent and replays
its completed stream. The test checks the reply text and final `finish`
chunk.
- Added a patch changeset for `@mastra/inngest`.

The PR description reports that the regression test and other listed
checks passed. Full Inngest end-to-end tests and the full aimock
scenario matrix were not run.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-10 09:05:04 -08:00
Arthur Fiorette
311d4342cc
fix(core): return processOutputResult text from durable agents (#29849) 2026-11-10 08:29:20 -08:00
Ward Peeters
da2b1a4b58
test(workflows): run the evented tripwire tests on a single copy of core (#29911)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-10 16:03:38 +01:00
dane-ai-mastra[bot]
7c4dd88400
chore: version packages (alpha) (#29889)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-10 15:48:17 +01:00
Ward Peeters
29eac207c0
test(workflows): run the agent tests in the shared workflow suite (#29904)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-10 14:07:53 +01:00
Ward Peeters
779cbcb7af
test(workflows): await agent callbacks instead of sleeping a fixed time (#29883)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-10 10:43:52 +01:00
dane-ai-mastra[bot]
6deb16047f
chore: version packages (alpha) (#29719)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases

> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.

## @mastra/slack@1.9.0-alpha.0


## @mastra/factory@0.28.0-alpha.3


## @mastra/code-sdk@1.19.0-alpha.3


## mastracode@0.50.0-alpha.3


## @mastra/observability@1.24.0-alpha.2


## @mastra/core@1.83.0-alpha.3


## @mastra/mcp-apps-components@0.1.0-alpha.3


## @mastra/playground-ui@69.0.0-alpha.4


## @mastra/server@1.83.0-alpha.3


## @mastra/server-adapters-test-suite@0.3.0-alpha.1


## @mastra/platform-workspace@1.13.0-alpha.0


## @mastra/auth-better-auth@1.1.11-alpha.1


## @mastra/client-js@1.60.0-alpha.3


## @mastra/react@1.15.0-alpha.3


## @mastra/deployer-cloud@1.83.0-alpha.3


## @mastra/longmemeval@1.1.42-alpha.3


## @mastra/opencode@0.1.42-alpha.3


## create-factory@0.4.3-alpha.3


## @mastra/arize@1.3.30-alpha.2


## @mastra/arthur@0.4.30-alpha.2


## @mastra/braintrust@1.3.27-alpha.2


## @mastra/datadog@1.4.22-alpha.2


## @mastra/deepeval@0.1.24-alpha.2


## @mastra/laminar@1.3.32-alpha.2


## @mastra/langfuse@1.6.6-alpha.2


## @mastra/langsmith@1.4.3-alpha.2


## @mastra/otel-bridge@1.5.22-alpha.2


## @mastra/otel-exporter@1.5.2-alpha.2


## @mastra/posthog@1.4.7-alpha.2


## @mastra/sentry@1.2.32-alpha.2


## @mastra/agent-builder@1.2.1-alpha.1


## mastra@1.39.0-alpha.3


## @mastra/connect@1.3.2-alpha.2


## @mastra/deployer@1.83.0-alpha.3


## @mastra/editor@0.17.1-alpha.2


## @mastra/fastembed@1.3.7-alpha.0


## @mastra/mcp-docs-server@1.3.13-alpha.6


## @mastra/memory@1.41.1-alpha.1


## @mastra/elysia@0.1.23-alpha.3


## @mastra/express@1.5.27-alpha.3


## @mastra/fastify@1.5.27-alpha.3


## @mastra/hono@1.7.25-alpha.3


## @mastra/koa@1.7.27-alpha.3


## @mastra/nestjs@0.3.0-alpha.3


## @mastra/next@0.2.41-alpha.3


## @mastra/tanstack-start@0.2.41-alpha.3


## @mastra/convex@1.8.1-alpha.0


## @mastra/pg@1.36.0-alpha.2


## @mastra/inngest@1.11.2-alpha.2


## @mastra/temporal@0.6.0-alpha.3


## create-mastra@1.39.0-alpha.3


## @internal/playground@1.39.0-alpha.3

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-10 10:33:29 +01:00
Arthur Fiorette
a836d09a43
fix(core, client-js, inngest): accept zod v3 schemas and newer ai v7 messages in public types (#29581)
## Description

zod v3 schemas work at runtime but some public types rejected them or
inferred `unknown`, and `UIMessage[]` from `ai@7.0.163` stopped
type-checking against core's vendored v7 types.

```ts
import { z } from 'zod/v3';
const schema = z.object({ answer: z.string() });

createStep(agent, { structuredOutput: { schema } }); // before: No overload matches this call
new Agent({ ...config, defaultOptions: { structuredOutput: { schema } } }); // before: TS2322
(await agent.generateLegacy('hi', { output: schema })).object; // before: unknown, now { answer: string }
await client.getAgent('a').generate('hi', { structuredOutput: { schema } }); // before: TS2590

import type { UIMessage } from 'ai'; // 7.0.163, has SteeringMessageUIPart
await agent.generate(uiMessages as UIMessage[]); // before: 'SteeringMessageUIPart' is not assignable
```

The zod v3 fixes point the user-facing option types at `PublicSchema` /
`PublicAgentExecutionOptions` (the runtime already converts with
`toStandardSchema`). `inferOutput` gets a zod v3 branch that reads
`_output` structurally, like `InferPublicSchema` in schema-compat, so a
v3 schema from a different zod install than core's is inferred too.
Client-js `StreamParamsBase` is built on `AgentExecutionOptionsBase`,
which avoids the TS2590.

For the v7 messages, `MessageInput` also accepts a v7 `UIMessage` whose
parts only need a string `type`. Core can't know the shape of part types
released after its vendored copy, so those parts are deliberately loose.
Inline object literals are still checked against the known part types,
because excess property checks rule out the loose member for them.
Mastra doesn't process the new part types yet.
`AIV6Adapter.fromUIMessage` paired parts with its v5 bridge output by
index, so a part the bridge drops (an unknown type, or an empty text
part) shifted every later part. It now drops those before pairing.

The PR also fixes two tests that failed CI on `main`: the stored board
runtime test still imported `factoryMemorySettingsUserId` after #29366
renamed it, and the redactor leak check matched `4111` inside a random
message UUID.

Type tests cover each case in the core and inngest type-test files, and
they fail with the fix reverted. The e2e client-js fixture imported
`zod/v3` and `zod/v4`, which don't resolve there, so its schemas were
`any`. This PR fixes those imports. The e2e type-check run still ignores
its exit code (`e2e-tests/type-check/test.ts:45`), which is left for a
separate PR.

## Related issue(s)

Fixes #29574
Fixes #29575

## Type of change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [x] I have linked the related issue(s) in the description above
- [ ] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have addressed all Coderabbit comments on this PR


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

This PR lets TypeScript accept Zod v3 schemas and newer AI SDK v7
messages in Mastra’s public APIs. It also fixes message-part matching so
ignored parts do not change how later parts are handled.

## Summary

- Updated agent and workflow option types to accept public schemas,
including Zod v3 schemas. Added Zod v3 output inference for
`generateLegacy`.
- Added forward-compatible input types for newer AI SDK v7 UI message
parts. Mastra accepts these messages, but does not yet process unknown
part types.
- Updated the AI v6 adapter to filter unsupported parts and empty text
parts before matching converted message parts.
- Updated client-js stream parameter types to avoid a complex TypeScript
union. Corrected Zod imports in a type-check fixture and added
regression type tests.
- Updated `getLastMessage` to use the last part only when it contains
nonempty string text.
- Added patch changesets for `@mastra/core`, `@mastra/client-js`, and
`@mastra/inngest`.

**Diff:** 19 files changed; 191 additions and 34 deletions.

**CI and test results:** not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-09 16:11:29 -03:00
NikAiyer
7358ad73a3
fix(core): stop durable streams waiting on frames the transport rejected (#29718)
## Summary

Two related problems in the durable agent stream surfaced while fixing
stale `@mastra/inngest` tests.

**A publish failure stalls a durable run for ten seconds.**
`CachingPubSub.publish` writes each frame to the replay log before it
publishes to the inner transport. When that publish rejects, the frame
stays in the log with its index — late subscribers can still replay it —
but the live subscription will never deliver it. `waitForLoggedEvents`
waited for that index until the pending-frame deadline
(`LOGGED_EVENT_DELIVERY_TIMEOUT_MS`, 10s in
`packages/core/src/agent/durable/stream-adapter.ts`), and `fail()` calls
`waitForEventDelivery()` before it delivers the error chunk. So a broken
transport delayed a run's error by the full watchdog: with a `publish`
that always throws, `stream`'s first chunk arrived at ~10.0s and
`generate` rejected at ~10.0s.

`CachingPubSub` now records the log indices whose live publish threw and
exposes them via `getUndeliveredIndices(topic)` (dropped in
`clearTopic`); `waitForLoggedEvents` skips those indices when computing
its wait target. The frame stays in the log for replay — only the wait
changes.

**Four `@mastra/inngest` tests assert durable-run behavior that has
since changed**, so `workflows/inngest` fails on `main` even though the
product is fine otherwise:

- **Publish-failure termination** (`create-inngest-agent.test.ts`,
stream + generate): previously raised the `Promise.race` budget to 15s
so the 10s watchdog could elapse. With the core fix the original 1s
budget bounds a genuine hang again, and the test fails if the stall
returns.
- **Resumed-chunk parity** (`resume() starts after cached history`,
`resumeGenerate() returns the resumed result`): durable delivery stamps
each chunk with its pubsub log index so late subscribers can order
frames, so exact chunk arrays are compared with the index stripped. The
two negative assertions had the same problem and could never have
failed; they now compare index-stripped chunks too.
- **Persisted request context** (`resume-runtime-rehydration.test.ts`):
the durable snapshot always records the run's scope set —
`mastra__scopes: []` included — so a cold resume can tell "held no
scopes" apart from a pre-upgrade snapshot
(`packages/core/src/agent/durable/preparation.ts`). The expectation
includes the marker.
- **Mocked suspended snapshot** (same file): the fixture predates the
resume guard that requires `status === 'suspended'`, so `resume()`
rejected it as "not suspended (no snapshot found)". The fixture now sets
a status.

The suite does not run in CI: the affected job runs only `--project
'unit:*' --project 'typecheck:*'` (`.github/workflows/test-suite.yml`),
and this package declares its suites as `e2e:workflows/inngest*`, so
only typechecking covers it. That is deliberate — those suites spawn
Inngest dev servers and are too slow for the affected-test job. Locally,
the package's plain `pnpm test` also aborts on a vitest project-config
conflict, so run the files directly.

## Test plan

```bash
pnpm exec vitest run --no-isolate --retry=1 --project e2e:workflows/inngest <file>
```

Run per file: this package's suites spawn dev servers on a shared port
and are unreliable as one serial invocation.

Branch rebased on `main` at `ed1cecba5c` (post-#29635):

- `src/__tests__/create-inngest-agent.test.ts` — 113 passed; the two
publish-failure tests now complete in ~15ms each instead of waiting out
the 10s deadline
- `src/resume-runtime-rehydration.test.ts`, `src/pubsub.test.ts` — 11
passed
- `packages/core`: `src/events` + `src/agent/durable/__tests__` — 324
files, 3356 passed / 4 skipped, no type errors
- `tsc -p workflows/inngest/tsconfig.json` clean; oxlint clean on the
changed files
- `pnpm turbo build --filter=@mastra/core --filter=@mastra/inngest` —
31/31

Before the core fix, the same two tests fail on the original 1s budget
and pass only when it is raised past the 10s watchdog.

Co-Authored-By: mastracode
<284800079+mastra-platform[bot]@users.noreply.github.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

When the event transport rejects a frame, a durable run no longer waits
up to ten seconds for a frame that was never delivered. The frame stays
in history for replay, and the tests now account for saved resume data
and changing log indices.

## Changes

- `CachingPubSub` records the indices of frames that were cached but
rejected by the live transport. `waitForLoggedEvents` skips those
indices while waiting for delivery.
- Added tests for rejected publishes, retained history, index tracking,
clearing tracked indices, and prompt durable-stream error handling.
- Resume comparisons ignore log indices. The rehydration test expects
`mastra__scopes: []` and a `suspended` workflow snapshot.

## Validation

- The supplied PR report says core event and durable-agent tests passed:
3,358 passed and 4 skipped.
- The report also says the Inngest tests, type checks, lint, and build
passed. The Inngest suites are not run by the affected-test CI job.
- Current-head CI results: not available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: review <review@local>
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-08 23:15:16 -08:00
mastra-platform[bot]
ed1cecba5c
refactor(core): share durable and Inngest run lifecycle setup (#29635)
## Summary

Extract reusable run-abort and cleanup helpers for durable agents, and
make Inngest reuse core's `AgentStreamBus` instead of maintaining its
own replay-cache/bus initialization.

This is the behavior-neutral shared-session lane (roadmap PR 1 /
COR-1509). It is independent of
[#29600](https://github.com/mastra-ai/mastra/pull/29600), which changes
server authorization organization. Neither branch includes the other.

- Replace four repeated DurableAgent abort-controller setups and two
Inngest setups with `createRunAbortController`, preserving
external-abort callbacks and remote-abort publication. The plain
publisher now uses the same forwarding helper and detaches it during
teardown, retaining its local no-reason abort default.
- Extract `createRunCleanup` for DurableAgent's stream, resume, recover,
generate and observe paths, retaining their different release policies,
ordering, timer behavior and retry-on-throw behavior.
- Resolve Inngest's bus through `AgentStreamBus`, preserving
`InngestPubSub`, its router, lazy cache resolution, already-cached
inner-bus reuse and fork bus identity.
- Replace the internal `super({ durable: false })` publisher-suppression
workaround with a protected publisher-ownership hook. DurableAgent still
does not start a plain publisher; its historically observable `durable
=== false` getter is preserved.

### Intentionally unchanged

Plain Agent continues to deliver directly to its original caller and
publish an observation copy. DurableAgent continues to publish and read
back. Public result shapes, event envelopes/topics, execution fencing
and path-specific cleanup policies remain distinct. Plain Agent retains
its existing publisher/abort/ownership policy rather than being forced
through durable-only lifecycle helpers.

No server authorization changes, topic-namespace redesign, output-path
convergence, durable-by-default execution, worker dispatch, checker
changes or propagated peer bumps. The separate streamed-delegation bug
fix shipped in [#29592](https://github.com/mastra-ai/mastra/pull/29592).

## Peer compatibility — unresolved review decision

Opening the exact reviewed candidate as requested; no additional
dependency or checker edits have been made for publication.

The candidate currently changes only Inngest's core peer range to
`>=1.83.0-0 <2.0.0-0`. Inngest imports the newly exported
`AgentStreamBus` and `createRunAbortController`, but published core
`1.83.0-alpha.0` satisfies that range and lacks those exports. The range
therefore does not yet establish compatibility with every version it
accepts.

- Stock import checking in the unprepared alpha checkout failed because
stable core `1.83.0` could not be downloaded at verification time.
- With authentic Changesets CI preparation, the unmodified checker
packed matching local core `1.83.0`: **18/18 import paths and 114 named
value imports passed**.
- Peer validation passed **128 valid / 0 invalid**; that is not proof
that published alpha.0 exposes the new APIs.

Decide the appropriate peer range during review. The suggested
stable-only range `>=1.83.0 <2.0.0-0` has **not** been applied. There is
no publish-first prerequisite or proposed checker workaround.

## Proof

Original reviewed candidate: `262c34adec`,
based on `f82c222119`. Twelve changed
files; no PR 2/server files or checker changes.

- Core TypeScript, changed-file repository formatting/type-aware lint,
peer validation and whitespace checks passed.
- Independent bounded core review: **24/24 test executions** across
standard and alternate-storage projects. Inspection covered abort
callback/reason ordering, cleanup retry/order, generate's lighter timer
release, publisher suppression and getter preservation.
- Strict retained before/after harness: **15/15 identical ordered frame
sequences** across plain, durable and evented engines for text,
tool-then-text, approval suspension, provider error and
abort-inside-tool. The comparison retains duplicate and
lifecycle/control frames. The scratch harness is not committed; this is
not generate, Inngest or remote-transport parity proof.
- Core suites reported **6,335 passing / 71 skipped**, with two parallel
worker processes terminating with SIGABRT in crash-recovery testing. The
crash-recovery file passed independently **20/20**. The broad parallel
run is not represented as clean.
- Earlier Inngest bus-integration-stage evidence: real dev-server abort
suites **9/9** and durable-agent suites **247/247**. Two create-agent
failures reproduced on main.

### Human-review follow-up — 541c618118

Two files changed: the plain publisher now reuses
`createRunAbortController(callerSignal)` and calls `detach()` on
teardown; the patch changeset explicitly documents `AgentStreamBus`,
`createRunAbortController`, and type `RunAbortController` with their
import path. Remote abort keeps its local `new Error('Aborted')`
default. No peer, checker, output-path, server, or cleanup-policy
change.

- Final focused stream/abort/cleanup suites: **66/66** across 11 test
files, including AIMock event parity and remote abort.
- Six disposable publisher-contract probes passed: live/already-aborted
caller signals with exact explicit/default reason identity, detachment
after teardown, and the local remote-abort default. Omitting
caller-signal forwarding made exactly **4 fail / 2 pass**; restoring it
returned **6/6**. Probe files are scratch evidence, not committed tests.
- Fresh `pnpm build:core`: **15/15 tasks passed**. Core `tsc --noEmit`
passed after rebuilding stale declarations. Changed-file type-aware
Oxlint, Oxfmt, Changesets status, whitespace checks, and the normal
commit hook passed.
- Broader cancellation/liveness/lease run: **300 passed / 2 failed / 24
skipped**. The lease case passed on isolated retry (12 matched cases
passed). The GC-child timeout reproduced with the publisher restored
byte-for-byte to original PR head `262c34ad`; this attributes it as
pre-existing to this two-file follow-up, **not** as proven main-baseline
behavior. The broader run remains non-clean.
- Fresh GitHub CI is pending. The disclosed Inngest peer-floor decision
remains unresolved and unchanged. No new hosted-service or two-process
verification was performed for this follow-up; earlier evidence below
retains its original provenance.

### Independent combined compatibility check

A disposable checkout applied only PR 2's three-file patch at
`71d868c384` to this exact PR 1 candidate.
Neither published branch was merged into the other, and the changed-file
sets have zero overlap.

Fresh core/server builds and both TypeScript checks passed. Executable
ESM/CJS source maps were checked against candidate sources, and fixture
imports resolved the freshly built distributions.

- Server handler tests: **281/281**.
- Real-TCP Hono access tests: **4/4**.
- Real Redis / two actual HTTP child processes: **8/8**, including
caching on/off, approval/finish observation, acknowledged remote abort,
unconfirmed abort false, tenant/anonymous/unknown/expired denials and
cleanup.
- Bounded Inngest cache/bus identity tests: **4/4**, plus the exported
`AgentStreamBus` check. This is wiring coverage, not a full Inngest
runtime run.

The subsequent PR 2 review additions are test-only and are not included
in that historical combined-run claim.

## Unverified

- Fresh GitHub CI and current-main integration after opening this PR.
- A compatible published minimum core version for all newly imported
APIs.
- A clean full-monorepo/full-parallel-core run; the parallel worker
SIGABRTs remain disclosed.
- Final-head full Inngest lifecycle/parity, generate parity, hosted
Redis, GCP Pub/Sub emulator, real-remote latency, browser
Stop/reconnect, load/failover and process-kill recovery.

## Risk

Lifecycle extraction can change abort-listener ordering, double
publication, cleanup timing or registry release even without intentional
behavior changes. Regression tests cover single control publication, the
absence of a second plain run record, one replay-cache wrapper, cache
identity, cleanup ordering and retry behavior. Explicit hooks preserve
path-specific policy rather than asserting identical output/result
contracts.

The unresolved peer range is a real package-compatibility issue and
remains a review/release gate, not a claim of runtime equivalence with
old core packages.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

This change gives durable and Inngest runs shared tools to handle
cancellation and cleanup. It also makes Inngest use core’s stream-bus
setup while keeping its own run behavior.

## Changes

- Added and exported helpers for forwarding abort signals and
coordinating run cleanup.
- Updated `DurableAgent` run paths to use the shared helpers. Added a
hook to prevent durable runs from using the plain Agent stream
publisher. Kept the `durable` getter set to `false`.
- Routed Inngest stream pubsub setup through `AgentStreamBus`. The setup
reuses an existing caching pubsub and resolves Mastra’s cache when
needed.
- Raised the Inngest package’s `@mastra/core` peer dependency minimum to
`1.83.0-0`.
- Added tests for abort forwarding, cleanup behavior, durable stream
publishing, and Inngest cache and bus setup.

## Validation and compatibility

The supplied PR notes report that focused stream, abort, and cleanup
suites passed 66/66 across 11 files. They also report 24/24 bounded core
test executions, 15/15 identical ordered frame sequences in a
before-and-after harness, six publisher-contract probes passed, and a
fresh `pnpm build:core` passed 15/15 tasks.

The broad core test run reportedly had 6,335 passing tests and 71
skipped tests, but two workers terminated with `SIGABRT`. Crash-recovery
reportedly passed 20/20 when run independently. A broader cancellation,
liveness, and lease run was not clean: 300 passed, 2 failed, and 24 were
skipped.

Fresh GitHub CI results and several runtime and recovery scenarios
remain unverified.

The PR notes identify a peer compatibility concern: published
`@mastra/core` version `1.83.0-alpha.0` lacks exports imported by
Inngest. The suggested stable-only peer range was not applied, so
compatibility with the published minimum remains unresolved.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-08 21:42:02 -08:00
dane-ai-mastra[bot]
009f632534 chore: version packages 2026-11-09 04:19:07 +00:00
dane-ai-mastra[bot]
f21e0742ce chore: version packages 2026-11-07 04:18:11 +00:00
Arthur Fiorette
aa79e280db
fix(inngest): claim suspended runs on resume and resume only the approved tool call (#29451) 2026-11-05 14:14:28 -08:00
Anurag Ojha
75983cdb05
fix(core,inngest): record durable agent span outcomes on any worker (#29261)
## Description

A durable agent run that finishes on a different worker from the one
that started it now records its real span outcome. Before, the facts
that decide the outcome lived only in the starting process's in-memory
run registry, so another worker ended `AGENT_RUN` and `MODEL_GENERATION`
with `unknown` or the wrong reason.

- Carry the stop cause, termination and delegation failure in the
durable loop's saved state, so the final step can read them on any
worker. Runs already in flight fall back to the registry, as they do
today.
- Keep `blocked / guardrail` and its `processorId` by passing the saved
step tripwire to the outcome classifier.
- Log a warning when a run reaches the final step with no recorded stop
cause.
- Inngest: end the caller's spans with the error when a run fails before
it reaches a worker. Nothing else ends them.

Real-run check with `gpt-4o-mini`, separate OS processes, main against
this PR:

| Case | main | This PR |
| --- | --- | --- |
| Observer failure on Inngest | `blocked / guardrail` in 6 of 9 runs |
`interrupted / observer_failed` in 9 of 9 |
| Delegation bail decided on one process, run ends on another |
`unknown` in 4 of 4 | Correct in 3 of 3 |

Stop conditions use the same field, but no real run crossed workers on
that path.

Unit tests run both final steps with the saved facts and no registry
entry, and they fail on `main`. The Inngest pool test stays as a smoke
test.

The rest of OBS-754 is in #29274 (span end that reaches storage first),
#29282 (resumed and recovered spans) and #29280 (Inngest deadlines and
workflow resume traces).

## Related Issue(s)


[OBS-754](https://linear.app/kepler-crm/issue/OBS-754/preserve-tracing-outcomes-and-resumed-span-identity-across-workers)
(part 1)

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

When a durable agent run finishes on a different worker, that worker now
has the saved facts it needs to describe how the run ended. This helps
the run’s tracing record show the real result instead of an unknown one.
If an Inngest start request fails before a worker receives it, the open
tracing records are also ended with an error.

## Summary

- Save stop causes, termination details, and delegation failures in
durable iteration state. Finalization reads those saved values and falls
back to the in-process run registry when needed.
- Classify final span outcomes from the saved state, including
completion, limits, guardrails, observer failures, and delegation
failures. Log a warning when a final step lacks a stop cause.
- End the caller’s agent and model spans with an error when the initial
Inngest trigger chain fails.
- Add tests for saved-state outcomes without a registry entry, state
propagation, and start-event dispatch failure. Add an Inngest
integration test for outcomes across workers.
- Add changesets for `@mastra/core` and `@mastra/inngest`. Remove
documentation describing the previous cross-worker tracing limitation.

**Diff:** 21 files changed; 679 lines added and 57 removed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-05 19:22:38 +05:30
mastra-platform[bot]
0c3212c66d
fix(temporal): return workflow input in run.start() step results (#29506)
Fixes #29502

## Problem
On Temporal workflows, `run.start()` and `run.resume()` stopped
returning the workflow input under `result.steps.input`. A local run
always returns it. Top-level `result.input` was not affected.

## Root cause
`run.ts` built `steps` with `normalizeStepResults(...)` on the success,
failed/tripwire and suspended paths. That function only converts the
worker's step records and never adds `input`. The worker's `steps`
object has no `input` key either, so `input` was missing for every
status.

A local run always puts the start input first in `steps` (`input: <start
input>`) for success, failed, tripwire, suspended and after resume. With
no start input it is `input: undefined`.

## Fix
- New helper `toStepResults(input, steps)` in
`workflows/temporal/src/run.ts` returns `{ input,
...normalizeStepResults(steps) }`. It is used on the success,
failed/tripwire and suspended paths.
- On success the start input is used. On resume no input is passed, so
it falls back to the worker's `output.input`.
- The suspended path uses `status.input`. `isSuspendedWorkflowResult`'s
type now includes `input?: unknown`.
- Patch changeset for `@mastra/temporal`.

## Tests
- `run.test.ts` has a test that runs the same workflows locally and
through the package's Temporal path, using the generated worker modules.
It now:
  - includes a successful workflow;
  - asserts that `steps.input` matches the local run's (`{ value: 5 }`);
  - no longer skips the `input` key.
- The mocked suspend/resume tests and lifecycle-hook tests asserted the
old shape without `input`, so they are updated to expect it.
- Without the fix, 6 of 32 tests in `run.test.ts` fail. With it, 32/32
pass in `run.test.ts` and 119/119 across the package.
- `tsc --noEmit` and `oxlint` (`pnpm lint`) are clean.

## Not verified
- Nothing was run against a real Temporal server. The tests use the
generated workflow and activity code with a mocked client.
- Suspend and resume are covered only by mocked tests.
- A run that fails after `resume()` has no `steps.input`, because the
worker's failure details carry no input. Top-level `input` is already
undefined there today.
- Canceled runs still return `steps: {}`.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

Temporal workflow results did not always include the original input
alongside their step results. This change adds it back so callers can
read `result.steps.input`, as they can with local runs.

## Summary

- Add `toStepResults` to normalize step records and include the workflow
input under `result.steps.input`.
- Include the input in successful, failed, tripwire, and suspended
results. For successful results, use `output.input` when the supplied
input is unavailable.
- Test local and Temporal results for success, failure, and tripwire
workflows. Add checks for suspended results and dispatch failures.
- Add a patch changeset for `@mastra/temporal`.

Failures after `resume()` may still lack `steps.input` when worker
failure details do not include the input. Canceled runs still return
`steps: {}`. The tests mock Temporal behavior and do not use a real
Temporal server.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastra-platform[bot] <mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: mastra-platform[bot] <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-04 21:40:54 -08:00
Arthur Fiorette
79d3099d44
fix(temporal): return step records and tripwire status from failed runs (#29169) 2026-11-04 20:19:51 -03:00
Tony Kovanen
f7320abd20
feat(core): continue runs in their stored storage version and get durable agents ready for storage v2 (#28708)
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
Co-authored-by: review <review@local>
2026-11-04 15:09:17 -08:00
dane-ai-mastra[bot]
675caf0613
chore: version packages (alpha) (#29336)
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.

⚠️⚠️⚠️⚠️⚠️⚠️

`main` is currently in **pre mode** so this branch has prereleases
rather than normal releases. If you want to exit prereleases, run
`changeset pre exit` on `main`.

⚠️⚠️⚠️⚠️⚠️⚠️

# Releases

> The changelog information of each package has been omitted from this
message, as the content exceeds the size limit.

## @mastra/client-js@1.60.0-alpha.0


## @mastra/react@1.15.0-alpha.0


## @mastra/factory@0.28.0-alpha.0


## @mastra/code-sdk@1.19.0-alpha.0


## mastra@1.39.0-alpha.0


## @mastra/core@1.83.0-alpha.0


## @mastra/mcp@2.9.0-alpha.0


## @mastra/mcp-apps-components@0.1.0-alpha.0


## @mastra/playground-ui@69.0.0-alpha.0


## @mastra/server@1.83.0-alpha.0


## @mastra/nestjs@0.3.0-alpha.0


## @mastra/clickhouse@1.30.0-alpha.0


## @mastra/duckdb@1.19.0-alpha.0


## @mastra/libsql@1.30.0-alpha.0


## @mastra/pg@1.36.0-alpha.0


## @mastra/temporal@0.6.0-alpha.0


## @mastra/auth-better-auth@1.1.11-alpha.0


## @mastra/deployer-cloud@1.83.0-alpha.0


## @mastra/deployer-cloudflare@1.2.42-alpha.0


## @mastra/deployer-netlify@1.2.42-alpha.0


## @mastra/deployer-sandbox@0.3.27-alpha.0


## @mastra/deployer-vercel@1.2.42-alpha.0


## @mastra/longmemeval@1.1.42-alpha.0


## @mastra/opencode@0.1.42-alpha.0


## create-factory@0.4.3-alpha.0


## mastracode@0.49.1-alpha.0


## @mastra/arize@1.3.30-alpha.0


## @mastra/arthur@0.4.30-alpha.0


## @mastra/braintrust@1.3.27-alpha.0


## @mastra/datadog@1.4.22-alpha.0


## @mastra/deepeval@0.1.24-alpha.0


## @mastra/laminar@1.3.32-alpha.0


## @mastra/langfuse@1.6.6-alpha.0


## @mastra/langsmith@1.4.3-alpha.0


## @mastra/observability@1.23.2-alpha.0


## @mastra/otel-bridge@1.5.22-alpha.0


## @mastra/otel-exporter@1.5.2-alpha.0


## @mastra/posthog@1.4.7-alpha.0


## @mastra/sentry@1.2.32-alpha.0


## @mastra/agent-builder@1.2.1-alpha.0


## @mastra/connect@1.3.2-alpha.0


## create-mastra@1.39.0-alpha.0


## @mastra/deployer@1.83.0-alpha.0


## @mastra/editor@0.17.1-alpha.0


## @mastra/mcp-docs-server@1.3.13-alpha.0


## @mastra/memory@1.41.1-alpha.0


## @mastra/rag@2.7.3-alpha.0


## @mastra/redis-streams@0.8.1-alpha.0


## @mastra/elysia@0.1.23-alpha.0


## @mastra/express@1.5.27-alpha.0


## @mastra/fastify@1.5.27-alpha.0


## @mastra/hono@1.7.25-alpha.0


## @mastra/koa@1.7.27-alpha.0


## @mastra/next@0.2.41-alpha.0


## @mastra/tanstack-start@0.2.41-alpha.0


## @mastra/spanner@1.11.2-alpha.0


## @mastra/turso@0.1.21-alpha.0


## @mastra/inngest@1.11.2-alpha.0


## @mastra/gcs@0.3.6-alpha.0


## @internal/playground@1.39.0-alpha.0

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-04 14:40:27 -08:00
Ehindero Israel
5d180ab8f1
fix(temporal): restore canceled workflow results (#29409)
PR #27020 accidentally removed the cancellation-specific result handling
while resolving its merge with lifecycle-hook changes. Temporal
cancellations were therefore returned as failed workflow results.

This restores canceled result classification for both `start()` and
background `startAsync()` settlement. Canceled runs invoke `onFinish`
with `status: canceled` and do not invoke `onError`.

Test plan: `pnpm test`, `pnpm typecheck`, `pnpm lint`, `pnpm build`,
formatting, and whitespace checks in `workflows/temporal`.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

Canceled Temporal workflows now show as canceled instead of failed. This
lets `onFinish` report the right status and keeps `onError` from
treating cancellation as an error.

## Changes

- `resolveResult` maps a `WorkflowFailedError` caused by
`CancelledFailure` to a canceled result for `start()` and background
`startAsync()` handling.
- Duplicate workflow-start errors do not trigger terminal hooks.
`start()` returns a failed result, while `startAsync()` rejects with the
original error.
- Workflow dispatch now passes `validateInputs` to Temporal.
- Added regression tests and patch changesets for cancellation handling,
duplicate starts, and input validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-04 17:52:15 +01:00
Ehindero Israel
374a7009f8
feat(temporal): support suspend and resume (#27020)
## Summary

Adds Temporal-backed suspend/resume support for Mastra workflows.

This lets Temporal-backed workflow steps call `suspend(...)`, wait
durably inside the Temporal workflow runtime, and continue when
`TemporalRun.resume(...)` or `resumeAsync(...)` sends resume data
through a Temporal signal.

Fixes #24024.

## What changed

- Adds `suspend(...)` to generated Temporal step activity contexts.
- Converts suspended step activity execution into an internal Temporal
suspension marker.
- Adds a `mastra.resume` signal handler to the generated Temporal
workflow runtime.
- Uses Temporal `condition(...)` to durably wait until the matching step
or label is resumed.
- Implements `TemporalRun.resume(...)`.
- Implements `TemporalRun.resumeAsync(...)`.
- Validates `resumeData` against the suspended step’s `resumeSchema`
where available.
- Preserves Mastra-facing step metadata after resume:
  - `suspendPayload`
  - `resumeData`
  - original step payload
  - final step output
- Updates generated workflow/activity fixtures and regression tests.
- Adds a changeset for `@mastra/temporal`.

## Notes

Temporal does not expose Mastra-style `suspended` as a separate platform
execution status for this flow. A suspended Mastra step is represented
as an open Temporal workflow waiting for the `mastra.resume` signal, so
Temporal UI may show the workflow as running/waiting until it is
resumed.

This PR does not implement full Studio live streaming parity, time
travel, or complete nested/foreach/parallel resume parity.

## Test plan

- `pnpm --filter @mastra/temporal test`
- `pnpm --filter @mastra/temporal typecheck`
- `pnpm --filter @mastra/temporal lint`
- `pnpm exec oxfmt --check workflows/temporal/src/run.ts
workflows/temporal/src/transforms/activities.test.ts
workflows/temporal/src/transforms/temporal-workflow-runtime.mjs`
- `pnpm --filter @mastra/temporal build`
- `git diff --check`

Local Temporal verification:

- Temporal server: `temporalio/auto-setup:1.22.0` (`temporal-server`
reported `1.22.0`); TypeScript SDK/worker: `1.22.0`; Temporal UI:
`2.17.0`.
- Started a workflow that suspended on `approval-step`.
- Confirmed Temporal UI/history showed the workflow open and waiting.
- Resumed it through `TemporalRun.resume(...)`.
- Confirmed `WorkflowExecutionSignaled` appeared before completion.
- Confirmed the resumed result included `suspendPayload` and
`resumeData`.
- Ran unsupported foreach suspension as workflow ID
`foreach-suspend-failure-1792067367063` (Temporal run ID
`c3a17543-56f9-461e-8edc-ddd098d4f23f`).
- Confirmed the execution closed as `Failed` with history length 11 and
exactly one `WorkflowExecutionFailed` event.
- Confirmed the failure was `ApplicationFailure` with `nonRetryable:
true`, message `Temporal workflows do not support suspending inside
foreach steps yet.`, and no repeated activity or workflow failure
attempts. History counts were one activity scheduled/started/completed
and one workflow execution failure; the two workflow-task cycles were
the normal activity scheduling and post-activity completion tasks.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

A workflow step can now pause and wait for someone to send it
information. Temporal keeps the workflow open while it waits, then runs
the step again with that information so the workflow can continue.

## Summary

- Generated step activities now provide `suspend(...)`. When a step
suspends, the activity returns a suspension marker instead of treating
suspension as a failure.
- The Temporal workflow runtime records suspended step state and waits
for a `mastra.resume` signal that matches the step ID or label. It
reruns the step with the supplied `resumeData`.
- A `mastra.status` query exposes the current workflow result.
`TemporalRun.start()` polls this query and can return a suspended result
while the Temporal workflow remains open.
- `TemporalRun.resume()` and `resumeAsync()` now signal the existing
workflow. Resume data is validated against the step’s resume schema when
available.
- Step results retain structured status, payload, output, timestamps,
and suspension or resume data where available.
- The change does not add full Studio live streaming, time travel, or
complete resume support for nested workflows, `foreach`, and parallel
paths.

## Verification

The PR description reports passing the Temporal package tests,
typecheck, lint, build, formatting check, and `git diff --check`. It
also reports a local Temporal verification in which a suspended workflow
resumed through `TemporalRun.resume()` and returned suspension and
resume data.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
2026-11-04 16:04:38 +01:00