0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
mastra/workflows
NikAiyer d929736435
fix(core): use one stream bus for durable agents (#30229)
Durable agents now use the wrapped Agent's stream bus instead of
constructing a second one. An Agent-level pubsub carries both run frames
and thread events, including after registration and forks; evented
execution still uses Mastra's shared transport. Standalone durable
agents remain observable, and plain-run records and abort checks stay
separate from durable execution.

Previously, observing across replicas required repeating the pubsub on
the durable wrapper. This now works with the Agent configuration alone:

```ts
const agent = createDurableAgent({
  agent: new Agent({ id: 'assistant', name: 'Assistant', instructions, model, pubsub }),
});
```

Inngest explicitly exposes its detached-streaming capability without
migrating its bus. The server retains the `isDurableAgentLike` fallback
in `hasResumableStreams` for compatibility with older Inngest wrappers:
their proxy can inherit `false` from a plain Agent or lack the method
entirely. Server and Inngest versions are not coupled, so removing the
fallback would silently turn HTTP disconnects into run cancellation on
supported mixed-version installs. This is a deliberate, user-approved
exception to the ticket's shortcut-removal bullet; no peer-dependency
bump is needed while the fallback remains. Regression coverage exercises
capability `true`, `false`, and missing; the latter two fail without the
fallback.

Server branch audit (`packages/server/src/server/handlers/agents.ts`):
`validateDurableToolCallAccess` (line 298) remains because approvals
need durable workflow snapshots; `authorizeRunAccess` (2676) remains
because durable ownership uses workflow rows/local and replay owners
rather than plain run records (COR-1542). The two abort-route branches
(2724 and 2742) retain durable cancellation support and its
missing/finished-run acknowledgement semantics, not separate transports.

The observe-route branch (3141) remains for generation-fenced
`__subscribeToRunStream` and the Inngest adapter's `pubsub` fallback;
consolidating observation is COR-1541, and migrating Inngest's bus is
COR-1539. The recover-route guard (3746) remains because only durable
execution has recoverable workflow snapshots. None of these remaining
branches requires a second stream bus on core DurableAgent.

The Mastra-fallback regression also runs through a fork created after
registration, both with and without resolving the bus before
registration. Another replica must receive replay plus live tail through
finish, and the terminal frame must be published on Mastra's transport.
The latest targeted rerun passed 33 tests, including cache-policy
warnings and plain-publisher isolation across both durable storage
variants; core typecheck, targeted lint, formatting and diff checks
passed.

Verified with 1,643 core durable/stream tests, 78 AIMock parity tests,
292 server handler tests (including the mixed-version capability cases),
119 Inngest tests, and six real Redis cross-server cases covering live
observation, thread events and HTTP Stop. Typechecks, lint and
formatting pass. The Agent-pubsub regression failed before the fix; a
built-package red/green demo also confirms the transport change. An
earlier approval-resume timing failure passed in isolation and on the
final full rerun.

Fixes COR-1540




## Transport precedence and upgrade scope

Default-engine durable run transport precedence is now: explicit
`createDurableAgent` pubsub → wrapped Agent pubsub → Mastra pubsub.
Evented runs retain Mastra's shared transport. Mastra-only
configurations are unchanged.

If the wrapped Agent already specifies a different transport from Mastra
and the wrapper has no override, this fix changes the effective
run-frame and remote Stop transport. To preserve routing across an
upgrade, explicitly pass the previous transport to `createDurableAgent`
or align the Agent and Mastra transports. The wrapper configures the
shared stream bus, so its override also affects resumable streams from
direct calls on the wrapped Agent. The core changeset includes this
upgrade note.

A local investigation used separately built pre-change main
(`049b31dd2c`) and this branch with real Redis in both owner/observer
version directions. Shared effective transport passed live completion,
replay, and remote Stop. Distinct selected transports lost live
completion and remote Stop acknowledgement; shared-cache replay still
worked. This is a configuration-dependent routing cutover, not a general
rolling-deploy incompatibility. The investigation-only second-checkout
harness and expected failures are not included in this PR; existing
same-version Redis coverage remains.

Permanent coverage now executes wrapper and fork runs with three
distinct wrapper/Agent/Mastra buses, verifies run frames only reach the
wrapper bus, and verifies replayed output. Dropping the wrapper override
makes this test fail on the intended publication assertion. Latest
focused verification: 38 tests passed; core typecheck, type-aware lint,
formatting, and diff checks passed.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

ELI5: Durable agents now send live updates through the same stream
channel as the normal agent they wrap. Before, they built a second
channel. Users can now set up cross-server viewing by configuring only
the wrapped agent.

## Summary

This PR changes durable agents to use the wrapped `Agent`'s stream bus.
They no longer build a second bus. Agent-level pubsub carries run frames
and thread events, including after registration and forks. Evented
execution continues to use Mastra's shared transport.

### Core changes (`packages/core`)

- `Agent` stores a fallback stream pubsub and a shared-run-log
capability callback.
- `__getStreamBus()` initializes a bus when wrapper stream configuration
exists or streams are enabled. It prefers wrapper configuration over
agent durable settings.
- `__getStreamBus()` uses the configured pubsub, or the agent pubsub. If
neither exists, it uses a reusable fallback `EventEmitterPubSub`.
- The shared-run-log check uses the wrapper callback when configured.
Otherwise, it calls `__usesSharedRunLog()`.
- `__ownsPlainStreamPublisher()` still selects the plain publisher. It
now gets its bus through `__getStreamBus()`.
- Plain publishing still requires enabled agent streams.
- `__configureDurableStreams(config, usesSharedRunLog)` overlays defined
wrapper values on the agent durable settings and any existing wrapper
settings. It stores the callback and resets the bus.
- Agent forks copy the durable stream configuration and share the
fallback pubsub.
- `DurableAgent` delegates stream-bus ownership and access to the
wrapped agent. Its cache, pubsub, cleanup scheduling, and replay-owner
operations use that bus.
- The `DurableAgent` constructor configures durable streams on the
wrapped agent. `__getStreamBus()` returns that bus.
- `DurableAgent` no longer copies a private bus configuration when
forked. It no longer adopts the inner pubsub. Mastra registration no
longer registers a separate bus.
- `DurableAgent` adds `__setLogger(logger: IMastraLogger)`. It forwards
logger updates to the wrapped agent.
- The `AgentStreamBus` constructor accepts an optional `defaultPubsub`.
The bus uses it when `config.pubsub` is absent, before the new
`EventEmitterPubSub` fallback.
- `AgentStreamBus.forkConfig` and `adoptInnerPubsub` are removed.
- Direct `stream` and `generate` calls on an agent with `durable: false`
stay unpublished. This holds after durable wrapping configures the
shared bus. The wrapper can still publish through that bus.
- Plain-run records and abort checks stay separate from durable
execution.

### Inngest changes (`workflows/inngest`)

- `InngestAgent` adds the public method `hasResumableStreams():
boolean`. It always returns `true`.
- The internal `Pick` type includes this method.
- Inngest exposes detached streaming. It does not migrate its bus.
- A new test checks that `createInngestAgent` reports resumable streams
when its wrapped `Agent` has `durable: false`.

### Server changes (`packages/server`)

- `hasResumableStreams` keeps the `isDurableAgentLike` fallback for
older Inngest wrappers in mixed-version installs.
- The observe-stream handler keeps the same fallback.
- The fallback applies when a wrapper reports `false` or does not report
the capability.
- Removing the fallback would cause the reported capability-`false` and
missing-method regression cases to fail.
- This PR does not require a peer-dependency bump while the fallback
remains.
- Other server branches handle durable tool-call access, run
authorization, abort, observation, and recovery. The PR states these
address separate durable-workflow behavior or deferred work. They do not
need a second stream bus on core `DurableAgent`.
- Comments in `hasResumableStreams` and the observe-stream handler were
revised. No executable behavior changed.
- The durable-wrapper stream-detachment test runs with
`hasResumableStreams` set to `true`, `false`, and absent. The stream
observer detaches in each case.

### Redis pubsub (`pubsub/redis-streams`)

- The cross-server abort test runs for the durable and evented engines
with the Mastra bus. It runs for the durable engine with the agent bus.
- Each child server receives the selected bus location.
- The successful-abort case subscribes the other server to the run's
thread. That server verifies a tool-call chunk. It then verifies
completion and a finish chunk after the abort.
- The remote abort response remains `{ aborted: true }`. The owner's run
ends aborted.
- The abort fixture adds `BUS_LOCATION=agent`. With this value, the
agent uses Redis Streams pubsub. The Mastra-level bus uses
`EventEmitterPubSub`.
- Without `BUS_LOCATION=agent`, the agent has no explicit pubsub. With
`ENGINE=evented`, the Mastra-level bus uses `CachingPubSub` over Redis
Streams. Otherwise, it uses Redis Streams directly.
- The `stream` command accepts optional `threadId` and `resourceId`. It
passes them as memory configuration when `threadId` is present.
- The fixture adds `observe` and `subscribe` commands. `observe` streams
run observations. `subscribe` streams thread subscription chunks.

### Tests

- The shared-bus tests add a standalone mode. They cover replay and
live-tail observation with and without forks. They also cover bus
resolution before registration.
- The tests verify finish-event publication.
- Replayed start and tool-call frames are asserted only for registered
replicas.
- A prior logger-warning assertion was removed.
- Thread-coordination tests abort across processes in every
`sharedFrames` case. They observe the aborted run's frames. They expect
an aborted finish reason and one model request.
- New tests check run-frame routing for plain, durable, and evented
agents. They cover configurations with both agent-level and Mastra
buses. They also check replay from an unregistered durable agent whose
wrapped agent has `durable: false`.
- The explicit durable-execution test expects the original agent and its
durable wrapper to share the same stream bus.
- A direct-execution test checks that `stream` and `generate` runs stay
unpublished when a `durable: false` agent is wrapped and registered.
Runs through the durable wrapper publish a finish event.

### Changesets

- Patch changesets cover `@mastra/core`, `@mastra/server`, and
`@mastra/inngest`.
- The `@mastra/core` changeset includes an upgrade note. Mastra-only
configurations keep their behavior. Users who need the previous
transport can keep it by configuring it explicitly.

### Verification

The PR description reports these results. This summary does not
independently verify them.

- 1,643 core durable and stream tests passed.
- 78 AIMock parity tests passed.
- 292 server handler tests passed.
- 119 Inngest tests passed.
- Six real Redis cross-server cases passed.
- A latest targeted rerun passed 33 tests. It covered cache-policy
warnings and plain-publisher isolation across both durable storage
variants.
- Typechecks, targeted lint, formatting, and diff checks passed.
- The Agent-pubsub regression test failed before the fix.
- An earlier approval-resume timing test failed. It passed in isolation
and on the final full rerun.

### Diff size

- 14 files changed.
- 341 lines added.
- 118 lines removed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: review <review@local>
Co-authored-by: mastracode <284800079+mastra-platform[bot]@users.noreply.github.com>
History 2026-11-16 07:21:54 -08:00
..
_test-utils fix(core): reset retryCount on each evented loop iteration (#30383) 2026-11-14 01:01:42 -03:00
inngest fix(core): use one stream bus for durable agents (#30229) 2026-11-16 07:21:54 -08:00
temporal chore: version packages (alpha) (#30430) 2026-11-16 14:09:31 +01:00
oxlint.config.ts chore: migrate linting to oxlint (#20173) 2026-07-25 01:04:02 +02:00
README.md fix(docs): Fix Inngest example (#5500) 2025-07-01 09:43:13 -07:00

Inngest Workflow Engine

Run Mastra workflows with Inngest.

Workflow and step execution functionality is exposed via serve from @mastra/inngest. This is a wrapper that detects all Mastra workflows created to run with Inngest and registers them to the Inngest server.

Steps and workflows are created using the createStep and createWorkflow functions initialized with init() from @mastra/inngest. This makes sure that all the steps are ready and able to connect with the Inngets runtime, publish events for watching, etc.

The Inngest runtime is horizontally scalable, with watching and eventing working through the @inngest/realtime package.

Usage

Running inngest dev server

docker run --rm -p 8288:8288 \
  inngest/inngest \
  inngest dev -u http://host.docker.internal:4111/inngest/api

Example

import { init, serve as inngestServe } from '@mastra/inngest';
import { PinoLogger } from '@mastra/loggers';
import { Inngest } from 'inngest';
import { z } from 'zod';

const inngest = new Inngest({
  id: 'mastra',
  baseUrl: `http://localhost:8288`, // if using local dev server
});

const { createWorkflow, createStep } = init(inngest);

const incrementStep = createStep({
  id: 'increment',
  inputSchema: z.object({
    value: z.number(),
  }),
  outputSchema: z.object({
    value: z.number(),
  }),
  execute: async ({ inputData }) => {
    return { value: inputData.value + 1 };
  },
});

const sideEffectStep = createStep({
  id: 'side-effect',
  inputSchema: z.object({
    value: z.number(),
  }),
  outputSchema: z.object({
    value: z.number(),
  }),
  execute: async ({ inputData }) => {
    console.log('log', inputData.value);
    return { value: inputData.value };
  },
});

const finalStep = createStep({
  id: 'final',
  inputSchema: z.object({
    value: z.number(),
  }),
  outputSchema: z.object({
    value: z.number(),
  }),
  execute: async ({ inputData }) => {
    return { value: inputData.value };
  },
});

const incrementWorkflow = createWorkflow({
  id: 'increment-workflow',
  inputSchema: z.object({
    value: z.number(),
  }),
  outputSchema: z.object({
    value: z.number(),
  }),
})
  .dountil(
    createWorkflow({
      id: 'increment-subworkflow',
      inputSchema: z.object({
        value: z.number(),
      }),
      outputSchema: z.object({
        value: z.number(),
      }),
      steps: [incrementStep, sideEffectStep],
    })
      .then(incrementStep)
      .then(sideEffectStep)
      .commit(),
    async ({ inputData }) => inputData.value >= 10,
  )
  .then(finalStep)
  .commit();

export const mastra = new Mastra({
  vnext_workflows: {
    incrementWorkflow,
  },
  server: {
    host: '0.0.0.0',
    apiRoutes: [
      {
        path: '/inngest/api', // this needs to match the path in inngest dev server (or production) config
        method: 'ALL',
        createHandler: async ({ mastra }) => inngestServe({ mastra, inngest }),
      },
    ],
  },
  logger: new PinoLogger({
    name: 'Mastra',
    level: 'info',
  }),
});