0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
mastra/explorations/batching-spec.md
Marvin Frachet d90a2e5ece
Revamp agent-builder onboarding: ready entry, identity step, back navigation, add-to-library step (#17687)
## Summary

Reworks the agent-builder onboarding wizard shown right after an agent
is created from the starter. The flow now opens on a polished "Your
agent is ready" screen, lets users step backward through the wizard, and
adds a dedicated "Add to library" step. A consistent back-to-list
affordance is wired across the create/edit/view top bars.

New onboarding step order:

`ready > identity > model > tools > instructions > skills > browser >
library > integrations > end`

(`ready`, `identity`, and `library` appear only in the starter
onboarding entry flow.)

### What changed

- **Ready entry step** — A new first step, "Your agent is ready", with
two actions: **Review my agent** (advances into the wizard) and **Try my
agent** (jumps to the agent's view page). It stays centered while the
builder is still composing the agent and animates into the split review
layout once the mandatory fields are populated.
- **Identity step** — The former `initial` name/description step is
renamed to `identity` to make its purpose explicit now that `ready` is
the true entry screen.
- **Backward navigation** — The wizard gains a `prev()` capability
mirroring `next()`, surfaced as a Back button in the step container
(hidden on the first step, disabled while streaming).
- **Add-to-library step** — A new copy/UI-only `library` step that
explains what adding an agent to the library means, placed before
`integrations`. No client-tool wiring.
- **Back-to-list affordance** — The create/edit/view top bars now share
one consistent control that returns to the agent list
(`/agent-builder/agents`) with a shared `data-testid`.
- **Animated ready background** — The ready panel has a premium
light-sweep / specular effect. It is driven once via the Web Animations
API so layout/view-transition repaints can never replay it, and it
respects `prefers-reduced-motion`.

## Test plan

- `pnpm --filter ./packages/playground test` — wizard-context, ready
step, library step, step-container back button, top-bar back-to-list,
create, and `edit-onboarding.msw` suites all pass.
- Typecheck and lint clean for the changed playground files.
- Manual: from the starter, create an agent → first screen is the
centered "Your agent is ready" panel; once the builder finishes it
animates into the split review; the light sweep plays exactly once;
**Review** → identity (name/description), **Try** → `/view`; Back steps
backward through the wizard; the "Add to library" step shows explanatory
copy + Continue and appears before integrations; the top-bar back
affordance returns to the agent list on create/edit/view.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

When you create a new agent from the starter, you now see a friendly
"Your agent is ready" screen letting you either try the agent or review
and finish setup; the onboarding flow was reordered to add identity and
library steps and you can go back to previous steps. The ready screen
has a one-time light-sweep animation (respects reduced-motion) and the
library step explains publishing without changing agent visibility.

---

## Overview

This PR revamps the agent-builder onboarding shown after creating an
agent from the starter. It introduces a new entry step (ready), renames
the initial name/description step to identity, adds backward navigation
(prev), inserts a UI-only library step before integrations, and
standardizes a back-to-list affordance across create/edit/view pages.
Visual polish includes an ambient aurora and a one-time specular sweep
animation on the ready panel. Starter onboarding order is now: ready →
identity → model → tools → instructions → skills → browser → library →
integrations → end.

---

## Key Changes

- AgentProfileReadyStep
  - New centered "Your agent is ready" entry screen with CTAs:
- "Review my agent": advances to identity (wrapped in
startViewTransition).
    - "Try my agent": navigates to agent view (with viewTransition).
- One-time Web Animations API specular sweep on mount; skipped when
prefers-reduced-motion is set.
- Keeps chat centered while composing and animates into split review
layout once mandatory fields are populated and the builder is idle.

- AgentProfileIdentityStep
- Renamed from AgentProfileInitialStep; same UI/behavior, now part of
onboarding order.

- AgentProfileLibraryStep
- New UI-only step explaining library publishing; Continue advances to
integrations without performing stored-agent visibility PATCH.

- WizardProvider / wizard model
- WizardStep type updated: removed 'initial', added 'ready', 'identity',
'library'.
- STEP_ORDER begins with 'ready'/'identity' and places 'library' before
integrations; includeInitial is computed from initialStep === 'ready'.
- Added prev() navigation; prev no-ops on first step. Wizard context now
exposes prev alongside next, step, steps, isLast.

- AgentStepContainer
  - New props: panelClassName and panelOverlay for panel decoration.
- Renders Back button (hidden on first step, disabled while streaming)
that calls prev() inside startViewTransition.
  - CTA layout adjusted to include Back control appropriately.

- Back-to-list affordance
  - Breadcrumb CrumbProps accepts optional data-testid.
- Back link in EditTopBar, ViewTopBar, and AgentBuilderCreate includes
data-testid="agent-builder-back-to-list" and points to
/agent-builder/agents for consistent navigation/testing.

- Edit page & layout behavior
- Edit flow initializes at 'ready' for starter onboarding when
initialUserMessage is present.
- Debounced stream-running state (3s debounce) avoids flicker between
centered and split layouts while builder runs.
- Mobile CTAs re-keyed to identity step gating; ProfileSlot maps wizard
onboarding steps to the new Ready/Identity/Library components.
- AgentBuilderEditLayout simplified to always render a grid wrapper for
chat and optional footer.

- Visual/CSS
- index.css adds .ready-stage, .ready-stage::before (aurora),
.ready-stage-sweep (specular sweep), .ready-stage-content and
`@keyframes` ready-aurora-drift.
  - Reduced-motion rules extended to neutralize ready-stage animations.

---

## Testing

- New/updated tests and utilities:
- wizard-context.test.tsx: starts from 'ready', adds prev() coverage,
revises step sequences and isLast assertions.
- ready.test.tsx: verifies ready UI, CTAs, navigation, and sweep
animation behavior (once-only; reduced-motion).
- library.test.tsx: verifies library copy and that Continue advances
without PATCHing stored-agent visibility.
- agent-step-container.test.tsx: Back button visibility, navigation, and
disabled state while streaming.
- edit-top-bar.test.tsx & view-top-bar.test.tsx: assert back-to-list
data-testid and href.
- edit-onboarding.msw.test.tsx: refactored to control streaming via MSW;
asserts centered vs split layout, onboarding transitions, and mobile CTA
behavior.
- Added test fixtures and render utilities (builder settings,
stored-agent, auth/workspace/platform fixtures, registerStepHandlers,
renderStep, flush).
- Playground suites exercising changed areas reported passing locally;
typecheck and lint pass for modified playground files.

---

## Exports & Public API

- Barrel exports: removed AgentProfileInitialStep; added
AgentProfileReadyStep, AgentProfileIdentityStep,
AgentProfileLibraryStep.
- Type/interface changes:
- WizardStep updated (removed 'initial', added
'ready','identity','library').
  - WizardContextValue: added prev: () => void.
  - Breadcrumb CrumbProps: added optional data-testid?: string.
- AgentStepContainerProps: added panelClassName?: string and
panelOverlay?: ReactNode.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Mastra Code (anthropic/claude-opus-4-8) <noreply@mastra.ai>
2026-06-10 10:39:24 +02:00

16 KiB

PubSub Batching — Spec (shipped scope)

Status: Shipped.

This document describes what landed under the pubsub-batching branch. Earlier drafts of this spec described a larger design — signal-stack migration to pubsub, native batching across Redis / GCP / Unix-socket adapters, cache-backed batching in CachingPubSub. None of that shipped on this branch. See §4 ("Future adapters") and §7 ("What did not ship") for what was cut and why.

For the public API surface as of this PR, the canonical source is the changeset (.changeset/pubsub-batching-primitives.md) and the types in packages/core/src/events/.


1. Problem

Agent signals (introduced experimentally in f0cecbe13a, documented at docs/src/content/en/docs/agents/signals.mdx) deliver out-of-band context into a running agent. They enter via Agent.sendSignal(), get appended to the agent's message list at safe chunk boundaries (text-end, reasoning-end, tool-result, finish), and force the loop to continue with the new input.

In the current implementation, every signal interrupts the model at the very next safe boundary. This is fine when signals are rare. It breaks down when they aren't:

A long generation is running. A file watcher producer is firing <file-changed> signals 3–4 times per second during a build. The user, meanwhile, sends one <user-message> "wait, also handle the auth case." The model is mid-way through a 2000-token response.

Today the model is re-prompted roughly every 250ms. Its context is rewritten on each iteration, its plan destabilizes, and per-token cost balloons because each "iteration" is a fresh model call with a growing input. There is no minIntervalMs, no maxWaitMs, no coalesce — just "drain everything every time."

The agent loop currently owns both "is it structurally safe to drain?" (a real concern only it can answer) and "is it strategically wise to drain?" (a producer/consumer cadence question that has nothing to do with the loop). This spec separates the second concern out.

This branch ships the primitives. Migrating signal delivery onto them is future work.


2. Solution summary (as shipped)

Introduce opt-in, per-subscription batching as a first-class capability of the PubSub abstraction.

await pubsub.subscribe(topic, cb, {
  group: 'active-run',
  batch: { maxSize: 8, maxWaitMs: 1500, minIntervalMs: 500, isImmediate, coalesce },
});

Subscribers that omit batch see no behavior change. Subscribers that pass batch receive their callback invocations grouped according to the policy. The callback signature is unchanged — a batch of N events is delivered as N consecutive callback invocations in publish order.

Concretely, this branch adds:

  • SubscribeBatchOptions and SubscribeOptions.batch in packages/core/src/events/types.ts.
  • PubSub.supportsNativeBatching capability flag (default false).
  • An in-process implementation in packages/core/src/events/event-emitter/ — BatchPolicy (decision engine), AckHandleBuffer (queue + dispatch), and EventEmitterPubSub wiring them together. These three live under event-emitter/ because EventEmitterPubSub is currently their only consumer; they are not re-exported as a public abstraction.
  • CachingPubSub is transparent to batching: it forwards options.batch to the inner PubSub and mirrors supportsNativeBatching.

There is no BatchingPubSub wrapper class. There is no cache-backed batching layer. There is no signal-saving subscriber, no topic scheme, no agent-loop wiring change. Those are explicitly deferred (§7).


3. PubSub core API changes

3.1 Event and EventCallback are unchanged

type Event = {
  type: string;
  id: string;
  data: any;
  runId: string;
  createdAt: Date;
  index?: number;
  deliveryAttempt?: number;
};

type EventCallback = (event: Event, ack?: () => Promise<void>, nack?: () => Promise<void>) => void;

No payload-shape changes.

3.2 SubscribeBatchOptions

In packages/core/src/events/types.ts:

export interface SubscribeBatchOptions {
  maxSize?: number;
  maxWaitMs?: number;
  minIntervalMs?: number;
  isImmediate?: (event: Event) => boolean;
  coalesce?: (events: Event[]) => Event[];
  maxBufferSize?: number;
  overflow?: 'drop-oldest' | 'drop-newest' | 'coalesce-or-drop-oldest';
}

Field semantics (full JSDoc on the type itself):

  • maxSize — flush when the buffer holds this many events.
  • maxWaitMs — flush when the oldest event has been waiting this long. Timer starts on empty→non-empty.
  • minIntervalMs — minimum wall time between consecutive batch deliveries. Even if maxSize / maxWaitMs would fire, the buffer holds until this interval elapses since lastDeliveredAt.
  • isImmediate(event) — per-event escape hatch. Flush on publish, subject to minIntervalMs.
  • coalesce(events) — applied to the batch before delivery. Reference-identity contract: must return a subset of its input events by reference. Returning fresh Event objects (even with matching id) discards the whole batch as a contract violation and acks every original as dropped. Rationale: ack/nack handles live on the original Event reference; manufactured events have no transport handle. If you need merged payloads, build them in the subscriber callback after delivery. Ordering must be preserved.
  • maxBufferSize — overflow trigger; defaults to 256. Immediate events are never dropped on overflow.
  • overflow — 'coalesce-or-drop-oldest' (default), 'drop-oldest', 'drop-newest'.

SubscribeOptions.batch?: SubscribeBatchOptions is added; group is unchanged.

3.3 Per-event invocation, not per-batch

A batch of N events delivered together produces N consecutive cb(event, ack, nack) invocations, in order, on the same event-loop tick. Rationale:

  • Every existing subscriber works unchanged.
  • The signal-saving callback (when that lands) wants per-signal handling anyway.
  • ack/nack per event preserves redelivery semantics on adapters that support them.

The observable batching property — temporal grouping — is captured by callback timing, not signature.

3.4 PubSub.supportsNativeBatching

get supportsNativeBatching(): boolean {
  return false;
}

Capability advertisement. Defaults to false. Implementations override and return true once they integrate options.batch (either via AckHandleBuffer or via their broker's own retention). CachingPubSub forwards its inner's value (§4.2).


4. Adapter behavior

4.1 EventEmitterPubSub (shipped — native, in-process)

The only adapter that natively honors options.batch on this branch.

This adapter is strictly in-process — producers, subscribers, and the batch buffer all live in the same Node process. The only failure mode for an in-memory buffer is "the process crashes", which kills every other piece of state too. There is no durability gap to close.

Implementation, in packages/core/src/events/event-emitter/:

  • EventEmitterPubSub.supportsNativeBatching returns true.
  • On subscribe(topic, cb, { batch }): construct an AckHandleBuffer keyed by cb (per-topic map). The emitter listener pushes events into the buffer; the buffer's BatchPolicy decides when to flush.
  • Ack/nack are no-ops at this layer (a synchronous emitter has no redelivery semantics). The buffer still surfaces them as no-op callbacks so the subscriber API is uniform.
  • flush() drains every per-subscriber buffer, then waits for any nack-scheduled redeliveries, looping until both are empty.
  • close() disposes every buffer.

Internals (BatchPolicy, AckHandleBuffer) are intentionally not re-exported through packages/core/src/events/index.ts — they describe how this adapter implements its policy, not a stable cross-adapter abstraction. If a future adapter wants to reuse them, that's the moment to evaluate whether they belong at a higher level.

4.2 CachingPubSub (shipped — transparent passthrough)

CachingPubSub is transparent to batching:

  • subscribe(topic, cb, options) forwards options (including options.batch) to the inner PubSub.
  • supportsNativeBatching returns this.inner.supportsNativeBatching.

Consequence to be aware of: wrapping a non-native inner adapter (the common case — that's why CachingPubSub exists) with { batch: {...} } results in the batch options being passed to an adapter that ignores them. Delivery will be unbatched. This is documented on the CachingPubSub class JSDoc; there is no runtime warn (it would fire on every subscribe).

An earlier draft of this spec had CachingPubSub implement cache-backed batching directly, using cache cursors instead of an in-memory buffer. That code was prototyped and removed because the policy state (BatchPolicy.size, firstQueuedAt, lastDeliveredAt, the setTimeout handle) lives in-process. Two CachingPubSub replicas sharing a subscriberId would each run their own BatchPolicy, race on flushOnce calls, and corrupt the cursor. A correct distributed implementation needs policy state in the cache, a shared scheduler, and a lease/coordinator — which is a substantial design, not a primitive. Deferred.

4.3 Future adapters (not in this branch)

  • RedisStreamsPubSub — would honor options.batch by tuning XREADGROUP COUNT/BLOCK, holding ack handles in AckHandleBuffer, ack'ing per delivered event. Coalesced-out messages get XACKed without delivery. Crash recovery via XAUTOCLAIM (existing). Would set supportsNativeBatching = true.
  • GoogleCloudPubSub — would honor options.batch via subscriber-side flowControl.maxOutstandingMessages, delayed Message.ack(), and the existing MaxExtension window. Cap maxWaitMs ≤ MaxExtension × 0.8; warn on exceed. Would set supportsNativeBatching = true.
  • UnixSocketPubSub and other non-native transports — would need either (a) a native broker, (b) the deferred distributed-batching design on top of CachingPubSub, or (c) accept unbatched delivery.

None of this is implemented on this branch.


5. Edge cases

5.1 Ordering

EventEmitterPubSub synchronous emit preserves publish order per topic. coalesce must preserve ordering for events it keeps (enforced by reference-identity contract — see §3.2). nack redelivery uses setTimeout(0) and can interleave with concurrent publishes; treat nack ordering as best-effort within a topic.

5.2 Buffer overflow

maxBufferSize defaults to 256. With typical signal size ~1KB, this caps in-memory cost at ~256KB per active subscription. isImmediate events are never dropped on overflow. If coalesce is provided, it runs first; if still over budget, the configured overflow strategy applies.

5.3 Test determinism

BatchPolicy accepts BatchPolicyDeps injection (now, setTimeout, clearTimeout) so tests use a fake clock deterministically. AckHandleBuffer propagates the same deps. The BatchPolicyTimerHandle is a branded type so test fakes are type-checked at the boundary.

5.4 ack/nack interaction with batching

AckHandleBuffer stores (eventRef, ack, nack) triples and invokes cb(event, ack, nack) at flush time, so the original transport ack handles reach the user callback. For EventEmitterPubSub, ack/nack are module-level no-ops.

Partial-batch failure: per-event semantics. If a cb throws or rejects for one event, the error is logged via the buffer's onError hook; subsequent events in the same flush still run. policy.size decrements once per event regardless of cb outcome — the invariant delivered + dropped === snapshot.length holds.


6. Non-goals

  • Changing the signal storage role or XML wrapping.
  • Removing the structural boundary check from llm-execution-step.ts. The chunk-type whitelist stays — it's about safety, not cadence.
  • A subscribeBatch API delivering Event[]. Per-event callback shape preserved.
  • A BatchingPubSub wrapper class. Batching state lives in the adapter that owns the retention.
  • Cache-backed / distributed batching. Deferred (§4.2).
  • Signal-stack migration onto pubsub. Deferred (§7).

7. What did not ship (deferred)

Listed for honest tracking; each is its own follow-up.

  • Signal delivery on pubsub. Both signal stacks (regular agent via AgentThreadStreamRuntime, durable agent via run-registry) still use their existing in-process queues. Migrating them is the whole point of having options.batch at all, but it's the larger change.
  • Per-Agent signalBatching config. No public surface yet. Once the signal-saving subscriber exists, this is the next step.
  • Native batching in RedisStreamsPubSub and GoogleCloudPubSub. Sketched in §4.3; not implemented.
  • Distributed batching in CachingPubSub. Removed from this branch (§4.2). Requires policy state in cache + lease/coordinator.
  • Observability. No batching-specific debug log emitted yet. Errors surface via the adapter's logger; cadence is silent.

8. Open questions (still open)

  1. Default maxBufferSize? 256 is a guess. Needs to be informed by real burst patterns once a producer (e.g. MastraCode file watcher) actually exercises this.
  2. Per-type vs. single-policy coalesce. Should the framework eventually support per-event-type sub-policies, or stay with "write a coalesce function"? Decide before any user-facing API freezes.
  3. Where does signalBatching configuration live? Per-Agent vs. per-Mastra with override. Proposed: both, Agent overrides. Confirm when the signal-stack migration starts.
  4. Error message when batching is requested but unsupported. Currently silent passthrough through CachingPubSub (§4.2). Worth a runtime warn if this becomes a real footgun.

9. Rollout

PR1 shipped (this branch). It is purely additive — no consumer code paths change unless options.batch is explicitly passed. PR2 (signal-stack migration) and PR3 (native batching in Redis/GCP adapters) are not yet planned and will have their own specs when they are.


10. Worked example (in-process)

Pubsub: EventEmitterPubSub (default; in-process).

Subscription:

await pubsub.subscribe('agent-signals:R:T', cb, {
  batch: {
    maxSize: 8,
    maxWaitMs: 1500,
    minIntervalMs: 750,
    isImmediate: e => e.type === 'user-message',
    coalesce: events => dedupeFileChangedByPath(events),
  },
});

Timeline (ms):

t Event Buffer Notes
0 subscribe with above options — AckHandleBuffer created
100 publish file-changed: src/a.ts [1] firstQueuedAt=100; deadline at t=1600
250 publish file-changed: src/b.ts [2]
400 publish file-changed: src/a.ts [3] will be coalesced with the first
550 publish file-changed: src/c.ts [4]
700 publish user-message: … [5] isImmediate matches → flush now
700 batch delivered [] coalesce returns 4 events (one per path + user-message); cb invoked 4× in order; lastDeliveredAt=700
900 publish file-changed: src/d.ts [1] firstQueuedAt=900; minIntervalMs floor t=1450 < deadline t=2400 → effective t=2400
1100 publish file-changed: src/d.ts [2] will coalesce
2400 maxWaitMs elapsed — batch delivered: 1 event after coalesce; lastDeliveredAt=2400

Net: subscriber invoked five times instead of seven separate dispatches, with the user-message preempting correctly and duplicate path events coalesced. No cache, no extra processes — just a buffer inside the emitter.