0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
Commit graph

3 commits

Author SHA1 Message Date
Michael 'Misha' Nightingale
84143a892e
feat(livekit)!: prepare 0.5.0 with recording and speech metrics (#24674)
Prepare the LiveKit 0.5.0 release with opt-in call recording and Studio
playback. `dispatchVoiceSession()` and `liveKitConnectionRoute()` can
create a fresh room with LiveKit auto egress before dispatching the
agent. The recording route resolves an authorized trace to a short-lived
playback URL, and Studio exposes it through **Review Audio**.

```ts
await dispatchVoiceSession({
  roomName: `call-${crypto.randomUUID()}`,
  metadata: { agentId: 'support' },
  recording, // Server-side LiveKit output settings.
});
```

Recording review requires an explicit `authorize({ traceId, context })`
callback. Only `true` permits trace lookup and playback URL resolution;
missing policies fail at registration. Recording room conflicts return
HTTP 409. The example and documentation cover S3 configuration, scoped
permissions, consent, and recording lifecycle implications. Browser
recording creation and playback in the unauthenticated example require
an explicit local-demo opt-in and are disabled in production.

The browser-safe `@mastra/livekit/client` entry point provides
`getLiveKitRecording(client, traceId)`, preserving the client's
authentication and request configuration. Studio uses the supported
React hook subpaths from main.

The integration also adds common per-turn metrics, speech-flush
boundaries, and an `onSpeechComplete` hook. `onTurnComplete` retains its
generation-only behavior. Custom reply-generator consumers must handle
flush boundaries, and `MastraLLM` integrations should use
`mastraLLMNode`. Playback completion describes server delivery, not
proof that the listener heard the audio.

The VAD opt-out is fixed. LiveKit Agents 1.7.1 is the supported minimum
and migration target, with matching plugin peer ranges and upgrade
instructions. The planned stable release remains **0.5.0**.

### Validation

- All 280 LiveKit tests across 22 files passed.
- All 18 Studio voice and recording-review tests passed.
- A local synthetic probe passed 96 example route-policy cases covering
room creation, dispatch, unrecorded tokens, storage access, and denied
playback.
- LiveKit, React SDK, shared Studio UI, and production Studio builds
passed.
- LiveKit, Studio, and voice-agent example typechecks passed; focused
lint and documentation checks passed.
- Production bundle inspection confirmed that the recording client adds
no LiveKit server or worker modules to the browser bundle.
- Changeset planning retains the 0.5.0 release target.

The core working-memory configuration fix and its regression tests are
in #26090 for separate review.
2026-10-06 14:38:25 -07:00
Marvin Frachet
f701530e7a
docs(react): add narrow API responsibility rule (#19336)
Adds a React best-practices rule for components, hooks, functions, and
utilities with oversized input or output APIs. It clarifies that
wrapping the same values in one object does not reduce responsibility
and shows how to split broad hooks into focused units composed at the
component level.\n\nThe rule catalog and quick reference now include the
new guidance and updated rule counts. Documentation structure, links,
counts, and fenced examples were validated.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5

This PR adds a new React “best practice” rule: if a
component/hook/function is taking in (or returning) too much at once,
you should split it into smaller, clearer pieces and then combine them
where it’s used. It also updates the rule catalog and adds a bit of
formatting/coverage cleanup in the voice-agent example tests.

## Changes

- Added a new React best-practices rule: `structure-narrow-apis` (“Keep
Input and Output APIs Narrow”) for components, hooks, functions, and
utilities.
- Clarified that wrapping unrelated things into a single
parameter/return object doesn’t truly narrow responsibility.
- Included incorrect vs. correct examples showing how to split an
oversized “mega-hook” into focused hooks and compose them in a
container/component.
- Updated the React best-practices rule catalog/quick reference (version
and rule/category counts), adding the new rule entry and revising totals
from 25 to 26 rules.
- Reformatted and lightly adjusted voice-agent example smoke coverage
and related example code without changing core behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-13 10:50:53 +02:00
Michael 'Misha' Litchev
25c1a4f6c0
feat(livekit): grouped worker configuration + MastraLLM plugin + voice-agent example (#19136)
## Description

This is the final PR of the stack split from #19019: memory
summarization (#19135, merged) → **livekit worker configuration +
MastraLLM plugin + voice-agent example (this PR)**. The example PR
(#19137) was consolidated into this one after review feedback landed, so
the package changes and the runnable example that exercises them ship
and get reviewed together.

> **Note for reviewers:** Tyler's split request listed the worker
`configuration` surface; the `MastraLLM` plugin work landed on the
original branch after that comment. Both are `@mastra/livekit` package
code, so they ride together here — happy to split the plugin into its
own 4th PR if you'd rather review it separately.

### `@mastra/livekit` — new grouped `configuration` worker option

One home for conversation & compliance policy on `createLiveKitWorker`,
so each control doesn't become another top-level option:

- **`configuration.greeting` — AI disclosure controls.** `text` spoken
at call start; `allowInterruptions: false` so a legally-required AI
disclosure plays through and can't be barged over (EU AI Act Art. 50);
`awaitPlayout: true` holds post-greeting work (persistence,
`onSessionStart`) until it finishes; `persist` saves it to the memory
thread; `repeatEvery`/`repeatText` re-disclose periodically on long
calls, spoken at the next turn boundary, never mid-sentence (CA SB
243-style requirements).
- **Per-tenant dynamic greeting.** `greeting.text` also accepts a
resolver `({ metadata, requestContext, roomName, ctx }) => string |
undefined`, resolved once per call *post-connect* — so one shared
multi-tenant agent opens with each tenant's own brand/disclosure
phrasing keyed off dispatch metadata / request context.
- **`configuration.stt` / `configuration.tts` — per-call resolvers.**
Pick the transcriber and voice per call — one voice or language per
tenant — falling back to the top-level `stt`/`tts` when they return
`undefined`.
- **`configuration.consentPolicy` — named consent model.** (Renamed from
`requireConsent` per review — it is declarative only; the worker
enforces nothing by itself.) Declares the data-use consents a call needs
as an extensible named set (starting with `summaryStorage`);
requirements are surfaced on `onCallEnd` for enforcement. Paired with
**`createConsentTool`** (new server-safe root export): captures each `{
item, granted }` decision at runtime with the caller identity from tool
context and hands it to an app-owned store.
- **`configuration.endCall` — agent-initiated hang-up.** Paired with
**`createEndCallTool`** (new server-safe root export). The worker waits
for the goodbye to finish playing out (`waitForAgentDoneSpeaking`,
capped by `maxWaitMs`), optionally speaks a non-interruptible closing
`message`, then deletes the room and shuts down the job — running
`onCallEnd` exactly as a caller hang-up does. Works on both the agent
and workflow reply paths.
- **Backwards compatible.** The released top-level `greeting` /
`persistGreeting` options remain as deprecated aliases;
`configuration.greeting` wins field-by-field.

### `@mastra/livekit/plugin` — `MastraLLM`, use Mastra as the LLM inside
a customer-owned LiveKit session

Until now, using Mastra for voice meant Mastra ran the *whole* call.
`MastraLLM` flips the ownership: the customer keeps their own LiveKit
`AgentSession` — their STT/TTS vendors, turn detection, deployment — and
drops Mastra into the LLM slot as a standard `llm.LLM` plugin.

- **New `@mastra/livekit/plugin` subpath export** (root stays
server-safe): `MastraLLM extends llm.LLM`, plugs straight into
`voice.AgentSession({ llm: new MastraLLM(...) })`.
- **Three interchangeable brain sources** (exactly one required):
`remote` (HTTP/SSE to a running Mastra server via
`createRemoteAgentReplyGenerator`, with abort propagation and
configurable timeouts), `agent` (in-process), or `generate` (fully
custom).
- **Mastra features surface cleanly:** thread/resource memory plumbed
per call; shared message ids dedupe what the server already persisted;
LiveKit-side `instructions` filtered rather than double-applied;
`memory` + LiveKit's preemptive generation guarded and documented.
- **Interrupted turns self-heal:** on barge-in the server skips
persisting the unheard reply; the plugin records only what the caller
actually heard, flags it interrupted, and reconciles it into memory on
the next turn.
- **Hooks:** `toolFeedback` (spoken filler while a tool runs),
`onToolCall`, `onTurnComplete` (per-turn
text/tool-calls/interrupted/usage, off the audio path).
`createLiveKitWorker` gained the same `onToolCall`/`onTurnComplete`
hooks, since both paths now share one reply seam.

### Drift fix vs the original branch

`main` bumped `@livekit/agents` 1.4.11 → 1.5.0 after #19019 branched.
1.5.0 turns `ToolContext` into a class that `AgentSession` always passes
to `llm.chat()` — `Object.keys()` on it saw only private fields, so
`MastraLLM`'s ignored-tools warning fired on every session (even
tool-less) with garbage names. Fixed in a dedicated commit (262d90d7fc):
tool names now come from the instance's `flatten()`, with plain-map
fallback for 1.4; two regression tests added.

## Scope

- All `integrations/livekit` changes: `configuration`
(greeting/consent/endCall/stt/tts), consent + end-call tools,
`MastraLLM` plugin, remote SSE transport, reply-seam hooks, README
- Docs: LiveKit voice guide (wrapper-vs-plugin comparison,
preemptive-generation warning) + full reference entries
- Changesets: 3 × `@mastra/livekit` minor (configuration, stt/tts
resolvers, plugin)

## Also included (consolidated from #19137)

- `examples/voice-agent`: permissive default worker + regulated demo
(consent-gated summarization) + customer-owned-session plugin worker
- `@mastra/memory` summarization — PR 1 (#19135); the README's
`memory.summarizeThread()` mentions depend on it, hence the stack order

## Verification

- `pnpm vitest run` in `integrations/livekit` — 186/186 passing across
15 files (was 184; +2 regression tests for the 1.5.0 ToolContext fix)
- `tsc --noEmit`, `pnpm build:lib`, `pnpm lint` — clean
- Verified live over LiveKit + Studio voice mode on the original branch:
disclosure playout, consent capture, agent-initiated hang-up, and the
plugin end-to-end against a remote Mastra server (barge-in
reconciliation, spoken tool feedback, `onTurnComplete` usage)

## Type of change

- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5
This PR upgrades the LiveKit voice agent so you can control—per call—how
it introduces itself, when/what consent it asks for, and how it ends
(including letting the agent hang up). It also adds a new
`@mastra/livekit/plugin` entry so customers can plug Mastra
(`MastraLLM`) into their own LiveKit `AgentSession`, plus tools to
record consent and trigger a safe end-call flow.

## Summary
- Updated `createLiveKitWorker` with a grouped per-call `configuration`
option for:
  - greeting / AI disclosure behavior,
  - consent via a named `consentPolicy` model,
  - optional agent-initiated hang-up via `endCall`,
- per-call `stt`/`tts` resolvers (run once per call) with fallback to
legacy top-level `stt`/`tts`.
- Kept legacy top-level `greeting` and `persistGreeting` as deprecated
aliases, with `configuration.greeting.*` taking precedence.
- Added new server-safe root exports:
- `createConsentTool` to record runtime consent decisions (including
caller identity fields) using schema validation and an `onGrant`
persistence callback.
- `createEndCallTool` to let the agent end the call after an optional
goodbye/closing workflow, while ensuring shutdown flow continues even if
bookkeeping fails (warns instead of breaking execution).
- Introduced `@mastra/livekit/plugin` with `MastraLLM`, a LiveKit
LLM-component for customer-owned `AgentSession`s, supporting exactly one
reply source: `remote`, `agent`, or `generate`.
- Includes remote HTTP/SSE reply generation, interrupted-turn
“self-healing” memory reconciliation, tool activity hooks
(`toolFeedback`, `onToolCall`, `onTurnComplete`), and token usage
surfaced via `result.usage`.
- Improved worker robustness and observability:
  - AI re-disclosure reminders,
  - mid-stream `onToolCall` notifications,
- stable message `id` propagation and filtering of LiveKit-injected
instruction messages,
  - wired `configuration.endCall` into the workflow reply path,
  - tightened watchdog + hook error handling, plus regression tests.
- Consolidated and refreshed `examples/voice-agent` (including new
regulated and plugin worker scripts) and updated LiveKit docs/reference
to reflect the new grouped configuration, `consentPolicy` rename, and
`@mastra/livekit/plugin` entrypoint.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 21:01:16 -07:00