mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
84143a892e |
feat(livekit)!: prepare 0.5.0 with recording and speech metrics (#24674)
Prepare the LiveKit 0.5.0 release with opt-in call recording and Studio
playback. `dispatchVoiceSession()` and `liveKitConnectionRoute()` can
create a fresh room with LiveKit auto egress before dispatching the
agent. The recording route resolves an authorized trace to a short-lived
playback URL, and Studio exposes it through **Review Audio**.
```ts
await dispatchVoiceSession({
roomName: `call-${crypto.randomUUID()}`,
metadata: { agentId: 'support' },
recording, // Server-side LiveKit output settings.
});
```
Recording review requires an explicit `authorize({ traceId, context })`
callback. Only `true` permits trace lookup and playback URL resolution;
missing policies fail at registration. Recording room conflicts return
HTTP 409. The example and documentation cover S3 configuration, scoped
permissions, consent, and recording lifecycle implications. Browser
recording creation and playback in the unauthenticated example require
an explicit local-demo opt-in and are disabled in production.
The browser-safe `@mastra/livekit/client` entry point provides
`getLiveKitRecording(client, traceId)`, preserving the client's
authentication and request configuration. Studio uses the supported
React hook subpaths from main.
The integration also adds common per-turn metrics, speech-flush
boundaries, and an `onSpeechComplete` hook. `onTurnComplete` retains its
generation-only behavior. Custom reply-generator consumers must handle
flush boundaries, and `MastraLLM` integrations should use
`mastraLLMNode`. Playback completion describes server delivery, not
proof that the listener heard the audio.
The VAD opt-out is fixed. LiveKit Agents 1.7.1 is the supported minimum
and migration target, with matching plugin peer ranges and upgrade
instructions. The planned stable release remains **0.5.0**.
### Validation
- All 280 LiveKit tests across 22 files passed.
- All 18 Studio voice and recording-review tests passed.
- A local synthetic probe passed 96 example route-policy cases covering
room creation, dispatch, unrecorded tokens, storage access, and denied
playback.
- LiveKit, React SDK, shared Studio UI, and production Studio builds
passed.
- LiveKit, Studio, and voice-agent example typechecks passed; focused
lint and documentation checks passed.
- Production bundle inspection confirmed that the recording client adds
no LiveKit server or worker modules to the browser bundle.
- Changeset planning retains the 0.5.0 release target.
The core working-memory configuration fix and its regression tests are
in #26090 for separate review.
|
||
|
|
f701530e7a |
docs(react): add narrow API responsibility rule (#19336)
Adds a React best-practices rule for components, hooks, functions, and utilities with oversized input or output APIs. It clarifies that wrapping the same values in one object does not reduce responsibility and shows how to split broad hooks into focused units composed at the component level.\n\nThe rule catalog and quick reference now include the new guidance and updated rule counts. Documentation structure, links, counts, and fenced examples were validated. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## ELI5 This PR adds a new React “best practice” rule: if a component/hook/function is taking in (or returning) too much at once, you should split it into smaller, clearer pieces and then combine them where it’s used. It also updates the rule catalog and adds a bit of formatting/coverage cleanup in the voice-agent example tests. ## Changes - Added a new React best-practices rule: `structure-narrow-apis` (“Keep Input and Output APIs Narrow”) for components, hooks, functions, and utilities. - Clarified that wrapping unrelated things into a single parameter/return object doesn’t truly narrow responsibility. - Included incorrect vs. correct examples showing how to split an oversized “mega-hook” into focused hooks and compose them in a container/component. - Updated the React best-practices rule catalog/quick reference (version and rule/category counts), adding the new rule entry and revising totals from 25 to 26 rules. - Reformatted and lightly adjusted voice-agent example smoke coverage and related example code without changing core behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
|
25c1a4f6c0 |
feat(livekit): grouped worker configuration + MastraLLM plugin + voice-agent example (#19136)
## Description This is the final PR of the stack split from #19019: memory summarization (#19135, merged) → **livekit worker configuration + MastraLLM plugin + voice-agent example (this PR)**. The example PR (#19137) was consolidated into this one after review feedback landed, so the package changes and the runnable example that exercises them ship and get reviewed together. > **Note for reviewers:** Tyler's split request listed the worker `configuration` surface; the `MastraLLM` plugin work landed on the original branch after that comment. Both are `@mastra/livekit` package code, so they ride together here — happy to split the plugin into its own 4th PR if you'd rather review it separately. ### `@mastra/livekit` — new grouped `configuration` worker option One home for conversation & compliance policy on `createLiveKitWorker`, so each control doesn't become another top-level option: - **`configuration.greeting` — AI disclosure controls.** `text` spoken at call start; `allowInterruptions: false` so a legally-required AI disclosure plays through and can't be barged over (EU AI Act Art. 50); `awaitPlayout: true` holds post-greeting work (persistence, `onSessionStart`) until it finishes; `persist` saves it to the memory thread; `repeatEvery`/`repeatText` re-disclose periodically on long calls, spoken at the next turn boundary, never mid-sentence (CA SB 243-style requirements). - **Per-tenant dynamic greeting.** `greeting.text` also accepts a resolver `({ metadata, requestContext, roomName, ctx }) => string | undefined`, resolved once per call *post-connect* — so one shared multi-tenant agent opens with each tenant's own brand/disclosure phrasing keyed off dispatch metadata / request context. - **`configuration.stt` / `configuration.tts` — per-call resolvers.** Pick the transcriber and voice per call — one voice or language per tenant — falling back to the top-level `stt`/`tts` when they return `undefined`. - **`configuration.consentPolicy` — named consent model.** (Renamed from `requireConsent` per review — it is declarative only; the worker enforces nothing by itself.) Declares the data-use consents a call needs as an extensible named set (starting with `summaryStorage`); requirements are surfaced on `onCallEnd` for enforcement. Paired with **`createConsentTool`** (new server-safe root export): captures each `{ item, granted }` decision at runtime with the caller identity from tool context and hands it to an app-owned store. - **`configuration.endCall` — agent-initiated hang-up.** Paired with **`createEndCallTool`** (new server-safe root export). The worker waits for the goodbye to finish playing out (`waitForAgentDoneSpeaking`, capped by `maxWaitMs`), optionally speaks a non-interruptible closing `message`, then deletes the room and shuts down the job — running `onCallEnd` exactly as a caller hang-up does. Works on both the agent and workflow reply paths. - **Backwards compatible.** The released top-level `greeting` / `persistGreeting` options remain as deprecated aliases; `configuration.greeting` wins field-by-field. ### `@mastra/livekit/plugin` — `MastraLLM`, use Mastra as the LLM inside a customer-owned LiveKit session Until now, using Mastra for voice meant Mastra ran the *whole* call. `MastraLLM` flips the ownership: the customer keeps their own LiveKit `AgentSession` — their STT/TTS vendors, turn detection, deployment — and drops Mastra into the LLM slot as a standard `llm.LLM` plugin. - **New `@mastra/livekit/plugin` subpath export** (root stays server-safe): `MastraLLM extends llm.LLM`, plugs straight into `voice.AgentSession({ llm: new MastraLLM(...) })`. - **Three interchangeable brain sources** (exactly one required): `remote` (HTTP/SSE to a running Mastra server via `createRemoteAgentReplyGenerator`, with abort propagation and configurable timeouts), `agent` (in-process), or `generate` (fully custom). - **Mastra features surface cleanly:** thread/resource memory plumbed per call; shared message ids dedupe what the server already persisted; LiveKit-side `instructions` filtered rather than double-applied; `memory` + LiveKit's preemptive generation guarded and documented. - **Interrupted turns self-heal:** on barge-in the server skips persisting the unheard reply; the plugin records only what the caller actually heard, flags it interrupted, and reconciles it into memory on the next turn. - **Hooks:** `toolFeedback` (spoken filler while a tool runs), `onToolCall`, `onTurnComplete` (per-turn text/tool-calls/interrupted/usage, off the audio path). `createLiveKitWorker` gained the same `onToolCall`/`onTurnComplete` hooks, since both paths now share one reply seam. ### Drift fix vs the original branch `main` bumped `@livekit/agents` 1.4.11 → 1.5.0 after #19019 branched. 1.5.0 turns `ToolContext` into a class that `AgentSession` always passes to `llm.chat()` — `Object.keys()` on it saw only private fields, so `MastraLLM`'s ignored-tools warning fired on every session (even tool-less) with garbage names. Fixed in a dedicated commit (262d90d7fc): tool names now come from the instance's `flatten()`, with plain-map fallback for 1.4; two regression tests added. ## Scope - All `integrations/livekit` changes: `configuration` (greeting/consent/endCall/stt/tts), consent + end-call tools, `MastraLLM` plugin, remote SSE transport, reply-seam hooks, README - Docs: LiveKit voice guide (wrapper-vs-plugin comparison, preemptive-generation warning) + full reference entries - Changesets: 3 × `@mastra/livekit` minor (configuration, stt/tts resolvers, plugin) ## Also included (consolidated from #19137) - `examples/voice-agent`: permissive default worker + regulated demo (consent-gated summarization) + customer-owned-session plugin worker - `@mastra/memory` summarization — PR 1 (#19135); the README's `memory.summarizeThread()` mentions depend on it, hence the stack order ## Verification - `pnpm vitest run` in `integrations/livekit` — 186/186 passing across 15 files (was 184; +2 regression tests for the 1.5.0 ToolContext fix) - `tsc --noEmit`, `pnpm build:lib`, `pnpm lint` — clean - Verified live over LiveKit + Studio voice mode on the original branch: disclosure playout, consent capture, agent-initiated hang-up, and the plugin end-to-end against a remote Mastra server (barge-in reconciliation, spoken tool feedback, `onTurnComplete` usage) ## Type of change - [x] New feature (non-breaking change that adds functionality) - [x] Documentation update 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## ELI5 This PR upgrades the LiveKit voice agent so you can control—per call—how it introduces itself, when/what consent it asks for, and how it ends (including letting the agent hang up). It also adds a new `@mastra/livekit/plugin` entry so customers can plug Mastra (`MastraLLM`) into their own LiveKit `AgentSession`, plus tools to record consent and trigger a safe end-call flow. ## Summary - Updated `createLiveKitWorker` with a grouped per-call `configuration` option for: - greeting / AI disclosure behavior, - consent via a named `consentPolicy` model, - optional agent-initiated hang-up via `endCall`, - per-call `stt`/`tts` resolvers (run once per call) with fallback to legacy top-level `stt`/`tts`. - Kept legacy top-level `greeting` and `persistGreeting` as deprecated aliases, with `configuration.greeting.*` taking precedence. - Added new server-safe root exports: - `createConsentTool` to record runtime consent decisions (including caller identity fields) using schema validation and an `onGrant` persistence callback. - `createEndCallTool` to let the agent end the call after an optional goodbye/closing workflow, while ensuring shutdown flow continues even if bookkeeping fails (warns instead of breaking execution). - Introduced `@mastra/livekit/plugin` with `MastraLLM`, a LiveKit LLM-component for customer-owned `AgentSession`s, supporting exactly one reply source: `remote`, `agent`, or `generate`. - Includes remote HTTP/SSE reply generation, interrupted-turn “self-healing” memory reconciliation, tool activity hooks (`toolFeedback`, `onToolCall`, `onTurnComplete`), and token usage surfaced via `result.usage`. - Improved worker robustness and observability: - AI re-disclosure reminders, - mid-stream `onToolCall` notifications, - stable message `id` propagation and filtering of LiveKit-injected instruction messages, - wired `configuration.endCall` into the workflow reply path, - tightened watchdog + hook error handling, plus regression tests. - Consolidated and refreshed `examples/voice-agent` (including new regulated and plugin worker scripts) and updated LiveKit docs/reference to reflect the new grouped configuration, `consentPolicy` rename, and `@mastra/livekit/plugin` entrypoint. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |