0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
mastra/voice/openai-realtime-api
History 2026-11-12 21:12:29 +00:00
..
src fix(voice): replace retired default OpenAI Realtime model (#30183) 2026-11-11 23:07:34 -08:00
CHANGELOG.md chore: version - exit prerelease mode 2026-11-12 21:12:29 +00:00
lint-staged.config.js refactor(lint): replace ESLint with type-aware oxlint (#27476) 2026-10-18 10:38:35 -07:00
package.json chore: version - exit prerelease mode 2026-11-12 21:12:29 +00:00
README.md fix(voice): replace retired default OpenAI Realtime model (#30183) 2026-11-11 23:07:34 -08:00
tsconfig.build.json chore: fix go-to-definition using tsc declaration maps (#6089) 2025-07-30 15:41:00 -07:00
tsconfig.json chore: migrate package builds to tsdown (#19829) 2026-07-28 12:32:21 +02:00
tsdown.config.ts chore: migrate package builds to tsdown (#19829) 2026-07-28 12:32:21 +02:00
vitest.config.ts refactor: reorganize core tests and namespace vitest projects across monorepo (#12662) 2026-02-09 10:24:48 +01:00

@mastra/voice-openai-realtime

OpenAI Realtime Voice integration for Mastra, providing real-time voice interaction capabilities using OpenAI's WebSocket-based API. This integration enables seamless voice conversations with real-time speech to speech capabilities.

Installation

npm install @mastra/voice-openai-realtime

Usage

import { OpenAIRealtimeVoice } from '@mastra/voice-openai-realtime';
import { getMicrophoneStream } from '@mastra/node-audio';

const voice = new OpenAIRealtimeVoice({
  apiKey: process.env.OPENAI_API_KEY,
  model: 'gpt-realtime-2.1-mini',
});

voice.updateSession({
  turn_detection: {
    type: 'server_vad',
    threshold: 0.5,
    silence_duration_ms: 1000,
  },
});

// Connect to the realtime service
await voice.connect();

// Audio data from voice provider
voice.on('speaking', (audioData: Int16Array) => {
  // Handle audio data
});

// Text data from voice provider
voice.on('writing', (text: string) => {
  // Handle transcribed text
});

// Error from voice provider
voice.on('error', (error: Error) => {
  console.error('Voice error:', error);
});

// Generate speech
await voice.speak('Hello from Mastra!', {
  speaker: 'echo', // Optional: override default speaker
});

// Listen to audio input
await voice.listen(audioData);

// Process audio input
const microphoneStream = getMicrophoneStream();
await voice.send(microphoneStream);

// Clean up
voice.close();

Documentation

Changelog

See the package changelog for version history and release notes.

Support

We have an open community Discord. Come and say hello and let us know if you have any questions or need any help getting things running.