mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
|
History
|
||
|---|---|---|
| .. | ||
| src | ||
| CHANGELOG.md | ||
| lint-staged.config.js | ||
| package.json | ||
| README.md | ||
| tsconfig.build.json | ||
| tsconfig.json | ||
| tsdown.config.ts | ||
| vitest.config.ts | ||
@mastra/voice-openai-realtime
OpenAI Realtime Voice integration for Mastra, providing real-time voice interaction capabilities using OpenAI's WebSocket-based API. This integration enables seamless voice conversations with real-time speech to speech capabilities.
Installation
npm install @mastra/voice-openai-realtime
Usage
import { OpenAIRealtimeVoice } from '@mastra/voice-openai-realtime';
import { getMicrophoneStream } from '@mastra/node-audio';
const voice = new OpenAIRealtimeVoice({
apiKey: process.env.OPENAI_API_KEY,
model: 'gpt-realtime-2.1-mini',
});
voice.updateSession({
turn_detection: {
type: 'server_vad',
threshold: 0.5,
silence_duration_ms: 1000,
},
});
// Connect to the realtime service
await voice.connect();
// Audio data from voice provider
voice.on('speaking', (audioData: Int16Array) => {
// Handle audio data
});
// Text data from voice provider
voice.on('writing', (text: string) => {
// Handle transcribed text
});
// Error from voice provider
voice.on('error', (error: Error) => {
console.error('Voice error:', error);
});
// Generate speech
await voice.speak('Hello from Mastra!', {
speaker: 'echo', // Optional: override default speaker
});
// Listen to audio input
await voice.listen(audioData);
// Process audio input
const microphoneStream = getMicrophoneStream();
await voice.send(microphoneStream);
// Clean up
voice.close();
Documentation
Changelog
See the package changelog for version history and release notes.
Support
We have an open community Discord. Come and say hello and let us know if you have any questions or need any help getting things running.