0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
mastra/voice/google-gemini-live-api
History 2026-11-12 21:12:29 +00:00
..
src refactor(lint): enable no-else-return at root (#28601) 2026-10-28 12:30:27 +01:00
architecture-llm.md fix(voice-google-gemini-live): include sample rate in realtime PCM frames (#23834) 2026-09-14 12:20:30 +01:00
CHANGELOG.md chore: version - exit prerelease mode 2026-11-12 21:12:29 +00:00
lint-staged.config.js refactor(lint): replace ESLint with type-aware oxlint (#27476) 2026-10-18 10:38:35 -07:00
package.json chore: version - exit prerelease mode 2026-11-12 21:12:29 +00:00
README.md chore(*): Update/unify/add READMEs (#22858) 2026-09-02 14:36:59 +02:00
tsconfig.build.json Feat/gemini live api integration (#6677) 2025-08-15 10:58:16 -07:00
tsconfig.json chore: migrate package builds to tsdown (#19829) 2026-07-28 12:32:21 +02:00
tsdown.config.ts chore: migrate package builds to tsdown (#19829) 2026-07-28 12:32:21 +02:00
vitest.config.ts refactor: reorganize core tests and namespace vitest projects across monorepo (#12662) 2026-02-09 10:24:48 +01:00

@mastra/voice-google-gemini-live

Google Gemini Live API integration for Mastra, providing real-time multimodal voice interactions with advanced capabilities including video input, tool calling, and session management.

Installation

npm install @mastra/voice-google-gemini-live

Usage

import { GeminiLiveVoice } from '@mastra/voice-google-gemini-live';

const voice = new GeminiLiveVoice({
  apiKey: process.env.GOOGLE_API_KEY,
  model: 'gemini-2.0-flash-live-001',
  speaker: 'Puck',
});

// Connect to the Live API
await voice.connect();

// Listen for responses
voice.on('speaking', ({ audioData }) => {
  // Handle audio response as Int16Array
  playAudio(audioData);
});

// Or subscribe to a concatenated audio stream per response
voice.on('speaker', audioStream => {
  audioStream.pipe(playbackDevice);
});

voice.on('writing', ({ text, role }) => {
  // role: 'user'      → speech-to-text of the caller
  // role: 'assistant' → speech-to-text of the model's spoken reply
  console.log(`${role}: ${text}`);
});

// Native-audio models only: model's internal reasoning
voice.on('thinking', ({ text }) => {
  console.log(`thinking: ${text}`);
});

// Drop queued playback when the user barges in over the model
voice.on('interrupt', ({ type, timestamp }) => {
  console.log(`interrupt by ${type} at ${timestamp}`);
});

// Send text to speech
await voice.speak('Hello from Mastra!');

// Send audio stream
const microphoneStream = getMicrophoneStream();
await voice.send(microphoneStream);

// When done, disconnect
voice.disconnect();

Documentation

Changelog

See the package changelog for version history and release notes.

Support

We have an open community Discord. Come and say hello and let us know if you have any questions or need any help getting things running.