mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
|
History
|
||
|---|---|---|
| .. | ||
| __recordings__ | ||
| src | ||
| .gitignore | ||
| CHANGELOG.md | ||
| lint-staged.config.js | ||
| package.json | ||
| README.md | ||
| tsconfig.build.json | ||
| tsconfig.json | ||
| tsdown.config.ts | ||
| turbo.json | ||
| vitest.config.ts | ||
@mastra/evals
@mastra/evals ships a collection of scoring utilities you can run locally or inside your own evaluation pipelines. These scorers come in two flavors:
- LLM scorers – leverage a judge model (e.g. OpenAI, Anthropic) to rate responses for qualities such as faithfulness or toxicity.
- Code/NLP scorers – deterministic heuristics (keyword coverage, similarity, etc.) that do not require an external model.
Installation
npm install @mastra/evals
Usage
Import prebuilt scorers from the package subpath.
import { createToolCallAccuracyScorerCode } from '@mastra/evals/scorers/prebuilt';
const scorer = createToolCallAccuracyScorerCode({ expectedTool: 'weather' });
Documentation
Changelog
See the package changelog for version history and release notes.
Support
We have an open community Discord. Come and say hello and let us know if you have any questions or need any help getting things running.