mirror of
https://github.com/mastra-ai/mastra.git
synced 2026-11-17 08:44:46 +00:00
## Description Fills the `timeToFirstTokenMs` span column that #28389 added empty, and lets trace, span and thread queries return, filter and sort by it. Time to first token is the time from the start of the model call (the `model_inference` span) to the first streamed chunk. Unlike `timeToFirstChunkMs`, it leaves out the work Mastra does before calling the model. - The tracker measures the value once, on the `model_inference` span, when its first chunk arrives, and copies that number up the agent's own chain (`model_step`, `model_generation`, `agent_run`) as a `timeToFirstTokenMs` attribute, first write wins, the same way `completionStartTime` is copied. Each step takes its own inference's value; the generation and agent run take the first step's. - Span export reads the attribute. Storage fills the column from it and drops the key from the attributes JSON. The Platform collector and the store row mappers already carry the column, so no DDL changes. - Durable agents (core and `@mastra/inngest`) carry the number through loop state beside `completionStartTime`, so a fresh tracker on another worker ends the generation and agent run with both. - Core contract, ClickHouse, Postgres and DuckDB return, filter and sort the field. Trace rows use the root span's value. Thread rows use the first trace's root when it is an agent run. Empty values sort last. Stores declare `trace-query-time-to-first-token` and `-sort`; the server returns 501 when a store lacks them. - A fallback model attempt opens its own `model_inference` span, named after the model it calls, so its value and label are its own, and a failed attempt's span ends with its error. Braintrust and LangSmith keep the attribute out of span metadata. - Client types regenerated, `useTraceQuery` keeps new traces in order under the new sort, docs and changesets added. No Studio changes. - Also fixes two gaps that already affected `timeToFirstChunkMs`: durable abort and tripwire exits now keep the clocks on the ended spans, and span aggregation gates the timing field instead of failing inside the store. - After a failed model call, in-process agents count the retry wait and durable agents do not. One test marked expected-to-fail pins that difference. Rebased on `main` after #28389 merged. ## Related Issue(s) N/A (OBS-722 follow-up) ## Type of Change - [ ] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Code refactoring - [ ] Performance improvement - [ ] Test update ## Checklist - [x] I have made corresponding changes to the documentation (if applicable) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have addressed all Coderabbit comments on this PR 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> <!-- coderabbit-ai-generated:v1 --> ## ELI5 This change measures how long a model takes to send its first piece of a response. Supported storage systems can return, filter, and sort by this time, which helps teams find slow responses. ## Summary - Records `timeToFirstTokenMs` from the start of a model inference to its first streamed chunk. The value is added to inference spans and propagated to related model-step, generation, and agent-run spans. - Gives each fallback model attempt its own inference span and timing value. Carries timing across durable agent steps, including resumed runs and abort or tripwire exits. - Adds `timeToFirstTokenMs` to trace, span, and thread query results and predicates. Trace values use the root span. Thread values use the first trace’s root agent span. - Adds sorting by this field for trace and span queries in ClickHouse, PostgreSQL, and DuckDB. Missing values sort last. The server returns `501` when a store does not support a requested predicate or sort. - Updates client types, `useTraceQuery`, documentation, storage capability declarations, and changesets. Braintrust and LangSmith exporters exclude the field from span metadata. - Adds and updates tests across tracing, durable workflows, query handling, and storage conformance. Test execution results were not provided. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
History
|
|
|---|---|---|
| .. | ||
| helpers.test.ts | ||
| helpers.ts | ||
| index.ts | ||
| metrics.test.ts | ||
| metrics.ts | ||
| tracing.early-data.test.ts | ||
| tracing.test.ts | ||
| tracing.ts | ||