0
Fork 0
mirror of https://github.com/mastra-ai/mastra.git synced 2026-11-17 08:44:46 +00:00
mastra/observability/langsmith/src
Anurag Ojha 81cf36f60d
feat(observability): record and query time to first token on spans, traces and threads (#28699)
## Description

Fills the `timeToFirstTokenMs` span column that #28389 added empty, and
lets trace, span and thread queries return, filter and sort by it. Time
to first token is the time from the start of the model call (the
`model_inference` span) to the first streamed chunk. Unlike
`timeToFirstChunkMs`, it leaves out the work Mastra does before calling
the model.

- The tracker measures the value once, on the `model_inference` span,
when its first chunk arrives, and copies that number up the agent's own
chain (`model_step`, `model_generation`, `agent_run`) as a
`timeToFirstTokenMs` attribute, first write wins, the same way
`completionStartTime` is copied. Each step takes its own inference's
value; the generation and agent run take the first step's.
- Span export reads the attribute. Storage fills the column from it and
drops the key from the attributes JSON. The Platform collector and the
store row mappers already carry the column, so no DDL changes.
- Durable agents (core and `@mastra/inngest`) carry the number through
loop state beside `completionStartTime`, so a fresh tracker on another
worker ends the generation and agent run with both.
- Core contract, ClickHouse, Postgres and DuckDB return, filter and sort
the field. Trace rows use the root span's value. Thread rows use the
first trace's root when it is an agent run. Empty values sort last.
Stores declare `trace-query-time-to-first-token` and `-sort`; the server
returns 501 when a store lacks them.
- A fallback model attempt opens its own `model_inference` span, named
after the model it calls, so its value and label are its own, and a
failed attempt's span ends with its error. Braintrust and LangSmith keep
the attribute out of span metadata.
- Client types regenerated, `useTraceQuery` keeps new traces in order
under the new sort, docs and changesets added. No Studio changes.
- Also fixes two gaps that already affected `timeToFirstChunkMs`:
durable abort and tripwire exits now keep the clocks on the ended spans,
and span aggregation gates the timing field instead of failing inside
the store.
- After a failed model call, in-process agents count the retry wait and
durable agents do not. One test marked expected-to-fail pins that
difference.

Rebased on `main` after #28389 merged.

## Related Issue(s)

N/A (OBS-722 follow-up)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Code refactoring
- [ ] Performance improvement
- [ ] Test update

## Checklist

- [x] I have made corresponding changes to the documentation (if
applicable)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have addressed all Coderabbit comments on this PR

🤖 Generated with [Claude Code](https://claude.com/claude-code)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## ELI5

This change measures how long a model takes to send its first piece of a
response. Supported storage systems can return, filter, and sort by this
time, which helps teams find slow responses.

## Summary

- Records `timeToFirstTokenMs` from the start of a model inference to
its first streamed chunk. The value is added to inference spans and
propagated to related model-step, generation, and agent-run spans.
- Gives each fallback model attempt its own inference span and timing
value. Carries timing across durable agent steps, including resumed runs
and abort or tripwire exits.
- Adds `timeToFirstTokenMs` to trace, span, and thread query results and
predicates. Trace values use the root span. Thread values use the first
trace’s root agent span.
- Adds sorting by this field for trace and span queries in ClickHouse,
PostgreSQL, and DuckDB. Missing values sort last. The server returns
`501` when a store does not support a requested predicate or sort.
- Updates client types, `useTraceQuery`, documentation, storage
capability declarations, and changesets. Braintrust and LangSmith
exporters exclude the field from span metadata.
- Adds and updates tests across tracing, durable workflows, query
handling, and storage conformance. Test execution results were not
provided.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
History 2026-10-29 14:09:27 +05:30
..
helpers.test.ts feat(langsmith): add vendor metadata support for dynamic project routing (#12690) 2026-02-05 12:19:37 -05:00
helpers.ts feat(langsmith): add vendor metadata support for dynamic project routing (#12690) 2026-02-05 12:19:37 -05:00
index.ts feat(langsmith): add vendor metadata support for dynamic project routing (#12690) 2026-02-05 12:19:37 -05:00
metrics.test.ts fix(observability): preserve Anthropic cache TTL usage (#21563) 2026-08-14 20:11:35 -07:00
metrics.ts fix(observability): preserve Anthropic cache TTL usage (#21563) 2026-08-14 20:11:35 -07:00
tracing.early-data.test.ts fix(langsmith): send queued batches in manual flush mode and after shutdown aborts (#26760) 2026-10-13 23:53:37 +05:30
tracing.test.ts feat(observability): record and query time to first token on spans, traces and threads (#28699) 2026-10-29 14:09:27 +05:30
tracing.ts feat(observability): record and query time to first token on spans, traces and threads (#28699) 2026-10-29 14:09:27 +05:30