I have sat in this incident more than once. A user reports a regression on Tuesday. On-call pulls the trace, and the span tree is complete: provider, model, tokens, finish reasons, retrieval chunks, eval scores. None of it carries the prompt version....
Every MCP server I have worked on is really two products under one binary.
The first is the functional surface: tool descriptions that match what the tools actually do, calls that succeed with the right arguments, results that flow into the next age...
I keep seeing the same failure land on teams that do everything right. The CI eval gate has been green for six weeks, faithfulness at 0.87, task completion at 0.91. Then a support thread surfaces a hallucination on a feature shipped in March. The tra...
How to evaluate a voice agent before launch with simulation
Evaluating a voice agent used to mean listening to recordings, then it meant scripts that replayed transcripts at the agent. Neither catches much.
The approach that works now is to simula...
The six conversation metrics that catch what voice agent dashboards miss
Most teams have shipped voice agents and most are not happy with them. In Hamming AI's State of Voice AI 2026, 87 percent of companies have deployed voice agents but only 12 p...
A trace is one of the more useful things you can capture from an LLM app, and many teams store it in a format they cannot move later. If your AI tracing uses its own private format, the spans sit in a silo, a backend change means re-instrumenting, an...
I want to start with a trace that still bothers me.
An agent fails to book a flight. The model called searchflights with departuredate="next Friday". The endpoint expected an ISO date, returned a 400, and the agent retried four times with the exact ...