A search-augmented agent looks like RAG until the citation list breaks. The first user clicks through, lands on a 404, and the trust budget spends in one impression. Three months of eval work, groundedness and context adherence and factual accuracy a...

Your multi-agent system works locally. Three agents coordinate, call tools, pass context, return a clean answer. In production the final output is wrong, and you cannot tell which agent failed, which tool call returned garbage, or where the reasonin...

Your custom metric is scoring noise unless it has these three parts A medical agent passes your faithfulness and refusal gates all quarter. A week after launch, a review flags outputs with unverified drug-interaction claims. The faithfulness rubric...

A Pydantic AI agent returns BookingResultconfirmed=True, flightid='AA127', totalusd=412.50. Every Pydantic check passes. The outputtype matches. The tool-call arguments validated against the Pydantic model on the way in. The retry counter is at zero....

A support model picks refundorder, fills the schema, validates clean, and pushes $54 to the right account. The user asked about $5,400. Tool selection scored 1.0. The arguments passed schema validation. The number was off by an order of magnitude, an...

You have eleven versions of the same prompt in a doc, three of them named "final," and no real idea which one scores best on last week's failures. So you tweak it again by hand, read ten outputs, decide it seems better, and ship. That is not optimiza...

I have sat in this incident more than once. A user reports a regression on Tuesday. On-call pulls the trace, and the span tree is complete: provider, model, tokens, finish reasons, retrieval chunks, eval scores. None of it carries the prompt version....

Every MCP server I have worked on is really two products under one binary. The first is the functional surface: tool descriptions that match what the tools actually do, calls that succeed with the right arguments, results that flow into the next age...

I keep seeing the same failure land on teams that do everything right. The CI eval gate has been green for six weeks, faithfulness at 0.87, task completion at 0.91. Then a support thread surfaces a hallucination on a feature shipped in March. The tra...

How to evaluate a voice agent before launch with simulation Evaluating a voice agent used to mean listening to recordings, then it meant scripts that replayed transcripts at the agent. Neither catches much. The approach that works now is to simula...

The six conversation metrics that catch what voice agent dashboards miss Most teams have shipped voice agents and most are not happy with them. In Hamming AI's State of Voice AI 2026, 87 percent of companies have deployed voice agents but only 12 p...

A trace is one of the more useful things you can capture from an LLM app, and many teams store it in a format they cannot move later. If your AI tracing uses its own private format, the spans sit in a silo, a backend change means re-instrumenting, an...

I want to start with a trace that still bothers me. An agent fails to book a flight. The model called searchflights with departuredate="next Friday". The endpoint expected an ISO date, returned a 400, and the agent retried four times with the exact ...