Interviewed by: Mehadi Hasan
Developer: Yaoshen Luo
Role: AI Agent Evaluation and Benchmarking Specialist
Location: Shanghai, China
Website: aiagentbenchmark.com
AI agents are becoming increasingly capable, but capability alone does not guarantee that they will perform reliably in real-world environments. An agent may pass a benchmark, use tools correctly, and produce convincing answers, yet still fail when confronted with ambiguous customer requests, changing conditions, or workflows that require judgment.
For Yaoshen Luo, understanding these gaps has become a central part of his work. His approach to AI agent evaluation is grounded in practical experience: testing systems, investigating failures, working with real customer data, and turning difficult-to-reproduce problems into repeatable benchmarks.
His perspective is shaped by a career that began with traditional software engineering and expanded through mobile internet, location-based services, machine learning, and modern AI applications. Along the way, he has learned that evaluating an AI agent requires more than measuring whether it can complete an isolated task. It requires understanding what successful performance actually means to the people who depend on it.
In this CoderLegion Developer Story, Yaoshen shares how his engineering background influenced his approach to AI, why real customer failures can reveal weaknesses that conventional benchmarks miss, and what developers need to reconsider as AI agents move into increasingly complex workflows.
From a Pascal Program to Applied AI
Yaoshen's interest in programming began when he was just ten years old. During a computer class, he followed his teacher's instructions to write a Pascal program that used a for loop to calculate the sum of the numbers from 1 to 100.
That small exercise sparked an interest that eventually shaped his academic and professional direction. When it came time to choose a university major, he decided to study software engineering.
After graduating in 2014, he joined Baidu and experienced China's rapid mobile internet expansion firsthand. He worked on Baidu Maps, a product comparable to Google Maps in China, where practical business requirements gave him opportunities to work across front-end engineering, back-end engineering, and algorithms.
This exposure to different areas of engineering helped him develop a broader understanding of how software systems operate beyond individual components.
His work also coincided with the growth of big data in location-based services. During this period, he began studying machine learning through Andrew Ng's courses on Coursera and found opportunities to apply what he learned to real map-related problems, including navigation and positioning.
His entry into AI was therefore not driven by theory alone. It emerged from the combination of technical curiosity, changing technology, and the practical problems he encountered at work.
That experience continues to influence how he approaches AI today: the value of a technology becomes clearer when it is tested against the demands of an actual business problem.
Discovering the Potential of AI Agents
When large language models first began attracting widespread attention, Yaoshen was not immediately convinced that they would fundamentally change software development.
He initially found the technology relatively slow and wondered whether ChatGPT might follow the trajectory of earlier applications built around language models.
His perspective changed in 2024, when he began using AI coding tools, particularly Cursor and Windsurf.
Through hands-on experimentation, he started to believe that the way software was developed could change fundamentally within six months to a year.
Rather than relying on predictions, he chose to learn by using the tools himself. He was willing to pay for tools that worked well because he wanted to understand their capabilities through direct experience.
This early-adopter mindset also shaped his later interest in evaluation. As AI systems became more capable, it became increasingly important to distinguish between what they could demonstrate in a controlled test and what they could reliably accomplish in practice.
Building or using an AI agent can reveal its potential. Evaluating it properly requires a different discipline: identifying where it fails, understanding why those failures occur, and determining whether the system is useful enough for its intended purpose.
Why Traditional AI Benchmarks Can Miss the Real Problem
A benchmark provides a structured way to assess an AI system. It defines tasks, establishes evaluation criteria, and measures performance against expected outcomes.
However, Yaoshen believes many benchmarks simplify the conditions under which real work happens.
He illustrates this through his investigation of procurement agents. When he discussed automation with procurement professionals, he found that purchasing highly standardized products could be relatively suitable for agents.
But procurement involves more than selecting a product from a list. Supplier relationships introduce additional considerations, and the appropriate decision can depend on the situation.
A benchmark might treat a task as though there were one correct answer. Actual procurement work may allow several reasonable choices, each appropriate under different circumstances.
This distinction matters because a technically correct answer does not necessarily represent a good business decision.
The same issue appears across many knowledge-work workflows. Tasks that look straightforward when isolated can become much more complicated when they involve incomplete information, competing priorities, or decisions that depend on context.
For Yaoshen, the challenge is not simply to create harder benchmarks. It is to design evaluations that reflect the real structure of the work an agent is expected to perform.
When an Agent's Capabilities Do Not Translate Into Product Success
One of Yaoshen's important learning experiences came from an agent developed for an elderly-care project associated with a foundation.
The agent was designed to process follow-up conversations with elderly people and generate answers to a questionnaire.
Working with this project helped him recognise a distinction that remains central to his thinking: an agent's capabilities are not the same as successful product deployment or successful business results.
A system may demonstrate that it can perform a particular function without establishing that it can deliver the expected outcome consistently in a real setting.
Deployment introduces additional considerations. The surrounding workflow, the quality of the available information, the cost of operating the system, and the consequences of an incorrect result can all influence whether the solution is useful.
This experience reinforced the importance of evaluating AI agents in the context of the problems they are meant to solve.
The question is not merely whether an agent can generate an answer. It is whether the complete system can perform its intended job under the conditions that matter to its users.
Building a Voice Agent Benchmark From Real Customer Failures
One of Yaoshen's clearest examples of practical evaluation comes from his work on a voice agent.
When he first took over the project, he was not yet fully familiar with how its relevant functions had been implemented. He had tested the system himself, and its voiceprint and speaker-recognition performance appeared promising.
The turning point came through customer feedback.
The customer explained that several people would deliberately interrupt one another during conversations. They would also ask questions about the conversation's history, such as who had said what.
These scenarios exposed a gap between the system's apparent performance during testing and the customer's expectations for real conversations.
Yaoshen's first objective was straightforward: reproduce the customer's problem.
He did not begin by designing a comprehensive benchmark or searching for an existing dataset. Instead, he focused on making one specific failure repeatable.
Once he could reproduce the interaction, he had a runnable example of the problem. That individual case then became the starting point for a repeatable annotation tool, and the benchmark developed from there.
Importantly, he did not rely on an existing speaker-recognition benchmark dataset. He built the benchmark by annotating real service data.
This process reflects a practical principle in his work: a useful benchmark can begin with a single well-understood failure. The important step is to transform that failure into something that can be reproduced, examined, and tested repeatedly.
Turning Real-World Failures Into Measurable Tests
For Yaoshen, the process of building a benchmark does not necessarily begin with a fully defined evaluation framework.
Instead, it begins with repeated interaction with the system.
He uses the relevant functionality until he develops a better understanding of how it behaves. Then he asks two fundamental questions: Does the information needed to evaluate the problem actually exist, and what elements are involved?
Once the main elements become clearer, he begins designing the annotation process.
He starts by performing the annotation manually. This gives him a more concrete understanding of the information an evaluator must examine and the judgments they must make.
Only after that does he consider turning the process into a batch-processing tool.
The goal is to help an operator focus on a group of tasks and annotate them carefully within a concentrated period of time.
He then implements the tool and creates a small dataset. He typically annotates the initial samples himself and, when possible, asks other people to annotate similar examples.
Comparing those annotations can reveal disagreements and contradictions.
Rather than treating every disagreement as a simple annotation mistake, Yaoshen sees it as a signal that the underlying problem may need further investigation. The evaluation criteria might be unclear, the task may involve ambiguity, or the annotators may have different interpretations of the same evidence.
Resolving these differences helps establish clearer standards. Once the main disagreements have been addressed, he uses real daily data logs for annotation.
The resulting process can be understood as a progression:
- Reproduce a real failure.
- Identify the information needed to evaluate it.
- Define the annotation process manually.
- Build a tool that supports repeatable annotation.
- Compare annotations and investigate disagreements.
- Refine the evaluation criteria.
- Annotate real operational data.
The key is that the benchmark develops alongside the team's understanding of the problem. It is not simply a collection of predefined tests; it is a structured way to preserve and investigate failures discovered through actual use.
Evaluating More Than Correct Answers
Correctness is important, but Yaoshen pays particular attention to two additional aspects of agent behaviour.
The first is the consistency between an agent's words and its actions.
An agent might explain that it is going to perform an operation, but the important question is whether its subsequent actions actually align with that explanation.
The second is how the agent responds when an operation fails.
Does it simply repeat the same unsuccessful action, or does it change its approach and attempt a different method?
These behaviours matter because real workflows rarely consist entirely of successful, straightforward steps. Agents must often respond to unexpected results and decide what to do next.
For Yaoshen, these observations are also relevant to management. Evaluating an agent involves examining not only the final result but also the behaviour that leads to it.
This broader perspective can help reveal weaknesses that a simple success-or-failure score might overlook.
Why Good Evaluation Still Does Not Guarantee Production Success
In his writing about AI agent evaluation, Yaoshen has explored why promising evaluation results do not automatically translate into successful production deployment.
He identifies two major differences between controlled evaluation environments and real-world operation.
The evaluation may simplify the actual problem
Evaluation tasks are often abstracted to make them easier to define and measure. In doing so, they may exclude factors that determine whether the agent can solve the complete business problem.
Even when an agent successfully completes a task, the result may not be useful if the cost of achieving it is unacceptable to the customer.
This introduces an economic dimension to evaluation. A system's technical ability must be considered alongside the resources required to use it and the value of the outcome it produces.
The operating environment continues to change
Real-world systems do not necessarily operate under the same conditions indefinitely.
The surrounding environment can change, and those changes may affect an agent's performance over time. Yaoshen notes that there are still important unanswered questions about how such changes influence the sustainability of an agent's performance.
This makes ongoing evaluation particularly important.
A benchmark result describes performance under particular conditions. It does not, by itself, establish that the system will continue to perform well as the environment evolves.
For developers, the implication is clear: evaluation should not be treated as a one-time milestone that permanently establishes an agent's reliability.
What a Real Issue-Resolution Bot Revealed
Another important example comes from an issue bot Yaoshen has been working on for a voice-agent product.
The bot helps identify issues based on customer feedback. When customers report a problem, their descriptions are often vague.
A customer might simply say, "The audio playback was stuttering."
To investigate the issue, the bot must use the customer's text, images, and videos as context, reason about the available evidence, and repeatedly use tools to identify the relevant problem.
During traditional capability testing, the bot performed well on tasks such as demonstrating its ability to use tools.
But when it was applied to real data, the results exposed weaknesses that those tests had not captured.
A single set of clues could correspond to multiple possible cases. The bot might miss relevant possibilities or select the wrong issue.
These failures mattered because identifying the correct issue was the actual business objective. Demonstrating tool use was only one part of the larger task.
Yaoshen initially assumed that once the bot had located an issue successfully several times, it would be able to handle future cases in a similar way.
That assumption proved incorrect.
When the system was exposed to a broader range of real cases, different execution failures appeared at different stages. It could not identify every issue as expected.
The response was not simply to keep testing the same capability. He had to redesign the business workflow, introduce human fallback mechanisms, and add newly discovered problems to the benchmark so that they could guide further development.
This example highlights a recurring theme in his work: successful demonstrations do not establish that a system can handle the full range of cases it will encounter.
Real data can expose combinations of clues, ambiguities, and failure patterns that a carefully controlled test may never represent.
Making Real Data Part of the Benchmark
The issue-resolution bot also illustrates why Yaoshen believes the real data environment should become part of the evaluation process.
Traditional benchmarks can be useful for testing specific capabilities, but the data encountered in actual workflows may be much more varied.
For an issue-resolution system, a task might require interpreting an incomplete description, examining several pieces of evidence, considering multiple possible cases, and using tools to narrow down the answer.
A test that checks only whether the system can call a tool does not adequately represent that full process.
Yaoshen argues that real-world data can be a unique asset for each company. It reflects the specific problems its customers encounter, the information available to investigate those problems, and the practical requirements of its business.
Turning that data into a benchmark creates a way to test the system against problems that genuinely matter.
Customer behaviour plays a central role in this process. In Yaoshen's view, customers are generally the most important validators because their behaviour, within a particular context, helps reveal whether a problem has actually been solved.
A benchmark formalises those behaviours so they can be tested repeatedly.
The objective is not to replace every traditional benchmark. It is to connect capability testing with the real conditions under which an agent must deliver value.
Defining When an Agent Is Ready for Production
If real-world failures are inevitable, how should developers decide when an AI agent is good enough to deploy?
Yaoshen believes there is a threshold at which an agent becomes suitable for production. At the current stage of AI development, he suggests that the performance of human workers already performing the same job can provide a useful reference point.
He also draws on the economic concept of marginal benefit.
The practical question is whether the additional value provided by the agent justifies the remaining cost and risk.
This perspective avoids treating production readiness as a purely technical milestone.
An agent might perform impressively on a benchmark but still be unsuitable for a particular workflow if its mistakes are too costly, its operating expenses are too high, or its performance does not meet the requirements of the business.
Conversely, an agent does not necessarily need to eliminate every failure to be useful. Its suitability depends on the work being performed and the balance between the value it creates and the risks that remain.
This is why Yaoshen emphasises appropriate evaluation design, economic considerations, and stable, repeatable evaluation as three important directions for making agent evaluation practical.
AI Agents Require Continuous Evaluation, Not One-Time Testing
Traditional deterministic software is generally built around behaviour that developers expect to remain relatively predictable under defined conditions. AI-agent software presents a different challenge because its observed effectiveness must be monitored continuously.
Yaoshen considers the ability to understand how well an agent is performing over time one of the most important aspects of working with this type of software.
Deployment may also reveal capabilities that developers did not initially anticipate. Once an agent begins operating in real workflows, teams may discover that it can perform a broader range of tasks than expected.
That discovery can create opportunities, but it can also mean that existing workflows need to change fundamentally.
Developers therefore need to think beyond the initial implementation. Building an agent may be the beginning of an ongoing process in which the system, its evaluation criteria, and the surrounding business workflow evolve together.
This approach also helps explain why new failures should be added back into the benchmark. Every newly discovered problem can become a test case that helps the team understand the system more deeply and assess whether subsequent changes have improved its behaviour.
Human Responsibility Remains Central
As AI agents become more capable, they are also being asked to take on more work. But Yaoshen makes an important distinction: greater capability does not mean that responsibility has been transferred to the agent.
People remain responsible for the systems they build and deploy. In some circumstances, the amount of responsibility carried by individuals may even increase as agents take on more consequential tasks.
He also believes that the responsibility assigned to any individual should have limits.
This observation has practical implications for agent evaluation. If an agent is going to perform work previously handled by people, developers must understand not only whether it can complete that work but also where human judgment, oversight, or intervention remains necessary.
Human fallback mechanisms, such as those introduced in his issue-resolution workflow, are one way to account for cases the agent cannot reliably handle.
For Yaoshen, the future of AI agent evaluation will depend in part on designing appropriate evaluations, understanding the economic realities of deployment, and ensuring that performance can be assessed consistently and repeatedly.
These considerations become especially important as agents move into knowledge-work roles that have traditionally been performed by people without technical backgrounds.
Advice for Developers: Start With a Problem That Matters
For developers who want to begin building AI agents, Yaoshen recommends a practical starting point: build an agent on top of an existing large language model.
He believes that direct experimentation is the fastest way to develop a practical understanding of the technology.
When the goal is learning, he follows a principle that has also shaped his benchmark-building process: do the work manually first.
Before automating a task, it is useful to understand what the task actually involves. The experience of doing it yourself can reveal details and requirements that might otherwise remain hidden.
He is currently working on a series of hands-on projects built around this approach.
One example involves a friend who works in human resources. He coached her as she explored building something around a workflow that had been troubling her for a long time: reconciling accounts with the finance department and distributing monthly salaries to employees.
Her project focused on learning through vibe coding, but Yaoshen sees a broader lesson in the experience.
A real problem provides a concrete reason to learn. It gives the developer a goal, a way to recognise progress, and a practical environment in which to discover what the technology can and cannot do.
His advice is to avoid waiting until you feel sufficiently prepared to begin.
As he explains, people learn fastest through interaction with their environment. Building something around a genuine problem creates opportunities to test assumptions, encounter limitations, and develop a deeper understanding through practice.
That principle applies to both AI development and evaluation. The process becomes much more meaningful when learning is grounded in real work rather than hypothetical tasks alone.
Conclusion: Reliability Begins With Understanding Failure
Yaoshen Luo's approach to AI agent evaluation is shaped by a simple but demanding idea: a system must be judged against the work it is expected to perform.
His engineering background, experience with applied machine learning, and hands-on work with AI agents have led him to focus on the gap between demonstrated capability and practical reliability.
By reproducing real customer failures, building annotation tools, formalising evaluation criteria, and feeding newly discovered problems back into benchmarks, he has developed a practical way to investigate that gap.
His examples also show why no single benchmark score can tell the entire story. Real workflows introduce ambiguity, economic constraints, changing conditions, and failure patterns that may not appear in controlled tests.
For developers building AI agents, the lesson is to start with real problems, investigate failures carefully, and treat evaluation as an ongoing part of system development.
The objective is not simply to build agents that can succeed on a test. It is to understand when they work, where they fail, what their performance is worth, and when people still need to intervene.
As AI agents take on more responsibility in real-world workflows, that understanding will be essential to making them genuinely useful.
About Yaoshen Luo
Yaoshen Luo is a software engineer and AI practitioner based in Shanghai, China, whose work focuses on AI agent benchmarks, evaluation, and real-world workflows. He graduated in software engineering and joined Baidu in 2014, where he worked on Baidu Maps across front-end engineering, back-end engineering, algorithms, and location-based services.
His experience with applied machine learning and modern AI coding tools has informed his work on evaluating AI agents through real customer data, reproducible failures, and practical business requirements. His CoderLegion articles include How I Built a Voice Agent Benchmark from Real Customer Failures, How to Evaluate AI Agent with Benchmark in Practice?, Why AI Agent Benchmark Matters, and Why Good Agent Evaluation Wasn’t Enough for Production.
Developer Stories is a CoderLegion interview series highlighting the people, ideas, projects, and engineering decisions behind software development.
