Test-driven development assumes a stable spec and a deterministic pass or fail. Agentic AI has neither. A grounded look at Evaluation-Driven Development (EDD): what it actually proposes, what the research confirms, and the honest gap between the theory and how evals are practiced today.
A few months into building an agent that triages support tickets, an engineer on our team wrote what looked like a perfectly ordinary unit test: call the agent with a sample ticket, assert that the output equals the expected routing decision. It passed on Monday. It failed on Tuesday, on the exact same input, with no code changes in between. Nobody had touched the model, the prompt, or the ticket. The test was written correctly. The assumption underneath it was not.
That assumption is the entire foundation of test-driven development: write a test against a stable specification, run it once, get a deterministic verdict of pass or fail, ship. It is a foundation that has run software engineering for two decades, and it works beautifully for a sorting function or a payment calculation. It does not survive contact with a probabilistic system that is being asked to behave like a deterministic one.
This is the real story behind the "TDD vs EDD" debate showing up in agent engineering blogs and academic papers over the past year. It is not a fad or a rebrand of testing with a fancier name. It is a genuine argument that the unit of correctness for an AI agent is different from the unit of correctness for a function, and that the discipline built around the old unit does not transfer cleanly to the new one.
Test-driven development, in its classic red-green-refactor form, quietly leans on three things being true:
An LLM-based agent violates all three. Its "specification" is often an under-specified goal in natural language ("resolve the customer's issue") rather than a fixed contract. Its behavior is non-deterministic, even at a temperature of zero, because of floating-point non-associativity, batching effects, and model updates outside your control. And its correctness is not a one-time question, because the agent keeps encountering situations after deployment that no pre-release test suite anticipated, and in many systems it keeps adapting based on that experience.
This is not an informal complaint. It is the explicit framing used in the most cited academic treatment of the problem, a 2024 CSIRO paper that proposes a formal alternative: traditional testing "assumes stable specifications, executable oracles, and predominantly pre-deployment test phases," while LLM agents "are non-deterministic, pursue under-specified goals, and continue to adapt after deployment." A 2025 ACM SIGKDD survey on evaluating LLM agents makes the same point independently: software testing is built for "deterministic and static behavior," while agents are "inherently probabilistic and behave dynamically." Two different research groups, working separately, converged on the same diagnosis.
The response gaining traction in both research and practice is Evaluation-Driven Development, usually shortened to EDD (the CSIRO paper calls its fuller version EDDOps, for evaluation-driven development and operations). The core move is simple to state and harder to build: instead of treating evaluation as a gate you pass through once before shipping, you treat it as a continuous, governing function that runs across the entire lifecycle of the agent, before release and after it, feeding what it learns back into the next iteration.
Two things are worth being precise about here, because they are easy to get wrong.
First, EDD is explicitly framed as a complement to TDD and BDD, not a replacement for them. The CSIRO paper states plainly that EDDOps "builds on the iterative principles of TDD and BDD while adapting them" for agentic systems. It is not "throw out your test suite." It is "your test suite was built for a different kind of correctness problem, and you need a second discipline for this one."
Second, EDD is not "run more evals." It specifically means unifying two loops that most teams currently run separately, if they run them at all: offline evaluation, done against curated cases before a release, and online evaluation, done against real production traffic after release. In EDD those two are meant to feed into one closed loop, rather than living in different tools, owned by different teams, on different schedules.
The academic framing is useful, but it would be easy to dismiss as ivory-tower reshuffling if it did not match what teams actually run into. Anthropic's own engineering team, writing about building evals for their agents, makes three claims that line up closely with the EDD argument, from the opposite direction of practice rather than theory.
Without automated evals, they note, debugging an agent becomes reactive: wait for a user complaint, try to reproduce it by hand, patch it, and hope the fix did not quietly break something else. Agent evaluation is also a genuinely different problem from single-turn testing, because agents operate over many turns, call tools, modify state, and adapt their next move based on what just happened, none of which a single input-output assertion can capture. And critically, they do not trust an eval score at face value until someone has actually read the underlying transcripts, because a high score can hide a shortcut the eval happened not to check for.
That last point deserves to be sat with, because it draws the sharpest line back to TDD. Owning and maintaining an eval suite, on this account, is not a one-time investment you build and forget. It has to be treated with the same ongoing discipline that a healthy team already applies to unit tests: something a team member reviews, extends, and occasionally rewrites when it turns out to be measuring the wrong thing. The tooling changed. The discipline of taking your own tests seriously did not.
There is a second, more informal data point worth including with appropriate caution: Andrew Ng has been quoted arguing that the single biggest predictor of how fast a team makes progress building an AI agent is whether they run a disciplined process for evals and error analysis. It is a strong claim, and it comes through a blog quoting a social media post rather than a peer-reviewed source, so it belongs here as an informed practitioner's opinion, not as settled evidence. But it is consistent with everything above it: velocity, not just correctness, is the thing evals are being asked to protect.
Here is where the story gets less tidy, and where an article like this one owes you the uncomfortable part rather than just the compelling part.
A Stanford-authored systematic review of 84 agentic AI evaluation papers published between 2023 and 2025, not yet peer-reviewed at the time of writing, found that current evaluation practice is heavily lopsided. Technical metrics show up in 83% of the papers reviewed. Safety metrics show up in 53%. Human-centered and economic dimensions each show up in only 30%. Put differently: the field talks a good game about evaluation being the answer, but most of what actually gets measured is still the easy, quantifiable, model-centric slice of the problem.
The same review ties this to real incidents, not hypothetical ones: agentic systems in healthcare, finance, and retail that performed well on the technical benchmarks they were measured against, and still failed once deployed, because of human, temporal, and contextual factors those benchmarks never captured. It cites, among others, the McDonald's and IBM drive-through AI project that was shut down after repeated public failures, DPD's chatbot going viral for insulting a customer, New York City's MyCity chatbot giving small businesses illegal advice, and the Air Canada case where a company was held liable for its chatbot's incorrect promise to a customer.
None of these are arguments against EDD as a concept. If anything, they are the argument for taking the "continuous, whole-lifecycle" part of EDD literally rather than as a slogan; a technically strong offline eval suite that never gets matched with a serious online, human-centered, or safety-focused counterpart is exactly the failure mode the CSIRO paper's own definition was written to prevent. But it means the honest current state of the field is: the theory of EDD is fairly well settled, and the practice of it is still catching up, unevenly, with technical metrics far ahead of the human and safety metrics that determine whether an agent survives contact with real customers.
None of this means TDD is obsolete inside an agentic system. Most production agents are not pure model calls; they are a mix of deterministic scaffolding, orchestration code, tool integrations, and structured output parsing, wrapped around a genuinely non-deterministic core. The scaffolding is still ordinary software. It still deserves ordinary unit tests, and TDD still works on it exactly as well as it always did.
The dividing line worth drawing is this: anything you can specify with a fixed input and a fixed correct output stays under TDD. Anything where correctness depends on judgment, on the appropriateness of a response to an open-ended situation, or on behavior that only shows up across a multi-turn interaction, moves to evals. Most real agents live on both sides of that line at once, which is exactly why EDD is framed as a complement rather than a replacement. You do not choose one discipline. You draw the boundary and run both.
TDD assumes a stable spec, a deterministic answer, and a finish line at deployment. Agentic AI has none of the three, which is why treating an agent's output like a function's return value produces tests that pass on Monday and fail on Tuesday for no reason anyone can find.
Evaluation-Driven Development answers this by making evaluation continuous instead of terminal, looping offline and online feedback across the whole lifecycle instead of stopping at release. The research case for that shift is solid. The practice is not there yet: most published evaluations are still technical-metric-heavy, and real deployments have failed in exactly the human and safety dimensions those metrics skip. Build the loop, but do not mistake a green eval dashboard for a safe one until you have read the transcripts and checked what it is not measuring.