The bottleneck in agentic engineering isn't intelligence. It's proof.

I haven’t written code by hand in two years. Over a thousand PRs I’ve shipped to production were written by agents I directed - and the one thing they still can’t do reliably is prove those PRs are correct.

Writing code is largely solved. For most changes, LLMs are faster than any human and at least as good. This may have been controversial a year ago. For anyone using frontier models in 2026, it's obvious.

Coding agents like Claude Code, Codex and others turn a ticket, docs and a prompt into large, fully functional changes spanning thousands of lines across hundreds of files. Automated adversarial AI review from a different model, auto-addressing human and AI review comments and merging a PR only when CI is green are nothing new either.

Nearly every stage of the loop is easily automated except one: proving the change is right. I call that stage validation - checking that a change does what was intended, meets all requirements, and breaks nothing else.

Writing tests is automated too - and most unit tests are now close to worthless anyway. Frontier models don't make function-scope mistakes; the bugs that survive live between components, and only end-to-end tests see those. An end-to-end test is only as good as what surrounds it: a real environment to run in, a judge to render the verdict, a contract to check against. Writing the test was never the hard part.

Validation is where the work lives now.

Validation is task-dependent

The main reason validation hasn't been automated away is that it's very hard to generalize. The way we validate work depends strictly on the nature of the work itself.

Testing and validating performance tuning of a database is completely different from making UI changes. For the first, you need to reproduce production-like data volumes and load, benchmark before and after the change, and read the query plans to confirm the improvement comes from where you think it does. The second involves rendering changes in a browser and doing a visual assessment, usually on top of mechanical tests through frameworks like Selenium or Playwright.

A bug fix in business logic is different again - validation means first reproducing the bug, then showing it's gone - and that nothing else broke. A change to an agentic system means something stranger: there's no pass/fail, only evals - replaying scenarios and scoring whole trajectories against a baseline to decide whether behavior got better or worse.

Contrast that with the coding side: LLMs are inherently universal - they’re capable of writing arbitrary code, whether it’s bare-metal C++ or frontend TypeScript. There isn’t a single universal procedure, nor framework to run arbitrary validations - and that’s where a big chunk of work remains. In other words, we’re dealing with universal writers, but we lack universal verifiers.

No verification, no trust. No trust, no closed loop. No closed loop, and the human is still the slowest component in the pipeline.

Four aspects of validation

There's no established framework for what validation actually consists of, so what follows is my own decomposition - one of many possible cuts, but the one that felt most natural in my mental model. There are other parts I'm leaving out - deciding how much validation a change deserves, what happens on escalation - but these four are where the work concentrates. The four aspects, in roughly the order they come into play:

Contract - a set of requirements, stated in a form that is checkable - something a verdict can be rendered against.

How formal a contract needs to be depends on the task, but ideally, it should not leave any room for interpretation and it’s written for a reader that won’t need to fill gaps with judgment - an AI agent. The contract is where intent stops being fuzzy - everything upstream is ideation and product thinking, everything downstream is mechanical execution.

Environment - the infrastructure where checks can actually run.

It takes many forms: a laptop running unit tests, an ephemeral Kubernetes cluster mimicking a real deployment, a browser with screenshot tooling, production-scale data volumes for benchmarks. Whatever the contract demands, the environment is what makes checking it physically possible.

Judge - whatever renders the verdict: a way of measuring whether the contract was fulfilled.

A judge can be as dumb as an exit code or as expensive as a human opinion - mechanical at one end, subjective at the other: green CI and benchmark deltas at the cheap end; screenshot assessment, eval scoring, human eyes at the other. (The software testing literature calls this the 'test oracle'.) The judge is often the missing piece even when the contract is clear - 'no visual regressions' is a perfectly clear requirement, and nothing mechanical can check it.

Evidence - an artifact that makes the verdict portable. It records what was checked and what the results were: benchmarks, screenshots, test reports, trajectory scores.

It should be attached to the change proposal (PR) and be easy for a human reviewer to understand - in my pipeline it's an HTML artifact, concise enough to assess the change without reading the code. It's catching on: more teams I talk to are shipping some version of this.

Evidence matters most while trust is being established; once a loop has enough clean runs behind it, reviewing evidence becomes sampling rather than gating.

In the ideal loop, humans touch validation at exactly two points: writing the contract at the start, reviewing the evidence at the end.

Designing ‘done’

Agentic Driven Development looks like the final form of TDD: writing the test first was always about designing 'done' before designing the solution. Now that AI is able to creatively solve complex engineering problems, the task specification becomes the contract the agent validates against. Vague criteria do not produce vague code anymore - instead they produce well-written, confidently wrong code that passes its own tests.

In practice, contracts are hard. I think of them as test suites at a higher level of abstraction: just as writing a good low-level test suite required thought and intention, so does writing a good contract. Formalization fails on taste ('make it look right'), fuzzy targets ('make it faster') and unbounded scope ('don't break anything').

Vague intent hides the real contract: when a nuanced goal gets reduced to a short ticket, unstated constraints and expectations are lost with it. The ticket says 'improve performance' but the actual contract is 'bring p99 under 200ms without hurting write throughput or storage costs'. Closing that gap is real work - and when it isn't closed, the loop can't converge: the agent either iterates against the wrong 'done' or escalates back to a human.

The missing environment

A concrete example from my own work. The product I work on connects to dozens of third-party systems, and building a new connector is exactly the split this post describes: writing the code is solved, validating it is not.

Real validation requires making actual calls to the third-party API - and we can't replicate every vendor's account, configuration and data internally.

For a long time we leaned on unit tests, which were comprehensive and green - yet bugs kept surfacing only in real environments, reported by customers. The contract was fine and the judge was fine; the environment was missing.

So we built it: unmerged code runs locally, but every call goes through a real account via a proxy that enforces read-only access and auditing, with credentials and data never leaving the customer's boundary.

Now a coding agent runs the end-to-end validation hands-off and attaches the evidence. Building a new connector dropped from a couple of days to a couple of hours.

Validation compounds

Building validation for a task-type is high-leverage work. Build it once and every future change of that type gets cheap trust. Screenshot diffs for UI, benchmark suites for perf tuning, integration tests for core business logic, evals for the agent harness. The catalog compounds. Each entry in the catalog, over time, moves a domain closer to a ‘dark factory’: software changes produced end to end with no human in the loop. We're not there yet. The orange boxes on the first diagram won't turn black until validation is battle-tested - until then, each one keeps a human in it.

A big part of engineering work used to be translating requirements into code. That part is mostly gone and it's been replaced by translating vague intent and high-level ideas into well-defined contracts that an agent can fulfill and verify.

The job now is setting an AI agent up for success: curating context, deciding what to build, and closing engineering loops by designing and building validations. A closed loop stops consuming human attention, and depending on the domain, that compounds enormously.

How far this goes, I don't know - some domains will close soon, others may keep a human in them for a long time. The answer will come from building validations, domain by domain.

Which domain have you managed to close - and which one refuses?