Already 6 months old when the scout caught it.
Eval engineering
What it is
Eval engineering is the discipline of designing, automating, and versioning the verification side of an agent system — building eval suites (curated datasets, scoring rubrics, LLM-judge or deterministic checks) as production infrastructure that gates agent behavior in CI, rather than judging outputs by eye. Where a single verifier checks one loop's output against a rubric, eval engineering is the practice across many loops: maintaining the datasets and gates that tell you whether the whole system is getting better or worse as prompts, tools, and models change underneath it.
Why it's emerging now
As teams moved from prototyping agents to running them in production, informal spot-checking of outputs stopped scaling — a prompt or model change that fixes one failure mode can silently regress another, and by the time it surfaces in production the cost of catching it has multiplied. The pattern crystallized through 2026 as agent teams that had already adopted loop engineering (designing the loop an agent runs) hit its natural bottleneck: the verifier inside the loop. Vendors built whole tooling categories around it — Splunk (via its Galileo acquisition) and FutureAGI both published dedicated eval-engineering guides and platforms over the spring and summer, and Andrew Ng has folded "evaluation-driven development" into his core AI-engineering curriculum.
How it relates to what you know
It's the direct sibling of loop engineering, already on your ledger: if loop engineering designs the loop an agent runs, eval engineering designs the bar that loop has to clear. It also formalizes LLM-as-judge and the older, more general notion of evals/benchmarks into a versioned, CI-gated engineering practice — the same move that turned ad hoc prompting into prompt engineering, and ad hoc context management into context engineering.