Double-blind AI evaluations
What it is
Double-blind AI evaluation is a testing methodology, piloted by Google DeepMind, that keeps a proprietary model and an external evaluator's benchmark mutually invisible during testing. Using confidential computing (Google Cloud's Confidential Space), the evaluator's test prompts run against the model inside a sealed enclave: the evaluator never sees the model weights, and the model's operator never sees the questions. Both sides get cryptographic proof the test ran as claimed, without either exposing their IP to the other.
Why it's emerging now
Benchmark contamination — where a model or its developer had prior access to test questions, inflating scores — has become a credibility problem for AI evaluation generally, and existing fixes (private held-out test sets, third-party auditors) still require trusting someone with either the model or the questions. DeepMind's pilot, run with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons against a Gemini Flash Lite model, is an attempt to remove that trust requirement using confidential-computing infrastructure that's only recently become practical to apply this way.
How it relates to what you know
It's a mutual-distrust variant of the same problem LLM-as-judge and evals/benchmarks already sit inside — how to get a trustworthy score out of a system that has an incentive to look good on the test. Where guardrails and red teaming police a model's behavior, and model cards disclose known limitations, double-blind evaluation is aimed one layer earlier: making the test itself tamper-proof for both parties before either behavior or disclosure comes into play.