Google Cloud Exposes Fatal Flaws in AI Benchmarks: A Single Word Change Can Crash Agent Performance to Zero

Google Cloud's new Discovery Bench framework exposes alarming flaws in AI benchmarking, revealing that a single vague word can crash an agent's performance score from perfect to zero, while also uncovering broken ground truth and mislabeled data in widely trusted benchmarks — raising urgent questions about whether the AI industry has been measuring intelligence correctly all along.

• Google Cloud's frontier AI team introduces Discovery Bench, a meta-benchmarking framework that uses information theory and a process called iterative surprisal-based query refinement (iSQR) to measure AI agent performance across calibrated levels of query ambiguity, revealing capability cliffs that traditional pass/fail benchmarks completely miss.
• Testing reveals that a single vague word change can drop an agent's retrieval score from a perfect F1 of 1.00 to 0.00, exposing sharp performance cliffs invisible to static benchmarks, while also uncovering that more specificity is not always better, as medium ambiguity sometimes outperforms low ambiguity for certain agents.
• A critical finding emerges that widely trusted benchmarks, including kramabench-astronomy, contain broken ground truth, mislabeled data, and structurally flawed evaluation cases, underscoring an urgent industry need to rigorously evaluate the evaluations themselves rather than blindly trusting existing benchmarks as reliable measuring sticks.