DoorDash Launches DashBench to Rigorously Evaluate AI Code Review Accuracy Beyond Simple Metrics

DoorDash launches DashBench, a rigorous AI code review benchmark that replays historical pull requests to surface real developer findings, revealing that multi-stage architectures dramatically outperform single-pass models and that trustworthy AI evaluation demands triangulating multiple imperfect signals rather than relying on any one metric.

• DoorDash introduces DashBench, a measurement layer that replays historical pull requests to evaluate whether its AI code review agent surfaces real, human-actionable findings rather than relying on flawed signals like acceptance rates or single aggregate scores.
• The benchmark reveals that staged scout-plus-reviewer architectures significantly outperform single-pass baselines in coverage, with the top configuration finding 537 real findings at 65.2% weighted recall, while no single model mix dominates across all metrics including precision, recall, cost, and latency.
• DashBench establishes that trustworthy AI evaluation requires triangulating across multiple imperfect signals — human annotations, production feedback, and agentic judgment — because each source alone is unreliable, and a single metric obscures where and how a system actually fails.