Perplexity Engineering Launches Q2D-Web, a Massive Multilingual Benchmark to Evaluate AI Retrieval Models at Scale

Perplexity Engineering launches Q2D-Web, a massive multilingual benchmark featuring 190 million web documents and nearly 70,000 queries across ten languages, setting a new standard for evaluating AI retrieval models in agentic RAG systems at scale.

• Perplexity Engineering introduces Q2D-Web, a large-scale private benchmark and public leaderboard designed to evaluate first-stage retrieval models in agentic RAG systems, featuring 190 million web documents and 69,721 agent-reformulated queries across ten languages.
• To address the limitations of existing benchmarks, Q2D-Web provides three distinct relevance judgment sets derived from agent citations, production web rankings, and LLM-judged pairs, averaging 99.6 positive relevance judgments per query to reduce false negatives and single-source bias.
• Evaluation results show no single model dominates across all judgment sets, with pplx-embed-v1-4b leading on Combined Recall@1000 and Nemotron-3-Embed-8B topping Citation Recall@1000, while a cost-saving RRF-based subsampling strategy reduces full-corpus evaluation costs by approximately two-thirds.