Small AI Model Beats GPT-5 and Claude on SQL Benchmark at a Fraction of the Cost
Summary
A small AI model called ReViSQL-K2.6, trained with reinforcement learning on a cleaned dataset, achieves human-level 92.97% accuracy on a text-to-SQL benchmark, outperforming GPT-5 and Claude at just 12–15% of their cost.
Key Points
- Researchers fine-tune a model called ReViSQL-K2.6 using reinforcement learning with verifiable rewards (RLVR) on a cleaned dataset called BIRD-Platinum, achieving human-level accuracy of 92.97% on the text-to-SQL benchmark Arcwise-Plat-SQL without relying on complex agentic scaffolding.
- A major breakthrough comes from purging massive annotation errors found in standard training data, with over 61% of sampled BIRD Train instances containing at least one error, and from improving the reward function to address false-positive execution matches and models ignoring externally provided knowledge.
- ReViSQL-K2.6 outperforms frontier models like GPT-5.6 Sol Ultra and Claude Fable 5 at 12–15% of their cost, demonstrating that embedding task expertise directly into model training via verified data and precise reward shaping scales better than adding scaffolding around a fixed model.