Liquid AI Pits Claude Opus 4.5 Against GPT-5.2 in Real-World Coding Challenge — One Succeeds, One Fails

Liquid AI pits Claude Opus 4.5 against GPT-5.2 in a real-world coding challenge to build a production-grade tokenizer trainer, and while both AI agents quickly produce basic working versions, only Claude Opus 4.5 ultimately succeeds at full scale after multiple iterations, exposing critical lessons about how engineers should write AI specifications and design verification loops using real production data.

• In late 2025, Liquid AI tasks two coding agents powered by Claude Opus 4.5 and Codex with GPT-5.2 with building a production-grade BPE tokenizer trainer called toktoktok, capable of processing trillions of tokens on a single machine to fill a gap left by existing tools like sentencepiece and Hugging Face tokenizers.
• Both agents quickly produce working toy trainers but fail to scale to production without iterative loops, uncovering critical issues such as memory overflow, improper parallelization, and encoding bugs that only surface on real, full-scale data — with Claude Opus 4.5 ultimately succeeding after multiple iterations while the GPT-5.2 track is halted due to persistent throughput issues.
• Two key lessons emerge and are now standard practice for the engineering team: write short, outcome-focused specifications that leverage agents' multi-domain expertise rather than detailing implementation steps, and design iteration loops that verify results against real production data using external, agent-proof harnesses to ensure genuine, unmanipulated correctness.