Salesforce AI Releases Random Attention, a Signal-Free KV-Cache Tool That Outpaces Leading AI Memory Selectors

Salesforce AI Research unveils Random Attention, a surprisingly simple yet powerful KV-cache eviction tool that matches or beats leading AI memory selectors on major benchmarks — without ever reading attention scores or requiring calibration data — while also being the fastest option available in popular AI serving stacks.

• Salesforce AI Research releases Random Attention, a signal-free KV-cache eviction policy for reasoning models that keeps the prompt and a uniformly random per-KV-head subset of generated tokens without reading attention scores, value statistics, or calibration data.
• On benchmarks including MATH-500, GPQA-Diamond, AIME, HMMT, and LiveCodeBench, Random Attention matches or beats learned selectors like SnapKV, R-KV, VaSE, and TriAttention on Qwen3 and Phi-4-reasoning models at matched budgets, while also being the fastest evictor in both Hugging Face and vLLM serving stacks.
• The open-source repository includes the full eviction engine, evaluation harness, significance tests, efficiency benchmarks, a vLLM port, and mechanism-study tooling, all released under the Apache License 2.0.