X Open-Sources 'For You' Feed Algorithm Powered by Grok Transformer Model
Summary
X open-sources its 'For You' feed algorithm powered by Grok transformer technology, revealing how the platform combines posts from followed accounts with AI-discovered content by predicting user engagement probabilities across likes, replies, reposts, and clicks.
Key Points
- X releases open-source code for its 'For You' feed algorithm powered by a Grok-based transformer model that combines in-network posts from followed accounts with out-of-network content discovered through ML-based retrieval
- The system eliminates hand-engineered features and uses the transformer to predict engagement probabilities for multiple actions (likes, replies, reposts, clicks) based on user engagement history, with final scores calculated as weighted combinations of these predictions
- The algorithm operates through four main components: Home Mixer for orchestration, Thunder for real-time in-network post storage, Phoenix for ML-based retrieval and ranking, and Candidate Pipeline as the framework connecting all stages from sourcing to filtering and scoring