Meta unveils MTTT: AI learns to learn at test time, surpassing linear attention
Summary
Meta unveils a groundbreaking AI framework, MTTT, that enables models to learn how to learn at test time, removing the need for human-designed self-supervised tasks and outperforming linear attention on long sequences while maintaining linear complexity.
Key Points
- Meta researchers develop a new AI framework called MTTT that allows models to learn how to learn at test time
- MTTT removes human ingenuity from designing self-supervised tasks by parameterizing the task itself
- MTTT with an MLP feature extractor outperforms linear attention on long sequences while retaining linear complexity