Hyena Edge: Liquid AI's Efficient Convolution Model Outpaces Transformers on Smartphones
Summary
Hyena Edge, a convolution-based AI model designed by Liquid AI using automated optimization, outperforms Transformer models in computational efficiency and quality on smartphones, demonstrating faster speed, lower memory usage, and better language modeling performance on a Samsung device.
Key Points
- Hyena Edge, a convolution-based multi-hybrid model, outperforms strong Transformer-based baselines in computational efficiency and model quality on edge hardware like smartphones.
- Liquid AI used its automated architecture optimization framework STAR to design Hyena Edge, favoring Hyena-Y convolutions for their efficiency-quality balance.
- Hyena Edge demonstrated faster latency, lower memory usage, and better language modeling performance compared to a parameter-matched GQA-Transformer++ baseline on a Samsung S24 Ultra smartphone.