MIT Study Reveals Medical AI Models Fail on Up to 75% of New Data Despite Strong Overall Performance
MIT researchers discover that medical AI models appearing highly effective can catastrophically fail on up to 75% of new data when deployed in different hospital settings, with chest X-ray diagnostic systems missing critical conditions like pleural diseases due to spurious correlations that aren't caught by standard performance metrics.