Base LLMs Show Strong Semantic Confidence Accuracy, But Fine-Tuning and Chain-of-Thought Reasoning Destroy It
New research reveals that base large language models possess strong semantic confidence accuracy, but popular techniques like fine-tuning and chain-of-thought reasoning actively destroy this calibration, raising urgent questions about the reliability of widely deployed AI systems.