ACL 2025 · Volume 1: Long Papers · pages 23411–23432
Language Models Resist Alignment: Evidence From Data Compression
TL;DR: LLMs exhibit alignment elasticity (resistance + rebound): alignment is shallow and compressible; inverse alignment with far less data can undo it — explained via compression / Theorem 4.2.
Large language models may exhibit unintended behaviors even after alignment. Empirically, post-alignment models tend to revert toward the pretraining behavior distribution upon further fine-tuning (elasticity). Leveraging compression theory, the authors deduce that fine-tuning disproportionately undermines alignment relative to pretraining, potentially by orders of magnitude. Experiments across model types and scales show rapid performance decline before reverting to the pretraining distribution, with elasticity positively correlated with model size and pretraining data volume.
