Abstract
Despite alignment efforts, anomalies show that even a well-conducted alignment process can be easily circumvented. We demonstrate the elasticity of post-alignment models — the tendency to revert to the behavior distribution formed during pre-training upon further fine-tuning. Leveraging compression theory, we deduce that fine-tuning disproportionately undermines alignment relative to pre-training, potentially by orders of magnitude.
① 段落功能：提出核心主张与理论承诺
② 逻辑角色：全文 thesis 句：弹性 + 压缩推导
③ 论证技巧或潜在漏洞：「orders of magnitude」依赖 $k\gg1$ 渐近，易被读者字面化为实测倍数——属修辞张力点。
Introduction
Training-based alignment (SFT, RLHF, and variants) is dominant. Yet highly safety-aligned models can be made unsafe again with very little fine-tuning, and even non-malicious fine-tuning can weaken safety. The authors ask: Why is alignment so fragile?
① 段落功能：建立问题紧迫性
② 逻辑角色：从现象到研究问题
③ 论证技巧或潜在漏洞：引用链（Qi et al. 等）支撑「脆弱」前提；本文贡献在机制而非再列案例。
We define inverse alignment: after aligning on $D_a$, a much smaller $D_b$ can pull the model back toward pre-training behavior. Elasticity = resistance + rebound.
① 段落功能：引入术语系统
② 逻辑角色：定义层，为后文实验协议铺路
③ 论证技巧或潜在漏洞：概念密度高；阅读时应分开记住两个操作化协议，避免混用。
Methods / Theory
Treat the LLM as a lossless compressor. The compression rate $\gamma$ proxies training/alignment loss. Normalized rate $\gamma^{{D_i/D}}=\gamma^{{D_i}}-\log M$ enables cross-dataset comparison. Under Pareto mass (Assumption A.7), Theorem 4.2 relates derivatives of normalized rates for alignment vs pre-training sets via factor $k=|D_p|/|D_a|$.
① 段落功能：给出形式机制
② 逻辑角色：方法论核心
③ 论证技巧或潜在漏洞：Pareto 与 $D_a\sim$ 预训练子集是主要可攻击假设（作者 Limitations 也承认）。
Spring analogy: $|D_i|$ ↔ stiffness $k$; $|\Delta\gamma|$ ↔ deformation. Large pre-training sets act as hard springs dominating equilibrium — hence inverse alignment.
① 段落功能：直觉转译定理
② 逻辑角色：类比论证
③ 论证技巧或潜在漏洞：类比有助传播，但不等价于证明；需与 Theorem 4.2 对照读。
Results
Table 1: across Alpaca / Truthful / Safe and Llama2-7B/13B, Llama3-8B, inverse loss is always lower than forward loss. Example Alpaca Llama2-7B $\theta_2\to\theta_1$: 0.1589 vs 0.2018.
① 段落功能：报告 Resistance 主结果
② 逻辑角色：证据支撑抗性
③ 论证技巧或潜在漏洞：协议为 1-epoch SFT 切片；外推到任意指令分布需谨慎。
Figure 4: more positive alignment data → faster drop under smaller negative fine-tuning; curves are steep then flatten (steep=rebound, flat=resistance). Figures 5–6: larger models and more pre-training tokens → stronger rebound. Figure 7: PPO/DPO/KTO/SimPO all rebound.
① 段落功能：报告 Rebound 与规模/算法普适
② 逻辑角色：证据三角：任务曲线 + 规模 + 算法消融
③ 论证技巧或潜在漏洞：趋势图无统一弹性标量；规模相关可能混杂容量与优化动力学。
Discussion / Limitations
Authors acknowledge: Pareto assumption; no full pretrain–align lifecycle sweep; multimodal generality and quantitative elasticity–scaling laws remain open. Compression rate / task score / KL are proxies, not ground-truth safety.
① 段落功能：自我设限
② 逻辑角色：concession 处理反驳
③ 论证技巧或潜在漏洞：诚实，但缓解方案（fine-tuning-robust alignment）仍偏愿景。
Conclusion
Language models possess elasticity and thus inherently resist alignment. Addressing this intrinsic elasticity is necessary for more robust / deeper alignment.
① 段落功能：收束主张
② 逻辑角色：规范结论句
③ 论证技巧或潜在漏洞：强规范性结尾；是否「inherent」取决于假设是否成立。