大規模言語モデルの逐次ポストトレーニングにおける表現崩壊 Representation Collapse in Sequential Post-Training of Large Language Models

Large language models are increasingly refined not through a single training pass, but through chains of post-training stages — instruction tuning, reinforcement learning from human feedback, safety alignment, domain adaptation, and more. Each stage is meant to add capability or constraint, but this paper raises a fundamental concern: stacking these stages sequentially may silently degrade the very internal representations that make the model capable in the first place.

The phenomenon the authors investigate is called representation collapse. Unlike catastrophic forgetting, which typically refers to the loss of previously learned knowledge at the parameter level, representation collapse operates at a more geometric level — the model's hidden states lose the diversity and structure needed to represent a rich semantic space. The result can manifest as homogenized outputs, reduced generalization, or subtle regressions in capability that are hard to pin down through standard benchmarks.

The distinction from single-pass instruction tuning is important. When a base model is fine-tuned in one continuous stage, the representation space tends to remain coherent. But when multiple independent objectives are applied sequentially, each stage can warp the geometry left by the previous one, compounding distortion in ways that are difficult to detect or reverse.

This connects to a broader body of research in continual learning, where the challenge of preserving prior knowledge while acquiring new skills has been studied extensively. Techniques like elastic weight consolidation, experience replay, and regularization-based approaches have been proposed in that context. Whether similar strategies translate effectively to the LLM post-training setting remains an open and practically urgent question.

The stakes are real for practitioners. Organizations routinely apply sequential post-training to adapt foundation models — adding domain-specific knowledge in one pass, safety guardrails in another, multilingual capability in a third. Major AI labs including OpenAI, Anthropic, and Google DeepMind use multi-stage alignment pipelines, and the integrity of the model's internal representations across those stages is not always rigorously evaluated.

By framing representation collapse as a measurable and analyzable failure mode, this paper could provide a useful lens for auditing multi-stage training pipelines. The findings may also motivate new design choices around training order, intermediate checkpointing, and regularization strategies that explicitly preserve representational diversity.

As LLM development cycles grow more complex and post-training becomes a modular, composable process, maintaining representational health across stages is likely to become a first-class engineering concern. This paper appears to be an early but substantive contribution to that effort.