Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions

Layer pruning efficiently reduces Large Language Model (LLM) computational costs but often triggers sudden performance collapse. Existing representation-based analyses struggle to explain this mechanism. We propose studying pruning through decision representation. Focusing on multiple-choice tasks, we introduce two metrics, Decision Margin and Option Frequency, and an Iterative Pruning method to analyze layer-wise decision dynamics. Our findings reveal a sharp decision transition that partitions the network into two stages: a Silent Phase, where the model cannot yet predict the correct answer, and a Decisive Phase, where the correct prediction emerges. We also find that pruning the Decisive Phase has minimal impact, whereas pruning the Silent Phase triggers immediate performance collapse, highlighting its extreme sensitivity to structural changes. Therefore, we conclude that pruning-induced collapse stems from disrupting the Silent Phase, which prevents the critical decision transition from occurring.

Modern AI systems such as chatbots can be expensive to run because they contain many internal processing steps. A common way to make them faster is to remove some of these steps, but this sometimes works well at first and then suddenly makes the system fail. We studied why this happens. Instead of only checking whether the system still stores similar information inside, we tracked when it actually becomes able to choose the correct answer in multiple-choice questions. We found that the system goes through two stages: an early stage where it has not yet formed a reliable answer, and a later stage where the correct answer becomes stable. Removing steps from the later stage often causes little damage, but removing steps from the early stage can stop the system from ever reaching the point where it can make a good choice. This explains why making these AI systems smaller can fail so abruptly. Our findings suggest that safer ways to shrink such systems should protect the early steps that prepare them to make decisions, instead of treating all removable parts as equally safe.