The Problem with Traditional Pruning in Reasoning Models

Standard model pruning techniques typically rely on minimizing the divergence between the original model's output distribution and the pruned model's distribution (e.g., KL-divergence). While effective for general-purpose language models, this approach often fails for reasoning-heavy models. Reasoning models rely on complex, multi-step chains of thought where the intermediate tokens are highly sensitive; traditional pruning often disrupts these chains, leading to a degradation in final answer accuracy even if the token-level perplexity remains relatively stable.

Outcome-Based Calibration (OBC)

OBC-Prune shifts the focus from token-level probability matching to outcome-based calibration. Instead of forcing the pruned model to mimic the exact probability distribution of the original model at every step, the method calibrates the pruning process based on the model's ability to reach the correct final answer. By evaluating the model's performance on reasoning-intensive datasets, the algorithm identifies and preserves the specific weights and attention heads that are most critical to the model's final reasoning success. This ensures that the pruned model maintains its problem-solving capability, even if the internal path taken to reach the solution deviates slightly from the original model's path.

Performance and Trade-offs

The research demonstrates that OBC-Prune allows for significant model compression—reducing parameter counts while maintaining higher accuracy on reasoning benchmarks compared to standard magnitude-based or distribution-matching pruning methods. The primary trade-off is the increased computational cost during the calibration phase, as it requires evaluating the model's reasoning outcomes rather than just its next-token predictions. However, for developers looking to deploy large reasoning models on resource-constrained hardware, this method provides a more robust path to maintaining high-level performance without the full memory footprint of the original model.