This paper addresses the limitations of existing structured pruning methods that assume prunable units are independent, a premise often broken by non-orthogonal weights and correlated activations. The authors propose a method that formulates pruning as a cardinality-constrained binary quadratic program to explicitly model cross-unit dependencies in reconstruction error. This approach aims to reduce inference costs for large language models while maintaining higher accuracy than isolation-based techniques.
- Moves beyond independent unit importance scoring to capture activation correlations.
- Uses a binary quadratic program to optimize pruning decisions jointly.
- Targets hardware-efficient inference with reduced performance degradation.
- Challenges the additive error assumption common in current pruning algorithms.
