The Influence of Fine-Tuning on Design Space Exploration in Joint Pruning-Quantization

Abstract

Joint pruning-quantization is now a widely adopted method for reducing complexity of deep neural networks. Specifically, layer-wise compression has achieved significant success by exploiting each layer’s varying sensitivity to pruning and quantization. Although effective at reducing complexity with minimal accuracy loss, this approach significantly complicates the design space exploration. Prior works estimate the order of complexity of the design space based on the number of pruning ratios, quantization bitwidth for weights and activations, layers, model variations, and hardware targets. However, none have considered the number of fine-tuning epochs required after joint pruning-quantization. This work hypothesizes that fine-tuning is often excluded from design space estimates because accuracy gains after the first epoch are minimal. To validate this, we identify layers in ResNet-20 that are highly sensitive to compression and evaluate their behavior under varying pruning ratios, bitwidth, and fine-tuning epochs. Results confirm that epochs beyond the first only provide marginal improvements, although fine-tuning is crucial for accuracy recovery. These findings support treating fine-tuning as a binary decision (applied or not) rather than a continuous design parameter, thus justifying its exclusion from the order of complexity in design space estimation.