深度學習因其在許多人工智慧應用中取得的巨大成功而引起了學術界與工業界的極大關注。隨著越來越多的應用與開發，在運算資源有限的邊緣終端設備上實現複雜類神經網路模型的需求變得益發重要。因此，我們提出一種新的優化方法以減少卷積類神經網路中的運算量。在這篇論文中我們認為某些卷積運算的輸出結果會因後續的激勵函數或池化層之特性而被浪費。卷積類神經網路中的卷積濾波器本質上會進行一連串乘積累加運算。我們提出的方法試圖在乘積累加運算中設置一系列檢查點，以確定濾波器能否依據該時間點之暫存結果提前終止運算。此外，我們亦探討由於設置檢查點提前終止乘積累加運算而導致的精確度損失之原因，並提出網路參數微調法予以解決。實驗結果顯示，我們提出的方法在兩個經典卷積類神經網路模型中能減少大約50%的乘積累加運算，精確度損失亦僅為1%，並且與以前的方法相比更具有競爭力。

Deep learning has been attracting enormous attention from academia as well as industry due to its great success in many artificial intelligence applications. As more applications are developed, the need for implementing a complex neural network model on an energy-limited edge device becomes more critical. Thus, this paper proposes a new optimization method for saving the computations of convolutional neural networks (CNNs). The method takes advantage of the fact that some convolutional operations are actually wasteful since their outputs are pruned by the following activation or pooling layers. Basically, a convolutional filter conducts a series of multiply-accumulate (MAC) operations. We propose to set a series of checkpoints in the MAC operations to determine whether a filter could terminate early according to the intermediate result. Furthermore, a fine-tuning process is conducted to recover the accuracy loss due to the applied checkpoints. The experimental results show that the proposed method can save approximately 50% MAC operations with only 1% accuracy loss for two classic CNN models and it is competitive with previous methods.

1 Introduction 1
2 Related works 5
2.0.1 Parameter pruning and sharing 5
2.0.2 Low-rank factorization 6
3 Proposed method 7
3.0.1 Step 1: Checkpoint determining 8
3.0.2 Step 2: Weight fine-tuning 9
3.0.3 Analysis and discussion on main ideas 9
4 Experimental Results 16
4.0.1 C10-Net 17
4.0.2 NiN 17
4.0.3 Summary 18
5 Conclusion 24
References 25

[1] Y. Cheng,D.Wang, P. Zhou, and T. Zhang.Model compression and acceleration for
deep neural networks: The principles, progress, and challenges. 35(1):126–136.
[2] S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural
networks with pruning, trained quantization and huffman coding. arXiv preprint
arXiv:1510.00149, 2015.
[3] S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for
efficient neural network. In Advances in neural information processing systems,
pages 1135–1143, 2015.
[4] Y. He, X. Zhang, and J. Sun. Channel pruning for accelerating very deep neural
networks. In International Conference on Computer Vision (ICCV), volume 2,
2017.
[5] G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network.
arXiv preprint arXiv:1503.02531, 2015.
[6] M. Jaderberg, A. Vedaldi, and A. Zisserman. Speeding up convolutional neural
networks with low rank expansions. arXiv preprint arXiv:1405.3866, 2014.
[7] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama,
and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In
Proceedings of the 22nd ACM international conference onMultimedia, pages 675–
678. ACM, 2014.
[8] A. Krizhevsky, V. Nair, and G. Hinton. The cifar-10 dataset. online: http://www. cs.
toronto. edu/kriz/cifar. html, 2014.
[9] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep
convolutional neural networks. In Advances in neural information processing systems,
pages 1097–1105, 2012.
[10] V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, and V. Lempitsky. Speedingup
convolutional neural networks using fine-tuned cp-decomposition. arXiv
preprint arXiv:1412.6553, 2014.
[11] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf. Pruning filters for efficient
convnets. arXiv preprint arXiv:1608.08710, 2016.
[12] M. Lin, Q. Chen, and S. Yan. Network in network. arXiv preprint arXiv:1312.4400,
2013.
[13] S. Shi and X. Chu. Speeding up convolutional neural networks by exploiting the
sparsity of rectifier units. arXiv preprint arXiv:1704.07724, 2017.
[14] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi. Inception-v4, inception-resnet
and the impact of residual connections on learning. In AAAI, volume 4, page 12,
2017.
[15] C. Tai, T. Xiao, Y. Zhang, X. Wang, et al. Convolutional neural networks with lowrank
regularization. arXiv preprint arXiv:1511.06067, 2015.