在邊緣端的不穩定溝通渠道與受限制運算資源為無人偵測機與機器人等的可移動式電池驅動設備的兩大限制。這些限制對於運算深度類神經網路的設備來說尤其嚴重。現在的趨勢是藉由把模型化的類神經網路層串接起來以因應對高精準度需求。在邊緣端運算深度網路會增加運算量與資源佔用量，進而導致電力消耗的增加。然而，在邊緣端用一個淺的網路並把運算量傳給伺服器則會因為不穩定的溝通渠道導致嚴重的延遲。因此，現在急需動態的深度類神經網路以管理傳輸量並且維持一定的精準度。這篇論文中，我們探討了可靠運算單元與動態網路架構。可靠運算單元為不同的類別定義了一系列的門檻值並用這些門檻值來決定是否要把這次的輸入資料傳到伺服器處理。動態網路架構則是根據溝通渠道的可用性來調整整個網路的深度。透過這兩個機制，我們可以有效率的分配邊緣端與伺服器端兩邊的運算量。

Unreliable communication channels and limited computing resources at the edge end are two primary constraints of battery-powered movable devices, such as autonomous robots and unmanned aerial vehicles (UAVs).
The impact is especially severe for those performing deep neural network (DNN) computations.
With increasing demand for accuracy, the trend in modern DNN designs is the use of cascaded modularized layers.
Implementing a deep network at the edge increases computational workloads and resource occupancy, leading to an increase in battery drain.
Using a shallow network and offloading workloads to backbone servers, however, incur significant latency overheads caused by unstable communication channels.
Hence, dynamic DNN design techniques for efficient workload allocation are urgently required to manage the amount of workload transmissions while achieving the required accuracy.
In this paper, we explore the use of authentic operation (AO) unit and dynamic network structure to enhance DNNs.
The AO unit determines a set of stochastic threshold values for different DNN output classes and determines at runtime if an input has to be transferred to backbone servers for further analysis.
The dynamic network structure adjusts its depth according to channel availability.
Experiments have been comprehensively performed on several well-known DNN models and datasets.
Our results show that, on an average, the proposed techniques based on a type of the DNN structure called residual neural network are able to reduce the amount of transmissions by up to 17% compared to previous methods under the same accuracy requirement.

1 Introduction 6
2 Background 11
2.1 CNN and Softmax Function 11
2.2 Modularized Components of CNN 12
2.3 Fast Inference 13
3 Authentic Operation Unit 15
3.1 AO Decision Flow 16
3.2 Fine-Grained AO Criteria 17
3.3 AO Threshold Selection by Kernel Density Estimation 19
4 Dynamic Network Sizing 21
4.1 Normal Mode and Workload Sharing Mode Structure 22
4.2 Network Sizing Methodology 23
4.3 Auxiliary Network Training Process 24
5 Experimental Results 26
5.1 Simulation Setup 26
5.2 Comparison of Different Threshold Criteria on Various Network Architectures 27
5.3 Comparison of Workload Sharing Efficiency 29
5.4 Validation of Dynamic Network Sizing 30
6 Conclusion

[1] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp.2278-2324, Nov. 1998.
[2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classiffication with deep convolutional neural networks," in Proc. Int. Conf. Neural Information Processing Systems (NIPS), pp. 1097-1105, Dec. 2012.
[3] F. N. Iandola et al., "Squeezenet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size," arXiv:1602.07360, Mar. 2016.
[4] S. Han et al., "DSD: Regularizing deep neural networks with dense-sparse-dense training flow," arXiv:1607.04381, Jul. 2016.
[5] S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in Proc. Int. Conf. Machine Learning (ICML), pp. 448-456, Feb. 2015.
[6] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting," J. Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, Jun.2014.
[7] M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv:1312.4400, Dec. 2013.
[8] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv:1409.1556, Sep. 2014.
[9] C. Szegedy et al., "Going deeper with convolutions," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1-9, Jun. 2015.
[10] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Jun. 2016.
[11] S. Zagoruyko and N. Komodakis, "Wide residual networks," arXiv:1605.07146, May 2016.
[12] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 580-587, Jun. 2014.
[13] E. Shelhamer, J. Long, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, Jun. 2015.
[14] J. Deng et al., "ImageNet: A large-scale hierarchical image database," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 248-255, Jun. 2009.
[15] R. Deng, R. Lu, C. Lai, T. H. Luan, and H. Liang, "Optimal workload allocation in fog-cloud computing toward balanced delay and power consumption," J. IEEE Internet of Things, vol. 3, no. 6, pp. 1171-1181, Dec. 2016.
[16] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, "Fog computing and its role in the internet of things," in Proc. MCC Wksp. Mobile Cloud Computing, pp. 13-16, Aug. 2012.
[17] P. G. Lopez et al., "Edge-centric computing: Vision and challenges," ACM SIGCOMM Computer Communication Review, vol. 45, no. 5, pp. 37-42, Oct. 2015.
[18] V. Mushunuri, A. Kattepur, H. K. Rath, and A. Simha, "Resource optimization in fog enabled IoT deployments," in Proc. IEEE Int. Conf. Fog and Mobile Edge Computing (FMEC), pp. 6-13, May 2017.
[19] P. Panda, A. Sengupta, and K. Roy, "Energy-efficient and improved image recognition with conditional deep learning," J. Emerging Technologies in Computing Systems, vol. 13, no. 3, pp. 33:1-33:21, Feb. 2017.
[20] S. Teerapittayanon, B. McDanel, and H. Kung, "Branchynet: Fast inference via early exiting from deep neural networks," in Proc. IEEE Int. Conf. Pattern Recognition (ICPR), pp. 2464-2469, Dec. 2016.
[21] S. Venkataramani, A. Raghunathan, J. Liu, and M. Shoaib, "Scalable-effort classifiers for energy-efficient machine learning," in Proc. Design Automation Conf., pp. 67:1-67:6, Jun. 2015.
[22] A. Krizhevsky, "Learning multiple layers of features from tiny images," Master's thesis, Department of Computer Science, University of Toronto, Apr. 2009.
[23] H. Zhao, J. Shi, X. Qi, X.Wang, and J. Jia, "Pyramid scene parsing network," in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), to appear, Jul. 2017.
[24] Y. Jia et al., "Caffe: Convolutional architecture for fast feature embedding," in Proc. ACM Int. Conf. Multimedia, pp. 675-678, Nov. 2014.