邊緣人工智能（AI）的一個關鍵約束是其有限的計算能力。因此，對邊緣AI的依賴將導致不可避免的準確性權衡。提高總體準確性的一種方法是引入工作負載分配方案，該方案將需要復雜計算的輸入數據分配給服務器AI，同時在邊緣AI處保留簡單的計算。為了實現這一點，我們利用可靠的操作（AO）來評估邊緣AI的預測置信度。我們的研究基於以前使用細粒度成對閾值處理的工作。在這項工作中，我們提出了粗粒度的群集層次閾值。此外，均方誤差（MSE）用於基於所獲得的閾值數據來規範邊緣AI的預測。我們通過添加第二級標準來進一步修改現有AO塊，該第二級標準用作驗證層，目的是進一步減少傳輸計數。我們的方法將10類數據集的閾值最小化了90％，並將數據傳輸率降低了15.20％，同時保持了整體準確性。

A critical constraint in Edge Artificial Intelligence (AI) is its limited computing power. Due to this, reliance on edge AI would result in an inevitable accuracy trade-off. One way to increase the overall accuracy is to introduce a workload allocation scheme that would assign input data requiring complex computations to a server AI while retaining simple ones at the edge AI. In order to achieve this, we utilize an authentic operation (AO) which assesses prediction confidence of the edge AI. We based our research on a previous work which uses fine-grained pair-wise thresholding. In this work, we proposed a coarse-grained cluster-wise hierarchical thresholding. Moreover, mean squared error (MSE) is used to regularize the edge AI’s prediction based on the obtained threshold data. We further modify the existing AO block by adding a second level criterion which serves as a validation layer with the aim of further reducing the transmission count. Our methodology minimizes the threshold values by 90% for a 10 class dataset and reduces data transmission by 15.20% while retaining overall accuracy.

1 Introduction 1
2 Background 6
2.1 Probability Difference Overview 6
2.2 Threshold Selection using PD 8
3 Methodologies 10
3.1 Hierarchical Thresholding 10
3.2 Probability Regularization 14
3.3 Authentic Operator: Two Level Validation 16
4 Accuracy Transmission Trade-off 19
5 Experimental Results 21
5.1 Comparison of different methodologies 21
5.2 Validation of probability regularization 23
6 Conclusion 26
References 27

[1] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278– 2324, 1998.

[2] M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in European conference on computer vision. Springer, 2014, pp. 818– 833.

[3] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.

[4] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.

[5] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.

[6] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.

[7] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.

[8] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.

[9] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size,” arXiv preprint arXiv:1602.07360, 2016.

[10] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks.” in CVPR, vol. 1, no. 2, 2017, p. 3.
[11] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A largescale hierarchical image database,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. Ieee, 2009, pp. 248–255. 27

[12] S. Teerapittayanon, B. McDanel, and H. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in Pattern Recognition (ICPR), 2016 23rd International Conference on. IEEE, 2016, pp. 2464–2469.

[13] P. Panda, A. Sengupta, and K. Roy, “Energy-efficient and improved image recognition with conditional deep learning,” ACM Journal on Emerging Technologies in Computing Systems (JETC), vol. 13, no. 3, p. 33, 2017.

[14] K. Berestizshevsky and G. Even, “Sacrificing accuracy for reduced computation: Cascaded inference based on softmax confidence,” arXiv preprint arXiv:1805.10982, 2018.

[15] C. Lo, Y.-Y. Su, C.-Y. Lee, and S.-C. Chang, “A dynamic deep neural network design for efficient workload allocation in edge computing,” in Computer Design (ICCD), 2017 IEEE International Conference on. IEEE, 2017, pp. 273–280.

[16] Z. Yan, H. Zhang, R. Piramuthu, V. Jagadeesh, D. DeCoste, W. Di, and Y. Yu, “Hd-cnn: hierarchical deep convolutional neural networks for large scale visual recognition,” in Proceedings of the IEEE international conference on computer vision, 2015, pp. 2740–2748.

[17] A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009.

[18] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org. [Online]. Available: https://www.tensorflow.org