深度估計是電腦視覺的重要領域之一,各類型無人載具或是自動駕駛等領域都需要使用到這項技術。近些年來機器學習領域蓬勃發展,深度估計這項技術也受益於機器學習的加持,以卷積神經網路或是 Vision Tramsformer 為基礎設計的深度估計網路架構可以達到非常優秀的精確度,但類似的神經網路架構都非常龐大,且需要大量的運算資源以及功耗才能計算出深度估計的結果,過往沒有以幀為基礎的深度估計突波神經網路的相關研究,故我們基於實驗室過去的研究結果,設計出以光流為基礎的神經網路算法,以非常簡單的架構便可以產生深度估計的結果,且相較於其他的神經網路架構,我們所設計的架構可以極快的速度運算出深度估計的結果。接著,我們再度簡化了神經網路,並將其應用於低功耗的裝置上,測試此深度估計結果應用於障礙物迴避任務的表現,也獲取了不錯的結果,進一步展現了此神經網路架構的輕量化以及實用性。

Depth estimation is one of the important techniques in computer vision, various types of unmanned vehicles or autonomous vehicles necessitate this technology. Recently, the machine learning technology is growing fast, depth estimation technology also benefits from the support of machine learning, The convolutional neural networks- based or Vision Tramsformer-based design of depth estimation network architecture can achieve outstanding performance for the mission. However, the calculation load for such neural network architectures is heavy, requiring a large amount of resources and energy to estimate the depth. Compared with other neural network architectures, our architecture obtains the depth much faster. We also put it into practice for edge computing. The simplified neural network can be implemented in a low power device to perform depth estimation and obstacle avoidance task with great performance, further demonstrating the lightweight and practicality of this neural network architecture.

Abstract iii
摘要 iv
誌謝 v
第一章 簡介 1
第一節 突波神經網路 1
第二節 光流 2
第三節 深度估計 3
第四節 障礙物迴避 4
第五節 Flowdep-基於光流的深度估計演算法 4
一、旋轉補償 5
二、從運動中估計深度 5
第六節 突波神經網路模擬器 7
第七節 論文架構 9
第二章 基於突波神經網路與人工神經網路的深度估計模型 10
第一節 神經網路模型 10
一、Flowdep-S架構 10
(一)、光流轉換 11
(二)、理想平移光流 11
(三)、理想旋轉光流 12
(四)、光流補償 14
(五)、歐幾里得距離(Euclidean Distance)計算 15
(六)、深度估計 15
二、Flowdep-A架構 17
(一)、訓練方式 18
第二節 實驗 19
一、實驗設置 19
二、資料集 20
三、評估 20
第三章 Flowdep為基礎的神經網路障礙物迴避模組 23
第一節 模組架構 23
一、Flowdep-SS 23
二、障礙物迴避邏輯設置 24
第二節 實驗 26
一、實驗設置 26
(一)、訓練方式 26
(二)、實驗細節以及無人車架構 27
(三)、環境設置 28
二、評估 29
第四章 結論與討論 31
第五章 參考文獻 34

1. Weihong, Wang, and Tu Jiaoyang. "Research on license plate recognition algorithms based on deep learning in complex environment." IEEE Access 8 (2020): 91661-91675.
2. Grigorescu, Sorin, et al. "A survey of deep learning techniques for autonomous driving." Journal of Field Robotics 37.3 (2020): 362-386.
3. Yang, Shuoheng, Yuxin Wang, and Xiaowen Chu. "A survey of deep learning techniques for neural machine translation." arXiv preprint arXiv:2002.07526 (2020).
4. McCulloch, Warren S., and Walter Pitts. "A logical calculus of the ideas immanent in nervous activity." The bulletin of mathematical biophysics 5.4 (1943): 115-133.
5. Kasabov, Nikola, et al. "Dynamic evolving spiking neural networks for on-line spatio-and spectro-temporal pattern recognition." Neural Networks 41 (2013): 188-201.
6. Akopyan, Filipp, et al. "Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip." IEEE transactions on computer-aided design of integrated circuits and systems 34.10 (2015): 1537-1557.
7. Hodgkin, Alan L., and Andrew F. Huxley. "A quantitative description of membrane current and its application to conduction and excitation in nerve." The Journal of physiology 117.4 (1952): 500.
8. Izhikevich, Eugene M. "Simple model of spiking neurons." IEEE Transactions on neural networks 14.6 (2003): 1569-1572.
9. Gerstner, Wulfram, et al. Neuronal dynamics: From single neurons to networks and models of cognition. Cambridge University Press, 2014.
10. Hebb, Donald Olding. The organization of behavior: A neuropsychological theory. Psychology Press, 2005.
11. Gibson, James J. "The perception of the visual world." (1950).
12. Horn, Berthold KP, and Brian G. Schunck. "Determining optical flow." Artificial intelligence 17.1-3 (1981): 185-203.
13. Lucas, Bruce D., and Takeo Kanade. An iterative image registration technique with an application to stereo vision. Vol. 81. 1981.
14. Barron, John L., David J. Fleet, and Steven S. Beauchemin. "Performance of optical flow techniques." International journal of computer vision 12.1 (1994): 43-77.
15. Kitt, Bernd, Benjamin Ranft, and Henning Lategahn. "Block-matching based optical flow estimation with reduced search space based on geometric constraints." 13th International IEEE Conference on Intelligent Transportation Systems. IEEE, 2010.
16. Becciu, Alessandro, et al. "A multi-scale feature based optic flow method for 3D cardiac motion estimation." International Conference on Scale Space and Variational Methods in Computer Vision. Springer, Berlin, Heidelberg, 2009.
17. Ilg, Eddy, et al. "Flownet 2.0: Evolution of optical flow estimation with deep networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
18. Shin, Jeongho, et al. "Optical flow-based real-time object tracking using non-prior training active feature model." Real-time imaging 11.3 (2005): 204-218.
19. Sengar, Sandeep Singh, and Susanta Mukhopadhyay. "Motion detection using block based bi-directional optical flow method." Journal of Visual Communication and Image Representation 49 (2017): 89-103.
20. Duman, Elvan, and Osman Ayhan Erdem. "Anomaly detection in videos using optical flow and convolutional autoencoder." IEEE Access 7 (2019): 183914-183923.
21. Braillon, Christophe, et al. "Real-time moving obstacle detection using optical flow models." 2006 IEEE Intelligent Vehicles Symposium. IEEE, 2006.
22. Yang, Zixin, et al. "Dense Depth Estimation from Stereo Endoscopy Videos Using Unsupervised Optical Flow Methods." Annual Conference on Medical Image Understanding and Analysis. Springer, Cham, 2021.
23. An image-interpolation technique for the computation of optic flow and egomotion
24. Wang, Yan, et al. "Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
25. Yang, Xin, et al. "Fast depth prediction and obstacle avoidance on a monocular drone using probabilistic convolutional neural network." IEEE Transactions on Intelligent Transportation Systems 22.1 (2019): 156-167.
26. Abuowaida, Suhaila FA, and Huah Yong Chan. "Improved Deep Learning Architecture for Depth Estimation from Single Image." Jordanian Journal of Computers and Information Technology 6.4 (2020).
27. Santoro, Michael, Ghassan AlRegib, and Yucel Altunbasak. "Misalignment correction for depth estimation using stereoscopic 3-D cameras." 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2012.
28. Zhang, Ruo, et al. "Shape-from-shading: a survey." IEEE transactions on pattern analysis and machine intelligence 21.8 (1999): 690-706.
29. Tsai, Yi-Min, Yu-Lin Chang, and Liang-Gee Chen. "Block-based vanishing line and vanishing point detection for 3D scene reconstruction." 2006 international symposium on intelligent signal processing and communications. IEEE, 2005.
30. Laina, Iro, et al. "Deeper depth prediction with fully convolutional residual networks." 2016 Fourth international conference on 3D vision (3DV). IEEE, 2016.
31. Risi, Nicoletta, Enrico Calabrese, and Giacomo Indiveri. "Instantaneous stereo depth estimation of real-world stimuli with a neuromorphic stereo-vision setup." 2021 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2021.
32. Haessig, Germain, et al. "A spiking neural network model of depth from defocus for event-based neuromorphic vision." Scientific reports 9.1 (2019): 1-11.
33. Jin, Yun, et al. "Design of an intelligent active obstacle avoidance car based on rotating ultrasonic sensors." 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER). IEEE, 2018.
34. Yang, Xin, et al. "Fast depth prediction and obstacle avoidance on a monocular drone using probabilistic convolutional neural network." IEEE Transactions on Intelligent Transportation Systems 22.1 (2019): 156-167.
35. Jiang, Ao, Xiang Yao, and Juan Zhou. "Research on path planning of real‐time obstacle avoidance of mechanical arm based on genetic algorithm." The Journal of Engineering 2018.16 (2018): 1579-1586.
36. Budiyanto, Almira, et al. "UAV obstacle avoidance using potential field under dynamic environment." 2015 International Conference on Control, Electronics, Renewable Energy and Communications (ICCEREC). IEEE, 2015.
37. Esrafilian, Omid, and Hamid D. Taghirad. "Autonomous flight and obstacle avoidance of a quadrotor by monocular SLAM." 2016 4th International Conference on Robotics and Mechatronics (ICROM). IEEE, 2016.
38. P. Chakravarty, K. Kelchtermans, T. Roussel, S. Wellens, T. Tuytelaars and L. Van Eycken, "CNN-based single image obstacle avoidance on a quadrotor," 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 6369-6374, doi: 10.1109/ICRA.2017.7989752.
39. Minguez, Javier, Florant Lamiraux, and Jean-Paul Laumond. "Motion planning and obstacle avoidance." Springer handbook of robotics. Springer, Cham, 2016. 1177-1202.
40. Chen-Fu Yeh. depth-from-motion. https://github.com/twetto/depth-from-motion, 2020.
41. Paszke, Adam, et al. "Pytorch: An imperative style, high-performance deep learning library." Advances in neural information processing systems 32 (2019).
42. Wei Fang, Yanqi Chen, Jianhao Ding, Ding Chen, Zhaofei Yu, Huihui Zhou, Yonghong Tian, and other contributors. Spikingjelly. https://github.com/fangwei1234 56/spikingjelly, 2020.
43. Chen-Fu Yeh. iq-neuron. https://github.com/twetto/iq-neuron, 2021.
44. Kroeger, Till, et al. "Fast optical flow using dense inverse search." European conference on computer vision. Springer, Cham, 2016.
45. cmpark0126. pytorch-polynomial-lr-decay. https://github.com/cmpark0126/pytorch-polynomial-lr-decay, 2020
46. Eigen, David, Christian Puhrsch, and Rob Fergus. "Depth map prediction from a single image using a multi-scale deep network." Advances in neural information processing systems 27 (2014).
47. Atapour-Abarghouei, Amir, and Toby P. Breckon. "Veritatem dies aperit-temporally consistent depth prediction enabled by a multi-task geometric and semantic scene understanding approach." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
48. Fu, Huan, et al. "Deep ordinal regression network for monocular depth estimation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
49. Godard, Clément, et al. "Digging into self-supervised monocular depth estimation." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.
50. Quigley, Morgan, et al. "ROS: an open-source Robot Operating System." ICRA workshop on open source software. Vol. 3. No. 3.2. 2009.
51. Hu, Yangfan, Huajin Tang, and Gang Pan. "Spiking Deep Residual Networks." IEEE Transactions on Neural Networks and Learning Systems (2018).
52. Mohemmed, Ammar, et al. "Span: Spike pattern association neuron for learning spatio-temporal spike patterns." International journal of neural systems 22.04 (2012): 1250012.
53. Ponulak, Filip. ReSuMe-new supervised learning method for spiking neural networks. Institute of Control and Information Engineering, Poznoń University of Technology. Tech. rep, 2005.
54. Tavanaei, Amirhossein, Zachary Kirby, and Anthony S. Maida. "Training spiking convnets by stdp and gradient descent." 2018 International Joint Conference on Neural Networks (IJCNN). IEEE, 2018.
55. Rueckauer, Bodo, et al. "Conversion of continuous-valued deep networks to efficient event-driven networks for image classification." Frontiers in neuroscience 11 (2017): 682.
56. Esser, Steven K., et al. "From the cover: Convolutional networks for fast, energy-efficient neuromorphic computing." Proceedings of the National Academy of Sciences of the United States of America 113.41 (2016): 11441.
57. Davies, Mike, et al. "Loihi: A neuromorphic manycore processor with on-chip learning." Ieee Micro 38.1 (2018): 82-99.
58. Almalioglu, Yasin, et al. "SelfVIO: Self-supervised deep monocular Visual–Inertial Odometry and depth estimation." Neural Networks 150 (2022): 119-136.
59. Ganganath, Nuwan, and Henry Leung. "Mobile robot localization using odometry and kinect sensor." 2012 IEEE International Conference on Emerging Signal Processing Applications. IEEE, 2012.