隨著科技進步，自動化是現代人們追求的目標，自駕車的發展越來越多樣化，我們根據這個主題進行研究。我們使用comma.ai在Github開源的自駕車系統openpilot以及他們的硬體設備。根據他們的模型與資料集comma2k19進行研究，建構出與openpilot相似的模型，並尋找合適的訓練方式。在未來，我們能慢慢建構自己的自駕車模型，並在這成熟的自駕車設備上運行。
ar
我們將一個完整的自駕車模型，細分成兩個模型：深度學習神經網路、路徑預測模型。前者為利用神經網路預測結果，後者則是將結果分析、預測成可視化的圖形。在深度學習神經網路中，我們利用YUV儲存方式的特性進行資料前處理，再組合不同的神經網路模型，並且比較backbone EfficientNet與EfficientNetV2在我們的更動與使用下兩者的差異。將訓練結果交予路徑預測模型進行分析，我們分析預測出的種種結果，著重於道路與道路線的預測，將預測出的世界座標轉換成二維座標，使其在圖形可視化。最後，藉由整套自駕車模型結構，我們便能順利預測出道路路徑與道路線。

With the advancement of technology, automation is a goal pursued by modern people, so the development of self-driving cars is becoming more and more diverse. We study this theme, using openpilot, comma.ai's open-source of the self-driving car system on Github, and their hardware devices. Our research is based on their software and hardware systems. We build a deep neural network (DNN), name it as OPNet similar to their DNN supercombo in openpilot, and train it on their dataset comma2k19. OPNet consists of three sub-networks, namely, an encoding NN, a recurrent NN, and a pose NN having regression outputs for path planning and self-driving maneuvers. In the future, we will create our own self-driving model and run on their self-driving devices.
ar

We also develop a path prediction (PP) model from OPNet's output and a simulator for visualizing, analyzing, and verifying OPNet-PP's performance in an end-to-end and real-time manner. Input images to OPNet are encoded in YUV (luminance, blue projection, red projection) instead of RGB (red, green, blue) format. We use and compare two encoding NNs, namely, EfficientNet and its variant V2. Our results show that OPNet-PP can successfully predict driving path and lane lines on the simulator in real time.

Contents
Acknowledgements
摘要 i
Abstract ii
1 緒論 1
1.1 研究動機 . . . . . . . . . . . . 1
1.2 問題陳述 . . . . . . . . . . . . 2
1.3 研究貢獻 . . . . . . . . . . . . 2
1.4 論文組織 . . . . . . . . . . . . 3
2 文獻回顧 5
2.1 深度學習神經網路 . . . . . . . . . 5
2.1.1 EfficientNet . . . . . . . . . 5
2.1.2 RNN . . . . . . . . . . . . . . 8
2.2 路線預測模型 . . . . . . 10
2.2.1 Openpilot 系統 . . . . . . 10
2.2.2 自駕車中的雷達應用 . . . . . . 11
3 深度學習神經網路 (Deep Learning Neural Networks) 13
3.1 模型總覽 . . . . . . 13
3.2 Conv Block . . . . . . 14
3.2.1 MBconv . . . . . . 14
3.2.2 Fuse_MBconv . . . . . . 15
3.3 EfficientNet . . . . . . 18
3.3.1 EfficientNet . . . . . . 18
3.3.2 EfficientNet V2 . . . . . . 18
3.4 RNN . . . . . . 19
3.5 PoseNet . . . . . . 19
3.6 損失函數 . . . . . . 19
4 路徑預測模型 (Path-Prediction Models) 23
4.1 路線預測 . . . . . . 23
4.2 實際預測結果 . . . . . . 23
4.3 Vandermonde matrix . . . . . . 25
4.4 道路預測長度與機率 . . . . . . 26
iii
5 實驗資料 (Experimental Data) 27
5.1 Comma2k19 資料集 . . . . . . 27
5.2 Data 前處理 . . . . . . . . 27
5.2.1 裁切 . . . . . . . . . . 28
5.2.2 YUV 格式轉換 . . . . . . 28
6 結果 (Results) 31
6.1 神經網路實驗結果 . . . . . . 31
6.2 路徑預測實驗結果 . . . . . . 32
7 結論 35
參考文獻 37

[1] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural net-
works,” International conference on machine learning, pp. 6015–6114, 2019.
[2] M. Tan and Q. Le, “Efficientnetv2: Smaller models and faster training,” International
conference on machine learning., pp. 10096–10106, 2021.
[3] P. Cunningham, M. Cord, and S. J. Delany, “Supervised learning,” Machine learning tech-
niques for multimedia, pp. 21–49, 2018.
[4] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent
neural networks on sequence modeling,” arXiv preprint, p. 1412.3555, 2014.
[5] Y. LeCun and et al., “Backpropagation applied to handwritten zip code recognition,” Neu-
ral Computation, pp. 541 – 551, 1989.
[6] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep con-
volutional neural networks,” Advances in neural information processing systems, p. 25,
2012.
[7] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image
recognition,” arXiv preprint, p. 1409.1556, 2014.
[8] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,”
Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–
778, 2016.
[9] A. G. Howard and et al, “Mobilenets: Efficient convolutional neural networks for mobile
vision applications,” arXiv preprint, p. 1704.04861, 2017.
[10] Elman and J. L, “Finding structure in time,” Cognitive science, pp. 179–211, 1990.
[11] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation,
pp. 1735–1780, 1997.
[12] Comma.ai, “Openpilot git repo.” https://github.com/commaai/openpilot.
[13] Comma.ai, “Comma2k19.” https://github.com/commaai/comma2k19.
[14] Y.-P. Liu, “Trajectory prediction and radar simulation on the highway with deep learning
methods,” Master’s Thesis, National Tsing Hua University, 202