本篇論文提出了一套適用於行車影像的穩定化方法。系統將光學失真納入考量，估測出相機在光學扭曲下的震動軌跡，並以反向校正而得到穩定化的行車影像。
影像穩定在視覺領域中是項日趨重要的技術，用於增強影像品質和去除振動。而在廣角行車紀錄器中，光學失真常會影響震動的估測。本系統首先針對原始影像進行扭曲參數的估測。我們由多張影像估測出影片的扭曲參數，並用除法失真校正模型以做後續的扭曲校正。接著，系統會從校正後的影像中估測光流場以得到各像素的位移向量；再藉由影像間的光流場以及三維運動模型來估測相機的自我運動。將各時刻所估測的震動位移參數，經過移動平均，而取得平順的位移軌跡。最後，穩定化後的位移參數用以合成具有扭曲校正的穩定化影像。在實驗部分，我們使用了合成與實際的行車影像，進行評估。除了視覺觀察之外，我們亦計算了偵間保真度以提供更為客觀的量化標準以作檢視。

In this thesis, we present a novel stabilization system for the videos recorded by dashcams. Our system takes the effects of optical distortion into consideration and attempt to estimate the shaky trajectories under radial distortion. Thus, we can obtain stable videos by removing the jitters and distortion effects.
Video stabilization is a technique to enhance video quality by removing vibration in the videos. In dashcam videos, the optical distortion often interferences the estimation of the camera motions. For the input video, we first estimate the distortion parameters from the video. We estimate the distortion parameters from point correspondences computed from multiple frames in the video. The division model is applied for distortion correction in this work. Then, we estimate the optical flow from consecutive frames in the corrected video. The ego-motion of the camera can be further estimated by fitting the estimated optical flow vectors to the 3D motion model. Then, a filtering procedure, moving average, is applied to the camera motion parameters to obtain a smooth camera trajectory. We use the smoothed parameters to synthesize the stabilized video. In order to justify our stabilization system, we demonstrate the proposed algorithm on synthetic and real-world videos. We also compute the inter-frame fidelity as a quantitative metric for evaluating the video stabilization result.

Chapter 1. Introduction 1
1.1 Motivation 1
1.2 Problem Description 2
1.3 Main Contribution 3
1.4 Thesis Organization 4
Chapter 2. Related Work 5
2.1 2D Video Stabilization 5
2.2 3D Video Stabilization 6
2.3 Gyroscope-Based Video Stabilization 7
Chapter 3. Proposed Method 8
3.1 Distortion Correction 9
3.1.1 Feature point selection 9
3.1.2 Radial Affine Model 11
3.1.3 Multi-frame Distortion Estimation 14
3.2 Ego-motion Estimation 15
3.2.1 Feature undistortion 16
3.2.2 Ego-motion model 16
3.3 Moving average 20
3.4 Image Warping 21
3.4.1 Image Undistortion 21
3.4.2 As-similar-as-possible warping 22
Chapter 4. Experimental Results 26
4.1 Comparison with other method 27
4.2 Evaluation with CarSim videos 30
4.3 Evaluation with Real-scene videos 32
4.4 Evaluation for distortion correction 35
Chapter 5. Conclusion 37
References 38

[1] L. Alvarez, L. Gomez, J. R. Sendra, “Accurate depth dependent lens distortion models: an application to planar view scenarios,” Journal of Mathematical Imaging and Vision, vol. 39, no. 1, pp. 75-85, January 2011.
[2] D.G. Lowe, “Object recognition from local scale-invariant features,” IEEE International Conference on Computer Vision, vol. 2, pp. 1150-1157, September 2006.
[3] H. Bay, T. Tuytelaars, and L.V. Gool, “Surf: speeded up robust features,” European Conference on Computer Vision, vol. 110, no.3, pp. 346-359, June 2008.
[4] CarSim http://www.cybernet-ap.com.tw/zh.php?m=449&t=92
[5] A .Litvin, J. Konrad and W.C. Karl, “Probabilistic video stabilization using Kalman filtering and mosaicking,” IS&T/SPIE Symposium on Electronic Imaging, Image and Video Communications, pp. 663-674, May 2003.
[6] Y. Matsushi Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H.Y. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1150-1163, July 2006.
[7] C. Morimoto and R. Chellappa, “Evaluation of image stabilization algorithms,” IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 5, no. 5, pp. 2789-2792, May 1998.
[8] H. Shen, Q. Pan, Y. Cheng, and Y. Yu, “Fast video stabilization algorithm for unmanned aerial vehicles,” IEEE International Conference on Intelligent Computing and Intelligent Systems, vol. 4, pp. 542-546, November 2009.
[9] M. Gleicher and F. Liu, “Re-cinematography: improving the camera dynamics of casual video,” International Conference on Multimedia, pp. 27-36. October 2007.
[10] K.-Y. Lee, Y.-Y. Chuang, B.-Y. Chen, and M. Ouhypung, “Video stabilization using robust feature trajectories,” IEEE International Conference on Computer Vision, pp. 1397-1404, September 2009.
[11] M. Grundmann, V. Kwatra, and I. Essa, “Auto-directed video stabilization with robust l1 optimal camera paths,” Computer Vision and Pattern Recognition, pp.225-232, June 2011.
[12] C. Buehler, M.Bosse, and L. McMillan, “Non-metric image-based rendering for video stabilization,” Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 609-614, December 2001.
[13] F. Liu, M. Gleicher, H. Jin, and A. Agarwala, “Content-preserving warps for 3D video stabilization,” ACM Transactions on Graphics, vol. 28, no. 3, July 2009.
[14] E. Ringaby and P.E. Forssén, “Efficient video rectification and stabilization for cell-phones,” International Journal of Computer Vision, vol. 96, no. 3, pp. 335, June 2011.
[15] A. Karpenko, D. Jacobs, J. Baek, and M. Levoy, “Digital video stabilization and rolling shutter correction using gyroscopes,” Stanford University Computer Science Tech Report, September 2011.
[16] S. Bell, A. Troccoli, and K. Pulli, “A non-linear filter for gyroscope-based video stabilization,” European Conference on Computer Vision, vol. 8692, pp. 294-308, September 2014.
[17] C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis,” Doctoral Thesis, Massachusetts Institute of Technology, May 2009.
[18] Z. Kukelova, J. Heller, M. Bujnak, and T. Pajdla, “Radial distortion homography,” Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 639-647, June 2015.
[19] S. Battiato, G. Gallo, G. Puglisi, and S. Scellato, “SIFT features tracking for video stabilization,” International Conference on Image Analysis and Processing, pp. 825-830, September 2007.
[20] J. Barreto, J. Roquette, P. Sturm, and F. Fonseca, “Automatic camera calibration applied to medical endoscopy,” in Proceedings of the 20th British Machine Vision Conference, London, UK, September 2009.