通常一段影片在以特定倍率放大後會有失真的情況，而其解決方法的程序便稱為超解析度(super resolution)演算法。在處理影片的超解析度過程中，取得相鄰幀之間的物體移動軌跡 (光流，optical flow)為重建高解析度影片的一重要步驟，而為了提高準確度及處理任意影片的能力，包含每個幀所有點構成之optical flow的計算方法被廣泛的使用，運用此方法所得到之輸出影片品質也十分突出，但卻需耗費龐大的時間複雜度。
我們在這篇論文提出一個以高頻邊緣為處理超解析度基底的方法，在達到減少時間複雜度目的的同時也能讓輸出的影片維持很好的品質。而如何減少時間複雜度的重點在於根據人類視覺系統(HVS)，只要抓取每個幀中針對人眼較為敏感的高頻邊緣部分並進行optical flow計算即可；optical flow通常是採雙向計算(順向、逆向)，這是為了得到與前後幀關係進而增加準確度，但也是造成計算時間加倍的原因之一，所以在這裡我們也提出由順向optical flow推出逆向optical flow的方法，有效的減少所需要的時間。
我們以不同的放大倍率與不同的影片做了一系列的實驗，並和現有的超解析度演算法比較，結果顯示我們提出的方法成功的達到四倍的加速並同時維持住高解析度影片的輸出品質。

Among researches of video super-resolution (VSR), reconstruction-based method is widely used for the capability of managing arbitrary scenes. And for accuracy, full-image based motion prediction is often adopted that results outstanding outputs. But it costs huge time complexity due to the multi-level processing. In this paper, we propose an edge-based motion and intensity prediction scheme to reduce the computation cost while maintain good enough quality simultaneously. The key point of reducing computation cost is to focus on extracted edges rather than the whole frame when finding optical flows of the video sequence in accordance with human vision system (HVS). Bi-directional optical flows are usually adopted to increase the prediction accuracy but it also increase the computation time. We also propose to obtain the backward flow from foregoing forward flow prediction which effectively save the heavy load. We perform a series of experiments and comparisons between existing VSR methods and our proposed edge-based method with different sequences and upscaling factors. The results reveal that our proposed scheme can successfully keep the super-resolved sequence quality and get about five times speed up in computation time for a two by two video scaling and six times speed up for a four by four one.

1 Introduction 1
1.1 Background of video super resolution (VSR) . . . . . . . . . . . . . 1
1.2 Motivation and Problem Description . . . . . . . . . . . . . . . . . 4
1.3 Goal and Contribution . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Related Works 9
2.1 Interpolation-based Methods . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Learning-based Methods . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Reconstruction-based Methods . . . . . . . . . . . . . . . . . . . . . 13
3 Proposed Edge-based Motion and Intensity Prediction for Video
Super-Resolution 15
3.1 Optical
ow prediction . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2 High resolution sequence construction . . . . . . . . . . . . . . . . . 24
4 Experimental Results 31
4.1 Quality Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 Computation Complexity Analysis . . . . . . . . . . . . . . . . . . 34
i
5 Conclusion and Future Work 52
5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

[1] R. Tsai and T. Huang, \Multiframe image restoration and registration," Ad-vances in Computer Vision and Image Processing, pp. 317319, 1984.
[2] S. Kim, N. Bose, and H. Valenzuela, \Recursive reconstruction of high resolu-tion image from noisy undersampled multiframes," IEEE Transactions Acous-
tic, Speech and Signal Process, vol. 38, no. 2, pp. 10131027, June. 1990.
[3] N. Bose, H. Kim, and H. Valenzuela, \Recursive total least squares algorithm
for image reconstruction from noisy, under sampled frames," Multidimen-
sional Systems and Signal Processing, vol. 4, no. 3, pp. 253268, 1993.
[4] R. Hardie, K. Barnard, and E. Armstrong, \Joint map registration and high-resolution image estimation using a sequence of undersampled images," IEEE
Transactions on Image Processing, vol. 6, no. 12, pp. 16211633, 1997.
[5] B. Bascle, A. Blake, and A. Zisserman, \Motion deblurring and super reso-lution from an image sequence," In Proceedings of the fourth European Con-ference on Computer Vision vol. 6, 1996.
[6] S. Fifman, \Digital recti_cation of ERTS multispectral imagery," in In Proceedings of Significant Results Obtained Earth Resources Technology.
Satellite-1, vol. 1, pp. 1131-1142, 1973.
[7] J. Parker, R. Kenyon, and D. Troxel, \Comparison of interpolation methods for image resampling," IEEE Transactions on Medical Imaging, vol. 2, no. 3, pp. 31-39, September 1983.
[8] H. Hou and H. Andrews, \Cubic splines for image interpolation and digital ltering," IEEE Transaction on Acoustics, Speech and Signal Processing, vol. ASSP-26, no. 6, pp. 508-517, December 1978.
[9] K. Turkowski, \Filters for common resampling tasks," in Graphics Gems. Academic Press Professional, 1990.
[10] Z. Yu, H. Li, Z. Wang, Z. Hu, and C. Chen, \Multi-level Video Frame Interpolation: Exploiting the Interaction Among Dierent Levels," IEEE Transactions on circuits and system for video technology, vol. 23, no. 7, July 2013
[11] Z.Wu, H. Yu, and C. Chen, \A New Hybrid DCT-Wiener-Based Interpolation Scheme for Video Intra Frame Up-Sampling," IEEE signal processing letters,vol. 17, no. 10, October 2010
[12] J. Yu, X. Gao, D. Tao, X. Li and K. Zhang \A Uni
ed Learning Framework for Single Image Super-Resolution," IEEE Transactions on Neural networks and Learning systems, vol. 25, no. 4, April 2014
[13] K. Guo, X. Yang, W. Lin, R. Zhang and S. Yu \Learning-based super-resolution method with a combining of both global and local constraints,"
Institute of Image Communication and Information Processing, vol. 6, Iss. 4,
pp. 337 344, 2012
[14] P. Gajjar and M. Joshi, \New Learning Based Super-Resolution: Use of DWT
and IGMRF Prior," IEEE Transactions on Image Processing, vol. 19, no. 5,May 2010
[15] Y. Kang, S. Jeong B. Song, \Fast super-resolution algorithms using one-dimensional patch-based training and directional interpolation," IET Image Processing, vol. 6, Iss. 5, pp. 548557, 2012
[16] K. Zhang, X. Gao, D. Tao and X. Li \Single Image Super-Resolution With Multiscale Similarity Learning," IEEE Transactions on neural networks and learning systems, vol. 24, no. 10, October 2013
[17] C. Dang, M. Aghagolzadeh, and H. Radha, \Image Super-Resolution via Local Self-Learning Manifold Approximation," IEEE signal processing letters, vol. 21, no. 10, October 2014
[18] J. Yang, Z. Wang, Z. Lin, S. Cohen and T. Huang, \Coupled Dictionary Training for Image Super-Resolution," IEEE Transactions on Image Processing, vol. 21, no. 8, August 2012
[19] S. Dai, S. Baker, and S. Kang, \An MRF-Based DeInterlacing Algorithm With Exemplar-Based Re_nement," IEEE Transactions on Image Processing, vol. 18, no. 5, May 2009
[20] Z. Xiong, X. Sun, and Feng Wu, \Robust Web Image/Video Super-Resolution,", IEEE Transactions on Image Processing, vol. 19, no. 8, August 2010.
[21] S. Keller, F. Lauze, and M. Nielsen, \Video super-resolution using simultaneous motion and intensity calculations," IEEE Transactions on Image Pro-
cessing, vol. 20, no. 7, pp. 18701884, 2011.
[22] C. Hsieh, Y. Huang, Y. Chen and C. Fuh, \Video Super-Resolution by Motion Compensated Iterative Back-Projection Approach," Journal of Information Science and Engineering vol. 27, pp. 1107-1122 ,2011
[23] S. Belekos, N. Galatsanos, and A. Katsaggelos, \Maximum a Posteriori Video Super-Resolution Using a New Multichannel Image Prior," IEEE Transactions on Image Processing, vol. 19, no. 6, pp. 14511464 , June 2010.
[24] K. Yap, Y. He, Y. Tian and L. Chau, \A Nonlinear L1-Norm Approach for Joint Image Registration and Super-Resolution," IEEE Transactions on
Signal Processing Letters. vol. 16, no. 11, pp. 981-982, November 2009
[25] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, \Fast and robust multi-frame super-resolution," IEEE Transactions on Image Processing, vol. 13, pp. 13271344, 2004.
[26] J. Ge, J. Liu, C. Ge, and X. Yang, \A robust video super-resolution based on adaptive overlapped block motion compensation," Signals, Systems and Computers (ASILOMAR) Conference vol. 0, pp. 187194, 2013.
[27] S. Naja_ and S. Shirani, \Regularization function for video superresolution
using auxiliary high resolution still images," Signal-Image Technology and Internet-Based Systems, pp. 1713 1717, 2012.
[28] B. Song, S. Jeong, and Y. Choi, \Video super-resolution algorithm using bidirectional overlapped block motion compensation and on-the-y dictionary training," IEEE Transactions on Circuits and Systems for Video Technology,
vol. 21, pp. 274285, 2011.
[29] E. Hung, R. Queiroz, F. Brandi, K. Oliveira, and D. Mukherjee, \Video super-resolution using codebooks derived from key-frames," IEEE Transactions on Circuits and System for Video Technology, vol. 22, no. 9, pp. 13211331,
September 2012.
[30] H. Takeda, P. van Beek, P. Milanfar, \Spatio-temporal video interpolation
and denoising using motion-assisted steering kernel (MASK) regression,"
IEEE International Conference on Image Processing 2008.
[31] H. Takeda, P. Milanfar, M. Protter, and M. Elad, \Super-Resolution Without
Explicit Subpixel Motion Estimation," IEEE Transactions on image processing, vol. 18, no. 9, September 2009
[32] M. Protter, M. Elad, H. Takeda, and P. Milanfar, \Generalizing the Nonlocal-Means to Super-Resolution Reconstruction," IEEE Transactions on image processing, vol. 18, no. 1, January 2009.
[33] H. Su, Y. Wu, and J. Zhou, \Super-Resolution Without Dense Flow," IEEE
Transactions on image processing, vol. 21, no. 4, April 2012.
[34] C. Liu and D. Sun, \On Bayesian Adaptive Video Super Resolution," IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2,February 2014.
[35] W.Wu, T.Wang, and C. Chiu, \Edge curve scaling and smoothing with cubic
spline interpolation," in IEEE International Conference on Image Processing,
pp. 859862, 2013.
[36] \Cimg library" Available: http://cimg.sourceforge.net/, 2014.
[37] N. Barzigar, A. Roozgard, S. Cheng, and P. Verma, \A robust super resolu-
tion method for video," IEEE Asilomar conference on Signals, Systems and Computers pp. 47, 2012.
[38] \Youtube" Available: https://www.youtube.com/, 2014.
[39] \Video Enhencer Version 1.9.10 [Online]" Available:
http://www.infognition.com/VideoEnhancer/, 2014.
[40] G. Zongliang, \Low Complexity Image and Video Super Resolution Using Edge and Nonlocal Self-Similarity Constraint," IEICE Transactions on Information and Systems, vol. 96, no. 7, pp. 1569-1572, July 2013