一般影格內插法中，需要以精確的對應關係描述像素間的關係。而在現代建築中，廣泛地使用易於反射光線的材質，因而產生非逐像素移動的情況，並產生錯誤的像素間關係估計值。藉由簡單的動態預測法作為輔助，我們提供一種新型之基於相位、用於非逐像素移動的影格內插方法。不同於逐像素的方法，我們將一個像素，分解並表示為許多次波段。該量在被解構後，經編碼成為對應的相位數值，再經由相較於先前方法更加精確的移動校正，以及同步等步驟之後，產生新的內插影格。
以Middlebury網站上之官方資料庫作為輸入資料，我們的方法雖然相較於Brox.等人之逐像素演算法，平均降低了0.037之PSNR，但在產生新的影格時，我們並不需使用全域最佳化，因而減少了許多計算量。而在使用的資料中存在非逐像素移動的情況，我們的方法相較於Brox.等人之方法，相對提昇了0.027的峰值信噪比。在此論文中，我們提出了一種適用於具有多重材質，高解析度、高影格數目之影格內插法。

Standard frame interpolation approaches require accurate pixel-wise correspondences between frames. Nowadays, materials that reflect light are widely applied on architectures, leading to non-pixel-wise movements in images. Wrong motion estimations are therefore induced. With simple motion estimation as an auxiliary, we offer a new phase-based approach to deal with such movements. Different from pixel-based methods, we decompose a pixel into several sub-bands. The corresponding motions are then encoded into phase values. By elaborated shift-correction and novel synchronizing strategies, interpolated frames are produced in our work.
With data sequences of pixel-wise movements from Middlebury website, the average PSNR of our method turned out to decrease by 0.037 compared to the method of Brox et al. Although our performance is worse in this case, we are capable of producing in-between frames without global optimization and are tend to cost less as a consequence.
In contrast, with data sequences of non-pixel-wise movements, objective statistics are totally different. The non-pixel-wise motions induced by multiple objects are carefully handled in the parts of shift-correction and synchronization. The average PSNR of our method increases by 0.027 compared to the method of Brox et al.[2]. In this thesis, we propose an efficient method for high-resolution frame interpolation and high frame rate videos on non-pixel-wise movements.

1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Related Work 4
2.1 The Design and Use of Steerable Filters . . . . . . . . . . . . . . . 4
2.2 Phase-based Video Motion Processing . . . . . . . . . . . . . . . . . 4
2.3 Phase-based Frame Interpolation . . . . . . . . . . . . . . . . . . . 6
2.4 Motion Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 Optical Flow Rendering . . . . . . . . . . . . . . . . . . . . . . . . 12
3 Propose Techniques for Non-pixel-wise Movements 14
3.1 Shift Correction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Shift Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2.1 Radial Synchronization . . . . . . . . . . . . . . . . . . . . . 19
3.2.2 Angular Analysis . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2.3 Angular Synchronization . . . . . . . . . . . . . . . . . . . . 25
4 Results 34
5 Discussion and Limitation 41
5.1 Wavelet Shifting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.2 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6 Conclusion 45
6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

[1] Simone Meyer, Oliver Wang, Henning Zimmer, Max Grosse, and Alexander
Sorkine-Hornung, \Phase-based frame interpolation for video," 2015 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1410{
1418, 2015.
[2] William T. Freeman and Edward H. Adelson, \The design and use of steerable
lters," IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, pp. 891{906, 1991.
[3] Hayit Greenspan, Serge J. Belongie, Rodney M. Goodman, Pietro Perona,
Subrata Rakshit, and Charles H. Anderson, \Overcomplete steerable pyramid
lters and rotation invariance," in CVPR, 1994.
[4] Eero P. Simoncelli and William T. Freeman, \The steerable pyramid: a

exible architecture for multi-scale derivative computation," in ICIP, 1995.
[5] Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Fredo Durand,
and William T. Freeman, \Eulerian video magnication for revealing subtle
changes in the world," ACM Transactions on Graphics (Proc. SIGGRAPH
2012), vol. 31, no. 4, 2012.
[6] Neal Wadhwa, Michael Rubinstein, Fredo Durand, and William T. Freeman,
\Phase-based video motion processing," ACM Trans. Graph. (Proceedings
SIGGRAPH 2013), vol. 32, no. 4, 2013.
[7] Javier Portilla and Eero P. Simoncelli, \A parametric texture model based
on joint statistics of complex wavelet coecients," International Journal of
Computer Vision, vol. 40, pp. 49{70, 2000.
47
48
[8] Piotr Didyk, Pitchaya Sitthi-Amorn, William T. Freeman, Fredo Durand, and
Wojciech Matusik, \Joint view expansion and ltering for automultiscopic
3d displays," ACM Transactions on Graphics (Proceedings SIGGRAPH Asia
2013, Hong Kong), vol. 32, no. 6, 2013.
[9] Petr Kellnhofer, Piotr Didyk, Szu-Po Wang, Pitchaya Sitthi-amorn, William
Freeman, Fredo Durand, and Wojciech Matusik, \3dtv at home: eulerianlagrangian
stereo-to-multiview conversion," ACM Trans. Graph., vol. 36, pp.
146:1{146:13, 2017.
[10] Thomas Brox, Andres Bruhn, Nils Papenberg, and Joachim Weickert, \High
accuracy optical
ow estimation based on a theory for warping," in ECCV,
2004.
[11] Simon Baker, Daniel Scharstein, J. P. Lewis, Stefan Roth, Michael J. Black,
and Richard Szeliski, \A database and evaluation methodology for optical

ow," 2007 IEEE 11th International Conference on Computer Vision, pp.
1{8, 2007.
[12] Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra,
\Overview of the h.264/avc video coding standard," IEEE Trans. Circuits
Syst. Video Techn., vol. 13, pp. 560{576, 2003.
[13] J. Ostermann, Jan Bormans, Peter List, Detlev Marpe, Matthias Narroschke,
Fernando da Cruz Pereira, Thomas Stockhammer, and Thomas Wedi, \Video
coding with h.264/avc: tools, performance, and complexity," IEEE Circuits
and Systems Magazine, vol. 4, pp. 7{28, 2004.
[14] Sing Bing Kang Harry Shum, \Review of image-based rendering techniques,"
in Visual Communications and Image Processing, 2000, vol. 4067.
[15] Steve Seitz Evan Herbst and Simon Baker, \Occlusion reasoning for temporal
interpolation using optical
ow," 2009, Microsoft Research.
[16] Taiki Fukiage, Takahiro Kawabe, and Shin'ya Nishida, \Hiding of phase-based
stereo disparity for ghost-free viewing without glasses," ACM Transactions
on Graphics (Proc. of SIGGRAPH 2017), vol. 36, no. 4, 2017.
49
[17] William T. Freeman, Edward H. Adelson, and David J. Heeger, \Motion
without movement," in SIGGRAPH, 1991.
[18] Felix Scholkmann, Jens Boss, and Martin Wolf, \An ecient algorithm for
automatic peak detection in noisy periodic and quasi-periodic signals," Al-
gorithms, vol. 5, pp. 588{603, 2012.
[19] Christopher Torrence and Gilbert P. Compo, \A practical guide to wavelet
analysis," Bulletin of the American Meteorological Society, vol. 79, no. 1, pp.
61{78, 1998.
[20] Marc Levoy, \Display of surfaces from volume data," IEEE Computer Graph-
ics and Applications, vol. 8, pp. 29{37, 1988.