影片/影像縮放是影像處理與電腦視覺的研究領域中，相當具有知名度的技術，這門技術的功能是將影片/影像縮放至我們指定的長與寬的比例，同時保持影片/影像中重要物件的形狀與結構。由於播放裝置的快速成長，在各式裝置上觀看多媒體資料的情形日益普遍，如手機、平板電腦、電視機等，用於將影片/影像改變為符合播放解析度的技術，成為一樣相當有用的工具。
在過去的研究中，保持物體的影像縮放技術，已經可以產生令人滿意的視覺效果。然而，在影片縮放方面，物件形狀與時間軸的一致性都需要保存，因此影片的縮放技術比起影像來說更加困難。直接將影像縮放技術使用在影片上會造成不自然的晃動，在播放時，觀看者會發現明顯的畫面不連續。過去的技術會取出並使用影片的整體資訊，以保持播放影片時，時間軸上的一致性。然而，這些做法需要許多暫存器存取影片中的所有幀，成本也大幅增加。
在這篇論文中，我們提出了有限資源下逐幀運算的演算法。首先，為了減少使用的暫存器數量，我們的做法是於兩個幀之間進行縮放，而不是對整個影片進行處理，再者，我們在處理當前幀時，只會使用到已最佳化處理與已播放的前一個幀的結果。實驗結果顯示即便在資源的限制下，我們的方法比起過去方法，仍有相當好的結果。

Image/video retargeting is a well-known technique in image processing and computer vision. This technique retargets an image/video to a desired aspect ratio, while simultaneously retain the shape and structure of important objects. Due to the development of display devices, displaying media contents in various devices, such as smart phones, TV, and Tablets, is getting common, and image/video retargeting technique becomes a useful tool.
Content-aware image retargeting has been proven to produce satisfying result. However, in video retargeting, both important content and temporal consistency should be preserved. As a result, video retargeting is a more complicate task in comparison with image retargeting. Extending the current image retargeting technique to individually resize video frames may cause jittering artifacts, leading to noticeable discontinuity when playing videos. Many approaches utilize global information of an entire video frame to preserve temporal coherence. However, in implementation, these methods need a number of buffers to save frames, which is comparably expensive.
In our method, we propose a resource-limited frame-by-frame algorithm of video retargeting. First, in order to reduce frame buffer of usage, instead of optimizing over the video cube, we perform our optimization in a frame-by frame manner. Second, in the process of resizing current frame, our method only considers the information of previous frame, which is already optimally deformed and streamed. Experiment shows that our proposed method produces promising results compared to previous works, even under limitation of resources.

摘 要 i
Abstract ii
Content iii
Chapter 1 Introduction 5
Chapter 2 Related Work 8
2.1 Image Retargeting 8
2.1.1 Discrete method 8
2.1.2 Continuous method 9
2.2 Video Retargeting 10
2.2.1 Discrete method 10
2.2.2 Continuous method 10
Chapter 3 Proposed method 12
3.1 Overview 12
3.2 Initialization 13
3.3 Optimized Video Retargeting 15
3.3.1 Motion Estimation 15
3.3.2 Optimization 20
Chapter 4 Experiments and Discussion 25
4.1 Performance Evaluation 25
4.1.1 Qualitative comparisons 26
4.1.2 User Study 28
4.2 Limitations 31
Chapter 5 Conclusion 32
References 33

[1] M. Nishiyama, T. Okabe, Y. Sato, and I. Sato, “Sensation-based photo cropping,” ACM Int. Conf. Multimedia, 669–672 , 2009.
[2] L. Zhang, M. Song, Yi Yang, Qi Zhao, Chen Zhao, and Nicu Sebe,“Weakly supervised photo cropping,” IEEE Trans. Multimedia, vol. 16, no. 1, Jan. 2014
[3] J. Yan, S. Lin, S.-B Kang, and X. Tang. “Learning the change for automatic image cropping.” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2013, pp. 971–978.
[4] T. Deselaers, P. Dreuw, and H. Ney, “Pan, zoom, scan—Time-coherent, trained automatic video cropping,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Jun. 2008, pp. 1–8.
[5] Z. Yuan, T. Lu, Y. Huang, D. Wu, and H. Yu, “Video retargeting: a visual-friendly dynamic programming approach,” in IEEE Int. Conf. Image Processing (ICIP), 2010.
[6] Z. Yuan, T. Lu, Y. Huang, D. Wu, and H. Yu, “Addressing visual consistency in video retargeting: A refined homogeneous approach,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 6, pp. 890–903, Jun.2012.
[7] S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph. (TOG), vol. 26, no. 3, pp. 1–10, Jul. 2007.
[8] W. Dong, N. Zhou, J.-C. Paul, and X. Zhang, “Optimized image resizing using seam carving and scaling,” ACM Trans. Graph. (TOG), vol. 29, no. 5, pp. 1–10, Dec. 2009.
[9] B. Yan, K. Li, X.-C Yang, and T.-X Hu, “Seam searching-based pixel fusion for image retargeting, ” IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 1, Jan. 2015
[10] M. Rubinstein, A. Shamir, and S. Avidan, “Improved seam carving for video retargeting,” ACM Trans. Graph. (TOG), vol. 27, no. 3, p. 16, Aug. 2008.
[11] M. Grundmann, V. Kwatra, M. Han, and I. Essa, “Discontinuous seam carving for video retargeting,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Jun. 2010, pp. 569–576.
[12] B. Yan, K. Sun, and L. Liu, “Matching area based seam carving for video retargeting,” IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 2, pp. 302–310, Feb. 2013.
[13] M. Rubinstein, A. Shamir, and S. Avidan, “Multi-operator media retargeting,” ACM Trans. Graph. (TOG), vol. 28, no. 3, p. 23, Aug. 2009
[14] W. Dong, G. Bao, X. Zhang, and J.-C. Paul, “Fast multi-operator image resizing and evaluation,” Journal of Computer Science and Technology, vol. 27, no. 1, pp. 121–134, 2012.
[15] Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, “Optimized scale-and-stretch for image resizing,” ACM Trans. Graph. (TOG), vol. 27, no. 5, p. 118,Dec. 2008.
[16] Y. Guo, F. Liu, J. Shi, Z. Zhou, and M. Gleicher, “Image retargeting using mesh parametrization,” IEEE Trans. Multimedia, vol. 11, no. 5, pp. 856–867, Aug. 2009.
[17] S. Sugimoto, S. Shimizu, H. Kimata, A. Kojima, “Multi-layered image retargeting,” in IEEE Int. Conf. Image Processing (ICIP), 2012
[18] S.-S. Lin, I.-C. Yeh, C.-H. Lin, and T.-Y. Lee, “Patch-based image warping for content-aware retargeting,” IEEE Trans. Multimedia, vol. 15, no. 2, pp. 359-368, Feb. 2013.
[19] L. Wolf, M. Guttmann, and D. Cohen-Or, “Non-homogeneous content-driven video-retargeting,” in Proc. IEEE Int. Conf. Computer Vision (ICCV), Oct. 2007, pp. 1–6.
[20] Y.-F. Zhang, S.-M. Hu, and R. R. Martin, “Shrinkability maps for content-aware video resizing,” Comput. Graph. Forum, vol. 27, no. 7, pp. 1797–1804, Oct. 2008.
[21] Y.-S. Wang, H. Fu, O. Sorkine, T.-Y. Lee, and H.-P. Seidel, “Motion-aware temporal coherence for video resizing,” ACM Trans. Graph. (TOG),vol. 28, no. 5, p. 127, Dec. 2009.
[22] Y.-S. Wang, H. Lin, O. Sorkine, and T. -Y. Lee, “Motion-based video retargeting with optimized crop-and-warp,” ACM Trans. Graph. (TOG), vol. 29, no. 4, p. 90, 2010.
[23] T.-C. Yen, C.-M. Tsai, and C.-W. Lin, “Maintaining temporal coherence in video retargeting using mosaic-guided scaling,” IEEE Trans. Image Process., vol. 20, no. 8, pp. 2339–2351, Aug. 2011.
[24] B. Li, L.-Y. Duan, J. Wang, R. Ji, C.-W. Lin, and W. Gao, “Spatiotemporal grid flow for video retargeting,” IEEE Trans. Image Process., vol. 23, pp. 1615–1628. 2014
[25] P. Krähenbühl, M. Lang, A. Hornung, and M. Gross, “A system for retargeting of streaming video,” ACM Trans. Graph. (TOG), vol. 28, no. 5, pp. 1–10, Dec. 2009.
[26] B. Yan, B. Yuan, and B. Yang, “Effective video retargeting with jittery assessment,” IEEE Trans. Multimedia, vol. 16, no. 1, pp. 272–277, Jan. 2014.
[27] L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254–1259, Nov. 1998.
[28] D. Sun, S. Roth, and M. J. Black, ”Secrets of optical flow estimation and their principles” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), June 2010.
[29] D. Panozzo, O. Weber, and O. Sorkine, “Robust image retargeting via axis-aligned deformation,” Comput. Graph. Forum, vol. 31, no. 2, pp.229–236, 2012.