在此篇論文中，我們提出了兩步驟-監督式人臉超解析度技術 (Supervised face hallucination) 處理方法，在此架構下，可讓低解析率（Low-resolution, LR）的輸入人臉影像經更有效處理被處理成高解析率（High-resolution, HR）影像。
這篇論文的主要工作框架主要分成為：經全局的人臉估計及區域的人臉的修正並結合分類方法選擇基底 (Selection bases step) 重建人臉影像，並修正人臉的局部細節 (Local facial-parts refinement)。本研究利用在支持向量機器 (Support vector machine, SVM) 和標籤信息 (Label information) 做人臉辨識做監督式資料分類 ( Supervised learning)，並將分類後的每群資料建立其對應的基底 (Global and local bases)，其中為了改善人臉超解析度技術的重建效果，在輸入低解析度人臉影像的重建過程中，利用人臉識別找出和輸入低解析度影像，擁有相似特性之全域及區域基底 (Global and local bases) ，再找出相似特性對應的基底後，以改進重建效果。於全局人臉估計過程中使用人臉識別挑選較佳的全域基底 (Global bases) 後，使用最大後驗 (Maximum a posteriori, MAP) 估計方法，在低維空間 (low-dimensional coefficient domain) 估計出最佳化的重建係數，並利用全域基底 (Global bases) 和最佳化的重建係數做線性組合重建全域的人臉高解析度影像。至於在修正人臉的局部細節此一步驟，我們選擇同樣使用人臉識別找出接近類似對應的區域基底 (local bases)，即超完備非負矩陣分解 (Overcomplete nonnegative matrix factorization, ONMF) 基底來重建人臉的局部細節。
經實驗結果證明此改進的此兩步驟-監督式人臉超解析度技術的架構，可有效處理的人臉超解析度技術的問題，成功的造就從輸入低解析率的人臉影像構建成高解析率 (High-resolution, HR) 人臉影像，不僅可有效提升視覺效果 (Visual quality)，實驗也使用人臉辨識 (Face recognition) 當作客觀評估之標準，驗證其重建之人臉影像的視覺品質和人臉辨識率之結果更由優於目前現今所有主流的人臉超解析度技術。

This thesis presents an improved two-step supervised face hallucination framework termed from the input low-resolution (LR) face image to the high-resolution (HR) image. To solve the special facial problem, we propose a novel face hallucination using Bayesian global estimation, local basis selection with support vector machine (SVM) and label information to achieve supervised learning for constructing super-resolution (SR) frontal images from the input LR face image. This proposed framework mainly consists of two steps: the global estimation step and the local facial-parts refinement using selection local bases selection step. In order to improve the face hallucination performance, we further employ face recognition (SVM) to find the similar face structure bases (global and local bases) as an input face image. In the global estimation, we use face recognition to select global/PCA bases and adopt a maximum a posteriori (MAP) estimator to estimate the optimum set of coefficients in the low-dimensional domain for hallucinating HR face image via a linear combination of the global bases. In the local refinement step, we use face recognition to select local/overcomplete nonnegative matrix factorization (ONMF) bases to refine the facial parts (i.e. eyes, nose and mouth). Experimental results show that our improved framework can effectively enhance visual effects and demonstrate that the good performance of our approach with face recognition is justified in that our reconstruction results are better than those produced by the other hallucination methods, such as visual quality and objective quality assessment (face recognition).

摘 要
Abstract
Content
Chapter 1 Introduction
1.1 Research Background
1.2 Motivation and Objective
1.3 Thesis Organization
Chapter 2 Related Work
2.1 Example-based Approach
2.1.1 Prototype-based
2.1.2 Model-based
2.1.3 Sparse Representation
2.2 Statistical model approach
2.2.1 Pyramid-based
2.2.2 Mapping-based
Chapter 3 Proposed Method
3.1 Overview of Proposed Method
3.2 Bayesian Global Face Estimation
3.3 Local Refinement using Clustering Local Bases
Chapter 4 Experiments and Discussion
4.1 Performance Evaluation
4.1.1 Database and Settings
4.1.2 Experiment on Visual quality
4.1.3 Experiment on Recognition Rate
4.1.4 Experiment on Objective Assessment
Chapter 5 Conclusion
References

[1] B. Baker, and T. Kanade, “Limits on super-resolution and how to break them,” IEEE Trans. Pattern Anal. Match. Intell, vol. 24, no. 9, pp. 1167–1183, 2002.
[2] G. S. Huang, R. Hu, Z. Han, T. Lu, J. Jiang, and F. Wang, “Face image superresolution via locality preserving projection and sparse coding,” Journal of Software, vol. 8, no. 8, pp. 2039-2046, 2013.
[3] J. L. Harris, “Diffraction and Resolving Power,” Journal of the Optical Society of America, vol. 54, no. 7, pp. 931-936, 1964.
[4] J. W. Goodman, Introduction to Fourier Optics, New York: McGraw-Hill, 1968.
[5] R. Y. Tsai, and T. S. Huang, “Multiple frame image restoration and registration,” advances in computer vision and image processing. Greenwich, Greenwich, CT: JAI Press Inc., 1984.
[6] W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based superresolution,” IEEE Computer Graphics & Applications, vol. 22, no. 2, pp. 56–65, 2002.
[7] C. Liu, H. Y. Shum, and C. S. Zhang, “A two-step approach to hallucinating faces: global parametric model and local nonparametric model,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit, vol. 1, pp. 192–198, 2001.
[8] X. Wang, and X. Tang, “Hallucinating face by eigentransformation,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 35, no. 3, pp. 425-434, 2005.
[9] J. S. Park, and S. W. Lee, “An example-based face hallucination method for single-frame, low-resolution facial images,” IEEE Trans Image Process, vol. 17, no. 10, pp. 1806-16, Oct, 2008.
[10] H. H. Chang, D. Yeung, and Y. Xiong, “Super-resolution through neighbor embedding,” Proc. 2004 Computer Vision and Pattern Recognition, pp. 275-282, 2004.
[11] W. Fan, and D. Y. Yeung, “Image hallucination using neighbor embedding over visual primitive manifolds,” Proc. 2007 Computer Vision and Pattern Recognition, pp. 1-7, 2007.
[12] T. Lu, R. Hu, Z. Jiang, and J. Chang, “Face hallucination based on sample selection bias correction,” International Journal of Advancements in Computing Technology, vol. 4, pp. 91-98, 2012.
[13] J. C. Yang, S. W. Ma, and T. Huang, “Face hallucination via sparse coding,” Proc. IEEE Int. Conf. Image Process., pp. 1264-1267, 2008.
[14] L. Chang, and M. Zhou, “Face sketch synthesis via sparse representation,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit, pp. 2146-2149, 2010.
[15] C. A. Jung, L. Jiao, and M. Giong, “Position-patch based face hallucination using convex optimization,” IEEE Signal Processing Letters, vol. 18, no. 6, pp. 367-370, 2011.
[16] X. Zhang, S. Peng, and J. Jiang, “An adaptive learning method for face hallucination using locality preserving projections,” 8th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 1-8, 2008.
[17] X. Jin, J. Bao, and J. Du, “Image enhancement based on selective - retinex fusion algorithm,” Journal of Software, vol. 7, no. 6, pp. 1187-1194, 2012.
[18] Y. Peng, A. Ganesh, J. Wright, W. Xu, and Yi Ma, “Robust Batch Alignment of Images by Sparse and Low-Rank Decomposition” IEEE Trans. Pattern Anal. Match. Intell, 2012.
[19] C. C. Hsu, C. W. Lin, C. T. Hsu, and H. Y. Mark Liao, “Face hallucination using Bayesian global estimation and local basis selection,” in Proc. IEEE Workshop Multimedia Signal Processing (MMSP) , Saint-Malo, France, 2010.
[20] W. Zhang, and W. K. Cham, “Learning-based face hallucination in DCT domain,” Proc. IEEE Conf. Comput. Vis. Pattern Recognit, 2008.
[21] C. Liu, H. Y. Shum, and W. T. Freeman, “Face hallucination: theory and practice,” Int. J. Comput. Vis.,, vol. 75, no. 1, pp. 115–134, 2007.
[22] S. W. Park, and M. Savvides, “Breaking the limitation of manifold analysis for super-resolution of facial images,” Proc. IEEE Int. Conf. Acoustics, Speech Signal Process., vol. 1, pp. I-573–I-576, 2007.
[23] T. P. Zhang, B. Fang, Y. Y. Tang, and G. H. He, “Topology preserving non-negative matrix factorization for face recognition,” Image Process, vol. 17, no. 4, pp. 574–584, 2008.
[24] A. Hyvärinen, J. Hurri, and P. O. Hoyer, Ch 13: Overcomplete and nonnegative models: Springer, 2009.
[25] J. Eggert, and E. Korner, “Sparse coding and NMF,” Proc. IEEE Int. Joint Conf. Neural Networks, vol. 4, pp. 2529-2533, 2004.
[26] W. W. Zou, and P. C. Yuen, “Very low resolution face recognition problem,” IEEE Transactions on Image Processing, 2012.
[27] Y. Li, C. Q. Cai, G. Quiu, and K. M. Lam, “Face hallucination based on sparse local-pixel structure,” Pattern Recognition, vol. 47, pp. 1261-1270, 2014.W. T. Freeman, E. C. Pasztor, O. T. Carmichael, “Learning Low-level Vision,”Int. J. Comput. Vis.,, vol. 40, no. 1, pp. 25–47, 2000.
[28] http://www.csie.ntu.edu.tw/~cjlin/libsvm/
[29] G. Cristóbal, E. Gil, F. Sroubek, J. Flusser, C. Miravet, and F. Rodrıa-cute; guez, “Superresolution imaging: A survey of current techniques,” in Proc. Adv. Signal
Process. Algorithms, Architectures, Implementations XVIII, 2008, vol. 7074, pp. 0C1–0C18, 2008.
[30] G. D. Guo, Y. Fu, C. R. Dyer, and T. S. Huang, “Image-based human age estimation by manifold learning and locally adjusted robust regression,” IEEE Trans. Image Process., vol.17, no.7, pp. 1178–1188, July 2008.
[31] http://vision.ucsd.edu/content/extended-yale-face-database-b-b
[32] M. Yang, L. Zhang, J. Yang, D. Zhang, “Robust sparse coding for face recognition,”
Proc. 2010 Computer Vision and Pattern Recognition, pp. 625-632, 2010.
[33] M. Yang, L. Zhang, J. Yang and D. Zhang, “Metaface learning for sparse representation based face recognition,” Proc. IEEE Int. Conf. Image Process., pp. 1601-1604, 2010.
[34] L. Zhang, M. Yang, X. Feng, “Sparse representation or collaborative representation: Which helps face recognition?,” in Proc. IEEE Int.Conf. Comput. Vis., Nov. 2011, pp. 471–478, 2011.
[35] http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio
[36] https://ece.uwaterloo.ca/~z70wang/research/ssim/
[37] S. Baker, T. Kanade, ”Hallucinating Faces,” IEEE International Conference on Automatic Face & Gesture Recognition, pp. 83-88, 2000.