近年來由於電腦硬體的進步，各種應用軟體對三維模型的需求越來越高，如何在短時間內完成一個高品質的三維模型的建模以及編修便成了一門重要的研究課題。且關於使用者人臉模型的重建與客製化在如：VR、社群網站、雲端會議、3D列印公仔...等等相關產業的應用將會是相當重要的一環，為因應此一潛在需求，我們開發了一套能夠在短時間內讓毫無任何三維建模相關經驗的使用者完成一個屬於自己的客製化三維人臉模型的系統流程，使用者只需提供自己的正面照片，透過Facegen手動選取一些臉部特徵點後，產生較為精細之使用者三維人臉模型，而對於想全自動化且快速的產生人頭的使用者且對於人臉細節較不要求的使用者，我們有另外開發了一套利用攝影機重建使用者人臉模型的系統。在重建好人頭之後，接著利用我們所開發的客製化風格編輯系統，透過三種簡單的形變操作 (1)區塊形變(Widget-based deformation) (2)勾勒形變(Sketch-based deformation) (3)膨脹形變(Bulb-based deformation)，並且系統中內建了數種針對臉部各部位的預設風格可供使用者直接套用。藉由操作畫面上的互動式UI，選取想要誇飾化的部位，一步一步地針對人臉上的各個部位進行自己所想要的風格化操作，經過幾次操作之後，使用者就能夠產生高度個人化的三維人臉。

Recently, due to the improvement of hardware, the requirement for 3D models is getting more and more. To complete a high quality 3D model and its editing in a short term becomes an important research topic. And in the near future, the requirement for human face model reconstruction and customization will be an important part in VR industry, social websites, cloud conference, 3D printing and so on. To meet such a potential requirement, we develop a system pipeline that can let user, without any 3D modeling experience, complete their own caricature face model in a short time. User need to provide their front photo, and then manually choose facial features, reconstruct fine quality 3D face model by Facegen. And for those user who want to automatically and quickly complete the reconstruction process and don't care too much about the detail of reconstructed face, we have developed another automatic face construction system using cameras. After the reconstruction process, using the user-customized stylization and editing system by using three simple deform operations, (i) Widget-based deformation (ii) Sketch-based deformation (iii) Bulb-based deformation. And user can apply several built-in styles directly on face model. By interacting with the program, choose the face part that user wants to exaggerate, apply stylization operations on each face part on the reconstructed face model, and after several operation, a highly personalized 3D face model is completed.

中文摘要 i
Abstract ii
目錄 iii
圖目錄 v
1 續論 1
2 相關研究 4
2.1 人頭與身體重建 4
2.2 模型風格化 5
3 系統概觀 7
4 人頭風格化與編輯系統 9
4.1 分區 10
4.2 形變操作 10
4.2.1 區塊形變 11
4.2.2 勾勒形變 11
4.2.3 膨脹形變 13
4.3 編輯步驟及風格資料庫的建立 14
4.4 風格套用及使用者自定義風格 14
5 姿勢追蹤系統 16
6 風格化與最終成果 20
6.1 風格化結果 20
6.2 公仔模型成品 21
7 結論 24
7.1 結語 24
7.2 系統限制及未來規劃 24
8 附錄-自動化人頭重建系統25
8.1 二維臉部特徵點偵測 25
8.2 樣版三維臉部特徵點重建 25
8.3 人臉角度估測 27
8.4 特徵對位形變 28
8.4.1 三維特徵點校正 28
8.4.2 特徵點對位 29
8.4.3 位移形變 29
8.5 材質紋理擷取 29
8.5.1 材質擷取 30
8.5.2 色彩轉移 31
8.5.3 圖層混合 32
Bibliography 35

[1] Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’99, pages 187–194, New York, NY, USA, 1999. ACM Press/
Addison-Wesley Publishing Co. ISBN 0-201-48560-5. doi: 10.1145/311535.311556. URL: http://dx.doi.org/10.1145/311535.311556.

[2] Hao Li, Etienne Vouga, Anton Gudym, Linjie Luo, Jonathan T. Barron, and Gleb Gusev. 3d self-portraits. ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2013), 32(6), November 2013.

[3] Michael Zollhöfer, Matthias Nießner, Shahram Izadi, Christoph Rhemann, Christopher Zach, Matthew Fisher, Chenglei Wu, Andrew Fitzgibbon, Charles Loop, Christian Theobalt, and Marc Stamminger. Real-time non-rigid reconstruction using an rgb-d camera. ACM Transactions on Graphics (TOG), 33(4), 2014.

[4] E. Brennan and Susan E. Brennan. Caricature generator: The dynamic exaggeration of faces by computer. Leonardo, pages 170–178, 1985.

[5] Hong Chen, Nan-Ning Zheng, Lin Liang, Yan Li, Ying-Qing Xu, and Heung-Yeung Shum. Pictoon: A personalized image-based cartoon system. In Proceedings of the Tenth ACM International Conference on Multimedia, MULTIMEDIA ’02, pages 171–178, New York, NY, USA, 2002. ACM. ISBN 1-58113-620-X. doi: 10.1145/641007.641040. URL http://doi.acm.org/10.1145/641007.641040.

[6] Lin Liang, Hong Chen, Ying-Qing Xu, and Heung-Yeung Shum. Example-based caricature generation with exaggeration. In 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings., pages 386–393, 2002. doi: 0.1109/PCCGA.2002.1167882.

[7] Chien-Chung Tseng and Jenn-Jier James Lien. Synthesis of Exaggerative Caricature with Inter and Intra Correlations, pages 314–323. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007. ISBN 978-3-540-76386-4. doi: 10.1007/978-3-540-76386-4_29. URL http://dx.doi.org/10.1007/978-3-540-76386-4_29.

[8] Wen-Hung Liao Pei-Ying Chiang and Tsai-Yen Li. Automatic caricature generation by analyzing facial features. ACCV, 2004.

[9] Zhenyao Mo, J. P. Lewis, and Ulrich Neumann. Improved automatic caricature by feature normalization and exaggeration. In ACM SIGGRAPH 2004 Sketches, SIGGRAPH ’04, pages 57–, New York, NY, USA, 2004. ACM. ISBN 1-58113-896-2. doi: 10.1145/1186223.1186294. URL http://doi.acm.org/10.1145/1186223.1186294.

[10] Alexander Fedorov Yeong-Kyu Lim and Shin-Dug Kim. 3d caricature generation system on the mobile handset using a single photograph. ICPPW, page 37, 2007.

[11] Zhang Mingming, Liu Shoukuai, Wang Jiajun, Shen Huaqing, and Pan Zhigeng. The 3d caricature face modeling based on aesthetic formulae. In Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry,
VRCAI ’10, pages 191–198, New York, NY, USA, 2010. ACM. ISBN 978-1-4503-0459-7. doi: 10.1145/1900179.1900220. URL http://doi.acm.org/10.1145/1900179.1900220.

[12] Thomas Lewiner, Thales Vieira, Dimas Martínez, Adelailson Peixoto, Vinícius Mello, and Luiz Velho. Interactive 3d caricature from harmonic exaggeration. Computers & Graphics, 35(3):586 – 595, 2011. ISSN 0097-8493. doi: https://doi.org/10.1016/j.cag.2011.03.
005. URL http://www.sciencedirect.com/science/article/pii/S0097849311000446. Shape Modeling International (SMI) Conference 2011.

[13] Lyndsey Clarke, M. Chen, and B. Mora. Automatic generation of 3d caricatures based on artistic deformation styles. IEEE Transactions on Visualization and Computer Graphics, 17(6):808–821, June 2011. ISSN 1077-2626. doi: 10.1109/TVCG.2010.76.

[14] Andrew Nealen, Olga Sorkine, Marc Alexa, and Daniel Cohen-Or. A sketch-based interface for detail-preserving mesh editing. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH), 24(3):1142–1147, 2005.

[15] FaceGen Modeller. Singular Inversions Inc., Toronto, CANADA, 2017. URL https://facegen.com/modeller.htm.

[16] Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. Bounded biharmonic weights for real-time deformation. ACM Transactions on Graphics (proceedings of ACM SIGGRAPH), 30(4):78:1–78:8, 2011.

[17] Alec Jacobson, Daniele Panozzo, et al. libigl: A simple C++ geometry processing library, 2016. http://libigl.github.io/libigl/.

[18] Vahid Kazemi and Josephine Sullivan. One millisecond face alignment with an ensemble of regression trees. In CVPR, 2014.

[19] Davis E. King. Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10:1755–1758, 2009.

[20] M.I. A. Lourakis and A.A. Argyros. SBA: A Software Package for Generic Sparse Bundle Adjustment. ACM Trans. Math. Software, 36(1):1–30, 2009. doi: http://doi.acm.org/10.1145/1486525.1486527.

[21] S. Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(4):376–380, Apr 1991. ISSN 0162-8828. doi: 10.1109/34.88573.

[22] Erik Reinhard, Michael Ashikhmin, Bruce Gooch, and Peter Shirley. Color transfer between images. IEEE Comput. Graph. Appl., 21(5):34–41, September 2001. ISSN 0272-1716. doi: 10.1109/38.946629. URL http://dx.doi.org/10.1109/38.946629.

[23] Patrick Pérez, Michel Gangnet, and Andrew Blake. Poisson image editing. In ACM SIGGRAPH 2003 Papers, SIGGRAPH ’03, pages 313–318, New York, NY, USA, 2003. ACM. ISBN 1-58113-709-5. doi: 10.1145/1201775.882269. URL http://doi.acm.org/10.1145/1201775.882269.

[24] Itseez. Open source computer vision library. https://github.com/itseez/opencv, 2017.

[25] Satya Mallick. Opencv examples and tutorials ( c+ + / python ). http://www.learnopencv.com.