本論文提出一個可在智慧型手機上快速運作的手掌辨識、指尖辨識與追蹤方法，可用於建構即時的虛擬觸控系統。系統的輸入為使用智慧型手機HTC One(m8)擷取之連續的深度影像。本虛擬觸控系統的操作概念是以單手在鏡頭前張開作為系統啟動的手勢後，即開始進行指尖的觸控偵測。因此本系統主要可分為三個階段：開掌手勢偵測、指尖點的偵測追蹤、觸控點的偵測以及對於觸控點的軌跡進行分析。先以擷取之深度影像取出鏡頭前物件，以快速且低資源需求的幾何方法迅速識別開掌之手勢，同時將手分為手臂、手掌、手指三部分。隨後開始於深度影像上尋找端點作為可能的指尖點，以手掌與手臂的位置可對指尖點進行過濾以取得正確之指尖點。最後便可對指尖點進行追蹤並做觸控點的判斷，取得觸控點後便可回傳觸控位置至手機上的應用程式加以分析。可分析手勢包含單指之輕觸、拖曳與長按，雙指之旋轉與縮放，五指之抓取。手勢在擷取分析後可立即提供參數給應用程式執行對應動作。本方法使用較少的運算量且不需大量記憶體，其快速的處理速度使得將來在手機上以手勢於空中操作觸控系統的概念得以實現。

This thesis presents a virtual touchpad system for smartphone by using image processing technology based on the depth image captured by HTC One (m8). This system consists of three modules: (1) open palm gesture recognition, (2) active fingertip detection and tracking, and (3) finger gesture recognition. To recognize the open palm gesture, we separate the hand region into three parts: arm, palm, and fingers, and find the fingertips. Then, we locate the position of the fingertips in 3D space to detect the active fingertips. Finally, the trajectory of the active fingertips can be analyzed to recognize the finger gestures. This method can operate in real-time with high precision for implementing the virtual touchpad system.

Contents
Chapter 1 Introduction 6
1.1 Motivation 6
1.2 System Overview 6
1.3 Experiment Environment Error! Bookmark not defined.
1.4 Contribution Error! Bookmark not defined.
1.5 The organization of this thesis Error! Bookmark not defined.
Chapter 2 Related works 11
Chapter 3 Handshape Detection 13
3.1 Threshold selection 14
3.2 Distance Transform 15
3.3 Open palm detection 16
Chapter 4 Finger Gesture Recognition for Virtual Touchpad 23
4.1 Active Fingertip Detection 24
4.2 Active Fingertip Tracking 25
4.3 On-click Gesture detection 28
4.4 Finger Gesture Detection 30
Chapter 5 The Experimental Results 34
5.1 2D Fingertip detection 34
5.2 Open-hand detection 36
5.3 Fingertip tracking 37
5.4 On-click gesture detection 38
5.5 On-press gesture detection 39
5.6 Finger Gesture Gesture Recognition and Precision Analysis 40
5.7 Processing Time 46
Chapter 6 Conclusion 48
References Error! Bookmark not defined.

1. P. Kakumanu, S. Makrogiannis, and N. Bourbakis., "A survey of skin-color modeling and detection methods," Pattern Recognition, no. 40(3), p. 1106–1122, 2007.
2. M. Jones and J. Rehg., "Statistical color models with application to skin detection," in CVPR, 1999.
3. Jones, M.J.and Rehg, J.M., "Statistical Color Models with Application to Skin Detection," International Journal of Computer Vision 46(1), p. 81–96, 2002.
4. D. Mohr and G. Zachmann, "Segmentation of Distinct Homogeneous Color Regions in Images," LNCS 4673, p. 432–440, 2007.
5. L. Sigal, S. Sclaroff, and V. Athitsos, "Skin color-based video segmentation under time-varying illumination," IEEE Trans. on PAMI, no. 26(7), p. 862–877, 2004.
6. Cheng Li, Kris M. Kitani, "Pixel-level Hand Detection in Ego-Centric Videos," CVPR, 2013.
7. Jun Wan, Qiuqi Ruan, Gaoyun An, and Wei Li, "Hand Tracking and Segmentation via Graph Cuts and Dynamic Model in Sign Language Videos," ICSP, pp. 1135-1138, 2012.
8. Omer Rashid and Ayoub Al-Hamad, "Flow Modeling and Skin-based Gaussian Pruning to Recognize Gestural Actions using HMM," in ICOR, Tsukuba, Japan, 2012.
9. Yining Deng, and B.S. Manjunath, "Unsupervised Segmentation of Color-Texture Regions in Images and Video," IEEE Trans. on PAMI, Vols. Vol. 23, No.8, August 2001.
10. Chen Qian, Xiao Sun, Yichen Wei, and Xiaoou Tang, "Realtime and Robust Hand Tracking from Depth," CVPR, 2014.
11. Qunqun Xie, Guoyuan Liang, Cheng Tang, and Xinyu Wu, "A Fast and Robust Fingertips Tracking Algorithm for Vision-Based Multi-touch Interaction," in 10th IEEE International Conference on Control and Automation, Hangzhou, China, 2013.
12. Z. Mo and U. Neumann, "Real-time hand pose recognition using lowresolution depth images," Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference, vol. 2, p. 1499–1505, 2006.
13. D. Schlegel, A. Chen, C. Xiong, J. Delmerico, and J. Corso, "Airtouch: Interacting with computer systems at a distance," Applications ofComputer Vision (WACV), 2011 IEEE Workshop, pp. 1-8, 2011.
14. Paul F. Whelan and Derek Molloy, Machine Vision Algorithms in Java: Techniques and Implementation, Springer Science & Business Media, 2000.
15. Nikolaos Oikonomidis, Iason. Kyriazis and Argyros Antonis, "Efficient model-based 3d tracking of hand articulations using kinect.," BMVA Press, pp. 101.1-101.11, 2011
16. Omer Rashid and Ayoub Al-Hamad, "Flow Modeling and Skin-based Gaussian Pruning to Recognize Gestural Actions using HMM," 21st Int. Conf. on Pattern Recognition (ICPR 2012), Tsukuba, Japan, 2012.
17. Yining Deng, and B.S. Manjunath, Member, IEEE, "Unsupervised Segmentation of Color-Texture Regions in Images and Video," IEEE Trans. on PAMI, Vols. Vol. 23, No.8, August 2001.