在這篇論文中，我們提出了一套基於階層式手部解析的手勢辨識演算法來對單張深度影像進行手勢辨識。我們所提出的方法，首先從原圖中擷取出手部區域的三維資料點，並透過計算手的方向與垂直軸之間的轉換，以光軸為轉軸，將手的姿勢做旋轉歸一化。我們根據定義好的手部組態，將整個手部切分成十一個彼此不重疊的子區塊，並利用深度範圍特徵訓練出三階層的隨機森林分類器，手部像素點將參考分類器計算出的後驗機率來決定其所屬的子區塊。在第一層中將利用分類器判別像素點是否屬於手掌區塊，接著在第二層中，進一步辨識非手掌類別中的像素點所屬的手指類別，最後在第三層中，將會辨識出位於手指的像素點是屬於指尖區塊還是手指根部區塊。最後，解析完成的手部資訊將會被進一步利用來組成三種特徵，包含手部姿勢特徵、手指角度特徵、手部區塊比例特徵，並透過支持向量機器來達到手勢辨識的目的。在實驗部分，我們將提出的方法分成手部解析以及手勢辨識兩大部分，利用不同的手勢真實影像資料庫來呈現我們方法的效能。

In this thesis, we proposed a hand gesture recognition algorithm based on hierarchical hand parsing from a single depth image. In the proposed system, we first normalize in-plane rotation of the hand pose. According to hand configuration, we propose to segment a hand into 11 non-overlapping parts with a novel 3-layer hierarchical Random Decision Forest (RDF) per-pixel classifier. In the first layer, the hand region is divided into two parts: palm and fingers. In the second layer, pixels are classified into different finger classes: thumb, index finger, middle finger, ring finger and pinky finger. In the third layer, a finger pixel is classified into upper and lower part. In each layer, per-pixel classification is executed to assign a set of posterior probabilities corresponding to different hand parts to each pixel based on depth-context features. To develop hand gesture recognition, the information of parsed hand is employed to compute three kinds of features including posture feature, finger angle feature and hand part ratio feature, for Support Vector Machines (SVMs). Our experiments show superior performance of hand parsing and gesture recognition by using the proposed algorithm compared to some previous methods on different real hand pose datasets.

Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Problem Description 2
1.3 Main Contribution 3
1.4 Thesis Organization 4
Chapter 2 Previous Works 5
2.1 Model-based approach 5
2.2 Appearance-based approach 6
Chapter 3 Proposed Hand Gesture Recognition System 8
3.1 System overview 8
3.2 Pre-processing work 9
3.3 Hand partition model 10
3.4 Hierarchical hand parsing framework 11
3.4.1 Hierarchical hand partition model 12
3.4.2 Training multiple random forests for hand parsing classification 13
3.4.3 Hierarchical per-pixel hand parsing process 16
3.5 Hand gesture recognition using the hand parsing information 19
3.5.1 Hand gesture feature 19
3.5.2 Mechanism for missing hand part 21
Chapter 4 Experimental Results 23
4.1 Our hand depth image dataset 23
4.2 RF-based hand parsing 26
4.3 Hand gesture recognition 31
4.4 Experiment on public datasets 33
Chapter 5 Conclusion 37
References 38

[1] C. Oz, and M. C. Leu. Recognition of finger spelling of American Sign Language with artificial neural network using position/orientation sensors and data glove, International Symposium on Neural Networks, pp. 157-164, 2005.
[2] C. Oz, and M. C. Leu. Linguistic properties based on American sign language recognition with artificial neural networks using a sensory glove and motion tracker. Computational Intelligence and Bioinspired System, Neurocomputing, vol. 70, no. 16–18, pp. 2891-2901, 2007.
[3] P. Kumar, Verma J, Prasad S. Hand data glove: a wearable real-time device for human–computer interaction, International Journal of Advanced Science and Technology, vol. 43, pp. 15-25, 2012.
[4] D. Kim, O. Hilliges, S. Izadi, AD. Butler, J. Chen, I. Oikonomidis, P. Olivier. Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. Proceedings of the 25th annual ACM symposium on User Interface Software and Technology, pp. 167–76, 2012.
[5] M. de La Gorce, Martin, D. J. Fleet, and N. Paragios. Model-based 3d hand pose estimation from monocular video, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(9):1793-1805, 2011.
[6] S. Mitra and T. Acharya. Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, 37:311–324, 2007.
[7] V. I. Pavlovic, R. Sharma, and T. S. Huang. Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19:677–695, 1997.
[8] J. Suarez and R.R. Murphy. Hand gesture recognition with depth images: A review. IEEE International Symposium on Robot and Human Interactive Communication, pp. 411-417, 2012.
[9] C.-S. Fahn and K.-Y. Chu. Hidden-Markov-model-based hand gesture recognition techniques used for a human-robot interaction system, Human-Computer Interaction: Interaction Techniques and Environments. vol. 6762, pp. 248-258, 2011.
[10] Y. Yao and Y. Fu. Contour model-based hand gesture recognition using the Kinect Sensor, IEEE Transactions on Circuits and Systems for Video Technology, 24 (11): 1935–1944, 2014.
[11] L. Oikonomidis, N. Kyriazis, and A. A. Argyros. Markerless and efficient 26-DOF hand pose recovery. British Machine Vision Conference, 2011.
[12] P. Gurjal, and K. Kunnur. Real time hand gesture recognition using SIFT. International Journal of Electronics and Electrical Engineering, Vol. 2, Issue 3, pp 19-33, 2012.
[13] N. Pugeault, and R. Bowden. Spelling it out: Real-time ASL fingerspelling recognition. IEEE Workshop on Consumer Depth Cameras for Computer Vision, pp. 1114-1119, 2011.
[14] C. R. Mihalache, B. Apstol. Hand pose estimation using HOG features from RGB-D data. System Theory, Control and Computing, pp. 356-361, 2013.
[15] G. Marin, F. Dominio, and P. Zanuttigh. Hand gesture recognition with leap motion and Kinect devices. IEEE International Conference on Image Processing, pp. 1565–1569, 2014.
[16] C. Keskin, F. Kirac, Y. E. Kara, and L. Akarun. Real time hand pose estimation using depth sensors. IEEE International Conference on Computer Vision Workshops, pp. 1228-1234, 2011.
[17] H. Liang, J. Yuan and D. Thalmann. Parsing the hand in depth images. IEEE Transactions on Multimedia, vol. 16, no. 5, pp. 1241-1253, 2014.
[18] H. Liang and J. Yuan. Hand parsing and gesture recognition with a commodity depth camera. Computer Vision and Machine Learning with RGB-D Sensors, Springer, pp. 239-265, 2014.
[19] C. Cortes, and V. Vapnik. Support-vector networks. Machine learning, 20(3): 273-297, 1995.
[20] D. Comaniciu, and P. Meer. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 603-619, 2002.
[21] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake. Real-time human pose recognition in parts from single depth images. Computer Vision and Pattern Recognition, pp. 1297-1304, 2011.
[22] A. Memo, L. Minto, P. Zanuttigh. Exploiting silhouette descriptors and synthetic data for hand gesture recognition. STAG: Smart Tools & Apps for Graphics, 2015.
[23] Z. Ren, J. Yuan, and Z. Zhang. Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. Proc. of ACM Multimedia, pp. 1093-1096, 2011.
[24] Z. Ren, J. Yuan, J. Meng, and Z. Zhang. Robust part-based hand gesture recognition using Kinect sensor. IEEE Transactions on Multimedia, vol. 15, no. 5, pp. 1110–1120, Aug. 2013.
[25] S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24:509–522, 2002.
[26] X. Bai and L. J. Latecki. Path similarity skeleton graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, pp. 1–11, 2008.