在本篇論文中，我們提出了一個手部骨架估測系統，能從單一張深度影像估測出影像中手部關節點的位置。手部骨架關節估測能廣泛運用於人機互動與手勢辨識等領域。最早有許多研究致力於人體的姿勢估測與識別，並發展出相應的感應器與套件應用於人機與體感方面；而相較於此，手勢由於單一的膚色造成特徵點不明顯，所占區域太小容易受限於環境與解析度，遮蔽與變化自由度高等困難，使得達成較為不易。目前也有基於模型或學習的方法來克服此困難。本方法結合了影像上的學習方法與主動形狀模型的技術，用於手部骨架關節估測，並實際以支持向量機測試估測結果，確實達到手勢辨識的應用。
提出的系統主要可分為兩部分：第一部分必須先針對輸入影像估測手部形狀類別，用以選擇使用相應的模型；第二部分則利用基於骨架的主動形狀模型，從初始位置開始遞迴地優化骨架模型。為此，在訓練過程中，首先會對資料庫進行隨機森林的訓練，得到能由輸入像素與影像組合的資料點預測屬於該群的機率值的多棵樹。同時也在各自群中的資料做關節位置的主成分分析與建立關節點表現模型。因此，對任一單張包含手部區域影像可由隨機森林決定出適合的主成分模型，並偵測三維骨架模型。
在實驗結果中，我們透過客觀的數據來展示提出的系統能夠有效地偵測關節點位置，並可運用於手勢辨識。

In this thesis, we propose a novel 3D hand skeleton estimation system that can estimate the positions of hand joints from a single depth image. The hand skeleton estimation can be widely used in the fields of Human Computer Interface (HCI) and gesture recognition. Numerous researches on depth sensors have been endeavored with applications in these domains. However, the monotonous skin color, self-occlusions, view variations and high degree of freedom are the difficulties for 3D hand skeleton model estimation and gesture recognition from color or RGBD images. Currently, the model-based and discriminative methods have been proposed for solving these problems. In this work, we combine the vision-based learning and Active Shape Model approaches for 3D hand skeleton estimation from a single depth image.
The proposed approach is decomposed into two principal steps: the first part is to select the corresponding ASM model from the depth image, and the second part uses the skeleton-based ASM to iteratively refine the joint positions. In the training phase, we first build a random forest from a dataset of annotated hand depth images, which are clustered via K-means algorithm. With the random forest, we can compute the probability of each cluster for the input data point. Meanwhile, the PCA skeleton models and joints profile models are constructed for each cluster. Thus, for an input hand depth image, the system first determines the associated ASM model from random forest and then estimates the 3D hand skeleton model with a modified ASM fitting process. Our experiments demonstrate the effective 3D hand skeleton estimation by using the proposed algorithm for quantitative evaluations.

Table of Content
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Problem Description 2
1.3 Main Contribution 3
1.4 Thesis Organization 4
Chapter 2 Previous Works 5
2.1 Model-based approach 5
2.2 Discrimination based approach 6
2.3 Gesture recognition 7
Chapter 3 Proposed hand skeleton estimation System 9
3.1 System overview 9
3.2 Hand skeleton model 10
3.3 Hand shape clustering 11
3.3.1 Training a random forest for hand shape clustering 12
3.3.2 Hand shape clustering in estimation phase 14
3.4 Multiple hand skeleton PCA models and joints appearance models 15
3.4.1 Multiple PCA models 16
3.4.2 Joints appearance models 19
3.5 Hand skeleton estimation process 20
3.6 Gesture recognition using 3D hand skeleton information 23
Chapter 4 Experimental Results 26
4.1 Introduction of dataset 27
4.2 Random forest for hand shape clustering 27
4.3 3D hand skeleton estimation 30
4.4 Gesture recognition 35
Chapter 5. Conclusion 37
References 38

[1] R.-Y. Wang, Y. Robert, and J. Popović. Real-time hand-tracking with a color glove. ACM Transactions on Graphics (TOG), 28(3):63, 2009.
[2] M. de La Gorce, Martin, D. J. Fleet, and N. Paragios. Model-based 3d hand pose estimation from monocular video. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 33(9):1793-1805, 2011.
[3] I. Oikonomidis, M. Lourakis, and A. A. Argyros. Evolutionary Quasi-Random Search for Hand Articulations Tracking. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[4] C. Keskin, F. Kıraç, Y. E. Kara and L. Akarun. Hand pose estimation and hand shape classification using multi-layered randomized decision forests. European Conference on Computer Vision (ECCV), pages 852-863, 2012.
[5] D. Tang, H.-J. Chang, A. Tejani and T.-K. Kim. Latent regression forest: Structured estimation of 3d articulated hand posture. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[6] I. Oikonomidis, N. Kyriazis, and A. A. Argyros. Tracking the articulated motion of two strongly interacting hands. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
[7] D. Tang, T.-H Yu, and T. K. Kim. Real-time articulated hand pose estimation using semi-supervised transductive regression forests. IEEE International Conference on Computer Vision (ICCV), 2013.
[8] Y. Wu,, J.Y. Lin, and T. S. Huang. Capturing natural hand articulation. IEEE International Conference on Computer Vision (ICCV), 2001.
[9] I. Oikonomidis, N. Kyriazis, and A. A. Argyros. Efficient model-based 3D tracking of hand articulations using Kinect. British Machine Vision Conference (BMVC), Vol. 1, No. 2, 2011.
[10] S. Sridhar, A.Oulasvirta, and C. Theobalt. Fast Tracking of Hand and Finger Articulations Using a Single Depth Camera. 2015.
[11] I. Oikonomidis, N. Kyriazis, and A. A. Argyros. Full dof tracking of a hand interacting with an object by modeling occlusions and physical constraints. IEEE International Conference on Computer Vision (ICCV), 2011.
[12] J. Shotton, T. Sharp, A. Kipman and A. fitzgibbon. Real-time human pose recognition in parts from single depth images. Communications of the ACM, 56(1):116-124, 2013.
[13] C. Keskin, F. Kira , Y. Kara and L. Akarun. Real time hand pose estimation using depth sensors. In Proceeding of the IEEE International Conference on Computer Vision Workshops, pp.1228 -1234, 2011.
[14] M. Elmezain, A. Al-Hamadi, J. Appenrodt and B. Michaelis. A hidden markov model-based continuous gesture recognition system for hand motion trajectory. International Conference on Pattern Recognition (ICPR), 2008.
[15] M. Elmezain, A. Al-Hamadi, and B. Michaelis. Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences. WSCG Journal, Vol. 16(1), pp. 65-72, 2008.
[16] Y. Yuan, and K. Barner. An active shape model based tactile hand shape recognition with support vector machines. IEEE Annual Conference on Information Sciences and Systems, 2006.
[17] T. F. Cootes, and C. J. Taylor. "Active shape models—‘smart snakes’. British Machine Vision Conference (BMVC), Springer London, pages 266-275, 1992.
[18] C. Cortes, and V. Vapnik. Support-vector networks. Machine learning, 20(3): 273-297, 1995.
[19] R. Szeliski. Computer Vision: Algorithms and Applications. Springer Science and Business Media, 2010.