本論文提出一個基於微軟的深度攝影機Kinect開發的即時手勢參數估測系統。和現在主流的兩種方法:model-based或appearance-based的方法不同的是，我們的系統結合了這兩種方法的優點，也就是可以在短時間內計算出值域連續的手部參數，並且不需要龐大的訓練資料量。本系統包含了兩個主要階段，第一階段是使用膚色偵測和邊緣偵測把從Kinect得到的彩色影像中的手部區域切割出來並進行標準化。接著使用預先訓練好的隨機森林分類器將經過極座標轉換的手部特徵進行分類，找出我們需要的幾個特徵點。在第二階段，我們將上一階段取得的特徵點轉換成特徵向量，並使用回歸森林訓練出來的回歸函式來取得所有的關節點參數，最後使用OpenGL將得到的關節角度和手指以及掌心位置建立出準確的3D虛擬手部模型。實驗上，我們會將電子手套得到的關節角度值當作基準，和本系統計算出來的角度值做比較，並統計手部在不同的距離下以及不同的手部旋轉角度對本系統的精準度造成的影響，最後提出數據證明本系統可以達到不受手部旋轉和深度距離影響。除此之外，不同的手勢在本系統的準確率和各個關節點在不同的情形下的錯誤分析也會加以討論。

In this thesis, we propose a real-time system to estimate hand pose from depth image captured by Kinect. Different from model-based or appearance-based method, our system generates continuous output in a short period of time. The system consists of two main stages: finger joint locating and joint angle computation. First, we segment the hand region from depth image, and extract some specific hand feature points by using random forest classifier. Then, we use the relative displacement of those feature points to build the new feature vector, and estimate the joint angle by regression forest. Based on the estimation results, we can emulate the hand model by OpenGL. We use data-glove as our ground truth for comparison of the estimated joint angle. The experimental results show the influence of different distance from camera and different in-plane rotation angle. We prove that our finger joint estimation system is distance-insensitive and rotation-insensitive.

1.Introduction
2.Related Work
2.1 Model-based Method & Appearance-based Method
2.2 Our Method and Comparison
3. Hand Segmentation
3.1 Background Removing
3.2 Finding Hand Centroid
3.3 Hand Feature Normalization
4.Feature Point Extraction
4.1 Random Forest Overview
4.2 Feature Region Determination
4.3 In-plane Rotation Problem
4.4 Pairwise Distance Method
5.Joint Angle Retrieving
5.1 Regression Forest
5.2 Training Stage in Regression Forest
6.Experimental Results
6.1 Training Parameters
6.2 Hand Model Rendering & Reliability Evaluation
6.3 Comparison of Regression Methods
6.4 Analysis of Time Consumption
7. Conclusion
Reference

[1] Microsoft Corp. Redmond WA. Kinect for xbox 360..
[2] V. Pavlovic, R. Sharma, and T. Huang, “Visual interpretation of hand gestures for human-computer interaction: A review,” IEEE Trans on Pattern Analysis and Machine Intelligence, vol. 19 no. 7, pp. 677-695, 1997.
[3] M. Półrola and A. Wojciechowski, “Real-Time Hand Pose Estimation Using Classifiers,” International Conference on Computer Vision and Graphics, pp. 573-580, 9 2012.
[4] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake, “Real-Time Human Pose Recognition in Parts from Single Depth Images,” CVPR, 2011.
[5] C. Xu, and L. Cheng, “Efficient Hand Pose Estimation from a Single Depth Image,” ICCV, 2013.
[6] Y. Yao, and Y. Fu, “Real-Time Hand Pose Estimation from RGB-D Sensor,” IEEE Multimedia and Expo, 2012.
[7] C. Keskin, F. Kıraç, Y. E. Kara and L. Akarun, “Real Time Hand Pose Estimation using Depth Sensors,” IEEE Int. Conference on Computer Vision Workshops, 2011.
[8] A. Erol, G. Bebis, M. Nicolescu, R. D. Boyle and X. Twombly, “Vision-based hand pose estimation: A review,” CVIU, pp. 52-73, 10 2007.
[9] I. Oikonomidis, N. Kyriazis and A. A. Argyros, “Efficient Model-based 3D Tracking of Hand Articulations using Kinect,” British Machine Vision Conference, 8 2011.
[10] L. Ballan, A. Taneja, J. Gall, L.V. Gool, and M. Pollefeys, “Motion Capture of Hands in Action Using Discriminative Salient Points,” ECCV, 2012.
[11] V. Athitsos and S. Sclaroff, “An Appearance-Based Framework for 3D Hand Shape Classification and Camera Viewpoint Estimation,” IEEE Automatic Face and Gesture Recognition, 2002.
[12] D. Tang, T-H, Yu, and T-K, Kim, “Real-time Articulated Hand Pose Estimation using Semi-supervised Transductive Regression Forests,” ICCV, 2013.
[13] R. Girshicky, J. Shottony, P. Kohliy, A. Criminisiy, and A. Fitzgibbon, “Efficient Regression of General-Activity Human Poses from Depth Images,” ICCV, 2011.
[14] H. Breu, J. Gil, D. Kirkpatrick and M. Werman, “Linear time Euclidean distance transform algorithms,” PAMI, pp. 529-533, 5 1995.
[15] L. Breiman, “Random Forests,” Machine Learning, vol. 45, pp. 5-32, 10 2001.
[16] H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola and V. Vapnik, “Support Vector Regression Machines,” Advances in Neural Information Processing Systems 9, vol. 9, pp. 155-161, 1997.
[17] A. Criminisi, J. Shotton, D. Robertson, and E. Konukoglu, “Regression Forests for Efficient Anatomy Detection and Localization in CT Studies,” Medical Image Computing and Computer Assisted Intervention Society, pp. 106-117, 2010.
[18] J. Romero, H. Kjellström and D. Kragic, “Hands in action: real-time 3D reconstruction of hands in interaction with objects,” pp. 458-463, 5 2010.
[19] H. Hamer, K. Schindler, E. Koller-Meier and L. V. Gool, “Tracking a Hand Manipulating an Object,” ICCV, pp. 1475 - 1482, 10 2009.
[20] D. Arthur and S. Vassilvitskii, “How Slow is the k-Means Method?,” Proceedings of the 21th annual symposium on Computational geometry, pp. 144-153, 2006.