本論文提出一個可以捕捉人體全身的肢體動作的系統。此系統利用微軟的深度攝影機Kinect所擷取的深度影像作為輸入，並針對影像作前處理與背景分離，接下來系統會進入三大步驟來做肢體動作的偵測。首先是特徵點的擷取，我們使用人體具有高度自由度的部位做為我們的特徵點，例如手部、雙腳、頭部。藉由系統所訓練的像素分類器，將深度影像的像素分類成各個部位，將分類的結果透過去雜訊與偵測中心的處理，產生所需要的特徵點。再來我們將特徵點轉為特徵向量並丟入動作搜尋系統，動作搜尋使用的方式是利用多元檢索樹，搜索在資料庫中對於特徵向量可能的數個動作候選。動作篩選部分是分成三個部分，第一是與主要特徵點的距離，第二是次要特徵點的懲罰函數，第三是根據時間相依性與上一張的動作去計算距離，透過這三個篩選的過程，系統會輸出最符合當前影像的動作作為系統輸出。另外本論文也提出自動化的標記特徵點的方法，使用色彩分離的方式幫助建立人體動作資料庫。在實驗上，我們證明了本系統可以達到相當程度的準確度，並且是個即時的肢體動作捕捉的系統。

　　In this thesis, we propose a real-time human full-body motion capturing system using the depth image from Kinect. Our system consists of three main steps to estimate human pose. First, we extract the characteristic landmarks on human body. By using pixel-based body part classifier, we segment the human silhouette into different body part regions. Then, we remove the outliers and extract the characteristic landmarks in the centers of body part regions. Second, we transform the landmarks to the feature vector with 3D position information. We apply the K-d tree to construct example-based system which will search several possible pose candidates. Third, we apply the voting to choose the best matching pose from candidates as the estimated pose. In experimental results, we prove that our system can operate in real-time and achieve sufficiently accuracy.

致謝 I
摘要 II
Abstract III
1. Introduction - 1 -
2. Model Database Generation - 5 -
2.1 Depth Image and Preprocessing - 5 -
2.2 Characteristic Landmarks Database - 5 -
2.3 Human Pose Database - 7 -
3. Characteristic Landmarks Extraction - 10 -
3.1 Random Forest Overview - 10 -
3.2 Random Forest Training Phase - 11 -
3.3 Characteristic Landmarks Location - 13 -
4. Human Pose Search Strategy - 17 -
4.1 Feature Vector Selection - 17 -
4.2 K-d Tree Searching - 18 -
4.3 Pose Voting - 21 -
5. Experimental Result - 24 -
5.1 Train Process - 24 -
5.2 Test Process - 25 -
6. Conclusion - 35 -
Reference - 36 -

[1] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio,, "real-time human pose recognition in parts from single depth images," In Proc. CVPR. IEEE, 2011.
[2] R. Girshick, J. Shotton, P. Kohli, A. Criminisi, A. Fitzgibbon, "Efficient Regression of General-Activity Human Poses from Depth Images," ICCV, 2011.
[3] A. Baak, M. Müller, G. Bharaj, H.-P Seidel, C. Theobalt, "A Data-Driven Approach for Real-Time Full Body Pose Reconstruction from a Depth Camera," IEEE, 2011.
[4] S. Knoop, S. Vacek, and R. Dillmann., "Fusion of 2d and 3d sensor data for articulated body tracking," Robotic and Autonomous Systems, pp. 321-329, 2009.
[5] V. Ganapathi,C. Plagemann,D. Koller,S. Thrun, "Real Time Motion Capture Using a Single Time-Of-Flight Camera," CVPR, 2010.
[6] Mao Ye, Ruigang Yang, "Real-time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera," CVPR, 2014.
[7] L. A. Schwarz, A. Mkhitaryan, D. Mateus, N. Navab, "Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow," AFGR, 2011.
[8] C. Plagemann, V. Ganapathi, D. Koller, S. Thrun, "Real-time identification and localization of body parts from depth images," ICRA, 2010.
[9] Microsoft Corp. Redmond WA., Kinect for Xbox 360..
[10] T. K.Ho, "The Random Subspace Method for Constructing Decision Forests," PAMI, 1998.
[11] T. Helten, A. Baak, G. Bharaj, M. Muller , "Personalization and Evaluation of a Real-time Depth-based Full Body Tracker," ECCV, 2012.
[12] M. Sun, P. Kohli, and J. Shotton, "Conditional Regression Forests for Human Pose Estimation," CVPR, 2012.
[13] Yen-Yu Lin, Ju-Hsuan Hua, Nick Tang, Min-Hung Chen, Hong-Yuan Liao, "Depth and Skeleton Associated Action Recognition without Online Accessible RGB-D Cameras," CVPR, 2014.
[14] Cewu Lu, Jiaya Jia, Chi-Keung Tang, "Range-Sample Depth Feature for Action Recognition," CVPR, 2014.
[15] R. Poppe, "Vision-based human motion analysis: An overview," CVIU, 2007.
[16] G. Fanelli, J. Gall, L. Van Gool, "Real Time Head Pose Estimation with Random Regression Forests," ICPR, 2010.
[17] Sebastian Handrich, Ayoub Al-Hamadi, "Upper-Body Pose Estimation Using Geodesic Distances and Skin-Color," Advanced Concepts for Intelligent Vision Systems. Springer International Publishing, pp. 150-161, 2013.
[18] R. Yang , L. Ren, and M. Pollefeys, "Accurate 3D Pose Estimation From a Single Depth Image," ICCV, 2011.
[19] Fang Wang, Yi Li, "Beyond Physical Connections: Tree Models in Human Pose Estimation," CVPR, 2013.
[20] K. Hara, T. Kurokawa, "Human Pose Estimation Using Patch-based Candidate Generation and Model-based Verification," FG, 2011.
[21] J. Charles and M. Everingham, "Learning shape models for monocular human pose estimation from the Microsoft Xbox Kinect," ICCV, 2011.
[22] N. H. Lehment, M. Kaiser, and G. Rigol, "Using Segmented 3D Point Clouds for Accurate Likelihood Approximation in Human Pose Tracking," ICCV, 2011.
[23] J. Romero, H. Kjellström and D. Kragic, "Hands in action: real-time 3D reconstruction of hands in interaction with objects," pp. 458-463, 5 2010.