本研究藉由電腦視覺技術發展一套應用於籃球競賽之球員狀態數據獲取系統，此系統具有以下幾個主要功能：(1) 球員追蹤：利用人物偵測與視覺追蹤技術將每個時間點球員在場上的位置記錄下來；(2) 球員運動軌跡圖：將追蹤技術獲得不同時間點的球員位置映射到球場模型圖上，讓使用者更清楚的了解球員在場上的運動狀況；(3) 球員表現之數據呈現：統計視覺追蹤技術與姿態辨識所獲得的球員資料，並將其呈現出來供使用者做進一步的應用或分析。
本研究使用三台攝影機架設在籃球場外的三個位置，以固定視角的方式錄下比賽。將此三個不同視角的影片輸入置系統後，利用Faster R-CNN進行人物位置偵測與動作辨識，以HSV特徵與K-近鄰算法(K-Nearest Neighbor, KNN)分類器將偵測到的人物分成隊伍A、隊伍B的球員以及裁判，並使用單應性矩陣(Homography matrix)把球員位置映射到籃球場模型圖上，透過匈牙利配對演算法(Hungarian Algorithm)與卡爾曼濾波器(Kalman Filter)對場上的10位球員進行追蹤，由Faster R-CNN的動作分類結果判別各個球員於每個時間點的動作，最後將記錄到的球員位置、動作進行統計及量化數據的呈現給使用者。

This study develops a player status data acquisition system in basketball competition by using computer vision technology. This system has the following functions: (1) Player tracking: Player position which is obtained by human detection and object tracking technology is recorded in each frame; (2) Player trajectory map: Player movement trajectory will be project into the basketball court model to allow user to understand player movement status; (3) Player status data display: the player status data which is gained by player tracking and action detection will be shown for user to use in more application or analysis.
In this study, three cameras are set up at three locations outside the basketball court, recording the basketball game with fixed view. While loading these three videos into the system, Faster R-CNN will detect human position and recognize action. In order to know who is player, HSV feature will be extracted from the human images and be input to KNN classifier to classify into team A, team B and referee. And then, player position will be projected by using Homography matrix to basketball court model. Ten players on the court are tracked by utilizing Kalman filter and Hungarian algorithm. Afterward, player status data which is calculated from player position and player action will be display for user.

摘要 I
Abstract II
致謝 III
目錄 VI
圖目錄 X
表目錄 XVI
第一章 緒論 1
1.1 前言 1
1.2 研究動機 1
第二章 文獻回顧 3
2.1 人物偵測 3
2.1.1 邊緣特徵 3
2.1.2 Haar-like特徵 5
2.1.3 HOG特徵 6
2.1.4 圖型結構模型 7
2.1.5 可變形部件模型 9
2.1.6 捲積類神經網路 13
2.2 目標物追蹤 19
2.2.1 單目標追蹤 20
2.2.2 多目標追蹤 28
2.3 人物動作辨識 31
2.3.1 基於部件模型 32
2.3.2 基於深度學習 35
2.4 籃球競賽應用 38
2.4.1 籃球轉播影片之分析 38
2.4.2 多固定視角畫面之分析 40
第三章 研究方法 42
3.1 實驗裝置 43
3.2 人物位置偵測與動作辨識 45
3.2.1 Faster R-CNN 46
3.2.2 Cascade DPM 50
3.2.3 非極大值抑制 52
3.3 人物分類 53
3.4 球場模型投影 54
3.5 球員追蹤 56
3.5.1 卡爾曼濾波器追蹤 57
3.5.2 匈牙利配對法 60
3.5.3 追蹤之限制與規則 62
3.6 球員數據統計 65
3.6.1 球員運動數據 65
3.6.2 球員動作數據 67
第四章 研究結果與討論 68
4.1 人物位置偵測與動作辨識結果 68
4.1.1 人物邊界框定位 69
4.1.2 人物動作辨識 71
4.2 人物分類效果 73
4.3 球員追蹤結果 75
4.3.1 同隊伍追蹤限制 76
4.3.2 球員追失處理 77
4.3.3 錯誤偵測處理 78
4.3.4 追蹤系統成效 78
4.4 球員數據統計結果 82
4.4.1 球員運動數據呈現 82
4.4.2 球員動作數據呈現 83
4.5 球員運動軌跡紀錄 86
第五章 結論 89
5.1 本研究之貢獻 89
5.2 未來展望 90
參考文獻 92

[1] NBA.com(2 December 2015), Available : http://stats.nba.com/tracking/
[2] D. M. Gavrila and V. Philomin. “Real-Time Object Detection for Smart Vehicles.” The Proceedings of the Seventh IEEE International Conference on Computer Vision and Pattern Recognition, Vol.1, pp.87-93, 1999.
[3] D. M. Gavrila. “Pedestrian Detection from a Moving Vehicle.” The Proceedings of the sixth European Conference on Computer Vision, pp. 37-49, 2000.
[4] P. Viola and M. J. Jones. “Robust Real-Time Face Detection.” International Journal of Computer Vision, Vol. 52, No. 2, pp. 137-154, 2004.
[5] P. Viola and M. J. Jones. “Detecting Pedestrians Using Patterns of Motion and Appearance.” International Journal of Computer Vision, Vol. 63, No. 2, pp. 153-161, 2005.
[6] N. Dalal and B. Triggs. “Histograms of Oriented Gradients for Human Detection.” Proceedings IEEE International Conference on Computer Vision and pattern Recognition, Vol.1, pp. 886-893, 2005.
[7] P. Felzenszwalb and D. Huttenlocher. “Pictorial Structures for Object Recognition.” International Journal Computer Vision, Vol. 61, No. 1, pp. 55-79, 2005.
[8] P. Felzenszwalb, D. McAllester, and D. Ramanan. “A Discriminatively Trained, Multiscale, Deformable Part Model.” Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[9] P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan. “Object Detection with Discriminatively Trained Part Based Models.” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, pp. 1627-1645, 2010.
[10] The PASCAL Visual Object Classes Homepage(9 October 2016), Available : http://host.robots.ox.ac.uk/pascal/VOC/
[11] P. Felzenszwalb, R. Girshick, and D. McAllester. “Cascade Object Detection with Deformable Part Models.” IEEE International Conference on Computer Vision and pattern Recognition, pp.2241-2248, 2010.
[12] J. Yan, et al. “The Fastest Deformable Part model for Object Detection.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 2497-2504, 2014.
[13] M. A. Sadeghi and D. Forsyth. “30hz Object Detection with DPM v5.” European Conference on Computer Vision, pp. 65-79, 2014.
[14] A. Krizhevsky, I. Sutskever, and G. E. Hinton. “Imagenet Classification with Deep Convolutional Neural Networks.” Advances in neural information processing systems, pp. 1097-1105, 2012.
[15] R. Girshick, et al. “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 580-587, 2014.
[16] J. R. R. Uijlings, et al. “Selective Search for Object Recognition.” International Journal of Computer Vision, Vol. 104, Issue 2, pp. 154-171, 2013.
[17] R. Girshick. “Fast R-CNN.” IEEE International Conference on Computer Vision, pp. 1440-1448, 2015.
[18] S. Ren, et al. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” Advances in neural information processing systems, pp. 91-99, 2015.
[19] NVIDIA Tesla K40(10 October 2016), Available: http://www.nvidia.com.tw/object/tesla-workstations-tw.html
[20] D. Comaniciu, V. Ramesh, and P. Meer. “Real-Time Tracking of Non-Rigid Objects Using Mean Shift.” Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 142-149, 2000.
[21] D. Comaniciu, V. Ramesh, and P. Meer. “Kernel-Based Object Tracking.” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 5, pp.564-577, 2003.
[22] R. T. Collins. “Mean-Shift Blob Tracking Through Scale Space.” Proceedings Conference on IEEE Computer Vision and Pattern Recognition, Vol. 2, pp. 234-240, 2003.
[23] S. T. Birchfield and S. Rangarajan. “Spatiograms versus Histograms for Region-Based Tracking.” Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 1158-1163, 2005.
[24] C. Ó Conaire, N. E. O’Connor, and A. F. Smeaton. “An Improved Spatiogram Similarity Measure for Robust Object Localisation.” IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. 1, pp. 1069-1072, 2007.
[25] G. Bradski and A. Kaehler. “Learning OpenCV: Computer Vision with the OpenCV Library” O'Reilly Media Inc. Sebastopol, CA, pp. 337-341, 2008.
[26] R. Faragher. “Understanding the Basis of the Kalman Filter via a Simple and Intuitive Derivation.” IEEE Signal processing magazine, Vol. 29, pp. 128-132, 2012.
[27] K. Nummiaro, E. Koller-Meier, and L. V. Gool. “An Adaptive Color-Based Particle Filter.” Image and Vision Computing, Vol. 21, Issue 1, pp. 99-110, 2003.
[28] Z. Kalal, K. Mikolajczyk, and J. Matas. “Tracking-Learning-Detection.” IEEE transactions on Pattern Analysis and Machine Intelligence, Vol. 34, Issue 7, pp. 1409-1422, 2012.
[29] Z. Kalal, J. Matas, and K. Mikolajczyk. “P-N Learning: Bootstrapping Binary Classifiers by Structural Constraints.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 49-56, 2010.
[30] K. Zhang, L. Zhang, and M.-H. Yang. “Real-time Compressive Tracking.” European Conference on Computer Vision, pp. 864-877, 2012.
[31] Y. Cai, N. de Freitas, and J. J. Little. “Robust Visual Tracking for Multiple Targets.” European Conference on Computer Vision, pp. 107-118, 2006.
[32] C. H. Kuo, C. Huang, and R. Nevatia. “Multi-Target Tracking by On-line Learned Discriminative Appearance Models.” Conference on Computer Vision and Pattern Recognition, pp. 685-692, 2010.
[33] A. R. Zamir, A. Dehghan, and M. Shah. “GMCP-Tracker: Global Multi-Object Tracking Using Generalized Minimum Clique Graphs.” 12th European Conference on Computer Vision, pp. 343-356, 2012.
[34] G. Shu, et al. “Part-Based Multiple-Person Tracking with Partial Occlusion Handling.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1815-1821, 2012.
[35] V. Ferrari, M. Marin-Jimenez, and A. Zisserman. “Progressive Search Space Reduction for Human Pose Estimation.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[36] C. Rother, V. Kolmogorov, and A. Blake. “Grabcut: Interactive Foreground Extraction Using Iterated Graph Cuts.” ACM Transactions on Graphics, Vol. 23, pp. 309-314, 2004.
[37] Y. Yang and D. Ramanan. “Articulated Pose Estimation with Flexible Mixtures-of-Parts.” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1385-1392, 2011.
[38] C. Desai, and D. Ramanan. “Detecting Actions, Poses, and Objects with Relational Phraselets.” European Conference on Computer Vision, pp. 158-172, 2012.
[39] L. Bourdev, and J. Malik. “Poselets: Body Part Detectors Trained using 3D Human Pose Annotations.” IEEE 12th International Conference on Computer Vision, pp. 1365-1372, 2009.
[40] D. Ramanan, “Part-Based Models for Finding People and Estimating Their Pose.” Visual Analysis of Humans, pp. 199-223, 2011.
[41] G. Gkioxari, et al. “R-CNNs for Pose Estimation and Action Detection.” arXiv e-prints, 2014.
[42] G. Gkioxari, R. Girshick, and J. Malik. “Actions and Attributes from Wholes and Parts.” Proceedings of the IEEE International Conference on Computer Vision, pp. 2470-2478, 2015
[43] G. Gkioxari, R. Girshick, and J. Malik. “Contextual Action Recognition with R* CNN.” IEEE International Conference on Computer Vision, pp. 1080-1088, 2015.
[44] G. Chéron, I. Laptev, and C. Schmid. “P-CNN: Pose-Based CNN Features for Action Recognition.” IEEE International Conference on Computer Vision, pp. 3218-3226, 2015.
[45] M. C. Hu, et al. “Robust Camera Calibration and Player Tracking in Broadcast Basketball Video.” IEEE Transactions on Multimedia, Vol. 13, pp. 266-279, 2011.
[46] K. H. Chang. “Player Tracking and Tactic Analysis for Basketball TV Broadcast.” Master’s Thesis, 2012.
[47] C. W. Lu, et al. “Identification and Tracking of Players in Sport Videos.” The Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service, pp. 113-116, 2013.
[48] H. Ben Shitrit, et al. “Tracking Multiple People under Global Appearance Constraints.” IEEE International Conference on Computer Vision, pp. 137-144, 2011.
[49] F. Fleuret, et al. “Multicamera People Tracking with a Probabilistic Occupancy Map.” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, pp. 267-282, 2008.
[50] Tracking Multiple Basketball Players using Global Appearance Constraints (ICCV11) (31 August 2016), Available : https://www.youtube.com/watch?v=74vHp4ozED4
[51] C. H. Yang. “Frames Evaluating and Selecting from Multi-Camera System – A Case Study for Playing Basketball Game Video.” Master’s Thesis, 2012.
[52] CX (球學) | 長耀盃(2 December 2015), Available : http://www.choxue.com/zh-tw/leagues/6
[53] HDR-CX900 - Sony(2 December 2015), Available : http://store.sony.com.tw/product/HDR-CX900#tabs-spec
[54] Faster R-CNN(8 April 2016), Available: https://github.com/ShaoqingRen/faster_rcnn
[55] Z. Zivkovic and F. van der Heijden. “Efficient Adaptive Density Estimation per Image Pixel for the Task of Background Subtraction.” Pattern Recognition Letters, Vol. 27, pp. 773-780, 2006.
[56] Discriminatively Trained Deformable Part Models(2 December 2015), Available : http://cs.brown.edu/~pff/latent-release4/
[57] Cascade Object Detection with Deformable Part Models(2 December 2015), Available : http://people.cs.uchicago.edu/~rbg/star-cascade/