在科技發達的社會裡發展了自動診斷系統來輔助專業醫生在進行判讀骨齡時能夠提供客觀的結果，在判讀的過程中會因為醫師的不同或是經驗豐富的與否造成判斷出的結果會不一樣，並透過輔助系統能夠適用於不同的醫院或地區。因此在本論文將著重於手掌腕骨的分割並運用於此類的自動診斷系統。
在論文流程中將所要檢測的手掌X光影像透過一系列的前處理，前處理的步驟包含切除背景、裁切影像尺寸和標記腕骨感興趣區域(Region of Interest, ROI)，接著使用機器學習的方式用於腕骨分割，以卷積神經網路U-Net作為實驗模型的基底，並使用多種特性對基底模型做疊加，這些特性包含殘差、循環、注意力閘和注意力機制，透過八種特徵的組合進行實驗與使用ResNeSt + U-Net [36]的目標模型進行效能比較。在加入注意力機制SENet後，從八種組合的結果中以SEAttention R2U-Net (t=3)具有最優秀的分割結果，此外在評估SEAttention R2U-Net (t=3)模型上的IOU指標達到了89.15%，F1 score指標達到了94.25%，Accuracy指標達到了99.84%，Precision指標達到了93.23%，在這些指標上都比ResNeSt + U-Net 稍微好一些，然而在Recall指標達到了95.59%則是比ResNeSt + U-Net的Recall指標96.39%差。最後將各個組合的神經網路模型分割出的結果做出分析與探討，從結果可以預期所提出的SEAttention R2U-Net (t=3)能夠成為骨齡判讀系統的一部分，並且能夠帶給醫生或著醫療相關人員更加便利且高效率的方式做出更加準確的判讀結果，並且不會因為人為因素造成誤判。

In the diagnosis process of the bone, the results may vary depending on physician’s different viewpoint or his/her experience for judgment. Therefore, in a technologically advanced society, automated diagnostic systems have been developed to assist the professional physician in providing objective results and to make the results more consistent and applicable to different hospitals or regions. Therefore, this thesis will focus on the carpal bone segmentation for use in such automated diagnostic systems.
In the thesis, the hand X-ray image is processed through a series of pre-processing steps, which include removing the background, cropping the image size and labeling the region of interest (ROI) of the wrist bone. Then, a machine learning approach, which uses the convolutional neural network U-Net as the substrate and combines with a variety of features to stack on the base model, is adopted for the carpal bone segmentation. These features include residual, recurrent, attention gate and attention mechanism. Eight combinations of the features are experimented and a ResNeSt + U-Net model [36] is used as the target model for performance comparison. After adding the attention mechanism SENet, the resulting SEAttention R2U-Net (t=3) has the best segmentation result among the eight various combinations. Moreover, the IOU indicator on the SEAttention R2U-Net (t=3) achieved 89.15%, F1 score indicator achieved 94.25%, Accuracy indicator achieved 99.84%, Precision indicator achieved 93.23%, which are all slightly better than those of ResNeSt + U-Net. However, the Recall indicator achieved 95.59%, which was worse than 96.39% of ResNeSt + U-Net. Finally, the segmentation results of each combination are analyzed and discussed. From the results, it can be expected that the proposed SEAttention R2U-Net (t=3) can become part of the bone age assessment system, which will enable doctors and medical professionals to make more accurate diagnoses in a more convenient and efficient way without human error.

摘要 I
Abstract II
致謝 III
目錄 IV
圖目錄 VI
表目錄 IX
第一章 緒論 1
1.1 研究背景 1
1.2 文獻回顧 2
1.3 研究動機/研究目的 4
1.4 論文架構 5
第二章 研究方法 6
2.1 卷積神經網路 6
2.1.1 卷積(Convolution) 6
2.1.2 池化(Pooling) 7
2.1.3 填補(Padding) 8
2.2 神經網路模型 9
2.2.1 U-Net 9
2.2.2 ResUNet 10
2.2.3 Attention U-Net 12
2.2.4 R2U-Net 14
2.2.5 ResNeSt 16
2.2.6 SENet(Squeeze-and-Excitation Networks) 19
2.2.7 SEAttentionR2U-Net 21
2.3 損失函數 22
2.3.1 Dice Loss 23
第三章 資料庫與資料前處理 24
3.1 資料庫 24
3.2 資料分割 25
3.3 資料前處理 26
3.4 資料增強(Data augmentation) 31
第四章 實驗結果與結果分析 40
4.1 實驗環境 40
4.2 實驗設計 40
4.3 實驗目標 42
4.4 錯誤評估指標 42
4.4.1 IoU(Intersection of Union) 43
4.4.2 準確率(Accuracy) 44
4.4.3 精確率(Precision) 44
4.4.4 召回率(Recall) 44
4.4.5 F1 score 44
4.5 實驗一：各神經網路架構實驗結果 45
4.6 實驗二：使用Attention注意力機制結合神經網路架構實驗結果 48
4.7 實驗三：使用SENet結合神經網路架構實驗結果 51
4.8 結果分析 55
第五章 結論與未來展望 68
5.1結論 68
5.2未來展望 69
參考文獻 70

[1] W. W. Greulich and S. I. Pyle, Radiographic Atlas of Skeletal Development of the Hand and Wrist, Stanford University Press, 1959.
[2] J. M. Tanner, R. H. Whitehouse , N. Cameron, W. A. Marshall, M. J. R. Healy and H. Goldstein , Assessment of Skeletal Maturity and Prediction of Adult Height (TW2 Method), Orlando, Fla: Academic Press, 1984.
[3] C. M. Tiu, T. C. Liu, C. W. Hsieh, D. M. Niu, J. D. Chen, T.L. Jong , "Turner syndrome phalangeal screening based on a two-stage linear regression concept," Pediatr Int, vol. 51, no. 4, pp. 453-459, Aug 2009.
[4] T. W. Todd, Atlas of Skeletal Maturation, St. Louis: The C. V. Mosby Company, 1937.
[5] Mora S, Boechat MI, Pietka E, Huang HK, Gilsanz V, "Skeletal age determinations in children of European and African descent: applicability of the Greulich and Pyle standards," Pediatr Res, vol. 50, no. 5, pp. 624-628, Nov 2001.
[6] Koc A, Karaoglanoglu M, Erdogan M, Kosecik M, Cesur Y, "Assessment of bone ages: is the Greulich-Pyle method sufficient for Turkish boys?," Pediatr Int, vol. 43, no. 6, pp. 662-665, Dec 2001.
[7] Bull RK, Edwards PD, Kemp PM, Fry S, Hughes IA, "Bone age assessment: a large scale comparison of the Greulich and Pyle, and Tanner and Whitehouse (TW2) methods," Arch Dis Child, vol. 81, no. 2, pp. 172-173, Aug 1999 .
[8] Milner GR, Levick RK, Kay R, "Assessment of bone age: a comparison of the Greulich and Pyle, and the Tanner and Whitehouse methods," Clin Radiol, vol. 37, no. 2, pp. 119-121, Mar 1986.
[9] Proos LA, Lönnerholm T, Jonsson B, Tuvemo T, "Can the TW3 bone age determination method provide additional criteria for growth hormone treatment in adopted girls with early puberty? A comparison of the Tanner-Whitehouse 3 method with the Greulich-Pyle and the Tanner-Whitehouse 2 methods," Horm Res Paediatr, vol. 73, no. 1, pp. 35-40, 2010.
[10] Frisch H, Riedl S, Waldhör T, "Computer-aided estimation of skeletal age and comparison with bone age evaluations by the method of Greulich-Pyle and Tanner-Whitehouse," Pediatr Radiol, vol. 26, no. 3, pp. 226-231, 1996.
[11] Ashizawa, K., Kumakura, C., Zhou, X., Jin, F., & Cao, J, "RUS skeletal maturity of children in Beijing," Annals of human biology, vol. 32, no. 3, pp. 316-325, 2005.
[12] J. M. Tanner, M. J. R. Healy, H. Goldstein and N. Cameron, Assessment of skeletal maturity and prediction of adult height (TW3 method), 3rd ed 編者, W.B Saunders, 2001.
[13] A. T. Al-Taani, I. W. Ricketts and A. Y. Cairns, "Classification of hand bones for bone age assessment," in Proceedings of Third International Conference on Electronics, Circuits, and Systems, 1996.
[14] S. Wastl and H. Dickhaus, "Computerized classification of maturity stages of hand bones of children and juveniles," in Proceedings of 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 1996.
[15] S. Mahmoodi, B. S. Sharif, E. G. Chester, J. P. Owen and R. E. J. Lee, "Automated vision system for skeletal age assessment using knowledge based techniques," in Sixth International Conference on Image Processing and Its Applications, 1997.
[16] M. S. Dinesh, Bhanuprakash and Ashok Rao, "Vision system for bone measurement from digital hand radiograph," in Proceedings of the First Regional Conference, IEEE Engineering in Medicine and Biology Society and 14th Conference of the Biomedical Engineering Society of India. An International Meet, 1995.
[17] D. J. Michael and A. C. Nelson, "HANDX: a model-based system for automatic segmentation of bones from digital hand radiographs," in IEEE Transactions on Medical Imaging, vol. 8, no. 1, pp. 64-69, March 1989.
[18] E. Pietka, M. F. McNitt-Gray, M. L. Kuo and H. K. Huang, "Computer-assisted phalangeal analysis in skeletal age assessment," in IEEE Transactions on Medical Imaging, vol. 10, no. 4, pp. 616-620, Dec 1991.
[19] E. Pietka, L. Kaabi, M. L. Kuo and H. K. Huang, "Feature extraction in carpal-bone analysis," in IEEE Transactions on Medical Imaging, vol. 12, no. 1, pp. 44-49, March 1993.
[20] E. Pietka, "Computer-assisted bone age assessment based on features automatically extracted from a hand radiograph," Comput Med Imaging Graph, vol. 19, no. 3, pp. 251-259, May-Jun 1995.
[21] C. W. Hsieh, T. L. Jong, C. M. Tiu, "The phalangeal morphological characteristics for bone age recognition," in Proceedings of the 13th IEEE-NPSS Real Time Conference, Montreal, Canada, 2003.
[22] C.W. Hsieh, B. C. Chu, T. L. Jong, "Bone age classification using fuzzy concept with support vector machine neural network," in 12th International Conference on BioMedical Engineering (ICBME), Singapore, 2005.
[23] Bo-Chun Chu, An Investigation of Computerized Automatic Bone Age Assessment System Based on Carpal Phalangeal, Master Thesis, EE NTHU, 2005.
[24] Chen-Ming Li, The Study of Fully Automatic Bone Age Development Using the Area Ratio of Carpals, Master Thesis, EE NTHU, 2011.
[25] Ka-Tong Leong, The Effect of Phalanx Contour Location via Alpha-Gamma Equalization, Master Thesis, EE NTHU, 2012.
[26] Guang-Tzu Wang, An Automatic Phalangeal Segmentation System with Carpal Information, Master Thesis, EE NTHU, 2013.
[27] Po-Chao Chang, Cloud Bone Age Assessment Service Based on Tanner and Whitehouse Method, Master Thesis, EE NTHU, 2012.
[28] You-Cyuan Yang, Implementation of a TW3-Based Bone Age Assessment Cloud Service System with Automatic Phalanx Segmentation, Master Thesis, EE NTHU, 2014.
[29] P. Sermanet, D. Eigen, . X. Zhang, M. Mathieu, R. Fergus and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
[30] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke and A. Rabinovich, "Going deeper with convolutions," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[31] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv preprint arXiv:1409.1556, 2014.
[32] C. Spampinato, S. Palazzo, D. Giordano, M. Aldinucci, R. Leonardi, "Deep learning for automated skeletal bone age assessment in X-ray images," Medical Image Analysis, vol. 36, pp. 41-51, February 2017.
[33] M. Chen, "Automated bone age classification with deep neural networks," in Technical Report, Stanford University, USA.
[34] V. I. Iglovikov, A. Rakhlin, A. A. Kalinin and A. A. Shvets , "Paediatric Bone Age Assessment Using Deep Convolutional Neural Networks," in Deep learning in medical image analysis and multimodal learning for clinical decision support, Springer, Cham, 2018.
[35] Wei-Liang Kuo, Bone Age Assessment and C. elegans Age Prediction Using Deep Convolutional Neural Network, Master Thesis, EE NTHU, 2018.
[36] Xin-Yi Lin, Carpal Bones Segmentation for Hand Bone X-ray Images Based on Deep Convolutional Neural Network, Master Thesis, EE NTHU, 2022.
[37] Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov 1998.
[38] M. Lin, Q. Chen and S. Yan, "Network In Network," arXiv preprint arXiv:1312.4400, 2013.
[39] Jie Hu, Li Shen, Gang Sun, "Squeeze-and-Excitation Networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[40] Q. Zhang, J. Sang, W. Wu, B. Cai, Z. Wu and H. Hu , "An Image Splicing and Copy-Move Detection Method Based on Convolutional Neural Networks with Global Average Pooling," in International Conference on Image and Graphics, Springer, Cham, 2019.
[41] O. Ronneberger, P. Fischer and T. Brox , "U-Net: Convolutional Networks for Biomedical Image Segmentation," in International Conference on Medical image computing and computer-assisted intervention, Springer, Cham, 2015.
[42] J. Long, E. Shelhamer and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
[43] F. I.Diakogiannis, F. Waldner, P. Caccetta and C. Wu, "ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 162, pp. 94-114, April 2020.
[44] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, "Deep Residual Learning for Image Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[45] O. Oktay, J. Schlemper, L. Le Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y Hammerla, B. Kainz, B. Glocker and D. Rueckert, "Attention U-Net: Learning Where to Look for the Pancreas," arXiv preprint arXiv:1804.03999, 2018.
[46] M. Zahangir Alom, M. Hasan, . C. Yakopcic, . T. M. Taha and V. K. Asari, "Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation," arXiv preprint arXiv:1802.06955, 2018.
[47] Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[48] Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R. Manmatha, Mu Li, Alexander Smola, "ResNeSt: Split-Attention Networks," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022.
[49] Saining Xie, Ross Girshick, Piotr Dollar, Zhuowen Tu, Kaiming He, "Aggregated Residual Transformations for Deep Neural Networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[50] Xiang Li, Wenhai Wang, Xiaolin Hu, Jian Yang, "Selective Kernel Networks," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[51] F. Milletari, N. Navab and S. Ahmadi, "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation," in 2016 Fourth International Conference on 3D Vision (3DV), 2016.
[52] L. Perez and J. Wang, "The Effectiveness of Data Augmentation in Image Classification using Deep Learning," in arXiv preprint arXiv:1712.04621, 2017.
[53] A. Buslaev, V. I. Iglovikov, E. Khvedchenya, A. Parinov , M. Druzhinin and A. A. Kalinin, "Albumentations: Fast and Flexible Image Augmentations," Information, vol. 11, no. 2, 2020.
[54] Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, Silvio Savarese, "Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[55] Bowden, B. D., "Epiphysial changes in the hand/wrist areas as indicators of adolescent stage," Australian Orthodontic Journal, vol. 4, no. 3, p. 87–104, 1976.
[56] Houston, W. J. B, "Relationships between skeletal maturity estimated from hand-wrist radiographs and the timing of the adolescent growth spurt," European Journal of Orthodontics, vol. 2, no. 2, p. 81–93, 1980.
[57] Smith, R. J, "Misuse of hand-wrist radiographs," American journal of orthodontics, vol. 77, no. 1, pp. 75-78, 1980.
[58] S. M. Garn, C. G. Rohmann and T. Blumenthal, "Ossification sequence polymorphism and sexual dimorphism in skeletal development," American Journal of Physical Anthropology, vol. 24, no. 1, pp. 101-115, 1966.
[59] Joseph MA, Natarajan J, "The Carpal and Tarsal Bones of the Human Body: Arabic mnemonics," Sultan Qaboos University Medical Journal, vol. 20, no. 2, pp. e223-e226, May 2020.