本論文前半部分主要在介紹機器學習的原理，多種基於梯度下降法下的優化演算法及深度學習神經網路多種不同的架構。後半部分在介紹本論文所做的實驗。
本論文主要以人工智慧的方式針對兩種類影像資料--掌骨影像與C線蟲顯微影像作處理。首先專注於在掌骨影像骨齡檢測上使用深度學習方式取代以往醫生人力判讀或是傳統影像處理特徵抽取技術。基於多種卷積神經網路模型下，針對原始掌骨影像及前處理去除雜訊後的影像做比較，在此所用的訓練資料為美國人掌骨影像資料庫，最後再加入額外的特徵考量，討論其預測結果並綜合比較各種架構下的準確率。本論文最後在訓練資料使用原始掌骨影像與前處理後掌骨影像混合之資料集，套用InceptionResNetV2的架構，獲得最低的測試資料Mean absolute error 5.71個月。最後使用美國人掌骨影像所訓練出來的深度學習模型，套用在台北榮總1到18歲孩童的掌骨X光影像集上做測試，得到了與以往醫生看法相近的結論，發現經訓練資料為美國人的模型判讀台灣人掌骨影像的骨齡值普遍都偏大，有一個正的偏差值，並針對為何發生此結果加以討論並修正模型。
本論文實驗的第二部分專注於秀麗隱桿線蟲(C線蟲)的日齡判讀。針對陽明大學許翱麟教授所提供的C線蟲顯微影像，透過深度學習的方式取代以往生物學者肉眼透過生理特徵做日齡判讀。後續在輸入項新增額外特徵，並透過InceptionResNetV2模型下，於測試資料的判斷誤差mean absolute error僅0.94天。使用影像分類的方式將結果分為14類，使用InceptionResNetV2模型，分類正確率在測試資料上達到了56%的正確率。以上的方式將能夠使得生物學者在使用C線蟲研究衰老時能有更精準的依據。

The first part of this thesis mainly introduces the principles of machine learning, a variety of optimization algorithms based on gradient descent, and different architectures of deep learning neural networks. The second part introduces the experiments on two sets of image databases conducted in this thesis.
The most important purpose in this thesis is handling two kinds of different image data sets by artificial intelligence. The first experiment focuses on the bone age assessment using deep learning methods to replace the traditional ways like interpretation by doctors or feature extraction technology in the past. Based on various convolutional neural network models, the prediction results of using original metacarpal images and images after pre-processing for noise removal are compared. Additional feature inputs are added to discuss the prediction results and comprehensively compare the accuracy under various architectures. Finally, I merge the original images and the pre-processed images to be the training data, and apply the architecture of InceptionResNetV2 to build a model for bone age assessment. It is found that the lowest test data mean absolute error of 5.71 months is achieved.
At the end of the first experiment, the deep learning model trained by the American metacarpal images was used to test the metacarpal X-ray image set of children aged 1 to 18 in Taiwan. It is found that the model trained from American training data predict a larger bone age value for the Taiwanese metacarpal image than its actual bone age. The offsetted results were consistent with doctors' previous knowledge. Discussion of why this result occurs is given and an offset can be added to the models to improve the MAE.
The second experiment in this thesis focuses on the age prediction of Caenorhabditis elegans. The microscopic C. elegans images are provided by Prof. Ao-Lin Hsu, Yangming University. To solve the age prediction problems of the C. elegans, deep learning method is used to replace age prediction made by biologists based on C. elegans’physiological characteristics. By augmenting other features to the input, and using the InceptionResNetV2 model, the mean absolute error in age prediction of the test data is only 0.94 days. Compared with human age prediction, it has successfully improved the accuracy and shortened the processing time, allowing biologists to have a more accurate basis when using C. elegans to study aging characteristics.

摘要 I
ABSTRACT II
目錄 IV
圖目錄 VIII
表目錄 XII
第一章 緒論 1
1.1 前言 1
1.2 研究背景 1
1.2.1 骨齡預測研究背景 1
1.2.2 線蟲日齡預測研究背景 1
1.3 研究動機 2
1.3.1 骨齡預測研究動機 2
1.3.2 線蟲日齡預測研究動機 2
1.4 文獻回顧 3
1.4.1 骨齡判讀文獻回顧 3
1.4.2 線蟲日齡判讀文獻回顧 4
1.5 論文貢獻 5
1.5.1 骨齡判讀論文貢獻 5
1.5.2 線蟲日齡判讀論文貢獻 5
1.6 數據介紹 5
1.6.1 手掌骨X光片影像資料 5
1.6.2 C線蟲顯微影像資料 7
1.7 論文架構 7
第二章 原理 8
2.1 前言 8
2.2 REGRESSION迴歸分析 9
2.2.1 挑選函數 10
2.2.2 函數的好壞 10
2.2.3 挑選最好的函數 10
2.2.4 Non Linear Regression 12
2.3 梯度下降法(GRADIENT DESCENT) 13
2.3.1 Gradient descent(GD) 13
2.3.2 學習率(Learning rate) 16
2.3.3 Adagrad 17
2.3.4 RMSporp 19
2.3.5 Momentum 20
2.3.6 Adam (Adaptive Moment Estimation) [13] 21
2.3.7 Full Batch Gradient Descent 22
2.3.8 Stochastic Gradient Descent 22
2.3.9 Mini-Batch Gradient Descent 23
2.3.10 總結 24
第三章 深度學習 25
3.1 前言 25
3.2 NEURAL NETWORK 26
3.2.1 神經網路 (Neural Network) 26
3.2.2 Activation function 27
3.2.3 Fully Connected Neural Network 30
3.3 反向傳播法(BACKPROPAGATION) [21] 33
3.3.1 Chain Rule 33
3.3.2 Forward Pass 34
3.3.3 Backward Pass 35
3.3.4 總結 37
3.4 過擬合(OVERFITTING) 38
3.4.1 權重衰減(Weight decay) 39
3.4.2 Dropout [22] 40
3.4.3 Earlystopping 41
3.5 卷積神經網路(CONVOLUTIONAL NEURAL NETWORK , CNN) [14] 41
3.5.1 卷積層(Convolutional Layer) 42
3.5.2 池化層Pooling Layer 44
3.5.3 Backprapogation in CNN 46
3.5.4 CNN Visualization 48
3.5.5 VGG(Visual Geometry Group) [27] 49
3.5.6 GoogLeNet [28] 50
3.5.7 ResNet [29] 51
3.6 FULL CONVOLUTIONAL NEURAL NETWORK [30] 52
3.6.1 原理及架構 52
3.6.1 步驟及結果 53
3.6.2 U-net [32] 54
第四章 分析方法與實驗結果 56
4.1 前言 56
4.2 掌骨影像前處理 57
4.2.1 介紹 57
4.2.2 整體流程圖 58
4.2.3 步驟 59
4.2.4 結果與討論 64
4.3 骨齡預測實驗方法及流程 65
4.3.1 資料前處理 65
4.3.2 資料增強 66
4.3.3 實驗步驟 68
4.4 骨齡預測實驗結果及討論 77
4.5 線蟲日齡估測實驗方法及流程 94
4.5.1 資料前處理 94
4.5.2 資料增強 95
4.5.3 實驗步驟 97
4.6 線蟲日齡估測實驗結果及討論 102
第五章 結論與未來展望 107
參考文獻 109

[1] Greulich WW, Pyle SI (1959): "Radiographic Atlas of Skeletal Development of the Hand and Wrist"
[2] Tanner JM, Whitehouse RH, Marshall WA (1983): "Assessment of skeletal maturity and prediction of adult height" (TW2 method). 2nd ed., Academic Press
[3] "Using C. elegans for aging research ", Program in Gene Function and Expression, Program in Molecular Medicine, University of Massachusetts Medical School, Worcester, MA, 01605, USA
[4] Vladimir Iglovikov, Alexander Rakhlin, Alexandr A. Kalinin, Alexey Shvets (2017): "Pediatric Bone Age Assessment Using Deep Convolutional Neural Networks"
[5] Olaf Ronneberger, Philipp Fischer, Thomas Brox (2015): "U-Net: Convolutional Networks for Biomedical Image Segmentation"
[6] C. Spampinato, S. Palazzo, D. Giordano, M. Aldinucci, R. Leonardi (2016) "Deep learning for automated skeletal bone age assessment in X-ray images"
[7] Matthew Chen Stanford University (2017) "Automated Bone Age Classification with Deep Neural Networks"
[8] Radiological Society of North America, http://www.rsna.org/
[9] Jonathan Long, Evan Shelhamer, Trevor Darrell, "Fully Convolutional Networks for Semantic Segmentation"
[10] Alphago DeepMind https://deepmind.com/research/alphago/
[11] Sam Abrahams, "Gradient Descent, Back Propagation, and Auto Differentiation", https://www.slideshare.net/cfregly/gradient-descent-back-propagation-and-auto-differentiation-advanced-spark-and-tensorflow-meetup-08042016
[12] 李宏毅 (Hung-yi Lee) http://speech.ee.ntu.edu.tw/~tlkagk/talk.html
[13] Diederik P. Kingma, Jimmy Ba, "Adam: A Method for Stochastic Optimization"
[14] Y. Lecun, L. Bottou, Y. Bengio (1998) "Gradient-based learning applied to document recognition"
[15] D.W Ruck, S.K. Rogers, M. Kabrisky (1990) "The multilayer perceptron as an approximation to a Bayes optimal discriminant function"
[16] Warren Sturgis McCulloch, (1943) "A Logical Calculus of the Ideas Immanent in Nervous Activity"
[17] Andrew Ng, "Machine Learning and AI via Brain simulations"
[18] Ivakhnenko, Alexey (1971). "Polynomial theory of complex systems"
[19] Han, Jun Morag, Claudio (1995). "The influence of the sigmoid function parameters on the speed of backpropagation learning"
[20] Vinod Nair and Geoffrey Hinton. (2010) "Rectified linear units improve restricted Boltzmann machines"
[21] Bryson, A.E.; W.F. Denham; S.E. Dreyfus. "Optimal programming problems with inequality constraints". I: Necessary conditions for extremal solutions. AIAA J. 1, 11 (1963)
[22] Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting", JMLR 2014
[23]. Test data and early stopping-
http://documentation.statsoft.com/STATISTICAHelp.aspx?path=SANN/ImportedFromProceed/TestDataandEarlyStopping
[24] Yeh James (2017) https://medium.com/@yehjames/資料分析-機器學習-第5-1講-卷積神經網絡介紹
[25] Jefkine (2016) "Backpropagation In Convolutional Neural Networks "
http://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/
[26] "State of Deep Learning in Computer Vision ", https://tech.showmax.com/2017/10/convnet-architectures/
[27] Karen Simonyan, Andrew Zisserman (2015)."Very Deep Convolutional Networks for Large-Scale Image Recognition"
[28] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich (2014): "Going deeper with convolutions"
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (2015)."Deep Residual Learning for Image Recognition"
[30] Jonathan Long, Evan Shelhamer, Trevor Darrell (2015): "Fully Convolutional Networks for Semantic Segmentation"
[31] 全卷積網絡FCN 詳解-https://read01.com/GdyPA2.html#.W0-Gq9IzZhE
[32] Olaf Ronneberger, Philipp Fischer, Thomas Brox (2015)."U-Net: Convolutional Networks for Biomedical Image Segmentation"
[33] Python - https://www.python.org/
[34] OpenCV - https://opencv.org/
[35] Keras - https://keras.io/
[36] 楊嘉棟: "α~γ值增強的前處理對掌骨影像切割之影響",國立清華大學碩士論文
[37] Jiunn-Liang Lin , Yung-Sheng Chen , Yi-Hao Huang , Ao-Lin Hsu , Tai-Lang Jong , Wen-Hsing Hsu, (2018) ."Approach to the Caenorhabditis Elegans Segmentation from Its Microscopic Image"
[38] Donglai Wei, Bolei Zhou, Antonio Torralba, William T. Freeman, (2015) ."mNeuron: A Matlab Plugin to Visualize Neurons from Deep Models"
[39] Dan Cires¸ Ueli Meier, Jurgen Schmidhuber, (2012). "Multi-column Deep Neural Networks for Image Classification"
[40] Cheng Huang, Chengjie Xiong, Kerry Kornfeld, (2004) "Measurements of age-related changes of physiological processes that predict lifespan of Caenorhabditis elegans"
[41] Theano, http://www.deeplearning.net/software/theano/
[42] Tensorflow, https://www.tensorflow.org/
[43] 16bits, https://16bit.ai/
[44] NVIDIA, http://www.nvidia.com.tw/page/home.html
[45] Walker, SH; Duncan, DB (1967). "Estimation of the probability of an event as a function of several independent variables".
[46] Harris, David and Harris, Sarah. Digital design and computer architecture (2nd ed.). San Francisco, Calif.: Morgan Kaufmann
[47] Pieter-Tjerk de Boer, Dirk Kroese, Shie Mannor, Reuven Y. Rubinstein (2005). A Tutorial on the Cross-Entropy Method