近年來，全球越來越多女性罹患乳癌，透過早期檢測與治療，才能及早發現異常並改善預後的治療。雖然深度學習技術在乳癌異常分類研究中有很大的進步，但是多數分類模型規模龐大且計算量複雜，著重在提高模型準確率，而沒有考慮運算資源的成本與限制。
本研究使用CBIS-DDSM資料集，並且提出一個新的串接分類模型與兩階段的方法，實現優化的輕量化乳癌腫瘤異常分類模型。串接分類模型在充足的資料量以及適當的資料前處理下，能夠加速模型訓練的過程以及從多樣化的特徵中學習。兩階段的方法包含透過知識蒸餾(Knowledge Distillation)建立輕量化串接分類模型以及使用啟發式演算法—簡化群體演算法(Simplified Swarm Optimization, SSO)對輕量化串接分類模型的結構進行優化。實驗結果顯示，透過知識蒸餾能夠輕量化模型規模與提升模型表現，而透過SSO針對輕量化模型的結構進行全變數更新，能夠在有限的時間下以及龐大的解空間中，取得不錯的模型表現。最後，本研究提出的優化輕量串接分類模型—SSO-Concatenated NASNetMobile(SSO-CNNM)的模型壓縮率、準確率、精確率、召回率、AUC依序為96.17%、96.47%、97.4%、94.94%、98.23%，相較於其他的方法有更好的模型表現。

In recent years, more and more women around the world are suffering from breast cancer. Abnormalities can be detected early only by early detection. In the study of breast cancer abnormality classification, most models using deep learning techniques are large-scale and computationally complex without considering the limitations of cost and computing resources.
This study uses the CBIS-DDSM dataset and proposes a new concatenated classification model and a two-stage method to achieve an optimized lightweight breast cancer mass abnormality classification model. Under data augmentation and image preprocessing, the performance of the proposed model can outperform the separate CNN model and DNN model. The two-stage method includes building a lightweight proposed model through knowledge distillation and optimizing the structure of the model using a heuristic algorithm—Simplified Swarm Optimization (SSO). The experimental results show that knowledge distillation can improve the performance of the proposed model. Besides, through the full variable update of SSO, the proposed model, SSO-Concatenated NASNetMobile(SSO-CNNM) achieves the optimal performance and the compression rate, accuracy, precision, recall, and AUC are 96.17%, 96.47%, 97.4%, 94.94%, and 98.23%, which are better than other methods.

摘要 i
Abstract ii
目錄 iii
表目錄 v
圖目錄 vii
第一章 緒論 1
1.1 研究背景與動機 1
1.2 研究目的 3
1.3 研究架構 4
第二章 文獻回顧 6
2.1 ROI圖像與全圖 6
2.2 卷積神經網路 9
2.3 知識蒸餾 14
2.4 簡化群體演算法 17
2.5 文獻回顧小結 18
第三章 研究資料與方法 19
3.1 資料集介紹 19
3.2 資料前處理 21
3.3 方法架構 23
3.4 模型評估指標 29
3.5 SSO優化模型 30
第四章 實驗結果與分析 36
4.1 實驗環境建置 36
4.2 串接分類模型表現 36
4.3 知識蒸餾實驗 44
4.4 SSO實驗設計 47
4.5 實驗結果比較 51
第五章 結論與未來研究方向 56
5.1 結論 56
5.2 未來研究方向 57

參考文獻

[1] G. N. Sharma, R. Dave, J. Sanadya, P. Sharma, and K. Sharma, "Various types and management of breast cancer: an overview," Journal of advanced pharmaceutical technology & research, vol. 1, no. 2, p. 109, 2010.
[2] W. H. Organization, "International Agency for Research on Cancer GLOBOCAN 2012: estimated cancer incidence, mortality and prevalence worldwide in 2012," Lung Cancer, 2012.
[3] R. M. Rangayyan, F. J. Ayres, and J. L. Desautels, "A review of computer-aided diagnosis of breast cancer: Toward the detection of subtle signs," Journal of the Franklin Institute, vol. 344, no. 3-4, pp. 312-348, 2007.
[4] F. Sardanelli et al., "Position paper on screening for breast cancer by the European Society of Breast Imaging (EUSOBI) and 30 national breast radiology bodies from Austria, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Israel, Lithuania, Moldova, The Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Spain, Sweden, Switzerland and Turkey," European radiology, vol. 27, no. 7, pp. 2737-2743, 2017.
[5] T. G. Debelee, F. Schwenker, A. Ibenthal, and D. Yohannes, "Survey of deep learning in breast cancer image analysis," Evolving Systems, vol. 11, no. 1, pp. 143-163, 2020.
[6] L. Abdelrahman, M. Al Ghamdi, F. Collado-Mesa, and M. Abdel-Mottaleb, "Convolutional neural networks for breast cancer detection in mammography: A survey," Computers in biology and medicine, vol. 131, p. 104248, 2021.
[7] M. P. Sampat, M. K. Markey, and A. C. Bovik, "Computer-aided detection and diagnosis in mammography," Handbook of image and video processing, vol. 2, no. 1, pp. 1195-1217, 2005.
[8] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, 2017.
[9] K. Weiss, T. M. Khoshgoftaar, and D. Wang, "A survey of transfer learning," Journal of Big data, vol. 3, no. 1, pp. 1-40, 2016.
[10] L. G. Falconí, M. Pérez, and W. G. Aguilar, "Transfer learning in breast mammogram abnormalities classification with mobilenet and nasnet," in 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), 2019: IEEE, pp. 109-114.
[11] W. Lotter, G. Sorensen, and D. Cox, "A multi-scale CNN and curriculum learning strategy for mammogram classification," in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Springer, 2017, pp. 169-177.
[12] B. Ibrokhimov and J.-Y. Kang, "Two-Stage Deep Learning Method for Breast Cancer Detection Using High-Resolution Mammogram Images," Applied Sciences, vol. 12, no. 9, p. 4616, 2022.
[13] L. Shen, L. R. Margolies, J. H. Rothstein, E. Fluder, R. McBride, and W. Sieh, "Deep learning to improve breast cancer detection on screening mammography," Scientific reports, vol. 9, no. 1, p. 12495, 2019.
[14] B. Zheng, S. W. Yoon, and S. S. Lam, "Breast cancer diagnosis based on feature extraction using a hybrid of K-means and support vector machine algorithms," Expert Systems with Applications, vol. 41, no. 4, pp. 1476-1482, 2014.
[15] S. Reddy, K. T. Reddy, V. V. Kumari, and K. V. Varma, "An SVM based approach to breast cancer classification using RBF and polynomial kernel functions with varying arguments," International Journal of Computer Science and Information Technologies, vol. 5, no. 4, pp. 5901-5904, 2014.
[16] B. Fu, P. Liu, J. Lin, L. Deng, K. Hu, and H. Zheng, "Predicting invasive disease-free survival for early stage breast cancer patients using follow-up clinical data," IEEE Transactions on Biomedical Engineering, vol. 66, no. 7, pp. 2053-2064, 2018.
[17] C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273-297, 1995.
[18] M. Pal, "Random forest classifier for remote sensing classification," International journal of remote sensing, vol. 26, no. 1, pp. 217-222, 2005.
[19] T. Chen et al., "Xgboost: extreme gradient boosting," R package version 0.4-2, vol. 1, no. 4, pp. 1-4, 2015.
[20] Q. Lou, Y. Li, Y. Qian, F. Lu, and J. Ma, "Mammogram classification based on a novel convolutional neural network with efficient channel attention," Computers in Biology and Medicine, vol. 150, p. 106082, 2022.
[21] S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International conference on machine learning, 2015: pmlr, pp. 448-456.
[22] M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:1312.4400, 2013.
[23] Q. Xiao and Y. Liang, "Fune: An FPGA tuning framework for CNN acceleration," IEEE Design & Test, vol. 37, no. 1, pp. 46-55, 2019.
[24] M. L. Richter, J. Schöning, A. Wiedenroth, and U. Krumnack, "Should you go deeper? optimizing convolutional neural network architectures without training," in 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), 2021: IEEE, pp. 964-971.
[25] O. Russakovsky et al., "Imagenet large scale visual recognition challenge," International journal of computer vision, vol. 115, no. 3, pp. 211-252, 2015.
[26] A. Baccouche, B. Garcia-Zapirain, and A. S. Elmaghraby, "An integrated framework for breast mass classification and diagnosis using stacked ensemble of residual neural networks," Scientific Reports, vol. 12, no. 1, pp. 1-17, 2022.
[27] C. Zhang, J. Zhao, J. Niu, and D. Li, "New convolutional neural network model for screening and diagnosis of mammograms," Plos one, vol. 15, no. 8, p. e0237674, 2020.
[28] H. Zhou, J. M. Alvarez, and F. Porikli, "Less is more: Towards compact cnns," in European conference on computer vision, 2016: Springer, pp. 662-677.
[29] S. Swaminathan, D. Garg, R. Kannan, and F. Andres, "Sparse low rank factorization for deep neural network compression," Neurocomputing, vol. 398, pp. 185-196, 2020.
[30] A. Polino, R. Pascanu, and D. Alistarh, "Model compression via distillation and quantization," arXiv preprint arXiv:1802.05668, 2018.
[31] J.-H. Luo, H. Zhang, H.-Y. Zhou, C.-W. Xie, J. Wu, and W. Lin, "Thinet: pruning cnn filters for a thinner net," IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 10, pp. 2525-2538, 2018.
[32] W. Chen, L. Gao, X. Li, and W. Shen, "Lightweight convolutional neural network with knowledge distillation for cervical cells classification," Biomedical Signal Processing and Control, vol. 71, p. 103177, 2022.
[33] E. Irmak, "Multi-classification of brain tumor MRI images using deep convolutional neural network with fully optimized framework," Iranian Journal of Science and Technology, Transactions of Electrical Engineering, vol. 45, no. 3, pp. 1015-1036, 2021.
[34] E. Irmak, "COVID‐19 disease severity assessment using CNN model," IET image processing, vol. 15, no. 8, pp. 1814-1824, 2021.
[35] J. Wu, X.-Y. Chen, H. Zhang, L.-D. Xiong, H. Lei, and S.-H. Deng, "Hyperparameter optimization for machine learning models based on Bayesian optimization," Journal of Electronic Science and Technology, vol. 17, no. 1, pp. 26-40, 2019.
[36] M. F. Aslan, K. Sabanci, A. Durdu, and M. F. Unlersen, "COVID-19 diagnosis using state-of-the-art CNN architecture features and Bayesian Optimization," Computers in Biology and Medicine, p. 105244, 2022.
[37] M. G. Ragab, S. J. Abdulkadir, and N. Aziz, "Random search one dimensional CNN for human activity recognition," in 2020 International Conference on Computational Intelligence (ICCI), 2020: IEEE, pp. 86-91.
[38] S. Loussaief and A. Abdelkrim, "Convolutional neural network hyper-parameters optimization based on genetic algorithms," International Journal of Advanced Computer Science and Applications, vol. 9, no. 10, 2018.
[39] X. Xiao, M. Yan, S. Basodi, C. Ji, and Y. Pan, "Efficient hyperparameter optimization in deep learning using a variable length genetic algorithm," arXiv preprint arXiv:2006.12703, 2020.
[40] W. Hamdy, I. Elansary, A. Darwish, and A. E. Hassanien, "An optimized classification model for COVID-19 pandemic based on convolutional neural networks and particle swarm optimization algorithm," in Digital Transformation and Emerging Technologies for Fighting COVID-19 Pandemic: Innovative Approaches: Springer, 2021, pp. 43-61.
[41] Y. Wang, H. Zhang, and G. Zhang, "cPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks," Swarm and Evolutionary Computation, vol. 49, pp. 114-123, 2019.
[42] W.-C. Yeh, Y.-P. Lin, Y.-C. Liang, and C.-M. Lai, "Convolution neural network hyperparameter optimization using simplified swarm optimization," arXiv preprint arXiv:2103.03995, 2021.
[43] C.-M. Lai, W.-C. Yeh, and C.-Y. Chang, "Gene selection using information gain and improved simplified swarm optimization," Neurocomputing, vol. 218, pp. 331-338, 2016.
[44] X. Zhang, W.-c. Yeh, Y. Jiang, Y. Huang, Y. Xiao, and L. Li, "A case study of control and improved simplified swarm optimization for economic dispatch of a stand-alone modular microgrid," Energies, vol. 11, no. 4, p. 793, 2018.
[45] W.-C. Yeh, Z. Liu, Y.-C. Yang, and S.-Y. Tan, "Solving dual-channel supply chain pricing strategy problem with multi-level programming based on improved simplified swarm optimization," Technologies, vol. 10, no. 3, p. 73, 2022.
[46] L. G. Falconi, M. Perez, W. G. Aguilar, and A. Conci, "Transfer learning and fine tuning in breast mammogram abnormalities classification on CBIS-DDSM database," Adv. Sci. Technol. Eng. Syst. J, vol. 5, pp. 154-165, 2020.
[47] X. Yu, W. Pang, Q. Xu, and M. Liang, "Mammographic image classification with deep fusion learning," Scientific Reports, vol. 10, no. 1, pp. 1-11, 2020.
[48] F. Mohanty, S. Rup, B. Dash, B. Majhi, and M. Swamy, "An improved scheme for digital mammogram classification using weighted chaotic salp swarm algorithm-based kernel extreme learning machine," Applied Soft Computing, vol. 91, p. 106266, 2020.
[49] K. U. Rehman, J. Li, Y. Pei, A. Yasin, S. Ali, and T. Mahmood, "Computer vision-based microcalcification detection in digital mammograms using fully connected depthwise separable convolutional neural network," Sensors, vol. 21, no. 14, p. 4854, 2021.
[50] X. Shu, L. Zhang, Z. Wang, Q. Lv, and Z. Yi, "Deep neural networks with region-based pooling structures for mammographic image classification," IEEE transactions on medical imaging, vol. 39, no. 6, pp. 2246-2255, 2020.
[51] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708.
[52] S. Zahoor, U. Shoaib, and I. U. Lali, "Breast cancer mammograms classification using deep neural network and entropy-controlled whale optimization algorithm," Diagnostics, vol. 12, no. 2, p. 557, 2022.
[53] W. Zhu, Q. Lou, Y. S. Vang, and X. Xie, "Deep multi-instance networks with sparse label assignment for whole mammogram classification," in International conference on medical image computing and computer-assisted intervention, 2017: Springer, pp. 603-611.
[54] L. Xie, L. Zhang, T. Hu, H. Huang, and Z. Yi, "Neural networks model based on an automated multi-scale method for mammogram classification," Knowledge-Based Systems, vol. 208, p. 106465, 2020.
[55] K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," IEEE transactions on pattern analysis and machine intelligence, vol. 37, no. 9, pp. 1904-1916, 2015.
[56] Y. Bengio, "Convolutional Networks for Images, Speech, and Time-Series Unsupervised Learning of Speech Representations View project Parsing View project," Tech. Rep., 1997.
[57] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[58] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
[59] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[60] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818-2826.
[61] A. G. Howard et al., "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
[62] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, "Learning transferable architectures for scalable image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 8697-8710.
[63] S. Sharma, S. Sharma, and A. Athaiya, "Activation functions in neural networks," towards data science, vol. 6, no. 12, pp. 310-316, 2017.
[64] R. Raj, J. Mathew, S. K. Kannath, and J. Rajan, "Crossover based technique for data augmentation," Computer Methods and Programs in Biomedicine, vol. 218, p. 106716, 2022.
[65] M. Alkhaleefah, P. K. Chittem, V. P. Achhannagari, S.-C. Ma, and Y.-L. Chang, "The influence of image augmentation on breast lesion classification using transfer learning," in 2020 International Conference on Artificial Intelligence and Signal Processing (AISP), 2020: IEEE, pp. 1-5.
[66] Y. Mednikov, S. Nehemia, B. Zheng, O. Benzaquen, and D. Lederman, "Transfer representation learning using Inception-V3 for the detection of masses in mammography," in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018: IEEE, pp. 2587-2590.
[67] A. S. A. Rahman, S. B. Belhaouari, A. Bouzerdoum, H. Baali, T. Alam, and A. M. Eldaraa, "Breast mass tumor classification using deep learning," in 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), 2020: IEEE, pp. 271-276.
[68] G. S. Charan and R. Yuvaraj, "Efficient Tumor Classification using GoogleNet Approach to Increase Accuracy in Comparison with ResNet," in 2022 3rd International Conference on Intelligent Engineering and Management (ICIEM), 2022: IEEE, pp. 490-494.
[69] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
[70] R. Ribani and M. Marengoni, "A survey of transfer learning for convolutional neural networks," in 2019 32nd SIBGRAPI conference on graphics, patterns and images tutorials (SIBGRAPI-T), 2019: IEEE, pp. 47-57.
[71] G. Carneiro, J. Nascimento, and A. P. Bradley, "Unregistered multiview mammogram analysis with pre-trained deep learning models," in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: Springer, pp. 652-660.
[72] F. Jiang, H. Liu, S. Yu, and Y. Xie, "Breast mass lesion classification in mammograms by transfer learning," in Proceedings of the 5th international conference on bioinformatics and computational biology, 2017, pp. 59-62.
[73] J. D. López-Cabrera, L. A. L. Rodríguez, and M. Pérez-Díaz, "Classification of breast cancer from digital mammography using deep learning," Inteligencia Artificial, vol. 23, no. 65, pp. 56-66, 2020.
[74] G. H. Aly, M. A. E.-R. Marey, S. El-Sayed Amin, and M. F. Tolba, "YOLO V3 and YOLO V4 for masses detection in mammograms with resnet and inception for masses classification," in International Conference on Advanced Machine Learning Technologies and Applications, 2021: Springer, pp. 145-153.
[75] Y. Chen, Q. Zhang, Y. Wu, B. Liu, M. Wang, and Y. Lin, "Fine-tuning ResNet for breast cancer classification from mammography," in The international conference on healthcare science and engineering, 2019: Springer, pp. 83-96.
[76] C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil, "Model compression," in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, 2006, pp. 535-541.
[77] G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," arXiv preprint arXiv:1503.02531, vol. 2, no. 7, 2015.
[78] J. Gou, B. Yu, S. J. Maybank, and D. Tao, "Knowledge distillation: A survey," International Journal of Computer Vision, vol. 129, no. 6, pp. 1789-1819, 2021.
[79] G. Chen, W. Choi, X. Yu, T. Han, and M. Chandraker, "Learning efficient object detection models with knowledge distillation," Advances in neural information processing systems, vol. 30, 2017.
[80] F. Zhang, X. Zhu, and M. Ye, "Fast human pose estimation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3517-3526.
[81] C. Wang et al., "Knowledge Distillation to Ensemble Global and Interpretable Prototype-Based Mammogram Classification Models," in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2022: Springer, pp. 14-24.
[82] J. DiPalma, A. A. Suriawinata, L. J. Tafe, L. Torresani, and S. Hassanpour, "Resolution-based distillation for efficient histology image classification," Artificial Intelligence in Medicine, vol. 119, p. 102136, 2021.
[83] E. Tuba, N. Bačanin, I. Strumberger, and M. Tuba, "Convolutional neural networks hyperparameters tuning," in Artificial intelligence: theory and applications: Springer, 2021, pp. 65-84.
[84] W.-C. Yeh, "A two-stage discrete particle swarm optimization for the problem of multiple multi-level redundancy allocation in series systems," Expert Systems with Applications, vol. 36, no. 5, pp. 9192-9200, 2009.
[85] R. Poli, J. Kennedy, and T. Blackwell, "Particle swarm optimization," Swarm intelligence, vol. 1, no. 1, pp. 33-57, 2007.
[86] W.-C. Yeh, "Orthogonal simplified swarm optimization for the series–parallel redundancy allocation problem with a mix of components," Knowledge-Based Systems, vol. 64, pp. 1-12, 2014.
[87] F. Dusberger and G. R. Raidl, "Solving the 3-staged 2-dimensional cutting stock problem by dynamic programming and variable neighborhood search," Electronic Notes in Discrete Mathematics, vol. 47, pp. 133-140, 2015.
[88] Q. Chen, Y. Cui, and Y. Chen, "Sequential value correction heuristic for the two-dimensional cutting stock problem with three-staged homogenous patterns," Optimization Methods and Software, vol. 31, no. 1, pp. 68-87, 2016.
[89] W.-C. Yeh, W.-W. Chang, and Y. Y. Chung, "A new hybrid approach for mining breast cancer pattern using discrete particle swarm optimization and statistical method," Expert Systems with Applications, vol. 36, no. 4, pp. 8204-8211, 2009.
[90] J.-H. Lee, W.-C. Yeh, and M.-C. Chuang, "Web page classification based on a simplified swarm optimization," Applied Mathematics and Computation, vol. 270, pp. 13-24, 2015.
[91] 賴智明, "Applying Simplified Swarm Optimization for Solving Clustering Problem," 2016.
[92] W.-C. Yeh, "New parameter-free simplified swarm optimization for artificial neural network training and its application in the prediction of time series," IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 4, pp. 661-665, 2013.
[93] C.-L. Huang, "A particle-based simplified swarm optimization algorithm for reliability redundancy allocation problems," Reliability Engineering & System Safety, vol. 142, pp. 221-230, 2015.
[94] A. Khamparia et al., "Diagnosis of breast cancer based on modern mammography using hybrid transfer learning," Multidimensional systems and signal processing, vol. 32, no. 2, pp. 747-765, 2021.