在資料探勘中，分類與預測為主要的議題之一。現今的研究當中，已有許多分類方法被應用於資料的分類問題中，如: 類神經網路(Artificial Neural Network, ANN)、向量支援器(Support Vector Machine, SVM)、馬氏距離(Mahalanobis Distance)、決策樹(Decision Tree)等。由於類神經網路強大的模式辨認與高度的容錯能力，使得類神經網路最常被用來進行分類的工作。


在利用類神經網路進行分類之前，資料的前處理對於類神經網路分類結果的好壞會有相當程度的影響，除了在使用類神經網路進行分類之前的正規化與標準化之外，並沒有對資料進行額外的處理，也就是說資料屬性的權重被當作是一模一樣，因此當有不恰當的資料屬性被拿來進行分類工作時，類神經網路可能會被此不恰當的屬性所誤導，進而讓分類結果不佳。所以在原始資料的前處理當中，資料屬性應該依其對分類結果的影響區分其權重，這樣才能讓對結果有顯著影響的因子被顯現出來，分類結果才會有所提升。


馬氏距離的計算主要是用來衡量樣本點之間的距離，而鮮少有利用馬氏距離計算屬性變數間的距離，利用屬性間馬氏距離的計算，可以看出哪些屬性對於分類的結果佔有較大的比重，哪些屬性對於分類結果的影響較小。本研究提出以轉置權重馬氏距離為基礎之類神經網路分類模型(TWMD-based Neural Network)來解決分類問題，藉由馬氏距離越大相似程度越小之概念來給予屬性權重，並使用權重處理過後之資料來訓練類神經網路。研究結果顯示，經屬性權重處理過後的資料來訓練類神經網路會比未經屬性權重處理的資料來訓練類神經網路的分類結果要來的好。










關鍵字：馬氏距離、類神經網路、屬性權重

Abstract

In the data mining field, classification and prediction are one of the most major issues. In resent research, there are many methodologies to apply to classification problem, like Neural Network, Support Vector Machine, Mahalanobis Distance, Decision Tree and so on. Due to the powerful pattern recognition and error tolerance ability in neural network, neural network is usually applied to do the classification works.


Before using neural network to do classification, there are significant influence on data preprocessing. Except scaling and normalization before using neural network, it doesn’t do additional process on data. All the attributes are regarded as the equal weight. If using irrelevant attributes to do the classification, neural network may misdirect by the irrelevant attributes. Therefore, neural network doesn’t do well on classification. So, in the data preprocessing, the weight of the attribute should be distinguish between different weights based on the classification results. It will highlight the attributes that have significant influence on outcome and the result of classification can increase.


The calculation of the Mahalanobis Distance is used to measure the distance of instances, but it is rarely to calculate the distance between attributes. Calculate the Mahalanobis Distances between attributes can know which attributes have significant influence on classification result. Our research use Neural Network based on Transposed Weighted Mahalanobis Distance (TWMD-based NN) to solve the classification problems. Use the concept of the similarity, the bigger Mahalanobis Distance, the smaller the weight. Finally, use the data which processed by attribute weight to train Neural Network. The research results show that the processed data by the attribute weights are better than the original data.







Keywords：Mahalanobis Distance、Neural Network、Feature weight

目錄

摘要.................................................................................................................................i
Abstract..........................................................................................................................ii
致謝...............................................................................................................................iii
目錄...............................................................................................................................iv
圖目錄...........................................................................................................................vi
表目錄..........................................................................................................................vii
第一章 緒論..................................................................................................................1
1.1 研究背景與動機.............................................................................................1
1.2 研究目的.........................................................................................................2
1.3 論文架構.........................................................................................................2
第二章 文獻探討..........................................................................................................3
2.1 馬氏距離文獻.................................................................................................3
2.2 類神經網路文獻.............................................................................................3
2.3 相關文獻小結.................................................................................................4
2.4 類神經網路.....................................................................................................4
2.4.1 倒傳遞類神經網路..............................................................................7
2.4.2 比例共軛梯度演算法........................................................................13
2.5 馬氏距離.......................................................................................................15
第三章 研究方法........................................................................................................18
3.1 研究方法架構...............................................................................................18
3.2 研究流程說明...............................................................................................19
3.2.1 收集資料............................................................................................19
3.2.2 資料轉置............................................................................................19
3.2.3 計算屬性變數之權重........................................................................20
3.2.4 將屬性權重乘回原始資料................................................................23
3.2.5 訓練類神經網路並進行分類............................................................24
3.3 實例說明.......................................................................................................26
3.3.1 收集資料............................................................................................27
3.3.2 資料進行轉置....................................................................................29
3.3.3 屬性變數之馬氏距離與權重............................................................30
3.3.4 將屬性權重乘回原始資料................................................................34
3.3.5 訓練類神經網路並進行分類............................................................36
第四章 實例驗證......................................................................................................41
4.1 實驗流程說明.............................................................................................41
4.2 衡量指標.....................................................................................................42
4.3 實驗結果.....................................................................................................43
4.3.1 Heart Disease (心臟病)....................................................................43
4.3.2 German Credit (德國信用)..............................................................47
4.3.3 Diabetes (糖尿病)............................................................................52
4.3.4 Letter Recognition (字母辨識)........................................................56
4.3.5 Landsat Satellite (衛星照片)........................................................61
4.4 實驗結論.....................................................................................................65
第五章 結論..............................................................................................................66
5.1 研究結論.....................................................................................................66
5.2 未來研究建議.............................................................................................68
參考文獻......................................................................................................................69





























圖目錄

圖2-1 類腦神經元之構造............................................................................................5
圖2-2 人工神經處理單元模型....................................................................................6
圖2-3 一層倒傳遞網路之網路架構圖........................................................................8
圖2-4 雙彎曲函數圖形................................................................................................9
圖2-5 雙曲線正切函數圖形........................................................................................9
圖2-6 馬氏距離與歐式距離......................................................................................16
圖3-1 研究方法架構圖..............................................................................................18
圖3-2 資料轉置..........................................................................................................20
圖4-1 實驗流程..........................................................................................................41




























表目錄

表3-1 原始資料..........................................................................................................19
表3-2 轉置原始資料之平均值與標準差..................................................................21
表3-3 標準化資料......................................................................................................21
表3-4 屬性變數之馬氏距離......................................................................................22
表3-5 屬性變數之權重..............................................................................................23
表3-6 新訓練資料......................................................................................................24
表3-7 心臟病資料庫屬性變數資訊..........................................................................26
表3-8 心臟病資料庫類別變數資料..........................................................................26
表3-9 類別變數型態....................................................................................………..27
表3-10心臟病資料庫(正常樣本) – 訓練集................................................……….27
表3-11心臟病資料庫(正常樣本) – 測試集................................................……….27
表3-12 心臟病資料庫(不正常樣本) – 訓練集...............................................…….28
表3-13 心臟病資料庫(不正常樣本) – 測試集................................................……28
表3-14 轉置心臟病資料庫(不正常樣本) – 訓練集................................................29
表3-15 轉置心臟病資料庫(不正常樣本) – 測試集................................................29
表3-16 標準化心臟病資料庫(正常樣本) – 訓練集................................................30
表3-17 標準化心臟病資料庫(不正常樣本) – 訓練集............................................31
表3-18 標準化心臟病資料庫(正常樣本)之共變異矩陣反矩陣– 訓練集….........31
表3-19 標準化心臟病資料庫(不正常樣本)之共變異矩陣反矩陣– 訓練集….…32
表3-20 標準化心臟病資料庫(正常樣本)之馬氏距離– 訓練集….........................32
表3-21 標準化心臟病資料庫(不正常樣本)之馬氏距離– 訓練集….....................33
表3-22 正常樣本之屬性變數權重– 訓練集............................................................33
表3-23 不正常樣本之屬性變數權重– 訓練集…....................................................34
表3-24 新心臟病資料庫(正常樣本) – 訓練集........................................................35
表3-25 新心臟病資料庫(不正常樣本) – 訓練集....................................................35
表3-26 正規化心臟病資料庫(正常樣本) – 訓練集………………………………36
表3-27 正規化心臟病資料庫(正常樣本) – 測試集………………………………37
表3-28 正規化心臟病資料庫(不正常樣本) – 訓練集……………………………37
表3-29 正規化心臟病資料庫(不正常樣本) – 測試集……………………………38
表3-30 網路輸出值 – 訓練集……………………………………………………..39
表3-31 網路判斷值 – 訓練集……………………………………………………..39
表3-32 真實目標值 – 訓練集目標………………………………………………..39
表3-33 網路輸出值 – 測試集……………………………………………………..40
表3-34 網路判斷值 – 測試集……………………………………………………..40
表3-35 真實目標值 – 測試集目標………………………………………………..40
表4-1 混淆矩陣……………………………………………………………………..42
表4-2資料庫屬性變數資訊(心臟病)………………………………………………43
表4-3資料庫類別變數資料(心臟病).……………………………………………...43
表4-4資料庫類別變數型態(心臟病).……………………………………………...44
表4-5轉換過後資料訓練類神經網路之評估準則(心臟病-訓練集)………………44
表4-6轉換過後資料訓練類神經網路之評估準則(心臟病-測試集)………………44
表4-7原始資料訓練類神經網路之評估準則(心臟病-訓練集)……………………45
表4-8原始資料訓練類神經網路之評估準則(心臟病-測試集)……………………45
表4-9交叉驗證之樣本評估指標(心臟病)…………………………………………45
表4-10 30次交叉驗證結果(心臟病)……..………………………………………...46
表4-11成對樣本檢定(心臟病)………………………………………...……………47
表4-12資料庫屬性變數資訊(德國信用)…………………………………………..47
表4-13資料庫類別變數資料(德國信用)…………………………………………..48
表4-14資料庫類別變數型態(德國信用)…………………………………………..48
表4-15轉換過後資料訓練類神經網路之評估準則(德國信用-訓練集)…………..49
表4-16轉換過後資料訓練類神經網路之評估準則(德國信用-測試集)…………..49
表4-17原始資料訓練類神經網路之評估準則(德國信用-訓練集)………………..49
表4-18原始資料訓練類神經網路之評估準則(德國信用-測試集)………………..50
表4-19交叉驗證之樣本評估指標(德國信用)……………………………………...50
表4-20 30次交叉驗證結果(德國信用)……………………………………………..50
表4-21成對樣本檢定(德國信用)…………………………………………………..52
表4-22資料庫屬性變數資訊(糖尿病)……………………………………………..52
表4-23資料庫類別變數資料(糖尿病)……………………………………………..53
表4-24資料庫類別變數型態(糖尿病)……………………………………………..53
表4-25轉換過後資料訓練類神經網路之評估準則(糖尿病-訓練集)……………..53
表4-26轉換過後資料訓練類神經網路之評估準則(糖尿病-測試集)……………..53
表4-27原始資料訓練類神經網路之評估準則(糖尿病-訓練集)…………………..54
表4-28原始資料訓練類神經網路之評估準則(糖尿病-測試集)…………………..54
表4-29交叉驗證之樣本評估指標(糖尿病)………………………………………..54
表4-30 30次交叉驗證結果(糖尿病)……………………………………………….55
表4-31成對樣本檢定(糖尿病)……………………………………………………..56
表4-32資料庫屬性變數資訊(字母辨識)…………………………………………..57
表4-33資料庫類別變數資料(字母辨識)…………………………………………..57
表4-34資料庫類別變數型態(字母辨識)…………………………………………..57
表4-35轉換過後資料訓練類神經網路之評估準則(字母辨識-訓練集)…………..58
表4-36轉換過後資料訓練類神經網路之評估準則(字母辨識-測試集)…………..58
表4-37原始資料訓練類神經網路之評估準則(字母辨識-訓練集)………………..58
表4-38原始資料訓練類神經網路之評估準則(字母辨識-測試集)………………..58

表4-39交叉驗證之樣本評估指標(字母辨識)……………………………………..59
表4-40 30次交叉驗證結果(字母辨識)…………………………………………….59
表4-41成對樣本檢定(字母辨識)…………………………………………………..61
表4-42資料庫屬性變數資訊(衛星照片)…………………………………………..61
表4-43資料庫類別變數資料(衛星照片)…………………………………………..61
表4-44資料庫類別變數型態(衛星照片)…………………………………………..61
表4-45轉換過後資料訓練類神經網路之評估準則(衛星照片-訓練集)…………..62
表4-46轉換過後資料訓練類神經網路之評估準則(衛星照片-測試集)…………..62
表4-47原始資料訓練類神經網路之評估準則(衛星照片-訓練集)…..……………62
表4-48原始資料訓練類神經網路之評估準則(衛星照片-測試集)…..……………62
表4-49交叉驗證之樣本評估指標(衛星照片)…………………………..…………63
表4-50 30次交叉驗證結果(衛星照片)…………………………………………….63
表4-51成對樣本檢定(衛星照片)…………………………………………………..65
表4-52 30次交叉驗證改善幅度…………………….………………………………65
表5-1資料轉換前後之隱藏層節點數比較………….……………………………...67
表5-2相同隱藏層節點數下類神經網路分類效果….……………………………...67

參考文獻

[1] J. Han and M. Kamber, Data Mining: Concepts and Techniques. Morgan
Kaufmann, 2000.
[2] P.C. Mahalanobis, “On the generalised distance in statistics”, Proceedings of the National Institute of Science of India, 1936 12, pp. 49–55.
[3] E. Krusinska and J. Liebhart, “Objective Evaluation of Degree of Illness with the Weighted Mahalanobis Distance. A Study for Patients Suffering from Chronic Obturative Lung Disease”, Computer in biology and medicine, Vol. 17, Issue 5, 1987, pp. 321-329.
[4] E. Krusinska, “A Valuation of State of Object Based on Weighted Mahalanobis Distance”, Pattern Recognition, Vol. 20, No. 4, 1987, pp 413-418.
[5] T. Marasovic and V. Papic, “Feature Weighted Nearest Neighbour Classification for Accelerometer-Based Gesture Recognition”, 20th International Conference on Software, Telcommunications and Computer Network (SoftCOM), 2012, pp. 1-5.
[6] F. Gu, D. Liu and X. Wang, “Semi-supervised weighted distance metric learning for kNN classification”, International Conference on Computer, Mechatronics, Control and Electronic Engineering (CMCE), Vol. 6, 2010, pp. 406-409.
[7] M. Wolfel and H. K. Ekenel, “Feature Weighted Mahalanobis Distance: Improved Robustness for Gaussian Classifiers”, In Proc. of 13th European Signal Processing Conference (EUSIPCO), 2005.
[8] K.Younis, M. Karim, R. Hardie, J. Loomis, S. Rogers and M. Desimio, “Cluster merging based on weighted mahalanobis distance with application in digital mammograph”, Proceedings of the IEEE National Aerospace and Electronics Conference, NAECON, 1998, pp. 525-530
[9] A. C. Sobieranski, D. D. Abdala, E. Comunello and A. V. Wangenheim, “Learning a color distance metric for region-based image segmentation ”, Pattern Recognition Letters, Vol. 30, No. 6, 2009, pp. 1496-1506.
[10] S. Xiang, F. Nie and C. Zhang, “Learning a Mahalanobis distance metric for data clustering and classification”, Pattern Recognition, Vol. 41, No. 12, 2008, pp. 3600-3612.
[11] Taguchi and Jugulum, 2002 G. Taguchi, R. Jugulum, The Mahalanobis–Taguchi strategy, John Wiley & Sons, New York (2002)
[12] S. A. Velastin and C. Xu, “Line and Circle Finding by The Weighted Mahalanobis Distance Transform and Extended Kalman Filtering”, IEEE International Symosium on Industrial Electronics, 1994, pp. 258-263.
[13] K. Gopal and T. R. Ioerger, “Distance Metric Learning through Optimization of Ranking”, Seventh IEEE International Conference on Data Mining Workshops (ICDM), 2007, pp. 201-206.
[14] D. Said, L. Stirling, P. Federolf and K. Barker, “Data preprocessing for Distance-based Unsupervised Intrusion Detection”, 9th Annual International Conference on Privacy, Security and Trust (PST), 2011, pp. 181-188.
[15] S. Kawai, T. Furukane, K. Shibata and Y. Horita, “Automatic Distinction of Road Surface Condition in Road Images at Night-time using PCA and Mahalanobis Weighting”, IEEE International Conference on Consumer Electronics (ICCE), 2012, pp. 231-232.
[16] M. Asasda, “Wafer yield prediction by the Mahalanobis-Taguchi system”, IEEE International Workshop on Statistical Methodology, 2001, pp. 25-28..
[17] J. Shi and L. Cheng, “Financial Crisis Dynamic Prediction Based on Sliding Window Technology and Mahalanobis-Taguchi system”, International Conference on Information Technology , Computer Engineering and Management Science (ICM), Vol. 4, 2011, pp. 65-68.
[18] H. Rabal, N. Cap, C. Criado and N. Alamo, “Holodiagrams using Mahalanobis distance”, Optik – International Journal for light Electron Optics, Vol. 123, No. 19, 2012, pp. 1725-1731.
[19] R. Das, “A comparison of multiple classification methods for diagnosis of Parkinson disease”, Expert Systems with Application, Vol. 37, No. 2, 2010, pp. 1568-1572.
[20] K. Rajeswari, V. Vaithiyanathan and T. R. Neelakantan, “Feature Selection in Ischemic Heart Disease Identification using Feed Forward Neural Network”, Pcocedia Engineering, Vol. 41, 2012, pp. 1818-1823.
[21] S. M. Jadhav, S. L. Nalbalwar and A. A. Ghatol, “Artificial Neural Network Based Cardiac Arrhythmia Disease Diagnosis”, International Conference on Process Automation, Control and Computing (PACC), 2011, pp. 1-6.
[22] N. G. B. Amma, “Cardiovascular disease prediction system using genetic algorithm and neural netwok”, International Conference on Computing, Communication and Application (ICCCA), 2012, pp. 1-5.
[23] G. Zhang, M. Y. Hu, B. E. Patuwo and D. C. Indro, “Artificial neural network in bankruptcy prdiction：General framework and cross-validation analysis”, European Journal of Operational Research, Vol. 116, No. 1, 1999, pp. 16-32.
[24] S. Lee and W. S. Choi, “A multi-industry bankruptcy prediction model using back- propagation neural network and multivariate discriminant analysis”, Expert Systems with Applications, Vol. 40, 2013, pp. 2941-2946.
[25] S. Domingues, P. Campoy and R. Aracil, “A neural network based quality
control system for steel strip manufacturing ”, Annual Review in Automatic
Programming, Vol. 19, 1994, pp. 185-190.
[26] G. Sciuto, B. Bonaccorso, A. Cancelliere and G. Rossi , “Quality control of daily rainfall data with neural networks”, Journal of Hydrology, Vol. 364, No, 1-2, 2009, pp. 13-22.
[27] O. Martin, M. Lopez and F. Martin, “Artificial neural networks for quality control by ultrasonic testing in resistance spot welding”, Journal of Materials Processing Technology, Vol. 183, No. 2-3, 2007, pp. 226-233.
[28] C.L. Wu, K.W. Chau and C. Fan, “Prediction of rainfall time series using modular artificial neural networks coupled with data-preprocessing techniques”, Journal of Hydrology, Vol. 389, No. 1-2, 2010, pp. 146-167.
[29] B. Cannas, A. Fanni, L. See and G. Sias, “Data preprocessing for river flow forecasting using neural networks: Wavelet transforms and data partitioning”, Physics and Chemistry of the Earth, Volume 31, No. 18, 2006 ,pp. 1164-1171.
[30] A. J. T. Ballesteros, C. H. Martinez, J. C. Riquelme and R. Ruiz, “Feature selection to enhance a two-stage evolutionary algorithm in product unit neural networks for complex classification problems”, Neurocomputing, Vol. 114, 2013, pp. 107-117.
[31] H. Yoon, C. S. Park, J. S. Kim and J. G. Baek, “Algorithm learning based neural network integrating feature selection and classification”, Expert Systems with Applcations, Vol. 40, No. 1, 2013, pp. 231-241.
[32] F. F. Navarrro, C. H. Martinez, J. C. Riquelme and R. Ruiz, “Evolutionary Generalized Radial Basis Function neural networks for improving prediction accuracy in gene classification using feature selection”, Applied Soft Computing, Vol. 12, No. 6, 2012, pp. 1787-1800.
[33] K. Rajeswari, V. Vaithiyanathan and T. R. Neelakantan, “Feature Selection in Ischemic Heart Disease Identification using Feed Forward Neural Networks”, Procedia Engineering, No. 41, 2012, pp. 1818-1823.
[34] I. C. Jou, S. S. You and L. W. Chang, “Analysis of hidden nodes for multi-layer perceptron neural networks”, Pattern Recognition, Vol. 27, Issue 6, 1994, pp. 859-864.
[35] M. Arai, “Bounds on the number of hidden units in binary-valued three-layer neural networks”, Neural Networks, Vol. 6, Issue 6, 1993, pp. 855-860.
[36] M. F. Moller, “A scaled conjugate gradient algorithm for fast supervised learning”, Neural Networks, Vol. 6, No. 4, 1993, pp. 525-533.
[37] 葉怡成，類神經網路模式應用與實作，儒林圖書有限公司，民國82年。
[38] 蔡依玲，「台灣股票市場報酬率之研究」，國立成功大學統計研究所碩士論文，民國90年。