阿茲海默症(AD)是一種持續且不可逆的神經退化型疾病，並且已經在人類社會中存在了幾個世紀。這種疾病的特點是會造成認知和記憶力衰退，最終導致身體機能衰退而死亡。AD的症狀常被與正常老化現象混淆，也對整個社會帶來經濟和心理負擔。目前針對AD的治療方法只能延緩症狀的加重速度，且如何區分AD與正常老化現象也是非常複雜的問題。針對延緩症狀，醫學界致力於研究如何及早發現AD，因AD的及早發現已被證實能大幅延緩症狀的加重速度。隨著科技的進步，過去幾十年越來越多的機器學習方法被用來識別AD。在這項研究中，我們關注於識別在未來有罹患AD風險的人。我們提出了多注意力與門控多模型單元模型(Multi-attention with GMU model)，其靈感來自於加型注意力機制(additive attention)以及門控多模型單元架構(Gated Multimodel Unit, GMU)。我們採用專門設計的資料分割方法，並用分割後的訓練集對模型進行訓練。此外，我們制訂了一種評估方法來計算模型在早期AD檢測中的能力。無論是準確率(accuracy)和F1分數(F1-score)等傳統的評估指標，或是我們制定的早期AD檢測評估方法，我們的模型都能比基線模型擁有更好的效能。此外相較於單注意力模型，我們的模型也擁有更高的解釋性。最後，我們利用注意力圖，以及使用Grad-CAM在特徵提取器中取得特徵圖的熱力圖，由此來深入了解模型的決策過程。

Alzheimer’s disease (AD) is an enduring, irreversible neurodegenerative disorder that has persisted in human society for centuries. This affliction is characterized by the deterioration of cognition and memory, ultimately culminating in physical decline and mortality. The resultant economic and psychological burdens on society are compounded by the challenge of distinguishing AD symptoms from the natural aging process. Present treatments can merely defer symptom exacerbation and the distinction between AD symptoms and normal aging is complex. Early detection of AD symptoms is a fundamental goal of the medical profession, as it offers the potential to attenuate symptom progression. With the advancement of technology, more and more machine-learning methods have been used to identify AD in the last few decades. In this study, our focus centers on identifying those at risk of developing AD in the future. We purpose a Multi-attention GMU model, which is inspired by additive attention and Gated Multimodel Unit (GMU) structure. Our model is trained using a specifically designed partitioning approach on a training dataset. Additionally, we formulate an evaluative methodology to demonstrate the model’s efficacy in early AD detection. In addition to traditional evaluation metrics such as accuracy and F1-score, our model outperform baseline models using our unique evaluation method in the context of early detection strategies. Besides, our model have more interpretability than singleattention model. Finally, we employ Grad-CAM heatmaps and attention maps of each stage block within the feature extractor to provide insights into the model’s decision-making process.

1 Introduction 1
2 Related works 4
3 Methodology 7
3.1 Data source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1.1 Electronic Health Record . . . . . . . . . . . . . . . . . . . . 7
3.1.2 Magnetic Resonance Imaging . . . . . . . . . . . . . . . . . 9
3.1.3 Label . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Data selection and matching . . . . . . . . . . . . . . . . . . . . . . 10
3.2.1 Data selection . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2.2 Feature selection . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3 Data pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.3.1 EHR pre-processing . . . . . . . . . . . . . . . . . . . . . . . 11
3.3.2 MRI pre-processing . . . . . . . . . . . . . . . . . . . . . . . 13
3.4 Data splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.5 Baseline models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.5.1 Algorithmic models of EHR . . . . . . . . . . . . . . . . . . 21
3.5.2 NN models of EHR . . . . . . . . . . . . . . . . . . . . . . . 23
3.5.3 ResNet models of MRI . . . . . . . . . . . . . . . . . . . . . 28
3.5.4 Fusion models . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.6 Purposed models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.7 Model interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4 Results 44
4.1 Performance analyze . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.1.2 Confusion matrix . . . . . . . . . . . . . . . . . . . . . . . . 47
4.1.3 Precision, Recall and F1-score . . . . . . . . . . . . . . . . . 48
4.1.4 ROC curves and AUC . . . . . . . . . . . . . . . . . . . . . 52
4.1.5 Tuning attention blocks and number of GMUs . . . . . . . . 54
4.2 Early detection of transition . . . . . . . . . . . . . . . . . . . . . . 57
4.2.1 Method of evaluation . . . . . . . . . . . . . . . . . . . . . . 57
4.2.2 Comparison of models . . . . . . . . . . . . . . . . . . . . . 62
4.2.3 Comparison of thresholds . . . . . . . . . . . . . . . . . . . 64
4.2.4 Comparison of different tolerance rate . . . . . . . . . . . . . 67
4.3 Model interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.3.1 Grad-CAM . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.3.2 Attention maps . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.3.3 Relationship of Grad-CAM and attention maps . . . . . . . 77
4.3.4 Relationship of EHR features and attention maps . . . . . . 80
4.4 Performance analysis of different splitting method . . . . . . . . . . 86
4.4.1 Data splitting and early-detecting evaluation . . . . . . . . . 86
4.4.2 Analysis of early detection . . . . . . . . . . . . . . . . . . . 87
4.4.3 Probability of AD in before-transition testing data . . . . . 88
5 Discussion 90
6 Conclusion 92
7 Appendix 98
7.1 Experiment of RIDs splitting . . . . . . . . . . . . . . . . . . . . . 98
7.2 Cropping method of pre-processing . . . . . . . . . . . . . . . . . . 100

[1] Fayyaz Ahmad et al. “Determination of affected brain regions at various stages of Alzheimer’s disease”. In: Neuroscience Research 192 (2023), pp. 77– 82.
[2] Rabah AlShboul et al. “The Application of Intelligent Data Models for Dementia Classification”. In: Applied Sciences 13.6 (2023), p. 3612.
[3] John Arevalo et al. “Gated multimodal units for information fusion”. In: arXiv preprint arXiv:1702.01992 (2017).
[4] Sercan O Arik and Tomas Pfister. “Tabnet: Attentive interpretable tabular¨ learning”. In: Proceedings of the AAAI conference on artificial intelligence. Vol. 35. 8. 2021, pp. 6679–6687.
[5] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. “Neural machine translation by jointly learning to align and translate”. In: arXiv preprint arXiv:1409.0473 (2014).
[6] Benjamin Billot et al. “SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining”. In: Medical image analysis 86 (2023), p. 102789.
[7] Leo Breiman. “Statistical modeling: The two cultures (with comments and a rejoinder by the author)”. In: Statistical science 16.3 (2001), pp. 199–231.
[8] Pierluigi Carcagnı et al. “Convolution Neural Networks and Self-Attention Learners for Alzheimer Dementia Diagnosis from Brain MRI”. In: Sensors
23.3 (2023), p. 1694.
[9] Tianqi Chen and Carlos Guestrin. “Xgboost: A scalable tree boosting system”. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 2016, pp. 785–794.
[10] Zhi Chen et al. “Orthogonal latent space learning with feature weighting and
Graph Learning for multimodal Alzheimer’s disease diagnosis”. In: Medical Image Analysis 84 (2023), p. 102698.
[11] Mingxi Dang et al. “Hippocampus-centred grey matter covariance networks predict the development and reversion of mild cognitive impairment”. In: Alzheimer’s Research & Therapy 15.1 (2023), p. 27.
[12] Lingyun Guo et al. “Multi-band network fusion for Alzheimer’s disease identification with functional MRI”. In: Frontiers in Psychiatry 13 (2022), p. 1070198.
[13] Kaiming He et al. “Deep residual learning for image recognition”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 770–778.
[14] Alekseı Grigorevich Ivakhnenko, Valentin Grigorevich Lapa, et al. “Cybernetics and forecasting techniques”. In: (No Title) (1967).
[15] Saumya Jetley et al. “Learn to pay attention”. In: arXiv preprint arXiv:1804.02391
(2018).
[16] Cheng Jiang et al. “Fusion of medical imaging and electronic health records with attention and multi-head machanisms”. In: arXiv preprint arXiv:2112.11710
(2021).
[17] Young Ju Kim et al. “Classification and prediction of cognitive trajectories of cognitively unimpaired individuals”. In: Frontiers in Aging Neuroscience 15 (2023), p. 1122927.
[18] Diederik P Kingma and Jimmy Ba. “Adam: A method for stochastic optimization”. In: arXiv preprint arXiv:1412.6980 (2014).
[19] Yann LeCun et al. “Gradient-based learning applied to document recognition”. In: Proceedings of the IEEE 86.11 (1998), pp. 2278–2324.
[20] Fei Liu et al. “MPS-FFA: A multiplane and multiscale feature fusion attention network for Alzheimer’s disease prediction with structural MRI”. In:
Computers in Biology and Medicine 157 (2023), p. 106790.
[21] Yuyang Liu et al. “An unsupervised learning approach to diagnosing Alzheimer’s disease using brain magnetic resonance imaging scans”. In: International Journal of Medical Informatics 173 (2023), p. 105027.
[22] Adam Paszke et al. “PyTorch: An Imperative Style, High-Performance Deep Learning Library”. In: Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 2019, pp. 8024–8035. url: http://papers.
neurips.cc/paper/9015-pytorch-an-imperative-style-high-performancedeep-learning-library.pdf.
[23] Fabian Pedregosa et al. “Scikit-learn: Machine learning in Python”. In: the
Journal of machine Learning research 12 (2011), pp. 2825–2830.
[24] Ronald Carl Petersen et al. “Alzheimer’s disease neuroimaging initiative
(ADNI): clinical characterization”. In: Neurology 74.3 (2010), pp. 201–209.
[25] Raveendra REDDY ENUMULA and Rama KRISHNA RAO. “Alzheimer’s
disease prediction and classification using CT images through machine learning.” In: Bratislava Medical Journal/Bratislavske Lekarske Listy 124.5 (2023).
[26] Jo Schlemper et al. “Attention gated networks: Learning to leverage salient regions in medical images”. In: Medical image analysis 53 (2019), pp. 197– 207.
[27] Ramprasaath R Selvaraju et al. “Grad-cam: Visual explanations from deep networks via gradient-based localization”. In: Proceedings of the IEEE international conference on computer vision. 2017, pp. 618–626.
[28] Philip H Swain and Hans Hauska. “The decision tree classifier: Design and potential”. In: IEEE Transactions on Geoscience Electronics 15.3 (1977), pp. 142–147.
[29] Mingxing Tan and Quoc Le. “Efficientnet: Rethinking model scaling for convolutional neural networks”. In: International conference on machine learning. PMLR. 2019, pp. 6105–6114.
[30] Janani Venugopalan et al. “Multimodal deep learning models for early detection of Alzheimer’s disease stage”. In: Scientific reports 11.1 (2021), p. 3254.
[31] Haibo Wang et al. “Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features”. In:
Journal of Medical Imaging 1.3 (2014), pp. 034003–034003.
[32] Bolei Zhou et al. “Learning deep features for discriminative localization”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 2921–2929.