嗅覺在日常生活中扮演不可或缺的重要角色，可以影響我們的情緒、行為以
及記憶等。在我們的研究中，我們試著去解析嗅覺是如何對生心理產生影響並
了解它們是否能反應出我們的主觀想法。我們首先將43種不同味道根據描述以
及主觀問卷進行標註後得到氣味印象空間。然後利用表徵相似性分析(RSA)及
階層式分群將氣味印象表示成表徵不相似性矩陣(RDM)。利用矩陣選出10種氣
味後，我們設計出自然氣味感知實驗。我們在實驗中記錄四種生理指標：腦電
圖、心電圖、嗅球電圖(EBG)和臉部表徵來構建多模態氣味生理空間。我們利
用RSA和機器學習分類器計算氣味印象與生理反應的相關性。根據RSA，我們
發現在氣味多模態生理空間中可以反應出氣味印象。在機器學習中，利用生理
資料所獲得的分類結果超過了基本水準。分類結果也被用來觀測研究大腦中處
理氣味的神經機制。總和來說，本研究利用兩種方式解析氣味知覺，並且了解
擁有眾多特徵的氣味是如何影響人類的反應。

The sense of smell plays a significant role in our daily lives and can have a dynamic influence on our emotions, behavior, and memories. In our study, we attempt to decode how odors affect the psychophysiological response and realize whether they could reveal subjective experience. We labeled 43 types of fragrances based on collective impressions extracted from description feedback and a subjective questionnaire. Then we computed these annotations with representational similarity analysis (RSA) and hierarchical clustering to obtain odor impressional representational dissimilarity matrix (RDM). After selecting ten odors by odor impressional RDM, we designed a naturalistic odor perception experiment. We analyzed four physiological metrics: electroencephalogram (EEG), electrocardiogram (ECG), electrobulbograms (EBG), and facial dynamics of ten participants while perceiving scents, and further built multimodal physiological spaces. Then we examined the correlation between odor impressional and physiological responses by spatial/temporal RSA and machine learning (ML) classifier. By RSA, our findings showed that the representation in the odor impressional space could be revealed by the representation in multimodal physiological spaces. By ML classification, the performances of classifying odors from physiological data were above the chance level. The classification results can also be used to investigate the neural mechanism for processing odors in the brain. Taken together, we decoded odor perception through two approaches, and from both of them, we recognized how odors with numerous characteristics affected human response.

Contents
Abstract (Chinese) I
Abstract II
Acknowledgements (Chinese) IV
Contents V
List of Figures VIII
List of Tables XI
1 Introduction 1
2 Related works 5
3 Methodlogy 10
3.1 Odors selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.1.1 Labeling collective experience of odors . . . . . . . . . . . . 12
3.1.2 Fragrance impressions questionnaire . . . . . . . . . . . . . . 13
3.2 Naturalistic odor perception experiment . . . . . . . . . . . . . . . 13
3.2.1 Procedure and stimulus . . . . . . . . . . . . . . . . . . . . 14
3.2.2 Olfactory display . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2.3 Multimodal data acquisition and recording . . . . . . . . . . 16
V
3.3 Data preprocessing and feature extraction . . . . . . . . . . . . . . 17
3.3.1 Electroencephalogram (EEG) . . . . . . . . . . . . . . . . . 17
3.3.2 Eelectrobulbogram (EBG) . . . . . . . . . . . . . . . . . . . 18
3.3.3 Eelectrocardiogram (ECG) . . . . . . . . . . . . . . . . . . . 18
3.3.4 Facial video recording . . . . . . . . . . . . . . . . . . . . . 19
3.4 Intersubject correlation (ISC) . . . . . . . . . . . . . . . . . . . . . 19
3.5 Representational Similarity Analysis (RSA) . . . . . . . . . . . . . 20
3.5.1 Odor Impressional RDM . . . . . . . . . . . . . . . . . . . . 21
3.5.2 Odor Physiological RDM . . . . . . . . . . . . . . . . . . . . 22
3.5.3 Spatial RSA . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5.4 Temporal RSA (tRSA) . . . . . . . . . . . . . . . . . . . . . 23
3.6 Machine learning (ML) classifier . . . . . . . . . . . . . . . . . . . . 24
3.6.1 Subject-wise binary classification model . . . . . . . . . . . . 24
3.6.2 Prediction polling . . . . . . . . . . . . . . . . . . . . . . . . 25
3.6.3 Differentiating impressions of odors using the classification
model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4 Result 27
4.1 Subjective impression . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Result of odor perception experiment . . . . . . . . . . . . . . . . . 29
4.2.1 Result of ISC . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2.2 Spatial RSA in session 1 . . . . . . . . . . . . . . . . . . . . 30
4.2.3 tRSA in session 1 . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2.4 Spatial RSA in session 2 . . . . . . . . . . . . . . . . . . . . 32
4.2.5 tRSA in session 2 . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2.6 Binary classification results . . . . . . . . . . . . . . . . . . 34
4.2.7 Prediction Polling . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.8 Result of differentiating impressions of odors . . . . . . . . . 40
5 Discussion 41
5.1 Odor characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.2 Challenges in decoding odor perception through ML . . . . . . . . . 42
5.3 Temporal processing in odor perception . . . . . . . . . . . . . . . . 42
5.4 Brain regions and EEG features associated with odor perception . . 43
5.5 Comparisons between RSA and ML classifier . . . . . . . . . . . . . 43
5.6 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.7 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
6 Conclusion 46
Bibliography 47

[1] Judith Amores, Javier Hernandez, Artem Dementyev, Xiqing Wang, and Pattie Maes. Bioessence: a wearable olfactory display that monitors cardiorespiratory information to support mental wellbeing. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 5131–5134. IEEE.
[2] Judith Amores and Pattie Maes. Essence: Olfactory interfaces for unconscious influence of mood and cognitive performance. In Proceedings of the 2017 CHI conference on human factors in computing systems, pages 28–34.
[3] Yijing Xiao, Burcin Becerik-Gerber, Gale Lucas, and Shawn C Roll. Impacts of working from home during covid-19 pandemic on physical and mental wellbeing of office workstation users. Journal of Occupational and Environmental Medicine, 63(3):181, 2021.
[4] Helen B Treloar, Alexandra M Miller, Arundhati Ray, and Charles A Greer. Development of the olfactory system. The neurobiology of olfaction, 20092457:131–155, 2010.
[5] Donald A Wilson, Mikiko Kadohisa, and Max L Fletcher. Cortical contributions to olfaction: plasticity and perception. In Seminars in cell developmental biology, volume 17, pages 462–470. Elsevier.
[6] H. Varendi and Rh Porter. Breast odour as the only maternal stimulus elicits crawling towards the odour source. Acta Paediatrica, 90(4):372–375, 2007.
[7] Janina Seubert, Amy F. Rea, James Loughead, and Ute Habel. Mood induction with olfactory stimuli reveals differential affective responses in males and females. Chemical Senses, 34(1):77–84, 2008.
[8] Mikiko Kadohisa. Effects of odor on emotion, with implications. Frontiers in systems neuroscience, 7:66, 2013.
[9] Behzad Iravani, Martin Schaefer, Donald A Wilson, Artin Arshamian, and Johan N Lundstr¨om. The human olfactory bulb processes odor valence representation and cues motor avoidance behavior. Proceedings of the National Academy of Sciences, 118(42):e2101209118, 2021.
[10] C Sarafoleanu, C Mella, M Georgescu, and C Perederco. The importance of the olfactory sense in the human behavior and evolution. Journal of Medicine and life, 2(2):196, 2009.
[11] Manon Genva, Tierry Kenne Kemene, Magali Deleu, Laurence Lins, and Marie-Laure Fauconnier. Is it possible to predict the odor of a molecule on the basis of its structure? International journal of molecular sciences, 20(12):3018, 2019.
[12] Artin Arshamian, Richard C Gerkin, Nicole Kruspe, Ewelina Wnuk, Simeon Floyd, Carolyn O’Meara, Gabriela Garrido Rodriguez, Johan N Lundstrom, Joel D Mainland, and Asifa Majid. The perception of odor pleasantness is shared across cultures. Current Biology, 32(9):2061–2066. e3, 2022.
[13] He Xu, Eleni Kroupi, and Touradj Ebrahimi. Functional connectivity from eeg signals during perceiving pleasant and unpleasant odors. In 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pages 911–916. IEEE.
[14] Judith Amores, Robert Richer, Nan Zhao, Pattie Maes, and Bjoern M Eskofier. Promoting relaxation using virtual reality, olfactory interfaces and wearable eeg. In 2018 IEEE 15th international conference on wearable and implantable body sensor networks (BSN), pages 98–101. IEEE.
[15] Jaeseung Jeong. Eeg dynamics in patients with alzheimer’s disease. Clinical neurophysiology, 115(7):1490–1505, 2004.
[16] MA Becerra, E Londo˜no-Delgado, SM Pelaez-Becerra, L Serna-Guarın, AE Castro-Ospina, D Marin-Castrill´on, and DH Peluffo-Ordonez. Odor pleasantness classification from electroencephalographic signals and emotional states. In Colombian Conference on Computing, pages 128–138. Springer.
[17] Eleni Kroupi, Ashkan Yazdani, Jean-Marc Vesin, and Touradj Ebrahimi. Eeg correlates of pleasant and unpleasant odor perception. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 11(1s):1–17, 2014.
[18] Mugihiko Kato, Toshiki Okumura, Yasuhiro Tsubo, Junya Honda, Masashi Sugiyama, Kazushige Touhara, and Masako Okamoto. Spatiotemporal dynamics of odor representations in the human brain revealed by eeg decoding. Proceedings of the National Academy of Sciences, 119(21):e2114966119, 2022.
[19] Behzad Iravani, Artin Arshamian, Kathrin Ohla, Donald A Wilson, and Johan N Lundstrom. Non-invasive recording from the human olfactory bulb. Nature communications, 11(1):1–10, 2020.
[20] Shafiul Omam, Mohammad Hossein Babini, Sue Sim, Rui Tee, Visvamba Nathan, Soheil Gohari, Colin Burvill, Kamil Kuca, Ondrej Krejcar, and Hamidreza Namazi. Decoding of the coupling between brain and skin activities in olfactory stimulation by analysis of eeg and gsr signals. Waves in Random and Complex Media, pages 1–15, 2021.
[21] Alessandro Tonacci, Jessica Di Monte, Maria Beatrice Meucci, Francesco Sansone, Anna Paola Pala, Lucia Billeci, and Raffaele Conte. Wearable sensors to characterize the autonomic nervous system correlates of food-like odors perception: A pilot study. Electronics, 8(12):1481, 2019.
[22] Najmeh Pakniyat, Mirra Soundirarajan, Soheil Gohari, Colin Burvill, Ondrej Krejcar, and Hamidreza Namazi. Decoding of facial muscle-brain relation by information-based analysis of electromyogram (emg) and electroencephalogram (eeg) signals. Waves in Random and Complex Media, pages 1–10, 2021.
[23] WR Klemm, SD Lutes, DV Hendrix, and Stephen Warrenburg. Topographical eeg maps of human responses to odors. Chemical senses, 17(3):347–361, 1992.
[24] Byung-Chan Min, Seung-Hyun Jin, In-Hyeng Kang, Dong Hyung Lee, Jin Kyu Kang, Sang Tae Lee, and Kazuyoshi Sakamoto. Analysis of mutual information content for eeg responses to odor stimulation for subjects classified by occupation. Chemical senses, 28(9):741–749, 2003.
[25] Agnieszka Sorokowska, Piotr Sorokowski, and Thomas Hummel. Crosscultural administration of an odor discrimination test. Chemosensory perception, 7(2):85–90, 2014.
[26] Hui-Rang Hou, Xiao-Nei Zhang, and Qing-Hao Meng. Odor-induced emotion recognition based on average frequency band division of eeg signals. Journal of neuroscience methods, 334:108599, 2020.
[27] Kiana Ezzatdoost, Hadi Hojjati, and Hamid Aghajan. Decoding olfactory stimuli in eeg data using nonlinear features: A pilot study. Journal of Neuroscience Methods, 341:108780, 2020.
[28] Pallavi Kaushik, Amir Moye, Marieke van Vugt, and Partha Pratim Roy. Decoding the cognitive states of attention and distraction in a real-life setting using eeg. Scientific Reports, 12(1):20649, 2022.
[29] Hossein Shahabi and Sahar Moghimi. Toward automatic detection of brain responses to emotional music through analysis of eeg effective connectivity. Computers in Human Behavior, 58:231–239, 2016.
[30] S. Koelstra, C. Muhl, M. Soleymani, J. S. Lee, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras. Deap: A database for emotion analysis; using physiological signals. IEEE Transactions on Affective Computing, 3(1):18–31, 2012.
[31] Mohammad Soleymani, Jeroen Lichtenauer, Thierry Pun, and Maja Pantic. A multimodal database for affect recognition and implicit tagging. IEEE transactions on affective computing, 3(1):42–55, 2011.
[32] Shing-On Leung. A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point likert scales. Journal of social service research, 37(4):412–421, 2011.
[33] Christian Kothe. Lab streaming layer (lsl), 2014.
[34] Fred Shaffer and Jay P Ginsberg. An overview of heart rate variability metrics and norms. Frontiers in public health, page 258, 2017.
[35] Nikolaus Kriegeskorte and Rogier A Kievit. Representational geometry: integrating cognition, computation, and the brain. Trends in cognitive sciences, 17(8):401–412, 2013.
[36] Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. Representational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience, page 4, 2008.
[37] Jorn Diedrichsen and Nikolaus Kriegeskorte. Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis. PLoS computational biology, 13(4):e1005508, 2017.
[38] Emily S Finn, Enrico Glerean, Arman Y Khojandi, Dylan Nielson, Peter J Molfese, Daniel A Handwerker, and Peter A Bandettini. Idiosynchrony: From shared responses to individual differences during naturalistic neuroimaging. NeuroImage, 215:116828, 2020.
[39] Ian Charest, Rogier A Kievit, Taylor W Schmitz, Diana Deca, and Nikolaus Kriegeskorte. Unique semantic space in the brain of each beholder predicts perceived similarity. Proceedings of the National Academy of Sciences, 111(40):14565–14570, 2014.
[40] Junichi Chikazoe, Daniel H Lee, Nikolaus Kriegeskorte, and Adam K Anderson. Population coding of affect across stimuli, modalities and individuals. Nature neuroscience, 17(8):1114–1122, 2014.
[41] Carmel A Levitan, Jiana Ren, Andy T Woods, Sanne Boesveldt, Jason S Chan, Kirsten J McKenzie, Michael Dodson, Jai A Levin, Christine XR Leong, and Jasper JF Van den Bosch. Cross-cultural color-odor associations. PloS one, 9(7):e101651, 2014.
[42] Linda Buck and Richard Axel. A novel multigene family may encode odorant receptors: a molecular basis for odor recognition. Cell, 65(1):175–187, 1991.
[43] Regina M Sullivan, Donald A Wilson, Nadine Ravel, and Anne-Marie Mouly. Olfactory memory networks: from emotional learning to social behaviors, 2015.
[44] Jasper HB de Groot, Monique AM Smeets, Matt J Rowson, Patricia J Bulsing, Cor G Blonk, Joy E Wilkinson, and G¨un R Semin. A sniff of happiness. Psychological science, 26(6):684–700, 2015.
[45] Tyler S Lorig, Amy C Sapp, Jamie Campbell, and William S Cain. Eventrelated potentials to odor stimuli. Bulletin of the Psychonomic Society, 31(2):131–134, 1993.
[46] Jean-Marie Heydel, Philippe Faure, and Fabrice Neiers. Nasal odorant metabolism: enzymes, activity and function in olfaction. Drug Metabolism Reviews, 51(2):224–245, 2019. doi: 10.1080/03602532.2019.1632890.
[47] Joana Pereira, Patr´ıcia Costa, Maria C Coimbra, and Al´ırio E Rodrigues. The trail of perfumes. AIChE Journal, 64(7):2890–2897, 2018.
[48] Jean Carles. A method of creation in perfumery. FAFAI JOURNAL, 8(3):43, 2006.
[49] Kandhasamy Sowndhararajan and Songmun Kim. Influence of fragrances on human psychophysiological activity: With special reference to human electroencephalographic response. Scientia pharmaceutica, 84(4):724–752, 2016.
[50] Veit Frederik Kepler, Manuel S Seet, Junji Hamano, Mariana Saba, Nitish V Thakor, Stavros I Dimitriadis, and Andrei Dragomir. Odor pleasantness modulates functional connectivity in the olfactory hedonic processing network. Brain Sciences, 12(10):1408, 2022.
[51] Thien Nguyen, Sangtae Ahn, Hyojung Jang, Sung Chan Jun, and Jae Gwan Kim. Utilization of a combined eeg/nirs system to predict driver drowsiness. Scientific reports, 7(1):1–10, 2017.
[52] Jie Zheng, Rebecca F Stevenson, Bryce A Mander, Lilit Mnatsakanyan, Frank PK Hsu, Sumeet Vadera, Robert T Knight, Michael A Yassa, and Jack J Lin. Multiplexing of theta and alpha rhythms in the amygdalahippocampal circuit supports pattern separation of emotional information. Neuron, 102(4):887–898, 2019.
[53] Drew B Headley and Denis Pare. In sync: gamma oscillations and emotional memory. Frontiers in behavioral neuroscience, 7:170, 2013.
[54] Asifa Majid, Niclas Burenhult, Marcus Stensmyr, Josje De Valk, and Bill S Hansson. Olfactory language and abstraction across cultures. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1752):20170139, 2018.