隨著服務設計越來越重要，許多方法論也發展出來已做使用者研究。然而這些方法論並非適用于所有情形，例如研究者無法直接觀察或記錄受試者。為開發一套分法論用以偵測受試者的心里狀態，此研究採用消費性腦機裝置，受試者穿戴此裝置並且接受英文聽力測驗，同時標示題目的難易度。我們採用支持向量機器，以五秒的特徵集來分類，並以 F-measure、精准率以及查全率來衡量，結果呈現最高的 F-measure 為 0.353。此外，部分結果顯示特徵值越多可能導致表現變差。此言就呈現了可行的分類器以及特徵集，並且設計現實生活中可能的語言學習方式。

As the service design becomes more important and highly related to customers’ needs, there are several methodologies could be used for user research. However, these existed methodologies may not be able to fit to all situations; for example, researchers cannot directly observe and record users’ mental states. In this research, we adopt a commercial EEG-based device with single electrode in the setting of listening to English to record their mental states. This research aims to develop a classifier that can predict users’ mental efforts via experiments in which 35 college students were asked to listen to English and respond their mental states, easy or difficult, before choosing the answer for each question. We chose Support Vector Machine (SVM) as the classifier, and valuated its performance in terms of recall, precision, and F-measure in five different feature models with different time windows of extracting EEG data. The results indicate that one of the proposed features gives the highest F-measure of 0.353 with precision rate of 65.8% and recall rate of 57.1% from EEG data extracted in the time window of five seconds. Besides, in some situations, the more features we include for classification, the lower F-measure scores the system obtains. This research has contributed to the literature that a classification of mental effort using SVM classifier is effective to capture users’ mental states in the process of listening English and comprehension. The embedded SVM classifier could be used for detecting users’ mental state in service design process in the real world application on language learning activities. Additional efforts can be made to extend its applications on different service contexts.

Chapter 1. Introduction 1
1.1 Research Background 1
1.1.1 Service design 1
1.1.2 EEG-based BCI wearable device 1
1.2 Research Motivation 2
1.3 Research Objectives 3
Chapter 2. Literature review 4
2.1 Service design process 4
2.2 English learning 6
2.3 Non-invasive Brain Computer Interface 7
2.3.1 Electroencephalography (EEG) 7
2.3.2 Magnetoencephalography (MEG) 9
2.3.3 Functional Magnetic Resonance Imaging (fMRI) 10
2.3.4 Conclusion of BCI 10
2.4 EEG-based Brain State Recognition using Classification Techniques 11
2.4.1 Bayes Classifier 15
2.4.2 Supervised Artificial Neural Network 15
2.4.3 K-Nearest Neighbors 16
2.4.4 Support Vector Machine 16
2.4.5 The Event Related Potential 18
2.4.6 The assessment of Neurosky headset 18
Chapter 3. Research Methodology 19
3.1 English listening system 19
3.2 The training of a mental state classification model 20
Chapter 4. Experimental Design 22
4.1 Experimental Scenarios 22
4.2 Experimental Setting 22
4.3 Experimental Process 23
4.4 Subjects 24
Chapter 5. Experimental Results and Discussion 25
5.1 Data Overview 25
5.2 Data Analysis 25
5.3 Feature Selection 25
5.4 F-measure from SVM Classifier 26
5.4.1 F-measure of Model 1 27
5.4.2 F-measure of Model 2 29
5.4.3 F-measure of Model 3 31
5.4.4 F-measure of Model 4 34
5.4.5 F-measure of Model 5 36
5.5 Summary of results of different models 37
Chapter 6. Limitations and Future work 41
Chapter 7. Conclusion 42
References 44
Appendix 49

Allen, J. J. B., Harmon-Jones, E., & CavenderJ.H. (2001). Manipulation of frontal EEG asymmetry through biofeedback alters self-reported emotional responses and facial EMG. Psychophysiology, 38, 685-693.
Arslan, B., Brouse, A., Castet, J., Lehembre, R., Simon, C., Filatriau, J. J., & Noirhomme, Q. (2006). A real time music synthesis environment driven with biological signals. Paper presented at the ICASSP.
Bergen, D. (1988). ENcyclopedia of neuroscience. JAMA, 260(1), 104-104. doi: 10.1001/jama.1988.03410010112050
Bos, D. (2007). EEG-based emotion recognition. The Influence of Visual and Auditory Stimuli. University of Twente. Retrieved from
Bos, D. O. (2006). EEG-based Emotion Recognition - The Influence of Visual and Auditory Stimuli. Emotion, 57(7), 1798-1806. doi: 10.1109/TBME.2010.2048568
Bradley, M. M., & Lang, P. J. (1994). Measuring Emotion: The Self-assessment Manikin and the Semantic Differential Journal of Behavior Therapy and Experimental Psychiatry, 25, 49-59.
Cahn, B. R., & John, P. (2006). Meditation states and traits: EEG, ERP, and neuroimaging studies. Psychological Bulletin, 132 (2), 180-211.
Chawla, N. V., Japkowicz, N., & Kotcz, A. (2004). Editorial: special issue on learning from imbalanced data sets. SIGKDD Explor. Newsl., 6(1), 1-6. doi: 10.1145/1007730.1007733
Coan, J., & Allen, J. J. B. (2004). Frontal EEG asymmetry as a moderator and mediator of emotion. Biological Psychology, 67, 7-49.
Coles, G.H., M., & Rugg, M. D. (1996). Event-related brain potentials: an introduction. 1-27.
Crowley, K., Sliney, A., Pitt, I., & Murphy, D. (2010). Evaluating a Brain-Computer Interface to Categorise Human Emotional Response. IEEE International Conference on Advanced Learning Technologies, 10, 276-278.
del R Millan, J., Mourino, J., Franze, M., Cincotti, F., Varsta, M., Heikkonen, J., & Babiloni, F. (2002). A local neural classifier for the recognition of EEG patterns associated to mental tasks. Neural Networks, IEEE Transactions on, 13(3), 678-686. doi: 10.1109/TNN.2002.1000132
Demšar, J., Curk, T., & Erjavec, A. (2013). Orange: Data Mining Toolbox in Python. Journal of Machine Learning Research, 2349−2353.
Ertl, M. (2013). Emotion regulation by cognitive reappraisal — The role of frontal theta oscillations. NeuroImage, 81, 412-421.
Fei, S., Liwen, X., Cai, A., Yibing, W., & Junshui, M. (2010, 23-26 Aug. 2010). EEG-based Personal Identification: from Proof-of-Concept to A Practical System. Paper presented at the Pattern Recognition (ICPR), 2010 20th International Conference on.
Garrett, D., Peterson, D. A., Anderson, C. W., & Thaut, M. H. (2003). Comparison of linear, nonlinear, and feature selection methods for EEG signal classification. Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 11(2), 141-144. doi: 10.1109/TNSRE.2003.814441
Goh, C. C. M. (2000). A cognitive perspective on language learners' listening comprehension problems. System, 28(1), 55-75.
Goldstein, S. M., Johnston, R., Duffy, J., & Rao, J. (2002). The service concept: the missing link in service design research? Journal of Operations Management, 20(2), 121-134.
Han, J. W. (2005). Data mining: concepts and techniques (3nd ed.). Singapore: Morgan kaufmann.
Harmony, T., Fernández, T., Silva, J., Bernal, J., Díaz-Comas, L., Reyes, A., . . . Rodríguez, M. (1996). EEG delta activity: an indicator of attention to internal processing during performance of mental tasks. International Journal of Psychophysiology, 24(1–2), 161-171.
He, H., & Garcia, E. A. (2009). Learning from Imbalanced Data. IEEE Trans. on Knowl. and Data Eng., 21(9), 1263-1284. doi: 10.1109/tkde.2008.239
Henriques, J. B., & Davidson, R. J. (1991). Left Frontal Hypoactivation in Depression Journal of Abnormal Psychology, 100, 535-545.
Huettel, S. A., McCarthy, G., & Song, A. W. (2009). Functional Magnetic Resonance Imaging (2 ed. ed.). Massachusetts: Sinauer.
Jain, A. K., Jianchang, M., & Mohiuddin, K. M. (1996). Artificial neural networks: a tutorial. Computer, 29(3), 31-44. doi: 10.1109/2.485891
Jrad, N., Congedo, M., Phlypo, R., Rousseau, S., Flamary, R., Yger, F., & Rakotomamonjy, A. (2011). sw-SVM: sensor weighting support vector machines for EEG-based brain–computer interfaces. Journal of Neural Engineering, 8(5), 056004.
Kahana, M. J., Sekuler, R., Caplan, J. B., Kirschen, M., & Madsen, J. R. (1999). Human theta oscillations exhibit task dependence during virtual maze navigation. Nature, 399(6738), 781-784.
Kaiyang, L., Xiaodong, Z., & Yuhuan, D. (2013, Oct. 30 2013-Nov. 2 2013). A SVM based classification of EEG for predicting the movement intent of human body. Paper presented at the Ubiquitous Robots and Ambient Intelligence (URAI), 2013 10th International Conference on.
Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews, 29(2–3), 169-195.
Kottaimalai, R., Pallikonda Rajasekaran, M., Selvam, V., & Kannapiran, B. (2013). EEG Signal Classification using Principal Component Analysis with Neural Network in Brain Computer Interface Applications. Paper presented at the IEEE International Conference on Emerging Trends in Computing, Communication and Nanotechnology.
Kumar, V. (2012). Design Method: A STRUCTURED APPROACH FOR DRIVING INNOVATION IN YOUR ORGANIZATION. New Jersey: John Willy & Sons.
Kwang-Ok, A., Jong-Bae, K., Won-Kyoung, S., & In-Ho, L. (2010, 26-29 Sept. 2010). Development of an emergency call system using a brain computer interface (BCI). Paper presented at the Biomedical Robotics and Biomechatronics (BioRob), 2010 3rd IEEE RAS and EMBS International Conference on.
Larsen, E. A. (2011). Classification of EEG Signals in a Brain-Computer Interface System. Norwegian University of Science and Technology.
Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., & Arnaldi, B. (2007). TOPICAL REVIEW: A review of classification algorithms for EEG-based brain computer interfaces. Journal of Neural Engineering, 4(2). doi: 10.1088/1741-2560/4/2/R01
Mak, J. N., Chan, R. H. M., & Wong, S. W. H. (2013, 10-13 Nov. 2013). Evaluation of mental workload in visual-motor task: Spectral analysis of single-channel frontal EEG. Paper presented at the Industrial Electronics Society, IECON 2013 - 39th Annual Conference of the IEEE.
Miltner, W. H. R., Braun, C., ArnoldMatthias, Witte, H., & Taub, E. (1999). Coherence of gamma-band EEG activity as a basis for associative learning. Nature, 397(6718), 434-436.
Mostow, J., Chang, K.-M., & Nelson, J. (2011). Toward exploiting EEG input in a reading tutor. Paper presented at the Proceedings of the 15th international conference on Artificial intelligence in education, Auckland, New Zealand.
Murugappan, M., & Murugappan, S. (2013). Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT). Paper presented at the IEEE International Colloquium on Signal Processing and its Applications 9th.
Murugappan, M., Ramachandran, N., & Sazali, Y. (2010). Classification of human emotion from EEG using discrete wavelet transform. Journal of Biomedical Science and Engineering, 3, 390-396.
NEUROSKY. (2009). Brain Wave Signal (EEG)
Niedermeyer, E., & Silva, F. L. d. (2004). Electroencephalography: Basic Principles, Clinical Applications, and Related Fields: Lippincot Williams & Wilkins.
Niemic, C. P. (2002). Studies of emotion: A theoretical and empirical review of psychophysiological studies of emotion. Journal of Undergraduate Research, 1, 15-18.
Peng, H., Hu, B., Liu, Q. Y., Dong, Q. X., Zhao, Q. L., & Moore, P. (2011, 23-26 May 2011). User-centered depression prevention: An EEG approach to pervasive healthcare. Paper presented at the Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2011 5th International Conference on.
Perez, J., & Delechelle, E. (2013, 17-19 June 2013). On the measurement of image quality perception using frontal EEG analysis. Paper presented at the Smart Communications in Network Technologies (SaCoNeT), 2013 International Conference on.
Pfurtscheller, G., & Lopes Da Silva, F. H. (1999). Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clinical Neurophysiology, 110 (11), 1842-1857.
Pine, I. I. B. J., & Gilmore, J. H. (1998). WELCOME TO THE EXPERIENCE ECONOMY. Harvard Business Review, 76(4), 97-105.
Sahito, F., & Wolfgang, S. (2012). Functional Magnetic Resonance Imaging and the Challenge of Balancing Human Security with State Security. Human Security Perspectives, 1, 38-66.
Schacter, D. L. (1977). EEG theta waves and psychological phenomena: A review and analysis. Biological Psychology, 5(1), 47-82.
Seo, S., Gil, Y., & Lee, J. (2008). The relation between affective style of stressor on EEG asymmetry and stress scale during multi-modal task. Paper presented at the Third 2008 International Conference on Convergence and Hybrid Information Technology.
Spengler, C., Wirth, W., & Sigrist, R. (2010). 360-Degree-Touchpoint-Management - How important is twitter for our brand? Content published in: Marketing Review 14-20.
Srinivasa, K. G., Singh, A., Thomas, A. O., Venugopal, K. R., & Patnaik, L. M. (2005, 14-17 Dec. 2005). Generic Feature Extraction for Classification using Fuzzy C - Means Clustering. Paper presented at the Intelligent Sensing and Information Processing, 2005. ICISIP 2005. Third International Conference on.
Stickdorn, M., & Schneider, J. (2010). THIS IS SERVICE DESIGN THINKING. Netherland: BIS Publishers.
Subasia, A., & Ercelebi, E. (2005). Classification of EEG signals using neural network and logistic regression. Computer Methods and Programs in Biomedicine, 78, 87-99.
Tatum, W. O., Husain, A. M., & Benbadis, S. R. (2008). Handbook of EEG Interpretation: Demos Medical.
Vempati, S., Vedaldi, A., Zisserman, A., & Jawahar, C. V. (2010). Generalized RBF feature maps for Efficient Detection (F. e. e. a. Z. Labrosse, Reyer and Liu, Yonghuai and Tiddeman, Bernie Ed.): BMVA Press.
Vidal, J. (1973). Toward direct brain-computer communication. Annual review of biophysics and bioengineering, 2, 157-180.
Vladimir, F., Ruth, R., & V., R. (2001). Multiplicity of the α Rhythm in Normal Humans. Journal of Clinical Neurophysiology, 18 (4), 331-344.
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clinical Neurophysiology, 113(6), 767-791.
Xu, H., & Plataniotis, K. N. (2012). Affect recognition using EEG signal. Paper presented at the Multimedia Signal Processing (MMSP), Banff, AB.
Yoon, H. (2013). Emotion Recognition of Serious Game Players Using a Simple Brain Computer Interface. IEEE ICTC, 783-786.
Yuan, Y., Li, Y., Yu, D., & D.P., M. (2008). Delay Time-Based Epileptic EEG Detection Using Artificial Neural Network. Paper presented at the Bioinformatics and Biomedical Engineering.
Yuen, C. T., San, W. S., Seong, T. C., & Rizon, M. (2009). Classification of Human Emotions from EEG Signals using Statistical Features and Neural Network. International Journal of Integrated Engineering, 1(3), 71 - 79.
Zhang, Y., Kong, F., Chen, H., Jackson, T., Han, L., Meng, J., . . . Hasan, A. N. (2011). Identifying Cognitive Preferences for Attractive Female Faces: An Event-Related Potential Experiment Using a Study-Test Paradigm. Journal of Neuroscience Research, 89, 1887-1893.