情緒表達一直是非常微妙的，常因群體或個人的經驗、知識、信仰而有所不同。為了進一步瞭解文字中潛藏的情緒，我們需要一套機制來捕捉語言中微妙的情緒表達現象。有鑒於此，我們提出了一套半監督型以圖論為基礎的演算法。在這套演算法生成結構化的「構句樣式」之上，我們賦予其語境意義，從而構句樣式能夠從文字中捕捉情感表達。為了衡量構句樣式作為情感特徵的有效性，我們訓練了幾個中文及英文的情緒識別模型並且比較其準確度。實驗結果發現，我們提出的方法能夠擊敗目前最先進、最佳的情緒模型。以此為基礎，藉由構句樣式所賦予的高解釋性，我們還探究了性別在情緒表達上扮演了什麼樣的角色。

Emotions are expressed in nuanced ways, which varies by collective or individual experiences, knowledge, and beliefs. Therefore, to understand emotion, as conveyed through text, a robust mechanism capable of capturing and modeling different linguistic nuances and phenomena is needed. We propose a semi-supervised, graph-based algorithm to produce rich structural descriptors which serve as the building blocks for constructing contextualized affect representations from text. The pattern-based representations are further mingled with word clusters as lexicons and evaluated through several multilingual emotion recognition tasks, including English and Chinese short texts. Our experimental results demonstrate that the proposed method outperforms state-of-the-art techniques on emotion recognition tasks. To examine generalizability of our proposed patterns, we also extend the proposed representations to analyze how female and male differ in emotional expressions through its interpretability and generalizability.

Contents
1 Introduction .................................... 1
2 Related Work.................................... 6
2.1 Conventional Lexica ............................. 6
2.2 Emotion Corpora ............................... 7
2.3 Feature Representations............................ 7
3 Methodology.................................... 8
3.1 Overview ................................... 8
3.2 EmoPattern Extraction ............................ 10
3.2.1 Normalization............................. 10
3.2.2 Graph Construction.......................... 10
3.2.3 Graph Aggregation .......................... 11
3.2.4 Token Categorization......................... 11
3.2.5 Pattern Candidates .......................... 14
3.2.6 EmoPattern Extraction ........................ 14
3.3 Contextualized Pattern Enrichment...................... 15
3.3.1 Pre-trained Word Embeddings .................... 16
3.3.2 Word Clusters as Lexicons ...................... 16
3.3.3 Contextualized Pattern Construction by Mingling Lexicons ..... 17
3.3.4 Emotion Pattern Vectorization .................... 19
3.4 Emotion Recognition in English Short Texts ............. 20
3.4.1 Overview ............................... 20
3.4.2 Fine-graining Preprocessing ..................... 20
3.4.3 MINION ............................... 20
3.5 Emotion Recognition in a Gender-aware Context .... 22
3.5.1 Overview ............................... 22
3.5.2 Gender Classifier ........................... 24
3.5.3 Gender-aware Emotion Model .................... 24
3.6 Emotion Recognition in Multilingual Short Texts .... 26
3.6.1 Overview ............................... 26
3.6.2 Chinese Emotion Classifier...................... 26
4 Experiment&Results............................... 28
4.1 Experiments for Emotion Recognition in English Short Texts ... 28
4.1.1 English Dataset Collection ...................... 28
4.1.2 Experiment Setup with Baseline, Traditional, and State-of-the-Art Models ................................ 29
4.1.3 Experiment Results in Plutchik’s Eight Emotions ... 32
4.1.4 Experiment Results in Ekman’s Six Emotions ........ 34
4.2 Experiment for Gender-related Models.................... 36
4.2.1 Gender Dataset Collection ...................... 36
4.2.2 Experiment Setup........................... 36
4.2.3 Experiment Results.......................... 37
4.3 Experiment for MultilingualCapabilities................... 39
4.3.1 Chinese Data Collection ....................... 39
4.3.2 Experiment Setup with Chinese Baseline Model ... 41
4.3.3 Experiment Results for Emotion Recognition in Chinese Short Texts 42
5 Analysis....................................... 43
5.1 Analysis on Contextualized Pattern...................... 43
5.1.1 Enrichment by Mingling Lexicons .................. 43
5.1.2 Pattern Coverage and Consistency .................. 44
5.1.3 What’s captured by MINION? .................... 46
5.2 Analysis on Emotional Expression of Gender ..... 47
5.2.1 How does the Gender Classifier work?................ 47
5.2.2 What actually is improved by the Gender-aware Emotion Model? . . 48
5.2.3 What has the Gender-aware Emotion Model learned? ... 51
6 Conclusions and Future Work .......................... 54
References ....................................... 55

References
[1] ABDUL-MAGEED, M., AND UNGAR, L. Emonet: Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2017), vol. 1, pp. 718–728. 

[2] BECKER, K., MOREIRA, V. P., AND DOS SANTOS, A. G. Multilingual emotion classification using supervised learning: Comparative experiments. Information Pro- cessing & Management 53, 3 (2017), 684–704. 

[3] BLITZER, J., DREDZE, M., AND PEREIRA, F. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics (2007), pp. 440– 447. 

[4] BOJANOWSKI, P., GRAVE, E., JOULIN, A., AND MIKOLOV, T. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics (2017).


[5] BOUREAU, Y.-L., PONCE, J., AND LECUN, Y. A theoretical analysis of feature pooling in visual recognition. In Proceedings of the 27th international conference on machine learning (ICML-10) (2010), pp. 111–118.
[6] CHOLLET, F., ET AL. Keras. https://github.com/keras-team/keras, 2015. 

[7] CHUNG, C., AND PENNEBAKER, J. W. The psychological functions of function words. Social communication 1 (2007), 343–359. 

[8] DERIU, J., GONZENBACH, M., UZDILLI, F., LUCCHI, A., LUCA, V. D., AND JAGGI, M. Swisscheese at semeval-2016 task 4: Sentiment classification using an ensemble of convolutional neural networks with distant supervision. In Proceedings of the 10th International Workshop on Semantic Evaluation (2016), no. EPFL-CONF- 229234, pp. 1124–1128. 

[9] DERIU, J., LUCCHI, A., DE LUCA, V., SEVERYN, A., MU ̈LLER, S., CIELIEBAK, M., HOFMANN, T., AND JAGGI, M. Leveraging large amounts of weakly super- vised data for multi-language sentiment classification. In Proceedings of the 26th International Conference on World Wide Web (2017), International World Wide Web Conferences Steering Committee, pp. 1045–1052. 

[10] EKMAN, P. An argument for basic emotions. Cognition & emotion 6, 3-4 (1992), 169–200.


[11] FELBO, B., MISLOVE, A., SØGAARD, A., RAHWAN, I., AND LEHMANN, S. Us- ing millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. arXiv preprint arXiv:1708.00524 (2017).

[12] GO, A., BHAYANI, R., AND HUANG, L. Twitter sentiment classification using dis- tant supervision. CS224N Project Report, Stanford 1, 2009 (2009), 12. 

[13] GONZA ́ LEZ-IBA ́ NEZ, R., MURESAN, S., AND WACHOLDER, N. Identifying sar- casm in twitter: a closer look. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers-Volume 2 (2011), Association for Computational Linguistics, pp. 581–586. 

[14] HINTON, G. E., SRIVASTAVA, N., KRIZHEVSKY, A., SUTSKEVER, I., AND SALAKHUTDINOV, R. R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012). 

[15] IVANOV, I. Sentiment bi-rnn. https://github.com/ilivans/ attention-sentiment, 2017. 

[16] KIM, Y. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014). 

[17] KINGMA, D. P., AND BA, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). 

[18] LESKOVEC, J., RAJARAMAN, A., AND ULLMAN, J. D. Mining of massive datasets. Cambridge university press, 2014.


[19] MIKOLOV, T., CHEN, K., CORRADO, G., AND DEAN, J. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[20] MOHAMMAD, S., BRAVO-MARQUEZ, F., SALAMEH, M., AND KIRITCHENKO, S. Semeval-2018 task 1: Affect in tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation (2018), pp. 1–17. 

[21] MOHAMMAD, S. M. # emotional tweets. In Proceedings of the First Joint Confer- ence on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (2012), Association for Computational Linguis- tics, pp. 246–255. 

[22] MOHAMMAD, S. M., AND KIRITCHENKO, S. Using hashtags to capture fine emo- tion categories from tweets. Computational Intelligence 31, 2 (2015), 301–326. 

[23] NAIR, V., AND HINTON, G. E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) (2010), pp. 807–814. 

[24] PEDREGOSA, F., VAROQUAUX, G., GRAMFORT, A., MICHEL, V., THIRION, B., GRISEL, O., BLONDEL, M., ET AL. Scikit-learn: Machine learning in Python. Jour- nal of Machine Learning Research 12 (2011), 2825–2830. 

[25] PENNEBAKER, J. W., BOOTH, R. J., AND FRANCIS, M. E. Linguistic inquiry and word count: Liwc [computer software]. Austin, TX: liwc. net (2007). 

[26] PLUTCHIK,R.Thenatureofemotionshumanemotionshavedeepevolutionaryroots, a fact that may explain their complexity and provide tools for clinical practice. Amer- ican scientist 89, 4 (2001), 344–350.
[27] PORIA, S., CHATURVEDI, I., CAMBRIA, E., AND HUSSAIN, A. Convolutional mkl based multimodal emotion recognition and sentiment analysis. In Data Mining (ICDM), 2016 IEEE 16th International Conference on (2016), IEEE, pp. 439–448. 

[28] QADIR, A., AND RILOFF, E. Bootstrapped learning of emotion hashtags# hash- tags4you. In Proceedings of the 4th workshop on computational approaches to sub- jectivity, sentiment and social media analysis (2013), pp. 2–11. 

[29] READ, J. Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In Proceedings of the ACL student research workshop (2005), Association for Computational Linguistics, pp. 43–48. 

[30] ROBERTS, K., ROACH, M. A., JOHNSON, J., GUTHRIE, J., AND HARABAGIU, S. M. Empatweet: Annotating and detecting emotions on twitter. In LREC (2012), vol. 12, pp. 3806–3813. 

[31] ROSENTHAL, S., FARRA, N., AND NAKOV, P. SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (Vancouver, Canada, August 2017), SemEval ’17, Association for Com- putational Linguistics. 

[32] ROTH, D. Incidental supervision: Moving beyond supervised learning. In AAAI (2017), pp. 4885–4890.


[33] SANTOS, L. B. D., CORREˆA JR, E. A., OLIVEIRA JR, O. N., AMANCIO, D. R., MANSUR, L. L., AND ALU ́ISIO, S. M. Enriching complex networks with word embeddings for detecting mild cognitive impairment from speech transcripts. arXiv preprint arXiv:1704.08088 (2017).
[34] SAP, M., PARK, G., EICHSTAEDT, J., KERN, M., STILLWELL, D., KOSINSKI, M., UNGAR, L., AND SCHWARTZ, H. A. Developing age and gender predictive lexica over social media. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014), pp. 1146–1151. 

[35] SARAVIA, E., LIU, H.-C. T., AND CHEN, Y.-S. Deepemo: Learning and enriching pattern-based emotion representations. arXiv preprint arXiv:1804.08847 (2018). 

[36] SAVIGNY, J., AND PURWARIANTI, A. Emotion classification on youtube comments using word embedding. In Advanced Informatics, Concepts, Theory, and Applications (ICAICTA), 2017 International Conference on (2017), IEEE, pp. 1–5. 

[37] SINTSOVA, V., MUSAT, C.-C., AND PU, P. Fine-grained emotion recognition in olympic tweets based on human computation. In 4th Workshop on computational approaches to subjectivity, sentiment and social media analysis (2013), no. EPFL- CONF-197185. 

[38] STRAPPARAVA, C., VALITUTTI, A., ET AL. Wordnet affect: an affective extension of wordnet. In LREC (2004), vol. 4, pp. 1083–1086.


[39] SUTTLES, J., AND IDE, N. Distant supervision for emotion classification with dis- crete binary values. In International Conference on Intelligent Text Processing and Computational Linguistics (2013), Springer, pp. 121–136.


[40] SWAYAMDIPTA, S., THOMSON, S., DYER, C., AND SMITH, N. A. Frame-semantic parsing with softmax-margin segmental rnns and a syntactic scaffold. arXiv preprint arXiv:1706.09528 (2017). 

[41] VOLKOVA, S., AND BACHRACH, Y. Inferring perceived demographics from user emotional tone and user-environment emotional contrast. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2016), vol. 1, pp. 1567–1578. 

[42] VOLKOVA, S., WILSON, T., AND YAROWSKY, D. Exploring sentiment in social media: Bootstrapping subjectivity clues from multilingual twitter streams. In Pro- ceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (2013), vol. 2, pp. 505–510. 

[43] WANG, W., CHEN, L., THIRUNARAYAN, K., AND SHETH, A. P. Harnessing twit- ter” big data” for automatic emotion identification. In Privacy, Security, Risk and Trust (PASSAT), 2012 International Conference on and 2012 International Confer- nece on Social Computing (SocialCom) (2012), IEEE, pp. 587–592. 

[44] WARD JR, J. H. Hierarchical grouping to optimize an objective function. Journal of the American statistical association 58, 301 (1963), 236–244. 

[45] ZHANG, X., ZHAO, J., AND LECUN, Y. Character-level convolutional networks for text classification. In Advances in neural information processing systems (2015), pp. 649–657.