自動產生相關且可懂的文字是一項困難的研究，很多現存的自然語言生成系統沒有考慮到詞和詞之間的關係，且沒有理解常識知識。詞和詞或句子間的關係是超出預期的緊密結合，因此缺乏理解常識知識的語言生成系統往往會造出非預期中的段落或句子。為了改進這個問題，我們從ConceptNet裡自動抓取常識知識並結合詞嵌入及深度神經網路篇章連貫性模型到蒙地卡羅搜尋樹裡，在極大的搜尋空間及有限的時間資源裡來尋找次佳解，最後根據使用者給定的初始詞來生成具連貫性的段落。我們也觀察到以統計記數方法而建成的詞嵌入經過調整後，在相似及相關性的任務上比用預測來建成的詞嵌入模型還要精準。我們的詞嵌入在相關性任務上得到了0.679的Spearman分數，勝過了其他預訓練的詞嵌入。我們最後採用人工來評測文字連貫性，評測結果為採用篇章連貫性模型及修改擴增後的ConceptNet所生成的文字更具有連貫性。

The objective of our research was to generate a coherent, understandable text which is a challenging task. Many of current natural language generation systems that based on word appearance frequency didn't consider the relations between words and lack understanding of commonsense knowledge. The relations between words or sentences combined so closely and subtlety that are often beyond the system's expectation. Therefore, unexpected paragraphs or sentences may be generated that leads to the incoherence of the generated text. To remedy this problem, we extracted commonsense knowledge templated from ConceptNet automatically. We combined a constructed word semantic embedding model and a designed Deep Neural Network of discourse coherence model with Monte-Carlo Tree Search to find suboptimal branches in a large search space and the limited time. Our system can generate a more coherent paragraph given user's input concept. We also observed that with proper techniques, count-based word embedding can perform better than prediction-based one on similarity/relatedness tasks. We get 0.67 Spearman's score on relatedness task which outperforms other pre-trained word embeddings. We evaluated generated paragraphs by human rating, our model can generate more coherent paragraphs when using the discourse coherence model and refined ConceptNet.

摘要 i
Abstract ii
1 Introduction 1
2 Background 5
2.1 Knowledge Base . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Commonsense Knowledge Base . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 Knowledge Acquisition and Representation . . . . . . . . . . . . . . . 8
2.1.3 ConceptNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2 Concept Association . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3 Discourse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.1 Discourse Coherence Evaluation . . . . . . . . . . . . . . . . . . . . 22
2.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.1 Human-Crafted Template-Based Generation. . . . . . . . . . . . . . . . 25
2.4.2 Monte-Carlo Tree Search. . . . . . . . . . . . . . . . . . . . . . . . 26
3 Methodology 29
3.1 ConceptNet Data Cleaning . . . . . . . . . . . . . . . . . . . . . . . . 30
3.1.1 Disadvantages of ConceptNet. . . . . . . . . . . . . . . . . . . . . . 31
3.1.2 Data Cleaning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1.3 ConceptNet Expansion . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Word Embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3 Discourse Coherence Model. . . . . . . . . . . . . . . . . . . . . . . . 54
3.3.1 Deep Neural Network Architecture . . . . . . . . . . . . . . . . . . . 54
3.3.2 Training of the neural network . . . . . . . . . . . . . . . . . . . . 58
3.4 Paragraph Generation . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.1 Templates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.4.2 MCTS-based generated system. . . . . . . . . . . . . . . . . . . . . . 64
4 Experiments and Results 73
4.1 ConceptNet Data Cleaning . . . . . . . . . . . . . . . . . . . . . . . . 73
4.2 Word embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 Discourse Coherence Model. . . . . . . . . . . . . . . . . . . . . . . . 84
4.3.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.4 Paragraph Generation . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5 Conclusion 95
5.1 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.2 Future Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Bibliography 98
Appendix 108
A. ConceptNet Relations. . . . . . . . . . . . . . . . . . . . . . . . . . . 108
B. Data Cleaning of Relations in ConceptNet. . . . . . . . . . . . . . . . . 113
C. Word Embedding Experiments. . . . . . . . . . . . . . . . . . . . . . . . 124
D. Recombine words and simplify POS. . . . . . . . . . . . . . . . . . . . . 126
E. Coherence model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
F. Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

[1] H. Liu and P. Singh, “Makebelieve: using commonsense knowledge to generate stories,” in AAAI/IAAI, (USA), pp. 957–958, American Association for Artificial Intelligence, 2002.
[2] S. Yu and E. Ong, “Using common-sense knowledge in generating stories,” in PRICAI 2012: Trends in Artificial Intelligence, (Berlin, Heidelberg), pp. 838–843, Springer Berlin Heidelberg, 2012.
[3] P. Yang, F. Luo, P. Chen, L. Li, Z. Yin, X. He, and X. Sun, “Knowledgeable storyteller: A commonsense-driven generative model for visual storytelling,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 5356–5362, International Joint Conferences on Artificial Intelligence Organization, 2019.
[4] H. Zhang, Z. Liu, C. Xiong, and Z. Liu, “Grounded conversation generation as guided traverses in commonsense knowledge graphs,” in ACL 2020, Association for Computational Linguistics, 2020.
[5] R. Speer, J. Chin, and C. Havasi, “Conceptnet 5.5: An open multilingual graph of general knowledge,” in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pp. 4444–4451, AAAI Press, 2017.
[6] R. Coulom, “Efficient selectivity and backup operators in monte-carlo tree search,” in Proceedings of the 5th International Conference on Computers and Games, CG’06, (Berlin, Heidelberg), p. 72–83, Springer-Verlag, 2006.
[7] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, 2016.
[8] B. Kartal, J. Koenig, and S. J. Guy, “User-driven narrative variation in large story domains using monte carlo tree search,” in Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS ’14, (Richland, SC), p. 69–76, International Foundation for Autonomous Agents and Multiagent Systems, 2014.
[9] J. Li, W. Monroe, T. Shi, S. Jean, A. Ritter, and D. Jurafsky, “Adversarial learning for neural dialogue generation,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, (Copenhagen, Denmark), pp. 2157–2169, Association for Computational Linguistics, 2017.
[10] K. Kumagai, I. Kobayashi, D. Mochihashi, H. Asoh, T. Nakamura, , and T. Nagai, “Natural language generation using monte carlo tree search,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 22, no. 5, pp. 777–785, 2018.
[11] S. Mukherjee, “An unsupervised approach to automatic response generation for conversational e-commerce agents using monte carlo tree search,” 2019.
[12] D. B. Lenat and R. V. Guha, Building Large Knowledge-Based Systems; Representation and Inference in the Cyc Project. USA: Addison-Wesley Longman Publishing Co., Inc., 1st ed., 1989.
[13] C. Fellbaum, “Wordnet: an electronic lexical database,” Language, vol. 76, p. 706, 2000.
[14] N. Tandon, G. de Melo, F. Suchanek, and G. Weikum, “Webchild: harvesting and organizing commonsense knowledge from the web,” in WSDM, (New York, NY, USA), pp. 523–532, Association for Computing Machinery, 2014.
[15] J. Romero, S. Razniewski, K. Pal, J. Z. Pan, A. Sakhadeo, and G. Weikum, “Commonsense properties from query logs and question answering forums,” in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, (New York, NY, USA), p. 1411– 1420, Association for Computing Machinery, 2019.
[16] M. Sap, R. LeBras, E. Allaway, C. Bhagavatula, N. Lourie, H. Rashkin, B. Roof, N. A. Smith, and Y. Choi, “Atomic: An atlas of machine commonsense for if-then reasoning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3027–3035, 2019.
[17] J. Gordon and B. Durme, “Reporting bias and knowledge acquisition,” in Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, AKBC ’13, (New York, NY, USA), pp. 25–30, Association for Computing Machinery, 2013.
[18] B. Van Durme, Extracting implicit knowledge from text. PhD thesis, University of Rochester, Rochester, NY 14627, 2010.
[19] J. F. Sowa, Semantic Networks. Encyclopedia of Artificial Intelligence, Shapiro, S.C. (Ed.). New York: Wiley and Sons, 1987.
[20] P. Singh, T. Lin, E. T. Mueller, G. Lim, T. Perkins, and W. Li Zhu, “Open mind common sense: Knowledge acquisition from the general public,” in On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE, vol. 2519, (Berlin, Heidelberg), pp. 1223–1237, Springer Berlin Heidelberg, 2002.
[21] Y.-L. Kuo, J.-C. Lee, K.-Y. Chiang, R. Wang, E. Shen, C. wei Chan, and J. Y.-J. Hsu, “Community-based game design: Experiments on social games for commonsense data collection,” in Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP ’09, (New York, NY, USA), p. 15–22, Association for Computing Machinery, 2009.
[22] L. v. Ahn, “Games with a purpose,” Computer, vol. 39, no. 6, p. 92–94, 2006.
[23] R. Rada, H. Mili, E. Bicknell, and M. Blettner, “Development and application of a metric on semantic nets,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 19, pp. 17–30, 1989.
[24] Z. Wu and M. Palmer, “Verbs semantics and lexical selection,” in Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics, ACL ’94, (USA), p. 133–138, Association for Computational Linguistics, 1994.
[25] P. Resnik, “Using information content to evaluate semantic similarity in a taxonomy,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence - Volume 1, IJCAI’95, (San Francisco, CA, USA), p. 448–453, Morgan Kaufmann Publishers Inc., 1995.
[26] D. Lin, “An information-theoretic definition of similarity,” in Proceedings of the Fifteenth International Conference on Machine Learning, ICML ’98, (San Francisco, CA, USA), p. 296–304, Morgan Kaufmann Publishers Inc.,1998.
[27] G. Hirst and D. St-Onge, Lexical Chains as Representations of Context for the Detection and Correction of Malapropisms, pp. 305–332. The MIT Press,1998.
[28] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119, Curran Associates, Inc., 2013.
[29] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), (New Orleans, Louisiana), pp. 2227–2237, Association for Computational Linguistics, 2018.
[30] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019.
[31] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), (Minneapolis, Minnesota), pp. 4171–4186, Association for Computational Linguistics, 2019.
[32] O. Levy and Y. Goldberg, “Dependency-based word embeddings,” in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), (Baltimore, Maryland), pp. 302–308, Association for Computational Linguistics, 2014.
[33] W. Che, Y. Shao, T. Liu, and Y. Ding, “SemEval-2016 task 9: Chinese semantic dependency parsing,” in Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), (San Diego, California), pp. 1074–1080, Association for Computational Linguistics, 2016.
[34] M. A. K. Halliday and R. Hasan, Cohesion in English. London:Longman, 1976.
[35] R. Kibble and R. Power, “Optimizing referential coherence in text generation,” Comput. Linguist., vol. 30, no. 4, p. 401–416, 2004.
[36] R. Soricut and D. Marcu, “Discourse generation using utility-trained coherence models,” in Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, (Sydney, Australia), pp. 803–810, Association for Computational Linguistics, 2006.
[37] M. Lapata, “Probabilistic text structuring: Experiments with sentence ordering,” in Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, ACL ’03, (USA), pp. 545–552, Association for Computational Linguistics, 2003.
[38] N. Okazaki, Y. Matsuo, and M. Ishizuka, “Coherent arrangement of sentences extracted from multiple newspaper articles,” Lecture Notes in Computer Science, vol. 3157, pp. 882–891, 2004.
[39] D. Bollegala, N. Okazaki, and M. Ishizuka, “A bottom-up approach to sentence ordering for multi-document summarization,” in Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, (Sydney, Australia), pp. 385–392, Association for Computational Linguistics, 2006.
[40] E. Miltsakaki and K. Kukich, “Automated evaluation of coherence in student essays,” in In Proceedings of LREC 2000, pp. 1–8, 2000.
[41] D. Higgins, J. Burstein, D. Marcu, and C. Gentile, “Evaluating multiple aspects of coherence in student essays,” in Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, (Boston, Massachusetts, USA), pp. 185–192, Association for Computational Linguistics, 2004.
[42] J. Hobbs, “On the coherence and structure of discourse,” tech. rep., Center for the Study of language and Information, Stanford University, 1985.
[43] J. C. Lester and B. W. Porter, “Developing and empirically evaluating robust explanation generators: The knight experiments,” Comput. Linguist., vol. 23, no. 1, p. 65–101, 1997.
[44] R. M. Young, “Using grice’s maxim of quantity to select the content of plan descriptions,” Artificial Intelligence, vol. 115, pp. 215–256, 1999.
[45] A. S. Law, Y. Freer, J. Hunter, R. Logie, N. McIntosh, and J. Quinn, “A comparison of graphical and textual presentations of time series data to support medical decision making in the neonatal intensive care unit,” Journal of Clinical Monitoring and Computing, vol. 19, pp. 183–194, 2005.
[46] P. Engelhardt, K. Bailey, and F. Ferreira, “Do speakers and listeners observe the gricean maxim of quantity?,” Journal of Memory and Language, vol. 54, no. 4, pp. 554–573, 2006.
[47] M. Elsner, J. Austerweil, and E. Charniak, “A unified local and global model for discourse coherence,” in Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, (Rochester, New York), pp. 436–443, Association for Computational Linguistics, 2007.
[48] R. Barzilay and M. Lapata, “Modeling local coherence: An entity-based approach,” Computational Linguistics, vol. 34, no. 1, pp. 1–34, 2008.
[49] S. Bangalore, O. Rambow, and S. Whittaker, “Evaluation metrics for generation,” in Proceedings of the First International Conference on Natural Language Generation - Volume 14, INLG ’00, (USA), p. 1–8, Association for Computational Linguistics, 2000.
[50] I. Langkilde-Geary, “An empirical verification of coverage and correctness for a general-purpose sentence generator,” in Proceedings of the 2nd International Conference on Natural Language Generation, (Harriman, New York, USA), pp. 17–24, Association for Computational Linguistics, 2002.
[51] A. Belz and E. Reiter, “Comparing automatic and human evaluation of NLG systems,” in 11th Conference of the European Chapter of the Association for Computational Linguistics, (Trento, Italy), pp. 313–320, Association for Computational Linguistics, 2006.
[52] J. Li and E. Hovy, “A model of coherence based on distributed sentence representation,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), (Doha, Qatar), pp. 2039–2048, Association for Computational Linguistics, 2014.
[53] J. Li and D. Jurafsky, “Neural net models of open-domain discourse coherence,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, (Copenhagen, Denmark), pp. 198–209, Association for Computational Linguistics, 2017.
[54] Q. Le and T. Mikolov, “Distributed representations of sentences and documents,” in Proceedings of the 31st International Conference on Machine Learning, vol. 32 of Proceedings of Machine Learning Research, (Bejing, China), pp. 1188–1196, PMLR, 2014.
[55] R. Kiros, Y. Zhu, R. Salakhutdinov, R. S. Zemel, A. Torralba, R. Urtasun, and S. Fidler, “Skip-thought vectors,” in Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS’15, (Cambridge, MA, USA), p. 3294–3302, MIT Press, 2015.
[56] S. Arora, Y. Liang, and T. Ma, “A simple but tough-to-beat baseline for sentence embeddings,” in ICLR, 2017.
[57] Z. Lin, M. Feng, C. N. dos Santos, M. Yu, B. Xiang, B. Zhou, and Y. Bengio, “A structured self-attentive sentence embedding,” 2017.
[58] D. Cer, Y. Yang, S.-y. Kong, N. Hua, N. Limtiaco, R. St. John, N. Constant, M. Guajardo-Cespedes, S. Yuan, C. Tar, B. Strope, and R. Kurzweil, “Universal sentence encoder for English,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, (Brussels, Belgium), pp. 169–174, Association for Computational Linguistics, 2018.
[59] O. Dušek, D. M. Howcroft, and V. Rieser, “Semantic noise matters for neural natural language generation,” in Proceedings of the 12th International Conference on Natural Language Generation, (Tokyo, Japan), pp. 421–426, Association for Computational Linguistics, 2019.
[60] V. W. Soo, T.-Y. Lai, K.-J. Wu, and Y.-P. Hsu, “Generate modern style chinese poems based on common sense and evolutionary computation,” 2015 Conference on Technologies and Applications of Artificial Intelligence (TAAI), pp. 315–322, 2015.
[61] X. Feng, M. Liu, J. Liu, B. Qin, Y. Sun, and T. Liu, “Topic-to-essay generation with neural networks,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 4078–4084, International Joint Conferences on Artificial Intelligence Organization, 2018.
[62] M. Elhadad and J. Robin, “An overview of SURGE: a reusable comprehensive syntactic realization component,” in Eighth International Natural Language Generation Workshop (Posters and Demonstrations), 1996.
[63] J. Coch, “Overview of AlethGen,” in 8th International Natural Language Generation Workshop (Posters and Demonstrations), 1996.
[64] B. Lavoie and O. Rainbow, “A fast and portable realizer for text generation systems,” in Fifth Conference on Applied Natural Language Processing, (Washington, DC, USA), pp. 265–268, Association for Computational Linguistics, 1997.
[65] A. Gatt and E. Reiter, “Simplenlg: A realisation engine for practical applications,” in Proceedings of the 12th European Workshop on Natural Language Generation, ENLG ’09, (USA), p. 90–93, Association for Computational Linguistics, 2009.
[66] S. W. Mcroy, S. Channarukul, and S. S. Ali, “An augmented template-based approach to text realization,” Nat. Lang. Eng., vol. 9, no. 4, p. 381–420, 2003.
[67] N. McIntyre and M. Lapata, “Learning to tell tales: A data-driven approach to story generation,” in Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, (Suntec, Singapore), pp. 217–225, Association for Computational Linguistics, 2009.
[68] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, (Philadelphia, Pennsylvania, USA), pp. 311–318, Association for Computational Linguistics, 2002.
[69] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems, vol. 27 of NIPS’14, (Cambridge, MA, USA), pp. 2672–2680, MIT Press, 2014.
[70] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, (Cambridge, MA, USA), p. 3104–3112, MIT Press, 2014.
[71] C. Hou, C. Zhou, K. Zhou, J. Sun, and S. Xuanyuan, “A survey of deep learning applied to story generation,” in Smart Computing and Communication, (Cham), pp. 1–10, Springer International Publishing, 2019.
[72] K. Kumagai, I. Kobayashi, D. Mochihashi, H. Asoh, T. Nakamura, and T. Nagai, “Human-like natural language generation using Monte Carlo tree search,” in Proceedings of the INLG 2016 Workshop on Computational Creativity in Natural Language Generation, (Edinburgh, UK), pp. 11–18, Association for Computational Linguistics, 2016.
[73] O. F. Zaidan and C. Callison-Burch, “Crowdsourcing translation: Professional quality from non-professionals,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, (Portland, Oregon, USA), pp. 1220–1229, Association for Computational Linguistics, 2011.
[74] X. Li, A. Taheri, L. Tu, and K. Gimpel, “Commonsense knowledge base completion,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), (Berlin, Germany), pp. 1445–1455, Association for Computational Linguistics, 2016.
[75] I. Saito, K. Nishida, H. Asano, and J. Tomita, “Commonsense knowledge base completion and generation,” in Proceedings of the 22nd Conference on Computational Natural Language Learning, (Brussels, Belgium), pp. 141–150, Association for Computational Linguistics, 2018.
[76] C. Malaviya, C. Bhagavatula, A. Bosselut, and Y. Choi, “Commonsense knowledge base completion with structural and semantic context,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 03, pp. 2925–2933, 2020.
[77] Y.-M. Hsieh, M.-H. Bai, S.-L. Huang, and K.-J. Chen, “Correcting chinese spelling errors with word lattice decoding,” ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), vol. 14, no. 4, 2015.
[78] K.-J. Chen, S.-L. Huang, Y.-Y. Shih, and Y.-J. Chen, “Extended-HowNet: A representational framework for concepts,” in Proceedings of OntoLex 2005 - Ontologies and Lexical Resources, 2005.
[79] D. A. Cruse, Lexical semantics. Cambridge University Press, 1986.
[80] K. W. Church and P. Hanks, “Word association norms, mutual information, and lexicography,” Computational Linguistics, vol. 16, no. 1, pp. 22–29,1990.
[81] Y. Niwa and Y. Nitta, “Co-occurrence vectors from corpora vs. distance vectors from dictionaries,” in COLING 1994 Volume 1: The 15th International Conference on Computational Linguistics, p. 304–309, Association for Computational Linguistics, 1994.
[82] B. Daille, Approche mixte pour l’extraction de terminologie : statistique lexicale et filtres linguistiques. PhD thesis, Université Paris VII, 1994.
[83] G. Bouma, “Normalized (pointwise) mutual information in collocation extraction,” Proceedings,of the Biennial GSCL Conference 2009, pp. 31–40, 2009.
[84] O. Levy and Y. Goldberg, “Neural word embedding as implicit matrix factorization,” in Advances in Neural Information Processing Systems, vol. 27, pp. 2177–2185, Curran Associates, Inc., 2014.
[85] J. Caron, “Experiments with lsa scoring: Optimal rank and basis,” in PROC. OF SIAM COMPUTATIONAL INFORMATION RETRIEVAL WORKSHOP, (USA), p. 157–169, Society for Industrial and Applied Mathematics, 2000.
[86] L. J. Bullinaria JA, “Extracting semantic representations from word cooccurrence statistics: stop-lists, stemming, and svd,” Behavior Research Methods, vol. 44, pp. 890–907, 2012.
[87] Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, and E. Hovy, “Hierarchical attention networks for document classification,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, (San Diego, California), pp. 1480–1489, Association for Computational Linguistics, 2016.
[88] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016.
[89] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37 of ICML’15, (Lille, France), p. 448–456, JMLR.org, 2015.
[90] L. Kocsis and C. Szepesvári, “Bandit based monte-carlo planning,” in Proceedings of the 17th European Conference on Machine Learning, ECML’06, (Berlin, Heidelberg), p. 282–293, Springer-Verlag, 2006.
[91] L.-W. Ku, Y.-T. Liang, and H.-H. Chen, “Opinion extraction, summarization and tracking in news and blog corpora,” in Proceedings of AAAICAAW-06, the Spring Symposia on Computational Approaches to Analyzing Weblogs, 2006.
[92] S.-M. Wang and L.-W. Ku, “ANTUSD: A large Chinese sentiment dictionary,” in Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), (Portorož, Slovenia), pp. 2697–2702, European Language Resources Association (ELRA), 2016.
[93] L.-C. Yu, L.-H. Lee, S. Hao, J. Wang, Y. He, J. Hu, K. R. Lai, and X. Zhang, “Building Chinese affective resources in valence-arousal dimensions,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, (SanDiego, California), pp. 540–545, Association for Computational Linguistics, 2016.
[94] L. Xu, H. Lin, Y. Pan, H. Ren, and J. Chen, “Constructing the affective lexicon ontology,” Journal of the China Society for Scientific and Technical Information, vol. 27, pp. 180–185, 2008.
[95] C.-R. Huang and S.-K. Hsieh, “Infrastructure for cross-lingual knowledge representation - towards multilingualism in linguistic studies,” 2010.
[96] Y. Wu and W. Li, “Overview of the nlpcc-iccpol 2016 shared task: Chinese word similarity measurement,” in Natural Language Understanding and Intelligent Applications, (Cham), pp. 828–839, Springer International Publishing, 2016.
[97] F. Hill, R. Reichart, and A. Korhonen, “SimLex-999: Evaluating semantic models with (genuine) similarity estimation,” Computational Linguistics, vol. 41, no. 4, pp. 665–695, 2015.
[98] E. Agirre, E. Alfonseca, K. Hall, J. Kravalova, M. Paşca, and A. Soroa, “A study on similarity and relatedness using distributional and WordNet-based approaches,” in Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, (Boulder, Colorado), pp. 19–27, Association for Computational Linguistics, 2009.
[99] E. Bruni, N. K. Tran, and M. Baroni, “Multimodal distributional semantics,” J. Artif. Int. Res., vol. 49, no. 1, p. 1–47, 2014.
[100] K. Radinsky, E. Agichtein, E. Gabrilovich, and S. Markovitch, “A word at a time: Computing word relatedness using temporal semantic analysis,” in Proceedings of the 20th International Conference on World Wide Web, WWW ’11, (New York, NY, USA), p. 337–346, Association for Computing Machinery, 2011.
[101] P. Jin and Y. Wu, “SemEval-2012 task 4: Evaluating Chinese word similarity,” in *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), (Montréal, Canada), Association for Computational Linguistics, 2012.
[102] L. Finkelstein, E. Gabrilovich, Y. Matias, E. Rivlin, Z. Solan, G. Wolfman, and E. Ruppin, “Placing search in context: The concept revisited,” ACM Trans. Inf. Syst., vol. 20, no. 1, p. 116–131, 2002.
[103] C.-Y. Chen and W.-Y. Ma, “Word embedding evaluation datasets and wikipedia title embedding for chinese,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), (Miyazaki, Japan), European Language Resources Association (ELRA), 2018.
[104] Z. Yin and Y. Shen, “On the dimensionality of word embedding,” 2018.
[105] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, 2017.
[106] E. Grave, P. Bojanowski, P. Gupta, A. Joulin, and T. Mikolov, “Learning word vectors for 157 languages,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), (Miyazaki, Japan), European Language Resources Association (ELRA), 2018.
[107] Y. Wang, L. Cui, and Y. Zhang, “How can bert help lexical semantics tasks?,” 2020.
[108] S. Gelly and Y. Wang, “Exploration exploitation in go: Uct for monte-carlo go,” in NIPS: Neural Information Processing Systems Conference On-line trading of Exploration and Exploitation Workshop, (Canada), 2006.
[109] P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Mach. Learn., vol. 47, no. 2–3, p. 235–256, 2002.