本論文提出一個詞彙語意解歧的方法，旨在提供給定句子中目標字的詞意。我們採取微調遮罩語言模型（Masked Language Model, MLM）技術來預測目標字主題。我們的方法包含自動將同義詞詞典詞目對應到詞典詞目，自動將詞典例句轉為訓練資料，自動提取線上百科全書的句子來擴充訓練資料，和微調一個遮罩語言模型來預測主題。執行時，我們預測目標字的主題，並對候選詞意進行重新排序。我們提出了一個詞意解歧系統原型，TopSense，將此方法應用在一個同義詞典、一個學習者詞典，和線上百科全書上。對一組新聞句子的初步實驗結果顯示，我們所提出的方法大幅優於最高頻詞意基準（Most Frequent Sense Baseline）。我們的方法支援結合知識庫與語料庫，進而得到額外的改進。

We introduce a method for learning to find the intended sense of a target word in a given sentence. In our approach, the target word sense is generalized as topics aimed at improving the chance of selecting the intended sense from candidate word senses. The method involves automatically mapping a thesaurus to a dictionary, automatically transforming dictionary examples into training data, automatically expanding training data with sentences extracted from an online encyclopedia, and fine-tuning a masked language model to predict topics. At run-time, topics of the target word are predicted, and reranking is performed on the candidate senses. We present a prototype word sense disambiguation system, TopSense, that applies the method to a thesaurus, a learner's dictionary, and an online encyclopedia. Preliminary evaluation on a set of news sentences shows that the proposed method substantially outperforms the baseline of most frequent sense. Our methodology cleanly supports combining knowledge base and corpus, resulting in additional improvement.

Abstract i
摘要 ii
致謝 iii
Contents iv
List of Figures vi
List of Tables vii
1 Introduction 1
2 Related Work 5
3 Methodology 9
3.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Preparing Training Data for Predicting Topics . . . . . . . . . . . . 10
3.2.1 Mapping Thesaurus to Dictionary . . . . . . . . . . . . . . . 11
3.2.2 Converting Sentences into Training Data . . . . . . . . . . . 14
3.2.3 Augmenting Training Data . . . . . . . . . . . . . . . . . . . 14
3.2.4 Training a Masked Language Model to Predict Topics . . . . 16
3.3 Run-Time Word Sense Disambiguation . . . . . . . . . . . . . . . . 19
4 Experiment 22
4.1 Datasets and Toolkits for Training TopSense . . . . . . . . . . . . . 23
4.2 Model Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.1 Masked Language Model . . . . . . . . . . . . . . . . . . . . 25
4.2.2 Sentence Embedding Model . . . . . . . . . . . . . . . . . . 26
4.3 Models Compared . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.4 Testing Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.5 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5 Evaluation Results 30
5.1 Results of Predicted Topics . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Results from 292 Random Selection Sentence . . . . . . . . . . . . . 32
6 Conclusion and Future Work 33
Reference 35

1. Satanjeev Banerjee, Ted Pedersen, et al. Extended gloss overlaps as a measure of semantic relatedness. In Ijcai, volume 3, pages 805–810. Citeseer, 2003.
2. Edoardo Barba, Tommaso Pasini, and Roberto Navigli. Esc: Redesigning wsd with extractive sense comprehension. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4661–4672, 2021.
3. Jordan Boyd-Graber, David Blei, and Xiaojin Zhu. A topic model for word sense disambiguation. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pages 1024–1033, 2007.
4. Devendra Singh Chaplot and Ruslan Salakhutdinov. Knowledge-based word sense disambiguation using topic models. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
5. Jen Nan Chen and Jason S Chang. Topical clustering of mrd senses based on information retrieval techniques. Computational Linguistics, 24(1):61–95, 1998.
6. William A Gale, Kenneth W Church, and David Yarowsky. A method for disambiguating word senses in a large corpus. Computers and the Humanities, 26(5):415–439, 1992
7. Christian Hadiwinoto, Hwee Tou Ng, and Wee Chung Gan. Improved word sense disambiguation using pre-trained contextualized word representations. arXiv preprint arXiv:1910.00194, 2019.
8. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strength Natural Language Processing in Python. 2020. doi: 10.5281/zenodo.1212303.
9. Luyao Huang, Chi Sun, Xipeng Qiu, and Xuanjing Huang. Glossbert:
Bert for word sense disambiguation with gloss knowledge. arXiv preprint
arXiv:1908.07245, 2019.
10. Caterina Lacerra, Michele Bevilacqua, Tommaso Pasini, and Roberto Navigli. Csi: A coarse sense inventory for 85% word sense disambiguation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8123–8130, 2020.
11. Michael Lesk. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on Systems documentation, pages 24–26, 1986.
12. Dekang Lin. Automatic retrieval and clustering of similar words. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 2, pages 768–774, 1998.
13. Bernardo Magnini and Gabriela Cavaglia. Integrating subject field codes into wordnet. In LREC, pages 1413–1418, 2000.
14. Bernardo Magnini, Carlo Strapparava, Giovanni Pezzulo, and Alfio Gliozzo. Using domain information for word sense disambiguation. In Proceedings of SENSEVAL-2 Second International Workshop on Evaluating Word Sense Disambiguation Systems, pages 111–114, 2001.
15. George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995.
16. George A Miller, Martin Chodorow, Shari Landes, Claudia Leacock, and Robert G Thomas. Using a semantic concordance for sense identification. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994, 1994.
17. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
18. Bianca Scarlini, Tommaso Pasini, and Roberto Navigli. Sensembert: Context-enhanced sense embeddings for multilingual word sense disambiguation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8758–8765, 2020.
19. Partha Pratim Talukdar and Koby Crammer. New regularized algorithms for transductive learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 442–457. Springer, 2009.
20. Stijn Marinus Van Dongen. Graph clustering by flow simulation. PhD thesis, 2000.
21. Dominic Widdows and Beate Dorow. A graph model for unsupervised lexical acquisition. In COLING 2002: The 19th International Conference on Computational Linguistics, 2002.
22. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Fun-
towicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online, October 2020. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/2020.emnlp-demos.6.
23. David Yarowsky. Word-sense disambiguation using statistical models of roget’s categories trained on large corpora. In COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics, 1992.
24. David Yarowsky. Unsupervised word sense disambiguation rivaling supervised methods. In 33rd annual meeting of the association for computational linguistics, pages 189–196, 1995.
25. Dayu Yuan, Julian Richardson, Ryan Doherty, Colin Evans, and Eric Altendorf. Semi-supervised word sense disambiguation with neural models. arXiv preprint arXiv:1603.07012, 2016.
26. Zhi Zhong and Hwee Tou Ng. It makes sense: A wide-coverage word sense disambiguation system for free text. In Proceedings of the ACL 2010 system demonstrations, pages 78–83, 2010.