本論文提出一個基於生成式預訓練模型 GPT-4 (Generative pre-trained transformer- 4) 推導文法規則下名詞參數之語義分類的方法，針對給定文法規則下的搭配名 詞參數提供語義資訊的使用建議。 我們採取GPT-4和利用檢索增強生成與提示 工程所構成的管線(Pipeline)技術來建立系統。基於研究方法我們提出了一個 系統，GrammarGenie，將該方法應用於系統設定。實驗結果顯示，我們的方法 能有效幫助使用者獲得更多語義資訊。

We introduce a method for inducing semantic noun arguments of a grammar pat- tern. In our approach, we combined Retrieval-Augmented Generation and Prompt Engineering to build up a semantic categorization generation pipeline aimed at op- timizing the outcome of pre-trained large language model. The method involves starting a description containing information on grammar patterns, as well as de- signing a prompt and adopting Chain-of-Thought strategy to elicit the semantic information of noun arguments within these grammar patterns. We present a pro- totype system, GrammarGenie, that applies the method to system setting. Blind evaluation on a set of sentence examples from Oxford Learner’s Dictionaries shows that the method significantly enhances both the user experience and the quality of semantic reference. Our methodology effectively combines generative AI with linguistic information from Collins Grammar Patterns to generate semantic refer- ences.

Abstract i
摘要 ii
致謝 iii
Contents iv
List of Figures vi
List of Tables vii
1 Introduction 1
2 Related Work 5
3 Methodology 8
3.1 ProblemStatement 8
3.2 Constructing a semantic categorization generation pipeline 9
3.3 Run-TimeInteractiveSystem 12
4 Experiment 14
4.1 DataPreprocessing 15
4.2 Implementation 16
4.3 EvaluationData 19
4.4 EvaluationMetrics 19
5 Evaluation Results 26
6 Conclusion and Future Work 30
Reference 32

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Ka- plan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https://arxiv.org/abs/2005.14165.
Katrin Erk. A simple, similarity-based model for selectional preferences. In Annie Zaenen and Antal van den Bosch, editors, Proceedings of the 45th Annual Meet- ing of the Association of Computational Linguistics, pages 216–223, Prague, Czech Republic, June 2007. Association for Computational Linguistics. URL https://aclanthology.org/P07-1028.
Daniil Larionov, Artem Shelmanov, Elena Chistova, and Ivan Smirnov. Semantic role labeling with pretrained language models for known and unknown predi- cates. In Ruslan Mitkov and Galia Angelova, editors, Proceedings of the Interna- tional Conference on Recent Advances in Natural Language Processing (RANLP2019), pages 619–628, Varna, Bulgaria, September 2019. INCOMA Ltd. doi: 10. 26615/978-954-452-056-4 073. URL https://aclanthology.org/R19-1073.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Ku ̈ttler, Mike Lewis, Wen-tau Yih, Tim Rockta ̈schel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented gen- eration for knowledge-intensive nlp tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Informa- tion Processing Systems, volume 33, pages 9459–9474. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/6b493230205f780e1bc26945df7481e5-Paper.pdf.
Dekang Lin. Automatic retrieval and clustering of similar words. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 2, pages 768–774, Montreal, Quebec, Canada, August 1998. Association for Computational Linguistics. doi: 10.3115/980691.980696. URL https://aclanthology.org/P98-2127.
Eleni Metheniti, Tim Van de Cruys, and Nabil Hathout. How relevant are se- lectional preferences for transformer-based language models? In Donia Scott, Nuria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Linguistics, pages 1266–1278, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguis- tics. doi: 10.18653/v1/2020.coling-main.109. URL https://aclanthology. org/2020.coling-main.109.
Tim Van de Cruys. A neural network approach to selectional preference acquisition. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors,
Proceedings of the 2014 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 26–35, Doha, Qatar, October 2014. As- sociation for Computational Linguistics. doi: 10.3115/v1/D14-1004. URL https://aclanthology.org/D14-1004.
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph em- bedding by translating on hyperplanes. In Proceedings of the AAAI conference on artificial intelligence, volume 28, 2014.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. CoRR, abs/2201.11903, 2022. URL https://arxiv.org/ abs/2201.11903.
Wilson Wong, Wei Liu, and Mohammed Bennamoun. Ontology learning from text: A look back and into the future. ACM Comput. Surv., 44(4), sep 2012. ISSN 0360-0300. doi: 10.1145/2333112.2333115. URL https://doi.org/10. 1145/2333112.2333115.
Deniz Ekin Yavas, Laura Kallmeyer, Rainer Osswald, Elisabetta Jezek, Marta Ricchiardi, and Long Chen. Identifying semantic argument types in predica- tion and copredication contexts: A zero-shot cross-lingual approach. In Ruslan Mitkov and Galia Angelova, editors, Proceedings of the 14th International Con- ference on Recent Advances in Natural Language Processing, pages 310–320, Varna, Bulgaria, September 2023. INCOMA Ltd., Shoumen, Bulgaria. URL https://aclanthology.org/2023.ranlp-1.35.