本論文提出一個閱讀測驗自動出題的方法，依據使用者輸入的文章，產生單字 題型的試題。 我們依據使用者指定的試題難易度，找出考點，並藉由語言模 型(Language Model, LM)和篩選機制所構成的管線(Pipeline)來產生正確答 案(Answer Key)和誘答項(Distractor)。 此方法涉及利用語言模型改寫部分文章 中的字彙，用以獲得候選詞，及利用篩選機制過濾不恰當的候選詞。 實驗結果 顯示，我們的方法產生的試題相當接近人工出題的結果。

We introduce a method for generating vocabulary questions on reading comprehension of a given English article. In our approach, the method involves selecting target words in the given English article, finding synonyms as answer keys, and generating seemingly reasonable words in context as distractors. At run-time, some target words in the inputted article will be identified as questions, and automatically generating one answer key and three distractors. We present a AQG (automatic question generation) system, JEFF, that applies the method to generate test items automatically. Evaluation on a set of test items generated by JEFF shows that the method is close to the human-designed ones.

Abstract i
摘要 ii
致謝 iii
Contents iv
List of Figures vi
List of Tables vii
1 Introduction 1
2 Related Work 5
3 Methodology 8
3.1 ProblemStatement 8
3.2 Generating Components of Test Items 9
3.2.1 Selecting Target Words 9
3.2.2 Generating Answer Keys 11
3.2.3 Generating Distractors 14
4 Experiment 17
4.1 Experimental Setting 17
4.2 Test Items Compared 19
4.3 Evaluation Metric 19
4.4 Evaluation Settings and Their Quality Judgments 20
5 Evaluation Results 22
5.1 Results of Average Scores 22
5.2 Results for the Distribution of the Score 23
6 Conclusion and Future Work 25
Reference 27
Appendices 30

Itziar Aldabe and Montse Maritxalar. Automatic distractor generation for domain specific texts. In International Conference on Natural Language Processing, pages 27–38. Springer, 2010.
Jonathan Brown, Gwen Frishkoff, and Maxine Eskenazi. Automatic question generation for vocabulary assessment. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 819–826, 2005.
Dhawaleswar Rao Ch and Sujan Kumar Saha. Automatic multiple choice question generation from text: A survey. IEEE Transactions on Learning Technologies, 13(1):14–25, 2018.
David Coniam. A preliminary inquiry into using corpus word frequency data in the automatic generation of english language cloze tests. Calico Journal, pages 15–33, 1997.
Takuya Goto, Tomoko Kojiri, Toyohide Watanabe, Tomoharu Iwata, and Takeshi Yamada. Automatic generation system of multiple-choice cloze questions and its evaluation. Knowledge Management & E-Learning: An International Journal, 2(3):210–224, 2010.
Shu Jiang and John SY Lee. Distractor generation for chinese fill-in-the-blank items. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pages 143–148, 2017.
Ghader Kurdi, Jared Leo, Bijan Parsia, Uli Sattler, and Salam Al-Emari. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30(1):121–204, 2020.
Chao-Lin Liu, Chun-Hung Wang, Zhao Ming Gao, and Shang-Ming Huang. Ap- plications of lexical information for algorithmically composing multiple-choice cloze items. In Proceedings of the second workshop on Building Educational Applications Using NLP, pages 1–8, 2005.
Keisuke Sakaguchi, Yuki Arase, and Mamoru Komachi. Discriminative approach to fill-in-the-blank quiz generation for language learners. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 238–242, 2013.
Eiichiro Sumita, Fumiaki Sugaya, and Seiichi Yamamoto. Measuring non-native speakers’ proficiency of english by using a test with automatically-generated fill-in-the-blank questions. In Proceedings of the second workshop on Building Educational Applications Using NLP, pages 61–68, 2005.
Yuni Susanti, Ryu Iida, and Takenobu Tokunaga. Automatic generation of english vocabulary tests. In CSEDU (1), pages 77–87, 2015.
Ratsameetip Wita, Sahussarin Oly, Sununta Choomok, Thanabhorn Treeratsakulchai, and Surarat Wita. A semantic graph-based japanese vocabulary learning game. In International Conference on Web-Based Learning, pages 140– 145. Springer, 2018.