本論文提出一個英文寫作多面向評分的方法，自動提供文章不同面向之 CEFR 等級，以利使用者更有效的修改文章並提高寫作等級。此研究從句法層面切 入，針對文法句構及字彙選擇進行評分。我們採用深度學習 (Deep Learning) 之 研究路線，先將個別句子轉換成能更充分反映各面向之型態，再訓練多個分類 器預測句子在不同面向的寫作等級。我們將上述方法應用於註釋語料庫及學習 者的文章，提出一個雛形寫作評分系統 WritingProfile，以協助使用者提昇寫作 能力。根據實驗結果顯示，我們在此研究中提出的方法能得到與基準 (baseline) 相比更佳的結果，為英語學習者提供了更全面的寫作反饋系統。

We introduce a method for learning to predict the holistic score as well as a num- ber of specific trait scores of a given essay. In our approach, essays are transformed into new sequences aimed at highlighting the characteristics of the various target writing traits. The method involves automatically learning the textual features of different writing traits from a training set of learner sentences, and automatically training a classifier to generate sentence-level scores. At run-time, essays are bro- ken down into sentences, and the predicted sentence level scores are aggregated to calculate the essay level scores. We present a prototype analytic automated essay scoring system, WritingProfile, that applies the method to a learner corpus. Blind evaluation on a set of real learner essays shows that the method signifi- cantly outperforms a baseline model trained on the original untransformed text, and outperforms several existing AES models. Our methodology clearly supports combining scores based on different writing traits, resulting in a more comprehen- sive writing feedback system for English language learners.

Abstract i
摘要 ii
致謝 iii
Contents iv
List of Figures vi
List of Tables vii
1 Introduction 1
2 Related Work 5
3 Methodology 9
3.1 Problem Statement 9
3.2 Learning to Model Writing Traits in a Sentence 10
3.2.1 Transforming Sentences for the Vocabulary Model 11
3.2.2 Transforming Sentences for the Grammar Model 12
3.2.3 Generating Sentence Scores 14
3.3 Run-time Analytic Scoring 14
4 Experimental Setting 17
4.1 Datasets and Toolkits 18
4.2 Training WritingProfile 21
4.3 Evaluation Metrics 22
4.4 Evaluation Sentences and Essays 23
5 Results and Discussion 26
5.1 Results from the Sentence-Level Evaluation 26
5.2 Results from the Essay-Level Evaluation 27
6 Conclusion and Future Work 29
References 31

Cristina Arhiliuc, Jelena Mitrović, and Michael Granitzer. Language proficiency scoring. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5624–5630, Marseille, France, 2020. European Language Resources Association.
Yigal Attali and Jill Burstein. Automated essay scoring with e-rater® v.2. Journal of Technology, Learning, and Assessment, 4(3):1–30, 2006.
Jill Burstein. The e-rater® scoring engine: Automated essay scoring with natural language processing. In M. D. Shermis J. Burstein (Eds.), Automated essay scoring: A cross-disciplinary perspective, pages 113–121. Lawrence Erlbaum Associates Publishers, 2003.
Andrew Caines and Paula Buttery. REPROLANG 2020: Automatic proficiency scoring of Czech, English, German, Italian, and Spanish learner essays. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5614–5623, Marseille, France, 2020. European Language Resources Association.
Mădălina Cozma, Andrei Butnaru, and Radu Tudor Ionescu. Automated essay scoring with string kernels and word embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 503–509, Melbourne, Australia, 2018. Association for Computational Linguistics.
Fei Dong, Yue Zhang, and Jie Yang. Attention-based recurrent convolutional neural network for automatic essay scoring. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 153–162, Vancouver, Canada, 2017. Association for Computational Linguistics.
Yasutake Ishii and Yukio Tono. Investigating japanese efl learners’ overuse/underuse of english grammar categories and their relevance to cefr levels. In Proceedings of Asia Pacific Corpus Linguistics Conference, pages 160–165, 2018.
Sungho Jeon and Michael Strube. Countering the influence of essay length in neural essay scoring. In Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing, pages 32–38, Virtual, 2021. Association for Computational Linguistics.
Sandeep Mathias and Pushpak Bhattacharyya. Can neural networks automatically score essay traits? In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 85–91, Seattle, WA, USA → Online, 2020. Association for Computational Linguistics.
Elijah Mayfield and Alan W Black. Should you fine-tune BERT for automated essay scoring? In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 151–162, Seattle, WA, USA → Online, 2020. Association for Computational Linguistics.
Ellis B Page. The imminence of... grading essays by computer. In The Phi Delta Kappan, 47(5), pages 238–243, 1966.
Isaac Persing and Vincent Ng. Modeling argument strength in student essays. In
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 543–552, Beijing, China, 2015. Association for Computational Linguistics.
Robert Ridley, Liang He, Xinyu Dai, Shujian Huang, and Jiajun Chen. Automated cross-prompt scoring of essay traits. In AAAI Conference on Artificial Intelligence, pages 13745–13753, 2021.
Lawrence M. Rudner, Veronica Garcia, and Catherine Welch. An evaluation of intellimetric TM essay scoring system using responses to gmat ® awa prompts. In GMAC Research report number, RR-05-08, 2006.
Veronica Juliana Schmalz and Alessio Brutti. Automatic assessment of english cefr levels using bert embeddings. In Proceedings of the Eighth Italian Conference on Computational Linguistics, 2021.
Takumi Shibata and Masaki Uto. Analytic automated essay scoring based on deep neural networks integrating multidimensional item response theory. In Proceedings of the 29th International Conference on Computational Linguistics, pages 2917–2926, Gyeongju, Republic of Korea, 2022. International Committee on Computational Linguistics.
Kaveh Taghipour and Hwee Tou Ng. A neural approach to automated essay scoring. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1882–1891, Austin, Texas, 2016. Association for Computational Linguistics.
Satoru Uchida and Masashi Negishi. Assigning cefr-j levels to english texts based on textual features. In Proceedings of the 4th Asia Pacific Corpus Linguistics Conference, pages 463–467, 2018.
Sowmya Vajjala and Taraka Rama. Experiments with universal CEFR classification. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 147–153, New Orleans, Louisiana, 2018. Association for Computational Linguistics.
Yongjie Wang, Chuang Wang, Ruobing Li, and Hui Lin. On the use of bert for automated essay scoring: Joint learning of multi-scale essay representation. In Proceedings of the 2022 Conference of the North American Chapter of the As- sociation for Computational Linguistics: Human Language Technologies, pages 3416–3425, Seattle, United States, 2022. Association for Computational Linguistics.
Patti West-Smith, Stephanie Butler, and Elijah Mayfield. Trustworthy automated essay scoring without explicit construct validity. In AAAI Spring Symposia, 2018.
Bronwyn Woods, David Adamson, Shayne Miel, and Elijah Mayfield. Formative essay feedback using predictive scoring models. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 2071–2080, 2017.
Jin Xue, Xiaoyi Tang, and Liyan Zheng. A hierarchical bert-based transfer learning approach for multi-dimensional essay scoring. IEEE Access, 9:125403–125415, 2021.