自主學習（Self-Regulated Learning, SRL）在台灣教育系統中日益受到重視。自主學習強調學生在學習過程中的自我控制與調整能力，旨在培養學生的批判性思維、創新能力及終身學習技能；然而實施自主學習面臨的挑戰之一是如何有效地提供適性化的指導和學習資源，以適應每個學生的不同需求和學習進度，尤其當利用大型語言模型融入自主學習時，不同的提示策略所產出的回應品質是值得討論的議題。
本論文旨在探討不同提示策略對大型語言模型在自主學習環境中的回答品質的影響。我們通過一系列實驗，比較了無策略、搜尋增強生成策略、鏈式提示策略及鏈式提示搭配搜尋增強生成策略四種方法，對其在回答忠實度、答案相關性及反問任務中的表現進行了詳細分析。
實驗結果顯示，搜尋增強生成策略在確保生成答案的忠實度和與知識文本的一致性方面具有顯著優勢；鏈式提示策略則在反問任務具有顯著的幫助。綜合實驗結果得知應用鏈式提示搭配搜尋增強策略能顯著提升大型語言模型的回答品質，特別是在需要高忠實度和高相關性的回答任務中表現優異。未來的研究建議可進一步優化提示策略，提升模型的穩定性和普適性，以更廣泛地應用於教育領域。

Self-Regulated Learning (SRL) is increasingly valued in Taiwan's education system, emphasizing students' ability to control and adjust their learning processes to cultivate critical thinking, innovation, and lifelong learning skills. One challenge in implementing SRL is providing personalized guidance and resources to meet individual student needs, especially when using large language models. The quality of responses generated by different prompting strategies is a key issue.
This thesis explores the impact of various prompting strategies on the response quality of large language models in SRL environments. We compared four methods: no strategy, search-enhanced generation, chain-of-thought prompting, and a combination of chain-of-thought prompting with search-enhanced generation. Our experiments analyzed response fidelity, answer relevance, and reverse questioning tasks.
Results showed that the search-enhanced generation strategy ensured high fidelity and consistency with knowledge texts, while chain-of-thought prompting was beneficial for reverse questioning. Combining both strategies significantly improved response quality, especially in tasks requiring high fidelity and relevance. Future research should focus on optimizing these strategies to enhance model stability and applicability in education.

第一章 緒論-----------------------------------------------1
1.1. 研究背景---------------------------------------------1
1.2. 研究目的---------------------------------------------3
1.3. 研究限制---------------------------------------------3
第二章 文獻探討-------------------------------------------5
2.1. 自主學習---------------------------------------------5
2.2. 聊天機器人-------------------------------------------6
2.2.1. 大型語言模型的發展---------------------------------7
2.2.2. 教育環境應用聊天機器人-----------------------------8
2.3. 大型語言模型使用與提示策略---------------------------9
2.3.1. 少樣本提示策略------------------------------------10
2.3.2. 搜尋增強生成策略----------------------------------11
2.3.3. 鏈式提示策略--------------------------------------13
2.4. 大型語言模型文本分析指標----------------------------15
2.4.1. 忠實度（Faithfulness）----------------------------16
2.4.2. 答案相關性（Answer Relevance）--------------------16
第三章 研究方法------------------------------------------18
3.1. 研究流程與系統架構----------------------------------18
3.1.1. 研究流程------------------------------------------18
3.1.2. 問答系統架構--------------------------------------19
3.2. 問答系統--------------------------------------------20
3.2.1. 問答知識資料搜集----------------------------------20
3.2.2. 知識資料文本處理與儲存----------------------------21
3.3. TRELLO、TRELLO WEBHOOK 與TRELLO API-----------------22
3.4. 問答回應系統運作流程--------------------------------23
3.5. 回應模組的資料前處理--------------------------------24
3.5.1. 萃取研究主題--------------------------------------24
3.5.2. 留言的文本處理與轉換------------------------------25
3.5.3. 整備適用於搜尋增強生成的相關知識文本--------------26
3.6. 生成回答--------------------------------------------26
3.6.1. 無策略的提示詞設定--------------------------------27
3.6.2. 搜尋增強生成策略的提示詞設定----------------------27
3.6.3. 鏈式提示的提示詞設定------------------------------27
3.6.4. 鏈式提示搭配搜尋增強生成策略的提示詞設定----------28
3.7. 問答資料分析----------------------------------------29
第四章 研究結果------------------------------------------31
4.1. 回應任務文本分析------------------------------------31
4.2. 回應任務忠實度分析----------------------------------32
4.3. 回應任務答案相關性分析------------------------------34
4.4. 反問任務完成度分析----------------------------------35
4.5. 實際應用於自主學習環境的表現------------------------36
第五章 研究結論------------------------------------------38
5.1. 研究結論--------------------------------------------38
5.2. 研究建議--------------------------------------------39
第六章 參考文獻------------------------------------------40
第七章 附錄----------------------------------------------44
7.1. 提問「我想知道亨利定律公式」的回應比較--------------44
7.2. 提問「我想知道虎克定律」的回應比較------------------45
7.3. 提問「我想知道電流公式」的回應比較------------------46
7.4. 提問「我想知道萬有引力公式」的回應比較--------------47
7.5. 提問「維生素Ｃ含量最多的水果？」的回應比較----------48
7.6. 提問「橡皮筋並聯數量會影響到彈性係數嗎？」的回應比較-49

國家教育研究院. (2021). 樂詞網 : 下載專區 教科書名詞. https://terms.naer.edu.tw/download/4/
梁雲霞. (2009). 學生能夠自主學習嗎? 華人教師對自主學習觀點之探究. 小學教育國際研討會 2009 (International Conference on Primary Education 2009), 香港教育學院 (HKIEd) 主辦, 2009-11-25~ 27,
陳科名. (2022). 高中彈性學習時間之學生自主學習實施現況與反思－以屏東縣為例. 中等教育, 73(2). https://doi.org/10.6249/SE.202206_73(2).0013
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., & Anadkat, S. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., & Lu, Y. (2020). Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977.
Anthropic. (2024). Chain prompts. https://docs.anthropic.com/claude/docs/chain-prompts
Bradeško, L., & Mladenić, D. (2012). A survey of chatbot systems through a loebner prize competition. Proceedings of Slovenian language technologies society eighth conference of language technologies,
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., & Askell, A. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
Cunningham-Nelson, S., Baktashmotlagh, M., & Boles, W. (2019). Visualizing student opinion through text analysis. IEEE Transactions on Education, 62(4), 305-311.
Davis, M. R. (2014). District's Ambitious Personalized Learning Effort Shows Progress. Education week, 34(9), s13.
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Devolder, A., van Braak, J., & Tondeur, J. (2012). Supporting self‐regulated learning in computer‐based learning environments: systematic review of effects of scaffolding in the domain of science education. Journal of Computer Assisted Learning, 28(6), 557-573.
Dignath, C., & Büttner, G. (2008). Components of fostering self-regulated learning among students. A meta-analysis on intervention studies at primary and secondary school level. Metacognition and learning, 3, 231-264.
Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2023). Ragas: Automated evaluation of retrieval augmented generation. arXiv preprint arXiv:2309.15217.
Feng, D., Shaw, E., Kim, J., & Hovy, E. (2006). An intelligent discussion-bot for answering student queries in threaded discussions. International Conference on Intelligent User Interfaces,
Gautier, I., & Grave, E. (2021). Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. arXiv.org. https://doi.org/10.48550/arxiv.2007.01282
Goda, Y., Yamada, M., Matsukawa, H., Hata, K., & Yasunami, S. (2014). Conversation with a chatbot before an online EFL group discussion and the effects on critical thinking. The Journal of Information and Systems in Education, 13(1), 1-7.
Haake, M., & Gulz, A. (2009). A look at the roles of look & roles in embodied pedagogical agents–a user preference perspective. International Journal of Artificial Intelligence in Education, 19(1), 39-71.
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020). Scaling laws for neural language models. arXiv preprint arXiv:2001.08361.
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023). ChatGPT - Reshaping medical education and clinical management. Pakistan journal of medical sciences, 39(2), 605-607. https://doi.org/10.12669/pjms.39.2.7653
Kim, J., Choi, S., Amplayo, R. K., & Hwang, S.-w. (2020, December). Retrieval-Augmented Controllable Review Generation. In D. Scott, N. Bel, & C. Zong, Proceedings of the 28th International Conference on Computational Linguistics Barcelona, Spain (Online).
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28(1), 973-1018. https://doi.org/10.1007/s10639-022-11177-3
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS digital health, 2(2), e0000198-e0000198. https://doi.org/10.1371/journal.pdig.0000198
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Wen-tau, Y., Rocktäschel, T., Riedel, S., & Kiela, D. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv.org. https://doi.org/10.48550/arxiv.2005.11401
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training.
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023). In-Context Retrieval-Augmented Language Models. Transactions of the Association for Computational Linguistics, 11, 1316-1331. https://doi.org/10.1162/tacl_a_00605
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., & Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
van der Graaf, J., Lim, L., Fan, Y., Kilgour, J., Moore, J., Gašević, D., Bannert, M., & Molenaar, I. (2022). The Dynamics Between Self-Regulated Learning and Learning Outcomes: an Exploratory Approach and Implications. Metacognition and learning, 17(3), 745-771. https://doi.org/10.1007/s11409-022-09308-9
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021). Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35, 24824-24837.
Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.
Wiley, D. A., & Edwards, E. K. (2002). ONLINE SELF-ORGANIZING SOCIAL SYSTEMS The Decentralized Future of Online Learning. Quarterly review of distance education, 3(1).
Wu, K., Wu, E., & Zou, J. (2024). How faithful are RAG models? Quantifying the tug-of-war between RAG and LLMs' internal prior. arXiv preprint arXiv:2404.10198.
Zimmerman, B. J. (1986). Becoming a self-regulated learner: Which are the key subprocesses? Contemporary Educational Psychology, 11(4), 307-313. https://doi.org/https://doi.org/10.1016/0361-476X(86)90027-5