近年來大規模開放式線上課程(MOOCs) 提供學習者只要網路，沒有時空限製
的新學習方式，吸引了全球數以萬計的學習者，而課程平台也不斷加入，提供
更多樣化的課程。同時學習者也在學習平台上累積了巨量的學習歷程，平台也
開始分析這些數據，用以提升教學及學習的效率。我們提出了一個題目分析及
概念診斷系統，提供題目的難度和鑑別度的分析結果以及學生容易誤解的迷思
概念，並將指引他們應該加強學習的方向。
我們的系統是基於學生的答題紀錄進行測驗題目分析(Test Item Analysis) 而
得到試題的難度及鑑別度，並將其顯示於題目教師題庫，讓教師在挑選題目時
對整份考題的難度有所參考依據，也將難度顯示於學生練習題旁，學生可以了
解自身程度。另外我們也從學生的答題紀錄及成績，可以分析出課程的迷思概
念，最後我們將迷思概念呈現於課程知識地圖上，彌補原先課程知識上知識節
點沒有權重的弱點，可以明確的指引學生需注意的概念。
我們希望此一自動化的輔助系統可以減少教師在出題上的不便，並且增加學
生學習的效率、減短學生試後檢討自身不足找到需加強章節所需花費的時間，
演算法與系統實作架構及步驟會在下接下來的內容詳細的介紹。

With the popularities of Massive Open Online Courses, a great number of enrollments
in MOOCs generate much educational big data in terms of online activities
and logs, which might be valuable for academia and practitioners. More personalized
and intelligent online learning environment could be potentially created
through educational data mining and learning analytics techniques. Based on Item
Response Theory (IRT), the current study builds an item analysis system to identify
alternative concepts (or misconceptions) from leaners’ response in exams. By
calculating difficulty parameter and discrimination parameter from massive learners,
our systems are believed to benefit both teaching faculties and online learners.
With the affordances of the system, teaching faculties could assess leaners’ learning
performance and quality of test items while alternative conception of leaners
would be identified for strategic learning. Beside, this system is combined with
Knowledge Map, a proper way to present the result of misconceptions weekly.
Other practical and technical implications will be discussed in this paper.
Keywords -Computer-Assisted Assessment, Item Response Theory, MOOC (Massive
Open Online Course), learning analytics, misconception, alternative conception,
Item Analysis, Learning Analytics Cycle, Knowledge Map.

Abstract
中文摘要
Acknowledgement
Contents
List of Figures
List of Tables
Chapter 1 Introduction 1
Chapter 2 Related Works 5
2.1 MOOCs 5
2.1.1 Overview 5
2.1.2 cMOOCs v.s. xMOOCs 5
2.1.3 edX 6
2.1.4 Open edX 7
2.1.5 Xuetangx 9
2.1.6 UOOC 11
2.1.7 ShareCourse 14
2.2 Learning Analysis 14
2.2.1 Testing Analysis on MOOCs 15
Chapter 3 System Architecture 17
3.1 MOOCs platform 18
3.1.1 Knowledge Map 19
3.2 Datacenter 20
3.2.1 Activity Record Schema 21
3.2.2 API Server 22
3.2.3 Analyzer 23
3.2.4 Database 23
3.2.5 Misconception Diagnosis Module 23
Chapter 4 System Implementation 25
4.1 Data Service Server Environment 25
4.1.1 Node.js 25
4.1.2 MongoDB 26
4.1.3 Google Cloud Platform 26
4.1.4 Ruby 26
4.2 Server Setup 26
4.3 Algorithm 27
4.3.1 Test Item Analysis 27
4.3.2 Alternative conception 29
4.4 Data Service API 31
4.4.1 API’s Collects User Activity Records 31
4.4.2 API’s Usage of Analysis 33
4.5 User Interface 34
4.5.1 UI techniques 34
4.5.1.1 D3-cloud.js 34
4.5.1.2 Bootstrap 34
4.5.1.3 jQuery 34
4.5.2 System Viewing 35
4.5.2.1 Chart Report 35
4.5.2.2 Item pool with Test Item Index 36
4.5.2.3 Exercise and Exam with Test Item Index 36
4.5.2.4 Misconception on Knowledge Map 38
Chapter 5 Experiment 40
5.1 Records 40
5.2 Course status 40
5.3 Index Improvement 41
5.4 learning intervention 43
Chapter 6 Conclusion and Future Work 45
Bibliography 47

[1] Misconceptions Mining and Visualizations for Chinese-based MOOCs Forum Based on NLP, International Conference on Bid Data Analysis. IEEE, March 2017.
[2] VideoMark: A Video-Based Learning Analytic Technique for MOOCs, International Conference on Bid Data Analysis. IEEE, March 2017.
[3] L. Breslow, D. E. Pritchard, J. DeBoer, G. S. Stump, A. D. Ho, and D. T.Seaton, “Studying learning in the worldwide classroom: Research into edx’sfirst mooc,” Research & Practice in Assessment, vol. 8, 2013.
[4] D. T. Seaton, Y. Bergner, I. Chuang, P. Mitros, and D. E. Pritchard, “Who does what in a massive open online course?” Communications of the ACM, vol. 57, no. 4, pp. 58–65, 2014.
[5] R. Ferguson and D. Clow, “Where is the evidence? a call to action for learning analytics,” 2017.
[6] K. E. Arnold and M. D. Pistilli, “Course signals at purdue: Using learning analytics to increase student success,” in Proceedings of the 2nd international conference on learning analytics and knowledge. ACM, 2012, pp. 267–270.
[7] D. Clow, “The learning analytics cycle: closing the loop effectively,” in Proceedings of the 2nd international conference on learning analytics and knowledge. ACM, 2012, pp. 134–138.
[8] S. M. North, R. Richardson, and M. M. North, “To adapt moocs, or not? that is no longer the question,” Universal Journal of Educational Research, vol. 2, no. 1, pp. 69–72, 2014.
[9] Y. Chengjie, “Challenges and changes of mooc to traditional classroom teaching mode,” Canadian Social Science, vol. 11, no. 1, p. 135, 2015.
[10] F. Amadieu, T. Van Gog, F. Paas, A. Tricot, and C. Mariné, “Effects of prior knowledge and concept-map structure on disorientation, cognitive load, and learning,” Learning and Instruction, vol. 19, no. 5, pp. 376–386, 2009.
[11] J. H. Lee and A. Segev, “Knowledge maps for e-learning,” Computers & Education, vol. 59, no. 2, pp. 353–364, 2012.
[12] C. D. Holley and D. F. Dansereau, Spatial learning strategies: Techniques, applications, and related issues. Academic Press, 2014.
[13] R. H. Hall, D. F. Dansereau, and L. P. Skaggs, “Knowledge maps and the presentation of related information domains,” The Journal of Experimental Education, vol. 61, no. 1, pp. 5–18, 1992.
[14] J. C. Nesbit and O. O. Adesope, “Learning with concept and knowledge maps: A meta-analysis,” Review of educational research, vol. 76, no. 3, pp. 413–448, 2006.
[15] L. Pappano, “The year of the mooc,” The New York Times, vol. 2, no. 12, p. 2012, 2012.
[16] M. I. of Technology and H. University, “edx,” Retrieved July 17, 2017, from the World Wide Web: https://www.edx.org/, 2012.
[17] ——, “openedx,” Retrieved June 15, 2017, from the World Wide Web: https://open.edx.org, 2012.
[18] J. S. Ruiz, H. J. P. Díaz, J. A. Ruipérez-Valiente, P. J. Muñoz-Merino, and C. D. Kloos, “Towards the development of a learning analytics extension in open edx,” in Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality. ACM, 2014, pp. 299–306.
[13] R. H. Hall, D. F. Dansereau, and L. P. Skaggs, “Knowledge maps and the presentation of related information domains,” The Journal of Experimental Education, vol. 61, no. 1, pp. 5–18, 1992.
[14] J. C. Nesbit and O. O. Adesope, “Learning with concept and knowledge maps: A meta-analysis,” Review of educational research, vol. 76, no. 3, pp. 413–448, 2006.
[15] L. Pappano, “The year of the mooc,” The New York Times, vol. 2, no. 12, p. 2012, 2012.
[16] M. I. of Technology and H. University, “edx,” Retrieved July 17, 2017, from the World Wide Web: https://www.edx.org/, 2012.
[17] ——, “openedx,” Retrieved June 15, 2017, from the World Wide Web: https://open.edx.org, 2012.
[18] J. S. Ruiz, H. J. P. Díaz, J. A. Ruipérez-Valiente, P. J. Muñoz-Merino, and C. D. Kloos, “Towards the development of a learning analytics extension in open edx,” in Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality. ACM, 2014, pp. 299–306.
[26] K. Kitto, S. Cross, Z. Waters, and M. Lupton, “Learning analytics beyond the lms: the connected learning analytics toolkit,” in Proceedings of the Fifth International Conference on Learning Analytics And Knowledge. ACM, 2015, pp. 11–15.
[27] J. Kean and J. Reilly, “Item response theory,” Handbook for Clinical Research: Design, Statistics and Implementation.(pp195-198). New York, NY: Demos Medical Publishing, 2014.
[28] P. Fotaris, T. Mastoras, I. Mavridis, and A. Manitsaris, “Extending lms to support irt-based assessment test calibration,” Technology Enhanced Learning. Quality of Teaching and Educational Reform, pp. 534–543, 2010.
[29] P. Fotaris and T. Mastoras, “Integrating irt analysis into lms for item pool optimization,” 2013.
[30] T. W. Kiat, “Test-item analysis,” 1981.
[31] T.-W. Sheu, J.-W. Tzeng, C.-P. Tsai, and T.-L. Chen, “Applying problemconcept chart combined with structural analysis to investigate the learning misconcept-simple equation with one variable for example,” Journal of Grey System, vol. 15, no. 1, pp. 55–66, 2012.
[32] R. Dahl, “Node.js,” Retrieved July 17, 2017, from the World Wide Web: https://nodejs.org/en/, 2009.
[33] M. Inc., “mongodb,” Retrieved July 17, 2017, from the World Wide Web: https://www.mongodb.com/, 2009.
[34] G. Inc., “Google cloud platform,” Retrieved July 17, 2017, from the World Wide Web: https://cloud.google.com/, 2011.
[35] Y. Matsumoto, “ruby,” Retrieved July 17, 2017, from the World Wide Web: https://www.ruby-lang.org/en/, 1995.
[36] J.-W. Tzeng, “The applications of the misconceptions’ order analysis method for concept diagnosis,” Ph.D. dissertation, National Taichung University of Education, 2013.
[37] D. Julong, “Introduction to grey system theory,” The Journal of grey system, vol. 1, no. 1, pp. 1–24, 1989.
[38] “d3-cloud.js,” Retrieved July 17, 2017, from the World Wide Web: https://github.com/jasondavies/d3-cloud.
[39] J. T. Mark Otto, “Bootstrap,” Retrieved July 17, 2017, from the World Wide Web: http://getbootstrap.com/.
[40] J. Resig, “jquery,” Retrieved July 17, 2017, from the World Wide Web: https://jquery.com/, 2006.
[41] M. Tavakol and R. Dennick, “Making sense of cronbach’s alpha,” International journal of medical education, vol. 2, p. 53, 2011.
[42]IBM, “Ibm spss software,” Retrieved July 17, 2017, from the World Wide Web: https://www.ibm.com/analytics/us/en/technology/spss/./, 2017.