帳號:guest(18.191.222.143)          離開系統
字體大小: 字級放大   字級縮小   預設字形  

詳目顯示

以作者查詢圖書館館藏以作者查詢臺灣博碩士論文系統以作者查詢全國書目
作者(中文):楊尚儒
作者(外文):Yang, Shan-Ru
論文名稱(中文):探討多媒體指令方式對於認知負荷和任務表現的影響
論文名稱(外文):Effects of Multimedia Instruction Methods on Mental Workload and Task Performance
指導教授(中文):李昀儒
指導教授(外文):Lee, Yun-Ju
口試委員(中文):盧俊銘
黃瀅瑛
口試委員(外文):Lu, Jun-Ming
Huang, Ying-Yin
學位類別:碩士
校院名稱:國立清華大學
系所名稱:工業工程與工程管理學系
學號:106034403
出版年(民國):108
畢業學年度:107
語文別:英文
論文頁數:51
中文關鍵詞:多媒體學習心智負荷語言眼動認知負荷指標
外文關鍵詞:multimedia learninglanguagemental workloadeye movementIndex of Cognitive Activity
相關次數:
  • 推薦推薦:0
  • 點閱點閱:52
  • 評分評分:*****
  • 下載下載:0
  • 收藏收藏:0
多媒體學習傳統上使用不同類型的資訊表現形式(如文字、圖等),來促進理解以及記憶相 關內容。雖然目前在科技的發展下有越來越多新穎的指導模式被發展出來,但是還未甚完備並讓學習者有更好的學習體驗以及效率。在學習時,語言是一個至關重要的因素。根據所學習的為母語或外語,很可能會影響最有效的多媒體學習方式,而目前多媒體學習仍然存在爭議:影片究竟是搭配旁白還是字幕會更佳。解決爭論所需的關鍵因素為評估使用各個類型時的心智負荷。心智負荷指的是在執行任務時有多少的認知資源被大腦所調用。為了量化認知資源,我們可以使用主觀和客觀工具,主觀測量為使用問卷,例如 NASA-TLX 等;客觀測量為使用物理參數進行測量(例如:心率或眨眼率)。

本研究旨在評估哪些模式在使用多媒體指令時將會更有效的學習,以及當受試者用外語學習時,這個結果是否會有所不同。參與者將以 4 種不同的教學方法:傳統的紙上文本、有旁白沒字幕的視頻、沒旁白和字幕的視頻以及帶有旁白和字幕的視頻來執行任務。受試者將分為兩組,一組參與者以母語學習(中文),第二組將以外語學習(英文)。透過讓參與者實際學習指定的食譜並以此評估參與者的學習表現。受試者將佩戴眼動儀,其眼動資訊如眼睛注視時間及使用認知活動指標 (Index of Cognitive Activity, ICA) 分析心智負荷。此外受試者將填寫有兩份問卷:NASA-TLX 和偏好問卷,以便了解受試者學習指定食譜時的心智負荷以及他們傾向於使用哪種方法。

總共從國立清華大學招募了30位受測者 年紀為20-27歲 (平均= 20.53, 標準差= 1.76, 18位男性, 12位女性)。雙因子重複測量ANOVA顯示語言的顯著主效應(F(8,18)= 4.802,p = 0.03),並且沒有交互作用。 PERCLOS(F(1,25)= 13.943,p = 0.001),左眼平均凝視時間(MFDL)(F(1,25)= 7.676,p = 0.01)和NASA-TLX(F(1,25)= 8.396 ,p = 0.008)顯示兩個語言組之間有顯著差異。在雙因子重複測量ANOVA的受試者內測試中,ICA(F(2.3,57.511)= 4.522,p = 0.012)在不同的學習方式下有了主效應但沒有交互作用。學習方式偏好問卷顯示,大多數參與者更喜歡的學習方式為有字幕和聲音的影片。隨後的配對t檢驗顯示,受測者的學習偏好對於表現有了顯著差異成為正相關(t = 4.25,p <0.001)。

研究結果顯示,在學習同樣的食譜時用第二語言學習相對於用第一語言,所需要的心智負荷會更高。其次,對於學習和製作烘焙食譜較低難度任務,在學習者自訂節奏的環境中,會消除認知負荷理論(Cognitive Load Theory) 和多媒體學習認知理論(Cognitive Theory of Multimedia Learning) 中的所提到的效果,尤其是多媒體模式的部分。第三,(PERCLOS)仍然處於成為心智負荷參數的早期階段,並且在描繪不同的認知狀態時仍然顯示出不確定性。
Multimedia Learning is a form of learning supported by different sources of information (e.g. texts and graphics) being handled jointly in order to understand and memorise a given content. Even though new instructing methods have been introduced, they have yet to be further developed and optimised so that they are able to achieve a better learning experience and performance for those learning. Language is a crucial factor when it comes to learning. Depending on whether you are learning in your first or second language, it will most likely affect the type of multimedia learning method that will be most effective or affect the conditions you prefer to be taught in. Taking Multimedia learning for example, there are still arguments to whether it is better to have narration within the video, or subtitles. A parameter that is needed to settle the argument is that of mental workload. Mental workload is a hypothetical construct that describes the extent to which the cognitive resources required to perform a task have been actively engaged by the operator. In order to measure it, both subjective and objective tools can be used. With subjective measurements being mainly questionnaires such as NASA-TLX and objective measurements using physical measurements such as heart rate, or eye parameters.
This thesis aimed to evaluate the effects of the modality principle and determine which modalities will have a more effective learning when using multimedia instructions. In addition to whether or not this result will differ when learning in your second language. The modality principle states that people learn better from graphics and narration than from graphics and printed text. Participants performed tasks in 4 different instructional methods, traditional text on paper, video with narration, video with subtitles, and video with narration and subtitles. The participants were divided into two groups, one where participants learned in their first language (mandarin Chinese) and the second group learned in their second language (English). We assessed the participants’ performance by making them physically enact the learned recipe. Participants also wore an eye tracker so we could analyse their eye movements such as PERCLOS (Percentage of Eyelid Closure) or eye fixation duration, and ICA (Index of Cognitive Activity), to determine their mental workload experienced during their learning phases. In addition, we had two questionnaires, NASA-TLX, and a preference questionnaire for them to answer how mentally challenging it was to learn the given recipe and which method they preferred overall.
A total of 30 participants aged 20-27 (mean= 23.36, SD= 1.76, 18 males, 12 females) were recruited from National Tsing Hua University in Hsinchu, Taiwan for the experiment. The two-way repeated measures ANOVA showed a significant main effect of language (F (8,18) =4.802, p=0.03) and no interaction. PERCLOS (F (1, 25) = 13.943, p= 0.001), mean fixation duration left (MFDL) (F (1, 25) = 7.676, p= 0.01) and NASA-TLX (F (1, 25) = 8.396, p= 0.008) showed significant differences between the two language groups. In the within-subjects test of the two-way repeated measures ANOVA, there was a main effect of ICA (F (2.3, 57.511) = 4.522, p= 0.012) on condition and no interaction. The preferred modality questionnaire showed that most participants preferred to learn the recipes in the video with subtitles and narration condition. A subsequent paired t-test showed that preferred modality scored a significantly higher score than the non-preferred modalities (t=4.25, p<0.001).

In conclusion when learning in your second language, the mental workload you experience will be higher in comparison to learning the same thing in your first language. Secondly, for low intrinsic tasks such as learning and making baking recipes, in a learner-paced environment, will mitigate and reverse effects mentioned in CLT and CTML especially the modality effect. Thirdly, PERCLOS is still in early stages of becoming a parameter for mental workload and still shows ambiguity when portraying different cognitive states. Finally, our study reciprocated the findings of previous research upon the correlation between preferences in learning and performance.
ABSTRACT II
摘要 IV
CONTENTS VI
TABLE CONTENTS VIII
FIGURE CONTENTS IX
CHAPTER 1 INTRODUCTION 1
1.1 BACKGROUND 1
1.2 MOTIVATION AND PURPOSE 3
CHAPTER 2 LITERATURE REVIEW 5
2.1 MULTIMEDIA LEARNING 5
2.1.1 Cognitive Load Theory 5
2.1.2 Cognitive Theory of Multimedia Learning 7
2.1.3 Modality effect 9
2.1.4 Instructing approaches 10
2.1.5 Summary of Multimedia Learning 12
2.2 LANGUAGE 13
2.2.1 Learning Languages 13
2.2.2 Subtitles 15
2.2.3 Summary of Language 16
2.3 MENTAL WORKLOAD 17
2.3.1 Subjective Measurements 17
2.3.2 Objective Measurements 19
2.3.3 Eye movement 20
2.3.4 Summary of Mental Workload 22
CHAPTER 3 METHOD 23
3.1 PILOT STUDY 23
3.1.1 Aim and Purpose 23
3.1.2 Materials and participants 24
3.1.3 Procedure 24
3.1.4 Statistical Analysis 25
3.1.5 Results 25
3.1.6 Summary of the pilot study 26
3.2 PARTICIPANTS 27
3.3 MATERIALS AND EQUIPMENT 27
3.4 EXPERIMENTAL DESIGN 29
3.5 DATA ANALYSIS 30
3.6 STATISTICS 31
CHAPTER 4 RESULTS 32
4.1 PARTICIPANTS 32
4.2 LANGUAGE 32
4.3 MODALITY 33
4.4 PREFERRED MODALITY 34
CHAPTER 5 DISCUSSION 36
5.1 LANGUAGE 36
5.2 MODALITY 38
5.3 PREFERRED MODALITY 40
5.4 RESEARCH LIMITATIONS 41
CHAPTER 6 CONCLUSION 43
6.1 RESEARCH CONTRIBUTIONS 43
6.2 FUTURE RESEARCH 43
REFERENCES 44

1. Akbari Chermahini, S., Ghanbari, A., &Ghanbari Talab, M. (2013). LEARNING STYLES AND ACADEMIC PERFORMANCE OF STUDENTS IN ENGLISH AS A SECOND-LANGUAGE CLASS IN IRAN. Bulgarian Journal of Science and Education Policy, 7(2), 322–333.
2. Alqahtani, N. D., Al-Jewair, T., Al-Moammar, K., Albarakati, S. F., &Alkofide, E. A. (2015). Live demonstration versus procedural video: A comparison of two methods for teaching an orthodontic laboratory procedure Approaches to teaching and learning. BMC Medical Education, 15(1), 1–4. https://doi.org/10.1186/s12909-015-0479-y
3. An, L., Wang, Y., &Sun, Y. (2017). Reading words or pictures: Eye movement patterns in adults and children differ by age group and receptive language ability. Frontiers in Psychology, 8(MAY), 1–8. https://doi.org/10.3389/fpsyg.2017.00791
4. Athanasopoulos, P. (2007). Do bilinguals think differently from monolinguals ? Evidence from non-linguistic cognitive categorisation of objects in Japanese-English. Selected Papers on Theoretical and Applied Linguistics (17th International Symposium on Theoretical and Applied Linguistics, Thessaloniki 14-17 April 2005), 17(2), 338–345.
5. Atkinson, R. C., &Shiffrin, R. M. (1971). THE CONTROL PROCESSES OF SHORT-TERM MEMORY. Builder. https://doi.org/10.1038/scientificamerican0871-82
6. Birulés-Muntané, J., &Soto-Faraco, S. (2016). Watching subtitled films can help learning foreign languages. PLoS ONE, 11(6), 1–10. https://doi.org/10.1371/journal.pone.0158409
7. Boeker, M., Andel, P., Vach, W., &Frankenschmidt, A. (2013). Game-based e-learning is more effective than a conventional instructional method: A randomized controlled trial with third-year medical students. PLoS ONE, 8(12), 1–11. https://doi.org/10.1371/journal.pone.0082328
8. Burt, M. (2004). Issues with outcomes in workplace ESL programs. U.S. Department of Education, Office of Adult and Vocational Education & The Institute for Work and the Economy, 1–13.
9. Casasanto, D., Boroditsky, L., Phillips, W., Greene, J., Goswami, S., Bocanegra-Thiel, S., …Gil, D. (2005). How deep are effects of language on thought? Time estimation in speakers of English, Indonesian, Greek, and Spanish. Proceedings of the Twenty-Sixth Annual Conference of the Cognitive Science Society, 186–191.
10. Charles, R. L., &Nixon, J. (2019). Measuring mental workload using physiological measures: A systematic review. Applied Ergonomics, 74(September 2016), 221–232. https://doi.org/10.1016/j.apergo.2018.08.028
11. Eckstein, M. K., Guerra-Carrillo, B., Miller Singley, A. T., &Bunge, S. A. (2017). Beyond eye gaze: What else can eyetracking reveal about cognition and cognitive development? Developmental Cognitive Neuroscience, 25, 69–91. https://doi.org/10.1016/j.dcn.2016.11.001
12. Ekkens, K., &Winke, P. (2009). Evaluating Workplace English Language Programs. Language Assessment Quarterly, 6(4), 265–287. https://doi.org/10.1080/15434300903063038
13. Elgart, D. B. (1975). oral reading, silent reading, and listening comprehension: a comparative study. Journal of Reading Behavior, (1), 203–207.
14. Fayaz, A., Mazahery, A., Hosseinzadeh, M., &Yazdanpanah, S. (2015). Video-based Learning Versus Traditional Method for Preclinical Course of Complete Denture Fabrication. Journal of Dentistry (Shiraz, Iran), 16(1 Suppl), 21–28. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/26106631%5Cnhttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4476117
15. Ferriol, J. L. M. (2013). Subtitle reading speeds in different languages : the case of Lethal Weapon. Quaderns, 20, 201–210.
16. Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15(4), 313–331. https://doi.org/10.1016/j.learninstruc.2005.07.001
17. Halverson, T., Estepp, J., Christensen, J., &Monnin, J. (2012). Classifying workload with eye movements in a complex task. Proceedings of the Human Factors and Ergonomics Society, (November), 168–172. https://doi.org/10.1177/1071181312561012
18. Hasegawa, M., Carpenter, P. A., &Just, M. A. (2002). An fMRI study of bilingual sentence comprehension and workload. NeuroImage, 15(3), 647–660. https://doi.org/10.1006/nimg.2001.1001
19. Hassanabadi, H., Robatjazi, E. S., &Savoji, A. P. (2011). Cognitive consequences of segmentation and modality methods in learning from instructional animations. Procedia - Social and Behavioral Sciences, 30, 1481–1487. https://doi.org/10.1016/j.sbspro.2011.10.287
20. Hercik, P., Milkova, E., &El-Hmoudova, D. (2015). Language Skills Development in E-learning Language Courses. Procedia - Social and Behavioral Sciences, 182, 653–659. https://doi.org/10.1016/j.sbspro.2015.04.803
21. Hill, S. G., Iavecchia, H. P., Byers, J. C., Bittner, A. C., Zaklad, A. L., &Christ, R. E. (1992). Comparison of four subjective workload rating scales. Human Factors, 34(4), 429–439. https://doi.org/10.1177/001872089203400405
22. Huib, K., Rob, L., Merrienboer, V., Jeroen, J. G., Martens, R. L., &Merriënboer, J. J. G.Van. (2001). The modality effect in multimedia instructions.
23. Ipek, H. (2009). Comparing and Contrasting First and Second Language Acquisition : Implications for Language Teachers. English Language Teaching, 2(2), 155–163. https://doi.org/10.5901/mjss.2013.v4n10p56
24. Iqbal, S. T., Zheng, X. S., &Bailey, B. P. (2004). Task-evoked pupillary response to mental workload in human-computer interaction. Extended Abstracts of the 2004 Conference on Human Factors and Computing Systems - CHI ’04, 1477. https://doi.org/10.1145/985921.986094
25. Kim, J. Y., &Ji, Y. G. (2013). A Comparison of Subjective Mental Workload Measures in Driving Contexts. Journal of the Ergonomics Society of Korea, 32(2), 167–177. https://doi.org/10.5143/JESK.2013.32.2.167
26. Klingner, J., Kumar, R., &Hanrahan, P. (2008). Measuring the task-evoked pupillary response with a remote eye tracker. Proceedings of the 2008 Symposium on Eye Tracking Research & Applications - ETRA ’08, 1(212), 69. https://doi.org/10.1145/1344471.1344489
27. Lowe, R. K. (2003). Animation and learning: Selective processing of information in dynamic graphics. Learning and Instruction, 13(2), 157–176. https://doi.org/10.1016/S0959-4752(02)00018-X
28. Mapp, A. P., Ono, H., &Barbeito, R. (2003). What does the dominant eye dominate ? A brief and somewhat contentious review, 65(2), 310–317.
29. Marquart, G., Cabrall, C., &deWinter, J. (2015). Review of Eye-related Measures of Drivers’ Mental Workload. Procedia Manufacturing, 3(Ahfe), 2854–2861. https://doi.org/10.1016/j.promfg.2015.07.783
30. Marshall, S. (2002). The index of cognitive activity: Measuring cognitive workload. Human Factors and Power Plants, 2002. Proceedings of the 2002 IEEE 7th Conference, 75–79. https://doi.org/10.1109/HFPP.2002.1042860
31. Marshall, S. P. (2000). Method and Apparatus for Eye Tracking and Monitorng Pupl Dlation To Evaluate Cognitive Activity, (19).
32. Martí Ferriol, J. L. (2012). Velocidades de lectura de subtítulos en alemán y español de películas norteamericanas. Estudios de Traducción, 2, 47–60. https://doi.org/10.5209/rev_ESTR.2012.v2.38977
33. Marx, W. (2012). Assessment of Cognitive Load in Multimedia Learning with Dual-Task Methodology: Auditory Load and Modality Effects. European Science Editing, 38(2), 35–37. https://doi.org/10.1023/B
34. Masuda, T., Ishii, K., Miwa, K., Rashid, M., Lee, H., &Mahdi, R. (2017). One label or two? Linguistic influences on the similarity judgment of objects between English and Japanese speakers. Frontiers in Psychology, 8(SEP), 1–12. https://doi.org/10.3389/fpsyg.2017.01637
35. Mayer, R. E. (2009). Multimedia Learning. Multi-Media Learning, 2, 318. https://doi.org/10.1016/S0079-7421(02)80005-6
36. Mayer, Richard E. (2010). Unique contributions of eye-tracking research to the study of learning with graphics. Learning and Instruction, 20(2), 167–171. https://doi.org/10.1016/j.learninstruc.2009.02.012
37. Miller, G. A. (1956). The Magical Number Seven, Plus or Minus Two: Some Limites on out Capacity for Processing Information. Psychological Review, 65(2), 81–97. https://doi.org/http://dx.doi.org/10.1037/h0043158
38. Miller, S. (2001). Workload measures. The University of Iowa, (August), 1–65.
39. Mitterer, H., &McQueen, J. M. (2009). Foreign subtitles help but native-language subtitles harm foreign speech perception. PLoS ONE, 4(11), 4–8. https://doi.org/10.1371/journal.pone.0007785
40. Molina, A. I., Navarro, Ó., Ortega, M., &Lacruz, M. (2018). Evaluating multimedia learning materials in primary education using eye tracking. Computer Standards and Interfaces, 59(January), 45–60. https://doi.org/10.1016/j.csi.2018.02.004
41. Mousavi, S. Y., Low, R., &Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87(2), 319–334. https://doi.org/10.1037/0022-0663.87.2.319
42. Palinko, O., Kun, A. L., Shyrokov, A., &Heeman, P. (2010). Estimating cognitive load using remote eye tracking in a driving simulator. Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA ’10, 141. https://doi.org/10.1145/1743666.1743701
43. Perani, D., &Abutalebi, J. (2005). The neural basis of first and second language processing. Current Opinion in Neurobiology, 15(2), 202–206. https://doi.org/10.1016/j.conb.2005.03.007
44. Psacharopoulos, G., Montenegro, C. E., &Patrinos, H. A. (2017). Education Financing Priorities in Developing Countries. Journal of Educational Planning and Administration, 31(1), 5–16. https://doi.org/10.1111/j.1464-0597.2004.00161.x
45. Ragab, N. (2016). Efficacy of Preferred Learning Styles and Teaching Methods of Post Graduate Nursing Students Enrolled In a Master ’ s Degree Program on Exam Achievements, 5(3), 8–20. https://doi.org/10.9790/1959-0503060820
46. Raleigh, M. F., Wilson, G. A., Moss, D. A., Reineke-Piper, K. A., Walden, J., Fisher, D. J., …Zakrajsek, T. (2018). Same content, different methods: Comparing lecture, engaged classroom, and simulation. Family Medicine, 50(2), 100–105. https://doi.org/10.22454/FamMed.2018.222922
47. Reid, G. B., &Nygren, T. E. (1988). THE SUBJECTIVE YORKLOAD ASSESSUENT TECHNIQUE: A SCALING PROCEDURE FOR HEASURING MENTAL WORKLOAD. Advances in Psychology, 52, 185–218. https://doi.org/10.1016/S0166-4115(08)62387-0
48. Ross, N. M., &Kowler, E. (2013). Eye movements while viewing narrated, captioned, and silent videos. Journal of Vision, 13(4), 1–1. https://doi.org/10.1167/13.4.1
49. Savoji, A. P., Hassanabadi, H., &Fasihipour, Z. (2011). The modality effect in learner-paced multimedia learning. Procedia - Social and Behavioral Sciences, 30, 1488–1493. https://doi.org/10.1016/j.sbspro.2011.10.288
50. Schwalm, M., Keinath, A., &Zimmer, H. D. (2008). Pupillometry as a method for measuring mental workload within a simulated driving task. Human Factors for Assistance and Automation, (January), 1–13.
51. Skirgård, H., Roberts, S. G., &Yencken, L. (2017). Why are some languages confused for others? Investigating data from the great language game. PLoS ONE (Vol. 12). https://doi.org/10.1371/journal.pone.0165934
52. Sorden, S. (2012). The cognitive theory of multimedia learning. Handbook of Educational Theories. Charlotte, NC: …, (July). https://doi.org/10.3102/0091732X09358129
53. Stiller, K. D., Freitag, A., Zinnbauer, P., &Freitag, C. (2009). How pacing of multimedia instructions can influence modality effects: A case of superiority of visual texts. Australasian Journal of Educational Technology, 25(2), 184–203.
54. Sweller, J., Merriënboer, J. J. G.Van, &Paas, F. G. W. C. (1998). Cognitive Architecture and Instructional Design.pdf. Educational Psychology Review, 10(3), 251–296.
55. Szarkowska, A., &Gerber-Morón, O. (2018). Viewers can keep up with fast subtitles: Evidence from eye movements. PLoS ONE, 13(6), 1–30. https://doi.org/10.1371/journal.pone.0199331
56. Tabbers, H., Martens, R., &Merriënboer, J.Van. (2000). Multimedia instructions and Cognitive Load Theory : Education, 1–10.
57. van derWel, P., &vanSteenbergen, H. (2018). Pupil dilation as an index of effort in cognitive control tasks: A review. Psychonomic Bulletin and Review, 1–11. https://doi.org/10.3758/s13423-018-1432-y
58. vanGog, T., &Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 20(2), 95–99. https://doi.org/10.1016/j.learninstruc.2009.02.009
59. Vulchanova, M., Aurstad, L. M. G., Kvitnes, I. E. N., &Eshuis, H. (2015). As naturalistic as it gets: Subtitles in the English classroom in Norway. Frontiers in Psychology, 5(JAN), 1–10. https://doi.org/10.3389/fpsyg.2014.01510
60. Wang, C. Y., Tsai, M. J., &Tsai, C. C. (2016). Multimedia recipe reading: Predicting learning outcomes and diagnosing cooking interest using eye-tracking measures. Computers in Human Behavior, 62, 9–18. https://doi.org/10.1016/j.chb.2016.03.064
61. Yang, T. X., Allen, R. J., Yu, Q. J., &Chan, R. C. K. (2015). The influence of input and output modality on following instructions in working memory. Scientific Reports, 5, 1–8. https://doi.org/10.1038/srep17657
62. http://www.human-memory.net/types_sensory.html
63. http://www.taipeitimes.com/News/front/archives/2018/08/28/2003699330
64. https://www.hongkongfp.com/2018/08/31/taiwan-make-english-official-language-next-year-says-official/
65. https://www.bbcgoodfood.com/recipes/collection/easy-baking
66. https://www.keithv.com/software/nasatlx/nasatlx.html

 
 
 
 
第一頁 上一頁 下一頁 最後一頁 top
* *