人機介面開發的過程中需進行多次易用性測試，但會耗費大量的時間，恐降低開發者採行的意願；因此，可針對使用者評估開發代理人系統（agent system）、模擬使用者與介面互動過程中的行為，做為介面開發初期的替代評估方法。然而，過去研究多沒有完整考慮使用者的認知歷程，使模擬結果的說服力有限，因此，本研究旨在建立以理論為基礎的模型、根據介面設計推估使用者的視覺注意力分配及操作意圖，以促成更貼近真實行為的模擬評估。
本研究鎖定二維圖形化介面，建構以人類訊息處理歷程為架構的模型，包含知覺歷程（以N-SEEV模型預測使用者與介面互動時最常見的視線軌跡）與認知決策（以基於相似度的決策模型預測最常見的點擊序列）的模擬：模型以介面設計與包含的感興趣區域（area of interest）之座標範圍、對應任務之正確操作步驟為輸入參數，首先預測多數使用者在各個感興趣區域之間的視覺注意力分配，並依序輸出為視線軌跡，接著比對當前注視之感興趣區域與子任務的操作目標之間的外觀相似程度以判斷是否可能點擊此感興趣區域，並輸出為點擊序列。
本研究透過驗證實驗蒐集使用者實際與數個標的介面互動之行為，並分別以ScanMatch演算法、點擊困惑度（click perplexity）量化「參與者實際的行為」與「模型預測的行為」之間的差異，以評估模型預測的視線軌跡與點擊序列是否確可反映真實使用者的視覺注意力分布與操作意圖。結果顯示，介面的預測視線軌跡與點擊序列皆通過驗證，然而，部分介面的視線軌跡預測結果並未與最常見之真實行為一致，說明視線軌跡的預測成效仍有改善空間；預測點擊序則都符合常見的真實行為，亦即能良好地反映使用者實際的操作意圖。
雖然因使用者的視線軌跡個體差異較大而較難以模型反映視線軌跡的多樣性，但仍得以據此有效預測使用者與設計單純的介面互動時的點擊序列；若進一步將預測的點擊序列與其他模擬使用者行為反應的理論結合，便能推估使用者的滑鼠移動時間等操作績效，藉以做為介面開發初期的快速評估工具、輔助比較不同設計方案之間的優劣。

Testing and evaluation are essential for refining interface design, but it’s often time-consuming. Thus, the agent system for user evaluation would be needed to enhance the acceptance of usability testing. Nevertheless, previous studies didn’t fully consider the cognitive processes of user, which weakens the simulation results. Therefore, this study aims to simulate the gaze trajectory and click sequence during the interaction between the user and the interface, so as to represent the user’s behavior more realistically.
Focusing on two-dimensional graphical user interfaces, this study constructed a prediction model based on the human information processing model. The N-SEEV model and the decision model based on similarity comparison were adopted for the simulation of gaze trajectory and click sequence respectively. Input parameters include interface design, coordinates of the areas of interest (AOIs), and operational steps for corresponding tasks. The model predicts the visual attention allocation of the majority of users among various AOIs, generating the gaze trajectory as the output. Subsequently, the appearance similarity between the fixated AOI and the operational target for the current subtask are compared to determine whether the AOI would be clicked, generating the click sequence of the majority of users as the output.
Besides, an experiment was designed to validate the simulation results. On the one hand, the ScanMatch algorithm and click perplexity were used to quantify the difference between the "predicted behavior" and the "actual behavior." Both measures were then compared against the prespecified criterion, suggesting that the predicted gaze trajectory and click sequence of the interface are both acceptable. Although the predicted gaze trajectories of four interfaces did not align with the most common behavior, the predicted click sequences all aligned well with the users' actual intention of operation. By further integrating the predicted click sequence with predictive models of movement time, user performance could be estimated for the rapid evalutation of interface design.

摘要 I
第一章 緒論 1
1.1.研究背景與動機 1
1.2.研究目的與範圍 4
1.3.研究架構與流程 5
第二章 文獻探討 8
2.1.人類訊息處理歷程 8
2.1.1.知覺 9
2.1.2.認知及決策 10
2.1.3.行為反應 11
2.1.4.新手的認知歷程 11
2.1.5.認知歷程小結 12
2.2.使用者行為模型 12
2.2.1.模擬使用者的視覺注意力 12
2.2.2.使用者認知決策 17
2.2.3.使用者行為的模擬方法 21
2.2.4.模型建立方法 21
2.3.視線軌跡分析方法 22
2.4.點擊序列的分析方法 25
2.5.小結 26
第三章 研究方法 27
3.1.建立預測模型 27
3.1.1.建立視覺注意力分布的預測模型 28
3.1.2.建立認知決策預測模型 32
3.2.驗證實驗 37
3.2.1.實驗設計 37
3.2.2.實驗採用之介面 38
3.2.3.參與者 41
3.2.4.實驗流程 41
3.2.5.分析方法 43
3.2.6.模型調整方法 48
3.2.7.前測結果 49
第四章 研究結果 50
4.1.介面易用性分類 50
4.2.視線軌跡的預測結果與驗證 52
4.2.1.「高易用性一頁式網頁」之預測視線軌跡 53
4.2.2.「低易用性一頁式網頁」之預測視線軌跡 56
4.2.3.「高易用性多頁式網頁」之預測視線軌跡 58
4.2.4.「低易用性多頁式網頁」之預測視線軌跡 59
4.3.點擊序列的預測結果與驗證 62
4.3.1.「高易用性一頁式網頁」之預測點擊序列 62
4.3.2.「低易用性一頁式網頁」之預測點擊序列 64
4.3.3.「高易用性多頁式網頁」之預測點擊序列 65
4.3.4.「低易用性多頁式網頁」之預測點擊序列 67
4.4.小結 68
第五章 討論 69
5.1.參與者行為的變異 69
5.1.1.視線軌跡的變異 69
5.1.2.點擊序列的變異 74
5.1.3.自覺易用性分數的變異 76
5.2.對於預測點擊序列驗證成效過度樂觀的風險 77
5.3.模型的應用與限制 78
第六章 結論 82
6.1.主要發現 82
6.2.研究貢獻與應用 83
6.3.研究限制與未來方向 84
參考文獻 88
附錄一、研究倫理審查核可證明 97
附錄二、前測所採用之介面 98
附錄三、介面易用性評估量表（譯自Brooke, 1986） 100
附錄四、感興趣區域（AOI）座標 101
附錄五、30位參與者的視線軌跡 102

中文：
1. 吴旭、完颜笑如、庄达民（2013）。多因素条件下注意力分配建模。北京航空航天大学学报（8），頁1086-1090。
2. 周庭瑱（2021）。基於使用者操作行為模擬及元件配置最佳化的介面設計：以二維圖形化使用者介面為例。清華大學工業工程與工程管理學系學位論文。
英文：
1. Aboufoul, M., Wesslen, R., Cho, I., Dou, W., & Shaikh, S. (2018). Using hidden markov models to determine cognitive states of visual analytic users. In Proceedings of the Machine Learning from User Interaction for Visualization and Analytics Workshop at IEEE VIS.
https://webpages.charlotte.edu/sshaikh2/pubs/c47.pdf
2. Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications, 37(4), 445-456.
https://www.academia.edu/download/6190316/10.1.1.94.381.pdf
3. Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences, 108(25), 10367-10371.
https://www.pnas.org/doi/abs/10.1073/pnas.1104047108
4. Anderson, N. C., Anderson, F., Kingstone, A., & Bischof, W. F. (2015). A comparison of scanpath comparison methods. Behavior research methods, 47(4), 1377-1392.
5. Anibal, S. (2003). Sequence Alignment Algorithms. MSc dissertation, King’s College London
6. Bailly, G., Oulasvirta, A., Brumby, D. P., & Howes, A. (2014, April). Model of visual search and selection time in linear menus. In Proceedings of the sigchi conference on human factors in computing systems (pp. 3865-3874).
7. Boccignone, G. & Ferraro, M. (2004). Modelling gaze shift as a constrained random walk. Physica A: Statistical Mechanics and its Applications, 331(1-2), 207-218.
8. Borji & L. Itti (2013). "State-of-the-Art in Visual Attention Modeling," in IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 185-207, Jan. 2013, https://ieeexplore.ieee.org/document/6180177
9. Brehmer (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81(3), 211-241.
10. Bruce, N. & Tsotsos, J. (2005). Saliency based on information maximization. Advances in neural information processing systems, 18.
11. Byrne, M. D. (2012). Unified theories of cognition. Wiley Interdisciplinary Reviews: Cognitive Science, 3(4), 431-438.
12. Byrne, M. D., Anderson, J. R., Douglass, S., & Matessa, M. (1999). Eye tracking the visual search of click-down menus. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 402-409).
13. Caplette, L., & Turk-Browne, N. (2022). Computational reconstruction of mental representations using human behavior. psyarxiv.com
14. Carmi, R. & Itti, L. (2006). Visual causes versus correlates of attentional selection in dynamic scenes. Vision Research, 46(26), 4333-4345.
15. Cavanagh, J. F., Wiecki, T. V., Kochar, A., & Frank, M. J. (2014). Eye tracking and pupillometry are indicators of dissociable latent decision processes. Journal of Experimental Psychology: General, 143(4), 1476.
16. Chase, W. G. & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4(1), 55-81
17. Chen, G. & Li, Y. (2015, November). Comparison Research on Bottom-up Visual Attention Models. In 2015 International Conference on Industrial Technology and Management Science (pp. 548-552). Atlantis Press.
18. Chen, M. C., Anderson, J. R., & Sohn, M. H. (2001, March). What can a mouse cursor tell us more? Correlation of eye/mouse movements on web browsing. In CHI'01 extended abstracts on Human Factors in Computing Systems (pp. 281-282).
19. Cowan, N. (2008). What are the differences between long-term, short-term, and working memory?. Progress in Brain Research, 169, 323-338.
20. Cristino, F., Mathôt, S., Theeuwes, J., & Gilchrist, I. D. (2010). ScanMatch: A novel method for comparing fixation sequences. Behavior Research Methods, 42(3), 692-700.
21. Diehl, E. & Sterman, J. D. (1995). Effects of feedback complexity on dynamic decision making. Organizational behavior and human decision processes, 62(2), 198-215.
22. Dumais, S. T., Buscher, G., & Cutrell, E. (2010, August). Individual differences in gaze patterns for web search. In Proceedings of the Third Symposium on Information Interaction in Context (pp. 185-194).
23. Dupret, G. E. & Piwowarski, B. (2008, July). A user browsing model to predict search engine click data from past observations. In Proceedings of the 31st annual international ACM SIGIR conference on Research and Development in Information Retrieval (pp. 331-338).
24. Egeth, H. E. & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual Review of Psychology, 48(1), 269-297.
25. Eraslan, S., Yesilada, Y., & Harper, S. (2016). Scanpath trend analysis on web pages: Clustering eye tracking scanpaths. ACM Transactions on the Web (TWEB), 10(4), 1-35.
26. Fahimi, R., & Bruce, N. D. (2021). On metrics for measuring scanpath similarity. Behavior Research Methods, 53, 609-628.
27. Findlay, J. M. (2005). Covert attention and saccadic eye movements. In Neurobiology of attention (pp. 114-116). Academic Press.
28. Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381.
29. Foulsham, T. & Kingstone, A. (2013). Fixation-dependent memory for natural scenes: an experimental test of scanpath theory. Journal of Experimental Psychology: General, 142(1), 41.
30. Frøkjær, E., Hertzum, M., & Hornbæk, K. (2000, April). Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 345-352).
31. Giraud, S., Thérouanne, P., & Steiner, D. D. (2018). Web accessibility: Filtering redundant and irrelevant information improves website usability for blind users. International Journal of Human-Computer Studies, 111, 23-35.
32. Goldberg, J. H. & Helfman, J. I. (2010, March). Scanpath clustering and aggregation. In Proceedings of the 2010 symposium on Eye-Tracking Research & Applications (pp. 227-234).
33. Goldstein E. B. & James R.Brockmole. (2016). Sensation and perception 10th edition. Cengage Learning.
34. Goodale, M. A., & Humphrey, G. K. (1998). The objects of action and perception. Cognition, 67(1-2), 181-207.
35. Grotov, A., Chuklin, A., Markov, I., Stout, L., Xumara, F., & de Rijke, M. (2015). A comparative study of click models for web search. In International Conference of the Cross-Language Evaluation Forum for European Languages (pp. 78-90). Springer, Cham.
36. Grundgeiger, T., Hohm, A., Michalek, A., Egenolf, T., Markus, C., & Happel, O. (2022). The validity of the SEEV model as a process measure of situation awareness: the example of a simulated endotracheal intubation. Human Factors, 64(7), 1181-1194.
37. Guo, F., Liu, C., Kannan, A., Minka, T., Taylor, M., Wang, Y. M., & Faloutsos, C. (2009, April). Click chain model in web search. In Proceedings of the 18th international conference on World Wide Web (pp. 11-20).
38. Hadler, F., Soiné, A., & Irrgang, D. (Eds.). (2018). Beyond UX. in Interface Critique Journal Vol. 1
39. Hamrick, J. B. (2019). Analogues of mental simulation and imagination in deep learning. Current Opinion in Behavioral Sciences, 29, 8-16.
40. Horrey, W. J., Wickens, C. D., & Consalus, K. P. (2006). Modeling drivers' visual attention allocation while interacting with in-vehicle technologies. Journal of Experimental Psychology: Applied, 12(2), 67.
41. Hou, X. & Zhang, L. (2007, June). Saliency detection: A spectral residual approach. In 2007 IEEE Conference on computer vision and pattern recognition (pp. 1-8). IEEE. https://pubmed.ncbi.nlm.nih.gov/20462309/
42. Itti, L. & Borji, A. (2015). Computational models: Bottom-up and top-down aspects. arXiv preprint arXiv:1510.07748.
43. Itti, L. & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2(3), 194-203.
44. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254-1259.
45. Jakob Nielsen. (1998, February). Cost of User Testing a Website.
https://www.nngroup.com/articles/cost-of-user-testing-a-website/
46. James, W. (2007). The principles of psychology (Vol. 1).
47. Jeffries, R. & Desurvire, H. (1992). Usability testing vs. heuristic evaluation: was there a contest?. ACM SIGCHI Bulletin, 24(4), 39-41.
48. Jiang, J., He, D., & Allan, J. (2014, July). Searching, browsing, and clicking in a search session: Changes in user behavior by task and over time. In Proceedings of the 37th international ACM SIGIR conference on Research & Development in Information Retrieval (pp. 607-616).
49. Johnston, W. A. (1978). The intrusiveness of familiar nontarget information. Memory & Cognition, 6(1), 38-42.
50. Josephson, S. & Holmes, M. E. (2002, March). Visual attention to repeated internet images: testing the scanpath theory on the world wide web. In Proceedings of the 2002 symposium on Eye Tracking Research & Applications (pp. 43-49).
51. Kahneman, D. (1973), Attention and Effort, Prentice-Hall,Englewood Cliffs, NJ.
52. Kahneman, D. (2003). A perspective on judgment and choice: mapping bounded rationality. American Psychologist, 58(9), 697.
53. Kangasrääsiö, A., Athukorala, K., Howes, A., Corander, J., Kaski, S., & Oulasvirta, A. (2017, May). Inferring cognitive models from data using approximate Bayesian computation. In Proceedings of the 2017 CHI conference on Human Factors in Computing Systems (pp. 1295-1306).
54. Kikolski, M. (2017). Study of production scenarios with the use of simulation models. Procedia Engineering, 182, 321-328.
55. Lawrence, D. & Tavakol, S. (2006). Balanced website design: optimising aesthetics, usability and purpose. Springer Science & Business Media.
56. Levenshtein, V. I. (1966, February). Binary codes capable of correcting deletions, insertions, and reversals. In Soviet Physics Doklady 10(8), 707-710.
57. Lewis, C. & Wharton, C. (1997). Cognitive walkthroughs. In Handbook of Human-Computer Interaction (pp. 717-732). North-Holland.
58. McKee, S. P. & Nakayama, K. (1984). The detection of motion in the peripheral visual field. Vision Research, 24(1), 25-32.
59. Mesbah, A. & Van Deursen, A. (2007, March). Migrating multi-page web applications to single-page Ajax interfaces. In 11th European Conference on Software Maintenance and Reengineering (CSMR'07) (pp. 181-190). IEEE.
60. Miguel Barreda-Ángeles, Ioannis Arapakis, Xiao Bai, B. Barla Cambazoglu, and Alexandre Pereda-Baños. (2015, August). Unconscious physiological effects of search latency on users and their click behaviour. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 203-212).
61. Moray, N. (1986). Monitoring behavior and supervisory control. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance, Vol. 2. Cognitive processes and performance (pp. 1–51). John Wiley & Sons.
62. Navalpakkam, V. & Itti, L. (2005). Modeling the influence of task on attention. Vision Research, 45(2), 205-231.
63. Newell, A. (1992). Unified theories of cognition and the role of Soar. In SOAR: A Cognitive Architecture in Perspective (pp. 25-79). Springer, Dordrech
64. Nielsen , J. (2001, January 20). Usability Metrics. https://www.nngroup.com/articles/usability-metrics/SEE
65. Nielsen, J. (1995). How to conduct a heuristic evaluation. Nielsen Norman Group, 1(1), 8.
66. Nielsen, Jakob (2006), “F-Shaped Pattern for Reading Web Content,” April 17, http://www.nngroup.com/articles/f-shaped-pattern-reading-web-content/.
67. Noton, D. & Stark, L. (1971). Scanpaths in eye movements during pattern perception. Science, 171(3968), 308-311.
68. Orquin, J. L., Bagger, M. P., & Loose, S. M. (2013). Learning Affects Top Down and Bottom Up Modulation of Eye Movements in Decision Making. Judgment and Decision making, 8(6), 700-716.
69. Payne, J. W. (1982). Contingent decision behavior. Psychological Bulletin, 92(2), 382.
70. Poole, D. L. & Mackworth, A. K. (2010). Artificial Intelligence: foundations of computational agents. Cambridge University Press.
71. Ray Crozier, Rob Ranyard, & Ola Svenson. (1997). Decision Making: Cognitive models and explanations (Vol. 1). Psychology Press.
72. Reani, M., Peek, N., & Jay, C. (2018). An investigation of the effects of n-gram length in scanpath analysis for eye-tracking research. In Proceedings of the 2018 acm symposium on eye tracking research & applications (pp. 1-8).
73. Rheem, H., Steelman, K. S., & Gutzwiller, R. S. (2021, September). SEEV Modeler: A GUI-Based Tool for the SEEV Model Simulation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 65(1), 675-679. Sage CA: Los Angeles, CA: SAGE Publications.
74. Robert J. Sternberg & Karin Sternberg (2016). Cognitive Psychology (7th edtion). Wadsworth Publishing.
75. Salvendy, G. (Ed.). (2012). Handbook of Human Factors and Ergonomics. John Wiley & Sons.
76. Savage, L.J. 1954. The Foundations of Statistics. John Wiley.
77. Senders, J. W. (1980). Visual Scanning Processes. JW Senders.
78. Sinha, T., Jermann, P., Li, N., & Dillenbourg, P. (2014). Your click decides your fate: Inferring information processing and attrition behavior from mooc video clickstream interactions. arXiv preprint arXiv:1407.7131.
79. Steelman, K. S., McCarley, J. S., & Wickens, C. D. (2011). Modeling the control of attention in visual workspaces. Human Factors, 53(2), 142-153.
80. Steelman-Allen, K. S., McCarley, J. S., Wickens, C., Sebok, A., & Bzostek, J. (2009, October). N-SEEV: A computational model of attention and noticing. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting 53(12), 774-778. Sage CA: Los Angeles, CA: SAGE Publications.fs
81. Torrente, M. C. S., Prieto, A. B. M., Gutiérrez, D. A., & De Sagastegui, M. E. A. (2013). Sirius: A heuristic-based framework for measuring web usability adapted to the type of website. Journal of Systems and Software, 86(3), 649-663.
82. Tversky, A. & Kahneman, D. (1989). Rational choice and the framing of decisions. In Multiple Criteria Decision Making and Risk Analysis Using Microcomputers (pp. 81-126). Springer, Berlin, Heidelberg.
83. Underwood, G., Foulsham, T., & Humphrey, K. (2009). Saliency and scan patterns in the inspection of real-world scenes: Eye movements during encoding and recognition. Visual Cognition, 17(6-7), 812-834.
84. Vecera, S. P., Cosman, J. D., Vatterott, D. B., & Roper, Z. J. (2014). The control of visual attention: Toward a unified account. In Psychology of Learning and Motivation 60 (pp. 303-347). Academic Press.
85. Von Neumann, J., & Morgenstern, O. (1947). Theory of games and economic behavior (2nd rev. ed.). Princeton University Press.
86. Wang, Y. & Ruhe, G. (2007). The cognitive process of decision making. International Journal of Cognitive Informatics and Natural Intelligence. IJCINI, 1(2), 73-85.
87. Wertheimer, M. (1938). Gestalt theory. In W. D. Ellis (Ed.), A Source Book of Gestalt Psychology (pp. 1–11). Kegan Paul, Trench, Trubner & Company. https://doi.org/10.1037/11496-001
88. Wickens, C. D. (1984). Processing resources in attention. In R. Para-suraman & R. Davies (Eds.), Varieties of Attention (pp. 63–101). Academic Press.
89. Wickens, C. D. (2015). Noticing events in the visual workplace: The SEEV and NSEEV models. In R. R. Hoffman, P. A. Hancock, M. W. Scerbo, R. Parasuraman, & J. L. Szalma (Eds.), The Cambridge Handbook Of Applied Perception Research, Vol. 2, pp. 749–768. Cambridge University Press. https://doi.org/10.1017/CBO9780511973017.046
90. Wickens, C. D., Goh, J., Helleberg, J., Horrey, W., & Talleur, D. A. (2003), “Attentional Models of Multitask Pilot Performance Using Advanced Display Technology,” Human Factors, 45(3), 360–380
91. Wolfe, J. M. (1994). Guided search 2.0 a revised model of visual search. Psychonomic Bulletin & Review, 1(2), 202-238.
92. Wolfe, J. M. & Gray, W. (2007). Guided search 4.0. Integrated Models of Cognitive Systems, 99-119.
93. Zhang, L., Zhang, L., Mou, X., & Zhang, D. (2011). FSIM: A feature similarity index for image quality assessment. IEEE transactions on Image Processing, 20(8), 2378-2386.
94. Zhang, W., Yang, H., Samaras, D., & Zelinsky, G. (2005). A Computational Model of Eye Movements during Object Class Detection. In Proceedings of the 18th International Conference on Neural Information Processing Systems (pp. 1609–1616). MIT Press.