研究人員可以使用經驗抽樣方法 (ESM) 立即收集有關參與者經驗的信息。 這是一種方法盛行用於設計和評估觀察研究參與者所需的科技技術，並且從 1977 年開始存在於 Csikszentmihalyi 等人的研究中。 研究人員還將這種方法應用於不同的主題，例如與健康相關的主題 (Fraser et al., 2014) (Haedt-Matt et al., 2011) 。 我們發現有不同的在線工具可以促進 ESM 研究，但它們存在不同的問題。對研究設計者來說，大多數工具需要高的學習曲線去設定和收取高額的使用費。此外，隨著第三方資源的日益豐富，不同的數據源可以使ESM研究的結果更加準確和豐富。活動跟踪是研究設計者可能增進研究結果準確性的方式。其中，收集日曆事件數據是活動追蹤的方法之一，可以查看研究參與者的日常活動。通過這種方式，我們可以了解研究參與者的日常活動，而不僅僅是通過問卷調查的回答來了解研究參與者的日常生活。
為了解決研究設計者使用 ESM 在線工具的問題，本研究實作了基於網頁的 ESM 線上工具。我們設計了兩個創新功能，發送通知和收集日曆事件數據以進行活動跟踪。我們將這些功能與第三方服務 Amazon Web Services 和 Google Calendar 結合。此系統是一個網頁應用程式並且開放原始碼為開源軟體，讓研究設計者可以客製化需求。我們對八名研究參與者和三名研究設計者進行了兩次驗證測試。其中一個測試是為研究參與者模擬為期一周的ESM 研究。另一測試是讓研究設計者執行幾個任務，以任務的完成情況作為測試結果。在驗證測試之後，我們收集了研究設計者和研究參與者的質性訪談資料。結果顯示，該系統對研究設計者來說是穩定的和有用的。 該系統的優勢在於可擴展性，具有開源、靈活、易於集成的特點。 我們解決了研究設計者的痛點，並為研究設計者帶來了有關活動追蹤的價值。

Researchers can gather information on participants' experiences right away using Experience Sampling Method (ESM). It is a method popular in designing and evaluating a technological artifact needed to observe participants in research studies and existing from 1977 in Csikszentmihalyi et al.’s study. Researchers also had applied this method in different topics such as health-related topics (Fraser et al., 2014) (Haedt-Matt et al., 2011). We find there were different online tools to facilitate ESM studies, but they have different problems. Most of them require a higher learning curve to set up and high cost for the study designers. Besides, with the increasing abundance of third-party resources, different data sources can make the result of the ESM study more accurate and abundant. Activity tracking is one way the study designers could support the result of the study. One form of activity tracking is collecting calendar event data to see the study participants’ daily activities. In this way, we can understand the daily activities of the study participants, not only the answers of the study participants through the responses of the survey.
To address the problems for the study designer using ESM online tool, our study sheds light on the implementation of a Web-based ESM online tool. We designed two novel features: sending notifications and collecting calendar event data for activity tracking. We integrated these features with third-party services of Amazon Web Services and Google Calendar. The system is not only supported over a web browser but its open-source nature also lets study designers further customize their needs. Two validation tests were conducted with eight study participants and three study designers. One test simulated an ESM study for a week for the study participants. The other test let study designers perform several tasks to see the completion of the tasks. After the validation test, we gathered feedback from both the study designers and participants in qualitative interviews. In the result, the system was stability and usefulness for the study designers. The system’s advantage was scalability with the feature of open source, flexibility, and easy integration. It addressed the study designers’ pain points and brought value to the result from the activity tracking that study designers can reference.

摘要..................................................................I
ABSTRACT...........................................................III
TABLE OF CONTENTS....................................................V
LIST OF TABLES.....................................................VII
LIST OF FIGURES....................................................VII
CHAPTER 1 INTRODUCTION...............................................1
CHAPTER 2 LITERATURE REVIEW..........................................4
2.1 EXPERIENCE SAMPLING METHOD (ESM).................................4
2.2 ESM IN APPLIED RESEARCH..........................................4
2.3 ONLINE TOOLS USED FOR ESM........................................5
2.4 SHORTCOMINGS OF CURRENT ESM TOOLS................................8
2.5 ACTIVITY TRACKING................................................9
CHAPTER 3 USER EXPERIENCE AND SYSTEM ARCHITECTURE...................11
3.1 STUDY DESIGNER EXPERIENCE.......................................12
3.1.1 Creating a study as a folder for survey collection............12
3.1.2 Creating an ESM study.........................................14
3.1.3 Creating a study and collecting calendar data.................19
3.1.4 Downloading study results.....................................21
3.2 STUDY PARTICIPANT EXPERIENCE....................................22
3.2.1 Confirm to Receive Email Notifications........................22
3.2.2 Share the Google Calendar to Study Designer...................24
3.3 SYSTEM ARCHITECTURE.............................................25
3.3.1 The Architecture of Sending Notifications.....................26
3.3.2 The Architecture of Storing Survey Responses..................28
3.3.3 The Architecture of Collecting Calendar Data..................28
CHAPTER 4 VALIDATION TEST...........................................29
4.1 VALIDATION TEST FOR STUDY PARTICIPANT...........................30
4.1.1 Study Design..................................................30
4.1.2 Recruitment and Onboarding survey.............................31
4.1.3 Repeating Survey and Notification Settings....................32
4.1.4 Sharing calendar data.........................................34
4.1.5 Qualitative Interviews........................................35
4.2 VALIDATION TEST FOR STUDY DESIGNER..............................35
4.2.1 Study Design..................................................35
4.2.2 Analysis methods..............................................37
4.2.3 Qualitative Interviews........................................37
CHAPTER 5 RESULTS...................................................38
5.1 VALIDATION TEST FOR STUDY PARTICIPANT...........................38
5.1.1 Notification Feature..........................................38
5.1.2 Activity Tracking Feature.....................................39
5.1.3 Participants’ Feedback on Notification Feature................41
5.1.4 Participants’ Feedback on Activity Tracking Feature...........42
5.2 VALIDATION TEST FOR STUDY DESIGNER..............................44
5.2.1 Notification Feature..........................................44
5.2.2 Activity Tracking Feature.....................................46
5.2.3 Study Designers’ Overall Feedback.............................46
CHAPTER 6 DISCUSSION................................................49
6.1 ADVANTAGES OF THE SYSTEM........................................49
6.1.1. System Stability and Usefulness..............................49
6.1.2 System Scalability - Open source, flexibility, and easy integration.........................................................50
6.1.3 Activity Tracking.............................................50
CHAPTER 7 LIMITATIONS & FUTURE WORK.................................51
7.1 LIMITATIONS.....................................................51
7.2 FUTURE WORK.....................................................52
7.2.1 Notification Feature..........................................52
7.2.2 Activity Tracking Feature.....................................52
CHAPTER 8 CONCLUSIONS...............................................53
REFERENCES..........................................................54

1. Chang, H. C., Chang, Y. J., Newman, M. W., & Lin, C. H. (2020, April). Combining participatory and ESM: A hybrid approach to collecting annotated mobility data. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-7).
2. Christensen, T. C., Barrett, L. F., Bliss-Moreau, E., Lebo, K., & Kaschub, C. (2003). A practical guide to experience-sampling procedures. Journal of Happiness Studies, 4(1), 53-78.
3. Conner, T. S. (2015, May). Experience sampling and ecological momentary assessment with mobile phones. Retrieved from http://www.otago.ac.nz/psychology/otago047475.pdf
4. Consolvo, S., & Walker, M. (2003). Using the experience sampling method to evaluate ubicomp applications. IEEE pervasive computing, 2(2), 24-31.
5. Csikszentmihalyi, M., Larson, R., & Prescott, S. (2014). The ecology of adolescent activity and experience. In Applications of Flow in Human Development and Education (pp. 241-254). Springer, Dordrecht.
6. Diego Menchaca. (2020, December 24). 6 Apps for Experience Sampling Method (ESM). Teamscope. https://www.teamscopeapp.com/blog/6-apps-for-experience-sampling-method-complete-guide
7. Evans, E. (2014). Domain-Driven Design Reference: Definitions and Pattern Summaries. Dog Ear Publishing.
8. Fraser, D., Kobinsky, K., Smith, S. S., Kramer, J., Theobald, W. E., & Baker, T. B. (2014). Five population-based interventions for smoking cessation: a MOST trial. Translational behavioral medicine, 4(4), 382-390.
9. Gorm, N., & Shklovski, I. (2017, February). Participant driven photo elicitation for understanding activity tracking: Benefits and limitations. In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing (pp. 1350-1361).
10. Haedt-Matt, A. A., & Keel, P. K. (2011). Revisiting the affect regulation model of binge eating: a meta-analysis of studies using ecological momentary assessment. Psychological bulletin, 137(4), 660.
11. Harris, P. A., Swafford, J., Serdoz, E. S., Eidenmuller, J., Delacqua, G., Jagtap, V., ... & Duda, S. N. (2022). MyCap: a flexible and configurable platform for mobilizing the participant voice. JAMIA open, 5(2), ooac047.
12. Hogarth, R. M., Portell, M., & Cuxart, A. (2007). What risks do people perceive in everyday life? A perspective gained from the experience sampling method (ESM). Risk Analysis: An International Journal, 27(6), 1427-1439.
13. Jakobsen, M. R., & Hornbæk, K. (2009, April). Fisheyes in the field: using method triangulation to study the adoption and use of a source code visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1579-1588).
14. Napa Scollon, C., Prieto, C. K., & Diener, E. (2009). Experience sampling: promises and pitfalls, strength and weaknesses. In Assessing well-being (pp. 157-180). Springer, Dordrecht.
15. Perrey, R., & Lycett, M. (2003, January). Service-oriented architecture. In 2003 Symposium on Applications and the Internet Workshops, 2003. Proceedings. (pp. 116-119). IEEE.
16. Randall, W. M., & Rickard, N. S. (2012). Development and trial of a mobile experience sampling method (m-ESM) for personal music listening. Music Perception: An Interdisciplinary Journal, 31(2), 157-170.
17. REDCap Reviews & Ratings. TrustRadius. https://www.trustradius.com/products/redcap/reviews?qs=pros-and-cons#reviews
18. REDCap Reviews. Capterra. https://www.capterra.com/p/234600/RedCap/#reviews
19. Rough, D., & Quigley, A. (2015, October). Jeeves-a visual programming environment for mobile experience sampling. In 2015 IEEE symposium on visual languages and human-centric computing (VL/HCC) (pp. 121-129). IEEE.
20. Shiffman, S. (2009). Ecological momentary assessment (EMA) in studies of substance use. Psychological assessment, 21(4), 486.
21. Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annu. Rev. Clin. Psychol., 4, 1-32.
22. Sloboda, J. A., O'Neill, S. A., & Ivaldi, A. (2001). Functions of music in everyday life: An exploratory study using the Experience Sampling Method. Musicae scientiae, 5(1), 9-32.
23. Stawarczyk, D., Majerus, S., Maj, M., Van der Linden, M., & D'Argembeau, A. (2011). Mind-wandering: Phenomenology and function as assessed with a novel experience sampling method. Acta psychologica, 136(3), 370-381.
24. Teamscope Reviews. Capterra. https://www.capterra.com/p/179110/Teamscope/#reviews
25. Van Berkel, N., Ferreira, D., & Kostakos, V. (2017). The experience sampling method on mobile devices. ACM Computing Surveys (CSUR), 50(6), 1-40.
26. Wen-Yun She. (2018). How Rating Scales Affect Survey Satisficing Behavior in Online Surveys
27. Xie, K., Heddy, B. C., & Greene, B. A. (2019). Affordances of using mobile technology to support experience-sampling method in examining college students' engagement. Computers & Education, 128, 183-198.
28. Xie, K., Heddy, B. C., & Vongkulluksn, V. W. (2019). Examining engagement in context using experience-sampling method with mobile technology. Contemporary Educational Psychology, 59, 101788.
29. Yue, Z., Litt, E., Cai, C. J., Stern, J., Baxter, K. K., Guan, Z., ... & Zhang, G. (2014, April). Photographing information needs: the role of photos in experience sampling method-style research. In Proceedings of the sigchi conference on human factors in Computing Systems (pp. 1545-1554).