在地球村的時代，專家透過遠距溝通與合作傳遞專業知識的行為愈來愈熱門，常見的情況像是在異地的同事們使用視訊軟體開會，或是詢問在遠方的修繕專家如何立即修理家中漏水的水管。但在遠距溝通的過程中，由於無法對需要提及的物體或目標進行立即的操作，如「拿」、「取」等動作，我們常常會使用過多的代名詞與指涉名來指稱所要提及的物品，例如「這個」、「那個」、「這裡」或「那裡」;或是使用像「那棟建築物的三樓最左邊的那個房間」這般複雜的形容，這些模稜兩可的句子和模糊的描述都會增加遠距溝通的困難。因此我們期望藉由影像標註系統AlphaRead，讓使用者在進行以視訊輔助溝通的任務中，能以可讀出的標籤(例如英文字母Ａ、Ｂ、Ｃ等)標注出在對話中需要指涉的物體，進而在對話中使用。
我們在使用者研究中，讓使用者進行以視訊輔助的遠距溝通任務，並對視訊加上可讀之物件標註或非可讀之物件標註，比較使用者在不同視訊輔助工具或沒有任何輔助下完成任務的效率。我們發現使用可讀之物件標註可以有效改善使用者的溝通效率與使用滿意度，我們同時也歸納出使用者如何在對話中使用可讀之物件標註和討論其在未來設計遠距協作工具時所能扮演的角色。

As experts and expertise are increasingly distributed across distance, remote collaboration on physical tasks also becomes popular. Physical collaboration requires collaborators to produce and resolve references to physical objects unambiguously. We present a novel annotation system called AlphaRead that enables users to add and see readable annotations of physical objects, such as labels in letter, in a dynamic video-mediated workspace. Explicit support for object readability can help people coordinate language and vision for collaboration, and allow them to directly read out object labels as a way to make unambiguous references. Object readability as a resource of linguistic references can reduce the ambiguity and complexity associated with traditional methods of referential expressions such as deictic pronouns (“this” or “that”) or descriptions of object attributes. In a video- mediated collaboration study, by making objects referable with readable labels, we improved the communication efficiency over the alternative options of using raw video or video with non-readable annotations to collaborate. We also identified patterns of language behaviors that people exhibited with readable labels and discussed the implications to the design of collaboration support tools.

Chapter 1 Introduction 9
Chapter 2 Background 11
2.1 Conversational Grounding in Collaboration 12
Chapter 3 System Design 14
3.1 Overview of features 14
3.1.1 Adding Readable Annotations by Scribbling 14
3.1.2 Object Tracking 15
3.1.3 Interface Features for Helper and Worker 16
3.2 Implementation 18
3.2.1 Object Annotation 18
3.2.2 Object Tracking and Detection 19
Chapter 4 Evaluation 21
4.1 Hypotheses 21
4.2 Design 22
4.3 Participants 23
4.4 Equipment 24
4.5 Tasks 24
4.6 Procedure and Measures 26
4.7 Results 27
4.7.1 Time Efficiency 27
4.7.2 Expression Efficiency 28
4.7.3 User Satisfaction 29
4.7.4 Cognitive Task Load 30
4.7.5 Efficacy of Collaboration 31
Chapter 5 Observation of Language Behaviors 32
5.1 Readable Annotations Eliminate Ambiguous Expression 32
5.1.1 Referring to Objects 32
5.1.2 Referring to Locations 33
5.2 Ambiguity in Conditions without Readable Annotations 34
5.2.1 Ambiguity due to Subjectivity in Color Interpretation 35
5.2.2 Ambiguity due to Subjectivity in Shape Interpretation 35
5.2.3 Comparing Deictic References and Readable Annotations 36
Chapter 6 Discussion 38
6.1 Summary of Results 38
6.2 Subtask Differences 39
6.3 Limitation of the Study 40
6.4 Features to Add and Investigate 41
6.4.1 Re-detection 41
6.4.2 Memorability 41
6.4.3 Temporal Annotations 42
6.4.4 Readability of Global and Compositional Objects 43
6.5 Implications to Everyday Physical Collaboration 43
Chapter 7 Conclusion 44
REFERENCE 45

1. Herbert H. Clark and Deanna Wilkes-Gibbs. 1986. Referring as a collaborative process, Cognition, Volume 22, Issue 1, 1-39. http://dx.doi.org/10.1016/0010-0277(86)90010-7
2. Herbert H. Clark and Susan E. Brennan. 1991. Grounding in communication. In Lauren Resnick, Levine B., M. John, Stephanie Teasley & D. (eds.), Perspectives on Socially Shared Cognition. American Psychological Association 13—1991.
3. Robert Dale and Jette Viethen. 2009. Referring expression generation through attribute-based heuristics. In Proceedings of the 12th European Workshop on Natural Language Generation (ENLG '09). Association for Computational Linguistics, Stroudsburg, PA, USA, 58-65.
4. Fred D. Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 3 (September 1989), 319-340. DOI=http://dx.doi.org/10.2307/249008
5. Susan R. Fussell, Leslie D. Setlock, Jie Yang, Jiazhi Ou, Elizabeth Mauer, and Adam D. I. Kramer. 2004. Gestures over video streams to support remote collaboration on physical tasks. Hum.-Comput. Interact. 19, 3 (September 2004), 273-309. DOI=10.1207/s15327051hci1903_3 http://dx.doi.org/10.1207/s15327051hci1903_3
6. Susan R. Fussell, Leslie D. Setlock, and Robert E. Kraut. 2003. Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 513-520. DOI=10.1145/642611.642701 http://doi.acm.org/10.1145/642611.642701
7. Steffen Gauglitz, Cha Lee, Matthew Turk, and Tobias Höllerer. 2012. Integrating the physical environment into mobile remote collaboration. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI '12). ACM, New York, NY, USA, 241-250. DOI=10.1145/2371574.2371610 http://doi.acm.org/10.1145/2371574.2371610
8. Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). ACM, New York, NY, USA, 449-459. DOI=10.1145/2642918.2647372 http://doi.acm.org/10.1145/2642918.2647372
9. Darren Gergle and Alan T. Clark. 2011. See what i'm saying? : using Dyadic Mobile Eye tracking to study collaborative reference. In Proceedings of the ACM 2011 conference on Computer supported cooperative work (CSCW '11). ACM, New York, NY, USA, 435-444. DOI=10.1145/1958824.1958892 http://doi.acm.org/10.1145/1958824.1958892
10. Sandra G. Hart, and Lowell E. Staveland. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52 (1988): 139-183.
11. Andreas Hofhauser, Carsten Steger, and Nassir Navab. 2008. Edge-Based Template Matching and Tracking for Perspectively Distorted Planar Objects. In Proceedings of the 4th International Symposium on Advances in Visual Computing (ISVC '08), George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Paolo Remagnino, Fatih Porikli, Jörg Peters, James Klosowski, Laura Arns, Yu Ka Chun, Theresa-Marie Rhyne, and Laura Monroe (Eds.). Springer-Verlag, Berlin, Heidelberg, 35-44.
12. Brennan Jones, Anna Witcraft, Scott Bateman, Carman Neustaedter, and Anthony Tang. 2015. Mechanics of Camera Work in Mobile Video Collaboration. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 957-966. DOI=10.1145/2702123.2702345 http://doi.acm.org/10.1145/2702123.2702345
13. David Kirk and Danae Stanton Fraser. 2006. Comparing remote gesture technologies for supporting collaborative physical tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), Rebecca Grinter, Thomas Rodden, Paul Aoki, Ed Cutrell, Robin Jeffries, and Gary Olson (Eds.). ACM, New York, NY, USA, 1191-1200. DOI=http://dx.doi.org/10.1145/1124772.1124951
14. Robert E. Kraut, Susan R. Fussell, and Jane Siegel. 2003. Visual information as a conversational resource in collaborative physical tasks. Hum.-Comput. Interact. 18, 1 (June 2003), 13-49. DOI=10.1207/S15327051HCI1812_2 http://dx.doi.org/10.1207/S15327051HCI1812_2
15. Robert E. Kraut, Mark D. Miller, and Jane Siegel. 1996. Collaboration in performance of physical tasks: effects on outcomes and communication. In Proceedings of the 1996 ACM conference on Computer supported cooperative work (CSCW '96), Mark S. Ackerman (Ed.). ACM, New York, NY, USA, 57-66. DOI=http://dx.doi.org/10.1145/240080.240190
16. Bruce D. Lucas and Takeo Kanade. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th international joint conference on Artificial intelligence - Volume 2 (IJCAI'81), Vol. 2. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 674-679.
17. Hideaki Kuzuoka. 1992. Spatial workspace collaboration: a SharedView video support system for remote collaboration capability. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92), Penny Bauersfeld, John Bennett, and Gene Lynch (Eds.). ACM, New York, NY, USA, 533-540. DOI=http://dx.doi.org/10.1145/142750.142980
18. Steve Mann. 2000. Telepointer: Hands-Free Completely Self Contained Wearable Visual Augmented Reality without Headwear and without any Infrastructural Reliance. In Proceedings of the 4th IEEE International Symposium on Wearable Computers (ISWC '00). IEEE Computer Society, Washington, DC, USA, 177-.
19. Jens Müller, Roman Rädle, and Harald Reiterer. 2016. Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI '16). ACM, New York, NY, USA, 1245-1249. DOI=http://dx.doi.org/10.1145/2858036.2858043
20. James Norris, Holger Schnädelbach, and Guoping Qiu. 2012. CamBlend: an object focused collaboration tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 627-636. DOI=http://dx.doi.org/10.1145/2207676.2207765
21. Gary M. Olson and Judith S. Olson. 2000. Distance matters. Hum.-Comput. Interact. 15, 2 (September 2000), 139-178. DOI=http://dx.doi.org/10.1207/S15327051HCI1523_4
22. Jiazhi Ou, Xilin Chen, Susan R. Fussell, and Jie Yang. 2003. DOVE: drawing over video environment. In Proceedings of the eleventh ACM international conference on Multimedia(MULTIMEDIA '03). ACM, New York, NY, USA, 100-101. DOI=http://dx.doi.org/10.1145/957013.957034
23. Jiazhi Ou, Susan R. Fussell, Xilin Chen, Leslie D. Setlock, and Jie Yang. 2003. Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proceedings of the 5th international conference on Multimodal interfaces (ICMI '03). ACM, New York, NY, USA, 242-249. DOI=http://dx.doi.org/10.1145/958432.958477
24. Nikhil R. Pal and Sankar K. Pal. A review on image segmentation techniques. Pattern recognition 26.9 (1993): 1277-1294.
25. Jette Viethen and Robert Dale. 2008. The use of spatial relations in referring expression generation. In Proceedings of the Fifth International Natural Language Generation Conference(INLG '08). Association for Computational Linguistics, Stroudsburg, PA, USA, 59-67.
26. Abhishek Ranjan, Jeremy P. Birnholtz, and Ravin Balakrishnan. 2007. Dynamic shared visual spaces: experimenting with automatic camera control in a remote repair task. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 1177-1186. DOI=http://dx.doi.org/10.1145/1240624.1240802
27. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. "GrabCut": interactive foreground extraction using iterated graph cuts. In ACM SIGGRAPH 2004 Papers (SIGGRAPH '04), Joe Marks (Ed.). ACM, New York, NY, USA, 309-314. DOI=10.1145/1186562.1015720 http://doi.acm.org/10.1145/1186562.1015720
28. Jianbo Shi and Carlo Tomasi. Good Features to Track. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1994, 593-600.
29. John C. Tang. 1991. Findings from observational studies of collaborative work. Int. J. Man-Mach. Stud. 34, 2 (February 1991), 143-160. DOI=http://dx.doi.org/10.1016/0020-7373(91)90039-A
30. John C. Tang and Scott L. Minneman. 1990. VideoDraw: a video interface for collaborative drawing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '90), Jane Carrasco Chew and John Whiteside (Eds.). ACM, New York, NY, USA, 313-320. DOI=10.1145/97243.97302 http://doi.acm.org/10.1145/97243.97302
31. Pierre Wellner, and Stephen Freeman. The Double DigitalDesk: Shared editing of paper documents. Tech. Rep. EPC-93-108, EuroPARC, 1993
32. Nelson Wong and Carl Gutwin. 2014. Support for deictic pointing in CVEs: still fragmented after all these years. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14). ACM, New York, NY, USA, 1377-1387. DOI=10.1145/2531602.2531691 http://doi.acm.org/10.1145/2531602.2531691
33. Naomi Yamashita and Toru Ishida. 2006. Effects of machine translation on collaborative work. In Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work (CSCW '06). ACM, New York, NY, USA, 515-524. DOI=http://dx.doi.org/10.1145/1180875.118095