已經有心理學研究指出，成員之間的互動行為與小組成效息息相關。目前已經有許多科學家試圖用機器學習，甚至是深度學習的方式來找出成員行為與小組成效間的關係，不過以往的研究都專注於單一場次的單一互動行為，而非關注小組成員長時間的互動中行為改變對小組成效的影響。除此之外，角色在小組中也是相當重要的一環，不過在這些研究中，卻常常被忽略。因此，我們的研究希望能提出一個工程的方法用角色的聲音、對話、文字行為變化來預測小組任務成效的進退步。單純使用聲音特徵，我們的預測準確率能夠達到50.3%，加入其他兩種特徵後我們的準確率能夠達到52.7，比基本分高出10%。

有趣的是，我們發現預測結果最好的竟然是在最低統治力的角色上，因此針對這個角色的對話以及文字行為分析過後，我們發現進步組別裡最低統治力角色在整個互動過程會盡量保持自己的對話行為一致性。除此之外，進步組別裡最低統治力的角色在兩次連續互動過程中會漸漸將自己轉換為輔助的角色。

Studies have shown during small group interaction, members behaviors are closely related to the overall group performances. While several works have modeled behaviors in predicting group performances using deep learning methods, most research focuses on predicting in a single-session setting instead of the progression of groups in a multi-session scenario. In this work, we present a computational study in predicting group progression using role’s acoustic, conversational, and verbal behavior difference features. The unweighted average recall (UAR) using only acoustic features achieves 50.3% in the 3-class classification, and can be improved to 52.7% by multimodal fusion, which outperforms the baseline by 10.0%.

Interestingly, our experimental result suggests that the least talkative person has the best predicting result. By analyzing the least talkative person’s conversational and verbal behavior differences, we find that he/she tends to behave similarly in both pre and post interaction episodes and he/she turns into an supporting role in post interaction episode for those groups that obtain an improvement in the performance scores.

誌謝 I
摘要 II
ABSTRACT III
CONTENTS IV
CHAPTER 1 INTRODUCTION 1
CHAPTER 2 METHODOLOGY 7
2.1 THE NTUBA AUDIO-VIDEO DATABASE 7
2.1.1 Group Progress (Label) 9
2.1.2 Quantitative Dominance 10
2.2 FEATURE EXTRACTION 11
2.2.1 Acoustic Feature 11
2.2.2 Conversational Behavior Feature 12
2.2.3 Verbal Feature 13
2.3 DEEP CONVERSATION DISCREPANCY NETWORK 14
2.3.1 Deep Acoustic Encoder with Triplet Loss 16
CHAPTER 3 EXPERIMENT SETUP AND RESULT 18
3.1 EXPERIMENT SETUP 18
3.1.1 Network Structure and Training Parameters 18
3.1.2 Experiment Setup 19
3.2 EXPERIMENT RESULT 22
CHAPTER 4 ANALYSIS 26
4.1 ANALYSIS ON LQD’S CONVERSATIONAL BEHAVIOR 26
4.2 ANALYSIS ON LQD’S VERBAL BEHAVIOR 29
4.2.1 Insight-related Words 29
4.2.2 Task-related Words 33
CHAPTER 5 CONCLUSION 36
REFERENCE 38

[1] Daniel Gatica-Perez, Oya Aran, and Dinesh Jayagopi. 2017. Analysis of Small Groups. Cambridge University Press, 349–367. https://doi.org/10.1017/ 9781316676202.025
[2] J Richard Hackman and Charles G Morris. 1975. Group tasks, group interaction process, and group performance effectiveness: A review and proposed integration. In Advances in experimental social psychology. Vol. 8. Elsevier, 45–99.
[3] Warren E Watson and Larry K Michaelsen. 1988. Group interaction behaviors that affect group performance on an intellective task. Group & Organization Studies 13, 4 (1988), 495–516.
[4] Kristin J Behfar, Elizabeth A Mannix, Randall S Peterson, and William M Trochim. 2011. Conflict in small groups: The meaning and consequences of process conflict. Small Group Research 42, 2 (2011), 127–176.
[5] Carsten KW De Dreu and Laurie R Weingart. 2003. Task versus relationship conflict, team performance, and team member satisfaction: a meta-analysis. Journal of applied Psychology 88, 4 (2003), 741.
[6] Kubasova, U., & Murray, G. (2020, October). Group Performance Prediction with Limited Context. In Companion Publication of the 2020 International Conference on Multimodal Interaction (pp. 191-195).
[7] Hettie A Richardson, Robert J Vandenberg, Terry C Blum, and Paul M Roman. 2002. Does decentralization make a difference for the organization? An examination of the boundary conditions circumbscribing decentralized decision-making and organizational financial performance. Journal of Management 28, 2 (2002), 217– 244.
[8] Ashtosh Sapru and Hervé Bourlard. 2015. Automatic recognition of emergent social roles in small group interactions. IEEE Transactions on Multimedia 17, 5 (2015), 746–760.
[9] Richard B Caple. 1978. The Sequential Stages of Group Development. Small Group Behavior 9, 4 (1978), 470–76.
[10] Roy B Lacoursiere. 1980.The lifecycle of groups: Group developmental stage theory. Human Sciences Press New York.
[11] Bruce W Tuckman and Mary Ann C Jensen. 1977. Stages of small-group devel- opment revisited. Group & Organization Studies 2, 4 (1977), 419–427.
[12] Fred RH Zijlstra, Mary J Waller, and Sybil I Phillips. 2012. Setting the tone: Early interaction patterns in swift-starting teams as a predictor of effectiveness. European Journal of Work and Organizational Psychology 21, 5 (2012), 749–777.
[13] Annika Wiedow and Udo Konradt. 2011. Two-dimensional structure of team process improvement: Team reflection and team adaptation. Small Group Research 42, 1 (2011), 32–54.
[14] Susan M Carter and Michael A West. 1998. Reflexivity, effectiveness, and mental health in BBC-TV production teams. Small group research 29, 5 (1998), 583–601.
[15] Wendy van Ginkel, R Scott Tindale, and Daan van Knippenberg. 2009. Team reflexivity, development of shared task representations, and the use of distributed information in group decision making. Group Dynamics: Theory, Research, and Practice 13, 4 (2009), 265.
[16] Umut Avci and Oya Aran. 2016. Predicting the performance in decision-making tasks: From individual cues to group interaction. IEEE Transactions on Multimedia 18, 4 (2016), 643–658.
[17] Gabriel Murray and Catharine Oertel. 2018. Predicting group performance in task-based interaction. In Proceedings of the 20th ACM International Conference on Multimodal Interaction. 14–20.
[18] Shun-Chang Zhong, Yun-Shao Lin, Chun-Min Chang, Yi-Ching Liu, and Chi- Chun Lee. 2019. Predicting Group Performances Using a Personality Composite- Network Architecture During Collaborative Task.. In INTERSPEECH. 1676–1680.
[19] Yun-Shao Lin and Chi-Chun Lee. 2020. Predicting Performance Outcome with a
Conversational Graph Convolutional Network for Small Group Interactions. In
ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8044–8048.
[20] Tripp Driskell, James E. Driskell, C. Shawn Burke, and Eduardo Salas. 2017. Team Roles: A Review and Integration. Small Group Re- search 48, 4 (2017), 482–511. https://doi.org/10.1177/1046496417711529 arXiv:https://doi.org/10.1177/1046496417711529
[21] Gary Gemmill. 1986. The Mythology of the Leader Role in Small Groups. Small GroupBehavior17,1(1986),41–50. https://doi.org/10.1177/104649648601700104 arXiv:https://doi.org/10.1177/104649648601700104
[22] John W Thibaut. 2017. The social psychology of groups. Routledge.
[23] Murray R Barrick, Greg L Stewart, Mitchell J Neubert, and Michael K Mount. 1998. Relating member ability and personality to work-team processes and team effectiveness. Journal of applied psychology 83, 3 (1998), 377.
[24] Nicolas Fay, Simon Garrod, and Jean Carletta. 2000. Group Discussion as Interac- tive Dialogue or as Serial Monologue: The Influence of Group Size. Psychological science 11 (12 2000), 481–6. https://doi.org/10.1111/1467- 9280.00292
[25] Mark Freiermuth. 2010. Native Speakers or Non-Native Speakers: Who Has the Floor? Online and Face-to-Face Interaction in Culturally Mixed Small Groups. Computer Assisted Language Learning April 2001 (08 2010), 169–199. https: //doi.org/10.1076/call.14.2.169.5780
[26] Thomas M. Brown and Charles E. Miller. 2000. Communication Networks in Task- Performing Groups: Effects of Task Complexity, Time Pressure, and Interpersonal Dominance.SmallGroupResearch31,2(2000),131–157. https://doi.org/10.1177/ 104649640003100201 arXiv:https://doi.org/10.1177/104649640003100201
[27] Umut Avci and Oya Aran. 2019. Analyzing group performance in small group interaction: Linking personality traits and group performance through the verbal content. In 2019 14th IEEE International Conference on Automatic Face Gesture Recognition (FG2019).1–7. https://doi.org/10.1109/FG.2019.8756531
[28] Anita Williams Woolley, Christopher F Chabris, Alex Pentland, Nada Hashmi, and Thomas W Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. science 330, 6004 (2010), 686–688.
[29] Lourdes Artigas Miralles, Anna Vilaregut Puigdesens, Guillem Feixas Viaplana, Clara Mateu Martínez, Jaakko Seikkula, and Berta Vall Castelló. 2020. Dia- logue and Dominance in Couple Therapy for Depression: Exploring Therapists’ Responses in Creating Collaborative Moments. Family Process 59, 3 (2020), 1080– 1093. https://doi.org/10.1111/famp.12494
[30] Hiroko Itakura. 2001.Describing conversational dominance. Journal of Pragmatics 33, 12 (2001), 1859–1880. https://doi.org/10.1016/S0378-2166(00)00082-5
[31] Fridanna Maricchiolo, Stefano Livi, Marino Bonaiuto, and Augusto Gnisci. 2011.
Hand Gestures and Perceived Influence in Small Group Interaction. The Spanish journal of psychology 14, 2 (2011), 755–764. https://doi.org/10.5209/rev_SJOP. 2011.v14.n2.23
[32] F. Eyben, K. R. Scherer, B. W. Schuller, J. Sundberg, E. André, C. Busso, L. Y. Devillers, J. Epps, P. Laukka, S. S. Narayanan, and K. P. Truong. 2016. The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing. IEEE Transactions on Affective Computing 7, 2 (2016), 190–202.
[33] Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. openSMILE – The Munich Versatile and Fast Open-Source Audio Feature Extractor. MM’10 - Proceedings of the ACM Multimedia 2010 International Conference, 1459–1462. https://doi.org/10.1145/1873951.1874246
[34] Shogo Okada, Yoshihiko Ohtake, Yukiko I Nakano, Yuki Hayashi, Hung-Hsuan Huang, Yutaka Takase, and Katsumi Nitta. 2016. Estimating communication skills using dialogue acts and nonverbal features in multiple discussion datasets. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 169–176.
[35] Umut Avci and Oya Aran. 2016. Predicting the performance in decision-making tasks: From individual cues to group interaction. IEEE Transactions on Multimedia 18, 4 (2016), 643–658.
[36] James Pennebaker, Roger Booth, Ryan Boyd, and Martha Francis. 2015. Linguistic Inquiry and Word Count: LIWC2015. (09 2015).
[37] Peng-Hsuan Li, Tsu-Jui Fu, and Wei-Yun Ma. 2020. Why Attention? Analyze BiLSTM Deficiency and Its Remedies in the Case of NER. Proceedings of the AAAI Conference on Artificial Intelligence 34 (04 2020), 8236–8244. https://doi.org/10. 1609/aaai.v34i05.6338
[38] Xingping Dong and Jianbing Shen. 2018. Triplet loss in siamese network for object tracking. In Proceedings of the European Conference on Computer Vision (ECCV). 459–474.
[39] Alexander Hermans, Lucas Beyer, and Bastian Leibe. 2017. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737 (2017).
[40] Hao-Chun Yang, Fu-Sheng Tsai, Yi-Ming Weng, Chip-Jin Ng, and Chi-Chun Lee. 2018. A triplet-loss embedded deep regressor network for estimating blood pressure changes using prosodic features. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 6019–6023.
[41] Sjir Uitdewilligen, Ramón Rico, and Mary J Waller. 2018. Fluid and stable: Dy- namics of team action patterns and adaptive outcomes. Journal of Organizational Behavior 39, 9 (2018), 1113–1128.
[42] Fred RH Zijlstra, Mary J Waller, and Sybil I Phillips. 2012. Setting the tone: Early interaction patterns in swift-starting teams as a predictor of effectiveness. European Journal of Work and Organizational Psychology 21, 5 (2012), 749–777.
[43] Ewa Kacewicz, James W. Pennebaker, Matthew Davis, Moongee Jeon, and Arthur C. Graesser. 2014. Pronoun Use Reflects Standings in Social Hierarchies. Journal of Language and Social Psychology 33, 2 (2014), 125–143. https://doi.org/ 10.1177/0261927X13502654 arXiv:https://doi.org/10.1177/0261927X13502654