本文提出了一種新的架構用以在給定音樂片段的情況下產出舞步。以往有關編舞生成的研究中，通常會利用到ＲＮＮ或Transformer這類費時也費硬體資源的架構。我們提出一個只使用卷積層的輕量網路的新架構，想探求在這樣條件下模型可以產出的成果。我們測試在非實驗室環境的影片上，計算產出舞步在拍點相關的指摽以及我們新提出的舞步自我相似指數，驗證了此方法的有效性。

This thesis proposes a new learning-based method to generate dance poses from given music clips. Prior approaches often address the choreography generation tasks using models that comprise recurrent networks or transformers and thus make the tasks hardware-demanding and time-consuming. We propose a network architecture that uses convolution layers to explore the extent of lightweight approaches. The experimental results on video in the wild provide a baseline of several beat-related indices and a new self-similarity metric on dance sequence generation and validate the effectiveness of our method.

1 Introduction . . . . .7
2 Related Work . . . . .9
2.1 Retrieval-Based Choreography Generation . . . . .9
2.2 Adversarial Learning-Based Choreography Generation . . . . .10
2.3 Sequence-to-Sequence Choreography Generation . . . . .10
3 Proposed Approach . . . . .12
3.1 Problem Definition . . . . .12
3.2 Input Features Extraction . . . . .13
3.3 Music-Pose Embedding Phase . . . . .13
3.4 Dance Sequence Inference Phase . . . . .14
3.5 Objective Functions . . . . .15
4 Experiments . . . . .16
4.1 Experimental Setup . . . . .16
4.1.1 Implementation Details . . . . .16
4.1.2 Evaluation Metrics . . . . .16
4.1.3 Baseline . . . . .18
4.1.4 Evaluation Settings . . . . .20
4.2 Quantitative Results . . . . .20
4.2.1 Ablation Study . . . . .21
4.3 Qualitative Results . . . . .23
5 Conclusion . . . . .28
6 Bibliography . . . . .29

[1]K. Chen, Z. Tan, J. Lei, S. Zhang, Y. Guo, W. Zhang, and S. Hu. Choreomas-ter: choreography-oriented music-driven dance synthesis.ACM Trans. Graph.,40(4):145:1–145:13, 2021.
[2]B. O. Community.Blender - a 3D modelling and rendering package. Blender Foun-dation, Stichting Blender Foundation, Amsterdam, 2018.
[3]R. Huang, H. Hu, W. Wu, K. Sawada, M. Zhang, and D. Jiang. Dance revolution:Long-term dance generation with music via curriculum learning. InInternationalConference on Learning Representations, 2021.
[4]H.-Y. Lee, X. Yang, M.-Y. Liu, T.-C. Wang, Y.-D. Lu, M.-H. Yang, and J. Kautz.Dancing to music. InNeurIPS, 2019.
[5]M. Lee, K. Lee, and J. Park. Music similarity-based approach to generating dancemotion sequence.Multim. Tools Appl., 62(3):895–912, 2013.
[6]R. Li, S. Yang, D. A. Ross, and A. Kanazawa. Ai choreographer: Music conditioned3d dance generation with aist++, 2021.
[7]M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: Askinned multi-person linear model.ACM Trans. Graphics (Proc. SIGGRAPH Asia),34(6):248:1–248:16, Oct. 2015.23
[8]B. McFee, C. Raffel, D. Liang, D. P. Ellis, M. McVicar, E. Battenberg, and O. Nieto.librosa: Audio and music signal analysis in python. InProceedings of the 14th pythonin science conference, volume 8, 2015.
[9]D. Pavllo, C. Feichtenhofer, D. Grangier, and M. Auli. 3d human pose estimationin video with temporal convolutions and semi-supervised training. InConference onComputer Vision and Pattern Recognition (CVPR), 2019.
[10]J. J. Taoran Tang and H. Mao. Dance with melody: An lstm-autoencoder approach tomusic-oriented dance synthesis. In2018 ACM Multimedia Conference on MultimediaConference, MM 2018, Seoul, Republic of Korea, October 22-26, 2018, pages 1598–1606. ACM, 2018.
[11]S. Tsuchida, S. Fukayama, M. Hamasaki, and M. Goto. Aist dance video database:Multi-genre, multi-dancer, and multi-camera database for dance information process-ing. InProceedings of the 20th International Society for Music Information RetrievalConference, ISMIR 2019, pages 501–510, Delft, Netherlands, Nov. 2019.
[12]Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick. Detectron2. https://github.com/facebookresearch/detectron2, 2019.
[13]Z. H. Xuanchi Ren, Haoran Li and Q. Chen. Self-supervised dance video synthesisconditioned on music. InACM MM, 2020.