隨著模型參數數量和訓練數據集規模的增加，近期基於深度學習發展的大型語言模型（Large Language Models, LLMs）進展顯著，其性能達到接近人類的水平。深度學習模型是啟發於人類大腦的架構，若能運用此架構模擬人腦腦波的生成，將有助於了解模型是否以類似於人腦的方式運作。本研究推出了一種基於Transformer架構的生成模型EEGen，用於產生連續的合成腦電圖（Electroencephalogram, EEG）訊號。由於EEG訊號有許多噪訊，我們使用了一種量化自動編碼器，將這些訊號壓縮為離散代碼，以更有效地擷取訊號中的時間特徵，實現不同數據集之間的泛化使用性。EEGen的編碼器以EEG訊號作為輸入，將訊號壓縮後傳入解碼器，讓解碼器自回歸地生成離散代碼。我們使用運動想象腦機介面（Brain-Computer Interface, BCI）作為評估我們方法的應用場景。本評估的挑戰在於，由於各個實驗與受試者存在顯著差異，跨數據集合併訓練與測試將是很大的挑戰。我們的實驗結果顯示，合成的EEG能有效地捕捉時間特徵，同時保持原始訊號的複雜度和頻譜。此外，分類任務結果表明，加入合成數據提高了分類準確度，並超過了基於生成對抗網路（Generative Adversarial Network, GAN）或擴散模型等方法的效果。這些發現強調了基於Transformer的生成模型在多個數據集上有效泛化並生成高質量合成EEG訊號的潛力。

Recent advancements in Large Language Models (LLMs) have been significant, with increases in both parameter count and training dataset size leading to performance levels nearing human-like capabilities. Despite their foundation in neural network architectures, it remains unclear if LLMs operate similarly to the human brain. This study introduces a transformer-based generative model, EEGen, designed to consecutively generate synthetic electroencephalogram (EEG) signals. Due to the intrinsic noise in EEG signals, we employ a quantized autoencoder that compresses these signals into discrete codes, effectively capturing their temporal features to facilitate generalization across diverse datasets. The encoder of EEGen processes EEG signals as inputs, and its decoder autoregressively produces discrete codes. We evaluate our method in a motor imagery Brain-Computer Interface (BCI) application, where merging data across datasets is challenging due to experimental differences. Our results demonstrate that the synthetic EEG effectively captures the temporal patterns while preserving the complexity and power spectrum of the original signals. Furthermore, the classification task results reveal that the inclusion of synthetic data enhances performance and surpasses that of Generative Adversarial Network (GAN)-based or diffusion-based methods. These findings highlight the potential of transformer-based generative models to effectively generalize across multiple datasets and produce high-quality synthetic EEG signals.

Abstract (Chinese)------------------------------------------I
Abstract---------------------------------------------------II
Acknowledgements (Chinese)--------------------------------III
Contents---------------------------------------------------IV
List of Figures--------------------------------------------VI
List of Tables-------------------------------------------VIII
1 Introduction----------------------------------------------1
2 Related Works---------------------------------------------6
2.1 Generative Models for Biological Signals----------------6
2.2 Vector Quantization-------------------------------------8
3 Method----------------------------------------------------9
3.1 Task Definition-----------------------------------------9
3.2 RVQ-----------------------------------------------------9
3.3 EEGen--------------------------------------------------11
3.4 CycleGAN-----------------------------------------------14
3.5 DDPM---------------------------------------------------14
4 Experiments----------------------------------------------15
4.1 Dataset------------------------------------------------15
4.2 Dataset Description------------------------------------15
4.3 Data Preprocessing-------------------------------------18
4.4 Channel Selection--------------------------------------19
4.5 Implementation Details---------------------------------19
4.5.1 RVQ Autoencoder Model Architecture-------------------19
4.5.2 EEGen Model Architecture-----------------------------21
4.5.3 CycleGAN Model Architecture--------------------------22
4.5.4 DDPM Model Architecture------------------------------23
4.5.5 Training and Inference of Generative Models----------24
4.6 Evaluation Metrics-------------------------------------25
5 Experimental Results-------------------------------------26
5.1 RVQ Codebook Utilization-------------------------------26
5.2 Data Visualization-------------------------------------29
5.3 Time Series Complexity Analysis------------------------30
5.4 BCI Classification Task--------------------------------31
5.5 Ablation Study-----------------------------------------37
5.6 Limitations--------------------------------------------39
6 Discussion-----------------------------------------------40
7 Conclusion-----------------------------------------------42
Bibliography-----------------------------------------------43

[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[2] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pages 8821–8831. Pmlr, 2021.
[3] Alec Radford, JongWook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492–28518. PMLR, 2023.
[4] Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A Hosseini, Nancy Kanwisher, Joshua B Tenenbaum, and Evelina Fedorenko. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45):e2105646118, 2021.
[5] Charlotte Caucheteux and Jean-R´emi King. Brains and algorithms partially converge in natural language processing. Communications biology, 5(1):134, 2022.
[6] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[7] Kay Gregor Hartmann, Robin Tibor Schirrmeister, and Tonio Ball. Eeggan: Generative adversarial networks for electroencephalograhic (eeg) brain signals. arXiv preprint arXiv:1806.01875, 2018.
[8] Fangzhou Xu, Fenqi Rong, Jiancai Leng, Tao Sun, Yang Zhang, Siddharth Siddharth, and Tzyy-Ping Jung. Classification of left-versus right-hand motor imagery in stroke patients using supplementary data generated by cyclegan. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29:2417–2424, 2021.
[9] Fatemeh Fahimi, Strahinja Dosen, Kai Keng Ang, Natalie Mrachacz-Kersting, and Cuntai Guan. Generative adversarial networks-based data augmentation for brain–computer interface. IEEE transactions on neural networks and learning systems, 32(9):4039–4051, 2020.
[10] Yun Luo and Bao-Liang Lu. Eeg data augmentation for emotion recognition using a conditional wasserstein gan. In 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pages 2535–2538. IEEE, 2018.
[11] Yun Luo, Li-Zhen Zhu, Zi-Yu Wan, and Bao-Liang Lu. Data augmentation for enhancing eeg-based emotion recognition with deep generative models. Journal of Neural Engineering, 17(5):056021, 2020.
[12] Zuochen Wei, Junzhong Zou, Jian Zhang, and Jianqiang Xu. Automatic epileptic eeg detection using convolutional neural network with improvements in time-domain. Biomedical Signal Processing and Control, 53:101551, 2019.
[13] Khansa Rasheed, Junaid Qadir, Terence J O’Brien, Levin Kuhlmann, and Adeel Razi. A generative model to synthesize eeg data for epileptic seizure prediction. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 29:2322–2332, 2021.
[14] Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
[15] Martin Arjovsky, Soumith Chintala, and L´eon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214–223. PMLR, 2017.
[16] Jordan J Bird, Michael Pritchard, Antonio Fratini, Anik´o Ek´art, and Diego R Faria. Synthetic biological signals machine-generated by gpt-2 improve the classification of eeg and emg through data augmentation. IEEE Robotics and Automation Letters, 6(2):3498–3504, 2021.
[17] Gert Pfurtscheller and Christa Neuper. Motor imagery and direct braincomputer communication. Proceedings of the IEEE, 89(7):1123–1134, 2001.
[18] Natasha Padfield, Jaime Zabalza, Huimin Zhao, Valentin Masero, and Jinchang Ren. Eeg-based brain-computer interfaces using motor-imagery: Techniques and challenges. Sensors, 19(6):1423, 2019.
[19] Jonathan R Wolpaw, Niels Birbaumer, Dennis J McFarland, Gert
Pfurtscheller, and Theresa M Vaughan. Brain–computer interfaces for communication and control. Clinical neurophysiology, 113(6):767–791, 2002.
[20] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
[21] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
[22] Yonghao Song, Qingqing Zheng, Bingchuan Liu, and Xiaorong Gao. Eeg conformer: Convolutional transformer for eeg decoding and visualization. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:710–719, 2022.
[23] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin
Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
[24] Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010.14701, 2020.
[25] Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12104–12113, 2022.
[26] Hiroshi Morioka, Atsunori Kanemura, Jun-ichiro Hirayama, Manabu
Shikauchi, Takeshi Ogawa, Shigeyuki Ikeda, Motoaki Kawanabe, and Shin
Ishii. Learning a common dictionary for subject-transfer decoding with resting calibration. NeuroImage, 111:167–178, 2015.
[27] Vinay Jayaram, Morteza Alamgir, Yasemin Altun, Bernhard Scholkopf, and Moritz Grosse-Wentrup. Transfer learning in brain-computer interfaces. IEEE Computational Intelligence Magazine, 11(1):20–31, 2016.
[28] Jenny Gu and Ryota Kanai. What contributes to individual differences in brain structure? Frontiers in human neuroscience, 8:262, 2014.
[29] Sujit Roy, Shirin Dora, Karl McCreadie, and Girijesh Prasad. Mieeg-gan: generating artificial motor imagery electroencephalography signals. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2020.
[30] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017.
[31] Jiaxin Xie, Siyu Chen, Yongqing Zhang, Dongrui Gao, and Tiejun Liu. Combining generative adversarial networks and multi-output cnn for motor imagery classification. Journal of neural engineering, 18(4):046026, 2021.
[32] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
[33] Bruno Aristimunha, Raphael Yokoingawa de Camargo, Sylvain Chevallier, Oeslle Lucena, Adam Thomas, M. Jorge Cardoso, Walter Lopez Pinaya, and Jessica Dafflon. Synthetic sleep EEG signal generation using latent diffusion models. In Deep Generative Models for Health Workshop NeurIPS 2023, 2023.
[34] Rui Niu, Yagang Wang, Haole Xi, Yulong Hao, and Mei Zhang. Epileptic seizure prediction by synthesizing eeg signals through gpt. In Proceedings of the 2021 4th International Conference on Artificial Intelligence and Pattern Recognition, pages 419–423, 2021.
[35] Robert Gray. Vector quantization. IEEE Assp Magazine, 1(2):4–29, 1984.
[36] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[37] Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507, 2021.
[38] Alexandre D´efossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. arXiv preprint arXiv:2210.13438, 2022.
[39] Biing-Hwang Juang and A Gray. Multiple stage vector quantization for speech coding. In ICASSP’82. IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 7, pages 597–600. IEEE, 1982.
[40] Yiqun Duan, Charles Zhou, Zhen Wang, Yu-Kai Wang, and Chin-teng Lin. Dewave: Discrete encoding of eeg waves for eeg to text translation. In Thirty seventh Conference on Neural Information Processing Systems, 2023.
[41] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234–241. Springer, 2015.
[42] Benjamin Blankertz, K-R Muller, Gabriel Curio, Theresa M Vaughan, Gerwin Schalk, Jonathan R Wolpaw, Alois Schlogl, Christa Neuper, Gert
Pfurtscheller, Thilo Hinterberger, et al. The bci competition 2003: progress and perspectives in detection and discrimination of eeg single trials. IEEE transactions on biomedical engineering, 51(6):1044–1051, 2004.
[43] Michael Tangermann, Klaus-Robert M¨uller, Ad Aertsen, Niels Birbaumer, Christoph Braun, Clemens Brunner, Robert Leeb, Carsten Mehring, Kai J Miller, Gernot R M¨uller-Putz, et al. Review of the bci competition iv. Frontiers in neuroscience, 6:55, 2012.
[44] Robin Tibor Schirrmeister, Jost Tobias Springenberg, Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for eeg decoding and visualization. Human Brain Mapping, aug 2017.
[45] Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vectorquantized image modeling with improved VQGAN. In International Conference on Learning Representations, 2022.
[46] Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance. Eegnet: a compact convolutional neural network for eeg-based brain-computer interfaces. Journal of neural engineering, 15(5):056013, 2018.