根據過去諸多文獻顯示，投資人的情緒可以幫助預測未來股價報酬，因此在分析投資人情緒領域上，一直是業界實務人士與研究者非常關心的議題。本文運用 Wall Street Journal 在2008至2021年所刊登的新聞，參考 Garcia (2013) 和 Obaid and Pukthuanthong (2021) 中建構文字、圖片負面情緒指標的方法，分析投資人情緒與市場報酬的關係。本文的主要貢獻除將新聞的文字資料更加完整化之外，進一步運用機器學習模型 BERT (bidirectional encoder representations from transformers) 將新聞文本依據新聞主題細分成政治、經濟、商業、科技、金融市場、房地產、藝術、生活、雜誌和體育共十種。並運用前述政治、經濟、商業、科技、金融市場、房地產和全部新聞等七種主題的新聞，建構屬於各自新聞主題的文字與圖片之負面情緒指標，探討不同主題的新聞文本與圖片所帶來的投資人情緒對市場報酬是否有不同的影響力。此外本文運用機器學習模型 CNN (convolutional neural network) 對新聞圖片進行情緒分析，幫助我們將圖片分成正面、負面，更有效率的建構圖片的負面情緒指標。本文發現在一般情形之下，上一期金融市場新聞文字的負面情緒可以預測當期市場報酬的下跌，並在後續會有股價報酬反轉 (return reversal) 情況。反之，在一般情形之下，新聞圖片的負面情緒指標與未來市場報酬較無關聯。而控制在發生天災而造成死亡人數遽增的時期時，我們發現當上一期經濟新聞與房地產新聞圖片的負面情緒上升，可以預測當期市場報酬的下跌。

Analyzing investor sentiment is an important issue for both academia and practice since investor sentiment can help us predict future market trends. In this study, we examine the relationship between investor sentiment and market return using news photos and texts from Wall Street Journal. We construct two types of daily investor sentiment index (Photo Pessimism and Text Pessimism) refers to Garcia (2013) and Obaid and Pukthuanthong (2021). The first contribution in this study is that we fulfill the news texts data. Second, we split the news by ten topics which are “Politics”, “Economy”, “Business”, “Technology”, “Financial Markets”, “Real Estates”, “Books & Arts”, “Life & Work”, “Magazine” and “Sports” by BERT (bidirectional encoder representations from transformers) model. In order to examine how different topics of investor sentiment could impact the financial market, we build seven investor sentiment index for both text and photo using the corresponding news topics including “Politics”, “Economy”, “Business”, “Technology”, “Financial Markets”, “Real Estates” and all news. Next, we applied a photo classification model called CNN (convolutional neural network) to efficiently classify sentiment from each photo into positive and negative. We find that when Text Pessimism from financial news increase, it can predict next day stock return. In contrast, we find no relationship between Photo Pessimism and future stock return. However, during periods of traumatic events, Photo Pessimism from economy and real estate news can predict stock return in the following trading day.

1. 前言 1
2. 文獻回顧 4
3. 資料與研究方法 7
4. 實證結果 15
5. 結論 34
附錄 36
參考文獻 40

1. Calomiris, C. W. and H. Mamaysky (2019), “How news and its context drive risk and returns around the world,” Journal of Financial Economics, 133, 299-336.
2. Chemtob, C. M., H. L. Roitblat, R. S. Hamada, M. Y. Muraoka, J. G. Carlson and G. B. Bauer (1999), “Compelled Attention: The Effects of Viewing Trauma-Related Stimuli on Concurrent Task Performance in Posttraumatic Stress Disorder,” Journal of Traumatic Stress, 12, 309-326.
3. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018) “Bert: Pre- training of deep bidirectional transformers for language understanding,“ arXiv preprint arXiv:1810.04805.
4. Dietterich, T. G. (2000) “Ensemble Methods in Machine Learning,” In Proceedings of the International Workshop on Multiple Classifier Systems, 1-15.
5. Edmans, A., A. Fernandez-Perez, A. Garel and I. Indriawan (2021), “Music Sentiment and Stock Returns Around the World,” Journal of Financial Economics, Forthcoming.
6. Edmans, A., D. Garcia and O. Norli (2007), “Sports Sentiment and Stock Returns,” Journal of Finance, 62, 1967-1998.
7. Frid-Adar, M., I. Diamant, E. Klang, M. Amitai, J. Goldberger and H. Greenspan (2018), “GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification,” Neurocomputing, 321, 321-331.
8. Garcia, D. (2013), “Sentiment during Recessions,” Journal of Finance, 68, 1267-1300.
9. Gu, J., Z. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai and T. Chen (2018), “Recent advances in convolutional neural networks,” Pattern Recognition, 77, 354-377.
10. Hirshleifer, D. and T. Shumway (2003), “Good Day Sunshine: Stock Returns and the Weather,” Journal of Finance, 58, 1009-1032.
11. Loey, M., G. Manogaran, M. H. N. Taha and N. E. M. Khalifa (2021), “A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic,” Measurement, 167, 108288.
12. Loughran, T. and B. Mcdonald (2011), “When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks,” Journal of Finance, 66, 35-65.
13. Manela, A. and A. Moreira (2017), “News implied volatility and disaster concerns,” Journal of Financial Economics, 123, 137-162.
14. O’shea, K. and R. Nash (2015), “An Introduction to Convolutional Neural Networks,” arXiv preprint arXiv: 1511.08458.
15. Obaid, K. and K. Pukthuanthong (2021), “A Picture is worth a Thousand Words: Measuring Investor Sentiment by Combining Machine Learning and Photos from News,” Journal of Financial Economics, 144, 273-297.
16. Opitz, D. and R. Maclin (1999), “Popular Ensemble Methods: An Empirical Study,” Journal of Artificial Intelligence Research, 11, 169-198.
17. Pires, T., E. Schlinger and D. Garrette (2019), “How multilingual is Multilingual BERT?,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 4996-5001.
18. Powell, T. E., H. G. Boomgaarden, K. D. Swert and C. H. de Vreese (2015), “A Clearer Picture: The Contribution of Visuals and Text to Framing Effects,” Journal of Communication, 65, 997-1017.
19. Reimers, N. and I. Gurevych (2019), “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” arXiv preprint arXiv: 1908.10084.
20. Rokach, L. (2010), “Ensemble-based classifiers,” Artificial Intelligence Review, 33, 1-39.
21. Sharma, C., D. Bhageria, W. Scott, S. PYKL, A. Das, T. Chakraborty, V. Pulabaigari and B. Gamback (2020), “SemEval-2020 Task 8: Memotion Analysis- The Visuo-Lingual Metaphor!,” Proceedings of the 14th International Workshop on Semantic Evaluation, 759-773.
22. Sun, C., X. Qiu, Y. Xu and X. Huang (2019), “How to Fine-Tune BERT for Text Classification?,” arXiv preprint arXiv: 1905.05583.
23. Tan, M. and Q. V. Le (2019), “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” arXiv preprint arXiv: 1905.11946.
24. Tetlock, P. C. (2007), “Giving Content to Investor Sentiment: The Role of Media in the Stock Market,” Journal of Finance, 62, 1139-1168.
25. You, Q., J. Lou, H. Jin and J. Yang (2015), “Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks,” arXiv preprint arXiv: 1509.06041.
26. Zadeh, A., M. Chen, S. Poria, E. Cambria and L. P. Morency (2017), “Tensor Fusion Network for Multimodal Sentiment Analysis,” Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 1103-1114.