本研究開發了基於擴增實境工程諮詢機器人的系統框架、關鍵模塊整合技術和測試案例演示。為了處理聊天機器人對話中的自然語言，採用了contextualized word embedding models，如 BERT 和 KeyBERT。其中，Wiki corpus用於詞嵌入模型訓練，以及千頁的工程技術文本用於該領域的上下文訓練。諮詢過程中的問題領域分類（存儲/運輸、組立和維修）及其子領域下的任務可以使用訓練好的模型、問題關鍵詞提取與相似性分析進行預測。擴增實境能檢測空間中的物理對象並做出特定反應，例如在顯示裝置上疊加虛擬資訊以提供零件組裝指令。透過將智慧諮詢機器人的對話能力與 AR 使用者介面的優勢相結合，用戶可以更好地理解複雜的工程任務，如組立、檢查和維護。與電力變壓器領域相關的工程知識複雜且難以傳達，例如組裝程序、檢查的關鍵點和維護預防措施。進行變壓器組立時，需要領域專家前往裝配現場進行安全安裝、檢查甚至維護。由於大型電力變壓器領域具有高知識的專家鮮少，導致對執行工程任務的需求很高。因此，本研究以電力變壓器製造業為測試案例，驗證所提出的以擴增實境為基之工程諮詢機器人系統的準確性及有效性。該研究收集了三類工程諮詢（存儲/運輸、組裝和維護）下的共 195 個問題，其實驗諮詢問答結果顯示準確率超過86%。

This research develops the system framework, integrated technologies for key modules, and the case demonstration of the augmented reality (AR) based engineering consultation chatbot platform. For processing the natural language in the chatbot dialogs, the contextualized word embedding models, i.e., BERT and KeyBERT, are applied. The Wiki corpus is used for word embedding model training and over a thousand pages of engineering technical document text are used for domain context training. The consultation domain classification (storage/shipment, assembly, and maintenance) and the specific tasks under their sub-domains can be predicted using the trained models and the question keywords extraction and similarity analysis. Augmented reality detects physical objects in space and makes specific responses, such as superimposing virtual images on a physical device for component assembly instruction. By combining intelligent chatbot dialogue capability with the advantages of AR interfaces, users can better understand complex engineering tasks, such as assembly, inspection, and maintenance. Engineering knowledge related to the power transformer domain is complicated and difficult to convey, e.g., assembly procedures, the critical points of inspection, and maintenance precautions. In addition, experts with high knowledge in the transformer domain are usually rare, which results in high demand for executing engineering tasks. Domain experts may be required to travel frequently to assembly sites for safety installation, inspection, and even maintenance. Therefore, this research uses the power transformer manufacturing industry as the test case and verifies the accuracy and efficacy of the proposed AR-enabled engineering consultation chatbot system. The research collects a total of 195 questions under three categories of engineering consultations (i.e., storage/shipment, assembly, and maintenance) with specific sub-domains and corresponding engineering tasks. The experimental consultation Q&A results show accuracy rate exceeding 86%.

Abstract 2
1. Introduction 9
2. Literature review 11
2.1 Chatbot development 12
2.1.1 chatbot type 13
2.1.2 chatbot application 16
2.2 Natural language technology 17
2.2.1 text mining technology 17
2.2.2 Word embedding model 18
2.2.3 Dialogue module design 21
2.3 Augmented reality 22
2.3.1 Engineering services - patterns 22
2.3.2 Immersive environment 23
2.3.3 Augmented reality for industrial application 24
2.3.4 Dialogue system for engineering task using augmented reality 24
3. Methodologies and modules of AR based engineering chatbot 25
3.1 Knowledge management 26
3.1.1 Knowledge base construction 27
3.1.2 Knowledge base collection 29
3.2 Dialogue module 30
3.2.1 Data pre-process 31
3.2.2 Contextualized word embedding fine-tuning 32
3.2.3 Response selection 33
3.3 Augmented reality module 33
3.3.1 3D model building 33
3.3.2 System interface design 34
3.3.3 Software and Hardware connection 35
4. A test case for the power transformer industry 35
4.1 Knowledge management module building 36
4.1.1 Domain distinction for the instruction manual 36
4.1.2 The knowledge base structure and content in the test case 37
4.2 Dialogue module design. 40
4.2.1 Workflow of the dialogue module. 41
4.2.2 BERT model fine tuning 42
4.2.3 Word embedding model training 48
4.2.4 Cosine similarity for comparing the similarity of two sentences. 49
4.2.5 NLP evaluation 50
4.3 AR module integration 54
4.3.1 Transformer 3D object modeling 54
4.3.2 Object detection method selection 56
4.3.3 Scenarios design for the answers 57
5. Conclusion. 60
References 62

1. Adikari, A., De Silva, D., Alahakoon, D., & Yu, X. (2019). A Cognitive Model for Emotion Awareness in Industrial Chatbots. Paper presented at the 2019 IEEE 17th International Conference on Industrial Informatics (INDIN).
2. Aschenbrenner, D., Rojkov, M., Leutert, F., Verlinden, J., Lukosch, S., Latoschik, M. E., & Schilling, K. (2018). Comparing different augmented reality support applications for cooperative repair of an industrial robot. Paper presented at the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).
3. Bettencourt, L. A., Brown, S. W., & Sirianni, N. J. (2013). The secret to true service innovation. Business Horizons, 56(1), 13-22.
4. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022.
5. De Crescenzio, F., Fantini, M., Persiani, F., Di Stefano, L., Azzari, P., & Salti, S. (2010). Augmented reality for aircraft maintenance training and operations support. IEEE Computer Graphics and Applications, 31(1), 96-101.
6. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
7. Divya, S., Indumathi, V., Ishwarya, S., Priyasankari, M., & Devi, S. K. (2018). A self-diagnosis medical chatbot using artificial intelligence. Journal of Web Development and Web Designing, 3(1), 1-7.
8. Egger, J., & Masood, T. (2020). Augmented reality in support of intelligent manufacturing–a systematic literature review. Computers & Industrial Engineering, 140, 106195.
9. Hoy, M. B. (2018). Alexa, Siri, Cortana, and more: an introduction to voice assistants. Medical reference services quarterly, 37(1), 81-88.
10. Hussain, S., Sianaki, O. A., & Ababneh, N. (2019). A survey on conversational agents/chatbots classification and design techniques. Paper presented at the Workshops of the International Conference on Advanced Information Networking and Applications.
11. Kapočiūtė-Dzikienė, J. (2020). A Domain-Specific Generative Chatbot Trained from Little Data. Applied Sciences, 10(7), 2221.
12. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
13. Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., & Shazeer, N. (2018). Generating wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198.
14. Liu, X., He, P., Chen, W., & Gao, J. (2019). Multi-task deep neural networks for natural language understanding. arXiv preprint arXiv:1901.11504.
15. Lokman, A. S., Zain, J. M., Komputer, F., & Perisian, K. (2009). Designing a Chatbot for diabetic patients. Paper presented at the International Conference on Software Engineering & Computer Systems (ICSECS'09).
16. Malik, A. A., Masood, T., & Bilberg, A. (2020). Virtual reality in manufacturing: immersive and collaborative artificial-reality in design of human-robot workspace. International Journal of Computer Integrated Manufacturing, 33(1), 22-37.
17. Manual, O. (2005). Guidelines for collecting and interpreting innovation data (2005). A joint publication of OECD and Eurostat, Organization for Economic Co-Operation and Development. Statistical Office of the European Communities.
18. Margetis, G., Papagiannakis, G., & Stephanidis, C. (2019). Realistic natural interaction with virtual statues in x-reality environments. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences.
19. Márquez, A. C., de León, P. M., Fernández, J. G., Márquez, C. P., & Campos, M. L. (2009). The maintenance management framework: A practical view to maintenance management. Journal of Quality in Maintenance Engineering.
20. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
21. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Paper presented at the Advances in neural information processing systems.
22. Nilsson, J., & Bertling, L. (2007). Maintenance management of wind power systems using condition monitoring systems—life cycle cost analysis for two case studies. IEEE Transactions on energy conversion, 22(1), 223-229.
23. Oliva, R., & Kallenberg, R. (2003). Managing the transition from products to services. International journal of service industry management.
24. Peng, Y., Ding, L., Xu, Z., Jiang, Y., & Chen, J. (2017, 15-17 Dec. 2017). Design and realization of augmented reality based operation training system for operation and maintenance personnel of intelligent transformer substation. Paper presented at the 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).
25. Pennington, J., Socher, R., & Manning, C. D. (2014). Glove: Global vectors for word representation. Paper presented at the Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP).
26. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365.
27. Quamar, A., Lei, C., Miller, D., Ozcan, F., Kreulen, J., Moore, R. J., & Efthymiou, V. (2020). An Ontology-Based Conversation System for Knowledge Bases. Paper presented at the Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data.
28. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. In.
29. Rosen, E., Whitney, D., Phillips, E., Chien, G., Tompkin, J., Konidaris, G., & Tellex, S. (2020). Communicating robot arm motion intent through mixed reality head-mounted displays. In Robotics Research (pp. 301-316): Springer.
30. Schroeder, G., Steinmetz, C., Pereira, C. E., Muller, I., Garcia, N., Espindola, D., & Rodrigues, R. (2016). Visualising the digital twin using web services and augmented reality. Paper presented at the 2016 IEEE 14th International Conference on Industrial Informatics (INDIN).
31. Serras, M., García-Sardiña, L., Simões, B., Álvarez, H., & Arambarri, J. (2020). Dialogue Enhanced Extended Reality: Interactive System for the Operator 4.0. Applied Sciences, 10(11), 3960.
32. Sharma, P., & Li, Y. (2019). Self-Supervised Contextual Keyword and Keyphrase Retrieval with Self-Labelling.
33. Shawar, B. A., & Atwell, E. (2002). A comparison between Alice and Elizabeth chatbot systems: University of Leeds, School of Computing research report 2002.19.
34. Singh, J., Joesph, M. H., & Jabbar, K. B. A. (2019). Rule-based chabot for student enquiries. Paper presented at the Journal of Physics: Conference Series.
35. Smith, L. N. (2017). Cyclical learning rates for training neural networks. Paper presented at the 2017 IEEE winter conference on applications of computer vision (WACV).
36. Thomas, N. (2016). An e-business chatbot using AIML and LSA. Paper presented at the 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI).
37. Trappey, A., Trappey, C. V., Chang, C.-M., Kuo, R. R., Lin, A. P., & Nieh, C. (2020). Virtual Reality Exposure Therapy for Driving Phobia Disorder: System Design and Development. Applied Sciences, 10(14), 4860.
38. Trischler, A., Ye, Z., Yuan, X., & Suleman, K. (2016). Natural language comprehension with the epireader. arXiv preprint arXiv:1606.02270.
39. Turing, A. M. (2009). Computing machinery and intelligence. In Parsing the turing test (pp. 23-65): Springer.
40. Wallace, R. S. (2009). The anatomy of ALICE. In Parsing the Turing Test (pp. 181-210): Springer.
41. Wanner, L., André, E., Blat, J., Dasiopoulou, S., Farrùs, M., Fraga, T., . . . Martínez, O. (2017). Kristina: A knowledge-based virtual conversation agent. Paper presented at the International conference on practical applications of agents and multi-agent systems.
42. Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.
43. Wen, T.-H., Vandyke, D., Mrksic, N., Gasic, M., Rojas-Barahona, L. M., Su, P.-H., . . . Young, S. (2016). A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562.
44. Wu, Y., Li, Z., Wu, W., & Zhou, M. (2018). Response selection with topic clues for retrieval-based chatbots. Neurocomputing, 316, 251-261.
45. Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., . . . Macherey, K. (2016). Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144.
46. Yan, Z., Duan, N., Bao, J., Chen, P., Zhou, M., Li, Z., & Zhou, J. (2016). Docchat: An information retrieval approach for chatbot engines using unstructured documents. Paper presented at the Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
47. Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018). Recent trends in deep learning based natural language processing. ieee Computational intelligenCe magazine, 13(3), 55-75.
48. Zhang, H., Xu, J., & Wang, J. (2019). Pretraining-based natural language generation for text summarization. arXiv preprint arXiv:1902.09243.
49. Zhang, M.-L., & Zhou, Z.-H. (2013). A review on multi-label learning algorithms. IEEE transactions on knowledge and data engineering, 26(8), 1819-1837.