工業4.0正在飛快地擴散至整個製造業，對於製造業來說，數位轉型已然不是選擇，而是所有企業所需正視的挑戰。透過人工智慧(artificial intelligence, AI)和物聯網(Internet of Things, IoT)技術驅動的數位孿生(digital twin)智慧製造(smart manufacturing)系統成為當今熱門的研究議題，製造業藉此得以追求更高的製造彈性、製造效率及製造品質。製造品質(manufacturing quality, MQ)預測是智慧製造的基礎之一，針對某些無法快速或是較難測量品質的產品，希望可以快速並且準確地預測製造品質，以滿足製造業品質控管等需求。
本論文聚焦於線切割放電加工(wire electrical discharge machining, WEDM)製造品質預測，具體地說是利用生產前設定的靜態製造參數以及生產過程中收集的動態時間序列資料進行的WEDM工件表面粗糙度(surface roughness)預測。文獻中存在一個WEDM工件表面粗糙度預測研究，利用格拉姆角度域(Gramian angular field, GAF)與馬可夫轉換域(Markov transition field, MTF)將時間序列資料轉換為二維圖像，並輸入至卷積長短期記憶(convolutional long short-term memory network, CLSTM)神經網路深度神經網路來預測工件表面粗糙度。基於這個研究，本論文完成以下三項研究工作: 基於這個研究，本論文完成以下三項研究工作: 第一、提出以特殊的卷積模塊長短期記憶(Convolution Block Long Short-Term Memory, CB-LSTM)神經網路架構為基礎，搭配GAF及MTF對時間序列資料的前處理來進行WEDM工件表面粗糙度預測。第二、使用超參數優化框架Keras Tuner搭配超參數優化演算法Hyperband，來優化CB-LSTM的超參數。第三、搭配遷移學習(transfer learning)技術，使用數量較多的材質A加工資料為來源域(source domain)資料訓練CB-LSTM來源域模型，然後以數量較少的材質B加工資料為目標域(target domain)資料，透過遷移學習中的微調(fine-tune)技術快速產出準確度不錯的目標域模型，藉此訓練出適用於材質B的模型。本論文並以實際WEDM生產製造資料進行優化CB-LSTM模型預測準確度評估，並將評估結果與其他相關方法進行比較。比較的結果顯示，本論文所產出優化的CB-LSTM神經網路，不論搭配GAF或是MTF都有最好的預測準確度。另外，在遷移學習方面，評估與比較結果顯示，我們可以僅透過微調的技術，將資料較多的來源域預測模型快速遷移到資料相對較少的目標域模型上，同樣也達到非常好的預測效果，達到遷移學習的主要目的。

The vision of Industry 4.0 has been rapidly spreading to the entire manufacturing industry. For the manufacturing industry, digital transformation is no longer an option, but a challenge that all companies need to face. The digital twin smart manufacturing system driven by the artificial intelligence (AI) and Internet of Things (IoT) technologies has become a hot research topic today, enabling the manufacturing industry to pursue higher manufacturing flexibility, efficiency and quality. Manufacturing quality (MQ) prediction is one of the foundations of smart manufacturing. For certain products whose qualities cannot be measured quickly or are difficult to measure, it is desirable that their manufacturing quality can be predicted quickly and accurately to realize real-time manufacturing quality control.
This thesis focuses on the prediction of wire electrical discharge machining (WEDM) manufacturing quality. Specifically, it focuses on the prediction of WEDM workpiece surface roughness based on static manufacturing parameters set before manufacturing and dynamic time series data collected during manufacturing. There is an existing method for such prediction. It uses the Gramian angular field (GAF) and the Markov transition field (MTF) to convert time series data into two-dimensional images, which in turn are fed into a convolutional long short-term memory (CLSTM) neural network for prediction. Based on the existing method, this thesis has completed the following three research tasks: First, using a special convolution block long short-term memory (CB-LSTM) neural network along with GAF and MTF to predict the surface roughness of WEDM workpieces. Second, utilizing the hyperparameter optimization framework Keras Tuner with the hyperparameter optimization algorithm Hyperband, to optimize the CB-LSTM neural network hyperparameters. Third, using the transfer learning technology to transfer a source-domain CB-LSTM model S into a target-domain CB-LSTM model T. Note that S is trained by a large amount of source domain manufacturing data of a specific workpiece material and T is for another material. Real WEDM manufacturing data are used to evaluate the prediction accuracy of the optimized CB-LSTM model, and the evaluation results are compared with other related methods. The comparison results show that the optimized CB-LSTM neural network has the best prediction accuracy no matter whether it is combined with GAF or MTF. Furthermore, the evaluation also shows that the transfer leaning mechanism can build target domain model with comparably good prediction accuracy.

中文摘要 I
Abstract III
目錄 V
圖目錄 VII
一、 緒論 1
1.1. 研究背景與動機 1
1.2. 研究目的與方法 2
1.3. 相關研究 3
1.4. 論文架構 5
二、 背景知識 5
2.1. 線切割放電加工 5
2.1.1. 放電加工 5
2.1.2. 線切割放電加工 6
2.2. 格拉姆角度域 7
2.2.1. 格拉姆矩陣 7
2.2.2. 格拉姆角度域 8
2.3. 馬可夫轉換域 9
2.3.1. 馬可夫鏈 9
2.3.2. 馬可夫鏈轉移矩陣 10
2.3.3. 馬可夫轉換域 11
2.4. 深度學習 13
2.4.1. 深度學習介紹 13
2.4.2. 深度神經網路 14
2.4.3. 卷積神經網路 15
2.4.4. 卷積模塊介紹 16
2.4.5. 長短期記憶遞歸神經網路 17
2.5. 超參數最佳化調整 18
2.5.1. 超參數調整介紹 18
2.5.2. Keras Tuner 介紹 18
2.5.3. 貝葉斯優化(Bayesian Optimization) 19
2.5.4. Hyperband演算法 21
2.6. 遷移學習 23
2.6.1. 遷移學習介紹 24
2.6.2. 深度遷移學習介紹 25
三、 問題定義及研究 28
3.1. 問題定義 28
3.2. 標籤定義 30
四、 研究方法 31
4.1. 資料前處理 31
4.1.1. 資料切割 32
4.1.2. 格拉姆角度域處理時間序列資料 32
4.1.3. 馬可夫轉換域處理時間序列資料 33
4.2. 模型架構 36
4.3. 超參數演算法及模型優化 39
4.3.1. 超參數演算法 39
4.3.2. 提早停止 39
4.4. 遷移學習 40
五、 實驗與效能評估 41
5.1. 實驗環境 41
5.2. 實驗結果比較 42
5.3. 遷移學習資料筆數分析 48
六、 結論與未來展望 51
參考文獻 53

[1] Ulas, M., Aydur, O., Gurgenc, T., & Ozel, C. (2020). Surface roughness prediction of machined aluminum alloy with wire electrical discharge machining by different machine learning algorithms.Journal of Materials Research and Technology, 9(6), 12512-12524.
[2] Ho, K. H., Newman, S. T., Rahimifard, S., & Allen, R. D. (2004). State of the art in wire electrical discharge machining (WEDM). International Journal of Machine Tools and Manufacture, 44(12-13), 1247-1259.
[3] Esme, U., Sagbas, A., & Kahraman, F. (2009). Prediction of surface roughness in wire electrical discharge machining using design of experiments and neural networks.
[4] Kumar, A., Kumar, V., & Kumar, J. (2012). Prediction of surface roughness in wire electric discharge machining (WEDM) process based on response surface methodology. international journal of engineering and technology, 2(4), 708-719..
[5] Fan, C. L., & Jiang, J. R. (2019, October). Surface roughness prediction based on Markov chain and deep neural network for wire electrical discharge machining. In 2019 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE) (pp. 191-194). IEEE.
[6] Jiang, J. R., & Yen, C. T. (2020, September). Markov Transition Field and Convolutional Long Short-Term Memory Neural Network for Manufacturing Quality Prediction. In 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan) (pp. 1-2). IEEE.
[7] Jiang, J. R., & Cheng, Z. K. (2020, September). Product Quality Prediction with Deep Transfer Learning for Smart Factories. In 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan) (pp. 1-2). IEEE.
[8] Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017). Hyperband: A novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research, 18(1), 6765-6816.
[9] Pan, S. J., & Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359.
[10] Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., & Liu, C. (2018, October). A survey on deep transfer learning. In International conference on artificial neural networks (pp. 270-279). Springer, Cham.
[11] Convolutional layers:
https://machinelearningmastery.com/convolutional-layers-for-deep-learning-neural-networks/
[12] Keras Tuner:
https://keras-team.github.io/keras-tuner/
[13]Deep Transfer Learning:
https://zh-tw.coderbridge.com/series/d4b5a1a1565e4e7a9cd14618ffe6146f/posts/54584ea6d4c240aeb3b8ae4af3a0531a
[14]Hyperband:
https://zhuanlan.zhihu.com/p/53088201
[15]Convolution blocks:
https://towardsdatascience.com/history-of-convolutional-blocks-in-simple-code-96a7ddceac0c
[16]Bayesian :
https://www.itread01.com/content/1541298849.html
[17] Keras-Tuner:
https://github.com/keras-team/keras-tuner
[18] Fahim, M., Fraz, K., & Sillitti, A. (2020). TSI: Time series to imaging based model for detecting anomalous energy consumption in smart buildings. Information Sciences, 523, 1-13.
[19] Jiang, J. R. (2018). An improved cyber-physical systems architecture for Industry 4.0 smart factories. Advances in Mechanical Engineering, 10(6), 1687814018784192.
[20] Wang, Z., & Oates, T. (2015, January). Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In Workshops at the twenty-ninth AAAI conference on artificial intelligence (Vol. 1).
[21] CHMER - Q4025L:
http://www.chmer.com/tw/products-view.php?id=76, accessed in June 2020.
[22] Tokyo Seimitsu – Surfcom 130A:
http://www.accretech.com.cn/surfcom.html, accessed in June 2020.
[23] Surface roughness:
https://tw.misumi-ec.com/pdf/tech/MSM1/Surface_Roughness.pdf, accessed in June 2020.
[24] Qureshi, A. S., Khan, A., Zameer, A., & Usman, A. (2017). Wind power prediction using deep neural network based meta regression and transfer learning. Applied Soft Computing, 58, 742-755..
[25] Qureshi, A. S., & Khan, A. (2019). Adaptive transfer learning in deep neural networks: Wind power prediction using knowledge transfer from region to region and between different task domains. Computational Intelligence, 35(4), 1088-1112.
[26] 放電加工原理:
http://120.114.52.149/~4970H089/wiki/index.php/%E6%94%BE%E9%9B%BB%E5%8A%A0%E5%B7%A5
[27] Batch Normalization:
http://violin-tao.blogspot.com/2018/02/ml-batch-normalization.html
[28] Global Average Pooling:
https://www.cnblogs.com/hutao722/p/10008581.html
[29] LSTM:
https://daniel820710.medium.com/%E5%88%A9%E7%94%A8keras%E5%BB%BA%E6%A7%8Blstm%E6%A8%A1%E5%9E%8B-%E4%BB%A5stock-prediction-%E7%82%BA%E4%BE%8B-1-67456e0a0b
[30] 馬可夫鏈:
https://zh.wikipedia.org/wiki/%E9%A9%AC%E5%B0%94%E5%8F%AF%E5%A4%AB%E9%93%BE
[31] Duarte, D., Nex, F., Kerle, N., & Vosselman, G. (2018). Multi-resolution feature fusion for image classification of building damages with convolutional neural networks. Remote sensing, 10(10), 1636.
[32] Singh, T., Kumar, P., & Misra, J. P. (2019). Surface roughness prediction modelling for wedm of aa6063 using support vector machine technique. In Materials Science Forum (Vol. 969, pp. 607-612). Trans Tech Publications Ltd.
[33] Thankachan, T., Prakash, K. S., Malini, R., Ramu, S., Sundararaj, P., Rajandran, S., ... & Jothi, S. (2019). Prediction of surface roughness and material removal rate in wire electrical discharge machining on aluminum based alloys/composites using Taguchi coupled Grey Relational Analysis and Artificial Neural Networks. Applied Surface Science, 472, 22-35.
[34] Yusoff, Y., Zain, A. M., Sharif, S., Sallehuddin, R., & Ngadiman, M. S. (2018). Potential ANN prediction model for multiperformances WEDM on Inconel 718. Neural Computing and Applications, 30(7), 2113-2127.
[35] Phate, M. R., & Toney, S. B. (2019). Modeling and prediction of WEDM performance parameters for Al/SiCp MMC using dimensional analysis and artificial neural network. Engineering Science and Technology, an International Journal, 22(2), 468-476.
[36] Surya, V. R., Kumar, K. V., Keshavamurthy, R., Ugrasen, G., & Ravindra, H. V. (2017). Prediction of machining characteristics using artificial neural network in wire EDM of Al7075 based in-situ composite. Materials Today: Proceedings, 4(2), 203-212.
[37] Gurupavan, H. R., Devegowda, T. M., Ravindra, H. V., & Ugrasen, G. (2017). Estimation of machining performances in WEDM of aluminium based metal matrix composite material using ANN. Materials Today: Proceedings, 4(9), 10035-10038.
[38] WEDM:
https://www.researchgate.net/figure/Working-principle-of-WEDM_fig2_260107358
[39] CNN:
https://shihs.github.io/blog/machine%20learning/2019/02/25/Machine-Learning-Covolutional-Neural-Networks(CNN)/
[40]Batch Normalization:
Ioffe, S., & Szegedy, C. (2015, June). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448-456). PMLR.
[41]CBR:
Hazirbas, C., Ma, L., Domokos, C., & Cremers, D. (2016, November). Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture. In Asian conference on computer vision (pp. 213-228). Springer, Cham.
[42] Jiang, J. R., & Yen, C. T. (2021). Product Quality Prediction for Wire Electrical Discharge Machining with Markov Transition Fields and Convolutional Long Short-Term Memory Neural Networks. Applied Sciences, 11(13), 5922.