本文探討如何利用遷移學習建模，解決工業上常見數據量不足的問題，採用的機器學習模型為高斯過程，藉由修改高斯過程中的核矩陣(kernel matrix)使原本只適用於單個任務的模型可以延伸至多個任務的處理，修改過後的多任務高斯過程可以將多個任務之間數據點的資訊量藉由模型參數訓練的過程自動決定各個任務之間需要共享的量是多少。
在化學工業製程中我們常需要讓各項變數維持在一個我們所希望的範圍當中，如何能準確的預測變數趨勢在化工廠的控制中相當重要，隨著製程越來越複雜，各種單元的變數不僅個數多，彼此間的關係也漸趨複雜，在實際的工廠數據中往往難以清晰地指出各種變數間的物理關係，而數據驅動模型正好可以在不明白變數間複雜關係的情況下，利用機器學習演算法去提取工業上龐大數據中隱含的關係，並輔助建立製程預測的軟儀表模型，有了基於數據所產生的軟儀表模型再進一步搭配製程物理理論所推論而得的模型，期望能準確地預測出製程的動態行為。
在化學工業中因為同一種化學品會依照不同的品質標準而有不同的應用，像是高分子化學品依照高分子鏈結的程度有不同的使用方式，所以多規格的製程生產方式非常常見，所謂多規格製程就是同一個製程生產線在不同的產品規格中做切換，不同規格有不同的製程參數設定，雖然製程參數設定不同但是基本上用的是同一個化工單元，製程變數的種類以及個數也都相同，通常多規格製程的特色是大部分規格中的數據量都很少，因為這些規格的數據有相同的製程資訊，若單獨建模會因為數據量缺少使模型預測效果不好，所以我們希望設計一個模型可以同時對各個規格的數據建模，這個多任務模型有機制可以使各個任務之間的數據分享訊息，我們期望藉由多任務的模型從各個任務中提取出相似的製程資訊以補足單個任務製程資訊缺少的問題。
高斯過程是一種基於貝葉斯定理(Bayes’ theorem)的機率模型，它的好處是在假設數據機率分布的前提下可以建立出目標數據(target data)可能的數據機率分布，而非單純訓練出一個預測的目標數據，因為結果是一個機率分布，我們可以藉由信賴區間分析出各數據有可能出現的範圍，而且相較於深度學習的神經網絡需要大量的參數訓練，高斯過程的訓練參數遠小於神經網絡因此不需要大量疊代訓練，高斯過程另一項特色在於核函數(kernel function)的選擇，選定核函數的形式就如同選定模型會有怎麼樣的表現形式，核函數通常依據各個任務數據的特性來做選擇。
本次使用兩種化工數據，第一種是螺桿元件數據，分析不同元件排列組合的結果，第二部分使用多規格CSTR數據，藉由上述兩種化工製程的數據集來探討兩種在多規格製程上常見的問題，第一種是當某些任務有充足數據的情況下是否可以藉由多任務模型將充足的製程資訊遷移至其他少量數據的任務中，第二種則是當各個具有關聯的製程之間數據量偏少的情況下是否可以藉由多任務模型將任務之間數據的資訊區分出哪些是互相關聯而哪些又是不相關，共享相關聯的資訊以彌補單個任務資訊量偏少使模型沒辦法有效訓練參數的缺點來解決化工製程小量數據建模的問題。

This thesis discusses how to use transfer learning to solve the problem of insufficient data in process modeling. The machine learning model we adopt herein is Gaussian process. By modifying the kernel matrix in the conventional Gaussian process, the model original only applicable to a single task can be extended to the processing of multiple tasks. The modified multi-task Gaussian process can automatically determine the amount of information to share between tasks by parameter training.
In chemical industry processes, we often need to keep process variables in control. How to predict variable trends accurately is an important issue . As the process becomes more complex, the number of operation units is larger, while the relation between units become difficult to handle. In this situation, the data-driven methods that use machine learning algorithms to extract the relation between huge data without deep understanding of the physical relationship between variables can assist the establishment of the prediction models.
In the chemical industry, the products have different applications according to the different quality standards. Taking polymers as an example, there are different grades in a polymer manufacturing process according to the degree of polymerization. As a result, multi-grade processes are common in industry, which often switch the settings of process variables during operating. Although the settings are often changed, the same operation unit is used, while the process variables to manipulated or recorded are also same.
The characteristic of multi-grade process is that the amount of data collected in some grades is often small and the process information is unbalanced between different grades. Therefore, we want to design a model which can model different grades at the same time by sharing information between them.
Gaussian process is a probabilistic model based on the Bayes’ theorem which has the advantage of establishing a probability distribution of the predicted target. Gaussain process models allow to choose the kernel functions,providing a significant flexibility in different modeling tasks.
The proposed method was illustrated with two different datasets. The first one is about a simulated twin-screw extruder and the second one contain the data of a multi-grade CSTR process. Based on the results of the case studies,the following two issues were discussed. The first issue to discuss is whether sufficient information of auxiliary task data can be migrated to assist the modeling of the target task which only contains a small amount of data. The second problem to answer is whether the multi-task model can identifyand quantify the similarity between tasks.

第一章 緒論---------------------1
1.1前言------------------------1
1.2 文章架構--------------------3
第二章 多任務高斯過程小數據建模---4
2.1 文獻回顧與研究動機-----------4
2.2 研究方法:高斯過程回歸--------6
2.2.1 權重參數空間角度-----------6
2.2.2 函數空間角度---------------9
2.3 多任務高斯過程---------------10
第三章 案例分析與討論-------------14
3.1 螺桿元件數據-----------------14
3.2 螺桿元件數據建模--------------15
3.3 螺桿元件數據結果討論-----------16
3.4 多規格CSTR數據----------------35
3.5 CSTR數據建模-----------------35
3.6 CSTR數據結果討論--------------36
第四章 結論-----------------------54
第五章 參考文獻------------------55

[1] Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)
[2] W. Yan, D. Tang, Y. Lin. A data-driven soft sensor modeling method based on deep learning and its application. IEEE Transactions on Industrial Electronics. DOI: 10.1109/TIE.2016.2622668. (2016)
[3] Huang, Z. & Pan, Z. & Lei, B. Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data. Remote Sens. (2017)
[4] E. Bonilla, K. M. Chai, & C. Williams. Multi-task gaussian process prediction. Proceedings of the 20th Annual Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, pp. 153–160. (2008)
[5] Evgeniou, T., Pontil, M. Regularized multi-task learning. Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, Washington, USA 109–117 (2004)
[6] Bakker, B., Heskes, T. Task clustering and gating for Bayesian multitask learning. Journal of Machine Learning Research (2003)
[7] Xue, Y., Liao, X., Carin, L., Krishnapuram, B. Multi-task learning for classification with Dirichlet process priors. Journal of Machine Learning Research 8 (2007)
[8] S.W. Lee, K.L. McCarthy.Effect of screw configuration and speed on RTD and expansion of rice extrudate. Journal of Food Process Engineering. (1996)
[9] Yu, K., Tresp, V., Schwaighofer, A. Learning Gaussian processes from multiple tasks. Proceedings of the Twenty-Second International Conference on Machine Learning, Bonn, Germany (2005)
[10] Edwin V. Bonilla, Felix V. Agakov, and Christopher K. I. Williams. Kernel Multi-task Learning using Task-specific Features. In Proceedings of the 11th AISTATS, March (2007)
[11] C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massachusetts (2006)
[12] C. K. I. Williams, K. M. A. Chai, and E. V. Bonilla. A note on noise-free Gaussian process prediction with separable covariance functions and grid designs. Technical report, University of Edinburgh, (2007)