隨著晶片製成的發展，不同種類的整合封裝技術相繼被提出，例如利用矽中介板或嵌入式多晶片互聯橋的2.5D封裝技術，或是利用堆疊方式的3D封裝技術。但是這些技術都各有優缺點，例如堆疊封裝雖然縮小晶片面積但伴隨著散熱問題，或是2.5D技術難以使晶片面積縮小等。因此我們希望有一個高效率的方法能對不同的晶片設計來產生一個異質結構已提高晶片設計之效能。
在本論文中，我們首先考慮各種不同的整合技術和其優缺點、製造限制與成本。我們考慮的整合技術為上述所提到的，其這些技術所包含的載體有印刷電路板、封裝基板、矽中介板、嵌入式多晶元互聯橋和集成扇出。我們提出一個效能導向的多晶片整合方法其能夠用不同的載體結合出異質整合結構。我們提出了一個結合強化式學習模型與模擬退火的方法，該方法包含兩大部分，第一部分是訓練一個強化式學習模型來建構出一個載體結構，第二部分為利用一個模擬退火演算法對建構出來的載體結構做晶片位置的分配。當晶片位置分配結束後評估該結果效能如何並回傳給模型進行模型的更新。
我們的實驗結果說明了我們提出的方法在效能方面優於現有的方法，而在時間方面在較大晶片設計下耗時縮短6倍以上的時間。

Along with the revolution of IC design, advanced integration technologies for multiple dies have been proposed by packaging and manufacturing companies. For instance, silicon interposer and embedded silicon bridge are two kinds of 2.5D technologies to integrate multiple dies. Besides 2.5D integration, 3D integration is another advanced technology by vertically stacking dies. However, there are some pros and cons for each technology. Therefore, we aim to propose a method that integrates multiple dies with different package technologies to achieve high performance.

In this thesis, we adopt different kinds of multi-die integrations technologies in our problem. Additionally, we also consider their manufacturing constraints and costs. The carriers from these integration technologies can include printed circuit board (PCB), package substrate, interposer, and silicon bridge. We propose a performance-driven integration methodology that aims to generate a high-performance carrier structure and a die assignment with respect to the structure. This methodology combines a reinforcement learning (RL) network and a simulated annealing (SA) algorithm, by training the network to modify the carrier structure and using SA to do die assignments for the structure. The experimental results show that our methodology outperforms a previous work.

摘要
Abstract ------------------------------i
1 Introduction ------------------------1
1.1 Motivation ------------------------1
1.2 Previous Work ---------------------3
1.3 Our Contributions -----------------4
1.4 Thesis Organization ---------------4
2 Preliminaries -----------------------5
2.1 Reinforcement Learning ------------5
2.1.1 Overview ------------------------5
2.1.2 Environment ---------------------6
2.1.3 Agent ---------------------------7
2.2 Simulated Annealing ---------------8
2.3 Problem Description ---------------9
2.3.1 Area Constraint -----------------11
2.3.2 Stacking Constraint -------------12
2.3.3 Cost Constraint -----------------12
2.3.4 Objective -----------------------13
3 Our Methodology ---------------------14
3.1 Overall Flow ----------------------14
3.2 Our RL Network --------------------16
3.2.1 States --------------------------16
3.2.2 Actions -------------------------17
3.2.3 Terminal Condition --------------18
3.2.4 Reward Function -----------------18
3.3 Die Assignment --------------------19
3.3.1 Initial Die Assignment ----------19
3.3.2 Die Assignment Refinement by SA -21
3.3.3 Silicon Bridge Insertion --------24
4 Experimental Results ----------------26
4.1 Experiment Setup ------------------26
4.2 Results ---------------------------27
5 Conclusion --------------------------30
References ----------------------------31

[1] M. Matsuo, N. Hayasaka, K. Okumura, E. Hosomi, and C. Takubo, “Silicon interposer technology for high-density package,” in Electronic Components and Technology Conference, pp. 1455–1459, 2000.
[2] C.F. Tseng, C.S. Liu, C.H. Wu, and D. Yu, “Info (wafer level integrated fanout) technology,” in Electronic Components and Technology Conference, pp. 1–6, 2016.
[3] R. Mahajan, R. Sankman, N. Patel, D.W. Kim, K. Aygun, Z. Qian, Y. Mekonnen, I. Salama, S. Sharan, D. Iyengar, et al., “Embedded multi-die interconnect bridge (emib)– a high density, high bandwidth packaging interconnect,” in Electronic Components and Technology Conference, pp. 557–565, 2016.
[4] W. Zeng, A. Davoodi, and R. O. Topaloglu, “Explainable drc hotspot prediction with random forest and shap tree explainer,” in Design, Automation & Test in Europe Conference, pp. 1151–1156, 2020.
[5] M. B. Alawieh, W. Li, Y. Lin, L. Singhal, M. A. Iyer, and D. Z. Pan, “High-definition routing congestion prediction for largescale fpgas,” in Asia and South Pacific Design Automation Conference, pp. 26–31, 2020.
[6] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018.
[7] A. Goldie and A. Mirhoseini, “Placement optimization with deep reinforcement learning,” in International Symposium on Physical Design, pp. 3–7, 2020.
[8] A. Agnesina, K. Chang, and S. K. Lim, “Vlsi placement parameter optimization using deep reinforcement learning,” in International Conference on ComputerAided Design, pp. 1–9, 2020.
[9] T.Y. Weng, A Performance-Driven Integration Methodology for Multi-Die Systems. Master thesis, National Tsing Hua University, 2021.
[10] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, 2016.
[11] M. L. Puterman, Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.