量子計算由於它極大的潛力與量子電腦持續穩定地發展而獲得越來越多的關注。IBM和Google已經發表了擁有50個量子位元以上的的量子電腦架構。然而在這些機器中，物理量子位元並沒有全部被兩兩連接，所以兩個量子位元的運算只能執行在特定的一組物理量子位元上。為了要運行量子電路，須將它轉換至功能一樣且滿足特定量子電腦架構限制的電路。此轉換不可避免會添加額外的邏輯閘，而這會減少量子電路的保真度。所以設計一個演算法用最小的代價完成轉換是非常重要。轉換分成兩個步驟，分別是量子位元佈局及量子位元繞線。在這裡我們提出一個基於強化學習的模型來解量子位元佈局及問題。此問題被定義成一個序列至序列學習，且利用自注意力網路來提取電路的特徵。實驗結果顯示我們的強化學習模型生成比目前已知最好的演算法產生更好的量子位元佈局，在量子繞線的階段減少了7%額外邏輯閘的添加。

Quantum computing is gaining more and more attention due to its huge potential and the constant progress in quantum computer development. IBM and Google have released quantum architectures with more than 50 qubits. However, in these machines, the physical qubits are not fully connected so two-qubit interaction can only be performed between specific pairs of the physical qubits. To execute a quantum circuit, it is necessary to transform it into a functionally equivalent one that respects the constraints imposed by the target architecture. Quantum circuit transformation inevitably introduces additional gates which reduces the fidelity of the circuit. Therefore, it is important that the transformation method completes the transformation with minimal overheads. It consists of two steps, qubit placement and qubit routing. Here we propose a reinforcement learning-based model to solve the qubit placement problem. Qubit placement is formulated as sequenceto- sequence learning and a self-attention network is used to extract features from a circuit. The experimental results show that our RL-model generates better qubit placement than the best-known algorithms with 7% fewer additional gates in the qubit routing stage.

Abstract i
1 Introduction 1
2 Background 3
2.1 Quantum Computing 3
2.2 Previous Works 6
2.3 Reinforcement Learning 9
2.4 Problem Formulation 10
3 Proposed Approach 13
3.1 Overview 13
3.2 Neural Network Architecture 14
3.2.1 Encoder 14
3.2.2 Gate Pooling 17
3.2.3 Decoder 18
3.3 Learning Algorithm 19
4 Experimental Results 21
5 Conclusion 29
Bibliography 31

[1] X. Cheng, Z. Guan, and P. Zhu, “Nearest neighbor transformation of quantum circuits in 2d architecture,” IEEE Access, vol. 8, pp. 222466–222475, 2020.
[2] P. Zhu, Z. Guan, and X. Cheng, “A dynamic look-ahead heuristic for the qubit mapping problem of nisq computers,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 39, no. 12, pp. 4721–4735, 2020.
[3] X. Zhou, S. Li, and Y. Feng, “Quantum circuit transformation based on simulated annealing and heuristic search,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 39, no. 12, pp. 4683–4694, 2020.
[4] P. W. Shor, “Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer,” SIAM Journal on Computing, vol. 26, no. 5, pp. 1484–1509, 1997.
[5] C. Zalka, “Grover’s quantum searching algorithm is optimal,” Physical Review A, vol. 60, no. 4, pp. 2746–2751, 1999.
[6] N. Gisin, G. Ribordy, W. Tittel, and H. Zbinden, “Quantum cryptography,” Rev. Mod. Phys., vol. 74, pp. 145–195, 2002.
[7] IBM, “Ibm quantum services,” https://quantumcomputing.ibm.com/services?services=systems, 2021.
[8] A. Sinha, U. Azad, and H. Singh, “Qubit routing using graph neural network aided monte carlo tree search,” arXiv preprint arXiv:2104.01992, 2021.
[9] M. G. Pozzi, S. J. Herbert, A. Sengupta, and R. D. Mullins, “Using reinforcement learning to perform qubit routing in quantum compilers,”arXiv preprint arXiv:2007.15957, 2020.
[10] S. Sivarajah, S. Dilkes, A. Cowtan, W. Simmons, A. Edgington, and R. Duncan, “t|ket>: a retargetable compiler for NISQ devices,” Quantum Science and Technology, vol. 6, no. 1, p. 014003, 2020.
[11] A. Barenco, C. H. Bennett, R. Cleve, D. P. DiVincenzo, N. Margolus, P. Shor, T. Sleator, J. A. Smolin, and H. Weinfurter “Elementary gates for quantum computation,” Physical Review A, vol. 52, no. 5, pp. 3457–3467, 1995.
[12] M. Amy, D. Maslov, M. Mosca, and M. Roetteler, “A meet-in-the-middle algorithm for fast synthesis of depth-optimal quantum circuits,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 32, p. 818–830, jun 2013. 31
[13] K. Matsumoto and K. Amano, “Representation of quantum circuits with clifford and pi/8 gates,” arXiv preprint arXiv:0806.3834, 2008.
[14] S. Niu, A. Suau, G. Staffelbach, and A. Todri-Sanial, “A hardware-aware heuristic for the qubit mapping problem in the NISQ era,” IEEE Transactions on Quantum Engineering, vol. 1, pp. 1–14, 2020.
[15] S. Li, X. Zhou, and Y. Feng, “Qubit mapping based on subgraph isomorphism and filtered depth-limited search,” IEEE Transactions on Computers, vol. 70, no. 11, pp. 1777–1788, 2021.
[16] G. Li, Y. Ding, and Y. Xie, “Tackling the qubit mapping problem for nisq-era quantum devices,” Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, p. 1001–1014, 2019.
[17] X. Zhou, Y. Feng, and S. Li, “A monte carlo tree search framework for quantum circuit transformation,” 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1–7, 2020.
[18] H. Deng, Y. Zhang, and Q. Li, “Codar: A contextual duration-aware qubit mapping for various nisq devices,” 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1–6, 2020.
[19] C. Zhang, A. B. Hayes, L. Qiu, Y. Jin, Y. Chen, and E. Z. Zhang, “Time-optimal qubit mapping,” Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, p. 360–374, 2021.
[20] P. Murali, J. M. Baker, A. J. Abhari, F. T. Chong, and M. Martonosi, “Noise-adaptive compiler mappings for noisy intermediate-scale quantum computers,” arXiv preprint arXiv:1901.11054, 2019.
[21] B. Tan and J. Cong, “Optimal layout synthesis for quantum computing,” 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1–9, 2020.
[22] M. Y. Siraichi, V. F. d. Santos, C. Collange, and F. M. Q. Pereira, “Qubit allocation,” Proceedings of the 2018 International Symposium on Code Generation and Optimization, p. 113–125, 2018.
[23] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
[24] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
[25] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I.Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, 2016.
[26] I. Bello, H. Pham, Q. V. Le, M. Norouzi, and S. Bengio, “Neural combinatorial optimization with reinforcement learning,” arXiv preprint arXiv:1611.09940, 2016. 32
[27] A. Mirhoseini, H. Pham, Q. V. Le, B. Steiner, R. Larsen, Y. Zhou, N. Kumar, M. Norouzi, S. Bengio, and J. Dean, “Device placement optimization with reinforcement learning,” arXiv preprint arXiv:1611.09940, 2017.
[28] A. Mirhoseini, A. Goldie, M. Yazgan, J. W. Jiang, E. Songhori, S. Wang, Y.-J. Lee, E. Johnson, O. Pathak, A. Nazi, J. Pak, A. Tong, K. Srinivasa, W. Hang, E. Tuncer, Q. V. Le, J. Laudon, R. Ho, R. Carpenter, and J. Dean, “A graph placement methodology for fast chip design,” Nature, vol. 594, no. 7862, pp. 207–212, 2021.
[29] R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine Learning, vol. 8, no. 3, pp. 229–256, 1992.
[30] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in Proceedings of The 33rd International Conference on Machine Learning, vol. 48, pp. 1928–1937, 2016.
[31] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
[32] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
[33] S. J. Rennie, E. Marcheret, Y. Mroueh, J. Ross, and V. Goel, “Self-critical sequence training for image captioning,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1179–1195, 2017.
[34] D. Amodei, R. Anubhai, and E. B. et al., “Deep speech 2: End-to-end speech recognition in english and mandarin,” arXiv preprint arXiv:1512.02595, 2015.
[35] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” CoRR, vol. abs/1706.03762, 2017.
[36] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.