我們使用深度神經網路來解決物理學、量子力學上著名的方程式–薛丁格方程式。薛丁格方程式有分為含時與不含時的形式、單粒子或多粒子問題，不同的位勢也會有不同的解，甚至能將方程式推廣到更高維度空間。問題其實是很廣泛且複雜的，因此，我們在此篇論文將針對討論的是不含時的單電子問題，並將其定義域限制在二維的盒中(2D box)。

在程式語言方面，我們使用Python以及Tensorflow套件來撰寫深度神經網路。我們將使用全連接神經網路(FCNN)和深度殘差網路(ResNet)的模型架構，去探討薛丁格方程式在兩種不同的位勢下的波函數以及其對應的能量，包含無限深位勢阱(infinite potential well)以及簡諧振子(simple harmonic oscillator)兩種位勢。此程式的特色有幾點：(1)使用非監督式學習。(2)使用3種不同方式將總損失函數做改變，並增加了1∼2個懲罰參數。(3)使用二階優化演算法—擬牛頓法的BFGS，來取代一階優化演算法—梯度下降法、Adagrad、Adam等。(4)使用兩次BFGS來提升解(波函數及其對應的能量)的精準度。(5)可訓練參數的個數少(800∼1000個參數)，相對訓練時間短。

此篇論文除了使用深度神經網路解決偏微分方程式的特徵值問題以外，最大的突破為能夠同時訓練出薛丁格方程式的波函數(即特徵向量)跟能量(即特徵值)，並且使用了Wielandt deflation的技巧讓其能夠依照能階小到大依次訓練出其解。

We use deep neural networks to solve the Schrödinger equation which is well-known in physics and quantum mechanics. The biggest breakthrough in this thesis is that we are able to train a model to get several energies and corresponding wave functions simultaneously. Together with the Wielandt deflation technique, the obtained energies are in the ascending order of energy levels. In addition, we use fully connected neural network (FCNN) and residual network (ResNet) as models to find energy levels and wave functions with the systems under two different external potentials, infinite potential well and simple harmonic oscillator. There are some features in our method: (1) It is hard to create labels for our training data, so we use unsupervised learning. (2) We change the total loss function in three different ways through adding one or two penalty parameters. (3) We use a quasi-newton method, BFGS, which is a second-order optimization algorithm instead of using first-order optimization algorithms, such as gradient descent, Adagrad and Adam. (4) In order to improve accuracy of the solution (wave function and its corresponding energy), we use BFGS twice. (5) There are just a small number of trainable parameters (800∼1000) in our models so that it takes less time to train a model.

abstract---------------i
Acknowledgement---------------iv
1 Introduction---------------1
2 Neural Network---------------3
2.1 Model---------------3
2.1.1 Fully Connected Neural Network---------------3
2.1.2 Residual Network---------------5
2.2 Optimizer---------------6
2.2.1 Newton's method---------------7
2.2.2 Quasi-Newton method (BFGS)---------------8
2.3 Initialization---------------12
2.3.1 Xavier initialization---------------12
3 Schrödinger Equation---------------15
3.1 Time-independent Schrödinger Equation---------------15
3.1.1 Infinite Potential Well---------------16
3.1.2 Simple Harmonic Oscillator---------------20
3.2 Properties---------------25
4 Methods---------------28
4.1 Training set---------------28
4.2 Neural Network Models---------------30
4.3 Loss functions---------------32
5 Results---------------40
5.1 Model 1 (Fully Connected Neural Network)---------------40
5.1.1 Infinite Potential Well---------------40
5.1.2 Simple Harmonic Oscillator---------------47
5.2 Model 2 (Residual Network)---------------53
5.2.1 Infinite Potential Well---------------53
5.2.2 Simple Harmonic Oscillator---------------56
6 Conclusion---------------58
Reference---------------60
Appendix---------------65

[1] 陳俊瑋，「卷積-遞迴神經網路計算薛丁格方程的激發態能量」，國立清華大
學計算與建模科學研究所，碩士論文，2020。
[2] https://en.wikipedia.org/wiki/Hermite_polynomials.
[3] https://en.wikipedia.org/wiki/Quantum_harmonic_oscillator.
[4] https://zh.wikipedia.org/wiki/%E7%84%A1%E9%99%90%E6%B7%B1%E6%
96%B9%E5%BD%A2%E9%98%B1.
[5] T Aboiyar, T Luga, and BV Iyorter. Derivation of continuous linear multistep methods using hermite polynomials as basis functions. American Journal of Applied Mathematics and Statistics, 3(6):220–225, 2015.
[6] Azam Asl and Michael L Overton. Analysis of the gradient method with an armijo–wolfe line search on a class of non-smooth convex functions. Optimization methods and software, 35(2):223–242, 2020.
[7] Amir Beck. First-order methods in optimization. SIAM, 2017.
[8] Adam Berger, Stephen A Della Pietra, and Vincent J Della Pietra. A maximum entropy approach to natural language processing. Computational linguistics, 22(1):39–71, 1996.
[9] Riccardo Borghi. The variational method in quantum mechanics: an elementary introduction. European Journal of Physics, 39(3):035410, 2018.
[10] Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST), 2(3):1–27, 2011.
[11] Yang Ding, Enkeleida Lushi, and Qingguo Li. Investigation of quasi-newton methods for unconstrained optimization. International Journal of Computer Application, 29:48–58, 2010.
[12] Tim Dockhorn. A discussion on solving partial differential equations using neural networks. arXiv preprint arXiv:1904.07200, 2019.
[13] Siegfried Flügge. Practical quantum mechanics. Springer Science & Business Media, 2012.
[14] Philip E Gill and Walter Murray. Quasi-newton methods for unconstrained optimization. IMA Journal of Applied Mathematics, 9(1):91–108, 1972.
[15] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
[16] David J Griffiths and Darrell F Schroeter. Introduction to quantum mechanics. Cambridge University Press, 2018.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
[18] Jan Hermann, Zeno Schätzle, and Frank Noé. Deep-neural-network solution of the electronic schrödinger equation. Nature Chemistry, 12(10):891–897, 2020.
[19] Jionghui Jiang, Xi’an Feng, Zhiwen Hu, Xiaodong Hu, Fen Liu, and Hui Huang. Medical image fusion using transfer learning and l-bfgs optimization algorithm. International Journal of Imaging Systems and Technology, 2021.
[20] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[21] Aristidis Likas and Andreas Stafylopatis. Training the random neural network using quasi-newton methods. European Journal of Operational Research, 126(2):331–339, 2000.
[22] Weibo Liu, Zidong Wang, Xiaohui Liu, Nianyin Zeng, Yurong Liu, and Fuad E Alsaadi. A survey of deep neural network architectures and their applications. Neurocomputing, 234:11–26, 2017.
[23] Lu Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. Deepxde: A deep learning library for solving differential equations. SIAM Review, 63(1):208–228, 2021.
[24] Kyle Mills, Michael Spanner, and Isaac Tamblyn. Deep learning and the schrödinger equation. Physical Review A, 96(4):042113, 2017.
[25] Nazri Mohd Nawi, Meghana R Ransing, and Rajesh S Ransing. An improved learning algorithm based on the broyden-fletcher-goldfarb-shanno (bfgs) method for back propagation neural networks. In Sixth International Conference on Intelligent Systems Design and Applications, volume 1, pages 152–157. IEEE, 2006.
[26] Jorge Nocedal and Stephen Wright. Numerical optimization. Springer Science & Business Media, 2006.
[27] A Pavelka and A Procházka. Algorithms for initialization of neural network weights. In In Proceedings of the 12th Annual Conference, MATLAB, pages 453–459, 2004.
[28] Dabal Pedamonti. Comparison of non-linear activation functions for deep neural networks on mnist classification task. arXiv preprint arXiv:1804.02763, 2018.
[29] Tong Qin, Kailiang Wu, and Dongbin Xiu. Data driven governing equations approximation using deep neural networks. Journal of Computational Physics, 395:620–635, 2019.
[30] Vijay K Rohatgi and AK Md Ehsanes Saleh. An introduction to probability and statistics. John Wiley & Sons, 2015.
[31] Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747, 2016.
[32] Youcef Saad. Numerical methods for large eigenvalue problems. Manchester University Press, 1992.
[33] Sagar Sharma and Simone Sharma. Activation functions in neural networks. Towards Data Science, 6(12):310–316, 2017.
[34] Hanif D Sherali and Osman Ulular. Conjugate gradient methods using quasi-newton updates with inexact line searches. Journal of Mathematical Analysis and Applications, 150(2):359–377, 1990.
[35] Zhen-Jun Shi and Jie Shen. A gradient-related algorithm with inexact line searches. Journal of computational and applied mathematics, 170(2):349–370, 2004.
[36] Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339–1364, 2018.
[37] Murray R Spiegel, John J Schiller, R Alu Srinivasan, and Mike LeVan. Probability and statistics, volume 2. Mcgraw-hill, 2001.
[38] Ingo Steinwart and Andreas Christmann. Support vector machines. Springer Science & Business Media, 2008.
[39] Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning. In International conference on machine learning, pages 1139–1147. PMLR, 2013.
[40] Georg Thimm and Emile Fiesler. Neural network initialization. In International Workshop on Artificial Neural Networks, pages 535–542. Springer, 1995.
[41] Nailong Wu. The maximum entropy method, volume 32. Springer Science & Business Media, 2012.
[42] Hongchao Zhang and William W Hager. A nonmonotone line search technique and its application to unconstrained optimization. SIAM journal on Optimization, 14(4):1043–1056, 2004.