最佳化技術向來是最佳化學者琢磨之標的，然而，隨著大數據時代的來臨，最佳化問題本身之決策變數、限制式的總量亦跟著數據量遽增而水漲船高，這樣的現象在在顯示出，如若期望大數據最佳化問題能被有效率地處理，則最佳化技術需藉由平行運算技術來改寫，使最佳化演算法得於多核心或叢集式環境進行分散式運算。又因近年來大型LPCC問題開始在各種如經濟、機器學習領域頻繁出現，故本研究以SQP作為LPCC之主要求解演算法，並導入Benders切割概念，以SQP本身之性質作為切割條件，於求解最佳化過程中將嚴重影響可行解區域之限制式逐條地加入子問題之中，使大數據問題能以小規模問題的樣態求解。於最佳化演算法重新建構後，本研究將演算法以資料平行化之方式進行分散式運算以期達到良好的加速結果。最後，經過數值結果的佐證與商業軟體的比較，我們認為本次提出的平行演算法確實能夠有效地處理具有巨量額外限制式的大規模LPCC問題。

The development of optimization technology has long been a popular research subject. However, with the expansion of the amount of data, the optimization algorithm is becoming more and more important for efficiently loading and solving large parameters. In order to deal with this tendency, this research aims to tackle linear programs with linear complementarity constraints problems (LPCC) with large scale side constraints, and developed an algorithm which combined sequential quadratic programming (SQP) with Bender’s generation cut to obtain the local optimal solutions of LPCC. Eventually, we parallel the algorithm mentioned above by message passing interface (MPI) framework in multiple clustering systems and evaluate the runtime of our algorithm against the prevailing commercial optimization software. From the numerical results, we consider this algorithm, no matter parallel or not, has a significant performance on solving LPCC problems with massive side constraints.

Content
Chapter 1 Introduction 1
Chapter 2 Literature review 3
2.1 A linear program with linear complementarity constraints 3
2.2 Sequential quadratic programming 4
2.3 Bender’s decomposition method 6
2.4 Parallel framework 8
2.5 Parallel efficiency 9
Chapter 3 method 12
3.1 Produce LPCC in parallel 12
3.2 Parallel SQP with generation cut 13
3.2.1 The matrix shift 13
3.2.2 The parameters computation for SQP 15
3.2.3 Elastic SQP mode construction 17
3.2.4 Parallel framework of the algorithm 18
3.2.5 Bender’s generation cut for SQP 20
3.2.6 Parallel algorithm 21
Chapter 4 Result 23
4.1 The environment of experiments 23
4.2 The setting of experiments 24
4.3 The design of experiments 24
4.4 The analysis of the experiments 31
4.5 The efficiency of the result 34
Chapter 5 Conclusion 38
Reference 40

References

Bertsekas, D. P. (2016). Nonlinear Programming 3rd Edition. U.S.A.: Athena Scientific.
Chang, J. F., Chu, S. C., Roddick, J. F., & Pan, J. S. (2005). Parallel Particle Swarm Optimization Algorithm with Communication Strategies. Journal of Information Science and Engineering, 21, pp. 809-818.
Chen, X. (1995). A Parallel BFGS-SQP Method for Stochastic Linear. Computational Techniques and Applications (pp. 1-8). World Scientific.
Chen, X., & Womersley, R. S. (1995). Random test problems and parallel methods for quadratic programs and quadratic stochastic programs. School of Mathematics, UNSW.
Dixon, L. C., & Jha, M. (1993). Parallel algorithms for global optimization. Journal of Optimization Theory and Applications, 72(2), pp. 385-395.
Fang, H. R., Leyffer, S., & Munson, T. S. (2009). A Pivoting Algorithm for Linear Programs with Complementarity Constraints. U.S.A: ARGONNE NATIONAL LABORATORY.
Fletcher, R., & Leyffer, S. (2004). Solving mathematical programs with complementarity constraints as nonlinear programs. Optimization Methods and Software, 19(1), pp. 15-40.
Fletcher, R., Leyffer, S., Ralph, D., & Scholtes, S. (2006). Local convergence of sqp methods for mathematical programs with equilibrium constraints. Society for Industrial and Applied Mathematics, 17(1), pp. 259-286.
Giannessi, F., & Tomasin, E. (1973). Non convex quadratic programs, linear complementarity problems, and integer linear programs. Conti, R., Ruberti, A. (eds.) Fifth Conference on Optimization Techniques. 3, pp. 437-449. Berlin: Springer.
Gill, P. E., Murray, W., & Saunders, M. A. (2005). SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization. Society for Industrial and Applied Mathematics, 47(1), pp. 99-131.
Gill, P. E., Saunders, M. A., & Wong, E. (n.d.). On the performance of sqp methods for nonlinear optimization.
Gockenbach, M. S. (n.d.). Introduction to sequential quadratic programming.
Goldsmith, M. J. (1999, 3). Seqential quardratic programming methods based on indefinite hessian approximations. P.hD thesis. Standford University.
Hu, J., Mitchell, J. E., & Pang, J. S. (2008). An LPCC approach to nonconvex quadratic programs. Mathematical Programming.
Hu, J., Mitchell, J. E., Pang, J. S., & Yu, B. (2012). On linear programs with linear complementarity constraints. J Glob Optim, 53, pp. 29-51.
Hu, J., Mitchell, J. E., Pang, J. S., Bennett, K. P., & Kunapili, G. (2008). On the global solution of linear programs with linear complementarity constraints. Society for Industrial and Applied Mathematics, 17(1), pp. 445-471.
Huang, Y., & Zheng, Q. P. (2012). Benders Decomposition. Department of Industrial and Management Systems Engineering West Virginia University.
Kang, S. J., Lee, S. Y., & Lee, K. M. (2015). Performance Comparison of OpenMP, MPI, and MapReduce in Practical Problems. Advances in Multimedia, pp. 1-9.
Karau, H., Konwinski, A., Wendell, P., & Zaharia, M. (2015). Learning Spark. U.S.A: O'Reilly Media.
Klabjan, D., Johnson, E. L., & Nemhauser, G. L. (2000). A Parallel Primal-Dual Simplex Algorithm. School of Industrial and Systems Engineering (GA 30332-0205). Georgia Institute of Technology.
Kulkarni, A. A., & Shanbhag, U. V. (2012). Recourse-based stochastic nonlinear programming: properties and Benders-SQP algorithms. Comput Optim Appl, 51, pp. 77-123.
Kunapuli, G., Bennett, K. P., Hu, J., & Pang, J. S. (2007). Bilevel Model Selection for Support Vector Machines.
Lee, Y. C., Pang, J. S., & Mitchell, J. E. (2015). An algorithm for global solution to bi-parametric linear complementarity constrained linear programs. Journal of Global Optimization, 62, pp. 263-297.
Migdalas, A., Toraldo, G., & Kumar, V. (2003). Nonlinear optimization and parallel computing. Parallel Computing., 29, pp. 375-391.
Monteiroa, M. T., & Rodrigues, H. S. (2011). Combining the regularization strategy and the SQP to solve MPCC — A MATLAB implementation. Journal of Computational and Applied Mathematics, 235, pp. 5348-5356.
Nocedal, J., & Wright, S. J. (2000). Numerical Optimization second edition. U.S.A: Springer.
Pacheco, P. (2011). An Introduction to Parallel Programming. Morgan Kaufmann.
Parsopoulos, K. E., & Vrahatis, M. N. (2002). Particle Swarm Optimization Method for Constrained Optimization Problems. Greece: Department of Mathematics, University of Patras Articial Intelligence Research Center.
Phua, P., & Ming, D. (2003). Parallel Nonlinear Optimization Techniques for Training Neural Networks. IEEE TRANSACTIONS ON NEURAL NETWORKS., 14(6), pp. 1460-1468.
Schittkowski, K. (n.d.). A New Fortran Implementation of a Sequential Quadratic Programming Algorithm for Parallel Computing.
Ta¸skın, Z. C. (n.d.). Benders Decomposition. Wiley Encyclopedia of Operations Research and Management Science., 1-16.
Yu, B. (2011). A Branch and Cut Approach to Linear Programs with Linear Complementarity Constraints. Ph.D. thesis. Rensselaer Polytechnic Institute.
Zaharia, M., Chowdhury, M., Franklin, M., Shenker, S., & Stoica, I. (2011). Spark: Cluster Computing with Working Sets. University of California, Berkeley.