統計學習是一門結合統計、最佳化、泛函分析使用資料來預測未來的學問。它已被應用在工業工程及電腦科學等領域。線性規劃是有著線性目標式及線性等式或不等式之數學規劃。但是鮮少有研究估計線性規劃的參數。因此，我們基於統計學習的框架建立了LP機的學習機器且證明解存在及得到KKT條件。進一步，我們將複雜度的概念融入線性規劃機當中並得到線性規劃機的學習邊界。

Statistical learning is a subject that integrates statistics, optimization and functional analysis techniques for using data to predict the future. This subject can be applied in many fields such as industrial engineering and computer science. Linear programming (LP) is a mathematical program design with linear objective function subject to linear equality and inequality constraints. However, studies rarely estimate the LP parameter. We develop a learning machine, namely, LP machine, based on the framework of statistical learning to estimate the unknown LP parameter. We verify the existence of a solution, and derive the Karush-Kuhn-Tucker condition. Furthermore, we incorporate the concept of complexity into the LP machine and investigate its learning bound.

1 Introduction 1
1.1 Background..........................................1
1.2 Motivation..........................................5
1.3 Organization of thesis..............................5
2 Literature review 6
2.1 LP..................................................6
2.1.1 Introduction..................................6
2.1.2 Duality.......................................7
2.1.3 Optimality condition..........................8
2.1.4 Sensitivity analysis..........................9
2.1.5 Parametric programming.......................10
2.2 Parameter estimation...............................12
2.2.1 Inverse problem..............................12
2.2.2 Ill-posed problem............................13
2.2.3 Modeling of physical system..................13
2.2.4 Inverse optimization.........................14
2.2.5 Optimal learning.............................14
2.3 Statistical learning...............................16
2.3.1 Classical model..............................16
2.3.2 Learnability.................................18
2.4 Summary............................................19
3 LP Machine 21
3.1 Problem statement..................................21
3.1.1 Framework of LP machine......................21
3.1.2 Assumptions of the LP machine................21
3.2 Modeling...........................................22
3.2.1 Notations....................................22
3.2.2 Modeling of LP machine.......................23
3.3 Training model.....................................24
3.3.1 Training model with parameter assumption.....25
3.3.2 Training model as quadratic programming......26
3.4 Existence of solutions.............................27
3.4.1 Learnability issue...........................27
3.4.2 Existence of optimal solution................29
3.4.3 KKT condition................................31
3.4.4 Geometric interpretation.....................33
4 Theoretical results of the LP machine 35
4.1 Issue of hypothesis set35
4.1.1 Growth function of LP machine................35
4.1.2 Minimum requirement of observations..........36
4.2 Complexity of LP machine...........................37
4.2.1 Leaning bound of LP machine..................37
4.3 Rademacher complexity bound of the LP machine......39
4.4 Sensitivity analysis of the LP machine.............40
5 Conclusions 42
5.1 Contributions......................................42
5.2 Limitation of research.............................43
5.3 Future direction...................................43
Reference 44

[1] Ahuja, R. K., Orlin, J. B. (2001). "Inverse Optimization ". Operations Research 49, 771-783.
[2] Apostol, T. (1974). "Mathematical Analysis". Pearson Education, New York.
[3] Aster, R., Borchers, B., Thurber, C. (2004)."Parameter Estimation and Inverse Problems". Academic Press.
[4] Bazaraa, M. S., Jarvis, J. J. Sherali, H. D. (2010)."Linear Programming and Network Flows". Wiley-Intersciences, New York.
[5] Bersimas, D., Tsitsiklis, J. N., (1997). "Introduction to Linear Optimization".Athena Scientic.
[6] Bersimas, D., Gupta, V., Paschalids, I. C., (2015). "Data-driven estimation in equalibrium using inverse opimization". Mathematical Programming, 153, 595-633
[7] Burton, D., Toint, P. L. (1992). "On an instance of the inverse shortest paths". Mathematical Programming, 53, 45-61.
[8] Chan, T. C., Craig, T., Lee, T., Sharpe, M. B. (2014). "Generalized inverse multiibjective optimization with application to cancer therapy". Operations Research, 62(3), 680-695.
[9] Chen, W., Tabak, E. (2017). "A data-driven linear programming methodology for optimal transport". manuscript.
[10] Danzig, G. B., Thapa, M. N. (1997). "Linear Programming: Introduction". Springer-Verlag, New York.
[11] Danzig, G. B., Thapa, M. N. (1997). "Linear Programming: Theory and Extensions". Springer-Verlag, New York.
[12] Gal, T. (1995). "Postoptimal Analyses, Parametric Programming, and Related Topics: degeneracy, multicriteria decision making redundacy". Berlin.
[13] Greeberg, H. J., Holder, A. G., Roos, K., Terlaky, T. (1998). "On the dimension of the set of rim perturbations for optimal partition invariance". SIAM Optimization, 9(1), 207-216
[14] Greenberg, H. (1999). "Matrix sensitivity analysis from interior solution of a linear program". INFORMS Journal on Computing, 11(3), 316-327
[15] Gyor, L., Kohler, M., Krz_zak, A., Walk., H. (2002). "A Distribution-Free Theory of Nonoparametric Regression". Springer-Verlag, New York.
[16] Greenberg, H. (2000). "Simutaneous primal-dual right-hand-side sensitivity from a strictly complementarity solution of a linear program". INFORMS Journal on Computing, 11(3), 316-327
[17] Hadigheh, A. G., Terlaky, T. (2006). "Sensitivity analysis in linear optimization: invariant support set intervals". European Journal of Operations Research, 169, 303-315.
[18] Hasatie, T., Tibshirani, R., Friedman, J. (2009). "The Element of Statistical Learning". Springer-Verlag, New York.
[19] Hillier, F. S., Liberman, G. J. (2015). \Introduction to Operations Research". McGraw-Hill, New York.
[20] Koltai, T., Terlaky, T. (2000). "The dierence between the managerial andmathematical interpretation of sensitivity analysis results in linear programming". International Journal of Production Economics, 65, 257-274.
[21] Mohri, M., Rostamizadeh, A., Talwalkar, A. (2012). "Foundation of Machine Learning". The MIT Press, Cambridge.
[22] Mostafa, Y. S., Magdon-Ismail, M., Lin, Hsuan-Tien. (2012). "Learning from Data-A Short Course". AML Book, New York.
[23] Powell, W. B., Ryzhov, Ilya O. (2012)."Optimal Learning". Wiley-Intersciences, New York.
[24] Powell, W. B., Ryzhov, Ilya O. (2012)."Information Collection for linear program with uncertain objective coecient ". SIAM 22(4), 1344-1368.
[25] Rumelhart, D. E., Hinton, G. E., Williams ,R.J. (1986). "Learning internal representations by error propagation. Parallel distributed processing: Explorations in macrostructure of cognition". Vol.I, Badford Books, Cambridge, MA., 318-362.
[26] Rosenblatt, F. (1962). \Principles of Nerodynamics: Percetron and the The-
ory of Brain Mechanics". Spartan Books, Washington.
[27] Rudin, W. (1976). "Principles of Mathemtical Analysis". McGraw-Hill, New York.
[28] Spall, J. (2003). "Introduction to Stochastic Search and Optimization". Wiley, New York.
[29] Tarantola, A. (2005). "Inverse Problem Theory and Methods for Model Parameter Estimation". SIAM
[30] Vapnik, V. (1998). "Statistical Learning Theory". Wiley, New York.
[31] Vapnik, V. (2000). "The Nature of Statistical Learning". Wiley, New York.
[32] Vapnik, V., Chervonenkis, A. (1974). "Theory of Pattern Recognotion(in Russia)". Nauka, Moscow.
[33] Valiant, L., G. (1984) "A theory of the learnable". ACM 27, 1049-1056.
[34] Wang, L. (2009). "Cutting plane algorithms for the inverse mixed integer linear prgramming problem.". Operations Research 37(2), 114-116.