It is shown in Huang and Chan (2013) that the local polynomial projection approach admits an equivalent mixed model formulation and they suggest a new smoothing approach using a combination of unpenalized polynomials and penalized trigonometric (Fourier) functions. We attempt to implement penalized Fourier regression by writing some R-functions in this thesis for easy and transparent usage of the methods. Our work is based on the book by Ruppert, Wand, and Carroll (2003) and improves some of their algorithms in the settings of univariate nonparametric regression, partial linear models, additive models, and nonparametric logistic and Poisson regression. We consider two forms of penalty for penalized Fourier regression, a quadratic penalty \alpha k^{2}, where k denotes the frequency of Fourier basis functions and \alpha\geqslant0, and the penalty estimated by REML mimicking that of penalized splines (Ruppert et al., 2003). Some data examples are used to illustrate our R-functions. A small simulation study is conducted to compare the penalized Fourier to spline approaches.

1 Introduction 6
2 Background 8
2.1 Penalized Spline Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.1 A Brief Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 Linear Mixed Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Why Penalized Fourier Regression? . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3 Penalized Fourier Regression 17
3.1 Univariate Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.1 Penalty Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.3 Elements of R-function pf:fit . . . . . . . . . . . . . . . . . . . . . . . . 20
3.1.4 R-function pf:fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.2 Partial Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1 Elements of R-function pf:plm:fit . . . . . . . . . . . . . . . . . . . . . . 25
3.2.2 R-function pf:plm:fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3 Additive Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3.1 Elements of R-function pf:add:fit . . . . . . . . . . . . . . . . . . . . . . 28
3.3.2 R-function pf:add:fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4 Generalize Linear Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4.1 Generalized Linear Mixed Model . . . . . . . . . . . . . . . . . . . . . . . 31
3.4.2 Elements of R-functions pf:logistic:fit and pf:pois:fit . . . . . . . . 34
1
3.4.3 R-function pf:logistic:fit . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.4.4 R-function pf:pois:fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4 Simulation Study 38
5 Discussion 43
Tables 45
Figures 53
Appendix A: Output of Examples 1-5 59
Appendix B: R code of PFR R-functions 70
References 99

• Berndt, E. R. (1991). The Practice of Econometrics. New York: Addison-Wesley.

• Bowman, A. W., and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analysis. London: Oxford.

• Clifford, M. H., Jeffrey, S. S., and Tasi, C. L. (1998). Smoothing Parameter Selection in Nonparametric Regression Using an Improved Akaike Information Criterion. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 60, No.2, 271–293.

• Draper, N. R., and Smith, H. (1998). Applied Regression Analysis, third ed. NewYork: Wiley.

• Eilers, P. H. C., and Marx, B. D. (1996). Flexible smoothing with B-splines and penalties (with discussion). Statistical Science, 11, 89-121.

• Eubank, R. L. (1988). Spline Smoothing and Nonparametric Regression. New York: Marcel Dekker.

• Eubank, R. L. (1999). Nonparametric Regression and Spline Smoothing. New York: Marcel Dekker.

• Eubank, R. L., and Speckman, P. (1990). Curve fitting by polynomial-trigonometric regression. Biometrika, 77, 1-9.

• Fan, J., and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. London: Chapman and Hall.

• Graybill, F. (1976). Theory and Application of the Linear Model. North Scituate, Duxbury.

• Garrett, M. F., Nan, M. L., James, H. W. (2004). Applied Longitudinal Analysis. Wiley Series in Probability and Statistics.

• Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72, 320-338

• Huang, L. S. and Chen, J. (2008). Analysis of variance, coecient of determination and F
-test for local polynomial regression. The Annals of Statistics, 36, 2085-2109.

• Huang, L. S. and Chan, K. S. (2013). Local polynomial and penalized trigonometric series regression. Statistica Sinica, tentatively accepted.

• Huang, L. S., and Davidson, P. W. (2010). Analysis of variance and F
-tests for partial linear models with applications to environmental health data. Journal of the American Statistical Association, 105, 991-1004.

• Long, J. S. and Freese, J. (2006). Regression Models for Categorical Dependent Variables Using Stata, Second ed. College -Station, TX: Stata Press.

• McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models. Second ed. London: Chapman and Hall.

• O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems (with discussion). Statistical Science, 1, 505–27.

• Rao, C. R. (1973). Linear Statistical Inference and Its Applications. NewYork: Wiley.

• Robinson, G. K. (1991). That BLUP is a good thing: The estimation of random effects. Statistical Science, 6, 15–51.

• Ruppert, D., Wand, M. P., and Carroll, R. J. (2003). Semiparametric Regression Cambridge University Press.

• Sigrist, M. (Ed.) (1994). Air Monitoring by Spectroscopic Techniques (Chemical Analysis Series, vol. 197). New York: Wiley.

• Wahba, G. (1990). Spline Models for Observational Data. Philadelphia: SIAM.

• Wang, Y. (2011). Smoothing Splines. CRC Press, Boca Raton.

• Wood, S. N. (2006). Genaralized Additive Models: An Introduction with R. CRC Press, Boca Raton.