人臉特徵點偵測通常會受到各種環境的影響，例如人臉姿勢的不同以及光影變化。我們觀察到人臉姿勢變化是一項影響人臉特徵點偵測準確率的一大因素。為了解決人臉姿勢對偵測準確率的影響，我們利用深度學習的方式去學習一個良好的迴歸器，並且提出了基於姿勢感知的卷積網路來解決姿勢變化的問題。我們首先提出了一個基於卷積網路的分類器來對人臉影像做姿勢的分類，之後我們提出了兩個卷積網路分別對應人臉的不同姿勢來偵測人臉特徵點。此外，我們利用了輪廓的限制來修改修正層。實驗結果驗證了姿勢感知的偵測器可以比原來的偵測器達到更好的效果。

Facial landmark detection usually suffers from the influence by the change of environment, such as pose variation and illumination. We observe that high pose variation is the one most influence the detection accuracy. To tackle the problem of pose variation, we adopt deep learning approach to learn a good regressor and propose a pose-aware CNN to tackle the pose variation. We first develop CNN classifier to classify facial image according to the pose. Next, we develop two CNN to detect the facial landmarks according to the corresponding pose. In addition, we adjust the refinement level by concluding the shape constraint. Our experimental results show that the pose-aware detector performs better than the original landmark detector.

中文摘要 1
Abstract 2
1. Introduction 4
2. Related work 6
2.1. Model-based approach 6
2.2. Detector-based approach 8
2.3. Regression-based approach 10
2.4. Deep learning-based approach 11
3. Proposed method 14
3.1. Preliminary work and discussion 14
3.2. Motivation 16
3.3. Pose-Aware Deep Convolutional Neural Networks 17
3.3.1. Structure 17
3.3.2. CNN for pose classification 18
3.3.3. Pose-aware CNN for facial landmark detection 19
3.3.4. CNN for refinement 22
3.4. Implementation detail 24
3.4.1. The structure in convolutional neural network 25
3.4.2. CNN training 26
4. Experimental results 30
4.1. Experimental setting 30
4.2. Pose classification result 31
4.3. Facial landmark detection result 31
5. Conclusions 36
6. Reference 37

[1] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, ”Active shape models their training and application,” Computer Vision and Image Understanding, vol. 61, no.1, pp.38-59, Jan. 1995.
[2] T. F. Cootes, G. J. Edwards, and C. J. Taylor, ”Active appearance models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, Jun. 2001.
[3] P. Sauer, T. F. Cootes, and C. J. Taylor, ”Accurate regression procedures for active appearance models,” In Proc. BMVC, 2011.
[4] P. A. Tresadern, P. Sauer, and T. F. Cootes, “Additive updatepredictors in active appearance models,” In Proc. BMVC, 2010.
[5] P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman, and N. Kumar, “Localizing parts of faces using a consensus of exemplars,” In Proc. CVPR, 2011.
[6] L. Gu and T. Kanade, “A generative shape regularization model for robust face alignment,” In Proc. ECCV, 2008.
[7] S. Milborrow and F. Nicolls, “Locating facial features with an extended active shape model,” In Proc. ECCV, 2008.
[8] X. Zhu and D. Ramanan, “Face detection, pose estimation, and landmark localization in the wild,” In Proc. CVPR, 2012.
[9] L. Liang, R. Xiao, F. Wen, and J. Sun, “Face alignment via component-based discriminative search,” In Proc. ECCV, 2008.
[10] Junjie Yan, Zhen Lei, Dong Yi, and Stan Z. Li, “Learn to Combine Multiple Hypotheses for Accurate Face Alignment,” In Proc. ICCVW, 2013.
[11] X. Cao, Y. Wei, F. Wen, and J. Sun, “Face alignment by explicit shape regression,” In Proc. CVPR, 2012.
[12] Shaoqing Ren, Xudong Cao, Yichen Wei, and Jian Sun, “Face Alignment at 3000 FPS via Regressing Local Binary Features,” In Proc. CVPR, 2014.
[13] Yi Sun, Xiaogang Wang, and Xiaoou Tang, “Deep Convolutional Network Cascade for Facial Point Detection,” In Proc. CVPR, 2013.
[14] Erjin Zhou , Haoqiang Fan, Zhimin Cao, Yuning Jiang, and Qi Yin, “Extensive Facial Landmark Localization with Coarse-to-fine Convolutional Network Cascade,” In Proc. ICCVW, 2013.
[15] Alexander Toshev and Christian Szegedy, “DeepPose: Human Pose Estimation via Deep Neural Networks,” In Proc. CVPR, 2014.
[16] Ning Zhang, Manohar Paluri, Marc’Aurelio Ranzato, Trevor Darrell and Lubomir Bourdev, ”PANDA: Pose Aligned Networks for Deep Attribute Modeling,” In Proc. CVPR, 2014.
[17] Dumitru Erhan, Christian Szegedy, Alexander Toshev and Dragomir Anguelov, ”Scalable Object Detection using Deep Neural Networks,” In Proc. CVPR, 2014.
[18] Y. LeCun, L. Bottou, G. Orr, and K. Muller, “Efficient backprop,” In G. Orr and M. K., editors, Neural Networks: Tricks of the trade. Springer, 1998.
[19] Hinton, G. E., Osindero, S., and Teh, Y., ”A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, No. 7, pp.1527–1554, 2006.
[20] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, No. 11, pp. 2278–2324, 1998.
[21] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller, “Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments,” University of Massachusetts, Amherst, Technical Report 07-49, October, 2007.