人臉反偽造對於依賴偵測臉部真實性的應用程式至關重要。最近輔助資訊如臉部深度圖和 rPPG 訊號，已被成功地應用以提高臉部部反偽造的性能。因此，輔助資訊的預測品質是分類真實/偽造有效性的關鍵。在本文中，我們關注輔助訊息之預測的魯棒性和潛在特徵的可辨別性。我們提出在對抗性學習框架中估計輔助資訊以真實/偽造分類器的訓練。我們在對比損失中加入了額外的約束，並提出了一個區別性的批量對比損失來學習潛在特徵。輔助資訊和區別性潛在特徵都用於分類真實/欺騙。此外，由於並非所有輔助監督都同樣可靠，因此我們提出了一種適應融合策略來融合來自不同輔助監督分支的預設結果。在數個基準資料集上的實驗結果表明，提出的方法明顯優於以前的方法。

Face anti-spoofing is critical to applications which heavily rely on the authenticity of detected faces. Recently, auxiliary information, such as facial depth maps and rPPG signals, have been successfully included to boost the performance of face anti-spoofing. Consequently, the quality of auxiliary estimation is key to the effectiveness of live/spoof classification. In this thesis, we focus on the robustness of auxiliary estimation and the discriminability of latent features. We propose to estimate the auxiliary information along with the training of live/spoof classifier in an adversarial learning framework. We include additional constraints in the contrastive loss and propose a discriminative batch-contrastive loss to learn the latent features. Both the auxiliary information and the discriminative latent features are included into the live/spoof classification. In addition, because not all the auxiliary supervisions are equally reliable, we propose an adaptive fusing strategy to fuse the estimation results from different auxiliary-supervised branches. Experimental results on several benchmark datasets show that the proposed method significantly outperforms previous methods.

摘要 i
Abstract ii
Acknowledgements
1. Introduction 1
2. Related Work 4
2.1 Texture­based Methods 4
2.2 Depth­based Methods 5
2.3 Temporal­based Methods 5
3. Method 7
3.1 Auxiliary Information 8
3.1.1 Facial Depth Map 9
3.1.2 Facial rPPG Map 9
3.2 Network Architecture of Single Branch 10
3.3 Auxiliary Estimation and Discriminative Feature Learning 10
3.3.1 Auxiliary Estimation 11
3.3.2 Adversarial Training 11
3.3.3 Discriminative Batch­Contrastive Loss 12
3.3.4 Total Loss 13
3.4 Live/Spoof Classification 14
3.4.1 Single Branch Detection 14
3.4.2 Detection by Adaptive Fusion 15
4. Experiments 17
4.1 Overview 17
4.2 Dataset 17
4.2.1 OULU­NPU 17
4.2.2 SiW 18
4.2.3 CASIA­MFSD 18
4.2.4 Replay­Attack 18
4.3 Evaluation metrics 19
4.4 Implementation Details 19
4.5 Network Structure 19
4.6 Ablation Study 22
4.6.1 Comparison between Different Contrastive Loss Terms 22
4.6.2 Comparison between Different Losses and Detection Scores 23
4.6.3 Adaptive Fusion Strategy 24
4.6.4 Intra Testing and Cross Testing 25
5. Conclusion 29
References 30

[1] L. Van der Maaten and G. Hinton, “Visualizing data using t­sne.,” Journal of machine learning research, vol. 9, no. 11, 2008.
[2] X. Yang, W. Luo, L. Bao, Y. Gao, D. Gong, S. Zheng, Z. Li, and W. Liu, “Face anti­-spoofing: Model matters, so does data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 3507–3516, 2019.
[3] Y. Liu, A. Jourabloo, and X. Liu, “Learning deep models for face anti­-spoofing: Binary or auxiliary supervision,” in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 389–398, 2018.
[4] T. de Freitas Pereira, A. Anjos, J. M. De Martino, and S. Marcel, “Can face anti­-spoofing countermeasures work in a real world scenario?,” in 2013 international conference on biometrics (ICB) , pp. 1–8, IEEE, 2013.
[5] Z. Boulkenafet, J. Komulainen, and A. Hadid, “Face anti­-spoofing based on color texture analysis,” in 2015 IEEE international conference on image processing (ICIP), pp. 2636–2640, IEEE, 2015.
[6] Z. Boulkenafet, J. Komulainen, and A. Hadid, “Face spoofing detection using colour tex­ture analysis,” IEEE Transactions on Information Forensics and Security, vol. 11, no. 8, pp. 1818–1830, 2016.
[7] A. Jourabloo, Y. Liu, and X. Liu, “Face de­-spoofing: Anti­-spoofing via noise modeling,” in Proceedings of the European Conference on Computer Vision (ECCV), pp. 290–306, 2018.
[8] Z. Wang, Z. Yu, C. Zhao, X. Zhu, Y. Qin, Q. Zhou, F. Zhou, and Z. Lei, “Deep spatial gra­dient and temporal depth learning for face anti­-spoofing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5042–5051, 2020.
[9] S.­Q. Liu, X. Lan, and P. C. Yuen, “Remote photoplethysmography correspondence fea­ture for 3d mask face presentation attack detection,” in Proceedings of the European Con­ference on Computer Vision (ECCV), September 2018.
[10] U. A. Ciftci, I. Demir, and L. Yin, “Fakecatcher: Detection of synthetic portrait videos us­ing biological signals,” IEEE Transactions on Pattern Analysis and Machine Intelligence, p. 1–1, 2020.
[11] Y. Atoum, Y. Liu, A. Jourabloo, and X. Liu, “Face anti­-spoofing using patch and depth­ based cnns,” in 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 319–328, 2017.
[12] B. Lin, X. Li, Z. Yu, and G. Zhao, “Face liveness detection by rppg features and contextual patch-­based cnn,” in Proceedings of the 2019 3rd International Conference on Biometric Engineering and Applications, pp. 61–68, 2019.
[13] Bharadwaj, Dhamecha, Vatsa, and Singh, “Computationally efficient face spoofing de­tection with motion magnification,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 105–110, 2013.
[14] J. Komulainen, A. Hadid, and M. Pietikäinen, “Context based face anti­-spoofing,” in 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8, 2013.
[15] K. Patel, H. Han, and A. K. Jain, “Secure face unlock: Spoof detection on smartphones,” IEEE Transactions on Information Forensics and Security, vol. 11, no. 10, pp. 2268–2283, 2016.
[16] K. Patel, H. Han, and A. K. Jain, “Cross-­database face anti-spoofing with robust feature representation,” in Chinese Conference on Biometric Recognition , pp. 611–619, Springer, 2016.
[17] J. Yang, Z. Lei, and S. Z. Li, “Learn convolutional neural network for face anti­-spoofing,” arXiv preprint arXiv:1408.5601, 2014.
[18] L. Li, X. Feng, Z. Boulkenafet, Z. Xia, M. Li, and A. Hadid, “An original face anti­-spoofing approach using partial convolutional neural network,” in 2016 Sixth Interna­tional Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6, 2016.
[19] G. Pan, L. Sun, Z. Wu, and S. Lao, “Eyeblink­based anti­-spoofing in face recognition from a generic webcamera,” in 2007 IEEE 11th international conference on computer vision, pp. 1–8, IEEE, 2007.
[20] Y.­Y. Tsou, Y.­A. Lee, and C.­T. Hsu, “Multi­task learning for simultaneous video genera­tion and remote photoplethysmography estimation,” in Proceedings of the Asian Confer­ence on Computer Vision, 2020.
[21] S. Liu, P. C. Yuen, S. Zhang, and G. Zhao, “3d mask face anti­-spoofing with remote photoplethysmography,” in Computer Vision – ECCV 2016 (B. Leibe, J. Matas, N. Sebe, and M. Welling, eds.), (Cham), pp. 85–100, 2016.
[22] X. Li, J. Komulainen, G. Zhao, P.­C. Yuen, and M. Pietikäinen, “Generalized face anti­ spoofing by detecting pulse from face videos,” in 2016 23rd International Conference on Pattern Recognition (ICPR) , pp. 4244–4249, 2016.
[23] J. Guo, X. Zhu, Y. Yang, F. Yang, Z. Lei, and S. Z. Li, “Towards fast, accurate and stable 3d dense face alignment,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2020.
[24] X. Zhu, X. Liu, Z. Lei, and S. Z. Li, “Face alignment in full pose range: A 3d total solu­tion,” IEEE transactions on pattern analysis and machine intelligence, 2017.
[25] G. Zhang and J. Xu, “Discriminative feature representation for person re­identification by batch­contrastive loss,” in Asian Conference on Machine Learning, pp. 208–219, PMLR, 2018.
[26] S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 539–546, IEEE, 2005.
[27] Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid, “OULU­NPU: A mobile face presentation attack database with real­world variations,” in IEEE International Con­ference on Automatic Face and Gesture Recognition, May 2017.
[28] Z. Zhang, J. Yan, S. Liu, Z. Lei, D. Yi, and S. Z. Li, “A face antispoofing database with diverse attacks,” in 2012 5th IAPR international conference on Biometrics (ICB), pp. 26–31, IEEE, 2012.
[29] I. Chingovska, A. Anjos, and S. Marcel, “On the effectiveness of local binary patterns in face anti­spoofing,” in 2012 BIOSIG­proceedings of the international conference of biometrics special interest group (BIOSIG), pp. 1–7, IEEE, 2012.
[30] I. Standard, “Information technology—biometric presentation attack detection—part 1:Framework,” ISO: Geneva, Switzerland, 2016.
[31] Z. Boulkenafet, J. Komulainen, Z. Akhtar, A. Benlamoudi, D. Samai, S. E. Bekhouche, A. Ouafi, F. Dornaika, A. Taleb­Ahmed, L. Qin, et al., “A competition on generalized software-­based face presentation attack detection in mobile scenarios,” in 2017 IEEE In­ternational Joint Conference on Biometrics (IJCB) , pp. 688–696, IEEE, 2017.
[32] A. Pinto, H. Pedrini, W. R. Schwartz, and A. Rocha, “Face spoofing detection through visual codebooks of spectral temporal cubes,”
IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 4726–4740, 2015