人臉反偽造旨在保護人臉識別系統免受於各種演示攻擊。為應對跨域測試場景，許多人臉反偽造方法採用了域泛化或域適應技術，在離線學習階段藉由使用所有可用的源域數據來適應模型。然而，由於攻擊手法不斷增加及演變，嘗試通過離線適應技術來模擬未知攻擊雖然不無可能但非常困難。測試時間適應專注於在線將現成模型適應到未標記的目標數據，而不參考任何源域數據，在圖像分類中已經取得成功，但在人臉反偽造方法中尚未被探索。在本文中，我們的目標是解決測試時間適應對於人臉反偽造的問題，以實現穩健的人臉反偽造。我們首先提出了一個新的測試時間適應基準，涵蓋不同域及各種攻擊，以模擬人臉反偽造面對新域數據和未知攻擊時所面臨的挑戰。接著，我們設計了一個新的框架3A-TTA，包含三個主要元件：基於激活的偽標籤生成、抗遺忘特徵學習和非對稱原型對比學習，以應對人臉反偽造中的測試時間適應問題。我們對所提出的基準進行了大量實驗，結果顯示，所提出的3A-TTA在在線檢測來自新域的已知和未知類型的面部攻擊方面取得了卓越的性能。

Face anti-spoofing (FAS) aims to defend face recognition systems from various presentation attacks. To deal with cross-domain testing scenarios, many FAS methods adopted domain generalization or domain adaptation approaches by using all the available source domain data to adapt the model in the offline training stage. However, as there exist ever-growing and ever-evolving attacks, attempting to simulate unseen attacks by offline adaptation techniques is extremely difficult if not impossible. Test-Time Adaptation (TTA), which focuses on on-line adapting an off-the-shelf model to unlabeled target data without referring to any source data, has been successfully adopted in image classification but is still unexplored in FAS methods. In this thesis, our goal is to address the TTA issues for robust face anti-spoofing. We first propose a novel TTA benchmark covering different domains and various attacks to simulate the challenges of FAS when facing new domain data and unseen attacks. Next, we develop a novel framework 3A-TTA, including three main components: activation-based pseudo-labeling, anti-forgetting feature learning, and asymmetric prototype contrastive learning to tackle the issues of \, TTA in FAS. Our extensive experiments on the proposed benchmark show that the proposed 3A-TTA achieves superior performance for on-line detecting both seen and unseen types of face presentation attacks from new domains.

摘要i
Abstract ii
Acknowledgements
1 Introduction 1
2 Related work 6
2.1 Face Anti-spoofing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Test-Time Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3 Method 8
3.1 Problem Statement and Overview of 3A-TTA framework . . . . . . . . . . . . 8
3.2 Activation-Based Pseudo-Labeling . . . . . . . . . . . . . . . . . . . . . . . . 10
3.3 Anti-Forgetting Feature Learning . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Asymmetric Prototype Contrastive Learning . . . . . . . . . . . . . . . . . . . 12
3.5 Total Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4 Experiments 15
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.2.1 OULU-NPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2.2 CASIA-MFSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2.3 MSU-MFSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2.4 Idiap Replay-Attack . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2.5 3DMAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2.6 HKBU-MARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3 Test-Time Adaptation Benchmark for FAS (TTA-FAS) . . . . . . . . . . . . . 17
4.4 Evaluation Metrics and Implementation Details . . . . . . . . . . . . . . . . . 18
4.5 Ablation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.5.1 Different Combinations of Loss Terms and Modules . . . . . . . . . . 21
4.5.2 Activation Maps and t-SNE Visualization . . . . . . . . . . . . . . . . 22
4.5.3 Sizes of Memory Bank and Numbers of Nearest Neighbors . . . . . . . 24
4.5.4 Experimental Comparisons on the Proposed TTA-FAS Benchmark . . . 25
4.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.6.1 The Vital Impact of Source Model Quality on TTA Performance . . . . 27
4.6.2 Error Rates for Each Class in Activation-Based Pseudo-Labeling . . . 27
i
5 Conclusion 28
References 29
ii

[1] Z. Boulkenafet, J. Komulainen, L. Li, X. Feng, and A. Hadid, “Oulu-npu: A mobile face
presentation attack database with real-world variations,” in 2017 12th IEEE international
conference on automatic face & gesture recognition (FG 2017), pp. 612–618, IEEE, 2017.
[2] Z. Zhang, J. Yan, S. Liu, Z. Lei, D. Yi, and S. Z. Li, “A face antispoofing database with
diverse attacks,” in 2012 5th IAPR international conference on Biometrics (ICB), pp. 26–
31, IEEE, 2012.
[3] D. Wen, H. Han, and A. K. Jain, “Face spoof detection with image distortion analysis,”
IEEE Transactions on Information Forensics and Security, vol. 10, no. 4, pp. 746–761,
2015.
[4] I. Chingovska, A. Anjos, and S. Marcel, “On the effectiveness of local binary patterns
in face anti-spoofing,” in 2012 BIOSIG-proceedings of the international conference of
biometrics special interest group (BIOSIG), pp. 1–7, IEEE, 2012.
[5] N. Erdogmus and S. Marcel, “Spoofing face recognition with 3d masks,” IEEE transactions
on information forensics and security, vol. 9, no. 7, pp. 1084–1097, 2014.
[6] S. Liu, P. C. Yuen, S. Zhang, and G. Zhao, “3d mask face anti-spoofing with remote photoplethysmography,”
in European Conference on Computer Vision, pp. 85–100, Springer,
2016.
[7] L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.,” Journal of machine
learning research, vol. 9, no. 11, 2008.
[8] Z. Chen, T. Yao, K. Sheng, S. Ding, Y. Tai, J. Li, F. Huang, and X. Jin, “Generalizable representation
learning for mixture domain face anti-spoofing,” in Proceedings of the AAAI
Conference on Artificial Intelligence, vol. 35, pp. 1132–1139, 2021.
[9] H. Feng, Z. Hong, H. Yue, Y. Chen, K. Wang, J. Han, J. Liu, and E. Ding, “Learning
generalized spoof cues for face anti-spoofing,” arXiv preprint arXiv:2005.03922, 2020.
[10] P.-K. Huang, M.-C. Chin, and C.-T. Hsu, “Face anti-spoofing via robust auxiliary estimation
and discriminative feature learning,” in Asian Conference on Pattern Recognition,
pp. 443–458, Springer, 2022.
[11] Y. Jia, J. Zhang, S. Shan, and X. Chen, “Single-side domain generalization for face antispoofing,”
in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition, pp. 8484–8493, 2020.
[12] A. Jourabloo, Y. Liu, and X. Liu, “Face de-spoofing: Anti-spoofing via noise modeling,”
in Proceedings of the European Conference on Computer Vision (ECCV), pp. 290–306,
2018.
[13] T. Kim, Y. Kim, I. Kim, and D. Kim, “Basn: Enriching feature representation using bipartite
auxiliary supervisions for face anti-spoofing,” in Proceedings of the IEEE/CVF
International Conference on Computer Vision Workshops, pp. 0–0, 2019.
[14] X. Li, J. Komulainen, G. Zhao, P.-C. Yuen, and M. Pietikäinen, “Generalized face antispoofing
by detecting pulse from face videos,” in 2016 23rd International Conference on
Pattern Recognition (ICPR), pp. 4244–4249, IEEE, 2016.
[15] S. Liu, K.-Y. Zhang, T. Yao, M. Bi, S. Ding, J. Li, F. Huang, and L. Ma, “Adaptive normalized
representation learning for generalizable face anti-spoofing,” in Proceedings of
the 29th ACM International Conference on Multimedia, pp. 1469–1477, 2021.
[16] S.-Q. Liu, X. Lan, and P. C. Yuen, “Remote photoplethysmography correspondence feature
for 3d mask face presentation attack detection,” in Proceedings of the European Conference
on Computer Vision (ECCV), pp. 558–573, 2018.
[17] Y. Liu, A. Jourabloo, and X. Liu, “Learning deep models for face anti-spoofing: Binary
or auxiliary supervision,” in Proceedings of the IEEE conference on computer vision and
pattern recognition, pp. 389–398, 2018.
[18] Y. Liu, J. Stehouwer, A. Jourabloo, and X. Liu, “Deep tree learning for zero-shot face antispoofing,”
in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition, pp. 4680–4689, 2019.
[19] Y. Liu, J. Stehouwer, and X. Liu, “On disentangling spoof trace for generic face antispoofing,”
in European Conference on Computer Vision, pp. 406–422, Springer, 2020.
[20] R. Shao, X. Lan, J. Li, and P. C. Yuen, “Multi-adversarial discriminative deep domain
generalization for face presentation attack detection,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pp. 10023–10031, 2019.
[21] R. Shao, X. Lan, and P. C. Yuen, “Regularized fine-grained meta face anti-spoofing,” in
Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11974–11981,
2020.
[22] G. Wang, H. Han, S. Shan, and X. Chen, “Cross-domain face presentation attack detection
via multi-domain disentangled representation learning,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pp. 6678–6687, 2020.
[23] J. Wang, J. Zhang, Y. Bian, Y. Cai, C. Wang, and S. Pu, “Self-domain adaptation for face
anti-spoofing,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35,
pp. 2746–2754, 2021.
[24] Y.-C. Wang, C.-Y. Wang, and S.-H. Lai, “Disentangled representation with dual-stage feature
learning for face anti-spoofing,” in Proceedings of the IEEE/CVF Winter Conference
on Applications of Computer Vision, pp. 1955–1964, 2022.
[25] Z. Wang, Z. Yu, C. Zhao, X. Zhu, Y. Qin, Q. Zhou, F. Zhou, and Z. Lei, “Deep spatial gradient
and temporal depth learning for face anti-spoofing,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pp. 5042–5051, 2020.
[26] Z. Wang, Z. Wang, Z. Yu, W. Deng, J. Li, T. Gao, and Z. Wang, “Domain generalization
via shuffled style assembly for face anti-spoofing,” in Proceedings of the IEEE/CVF
Conference on Computer Vision and Pattern Recognition, pp. 4123–4133, 2022.
[27] Z. Yu, X. Li, X. Niu, J. Shi, and G. Zhao, “Face anti-spoofing with human material perception,”
in European Conference on Computer Vision, pp. 557–575, Springer, 2020.
[28] Z. Yu, J. Wan, Y. Qin, X. Li, S. Z. Li, and G. Zhao, “Nas-fas: Static-dynamic central
difference network search for face anti-spoofing,” IEEE transactions on pattern analysis
and machine intelligence, vol. 43, no. 9, pp. 3005–3023, 2020.
[29] K.-Y. Zhang, T. Yao, J. Zhang, Y. Tai, S. Ding, J. Li, F. Huang, H. Song, and L. Ma,
“Face anti-spoofing via disentangled representation learning,” in European Conference
on Computer Vision, pp. 641–657, Springer, 2020.
[30] K.-Y. Zhang, T. Yao, J. Zhang, S. Liu, B. Yin, S. Ding, and J. Li, “Structure destruction and
content combination for face anti-spoofing,” in 2021 IEEE International Joint Conference
on Biometrics (IJCB), pp. 1–6, IEEE, 2021.
[31] S. Zhang, A. Liu, J. Wan, Y. Liang, G. Guo, S. Escalera, H. J. Escalante, and S. Z. Li,
“Casia-surf: A large-scale multi-modal benchmark for face anti-spoofing,” IEEE Transactions
on Biometrics, Behavior, and Identity Science, vol. 2, no. 2, pp. 182–193, 2020.
[32] Y. Zhang, Z. Yin, Y. Li, G. Yin, J. Yan, J. Shao, and Z. Liu, “Celeba-spoof: Large-scale
face anti-spoofing dataset with rich annotations,” in European Conference on Computer
Vision, pp. 70–85, Springer, 2020.
[33] Z. Yu, X. Li, J. Shi, Z. Xia, and G. Zhao, “Revisiting pixel-wise supervision for face antispoofing,”
IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 3, no. 3,
pp. 285–295, 2021.
[34] Y. Qin, Z. Yu, L. Yan, Z. Wang, C. Zhao, and Z. Lei, “Meta-teacher for face anti-spoofing,”
IEEE transactions on pattern analysis and machine intelligence, 2021.
[35] X. Tu, Z. Ma, J. Zhao, G. Du, M. Xie, and J. Feng, “Learning generalizable and identitydiscriminative
representations for face anti-spoofing,” ACM Transactions on Intelligent
Systems and Technology (TIST), vol. 11, no. 5, pp. 1–19, 2020.
[36] M. Fang, N. Damer, F. Kirchbuchner, and A. Kuijper, “Learnable multi-level frequency
decomposition and hierarchical attention mechanism for generalized face presentation attack
detection,” in Proceedings of the IEEE/CVF Winter Conference on Applications of
Computer Vision, pp. 3722–3731, 2022.
[37] N. Sanghvi, S. K. Singh, A. Agarwal, M. Vatsa, and R. Singh, “Mixnet for generalized face
presentation attack detection,” in 2020 25th International Conference on Pattern Recognition
(ICPR), pp. 5511–5518, IEEE, 2021.
[38] A. Agarwal, R. Singh, M. Vatsa, and A. Noore, “Boosting face presentation attack detection
in multi-spectral videos through score fusion of wavelet partition images,” Frontiers
in big Data, p. 53, 2022.
[39] P.-K. Huang, H.-Y. Ni, Y.-Q. Ni, and C.-T. Hsu, “Learnable descriptive convolutional
network for face anti-spoofing,” in BMVC, 2022.
[40] Y. Sun, Y. Liu, X. Liu, Y. Li, and W.-S. Chu, “Rethinking domain generalization for face
anti-spoofing: Separability and alignment,” arXiv preprint arXiv:2303.13662, 2023.
[41] C.-Y. Wang, Y.-D. Lu, S.-T. Yang, and S.-H. Lai, “Patchnet: A simple face anti-spoofing
framework via fine-grained patch recognition,” in Proceedings of the IEEE/CVF Conference
on Computer Vision and Pattern Recognition, pp. 20281–20290, 2022.
[42] Z. Yu, C. Zhao, Z. Wang, Y. Qin, Z. Su, X. Li, F. Zhou, and G. Zhao, “Searching
central difference convolutional networks for face anti-spoofing,” in Proceedings of the
IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5295–5305,
2020.
[43] B. Chen, W. Yang, H. Li, S. Wang, and S. Kwong, “Camera invariant feature learning for
generalized face anti-spoofing,” IEEE Transactions on Information Forensics and Security,
vol. 16, pp. 2477–2492, 2021.
[44] A. George and S. Marcel, “On the effectiveness of vision transformers for zero-shot face
anti-spoofing,” in 2021 IEEE International Joint Conference on Biometrics (IJCB), pp. 1–
8, IEEE, 2021.
[45] Z. Wang, Q. Wang, W. Deng, and G. Guo, “Face anti-spoofing using transformers with
relation-aware mechanism,” IEEE Transactions on Biometrics, Behavior, and Identity Science,
vol. 4, no. 3, pp. 439–450, 2022.
[46] Z. Wang, Q. Wang, W. Deng, and G. d. Guo, “Learning multi-granularity temporal characteristics
for face anti-spoofing,” IEEE Transactions on Information Forensics and Security,
vol. 17, pp. 1254–1269, 2022.
[47] P.-K. Huang, J.-X. Chong, H.-Y. Ni, T.-H. Chen, and C.-T. Hsu, “Towards diverse liveness
feature representation and domain expansion for cross-domain face anti-spoofing,”
in 2023 IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2023.
[48] R. Quan, Y. Wu, X. Yu, and Y. Yang, “Progressive transfer learning for face anti-spoofing,”
IEEE Transactions on Image Processing, vol. 30, pp. 3946–3955, 2021.
[49] P.-K. Huang, C.-L. Chang, H.-Y. Ni, and C.-T. Hsu, “Learning to augment face presentation
attack dataset via disentangled feature learning from limited spoof data,” in 2022
IEEE International Conference on Multimedia and Expo (ICME), IEEE, 2022.
[50] Q. Zhou, K.-Y. Zhang, T. Yao, R. Yi, K. Sheng, S. Ding, and L. Ma, “Generative domain
adaptation for face anti-spoofing,” in Computer Vision–ECCV 2022: 17th European
Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part V, pp. 335–356,
Springer, 2022.
[51] H.-P. Huang, D. Sun, Y. Liu, W.-S. Chu, T. Xiao, J. Yuan, H. Adam, and M.-H. Yang,
“Adaptive transformers for robust few-shot cross-domain face anti-spoofing,” in Computer
Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27,
2022, Proceedings, Part XIII, pp. 37–54, Springer, 2022.
[52] Y. Liu, Y. Chen, W. Dai, M. Gou, C.-T. Huang, and H. Xiong, “Source-free domain adaptation
with contrastive domain alignment and self-supervised exploration for face antispoofing,”
in Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel,
October 23–27, 2022, Proceedings, Part XII, pp. 511–528, Springer, 2022.
[53] D. Chen, D. Wang, T. Darrell, and S. Ebrahimi, “Contrastive test-time adaptation,” in
CVPR, 2022.
[54] S. Niu, J. Wu, Y. Zhang, Z. Wen, Y. Chen, P. Zhao, and M. Tan, “Towards stable test-time
adaptation in dynamic wild world,” in Internetional Conference on Learning Representations,
2023.
[55] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, “Tent: Fully test-time adaptation
by entropy minimization,” in International Conference on Learning Representations,
2021.
[56] Y. Iwasawa and Y. Matsuo, “Test-time classifier adjustment module for model-agnostic
domain generalization,” Advances in Neural Information Processing Systems, vol. 34,
pp. 2427–2440, 2021.
[57] M. Jang, S.-Y. Chung, and H. W. Chung, “Test-time adaptation via self-training with nearest
neighbor information,” in The Eleventh International Conference on Learning Representations,
2023.
[58] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam:
Visual explanations from deep networks via gradient-based localization,” in Proceedings
of the IEEE international conference on computer vision, pp. 618–626, 2017.
[59] D. Belli, D. Das, B. Major, and F. Porikli, “Online adaptive personalization for face antispoofing,”
in 2022 IEEE International Conference on Image Processing (ICIP), pp. 351–
355, IEEE, 2022.
[60] X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance
normalization,” in ICCV, 2017.
[61] A. Anjos and S. Marcel, “Counter-measures to photo attacks in face recognition: a public
database and a baseline,” in 2011 international joint conference on Biometrics (IJCB),
pp. 1–7, IEEE, 2011.
[62] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in
Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–
778, 2016.
[63] S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with
application to face verification,” in 2005 IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR’05), vol. 1, pp. 539–546, IEEE, 2005.