本論文提出了一個由形狀引導的專家模型學習框架，可以有效解決無監督式3D 異常檢測的問題。此方法依靠兩個專家模型的協同作用，結合顏色和形狀模態定位出異常區域。第一個專家模型利用幾何資訊，通過建模局部形狀的隱式距離場來探測三維結構異常。第二個專家模型則使用與第一個專家模型相關的二維色彩特徵，識別局部形狀上的顏色異常。我們利用這兩個專家模型，從正常的訓練樣本中建立雙重記憶庫，並通過形狀引導的推理，精確定位測試樣本中的缺陷。憑藉三維各點的表達能力，以及有效結合互補模態的策略，我們的異常檢測方法在MVTec 3D-AD 資料集上，與當前各種方法相比，取得了最佳表現，並具備更高的召回率和更低的誤報率，對於實際應用更具價值。

We present a shape-guided expert-learning framework to tackle the problem of unsupervised 3D anomaly detection. Our method is established on the effectiveness of two specialized expert models and their synergy to localize anomalous regions from color and shape modalities. The first expert utilizes geometric information to probe 3D structural anomalies by modeling the implicit distance fields around local shapes. The second expert considers the 2D RGB features associated with the first expert to identify color appearance irregularities on the local shapes. We use the two experts to build the dual memory banks from the anomaly-free training samples and perform shape-guided inference to pinpoint the defects in the testing samples. Owing to the per-point 3D representation and the effective fusion scheme of complementary modalities, our method efficiently achieves state-of-the-art performance on the MVTec 3D-AD dataset with better recall and lower false positive rates, as preferred in real applications.

List of Tables 3
List of Figures 4
摘要 6
Abstract 7

1 Introduction 8
2 Related Work 10
2.1 2D Anomaly Detection 10
2.2 3D Anomaly Detection 11
3 Approach 13
3.1 Shape-Guided Expert Learning 13
3.2 Shape-Guided Inference 16
4 Experiments 19
4.1 Experimental Setup 19
4.2 Implementation Details 21
4.3 Evaluation Metrics 21
4.4 Experimental Results 22
4.5 Computational Complexity 24
4.6 Patch Size Analysis 25
4.7 Qualitative Results 25
4.8 Additional Ablation 28
4.8.1 Benefit of Combining RGB and 3D Information 28
4.8.2 The Effectiveness of Sparse Coding 30
5 Conclusion 31

A More Experimental Results 32
Bibliography 33

[1] The mvtec anomaly detection dataset: A comprehensive real-world dataset for unsupervised anomaly detection. Int. J. Comput. Vis., 129(4):1038–1059, 2021.
[2] F. Behrendt, M. Bengs, F. Rogge, J. Krüger, R. Opfer, and A. Schlaefer. Unsupervised anomaly detection in 3d brain MRI using deep learning with impured training data. In 19th IEEE International Symposium on Biomedical Imaging, ISBI 2022, Kolkata, India, March 28-31, 2022, pages 1–4. IEEE, 2022.
[3] M. Bengs, F. Behrendt, M. Laves, J. Krüger, R. Opfer, and A. Schlaefer. Unsupervised anomaly detection in 3d brain MRI using deep learning with multi-task brain age prediction. In K. Drukker and K. M. Iftekharuddin, editors, Medical Imaging 2022: Computer-Aided Diagnosis, San Diego, CA, USA, February 20-24, 2022 / on-line, March 21-27, 2022, volume 12033 of SPIE Proceedings. SPIE, 2022.
[4] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger. Mvtec AD - A comprehensive real-world dataset for unsupervised anomaly detection. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 9592–9600. Computer Vision Foundation / IEEE, 2019.
[5] P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 4182–4191. Computer Vision Foundation / IEEE, 2020.
[6] P. Bergmann, X. Jin, D. Sattlegger, and C. Steger. The mvtec 3d-ad dataset for unsupervised 3d anomaly detection and localization. In G. M. Farinella, P. Radeva, and
K. Bouatouch, editors, Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISI- GRAPP 2022, Volume 5: VISAPP, Online Streaming, February 6-8, 2022, pages 202–213. SCITEPRESS, 2022.
[7] P. Bergmann and D. Sattlegger. Anomaly detection in 3d point clouds using deep geometric descriptors. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023, Waikoloa, HI, USA, January 2-7, 2023, pages 2612–2622. IEEE, 2023.
[8] N. Cohen and Y. Hoshen. Sub-image anomaly detection with deep pyramid correspondences. CoRR, abs/2005.02357, 2020.
[9] T. Defard, A. Setkov, A. Loesch, and R. Audigier. Padim: A patch distribution modeling framework for anomaly detection and localization. In A. D. Bimbo, R. Cucchiara,
S. Sclaroff, G. M. Farinella, T. Mei, M. Bertini, H. J. Escalante, and R. Vezzani, editors, Pattern Recognition. ICPR International Workshops and Challenges - Virtual Event, January 10-15, 2021, Proceedings, Part IV, volume 12664 of Lecture Notes in Computer Science, pages 475–489. Springer, 2020.
[10] D. A. Gudovskiy, S. Ishizaka, and K. Kozuka. CFLOW-AD: real-time unsupervised anomaly detection with localization via conditional normalizing flows. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022, pages 1819–1828. IEEE, 2022.
[11] E. Horwitz and Y. Hoshen. Back to the feature: Classical 3d features are (almost) all you need for 3d anomaly detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Workshops, Vancouver, BC, Canada, June 17-24, 2023, pages 2968–2977. IEEE, 2023.
[12] C. M. Jiang, A. Sud, A. Makadia, J. Huang, M. Nießner, and T. A. Funkhouser. Local implicit grid representations for 3d scenes. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 6000–6009. Computer Vision Foundation / IEEE, 2020.
[13] S. Lee, S. Lee, and B. C. Song. CFA: coupled-hypersphere-based feature adaptation for target-oriented anomaly localization. IEEE Access, 10:78446–78454, 2022.
[14] C. Li, K. Sohn, J. Yoon, and T. Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 9664–9674. Computer Vision Foundation / IEEE, 2021.
[15] K. Li, Y. Tang, V. A. Prisacariu, and P. H. S. Torr. Bnv-fusion: Dense 3d reconstruction using bi-level neural volume fusion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 6156–6165. IEEE, 2022.
[16] Z. Liu, Y. Zhou, Y. Xu, and Z. Wang. Simplenet: A simple network for image anomaly detection and localization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 20402-20411. IEEE, 2023.
[17] B. Ma, Z. Han, Y. Liu, and M. Zwicker. Neural-pull: Learning signed distance function from point clouds by learning to pull space onto surface. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 7246–7257. PMLR, 2021.
[18] B. Ma, Y. Liu, M. Zwicker, and Z. Han. Surface reconstruction from point clouds by learning predictive context priors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 6316–6327. IEEE, 2022.
[19] Y. Pang, W. Wang, F. E. H. Tay, W. Liu, Y. Tian, and L. Yuan. Masked autoencoders for point cloud self-supervised learning. In S. Avidan, G. J. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, editors, Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part II, volume 13662 of Lecture Notes in Computer Science, pages 604–621. Springer, 2022.
[20] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 77–85. IEEE Computer Society, 2017.
[21] K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox, and P. V. Gehler. Towards total recall in industrial anomaly detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 14298–14308. IEEE, 2022.
[22] M. Rudolph, T. Wehrbein, B. Rosenhahn, and B. Wandt. Asymmetric student-teacher networks for industrial anomaly detection. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023, Waikoloa, HI, USA, January 2-7, 2023, pages 2591–2601. IEEE, 2023.
[23] H. M. Schlüter, J. Tan, B. Hou, and B. Kainz. Natural synthetic anomalies for self-supervised anomaly detection and localization. In S. Avidan, G. J. Brostow, M. Cissé,
G. M. Farinella, and T. Hassner, editors, Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXI, volume 13691 of Lecture Notes in Computer Science, pages 474–489. Springer, 2022.
[24] H. Shi, Y. Zhou, K. Yang, X. Yin, and K. Wang. Csflow: Learning opticalflow via cross strip correlation for autonomous driving. In 2022 IEEE Intelligent Vehicles Symposium, IV 2022, Aachen, Germany, June 4-9, 2022, pages 1851–1858. IEEE, 2022.
[25] T. Takikawa, J. Litalien, K. Yin, K. Kreis, C. T. Loop, D. Nowrouzezahrai, A. Jacobson, M. McGuire, and S. Fidler. Neural geometric level of detail: Real-time rendering with implicit 3d shapes. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 11358–11367. Computer Vision Foundation / IEEE, 2021.
[26] J. S. Viana, E. de la Rosa, T. V. Vyvere, D. Robben, and D. M. Sima. Unsupervised 3d brain anomaly detection. In A. Crimi and S. Bakas, editors, Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part I, volume 12658 of Lecture Notes in Computer Science, pages 133–142. Springer, 2020.
[27] Y. Wang, J. Peng, J. Zhang, R. Yi, Y. Wang, and C. Wang. Multimodal industrial anomaly detection via hybrid fusion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 8032–8041. IEEE, 2023.
[28] M. Yang, P. Wu, J. Liu, and H. Feng. Memseg: A semi-supervised method for image surface defect detection using differences and commonalities. CoRR, abs/2205.00908, 2022.
[29] S. Zagoruyko and N. Komodakis. Wide residual networks. In R. C. Wilson, E. R. Hancock, and W. A. P. Smith, editors, Proceedings of the British Machine Vision Conference 2016, BMVC 2016, York, UK, September 19-22, 2016. BMVA Press, 2016.
[30] V. Zavrtanik, M. Kristan, and D. Skocaj. Dræm - A discriminatively trained reconstruction embedding for surface anomaly detection. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 8310–8319. IEEE, 2021.
[31] Y. Zheng, X. Wang, Y. Qi, W. Li, and L. Wu. Benchmarking unsupervised anomaly detection and localization. CoRR, abs/2205.14852, 2022.