我們提出了一種新的捲積神經網絡架構，稱為HarDNet-MSEG，應用於腸鏡檢查圖像的息肉分割。
HarDNet-MSEG由骨幹網路和解碼器組成。
骨幹網路採用 HarDNet，是一種低記憶體流量的 CNN 模型。
解碼器的靈感來自於以快速準確的顯著物體檢測而聞名的 Partial Decoder。

我們提出的模型有效地融合來自不同尺度的特徵。
在五個公開的息肉分割數據集上（Kvasir-SEG, CVC-ColonDB, EndoScene, ETIS-Larib Polyp DB 與 CVC-ClinicDB），不論是準確性或是推理速度上，皆達到世界第一的性能。
其中，在 Kvasir-SEG 上的表現，HarDNet-MSEG 是唯一達到超過 0.9 mean Dice 並且在 GeForce RTX 2080 Ti GPU 上有超過 100 FPS 的運行速度。
該代碼和所有實驗詳細信息均在 Github 上開源。

We propose a new convolutional neural network architecture called HarDNet-MSEG for polyp segmentation of colonoscopy images.
HarDNet-MSEG consists of a backbone and a decoder.
The backbone is a low memory traffic CNN called HarDNet.
The decoder is inspired by the Cascaded Partial Decoder, known for fast and accurate salient object detection.
Our proposed model aims to effectively fuse the features from different scales.
We have evaluated HarDNet-MSEG using five popular datasets (Kvasir-SEG, CVC-ColonDB, EndoScene, ETIS-Larib Polyp DB, and CVC-ClinicDB).
It achieves the state of the art performance in both accuracy and inference speed on all datasets.
For the Kvasir-SEG benchmark, HarDNet-MSEG delivers over 0.9 mean Dice running at over 100 FPS on a GeForce RTX 2080 Ti GPU.
The code and all experiment details are open-sourced on Github.

1 Introduction 1
2 Related Work 5
2.1 Architectures ................................. 5
2.2 Backbones...................................... 6
2.3 Enhancement Modules ........................... 8
2.3.1 Receptive Field Enhancement ................... 8
2.3.2 Attention Mechanism............................ 9
3 Proposed Architecture 11
3.1 HarDNet­-MSEG Overview.......................... 12
3.2 Backbone: HarDNet ............................. 13
3.2.1 HarDNet Introduction .......................... 13
3.2.2 HarDNet68 vs ResNet50.......................... 14
3.3 Architecture Selection ........................ 16
3.4 Additional Improvements........................ 18
3.4.1 Enhancement Modules............................ 18
3.4.2 Loss Function.................................. 20
3.4.3 Data Augmentation.............................. 21
3.5 Summary ....................................... 22
4 Experiment 23
4.1 Datasets and Experiment Setup ................. 24
4.1.1 Datasets ...................................... 24
4.1.2 Experiment setup............................... 25
4.1.3 Metrics ....................................... 26
4.2 Results........................................ 27
4.2.1 Results on the Kvasir-­SEG Data Set ............ 27
4.2.2 Results on the Multiple Data Sets.............. 28
4.2.3 Inference Analysis ............................ 30
5 Conclusion and Future work 33
5.1 Conclusion .................................... 33
5.2 Future work.................................... 35
Bibliography 37

[1] Abraham,N.,and Kahn,N.M. A novel focal tversky loss function with improved attention u­net for lesion segmentation. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) (2019), IEEE, pp. 683–687.
[2] Akbari, M., Mohrekesh, M., Nasr­Esfahani, E., Soroushmehr, S. R., Karimi, N., Samavi, S., and Najarian, K. Polyp segmentation in colonoscopy images using fully convolu­ national network. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2018), IEEE, pp. 69–72.
[3] Bernal, J., Sánchez, F. J., Fernández­Esparrach, G., Gil, D., Rodríguez, C., and Vilariño, F. Wm­dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized Medical Imaging and Graphics 43 (2015), 99–111.
[4] Bochkovskiy, A., Wang, C.­Y., and Liao, H.­Y. M. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020).
[5] Brandao, P., Mazomenos, E., Ciuti, G., Caliò, R., Bianchi, F., Menciassi, A., Dario, P., Koulaouzidis, A., Arezzo, A., and Stoyanov, D. Fully convolutional neural networks for polyp segmentation in colonoscopy. In Medical Imaging 2017: Computer­Aided Diagno­ sis (2017), vol. 10134, International Society for Optics and Photonics, p. 101340F.
[6] Chao, P., Kao, C.­Y., Ruan, Y.­S., Huang, C.­H., and Lin, Y.­L. Hardnet: A low memory traffic network. In Proceedings of the IEEE International Conference on Computer Vision (2019), pp. 3552–3561.
[7] Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 3213–3223.
[8] Deng, J., Dong, W., Socher, R., Li, L.­J., Li, K., and Fei­Fei, L. Imagenet: A large­scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (2009), Ieee, pp. 248–255.
[9] Eaton­Rosen, Z., Bragman, F., Ourselin, S., and Cardoso, M. J. Improving data augmen­tation for medical image segmentation.
[10] Erichsen, R., Baron, J. A., Hamilton­Dutoit, S. J., Snover, D. C., Torlakovic, E. E., Ped­ersen, L., Frøslev, T., Vyberg, M., Hamilton, S. R., and Sørensen, H. T. Increased risk of colorectal cancer development among patients with serrated polyps. Gastroenterology 150, 4 (2016), 895–902.
[11] Fan, D.­P., Ji, G.­P., Zhou, T., Chen, G., Fu, H., Shen, J., and Shao, L. Pranet: Parallel reverse attention network for polyp segmentation. MICCAI (2020).
[12] Fang, Y., Zhu, D., Yao, J., Yuan, Y., and Tong, K.­y. Abc­net: Area­boundary constraint network with dynamical feature selection for colorectal polyp segmentation. IEEE Sensors Journal (2020).
[13] He, K., Zhang, X., Ren, S., and Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE transactions on pattern analysis and machine intel­ ligence 37, 9 (2015), 1904–1916.
[14] He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 770–778.
[15] Huang,G.,Liu,Z.,VanDerMaaten,L.,andWeinberger,K.Q.Denselyconnectedconvolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (2017), pp. 4700–4708.
[16] J., T. N. G. S. L. Automated polyp detection in colonoscopy videos using shape and context information. IEEE TMI 35(2), 630–644 (2015).
[17] Jégou,S.,Drozdzal,M.,Vazquez,D.,Romero,A.,andBengio,Y.Theonehundredlayers tiramisu: Fully convolutional densenets for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (2017), pp. 11– 19.
[18] Jha, D., Ali, S., Johansen, H. D., Johansen, D. D., Rittscher, J., Riegler, M. A., and Halvorsen, P. Real­time polyp detection, localisation and segmentation in colonoscopy using deep learning. arXiv preprint arXiv:2011.07631 (2020).
[19] Jha, D., Riegler, M. A., Johansen, D., Halvorsen, P., and Johansen, H. D. Doubleu­net: A deep convolutional neural network for medical image segmentation. arXiv preprint arXiv:2006.04868 (2020).
[20] Jha, D., Smedsrud, P. H., Riegler, M. A., Halvorsen, P., de Lange, T., Johansen, D., and Johansen, H. D. Kvasir­seg: A segmented polyp dataset. In International Conference on Multimedia Modeling (2020), Springer, pp. 451–462.
[21] Jha, D., Smedsrud, P. H., Riegler, M. A., Johansen, D., De Lange, T., Halvorsen, P., and Johansen, H. D. Resunet++: An advanced architecture for medical image segmentation. In 2019 IEEE International Symposium on Multimedia (ISM) (2019), IEEE, pp. 225–2255.
[22] Liang­Chieh, C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. In IEEE transactions on pattern analysis and machine intelligence 40, no. 4 (2017), pp. 834–848.
[23] Liang­Chieh, C., Papandreou, G., Schroff, F., and Adam, H. Rethinking atrous convolu­tion for semantic image segmentation. In arXiv preprint arXiv:1706.05587 (2017).
[24] Liang­Chieh, C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. Encoder­decoder with atrous separable convolution for semantic image segmentation. In In Proceedings of the European conference on computer vision (ECCV), pp. 801­818. 2018.
[25] Lin, G., Milan, A., Shen, C., and Reid, I. Refinenet: Multi­path refinement networks for high­resolution semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (2017), pp. 1925–1934.
[26] Lin, T.­Y., Goyal, P., Girshick, R., He, K., and Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision (2017), pp. 2980–2988.
[27] Liu,S.,Huang,D.,etal.Receptivefieldblocknetforaccurateandfastobjectdetection.In Proceedings of the European Conference on Computer Vision (ECCV) (2018), pp. 385– 400.
[28] Long, J., Shelhamer, E., and Darrell, T. Fully convolutional networks for semantic seg­ mentation. In Proceedings of the IEEE conference on computer vision and pattern recog­ nition (2015), pp. 3431–3440.
[29] Ronneberger, O., Fischer, P., and Brox, T. U­net: Convolutional networks for biomed­ ical image segmentation. In International Conference on Medical image computing and computer­assisted intervention (2015), Springer, pp. 234–241.
[30] Shrestha, S., Khanal, B., and Ali, S. Ensemble u­net model for efficient polyp segmenta­ tion.
[31] Shu, L., Lu, Q., Haifang, Q., Jianping, S., and Jiaya, J. Path aggregation network for instance segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (2018), pp. 8759–8768.
[32] Silva, J., Histace, A., Romain, O., Dray, X., and Granado, B. Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. International Journal of Computer Assisted Radiology and Surgery 9, 2 (2014), 283–293.
[33] Tsung­Yi,L.,Piotr,D.,Ross,G.,Kaiming,H.,Bharath,H.,andSerge,B.Featurepyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (2017), pp. 2117–2125.
[34] Vázquez, D., Bernal, J., Sánchez, F. J., Fernández­Esparrach, G., López, A. M., Romero, A., Drozdzal, M., and Courville, A. A benchmark for endoluminal scene segmentation of colonoscopy images. Journal of healthcare engineering 2017 (2017).
[35] Wang, C.­Y., Liao, H.­Y. M., Wu, Y.­H., Chen, P.­Y., Hsieh, J.­W., and Yeh, I.­H. Csp­ net: A new backbone that can enhance learning capability of cnn. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (2020), pp. 390–391.
[36] Wu, Z., Su, L., and Huang, Q. Cascaded partial decoder for fast and accurate salient object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019), pp. 3907–3916.
[37] Yang, M., Yu, K., Zhang, C., Li, Z., and Yang, K. Denseaspp for semantic segmentation in street scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition (2018), pp. 3684–3692.
[38] Yang, X., Li, X., Ye, Y., Lau, R. Y., Zhang, X., and Huang, X. Road detection and center­ line extraction via deep recurrent convolutional neural network u­net. IEEE Transactions on Geoscience and Remote Sensing 57, 9 (2019), 7209–7220.
[39] Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., and Yoo, Y. Cutmix: Regularization strat­ egy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2019), pp. 6023–6032.
[40] Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez­Paz, D. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017).
[41] Zhang,R.,Li,G.,Li,Z.,Cui,S.,Qian,D.,andYu,Y.Adaptivecontextselectionforpolyp segmentation. In International Conference on Medical Image Computing and Computer­ Assisted Intervention (2020), Springer, pp. 253–262.
[42] Zhao,H.,Shi,J.,Qi,X.,Wang,X.,andJia,J.Pyramidsceneparsingnetwork.InProceed­ ings of the IEEE conference on computer vision and pattern recognition (2017), pp. 2881– 2890.
[43] Zhong, J., Wang, W., Wu, H., Wen, Z., and Qin, J. Polypseg: An efficient context­ aware network for polyp segmentation from colonoscopy videos. In International Confer­ ence on Medical Image Computing and Computer­Assisted Intervention (2020), Springer, pp. 285–294.