在利用基於機器學習的醫學影像識別模型進行影像分類的任務上，有關於模型於不同族群的診斷公平性議題變得越來越重要。在部署基於機器學習的醫療診斷系統時，如果對不同人口族群（如性別、人種）的診斷效能存在差異，這會損害弱勢群體的利益。在這篇論文中，我們觀察到，雖然來自較深層的神經網路的特徵，能在分類時提供更高的準確性，但當我們使用此特徵進行分類時，會降低分類的公平性。這一現象促使我們擴展多出口框架的概念，用來改善模型分類的公平性。與現有主要關注於改善準確性的多出口框架不同，我們的多出口框架是以提升模型分類公平性為導向的。我們的研究指出，透過在傳統的，用於分類圖像的深度模型內引進內部分類器，不僅可以更準確，而且更公平的分類皮膚病圖像，同時這個框架也具有很高的擴展性，可以應用於大多數現有的，改善模型分類公平性的方法。實驗結果表明，我們所提出的訓練框架，相較於最先進的方法，可以在兩個皮膚病數據集上提升更多的分類公平性。

In the task of image classification using machine learning-based medical image recognition models, ensuring diagnostic fairness across various demographic groups has become increasingly critical. When deploying machine learning-based medical diagnostic systems, bias in prediction accuracy across different demographic groups (such as gender and race) can harm the interests of underprivileged groups. In this thesis, we observe that while features extracted from the deeper layers of neural networks generally offer higher accuracy, using these features can reduce the fairness of classification. This phenomenon motivates us to extend the concept of multi-exit frameworks to improve the fairness of model prediction. Unlike existing multi-exit frameworks that focus on enhancing accuracy, our multi-exit framework is fairness-oriented. Our research demonstrates that by incorporating internal classifiers within models, it is possible to classify dermatological images with greater accuracy and fairness. Additionally, this framework is highly extensible and can be integrated with most existing bias mitigation methods. Experimental results show that the proposed framework improves fairness compared to the state-of-the-art in two dermatological disease datasets.

Abstract (Chinese)
Abstract
Acknowledgements
Contents
List of Figures
List of Tables 1
1 Introduction 2
2 Related Work 5
2.1 Muti-Exit Networks 5
2.2 Bias Mitigation Methods 6
3 Motivation 8
4 Method 10
4.1 Problem Formulation 10
4.2 Multi-Exit Training Framework 10
5 Experiment 13
5.1 Dataset 13
5.2 Implementation Details 13
5.3 Evaluation Metrics 14
6 Results 16
6.1 Comparison with State-of-the-art 16
6.2 Multi-Exit Training on Different Method 18
6.3 Ablation Study 19
7 Conclusion 22
Bibliography 23

[1] Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker. Turning a blind eye: Explicit removal of biases and variation from deep neural network embeddings. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 0–0, 2018.

[2] Yoshua Bengio and Yann LeCun. Scaling learning algorithms towards AI. In Large Scale Kernel Machines. MIT Press, 2007.

[3] Yu-Jen Chen, Yen-Jung Chang, Shao-Cheng Wen, Yiyu Shi, Xiaowei Xu, Tsung-Yi Ho, Qianjun Jia, Meiping Huang, and Jian Zhuang. Zero-shot medical image artifact reduction. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pages 862–866. IEEE, 2020.

[4] Yu-Jen Chen, Yen-Jung Chang, Shao-Cheng Wen, Xiaowei Xu, Meiping Huang, Haiyun Yuan, Jian Zhuang, Yiyu Shi, and Tsung-Yi Ho. “one-shot” reduction of additive artifacts in medical images. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 836–841. IEEE, 2021.

[5] Yu-Jen Chen, Wei-Hsiang Shen, Hao-Wei Chung, Jing-Hao Chiu, Da-Cheng Juan, Tsung-Ying Ho, Chi-Tung Cheng, Meng-Lin Li, and Tsung-Yi Ho. Representative image feature extraction via contrastive learning pretraining for chest x-ray report generation. arXiv preprint arXiv:2209.01604, 2022.

[6] Yu-Jen Chen, Cheng-Yen Tsai, Xiaowei Xu, Yiyu Shi, Tsung-Yi Ho, Meiping Huang, Haiyun Yuan, and Jian Zhuang. CT image denoising with encoder-decoder based graph convolutional networks. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pages 400–404. IEEE, 2021.

[7] Marc Combalia, Noel CF Codella, Veronica Rotemberg, Brian Helba, Veronica Vilaplana, Ofer Reiter, Cristina Carrera, Alicia Barreiro, Allan C Halpern, Susana Puig, et al. BCN20000: Dermoscopic lesions in the wild. arXiv preprint arXiv:1908.02288, 2019.

[8] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018.

[9] Mengnan Du, Fan Yang, Na Zou, and Xia Hu. Fairness in deep learning: A computational perspective. IEEE Intelligent Systems, 36(4):25–34, 2020.

[10] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pages 214–226, 2012.

[11] Nicholas Frosst, Nicolas Papernot, and Geoffrey Hinton. Analyzing and improving representations with the soft nearest neighbor loss. In International Conference on Machine Learning, pages 2012–2020. PMLR, 2019.

[12] Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. Evaluating deep neural networks trained on clinical images in dermatology with the Fitzpatrick 17k dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1820–1828, 2021.

[13] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems, 29, 2016.

[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.

[15] Sangwon Jung, Donggyu Lee, Taeeon Park, and Taesup Moon. Fair feature distillation for visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12115–12124, 2021.

[16] Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1):1–33, 2012.

[17] Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras. Shallow-deep networks: Understanding and mitigating network overthinking. In International Conference on Machine Learning, pages 3301–3310. PMLR, 2019.

[18] Newton M Kinyanjui, Timothy Odonga, Celia Cintas, Noel CF Codella, Rameswar Panda, Prasanna Sattigeri, and Kush R Varshney. Fairness of classifiers across skin tones in dermatology. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI, pages 320–329. Springer, 2020.

[19] Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Deeply-supervised nets. In Artificial Intelligence and Statistics, pages 562–570. PMLR, 2015.

[20] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (CelebA) dataset. Retrieved August, 15(2018):11, 2018.

[21] Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. Gender bias in neural natural language processing. In Logic, Language, and Security, pages 189–202. Springer, 2020.

[22] Mkhuseli Ngxande, Jules-Raymond Tapamo, and Michael Burke. Bias remediation in driver drowsiness detection systems using generative adversarial networks. IEEE Access, 8:55592–55601, 2020.

[23] Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. Discovering fair representations in the data domain. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8227–8236, 2019.

[24] Robik Shrestha, Kushal Kafle, and Christopher Kanan. OccamNets: Mitigating dataset bias by favoring simpler hypotheses. In European Conference on Computer Vision, pages 702–721. Springer, 2022.

[25] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.

[26] Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. BranchyNet: Fast inference via early exiting from deep neural networks. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 2464–2469. IEEE, 2016.

[27] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1):1–9, 2018.

[28] Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8919–8928, 2020.

[29] Shao-Cheng Wen, Yu-Jen Chen, Zihao Liu, Wujie Wen, Xiaowei Xu, Yiyu Shi, Tsung-Yi Ho, Qianjun Jia, Meiping Huang, and Jian Zhuang. Do noises bother human and neural networks in the same way? A medical image analysis perspective. In 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 1166–1170. IEEE, 2020.

[30] Yawen Wu, Dewen Zeng, Xiaowei Xu, Yiyu Shi, and Jingtong Hu. FairPrune: Achieving fairness through pruning for dermatological disease diagnosis. arXiv preprint arXiv:2203.02110, 2022.

[31] Zikang Xu, Shang Zhao, Quan Quan, Qingsong Yao, and S Kevin Zhou. FairAdaBN: Mitigating unfairness with adaptive batch normalization and its application to dermatological disease classification. arXiv preprint arXiv:2303.08325, 2023.

[32] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340, 2018.

[33] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint arXiv:1707.09457, 2017.