人體解剖學對現代醫學中有著舉足輕重的影響，因為許多疾病或病理狀況都以解剖學的形式呈現。如今，最先進的醫學影像技術，例如電腦斷層掃描等，已廣泛用於以無創方式評估這些疾病。然而，這些醫學圖像通常以灰階圖像顯示，因此需要具有專業訓練背景的人員才能對其進行醫學解釋。此外，有時色彩的提示對於診斷的準確性和術前計劃有著密切的關係，而色彩顯然是以灰階成像的電腦斷層影像所缺少的資訊。為此，我們在這篇論文中提出一個框架，以彌補電腦斷層影像和人體橫截面彩色圖像之間的差距。在此論文中，我們將問題定義為一種新穎的兩階段圖像轉換，並採用對抗式訓練來訓練模型。在模型的第一階段中，模型從電腦斷層圖像中提取與彩色圖像中關鍵的輪廓相對應的結構資訊。這是一個重要的部分，由於電腦斷層圖像和普通的彩色圖片之間的動態範圍差距很大。在第二階段中，我們通過將第一階段生成的結構訊息與原始CT圖像耦合來生成對應的彩色圖像。我們分別在第二以及第一階段中，使用大體橫截面的冷凍切片及其圖像梯度作為監督信號來訓練我們的模型。我們的實驗結果從數據以及生成圖片的品質證明我們的方法能有顯著的表現。為了研究模型的適用性，我們也透過問卷調查來獲得醫生的反饋。有部分醫生的反饋與我們的方法動機相吻合，並顯示了此模型在醫學上的適用性。

Human anatomy plays a pivotal role in modern medicine as many diseases or pathological conditions take the form of anatomical derangements. Nowadays, state-of-the-art technologies of medical imaging like Computed Tomography (CT) are widely used to evaluate these conditions in a noninvasive fashion. However, these medical images are typically displayed as grayscale images, which require a well-trained professional to interpret. Color cues, which are sometimes essential to make an accurate diagnosis and facilitate preoperative planning, are clearly lacking in these images. We hereby propose a framework to bridge the gap between CT images and cross-sectional color images. We formulate the problem as a novel two-stage image-to-image translation task and train our model with an adversarial training scheme. In the first stage, we focus on extracting the structural information corresponding to critical contours in the color image from a CT image. This is an important stage due to the large difference in dynamic range between CT and color images. In the second stage, we synthesize the color image by coupling the structural information with the original CT image. We train our framework using cadaveric cross-sectional cryosection and its image gradient as the supervision signal in the second and first stages, respectively. Our experimental results demonstrate the effectiveness of our proposed framework in both quantitative and qualitative aspects. To investigate the proposed framework's applicability, we also acquire doctors' feedback by conducting a human perceptual study. The feedback from a group of doctors aligns our approach's motivation and shows our framework's medical applicability.

摘要
Abstract
Acknowledgments
1 Introduction ............................................ 1
2 Background and Related Work ............................. 3
2.1 GenerativeAdversarialNetwork .......................... 3
2.2 Image­to­ImageTranslation ............................... 3
2.3 MedicalImageColorization .............................. 3
3 Method .................................................. 5
3.1 ImageGradientGenerator ................................ 6
3.2 ColorizationNetwork ................................... 7
4 Experiments ............................................. 9
4.1 ExperimentalSetup ..................................... 9
4.1.1 DatasetandImplementationDetail ...................... 9
4.1.2 ComparisonSetupandEvaluationMetrics ................. 9
4.2 Result ................................................ 10
4.2.1 QuantitativeResults ................................. 10
4.2.2 QualitativeResults .................................. 10
4.3 HumanPerceptualStudy .................................. 12
4.4 CrossDatasetAdaptation ................................ 13
5 Conclusion and Future Work .............................. 15
5.1 Conclusion ............................................ 15
5.2 FutureWork ............................................ 15
References ................................................ 17

[1] B. W. Turney, “Anatomy in a modern medical curriculum,” Annals of the Royal College of Surgeons of England, 2007.
[2] R. Zhang, P. Isola, and A. A. Efros, “Colorful image colorization,” in European Confer­ence on Computer Vision (ECCV), 2016.
[3] P. Isola, J.­-Y. Zhu, T. Zhou, and A.A.Efros, “Image­-to­-image translation with conditional adversarial networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[4] J. -­Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image­to­image translation using cycle­consistent adversarial networks,” in IEEE International Conference on Computer Vision (ICCV), 2017.
[5] K. Nazeri, E. Ng, T. Joseph, F. Z. Qureshi, and M. Ebrahimi, “Edgeconnect: Generative image inpainting with adversarial edge learning,” arXiv preprint arXiv:1901.00212, 2019.
[6] M. J. Ackerman, “The visible human project,” Proceedings of the IEEE, 1998.
[7] I. Goodfellow, J. Pouget­Abadie, M. Mirza, B. Xu, D. Warde­Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural In­ formation Processing Systems (NeurIPS), 2014.
[8] J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye, “A review on generative adversarial networks: Algorithms, theory, and applications,” arXiv preprint arXiv:2001.06937, 2020.
[9] L.A.Gatys, A.S.Ecker, and.Bethge, “Imagestyletransferusingconvolutionalneural networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[10] J. Johnson, A. Alahi, and L. Fei­Fei, “Perceptual losses for real­time style transfer and super­resolution,” in European Conference on Computer Vision (ECCV), 2016.
[11] J. M. Wolterink, A. M. Dinkla, M. H. Savenije, P. R. Seevinck, C. A. van den Berg, and I. Išgum, “Deep mr to ct synthesis using unpaired data,” in International workshop on simulation and synthesis in medical imaging, 2017.
[12] C.­B. Jin, H. Kim, M. Liu, W. Jung, S. Joo, E. Park, Y. S. Ahn, I. H. Han, J. I. Lee, and X. Cui, “Deep ct to mr synthesis using paired and unpaired data,” Sensors, 2019.
[13] M.U.G.Khan, Y.Gotoh, and N.Nida, “Medicalimagecolorizationforbettervisualization and segmentation,” in Medical Image Understanding and Analysis, 2017.
[14] X. Zeng, S. Tong, Y. Lu, L. Xu, and Z. Huang, “Adaptive medical image deep color per­ception algorithm,” IEEE Access, 2020.
[15] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014.
[16] N. Kanopoulos, N. Vasanthavada, and R. L. Baker, “Design of an image edge detection filter using the sobel operator,” IEEE Journal of Solid­State Circuits, 1988.
[17] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al., “Photo­realistic single image super­resolution using a generative adversarial network,” in IEEE Conference on Computer Vision and Pattern Recog­nition (CVPR), 2017.
[18] K.SimonyanandA.Zisserman, “Verydeepconvolutionalnetworksforlarge­scaleimage recognition,” arXiv preprint arXiv:1409.1556, 2014.
[19] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[20] J. Deng, W. Dong, R. Socher, L.­J. Li, K. Li, and L. Fei­Fei, “ImageNet: A Large­ Scale Hierarchical Image Database,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
[21] V. Spitzer, M. J. Ackerman, A. L. Scherzinger, and D. Whitlock, “The Visible Human Male: A Technical Report,” Journal of the American Medical Informatics Association, 1996.
[22] X.Mao,Q.Li,H.Xie,R.Y.Lau,Z.Wang,andS.PaulSmolley,“Leastsquaresgenerative adversarial networks,” in IEEE International Conference on Computer Vision (ICCV), 2017.
[23] Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, 2004.
[24] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effec­tiveness of deep features as a perceptual metric,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[25] D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[26] K. Yan, X. Wang, L. Lu, and R. M. Summers, “Deeplesion: automated mining of large­ scale lesion annotations and universal lesion detection with deep learning,” Journal of medical imaging, 2018.
[27] X.Ying, “Anoverviewofoverfittinganditssolutions,”inJournalofPhysics: Conference Series, 2019.
[28] Visible Body, “Human anatomy atlas.” https://www.visiblebody.com/ anatomy-and-physiology-apps/human-anatomy-atlas, 2021.
[29] M.­Y. Liu, X. Huang, A. Mallya, T. Karras, T. Aila, J. Lehtinen, and J. Kautz, “Few­shot unsupervised image­to­image translation,” in IEEE International Conference on Com­ puter Vision (ICCV), 2019.
[30] M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” Neurocomputing, 2018.
[31] Y. Li, N. Wang, J. Liu, and X. Hou, “Demystifying neural style transfer,” arXiv preprint arXiv:1701.01036, 2017.
[32] Z. Murez, S. Kolouri, D. Kriegman, R. Ramamoorthi, and K. Kim, “Image to image translation for domain adaptation,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.