這篇論文著重在無監督式域自適應學習的問題，目標是將使用成對的源域訓練資料與正確標註學習得到的深度學習網絡架構利用遷移學習的方式應用到無正確標注資料的目標域上。我們的目標主要有兩個：利用觀測到的源域與目標域資料來縮小兩域之間特性分布差異，並同時著重學習出可鑑別度高的目標域網絡架構。我們提出一個生成對抗式引導學習網絡來處理這個問題。我們延續了域對抗式學習的方式來處理源域與目標域之間的域偏差問題。並且，為了要學習出可鑑別度高的目標域網絡架構，我們提出利用生成學習網絡來引導分類網絡，將分類器的決策邊界推離目標域資料所在的高密度區域。我們提出的生成對抗式引導學習網絡是一個端到端的架構，因此可以同時學習分類網絡架構並且在生成網絡的引導下強化其決策邊界。我們的實驗結果顯示提出的生成對抗式引導學習網絡不僅呈現出超越域對抗式學習網絡成效，並且在數個標準的域自適應學習數據集上表現得與其他最先進的方法並駕齊驅，不分軒輊。

This thesis focuses on unsupervised domain adaptation problem, which aims to learn a classification model on an unlabelled target domain by referring to a fully-labelled source domain. Our goal is twofold: bridging the gap between source-target domains, and deriving a discriminative model for the target domain. We propose a Generative Adversarial Guided Learning (GAGL) model to tackle the task. To minimize the source-target domain shift, we adopt the idea of domain adversarial training to build a classification network. Next, to derive a target discriminative classifier, we propose to include a generative network to guide the classifier so as to push its decision boundaries away from high density area of target domain. The proposed GAGL model is an end-to-end framework and thus can simultaneously learn the classification model and refine its decision boundary under the guidance of the generator. Our experimental results show that the proposed GAGL model not only outperforms the baseline domain adversarial model but also achieves competitive results with state-of-the-art methods on standard benchmarks.

中文摘要......I
Abstract......II
1. Introduction......1
2. Related Work......4
2.1 Distribution-matching-based Learning via Pre-defined Metric......4
2.2 Adversarial Learning for Domain Adaptation......4
2.3 Non-distribution-matching-based Methods......5
2.4 Discriminative Learning......5
3. Proposed Method......6
3.1 Problem Statement and Notations......6
3.2 Review of Domain Adversarial Training......7
3.3 Generative Adversarial Guided Learning......8
4. Experiments......11
4.1 Domain Adaptation Bencemarks......11
4.2 Implementation Details......12
4.3 Experimental Results and Discussion......15
5. Conclusion......24
6. References......25

[1] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, " Contour detection and hierarchical image segmentation," In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 33, pages898-916, 2011.
[2] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan, “A theory of learning from different domains,” In Machine Learning, 79:151-175, 2010.
[3] O. Chapelle and A. Zien, “Semi-supervised classification by low density separation," In Proc. International Workshop on Artificial Intelligence and Statistics, pages 57–64, 2005.
[4] A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” In Proc. International Conference on Artificial Intelligence and Statistics, volume 15, pages 215-226, 2011.
[5] Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. Salakhutdinov, "Good semi-supervised learning that requires a bad GAN," In Advances in Neural Information Processing Systems, 2017.
[6] G. French, M. Mackiewicz, and M. Fisher, “Self-ensembling for visual domain adaptation,” arXiv:1706.05208v1, 2017.
[7] G. French, M. Mackiewicz, and M. Fisher, “Self-ensembling for visual domain adaptation,” In Proc. International Conference on Learning Representation, 2018.
[8] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” In Proc. International Conference on Machine Learning, volume 37, pages 1180–1189, 2015.
[9] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, " Generative adversarial networks," In Advances in Neural Information Processing Systems, 2014.
[10] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Scholkopf, and A. Smola, “A kernel two-sample test,” In Proc. International Conference on Machine Learning, volume 13, pages 723-773, 2012.
[11] P. Haeusser, T. Frerix, A. Mordvintsev, and D. Cremers, “Associative domain adaptation,” In International Conference on Computer Vision, 2017.
[12] J. Hoffman, E. Tzeng, T. Park, J. Y. Zhu, P. Isola, K. Saenko, A. A. Efros, and T. Darrell, “CyCADA: cycle-Consistent Adversarial Domain Adaptation,” arXiv:1711.03213v3, 2017.
[13] D. P. Kingma and J. L. Ba, “Adam: a method for stochastic optimization,” In Proc. International Conference on Learning Representation, 2015.
[14] A. Krizhevsky, “Learning multiple layers of features from tiny images,” Tech report, 2009.
[15] A. Kumar, P. Sattigeri, and P. T. Fletcher, “Semi-supervised learning with GANs: manifold invariance with improved inference,” In Advances in Neural Information Processing Systems, 2017.
[16] S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” In Proc. International Conference on Learning Representation, 2016.
[17] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” In Proc. of the IEEE, volume 86, pages 2278-2324, 1998.
[18] M. Long, Y. Cao, J. Wang, and M. I. Jordan, “Learning transferable features with deep adaptation networks,” In Proc. International Conference on Machine Learning, volume 37, pages 97-105, 2015.
[19] T. Miyato, S. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training : a regularization method for supervised and semi-supervised learning,” arXiv:1704.03976, 2017.
[20] B. Moiseyev, A. Konev, A. Chigorin, and A. Konushin, “Evaluation of traffic sign recognition methods trained on synthetically generated data,” In Advanced Concepts for Intelligent Vision Systems, pages 576–583, 2013.
[21] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” In NIPS workshop on Deep Learning and Unsupervised Fearture Learning, 2011.
[22] B. T. Polyak and A. B. Juditsky, “Acceleration of stochastic approximation by averaging,” In SIAM Journal on Control and Optimization, volume 30, pages 838-855, 1992.
[23] P. Russo, F. M. Carlucci, T. Tommasi, and B. Caputo, “From source to target and back: symmetric bi-directional adaptive GAN,” arXiv:1705.08824v2, 2017.
[24] K. Saito, Y. Ushiku, and T. Harada, “Asymmetric tri-training for unsupervised domain adaptation,” In Proc. International Conference on Machine Learning, volume 70, pages 2988-2997, 2017.
[25] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training GANs,” In Advances in Neural Information Processing Systems, 2016.
[26] S. Sankaranarayanan, Y. Balaji, C. D. Castillo, and R. Chellappa, “Generate to adapt: aligning domains using generative adversarial networks,” IarXiv:1704.01705v3, 2017.
[27] O. Sener, H. O. Song, A. Saxena, and S. Savarese, “Learning transferrable representations for unsupervised domain adaptation,” In Advances in Neural Information Processing Systems, 2016.
[28] J. Shen, Y. Qu, W. Zhang, and Y. Yu, “Wasserstein distance guided representation learning for domain adaptation,” In AAAI, 2018.
[29] R. Shu, H. H. Bui, H. Narui, and S. Ermon, “A DIRT-T approach to unsupervised domain adaptation,” In Proc. International Conference on Learning Representation, 2018.
[30] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “The German traffic sign recognition benchmark: a multi-class classification competition,” In IEEE International Joint Conference on Neural Networks, 2011..

[31] E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell, “Deep domain confusion: maximizing for domain invariance,” arXiv:1412.3474, 2014.
[32] E. Tzeng, J. Hoffman, K. Saenko, T. Darrell. “Adversarail discriminative domain adaptation.,” In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2962-2971, 2017.
[33] L. v. d. Maaten and G. Hinton, “Visualizing data using t-SNE,” In Journal of Machine Learning Research, volume 9, pages 2579-2605, 2008.
[34] R. Volpi, P. Morerio, S. Savarese, and V. Murino, “Adversarial feature augmentation for unsupervised domain adaptation,” arXiv:1711.08561v1, 2017.
[35] W. Zellinger, T. Grubinger, E. Lughofer, T. Natschlager, and S. Saminger-Platz, “Central moment discrepancy (CMD) for domain-invariant representation learning,” In Proc. International Conference on Learning Representation, 2017.