本論文中，我們提供了一個統一的架構來解決網路嵌入問題。網路嵌入問題的目標是將在高維空間中相似的點，投影到歐氏幾何空間中相近的點，進而學習到網路中節點的低維向量表示式。因為眾人對網路上哪些節點為“相似”可以有不同的看法，網路嵌入問題為一不適定問題。我們引入了一個機率架構來隨機取樣網路中的節點，進而將模組性嵌入延伸到可以由取樣網路(sampled graph)得到廣義的模組性嵌入(generalized modularity). 基於取樣網路，模組化的概念可以精確地被定義。廣義的模組性嵌入可以藉由應用不同的取樣方法，簡單地採用不同的看法，來解決網路嵌入問題。與網路嵌入問題相同，社群偵測問題也為一不適定問題。在社群偵測問題中，人們找尋大的模組值保存社群結構。我們的推導會展現出網路嵌入問題與社群偵測問題，這兩個問題在特定的限制下可以被化簡成相同的形式。廣義的模組性嵌入進而能夠代入多樣觀點，藉由應用不同取樣方法捕獲網路特徵。本論文提供了一種基於廣義的模組性嵌入，靈活且有效的網路嵌入方法。最後，我們在真實網路數據下進行實驗以說明我們方法的有效性。

In this thesis, we provide a unified framework of the network embedding problem. The embedding problem aims to map nodes that are similar to each other in a high- dimensional space to vectors in a Euclidean space that are close to each other. It then learns the low-dimensional representations of nodes in networks. The embedding problem is ill-posed since ones may hold different opinions about what kind of nodes are ”simi- lar”. Here we introduce a probabilistic framework to sample a network randomly, so as to extend modularity to generalized modularity, which can be obtained from the sampled graph. Based on sampled graphs, the notion of modularity can be explicitly defined. To solve the embedding problem, the generalized modularity can easily adopt different viewpoints by applying different sampling methods. Same as the embedding problem, the community detection problem is also an ill-posed problem. In the community detection problem, one looks for large values of modularity to preserve the community structure. Our derivation shows that the network embedding problem and the community detection problem can be formulated as similar forms under certain constraints. Generalized mod- ularity further brings a variety of viewpoints and captures characteristics by applying different sampling methods. This thesis provides a flexible and effective network embed- ding method based on the generalized modularity. Experiments are conducted on real datasets to illustrate the effectiveness of our approach.

Contents 1
List of Figures 3
1 Introduction 4
2 The probabilistic framework for generalized modularity embedding 8
2.1 The generalized modularity of a sampled graph . . . . . . . . . . . . . . 8
2.2 Community detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Modularity embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3 Extensions and applications of generalized modularity embedding 20
3.1 Modularity embedding for data points in a semi-metric space . . . . . . . 20
3.2 Dimensionality reduction and connections to PCA . . . . . . . . . . . . . 23
3.3 The Laplacian eigenmaps as a special case of generalized modularity embedding
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4 Nonnegative embedding 27
4.1 A softmax embedding/clustering algorithm . . . . . . . . . . . . . . . . . 28
4.2 Adding an annealing parameter . . . . . . . . . . . . . . . . . . . . . . . 29
4.3 Hardmax decision: assigning point u to the cluster with the largest positive
expected covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5 Experiments 31
5.1 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.1.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.1.2 Baseline algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.3 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.4 Evaluation metrics . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.2 Results and analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2.1 Performance on Six Clusters . . . . . . . . . . . . . . . . . . . . . 36
5.2.2 Performance on SBM . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2.3 Performance on Amazon Network . . . . . . . . . . . . . . . . . . 40
5.2.4 Performance on Flickr Network . . . . . . . . . . . . . . . . . . . 40
6 Conclusion and future work 44

[1] M. Belkin and P. Niyogi, “Laplacian eigenmaps and spectral techniques for embed- ding and clustering,” in Advances in neural information processing systems, 2002, pp. 585–591.
[2] W. Li, “Modularity embedding,” in International Conference on Neural Information Processing. Springer, 2013, pp. 92–99.
[3] B. Perozzi, R. Al-Rfou, and S. Skiena, “Deepwalk: Online learning of social repre- sentations,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2014, pp. 701–710.
[4] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei, “Line: Large-scale infor- mation network embedding,” in Proceedings of the 24th International Conference on WorldWideWeb. InternationalWorldWideWebConferencesSteeringCommittee, 2015, pp. 1067–1077.
[5] S. Cao, W. Lu, and Q. Xu, “Grarep: Learning graph representations with global structural information,” in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 2015, pp. 891–900.
[6] A. Grover and J. Leskovec, “node2vec: Scalable feature learning for networks,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2016, pp. 855–864.
[7] P. Goyal and E. Ferrara, “Graph embedding techniques, applications, and perfor- mance: A survey,” arXiv preprint arXiv:1705.02801, 2017.
[8] X. Wang, P. Cui, J. Wang, J. Pei, W. Zhu, and S. Yang, “Community preserving network embedding.” in AAAI, 2017, pp. 203–209.
[9] Y.-A. Lai, C.-C. Hsu, W. H. Chen, M.-Y. Yeh, and S.-D. Lin, “Preserving prox- imity and global ranking for node embedding,” in Advances in Neural Information Processing Systems, 2017, pp. 5261–5270.
[10] J. B. Tenenbaum, V. De Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” science, vol. 290, no. 5500, pp. 2319–2323, 2000.
[11] D. Wang, P. Cui, and W. Zhu, “Structural deep network embedding,” in Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2016, pp. 1225–1234.
[12] M. Ou, P. Cui, J. Pei, Z. Zhang, and W. Zhu, “Asymmetric transitivity preserving graph embedding,” in Proceedings of the 22nd ACM SIGKDD international confer- ence on Knowledge discovery and data mining. ACM, 2016, pp. 1105–1114.
[13] C.-S. Chang, C.-J. Chang, W.-T. Hsieh, D.-S. Lee, L.-H. Liou, and W. Liao, “Rel- ative centrality and local community detection,” Network Science, vol. 3, no. 4, pp. 445–479, 2015.
[14] P. J. Mucha, T. Richardson, K. Macon, M. A. Porter, and J.-P. Onnela, “Community structure in time-dependent, multiscale, and multiplex networks,” Science, vol. 328, no. 5980, pp. 876–878, 2010.
[15] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Gossip algorithms: Design, analysis and applications,” in INFOCOM 2005. 24th Annual Joint Conference of the IEEE
Computer and Communications Societies. Proceedings IEEE, vol. 3. IEEE, 2005, pp. 1653–1664.
[16] J. Han, J. Pei, and M. Kamber, Data mining: concepts and techniques. Elsevier, 2011.
[17] M. E. Newman, “Fast algorithm for detecting community structure in networks,” Physical review E, vol. 69, no. 6, p. 066133, 2004.
[18] C.-S. Chang, D.-S. Lee, L.-H. Liou, S.-M. Lu, and M.-H. Wu, “A probabilistic framework for structural analysis and community detection in directed networks,” IEEE/ACM Transactions on Networking, vol. 26, no. 1, pp. 31–46, 2018.
[19] M. Newman, Networks: an introduction. OUP Oxford, 2009.
[20] J. Shi and J. Malik, “Normalized cuts and image segmentation,” Pattern Analysis
and Machine Intelligence, IEEE Transactions on, vol. 22, no. 8, pp. 888–905, 2000.
[21] S. X. Yu and J. Shi, “Multiclass spectral clustering,” in Computer Vision, 2003.
Proceedings. Ninth IEEE International Conference on. IEEE, 2003, pp. 313–319.
[22] I. Dhillon, Y. Guan, and B. Kulis, A unified view of kernel k-means, spectral cluster- ing and graph cuts. Computer Science Department, University of Texas at Austin, 2004.
[23] C.-S. Chang, W. Liao, Y.-S. Chen, and L.-H. Liou, “A mathematical theory for clus- tering in metric spaces,” IEEE Transactions on Network Science and Engineering, vol. 3, no. 1, pp. 2–16, 2016.
[24] U. Von Luxburg, “A tutorial on spectral clustering,” Statistics and computing, vol. 17, no. 4, pp. 395–416, 2007.
[25] C.-S. Chang, C.-T. Chang, D.-S. Lee, and L.-H. Liou, “K-sets+: a linear-time clus- tering algorithm for data points with a sparse similarity measure,” arXiv preprint arXiv:1705.04249, 2017.
[26] C.-H. Chang, C.-S. Chang, C.-T. Chang, D.-S. Lee, and P.-E. Lu, “Exponentially twisted sampling for centrality analysis and community detection in attributed net- works,” submitted for publication, 2018.
[27] D. A. Spielman, “Spectral graph theory and its applications,” in Foundations of Computer Science, 2007. FOCS’07. 48th Annual IEEE Symposium on. IEEE, 2007, pp. 29–38.
[28] C. M. Bishop, “Pattern recognition,” Machine Learning, vol. 128, pp. 1–58, 2006.
[29] C.-S. Chang, Performance guarantees in communication networks. Springer Science
& Business Media, 2012.
[30] X. Huang, J. Li, and X. Hu, “Accelerated attributed network embedding,” in SIAM
International Conference on Data Mining, 2017, pp. 633–641.
[31] J. Yang and J. Leskovec, “Defining and evaluating network communities based on ground-truth,” Knowledge and Information Systems, vol. 42, no. 1, pp. 181–213, 2015.
[32] L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of machine learning research, vol. 9, no. Nov, pp. 2579–2605, 2008.
[33] C.-H. Chang, C.-S. Chang, D.-S. Lee, and P.-E. Lu, “Exponentially twisted sampling for centrality analysis in attributed networks.”
[34] C.-T. Chang, “A unified framework for sampling, clustering and embedding data points in semi-metric spaces,” Master’s thesis, National Tsing Hua University, 2017.