隨著物聯網 (Internet of Things, IoT) 和人工智慧 (Artificial Intelligence, AI) 的進 步，推動了具有協作性質的社交物聯網 (Social IoT, SIoT) 的興起。 對於社交物聯網 而言，在執行數據分析或協作事件處理時，防止隱私洩露是至關重要。 為此，聯 合學習（Federated learning, FL）已被視為分佈式資料處理的標準，通過本地端設 備使用自身收集的資料訓練模型，取代集中式學習來防止敏感數據洩露。 在本論 文中，我們探討了社交物聯網設備之訓練集群建立問題 (SIoT device Training Group Construction)，並命名為 STGC 。 我們在分層式聯邦學習 (Hierarchical FL) 的環境 中，通過差分隱私 (Differential Privacy, DP) 來滿足設備的隱私需求以及共同考慮數 據質量、平衡性和標籤覆蓋來選擇合適的社交物聯網設備並將其分群進行模型訓 練，同時最小化社交物聯網設備的總計算和通信成本、招聘私有設備參與訓練的 雇用成本和使用DP所產生的隱私保護成本。 此外，證明了 STGC 是 NP-Hard的問 題，而且無法設計出一演算法能夠在多項式時間內使用任何因子近似最佳解，除 非 P = NP。 為此，我們設計了一種具有隱私感知的社交物聯網設備選擇、聚類與 路由 (Privacy-aware SIoT device Selection, Clustering, and Routing) 的演算法，並命名 為PSSCR ，其中結合覆蓋效率指標 (Coverage Efficiency Indicator, CEI )、數據平衡 感知之雙重調整 (Data Balance-aware Dual Adjustment, DBDA)和隱私感知重新路由 (Privacy-Aware Rerouting, PAR) 的想法來選擇和分組社交物聯網設備，並決定每個 集群中的傳輸路徑以及本地聚合器(Aggregator, AG)。 最後，本論文通過兩個真實 的資料集 1) MNIST，2) Fashion-MNIST 評估PSSCR的效能。模擬結果表明 PSSCR 在總成本、模型準確性和訓練時間方面皆優於其他的演算法，且可以有效地減少 60% 以上的總成本和降低六成的收斂時間。

The Advances in the Internet of Things (IoT) and Artificial Intelligence (AI) have prompted the emerging of Social IoT (SIoT) with collaborative social relations. For SIoT, it is crucial to prevent the risk of privacy leakage such as data analysis, collaborative events. To this end, Federated learning (FL) has been regarded as the distributed learning paradigm to protect sensitive data via locally model training on the device side. On the other hand, Hierarchical FL (HFL) clusters devices into multiple local training groups to reduce communication overheads by local aggregation. In this thesis, we explore SIoT device Training Group Construction (STGC) to select and cluster SIoT devices for training model in HFL environment with privacy requirement via Differential privacy (DP) and joint consideration of the data quality, coverage, and balance requirements, while minimizing the total computation and communication costs of SIoT devices, hiring costs for hiring private SIoT devices to participate in the training task, and privacy cost for exploiting DP. We prove that STGC is NP-hard and there is no algorithm with a finite approximation ratio for STGC unless P = NP. Then, we design an algorithm, named Privacy-aware SIoT device Selection, Clustering, and Routing (PSSCR), with the ideas of Coverage Efficiency Indicator, Data Balance-aware Dual Adjustment, and Privacy-Aware Rerouting 1) to choose and cluster SIoT devices with greater data coverage and quality and 2) to determine the local aggregator and SIoT device routing to minimize the total communication and privacy costs, by carefully examining the social relation between each pair of SIoT devices in each cluster. Simulation results manifest that the proposed algorithm PSSCR outperforms other algorithms regarding the privacy cost, model accuracy, and training rounds and effectively reduces more than 60% of the total cost and convergence time on the two real datasets, MNIST and Fashion-MNIST.

1. Introduction … 1
2. Related Work … 5
2.1 Social Internet of Things with Artificial Intelligence … 5
2.2 Decentralized Learning Framework and Analysis … 6
2.3 Federated Learning with Privacy Preservation … 7
3. The SIoT device Training Group Construction Problem … 8
3.1 System Model … 8
3.2 Problem Formulation … 11
3.3 Summary … 15
4. Privacy-aware SIoT device Selection, Clustering, and Routing (PSSCR) … 16
4.1 Privacy-aware SIoT device Selection, Clustering, and Routing … 16
4.2 Complexity Analysis … 23
4.3 Summary … 24
5. Performance Evaluation … 25
5.1 Simulation Setup … 25
5.2 Different Number of SIoT devices and SIoT Degree … 26
5.3 Effect of Additional Training Cost … 27
5.4 Effect of Data Balance, Privacy Intensity, and Data Quantity on Accuracy … 28
5.5 Training Time by Different Privacy Intensity and Data Quantity … 30
5.6 Summary … 30
6. Conclusions … 31

[1] C.-H. Wang, J.-J. Kuo, D.-N. Yang, and W.-T. Chen, “Collaborative social internet of things in mobile edge networks,” IEEE Internet of Things Journal, vol. 7, no. 12, pp. 11 473–11 491, 2020.
[2] P. Dong, J. Ge, X. Wang, and S. Guo, “Collaborative edge computing for social internet of things: Applications, solutions, and challenges,” IEEE Transactions on Computational Social Systems, pp. 1–11, 2021.
[3] X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” in Proceedings of International Conference on Learning Representations (ICLR), 2020.
[4] S. R. Pandey, N. H. Tran, M. Bennis, Y. K. Tun, A. Manzoor, and C. S. Hong, “A crowdsourcing framework for on-device federated learning,” IEEE Transactions on Wireless Communications, vol. 19, no. 5, pp. 3241–3256, 2020.
[5] Q. Wu, X. Chen, Z. Zhou, and J. Zhang, “Fedhome: Cloud-edge based personalized federated learning for in-home health monitoring,” IEEE Transactions on Mobile Computing, pp. 1–1, 2020.
[6] W. Y. B. Lim, S. Garg, Z. Xiong, D. Niyato, C. Leung, C. Miao, and M. Guizani, “Dynamic contract design for federated learning in smart healthcare applications,” IEEE Internet of Things Journal, pp. 1–1, 2020.
[7] E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge ai: On-demand accelerating deep neural network inference via edge computing,” IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 447–457, 2020.
[8] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “CommunicationEfficient Learning of Deep Networks from Decentralized Data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 54, Apr. 2017, pp. 1273–1282.
[9] L. Liu, J. Zhang, S. Song, and K. B. Letaief, “Client-edge-cloud hierarchical federated learning,” in Proceedings of IEEE International Conference on Communications (ICC), 2020, pp. 1–6.
[10] N. Mhaisen, A. Awad, A. Mohamed, A. Erbad, and M. Guizani, “Optimal user-edge assignment in hierarchical federated learning based on statistical properties and network topology constraints,” IEEE Transactions on Network Science and Engineering, pp. 1–1, 2021.
[11] Z. Wang, H. Xu, J. Liu, H. Huang, C. Qiao, and Y. Zhao, “Resource-efficient federated learning with hierarchical aggregation in edge computing,” in Proceedings of IEEE Conference on Computer Communications (INFOCOM), 2021, pp. 1–10.
[12] F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek, “Robust and communicationefficient federated learning from non-i.i.d. data,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 9, pp. 3400–3413, 2020.
[13] Y. Zhang, C. Jiang, L. Song, M. Pan, Z. Dawy, and Z. Han, “Incentive mechanism for mobile crowdsourcing using an optimized tournament model,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 4, pp. 880–892, 2017.
[14] K.-Y. Chen, C.-H. Wang, S.-H. Chiang, D.-N. Yang, W.-T. Chen, and J.-P. Sheu, “Collaboration between social internet of things and mobile users for accuracy-aware detection,” in Proceedings of IEEE International Conference on Communications (ICC), 2021, pp. 1–6.
[15] A. Hard, K. Rao, R. Mathews, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage, “Federated learning for mobile keyboard prediction,” CoRR, vol. abs/1811.03604, 2018. [Online]. Available: http://arxiv.org/abs/1811. 03604.
[16] “Jamaica eye.” [Online]. Available: https://jamaicaeye.gov.jm/.
[17] Y. Yang, L. Wu, G. Yin, L. Li, and H. Zhao, “A survey on security and privacy issues in internet-of-things,” IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1250–1258, 2017.
[18] M. A. Al-Garadi, A. Mohamed, A. K. Al-Ali, X. Du, I. Ali, and M. Guizani, “A survey of machine and deep learning methods for internet of things (iot) security,” IEEE Communications Surveys Tutorials, vol. 22, no. 3, pp. 1646–1685, 2020.
[19] M. Song, Z. Wang, Z. Zhang, Y. Song, Q. Wang, J. Ren, and H. Qi, “Analyzing userlevel privacy attack against federated learning,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 10, pp. 2430–2444, 2020.
[20] K. Wei, J. Li, M. Ding, C. Ma, H. Su, B. Zhang, and H. V. Poor, “User-level privacypreserving federated learning: Analysis and performance optimization,” IEEE Transactions on Mobile Computing, pp. 1–1, 2021.
[21] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.
[22] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. S. Quek, and H. V. Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454–3469, 2020.
[23] E. Bagdasaryan, O. Poursaeed, and V. Shmatikov, “Differential privacy has disparate impact on model accuracy,” in Proceedings of Advances in Neural Information Processing Systems (NeurIPS), 2019.
[24] L. Cui, Y. Qu, S. Yu, L. Gao, and G. Xie, “A trust-grained personalized privacypreserving scheme for big social data,” in Proceedings of IEEE International Conference on Communications (ICC), 2018, pp. 1–6.
[25] M. Chen, H. V. Poor, W. Saad, and S. Cui, “Convergence time optimization for federated learning over wireless networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 4, pp. 2457–2471, 2021.
[26] H. Zhu and Y. Jin, “Multi-objective evolutionary federated learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 4, pp. 1310–1322, 2020.
[27] P. Hand and V. Voroninski, “Global guarantees for enforcing deep generative priors by empirical risk,” IEEE Transactions on Information Theory, vol. 66, no. 1, pp. 401– 418, 2020.
[28] D. Wang and J. Xu, “Differentially private empirical risk minimization with smooth non-convex loss functions: A non-stationary view,” in Proceedings of AAAI Conference on Artificial Intelligence, 2019.
[29] H. Li, K. Ota, and M. Dong, “Learning iot in edge: Deep learning for the internet of things with edge computing,” IEEE Network, vol. 32, no. 1, pp. 96–101, 2018.
[30] Z. Guo, K. Yu, Y. Li, G. Srivastava, and J. C.-W. Lin, “Deep learning-embedded social internet of things for ambiguity-aware social recommendations,” IEEE Transactions on Network Science and Engineering, pp. 1–1, 2021.
[31] G. Delnevo, R. Girau, C. Ceccarini, and C. Prandi, “A deep learning and social iot approach for plants disease prediction toward a sustainable agriculture,” IEEE Internet of Things Journal, pp. 1–1, 2021.
[32] T. Han, K. Muhammad, T. Hussain, J. Lloret, and S. W. Baik, “An efficient deep learning framework for intelligent energy management in iot networks,” IEEE Internet of Things Journal, vol. 8, no. 5, pp. 3170–3179, 2021.
[33] W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y.-C. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” IEEE Communications Surveys Tutorials, vol. 22, no. 3, pp. 2031–2063, 2020.
[34] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” in Proceedings of Machine Learning and Systems(MLSys), vol. 2, 2020, pp. 429–450.
[35] Y. Tu, Y. Ruan, S. Wagle, C. G. Brinton, and C. Joe-Wong, “Network-aware optimization of distributed learning for fog computing,” in Proceedings of IEEE Conference on Computer Communications (INFOCOM), 2020, pp. 2509–2518.
[36] T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” in Proceedings of IEEE International Conference on Communications (ICC), 2019, pp. 1–7.
[37] S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “Adaptive federated learning in resource constrained edge computing systems,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, pp. 1205–1221, 2019.
[38] J. Mills, J. Hu, and G. Min, “Communication-efficient federated learning for wireless edge intelligence in iot,” IEEE Internet of Things Journal, vol. 7, no. 7, pp. 5986– 5994, 2020.
[39] A. Triastcyn and B. Faltings, “Federated learning with bayesian differential privacy,” in Proceedings of IEEE International Conference on Big Data (Big Data), 2019, pp. 2587–2596.
[40] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1175–1191.
[41] M. Nitti, R. Girau, and L. Atzori, “Trustworthiness management in the social internet of things,” IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 5, pp. 1253–1266, 2014.
[42] I.-R. Chen, F. Bao, and J. Guo, “Trust-based service management for social internet of things systems,” IEEE Transactions on Dependable and Secure Computing, vol. 13, no. 6, pp. 684–696, 2016.
[43] C. Dwork and A. Roth, “The algorithmic foundations of differential privacy,” Found. Trends Theor. Comput. Sci., vol. 9, no. 3, pp. 211–407, 2014.
[44] G. Xu, B. Liu, L. Jiao, X. Li, M. Feng, K. Liang, L. Ma, and X. Zheng, “Trust2privacy: A novel fuzzy trust-to-privacy mechanism for mobile social networks,” IEEE Wireless Communications, vol. 27, no. 3, pp. 72–78, 2020.
[45] Y. Chen, Y. Ning, M. Slawski, and H. Rangwala, “Asynchronous online federated learning for edge devices with non-iid data,” in Proceedings of IEEE International Conference on Big Data (Big Data), 2020, pp. 15–24.
[46] M. Duan, D. Liu, X. Chen, R. Liu, Y. Tan, and L. Liang, “Self-balancing federated learning with global imbalanced data in mobile systems,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 1, pp. 59–71, 2021.
[47] N. Yoshida, T. Nishio, M. Morikura, K. Yamamoto, and R. Yonetani, “Hybrid-fl for wireless networks: Cooperative learning mechanism using non-iid data,” in Proceedings of IEEE International Conference on Communications (ICC), 2020, pp. 1–7.
[48] S. Wang, Y.-C. Wu, M. Xia, R. Wang, and H. V. Poor, “Machine intelligence at the edge with learning centric power allocation,” IEEE Transactions on Wireless Communications, vol. 19, no. 11, pp. 7293–7308, 2020.
[49] Z. Chen, D. Li, M. Zhao, S. Zhang, and J. Zhu, “Semi-federated learning,” in Proceedings of IEEE Wireless Communications and Networking Conference (WCNC), 2020, pp. 1–6.
[50] Z. Cui, X. Jing, P. Zhao, W. Zhang, and J. Chen, “A new subspace clustering strategy for ai-based data analysis in iot system,” IEEE Internet of Things Journal, vol. 8, no. 16, pp. 12 540–12 549, 2021.
[51] N. H. Tran, W. Bao, A. Zomaya, M. N. H. Nguyen, and C. S. Hong, “Federated learning over wireless networks: Optimization model design and analysis,” in Proceedings of IEEE Conference on Computer Communications (INFOCOM), 2019, pp. 1387–1395.
[52] Y. LeCun, C. Cortes, and C. J. C. Burges, “The mnist database,” accessed Aug. 12, 2021. [Online]. Available: http://yann.lecun.com/exdb/mnist/.
[53] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms,” CoRR, vol. abs/1708.07747, 2017. arXiv: 1708.07747. [Online]. Available: http://arxiv.org/abs/1708.07747.