帳號:guest(18.223.196.180)          離開系統
字體大小: 字級放大   字級縮小   預設字形  

詳目顯示

以作者查詢圖書館館藏以作者查詢臺灣博碩士論文系統以作者查詢全國書目
作者(中文):謝承翰
作者(外文):Hsieh, Cheng-Han
論文名稱(中文):糾刪碼(erasure code)架構之多雲儲存服務的資料擺放最佳化方法設計與實作
論文名稱(外文):Data Placement Optimization of Erasure Code-based Multi-Cloud Storage
指導教授(中文):周志遠
指導教授(外文):Chou, Chi-Yuan
口試委員(中文):金仲達
李哲榮
口試委員(外文):King, Chung-Ta
Lee, Che-Rung
學位類別:碩士
校院名稱:國立清華大學
系所名稱:資訊工程學系所
學號:104062510
出版年(民國):106
畢業學年度:105
語文別:英文
論文頁數:37
中文關鍵詞:糾刪碼線性規劃多雲端空間
外文關鍵詞:Erasure CodeLinear ProgrammingMulti-Cloud Storage
相關次數:
  • 推薦推薦:0
  • 點閱點閱:580
  • 評分評分:*****
  • 下載下載:30
  • 收藏收藏:0
隨著處存需求的逐年增加,雲端處存系統變得越來越重要。雲端供應商提供了巨大但是便宜的處存服務。人們或是公司可以在使用的同時減少了維護硬體及電力的費用。同時公司也能夠將其轉化為自己的服務。對於雲端儲存而言,糾刪碼被用來增進檔案可取得性而且有降低存取檔案時間的潛力。糾刪碼將檔案轉化成許多的片段。這些片段都比原本的檔案小,如此一來可以透過用平行下載的方式來降低下載時間。此外,這些片段也可以放在不同的儲存區域進而降低遇到發生區域毀壞的情況。然而,每個儲存區域都有不同的存取花費,儲存花費跟傳輸花費。有了這些議題,重點就在於如何選擇選擇候選儲存區域來擺放片段以滿足所有要求。在過去的研究中著重於單一的特性,並且不夠實際。在現實世界中需要考量的因素非常的多。在本篇論文中,我們使用了糾刪碼與線性規劃來找到最好的擺放方法。實驗顯示我們的方法可以省下有66%的金錢以及在效能上有50%的提升。
The cloud storage system has been popular recently due to the higher and higher demand of storage space. The cloud providers offer large but cheap storage service. People or companies use them and do not have to pay on hardware or electric utility. Companies can use these features to build its own storage service for benefit too. For cloud storage, erasure code can be used to improve data availability and have potential to reduce download time. Erasure code encode files into chunks and place them in different storage regions for higher availability. Besides, these chunks is smaller than the origin file that the download time can be reduced by using parallel downloading. These chunks can also improve the availability by placed at different regions for avoiding regions failure. However, each region owns different request cost, storage cost or even latency and bandwidth. Besides, users location can largely influence download latency and access cost. With these multiple issues, the main point is how to choose the candidate regions for chunks that can fulfill all requirements. In the past, most thesiss focus on specific features. However, the models from those research are not realistic enough. There are many aspects that we need to take into consideration for being closer to the real world. In this thesis, we propose the method with using erasure code and linear programming to include multiple requirements at the same time and find the best placement strategy. The experiment shows that our work can save money at most 66\% and have at most 50\% performance improvement.
List of Figures 4
1 Introduction 5
2 System Model 8
2.1 Architecture................................ 8 2.2 Assumption ................................ 9
3 Problem Formulation 10
3.1 NotationandAvailability ........................ 10 3.2 DecisionVariables............................. 12 3.3 EvaulatedVariables............................ 12 3.4 Constraints ................................ 13 3.5 ObjectiveFunctions............................ 14
4 Experiment Setup 15
5 Simulation Evaluation 18
5.1 CostMinimization ............................ 18 5.1.1 Popularity&NumberofFile................... 18 5.1.2 FaultToleranceLevel....................... 20 5.1.3 NumberofRegion ........................ 21
5.2 PerformanceMinimization ........................ 23 5.2.1 FaultToleranceLevel....................... 23 5.2.2 NumberofRegion ........................ 24
5.3 Summary ................................. 24
2
6 Real Testbed Experiments 26
6.1 Setup.................................... 26 6.2 DownloadTime.............................. 27 6.3 NumberofRequest&TotalCost .................... 28 6.4 ErasureCode(4,2)TimeProfiling.................... 29 6.5 Summary ................................. 30
7 Related work 31
7.1 TraditionalIssues............................. 31 7.2 ErasureCodeonMulti-cloud....................... 31
8 Conclusion Bibliography
33 34
[1] https://aws.amazon.com/tw/s3/pricing/.
[2] https://azure.microsoft.com.
[3] https://www.amazon.com.
[4] https://www.microsoft.com.
[5] H. Abu-Libdeh, L. Princehouse, and H. Weatherspoon. Racs: A case for cloud storage diversity. In Proceedings of the 1st ACM Symposium on Cloud Com- puting, SoCC ’10, pages 229–240, New York, NY, USA, 2010. ACM.
[6] A. Bessani, M. Correia, B. Quaresma, F. Andr ́e, and P. Sousa. Depsky: De- pendable and secure storage in a cloud-of-clouds. In Proceedings of the Sixth Conference on Computer Systems, EuroSys ’11, pages 31–46, New York, NY, USA, 2011. ACM.
[7] B. Calder, J. Wang, A. Ogus, N. Nilakantan, A. Skjolsvold, S. McKelvie, Y. Xu, S. Srivastav, J. Wu, H. Simitci, J. Haridas, C. Uddaraju, H. Khatri, A. Edwards, V. Bedekar, S. Mainali, R. Abbasi, A. Agarwal, M. F. u. Haq, M. I. u. Haq, D. Bhardwaj, S. Dayanand, A. Adusumilli, M. McNett, S. Sankaran, K. Mani- vannan, and L. Rigas. Windows azure storage: A highly available cloud storage service with strong consistency. In Proceedings of the Twenty-Third ACM Sym- posium on Operating Systems Principles, SOSP ’11, pages 143–157, New York, NY, USA, 2011. ACM.
[8] J. C. W. Chan, Q. Ding, P. P. C. Lee, and H. H. W. Chan. Parity logging with reserved space: Towards efficient updates and recovery in erasure-coded clus- tered storage. In Proceedings of the 12th USENIX Conference on File and Stor- age Technologies (FAST 14), pages 163–176, Santa Clara, CA, 2014. USENIX.
[9] H. Chang. Data replication management for geo-distributed cloud storage. In Master Thesis, National Tsing Hua University, 2015.
[10] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. J. Wainwright, and K. Ramchan- dran. Network coding for distributed storage systems. IEEE Transactions on Information Theory, 56(9):4539–4551, Sept 2010.
[11] A. G. Dimakis, K. Ramchandran, Y. Wu, and C. Suh. A survey on network codes for distributed storage. Proceedings of the IEEE, 99(3):476–489, March 2011.
[12] K. S. Esmaili, A. Chiniah, and A. Datta. Efficient updates in cross-object erasure-coded storage systems. In 2013 IEEE International Conference on Big Data, pages 28–32, Oct 2013.
[13] S. Frolund, A. Merchant, Y. Saito, S. Spence, and A. Veitch. A decentral- ized algorithm for erasure-coded virtual disks. In International Conference on Dependable Systems and Networks, 2004, pages 125–134, June 2004.
[14] S. Ghemawat, H. Gobioff, and S.-T. Leung. The google file system. In Pro- ceedings of the Nineteenth ACM Symposium on Operating Systems Principles, SOSP ’03, pages 29–43, New York, NY, USA, 2003. ACM.
[15] Y. Hu and D. Niu. Reducing access latency in erasure coded cloud storage with local block migration. In IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, pages 1–9, April 2016.
[16] S. Jiekak, A. M. Kermarrec, N. L. Scouarnec, G. Straub, and A. V. Kempen. Regenerating codes: A system perspective. In 2012 IEEE 31st Symposium on Reliable Distributed Systems, pages 436–441, Oct 2012.
[17] G. M. Kamath, N. Prakash, V. Lalitha, and P. V. Kumar. Codes with local regeneration. In 2013 IEEE International Symposium on Information Theory, pages 1606–1610, July 2013.
[18] O. Khan, R. Burns, J. Plank, W. Pierce, and C. Huang. Rethinking erasure codes for cloud file systems: Minimizing i/o for recovery and degraded reads. In Proceedings of the 10th USENIX Conference on File and Storage Technologies, FAST’12, pages 20–20, Berkeley, CA, USA, 2012. USENIX Association.
[19] O. T. Lee, S. D. M. Kumar, and P. Chandran. Erasure coded storage systems for cloud storage x2014; challenges and opportunities. In 2016 International Conference on Data Science and Engineering (ICDSE), pages 1–7, Aug 2016.
[20] P. Li, X. Jin, R. J. Stones, G. Wang, Z. Li, X. Liu, and M. Ren. Parallelizing degraded read for erasure coded cloud storage systems using collective commu- nications. In 2016 IEEE Trustcom/BigDataSE/ISPA, pages 1272–1279, Aug 2016.
[21] Y. Ma, T. Nandagopal, K. P. N. Puttaswamy, and S. Banerjee. An ensemble of replication and erasure codes for cloud file systems. In 2013 Proceedings IEEE INFOCOM, pages 1276–1284, April 2013.
[22] B. Mao, S. Wu, and H. Jiang. Improving storage availability in cloud-of-clouds with hybrid redundant data distribution. In 2015 IEEE International Parallel and Distributed Processing Symposium, pages 633–642, May 2015.
[23] S. Mu, K. Chen, P. Gao, F. Ye, Y. Wu, and W. Zheng. x00b5;libcloud: Pro- viding high available and uniform accessing to multiple cloud storages. In 2012 ACM/IEEE 13th International Conference on Grid Computing, pages 201–208, Sept 2012.
[24] D. S. Papailiopoulos and A. G. Dimakis. Locally repairable codes. IEEE Trans- actions on Information Theory, 60(10):5843–5855, Oct 2014.
[25] T. G. Papaioannou, N. Bonvin, and K. Aberer. Scalia: An adaptive scheme for efficient multi-cloud storage. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, pages 1–10, Nov 2012.
[26] N. Prakash, G. M. Kamath, V. Lalitha, and P. V. Kumar. Optimal linear codes with a local-error-correction property. In 2012 IEEE International Symposium on Information Theory Proceedings, pages 2776–2780, July 2012.
[27] K. Rashmi, P. Nakkiran, J. Wang, N. B. Shah, and K. Ramchandran. Having your cake and eating it too: Jointly optimal erasure codes for i/o, storage, and network-bandwidth. In 13th USENIX Conference on File and Storage Technolo- gies (FAST 15), pages 81–94, Santa Clara, CA, 2015. USENIX Association.
[28] K. V. Rashmi, N. B. Shah, K. Ramchandran, and P. V. Kumar. Regenerating codes for errors and erasures in distributed storage. In 2012 IEEE International Symposium on Information Theory Proceedings, pages 1202–1206, July 2012.
[29] A. S. Rawat, O. O. Koyluoglu, N. Silberstein, and S. Vishwanath. Optimal lo- cally repairable and secure codes for distributed storage systems. IEEE Trans- actions on Information Theory, 60(1):212–236, Jan 2014.
[30] M. Su, L. Zhang, Y. Wu, K. Chen, and K. Li. Systematic data placement opti- mization in multi-cloud storage for complex requirements. IEEE Transactions on Computers, 65(6):1964–1977, June 2016.
[31] H. Weatherspoon and J. D. Kubiatowicz. Erasure Coding Vs. Replication: A Quantitative Comparison, pages 328–337. Springer Berlin Heidelberg, Berlin, Heidelberg, 2002.
[32] Y. Xiang, T. Lan, V. Aggarwal, and Y. F. Chen. Optimizing differentiated latency in multi-tenant, erasure-coded storage. IEEE Transactions on Network and Service Management, 14(1):204–216, March 2017.
[33] Y. Zhu, J. Lin, P. P. C. Lee, and Y. Xu. Boosting degraded reads in het- erogeneous erasure-coded storage systems. IEEE Transactions on Computers, 64(8):2145–2157, Aug 2015.
 
 
 
 
第一頁 上一頁 下一頁 最後一頁 top
* *