隨著分散式計算系統的革新，對於高效能和大規模的計算需求日漸增加。雲端應用服務和使用者們使用雲端儲存時，希望儲存服務能同時提供安全性、可用性、可靠性與高效能。許多不同的雲端服務供應商與檔案儲存系統，都是用著不同的方式滿足使用者的需求；然而依據CAP理論，各家儲存供應商勢必對各式需求有所取捨，無法同時滿足所有期望。除此之外，各儲存供應商並沒有統一的介面，亦不存在不同儲存服務間有效率的檔案轉移工具，因而導致嚴重的廠商鎖定效應。於此，利用聯邦分散式儲存系統統合異質的儲存資源勢在必行。在這篇論文中，我們設計了能提供統一接口的分散式聯邦雲端儲存管理系統，並利用一個帶權重的仲介模型，有效考量到使用者需求、檔案應用特徵以及儲存系統特性，進行檔案與儲存系統的耦合。最後，我們在31個節點上，利用真實的檔案系統工作負載和紀錄進行測試，實驗結果顯示我們的作法帶來了35%~125%的效能改進。

As the evolution of distributed computing systems, the need of high performance and large-scale computing is getting increase. Cloud applications or users may demand a storage system with security, availability, performance, and reliability. A variety of public cloud storage providers and private storage systems tried to meet the requirements by applying different approaches. However, no one of cloud storage providers is able to fulfil every requirement of expectations at the same time because of the CAP theorem. In addition, cloud storage providers usually offer different APIs for the access so as to lead users face an issue about vender lock-in. Leveraging heterogeneous storage resources in federated cloud storages is a prospective manner to solve these issues. In this paper, we focus on proposing a federated cloud storage system with a uniform interface. Based on a prioritized brokerage model, a resource brokerage is further presented to benefit the matchmaking with the considerations of user requirements, file classifications, and storage characteristics. We evaluate the system performance with real traces and workloads on 31 nodes. Experimental results show that our approach improve 35%~125% performance gains.

Contents
1 Introduction
2 Related Work
3 Federated Cloud Storage System
3.1
Overview
3.2
System Architecture and Implementation
3.2.1
Client and User API
3.2.2
Storage Providers
3.2.3
Metadata Services
3.2.4
Management Services
3.3
File Classification and Storage Characteristics
3.3.1
File Classification
3.3.2
Storage Characteristics
3.4
User Requirement
3.4.1
Attributes in User Requirement
3.4.2
Constraint
3.4.3
Priority
3.4.4
Preference
3.4.5
Hinting
3.4.6
Example
3.5
Brokerage
3.5.1
Matchmaking
3.5.2
Filter
3.5.3
Prioritize Ranking
3.5.4
Resource Selection Policy
3.6
Resource Contention
3.6.1
Admission Control
3.6.2
Resource-Reservation
3.6.3
Worst-Fit
4 Workload Analysis
4.1
Workload of FTP Trace Analysis
4.1.1
Characteristics of Folders
4.1.2
Characteristics of Files
4.2
Workload Generation
4.2.1
Generators
4.2.2
Synthetic Workload
5 Simulation
5.1
Simulation Environment
5.1.1
Workload Generation
5.1.2
Resource Pool Generation
5.1.3
Requirement Generation
5.2
Result
5.2.1
Matchmaking
5.2.2
Quality Comparison
5.2.3
Resource Contention
5.2.4
Response Time
6 Experimental Evaluation
6.1
Evaluation Environment
6.1.1
Workload Generation with FABAN
6.1.2
Configuration of The Test Bay
6.1.3
Requirement Generation
6.2
Result
6.2.1
Matchmaking
6.2.2
Throughput
6.2.3
System Overhead
7 Conclusions
Reference

[1] Google Cloud Storage, https://cloud.google.com/products/cloud-storage
[2] Amazon AWS, http://aws.amazon.com/
[3] Microsoft Azure, http://www.windowsazure.com
[4] Rackspace Cloud Files, http://www.rackspace.com/cloud/public/files/
[5] Oracle CloudWorld, https://cloud.oracle.com
[6] Google Apps Incident Report, http://static.googleusercontent.com/external_content/untrusted_dlcp/www.google.com/en/us/appsstatus/ir/ej73a82sddnv7fb.pdf
[7] Amazon outage, http://edition.cnn.com/2012/10/22/tech/web/reddit-goes-down/index.html
[8] Microsoft online services hit by major failure, http://www.bbc.co.uk/news/technology-14851455
[9] Amazon’s Cloud Crash Disaster Permanently Destroyed Many Customers’ Data, http://articles.businessinsider.com/2011-04-28/tech/29958976_1_amazon-customer-customers-data-data-loss
[10] H. Abu-Libdeh, L. Princehouse, and H. Weatherspoon. RACS: A Case forCloud Storage Diversity. In Proc. of ACM SoCC, 2010.
[11] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G.Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. “A View of Cloud Computing,” Communications of the ACM, 53(4):50–58, 2010.
[12] Oracle Lustre, http://wiki.lustre.org/index.php/Main_Page
[13] Sage A. Weil, Scott A. Brandt, Ethan L. Miller, Darrell D. E. Long, andCarlos Maltzahn, “Ceph: A Scalable, High-Performance Distributed File System”, Proceedings of the 7th Symposium on Operating Systems Design and Implementation, OSDI, 2006.
[14] MooseFS, http://www.moosefs.org/
[15] Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung, “The Google File System,” SIGOPS - Operating Systems Review, 37(5) (2003), pp. 29–43.
[16] K. Shvachko, H. Huang, S. Radia, R. Chansler, “The hadoop distributed file system,“ in: 26th IEEE (MSST2010) Symposium on Massive Storage Systems and Technologies, May, 2010.
[17] Amazon S3, http://aws.amazon.com/s3/
[18] OpenStack Swift, http://openstack.org/projects/storage/
[19] GlusterFS, http://www.gluster.org/
[20] Bermbach, D.; Klems, M.; Tai, S.; Menzel, M.; , “MetaStorage: A Federated Cloud Storage System to Manage Consistency-Latency Tradeoffs,” Cloud Computing (CLOUD), 2011 IEEE International Conference on , vol., no., pp.452-459, 4-9 July 2011.
[21] Gifford, David K. (1979). “Weighted voting for replicated data,” SOSP ’79: Proceedings of the seventh ACM symposium on Operating systems principles. pp. 150–162.
[22] Alysson Bessani, Miguel Correia, Bruno Quaresma, Fernando Andr ́ e and Paulo Sousa. “DepSky: Dependable and Secure Storage in a Cloud-of-Cloud,” In 6th ACM SIGOPS/EuroSys European Systems Conference (EuroSys’11), pp 31-45. April 2011.
[23] Y. Hu et al., “NCCloud: Applying Network Coding for the Storage Repair in a Cloud-of-Clouds,” USENIX FAST, San Jose, 2012.
[24] Cheng Huang, Huseyin Simitci, Yikang Xu, Aaron Ogus, Brad Calder, Parikshit Gopalan, Jin Li, and Sergey Yekhanin. 2012. “Erasure coding in windows azure storage,” In Proceedings of the 2012 USENIX conference on Annual Technical Conference (USENIX ATC’12)
[25] Rajkumar Buyya, Rajiv Ranjan, and Rodrigo N. Calheiros. 2010. “InterCloud: utility-oriented federation of cloud computing environments for scaling of application services,” In ICA3PP’10, Berlin, 13-31.
[26] M. Mesnier, G. R. Ganger, and E. Riedel. “Object-based Storage”. Communications Magazine, 41(8):84–90. IEEE, August 2003.
[27] Percentage calculation in Availability, http://en.wikipedia.org/wiki/High_availability#Percentage_calculation
[28] Ruoming Pang , Vern Paxson, “A high-level programming environment for packet trace anonymization and transformation”, In Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, August 25-29, 2003, Karlsruhe, Germany
[29] George K. Zipf(1949), “Human Behaviour and the Principle of Least-Effort”, Addison-Wesley Press
[30] FABAN, http://faban.org/