雲端環境中共享實體資源的特性大大地提高了資源使用率，也成為節能以及節省成本的有效方法。而虛擬化技術是使實體資源可以共享的關鍵技術。但是虛擬化以及資源共享的計算模式會對系統效能造成的影響，至今仍無一個有系統的方法或工具可以分析它。這使得雲端環境上的效能預測以及保障效能的動態計算資源供應系統在實作上變得十分困難。此外，在雲端系統中經常存在實體資源的異質性，大大地加深了效能預測與管理的複雜度。
本論文提出了一套可自動化快速評估雲端平台效能的工具，用以評估虛擬化 (Virtualization)與資源共享(Consolidation)對於應用程式在雲端平台執行時的效能影響，主要功能是分析三種運算基本單元：CPU、Memory、Disk I/O在虛擬化平台上效能的變化，這樣分開來評估的方法可以更準確的看出雲端平台的優缺點。此外，為了使這樣的效能評估能夠更有效率，這個工具利用雲端平台的特性，將所有效能評估平行化，相較於傳統的效能評估軟體，本工具能夠有系統，準確，且快速的報告虛擬化與資源共享對效能的影響。
我們在數台雲端伺服器上進行了最快速以及最複雜的實驗，並且比較了他們數據上的差異。根據最複雜測試的結果，我們對此雲端平台詳細地分析效能趨勢，在Disk I/O的部分，也包括在不同映像檔格式以及SSD下的效能分析。然後我們針對最複雜實驗以平行化方式重新測試，並且與未平行化的結果相比較。實驗結果顯示最快速的結果可以準確地表現雲端平台最普遍的效能，而平行化測試的結果亦達到了預期的準確度。此外，我們還對每種資源類型的工作負載分析其效能趨勢及虛擬化的效能瓶頸。

Server consolidation in cloud environment largely enhances the resource utilization, and it also become a good solution to save energy and cost. Virtualization technology is a key technique to realize server consolidation. However, virtualization and server consolidation may cause some problems on system performance, and there are still not any systematic methods or tools for analyzing them. This makes it pretty difficult to predict performances for a cloud environment and to implement a performance guaranteed dynamic provisioning system of computing resources. By the way, there are often heterogeneities of physical resources existed in cloud environments, and they largely increase the complexities of performance prediction and management.
In this paper, we propose a suite of fast automated benchmarking tools for cloud platforms, which can be used to evaluate how virtualization and server consolidation effect the performances of applications running on cloud platforms. Its main functionalities are analyze the performance variations of CPU, memory, and disk I/O for virtualization platforms. We can more precisely see the goods and bad of a cloud platform by evaluating those resources separately. Furthermore, for enhancing the efficiency of those performance evaluations, the tools parallelize each performance evaluation by utilizing the property of cloud platforms. Comparing with traditional benchmarking software, the tools can systematically, precisely, and quickly report how virtualization and server consolidation effects the performances of applications.
We conduct the fastest and most complicated experiments on several of cloud servers and compare the differences among their results. According to the results of most complicated experiments, we analyze the performances of this cloud platform. In profiling disk I/O performances, we also conduct the experiments and analyze its performances under different image formats and SSD. Finally, we redo the most complicated experiments by parallel profiling. To verify the precision of parallel profiling, we compare its results with original results. According to the experiment results, the results of the fastest experiments can precisely show the most general performances for a cloud platform, and that of parallel profiling also achieve the expected precision. Besides, we also analyze the performance tendencies and virtualization overheads for the workloads of each resource type.

中文摘要 I
ABSTRACT II
TABLE OF CONTENTS IV
TABLE OF FIGURES VI
TABLE OF TABLES VIIII
CHAPTER 1 INTRODUCTION 1
CHAPTER 2 RELATED WORKS 6
CHAPTER 3 AUTOMATIC BENCHMARKING TOOLS FOR CLOUD PLATFORMS 10
3.1 BENCHMARKS IN AUTOBTC 10
3.1.1 CPU benchmark 11
3.1.2 Memory benchmark 13
3.1.3 DISK I/O benchmark 15
3.2 SYSTEM ARCHITECTURES IN AUTOBTC 19
3.2.1 Virtualization overhead profiling 21
3.2.2 Consolidation overhead profiling 22
3.3 WORKFLOW CHARTS IN AUTOBTC 25
3.3.1 Virtualization overhead profiling 25
3.3.2 Consolidation overhead profiling 26
3.4 AUTOMATIC BENCHMARKING METHODS 27
3.4.1 CPU Virtualization Overhead 28
3.4.2 Memory Virtualization Overhead 29
3.4.3 Disk I/O Virtualization Overhead 31
3.4.4 Workload Consolidation Overhead 32
CHAPTER 4 PARRELL AUTOBTC 36
4.1 SYSTEM ARCHITECTURES 36
4.2 SYSTEM WORKFLOWS 40
4.3 PARALLEL SCHEDULER 42
CHAPTER 5 EXPERIMENTAL RESULTS 43
5.1 EXPERIMENTAL SETTINGS AND ENVIRONMENT 43
5.2 COMPARISON WITH TRADITIONAL BENCHMARKING TOOLS 44
5.2.1 CPU Virtualization Overhead 44
5.2.2 Memory Virtualization Overhead 45
5.2.3 Disk I/O Virtualization Overhead 47
5.3 PRECISION COMPARISON AMONG SIMPLE MODE AND ADVANCED MODE 58
5.4 PERFORMANCE OF PARALLEL AUTOBTC 61
CHAPTER 6 CONCLUSION 63
REFERENCE 64

[1] Umeno, Hidenori; Parayno, M.L.C.; Teramoto, K.; Kawano, M.; Inamasu, H.; Enoki, S.; Kiyama, M.; Aoyama, T.; Fukunaga, T., "Performance Evaluation on Server Consolidation Using Virtual Machines," SICE-ICASE, 2006. International Joint Conference , vol., no., pp.2730,2734, 18-21 Oct. 2006
[2] Ramanathan, S.; Goel, S.; Alagumalai, S., "Comparison of Cloud database: Amazon's SimpleDB and Google's Bigtable," Recent Trends in Information Systems (ReTIS), 2011 International Conference on , vol., no., pp.165,168, 21-23 Dec. 2011
[3] http://www.cs.virginia.edu/stream/ref.html
[4] http://www.coker.com.au/bonnie++/readme.html
[5] http://pic.dhe.ibm.com/infocenter/sonasic/sonas1ic/index.jsp?topic=%2Fcom.ibm.sonas.doc%2Fmng_t_filesys_create_blocksize.html
[6] http://docs.oracle.com/cd/E11882_01/server.112/e17110/limits002.htm
[7] Reig, G.; Alonso, J.; Guitart, J., "Prediction of Job Resource Requirements for Deadline Schedulers to Manage High-Level SLAs on the Cloud," Network Computing and Applications (NCA), 2010 9th IEEE International Symposium on , vol., no., pp.162,167, 15-17 July 2010
[8] Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. “Benchmarking cloud serving systems with YCSB”. In Proceedings of the 1st ACM symposium on Cloud computing (SoCC '10). ACM, New York, NY, USA, 143-154.
[9] Gunarathne, T.; Tak-Lon Wu; Qiu, J.; Fox, G., "MapReduce in the Clouds for Science," Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on , vol., no., pp.565,572, Nov. 30 2010-Dec. 3 2010
[10] http://www.techhormone.com/top-10-cloud-services-for-everyone/
[11] Dawei Jiang, Beng Chin Ooi, Lei Shi, and Sai Wu. 2010. The performance of MapReduce: an in-depth study. Proc. VLDB Endow. 3, 1-2 (September 2010), 472-483.
[12] http://www.iozone.org/
[13] Paolo Bientinesi, Roman Iakymchuk, Jeff Napper. 2010. HPC on Competitive Cloud Resources. B. Furht, A. Escalante (eds.), Handbook of Cloud Computing.

[14] G. Mc Evoy and B. Schulze. 2011. Understanding scheduling implications for scientific applications in clouds. In Proceedings of the 9th International Workshop on Middleware for Grids, Clouds and e-Science (MGC '11), Bruno Schulze, Omer Rana, and Edmundo Madeira (Eds.). ACM, New York, NY, USA, , Article 3 , 6 pages.
[15] http://gizmodo.com/5990589/the-best-way-to-store-your-stuff-in-the-cloud
[16] Luis M. Vaquero, Luis Rodero-Merino, and Rajkumar Buyya. 2011. Dynamically scaling applications in the cloud. SIGCOMM Comput. Commun. Rev. 41, 1 (January 2011), 45-52.
[17] http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol
[18] http://docs.openstack.org/developer/nova/devref/rpc.html
[19] http://www.rabbitmq.com/
[20]http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-transport-v1.0.html#doc-flow-control
[21]http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-transport-v1.0.html#doc-idp489648
[22]http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-security-v1.0.html#section-security-layers
[23]http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-transport-v1.0.html#section-transport
[24] http://libvirt.org/
[25] http://en.wikipedia.org/wiki/Libvirt
[26] http://libguestfs.org/
[27] http://en.wikipedia.org/wiki/Physical-to-Virtual
[28] http://www.ohloh.net/p/virt-v2v
[29] http://matplotlib.org/index.html
[30] http://en.wikipedia.org/wiki/RabbitMQ
[31] http://en.wikipedia.org/wiki/Message-oriented_middleware
[32] Lian Yu, Wei-Tek Tsai, Xiangji Chen, Linqing Liu, Yan Zhao, Liangjie Tang, and Wei Zhao. 2010. Testing as a Service over Cloud. In Proceedings of the 2010 Fifth IEEE International Symposium on Service Oriented System Engineering (SOSE '10). IEEE Computer Society, Washington, DC, USA, 181-188.
[33] Luszczek, P., Dongarra, J., Koester, D., Rabenseifner, R., Lucas, B., Kepner, J., McCalpin, J., Bailey, D., Takahashi, D. "Introduction to the HPC Challenge Benchmark Suite," March, 2005.
[34] http://www.spec.org/cpu2006/
[35] http://icl.cs.utk.edu/hpcc/index.html
[36] http://www.netlib.org/benchmark/hpl/index.html
[37] http://www.spec.org/spec/
[38] http://www.iozone.org/docs/IOzone_msword_98.pdf
[39] http://www.spec.org/virt_sc2013/
[40] http://www.spec.org/virt_sc2013/docs/SPECvirt_ClientHarnessUserGuide.html
[41] IDC. Virtualization Market Accelerates Out of the Recession as Users Adopt
“Virtualize First” Mentality, According to IDC, April 2010.
[42] IT world, The IDG Network. Gartner’s data on energy consumption, virtualization,cloud.http://www.itworld.com/green-it/59328/gartners-data-energy-consumption-virtualization-cloud, 2008.
[43] Hien Nguyen Van; Tran, F.D.; Menaud, J.-M., "SLA-Aware Virtual Resource Management for Cloud Infrastructures," Computer and Information Technology, 2009. CIT '09. Ninth IEEE International Conference on , vol.1, no., pp.357,362, 11-14 Oct. 2009