越來越多軟體開發者和資訊技術人員擁抱容器技術，容器技術將需要的資源和設定封裝成映像檔，經由映像檔產生的容器可避免許多兼容性的問題。容器的叢集化工具配合自動化開發和複雜的系統的需求也是勢在必行。
在這篇論文中，我們提出了透過checkpoint和restore技術加強Docker Swarm叢集。主要有兩點：高可用性系統和容器遷移。我們使用多版本checkpoint可以針對特定的容器定期設定checkpoint儲存至雲端儲存空間，若叢集中的節點遇到不正常的離線時，可以及時回復最近的容器狀態到健康的節點上。另外，透過Docker Swarm的分配器，我們能更簡單的將容器在多個節點中相互搬移。根據實驗結果，在使用pre-dump和track-memory時可省下10%~20%的時間和節省200%以上的儲存空間。

More and more software developers and information-technology professionals embrace the container technology, because it packs the required libraries and settings of software into a single image so that the deployment can be easily done anywhere without the issues of compatibility. On top of single containers, orchestration tools are essential to automate the deployment and operation of complex systems that involve multiple containers on cluster of machines. In this thesis, we present the idea of using checkpoint-and-restore technique to enhance the functionality of Docker Swarm, a state-of-art orchestration tool for Docker containers. Two major functions are focused: high availability and migration. We used multi-version checkpoints to enhance the availability of containers and investigated the optimal storage and performance for checkpoint-and-restore. For migration, we leverage the shared storage and Docker Swarm's scheduler to make migration easier. We also studied the possibility of live migration for our implementation. Experiments show that pre-dump and track-memory will save about 10% ~ 20% container checkpoint frozen time and at least 200% storage space.

Chinese Abstract i
Abstract ii
Contents iv
List of Figures vi
List of Tables vii
List of Algorithms viii
1 Introduction 1
2 Background 5
2.1 Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 runC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Docker Swarm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1 Discovery services . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2 Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.3 High availability of Swarm Manager . . . . . . . . . . . . . . . 8
2.2.4 High availability of Docker Swarm containers . . . . . . . . . . 8
2.3 CRIU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.1 Checkpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3.2 Restore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 Design and Implementation 11
3.1 Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1.1 Docker Client . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1.2 Docker Daemon . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 Docker Swarm Conguration . . . . . . . . . . . . . . . . . . . . . . . 12
3.3 Docker Containers Migration in Docker Swarm . . . . . . . . . . . . . 13
3.4 Docker Swarm Checkpoint and Restoration
Rescheduling Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4.1 Docker Swarm Container Checkpoint Ticker . . . . . . . . . . 15
3.4.2 Container Checkpoint Ticker Strategy . . . . . . . . . . . . . 15
3.4.3 Docker Swarm Restore Rescheduling Policy . . . . . . . . . . 22
3.4.4 High Availability of Swarm Manager in Docker Swarm Checkpoint
and Restore Rescheduling Policy . . . . . . . . . . . . . 24
4 Experiments 25
4.1 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2 Container Migration Time . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 The In
uence of Container Checkpoint Time on Container Process
Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 The In
uence of Many Containers Checkpoint at The Same Time on
Container Checkpoint Time . . . . . . . . . . . . . . . . . . . . . . . 29
4.5 Performance of Checkpoint Time and Space . . . . . . . . . . . . . . 29
5 Conclusion 32

[1] Criu. https://criu.org/Main_Page.
[2] Docker swarm. https://docs.docker.com/swarm/overview/.
[3] runc. https://runc.io/.
[4] Michael Armbrust, Armando Fox, Rean Grith, Anthony D Joseph, Randy
Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica,
et al. A view of cloud computing. Communications of the ACM, 53(4):50{58,
2010.
[5] David Bernstein. Containers and cloud: From lxc to docker to kubernetes.
IEEE Cloud Computing, 1(3):81{84, 2014.
[6] Sukadev Bhattiprolu, Eric W. Biederman, Serge Hallyn, and Daniel Lezcano.
Virtual servers and checkpoint/restart in mainstream linux. SIGOPS Oper.
Syst. Rev., 42(5):104{113, July 2008.
[7] Eric W Biederman and Linux Networx. Multiple instances of the global linux
namespaces. In Proceedings of the Linux Symposium, volume 1, pages 101{112.
Citeseer, 2006.
[8] Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. Fog computing
and its role in the internet of things. In Proceedings of the rst edition
of the MCC workshop on Mobile cloud computing, pages 13{16. ACM, 2012.
[9] NM Mosharaf Kabir Chowdhury and Raouf Boutaba. A survey of network
virtualization. Computer Networks, 54(5):862{876, 2010.
[10] Jason H Christensen. Using restful web-services and cloud computing to create
next generation mobile applications. In Proceedings of the 24th ACM SIGPLAN
conference companion on Object oriented programming systems languages and
applications, pages 627{634. ACM, 2009.
[11] Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul,
Christian Limpach, Ian Pratt, and Andrew Wareld. Live migration of virtual
machines. In Proceedings of the 2nd conference on Symposium on Networked
Systems Design & Implementation-Volume 2, pages 273{286. USENIX Association,
2005.
[12] Brendan Cully, Georey Lefebvre, Dutch Meyer, Mike Feeley, Norm Hutchinson,
and Andrew Wareld. Remus: High availability via asynchronous virtual
machine replication. In Proceedings of the 5th USENIX Symposium on Net-
worked Systems Design and Implementation, pages 161{174. San Francisco,
2008.
[13] Rajdeep Dua, A Reddy Raja, and Dharmesh Kakadia. Virtualization vs containerization
to support paas. In Cloud Engineering (IC2E), 2014 IEEE Inter-
national Conference on, pages 610{614. IEEE, 2014.
[14] Robert P Goldberg. Survey of virtual machine research. Computer, 7(6):34{45,
1974.
[15] Jim Gray and Daniel P. Siewiorek. High-availability computer systems. Com-
puter, 24(9):39{48, 1991.
[16] Matt Helsley. Lxc: Linux container tools. IBM devloperWorks Technical Li-
brary, 2009.
[17] Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D
Joseph, Randy H Katz, Scott Shenker, and Ion Stoica. Mesos: A platform for
ne-grained resource sharing in the data center. In NSDI, volume 11, pages
22{22, 2011.
[18] Ann Mary Joy. Performance comparison between linux containers and virtual
machines. In Computer Engineering and Applications (ICACEA), 2015 Inter-
national Conference on Advances in, pages 342{346. IEEE, 2015.
[19] Alexey Kopytov. Sysbench: a system performance benchmark. URL:
http://sysbench. sourceforge. net, 2004.
[20] Oren Laadan and Serge E Hallyn. Linux-cr: Transparent application
checkpoint-restart in linux. In Linux Symposium, pages 159{172. Citeseer, 2010.
[21] Haikun Liu, Hai Jin, Cheng-Zhong Xu, and Xiaofei Liao. Performance and
energy modeling for live migration of virtual machines. Cluster computing,
16(2):249{264, 2013.
[22] Dirk Merkel. Docker: lightweight linux containers for consistent development
and deployment. Linux Journal, 2014(239):2, 2014.
[23] Karissa Miller and Mahmoud Pegah. Virtualization: virtually at the desktop. In
Proceedings of the 35th annual ACM SIGUCCS fall conference, pages 255{260.
ACM, 2007.
[24] Andrey Mirkin, Alexey Kuznetsov, and Kir Kolyshkin. Containers checkpointing
and live migration. In Proceedings of the Linux Symposium, volume 2, pages
85{90, 2008.
[25] Matti Paksula. Persisting objects in redis key-value database. University of
Helsinki, Department of Computer Science, 2010.
[26] Aameek Singh, Madhukar Korupolu, and Dushmanta Mohapatra. Serverstorage
virtualization: integration and load balancing in data centers. In Pro-
ceedings of the 2008 ACM/IEEE conference on Supercomputing, page 53. IEEE
Press, 2008.
[27] Stephen Soltesz, Herbert Potzl, Marc E Fiuczynski, Andy Bavier, and Larry
Peterson. Container-based operating system virtualization: a scalable, highperformance
alternative to hypervisors. In ACM SIGOPS Operating Systems
Review, volume 41, pages 275{287. ACM, 2007.
[28] Salvatore J Stolfo, Malek Ben Salem, and Angelos D Keromytis. Fog computing:
Mitigating insider data theft attacks in the cloud. In Security and Privacy
Workshops (SPW), 2012 IEEE Symposium on, pages 125{128. IEEE, 2012.
[29] Miguel G Xavier, Marcelo V Neves, Fabio D Rossi, Tiago C Ferreto, Timoteo
Lange, and Cesar AF De Rose. Performance evaluation of container-based
virtualization for high performance computing environments. In 2013 21st Eu-
romicro International Conference on Parallel, Distributed, and Network-Based
Processing, pages 233{240. IEEE, 2013.
[30] Li Yan. Development and application of desktop virtualization technology. In
Communication Software and Networks (ICCSN), 2011 IEEE 3rd International
Conference on, pages 326{329. IEEE, 2011.
[31] Chen Yang. Checkpoint and restoration of micro-service in docker containers.
2015.