手機操作的流暢度與電池續航能力是影響使用者體驗的兩個重要因素，如何在維持使用者對手機效能滿意的情況下，盡可能的節省電池電能是我們研究探討的問題。
目前的手機多使用動態電壓頻率調整，通過當下系統的資源負載狀況預測下一個適合的頻率，然而，不同的使用者在不同的環境下，對手機的流暢程度會有不同的要求，僅使用單一的規則作為效能調整的依據，並沒有考慮到使用者在不同情境的主觀感受，可能會造成額外的電能浪費。
為了解決這個問題，我們提出能感知情境的增強式調頻策略，利用手機上的陀螺儀、三軸加速器等等的感測器，感知使用者情境，將蒐集到的情境資料與手機系統資料整合，由於情境資料是高維度的巨量資料，我們使用增強式學習的架構訓練深度神經網路模型，利用深度神經網路的特性來處理複雜且高維度的情境資料，讓增強式學習的架構能分析出最佳調頻策略。
我們的模型可以依照使用者情境調整手機頻率，在不影響使用者的滿意度下盡可能的省電，找出最佳平衡點，相較於現行的動態電壓頻率調整，我們的方法可以在滿意度僅下降1.5%的情況下，節省10.4%的電能，並能即時感知使用者情境做最佳化的頻率調整決策。

The smoothness in using the smartphone and its battery life are two most important factors affecting user experiences. It is thus critical to reducing power consumption in smartphones while keeping their smooth operations. One of the most basic techniques in managing power in smartphones is Dynamic Voltage and Frequency Scaling (DVFS), which scales the frequency and voltage of CPUs according to the current system workloads. Unfortunately, current smartphones use a single DVFS policy for all users, while ignoring the possible different requirements of different users under different contexts. This leads to suboptimal management of power and a waste of energy. In this thesis, we address this issue by exploiting reinforcement learning (RL) to derive the best CPU frequency scaling for individual users under different contexts. The idea is to sense the current contexts using the available sensors on the smartphone, extract the context features with a neural network, and then learn the best CPU frequency setting through RL with simple satisfactory feedbacks from the user. To speed up the learning process, a generic model is first trained from daily usage data of multiple users and then adapted to individuals. Performance evaluations show that the proposed RL method can save about 10.4% of energy comparing to the default policy, while user satisfaction is decreased only by 1.5%

Contents

Chinese Abstract. . . . . . . . . . . . . . . . . . . . . . . . i
Abstract. . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . iii
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . iii
1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Background. . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Typical Energy Saving Methods . . . . . . . . . . . . . . . 4
2.2 Machine Learning . . . . . . . . . . . . . . . . . . . . . .5
2.2.1 Deep Learning . . . . . . . . . . . . . . . . . . . . . . 6
2.2.2 Reinforcement Learning . . . . . . . . . . . . . . . . . .7
3 Related Work. . . . . . . . . . . . . . . . . . . . . . . . . 9
4 Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.1 Generic Model . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 CPU Frequency Scaling Policy for Individual Users . . . . . 12
5 Experiment. . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.1 Experimental Environment . . . . . . . . . . . . . . . . . .16
5.2 Dataset Analysis . . . . . . . . . . . . . . . . . . . . . .18
5.3 Experimental Result . . . . . . . . . . . . . . . . . . . . 19
6 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . 26

[1] Chen-Xi Zu and Hong Li. Thermodynamic analysis on energy densities of
batteries. Energy Environ. Sci., 4:2614–2624, 2011.
[2] Matti Siekkinen, Mohammad Ashraful Hoque, Jukka K. Nurminen, and Mika
Aalto. Streaming over 3g and lte: How to save smartphone energy in radio
access network-friendly way. In Proceedings of the 5th Workshop on Mobile
Video, MoVid ’13, pages 13–18, New York, NY, USA, 2013. ACM.
[3] David T. Nguyen, Gang Zhou, Xin Qi, Ge Peng, Jianing Zhao, Tommy Nguyen,
and Duy Le. Storage-aware smartphone energy savings. In Proceedings of
the 2013 ACM International Joint Conference on Pervasive and Ubiquitous
Computing, UbiComp ’13, pages 677–686, New York, NY, USA, 2013. ACM.
[4] L. G. M. Ballesteros, S. Ickin, M. Fiedler, J. Markendahl, K. Tollmar, and
K. Wac. Energy saving approaches for video streaming on smartphone based
on qoe modeling. In 2016 13th IEEE Annual Consumer Communications Networking
Conference (CCNC), pages 103–106, Jan 2016.
[5] Aaron Carroll and Gernot Heiser. An analysis of power consumption in a
smartphone. In Proceedings of the 2010 USENIX Conference on USENIX Annual
Technical Conference, USENIXATC’10, pages 21–21, Berkeley, CA, USA,
2010. USENIX Association.
[6] Jian-Jia Chen, Chuan-Yue Yang, Tei-Wei Kuo, and Chi-Sheng Shih. Energyefficient
real-time task scheduling in multiprocessor dvs systems. 2007 Asia and
South Pacific Design Automation Conference, pages 342–349, 2007.
[7] Wen-Yew Liang, Po-Ting Lai, and Che Wun Chiou. An energy conservation
dvfs algorithm for the android operating system. 2011.
[8] J. Kwak, O. Choi, S. Chong, and P. Mohapatra. Dynamic speed scaling for
energy minimization in delay-tolerant smartphone applications. In IEEE INFOCOM
2014 - IEEE Conference on Computer Communications, pages 2292–
2300, April 2014.
[9] Lei Yang, Robert P Dick, Gokhan Memik, and Peter Dinda. Happe: Human
and application-driven frequency scaling for processor power efficiency. IEEE
transactions on mobile computing, 12(8):1546–1557, 2013.
[10] Sascha Bischoff, Andreas Hansson, and Bashir M Al-Hashimi. Applying of
quality of experience to system optimisation. In Power and Timing Modeling,
Optimization and Simulation (PATMOS), 2013 23rd International Workshop
on, pages 91–98. IEEE, 2013.
[11] Wei-Ming Chen, Sheng-Wei Cheng, Pi-Cheng Hsiu, and Tei-Wei Kuo. A usercentric
cpu-gpu governing framework for 3d games on mobile devices. In Proceedings
of the IEEE/ACM International Conference on Computer-Aided Design,
pages 224–231. IEEE Press, 2015.
[12] Youngho Ahn and Ki-Seok Chung. User-centric power management for embedded
cpus using cpu bandwidth control. IEEE Transactions on Mobile Computing,
15(9):2388–2397, 2016.
[13] Wook Song, Nosub Sung, Byung-Gon Chun, and Jihong Kim. Reducing energy
consumption of smartphones using user-perceived response time analysis. In
Proceedings of the 15th Workshop on Mobile Computing Systems and Applications,
page 20. ACM, 2014.
[14] B. Clarkson, K. Mase, and A. Pentland. Recognizing user context via wearable
sensors. In Digest of Papers. Fourth International Symposium on Wearable
Computers, pages 69–75, Oct 2000.
[15] Yonatan Vaizman, Nadir Weibel, and Gert Lanckriet. Context recognition inthe-wild:
Unified model for multi-modal sensors and multi-label classification.
1:1–22, 01 2018.
[16] Yann LeCun, Y Bengio, and Geoffrey Hinton. Deep learning. 521:436–44, 05
2015.
[17] Yoshua Bengio. Learning deep architectures for AI. Foundations and Trends in
Machine Learning, 2(1):1–127, 2009. Also published as a book. Now Publishers,
2009.
[18] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness,
Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland,
Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis
Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and
Demis Hassabis. Human-level control through deep reinforcement learning. Nature,
518:529 EP –, Feb 2015.
[19] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis
Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep
reinforcement learning, 2013.
[20] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification
with deep convolutional neural networks. In Proceedings of the 25th
International Conference on Neural Information Processing Systems - Volume
1, NIPS’12, pages 1097–1105, USA, 2012. Curran Associates Inc.
[21] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning
representations by back-propagating errors. Nature, 323:533 EP –, Oct 1986.
[22] Gongde Guo, Hui Wang, David Bell, Yaxin Bi, and Kieran Greer. Knn modelbased
approach in classification. In Robert Meersman, Zahir Tari, and Douglas
C. Schmidt, editors, On The Move to Meaningful Internet Systems 2003:
CoopIS, DOA, and ODBASE, pages 986–996, Berlin, Heidelberg, 2003. Springer
Berlin Heidelberg.
[23] T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman,
and A. Y. Wu. An efficient k-means clustering algorithm: analysis and implementation.
IEEE Transactions on Pattern Analysis and Machine Intelligence,
24(7):881–892, Jul 2002.
[24] Chih-wei Hsu, Chih-chung Chang, and Chih-Jen Lin. A practical guide to
support vector classification chih-wei hsu, chih-chung chang, and chih-jen lin.
11 2003.
[25] L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A
survey. 1996.
[26] Volodymyr Mnih, Adri Puigdomnech Badia, Mehdi Mirza, Alex Graves, Timothy
P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous
methods for deep reinforcement learning. 2016.
[27] Guangshuo Liu, Jinpyo Park, and Diana Marculescu. Procrustes1: Power constrained
performance improvement using extended maximize-then-swap algorithm.
IEEE Transactions on Computer-Aided Design of Integrated Circuits
and Systems, 34:1664–1676, 2015.
[28] Y. G. Kim, M. Kim, and S. W. Chung. Enhancing energy efficiency of multimedia
applications in heterogeneous mobile multi-core processors. IEEE Transactions
on Computers, 66(11):1878–1889, Nov 2017.
[29] Paul Bogdan, Radu Marculescu, and Siddharth Jain. Dynamic power management
for multidomain system-on-chip platforms: An optimal control approach.
ACM Trans. Des. Autom. Electron. Syst., 18(4):46:1–46:20, October 2013.
[30] R. David, P. Bogdan, and R. Marculescu. Dynamic power management for
multicores: Case study using the intel scc. In 2012 IEEE/IFIP 20th International
Conference on VLSI and System-on-Chip (VLSI-SoC), pages 147–152,
Oct 2012.
[31] Yu-Ming Chang, Pi-Cheng Hsiu, Yuan-Hao Chang, and Che-Wei Chang. A
resource-driven dvfs scheme for smart handheld devices. ACM Trans. Embed.
Comput. Syst., 13(3):53:1–53:22, December 2013.
[32] Yohan Chon, Elmurod Talipov, Hyojeong Shin, and Hojung Cha. Mobility
prediction-based smartphone energy optimization for everyday location monitoring.
In Proceedings of the 9th ACM Conference on Embedded Networked
Sensor Systems, SenSys ’11, pages 82–95, New York, NY, USA, 2011. ACM.
[33] A. M. Khan, Y. K. Lee, S. Y. Lee, and T. S. Kim. Human activity recognition
via an accelerometer-enabled-smartphone using kernel discriminant analysis.
In 2010 5th International Conference on Future Information Technology, pages
1–6, May 2010.
[34] J. Wannenburg and R. Malekian. Physical activity recognition from smartphone
accelerometer data for user context awareness sensing. IEEE Transactions on
Systems, Man, and Cybernetics: Systems, 47(12):3142–3149, Dec 2017.
[35] Yonatan Vaizman, Nadir Weibel, and Gert Lanckriet. Context recognition inthe-wild:
Unified model for multi-modal sensors and multi-label classification.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 1(4):168:1–168:22,
January 2018.
[36] C. Li, J. Bao, and H. Wang. Optimizing low memory killers for mobile devices
using reinforcement learning. In 2017 13th International Wireless Communications
and Mobile Computing Conference, pages 2169–2174, June 2017.
[37] Hado van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning
with double q-learning, 2015.
[38] Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot,
and Nando de Freitas. Dueling network architectures for deep reinforcement
learning, 2015.
[39] David Silver. Reinforcement learning, 2015