本研究透過強化學習中的Deep Q-Learning建立了一套空調及風扇的控制演算法，能夠對室內的空氣品質、熱舒適度以及能源消耗做平衡，以實驗方式對該演算法進行測試，並探討其效能。實驗場域為交通大學工程五館132教室，本研究與中華電信合作，使用中華電信的Prius智慧家電Wi-Fi系統作為無線傳輸架構。空氣品質定義二氧化碳800ppm以下為舒適，800-1000ppm為可以接受，1000ppm以上為不舒適；舒適度定義PMV在正負0.5之間為舒適；能源消耗以使用一節課五十分鐘內消耗多少度電為基準，改變實驗參數比較分析何種環境或控制方法能在教室人員舒適的情況下達到最大的節能效果。實驗參數分為不可控參數：教室人數(26-65人)、人員分佈情形、室外溫度(22-34℃) 、天氣(晴、雨與風)。可控參數：固定溫度控制(25-26℃)、DQN自動控制、人工手動控制。
研究結果顯示，未控制情況，人員使用132教室時大部分為開門關窗和空調定溫25度。教室人數超過30人時二氧化碳濃度就有機會超過1000ppm，使人不舒適。DQN Agent在維持PMV為舒適的情況下，比定溫25度節省13.7-45.0％的能源消耗。一般情況下建議冷氣定溫26度，比定溫25度節省6-18.7％的能源消耗。

This study establishes a control algorithm for air conditioning and ventilation fans through Deep Q-Learning, which balances indoor air quality, thermal comfort and energy consumption. The utility of DQN agent is tested and explored through experiment. The experiment was conducted in 132 classroom of Engineering building 5 in Nation Chiao Tung University. This research project is collaborated with Chunghwa Telecom and the Prius Wi-Fi system used in this study is provided by the corporation. We use carbon dioxide concentration level as indoor air quality index: below 800ppm for comfortable; 800-1000ppm for acceptable, and over 1000ppm for uncomfortable. For thermal comfort we use PMV as our index: between -0.5 and 0.5 are comfort; For energy consumption we consider the sum of electricity used by the AC and fans during the 50-minute-class. Different experimental setup and parameters are considered to find out which control strategy has the greatest energy savings while maintaining the same comfort level.
Most uncontrolled usage situations in room 132 are window closed and door opened, AC set at 25 °C. Experiment result shows that when the number of students in classroom exceeds 30, the carbon dioxide concentration has a chance of exceeding 1000 ppm, making people uncomfortable. DQN agent is able to save 13.7-45.0% energy consumption compared to constant temperature 25 °C while maintaining PMV at comfort range. For average situation it is recommended setting AC temperature at 26 °C, which saves 6-18.7% energy consumption than 25 °C.

目錄
摘要 i
ABSTRACT ii
目錄 iii
表目錄 v
圖目錄 vi
第一章 緒論 1
1.1 前言 1
1.2 文獻回顧 2
1.3 研究目的 12
1.4 實驗之操作參數 14
第二章 實驗原理 15
2.1 熱舒適度 15
2.1.1 熱舒適度之定義 15
2.1.2 熱舒適度的國際標準及相關驗證 18
2.2 空氣品質─二氧化碳 22
2.3 物聯網和無線網路技術 24
2.4 強化學習(Deep Reinforcement Learning)與Deep Q Learning 25
2.4.1 強化學習的基本概念 25
2.4.2 Q-Learning的基本概念 27
2.4.3 Deep Q-Learning的基本概念 29
第三章 實驗設備與量測方法 31
3.1 實驗地點 31
3.2 實驗系統架構 32
3.3 實驗儀器設備 33
3.4 實驗儀器設備裝設點 40
第四章 Deep Q-Learning實驗演算法流程 44
4.1 系統狀態 45
4.2 控制行動 46
4.3 獎勵函數 47
4.4 演算流程 50
4.5 神經網路參數 53
4.6 訓練方法及模擬結果 54
4.6.1 訓練方法 54
4.6.2 模擬結果 58
第五章 實驗結果與討論 61
5.1 智慧教室參數設定與實驗總表 61
5.2 教室環境空氣品質─二氧化碳 66
5.2.1 人數之影響 68
5.2.2 人員分佈之影響 68
5.2.3 使用風扇之影響 69
5.3 熱舒適度─PMV 77
5.3.1 人數之影響 77
5.3.2 人員分佈之影響 77
5.3.3 控制方法之影響 78
5.4 能源消耗 83
5.4.1 室外溫度之影響 83
5.4.2 人數之影響 83
5.4.3 控制方法之影響 85
第六章 結論與未來展望 86
6.1 結論 86
6.2 未來展望及工作 87
參考文獻 88
附錄：實驗不準度計算 91

參考文獻
[1] 經濟部能源局, 能源產業技術白皮書, in, 2014.
[2] P.O.J.T.c.A. Fanger, a.i.e. engineering., Thermal comfort. Analysis and applications in environmental engineering, (1970).
[3] A.J.I.J.o.B. Auliciems, Towards a psycho-physiological model of thermal perception, 25(2) (1981) 109-122.
[4] R. De Dear, G.S.J.I.j.o.b. Brager, The adaptive model of thermal comfort and energy conservation in the built environment, 45(2) (2001) 100-108.
[5] K.E. Charles, Fanger's thermal comfort and draught models, (2003).
[6] K.W.H. Mui, W.T.D.J.B. Chan, environment, Adaptive comfort temperature model of air-conditioned building in Hong Kong, 38(6) (2003) 837-852.
[7] S. Lin, S. Wei, C. Huang, W.J.J.o.A. Chen, Thermal comfort study of an air-conditioned presentation room in Taiwan, 65 (2008) 125-138.
[8] R.-L. Hwang, T.-P. Lin, N.-J.J.E. Kuo, Buildings, Field experiments on thermal comfort in campus classrooms in Taiwan, 38(1) (2006) 53-62.
[9] N.H. Wong, S.S.J.E. Khoo, buildings, Thermal comfort in classrooms in the tropics, 35(4) (2003) 337-351.
[10] 林.J. 成功大學工業設計學系學位論文, 應用遺傳演算法於智慧型空間熱舒適度系統之研究, (2007) 1-114.
[11] K. Ku, J. Liaw, M. Tsai, T.J.I.T.A.S. Liu, Engineering, Automatic Control System for Thermal Comfort Based on Predicted Mean Vote and Energy Saving, 12(1) (2015) 378-383.
[12] Y. Kim, T. Schmid, Z.M. Charbiwala, M.B. Srivastava, ViridiScope: design and implementation of a fine grained power monitoring system for homes, in: Proceedings of the 11th international conference on Ubiquitous computing, ACM, 2009, pp. 245-254.
[13] S.S. Intille, K. Larson, E.M. Tapia, J.S. Beaudin, P. Kaushik, J. Nawyn, R. Rockinson, Using a live-in laboratory for ubiquitous computing research, in: International Conference on Pervasive Computing, Springer, 2006, pp. 349-365.
[14] D. Kolokotsa, G. Stavrakakis, K. Kalaitzakis, D.J.E.A.o.A.I. Agoris, Genetic algorithms optimized fuzzy controller for the indoor environmental management in buildings implemented using PLC and local operating networks, 15(5) (2002) 417-428.
[15] K. Dalamagkidis, D. Kolokotsa, K. Kalaitzakis, G.S.J.B. Stavrakakis, environment, Reinforcement learning for energy conservation and comfort in buildings, 42(7) (2007) 2686-2698.
[16] D. Kolokotsa, A. Pouliezos, G. Stavrakakis, C. Lazos, Predictive control techniques for energy and indoor environmental quality management in buildings, Building and Environment, 44(9) (2009) 1850-1863.
[17] D. Kolokotsa, K. Gobakis, S. Papantoniou, C. Georgatou, N. Kampelis, K. Kalaitzakis, K. Vasilakopoulou, M. Santamouris, Development of a web based energy management system for University Campuses: The CAMP-IT platform, Energy and Buildings, 123 (2016) 119-135.
[18] P.M. Ferreira, A.E. Ruano, S. Silva, E.Z.E. Conceição, Neural networks based predictive control for thermal comfort and energy savings in public buildings, Energy and Buildings, 55 (2012) 238-251.
[19] T. Wei, Y. Wang, Q. Zhu, Deep Reinforcement Learning for Building HVAC Control, in: Proceedings of the 54th Annual Design Automation Conference 2017 on - DAC '17, 2017, pp. 1-6.
[20] 經濟部能源委員會, 學校節約能源技術手冊.
[21] 交通大學營繕組, 校區電費(104).
[22] 交通大學營繕組, 館舍用電資料(104).
[23] A.S.o. Heating, Refrigerating, A.-C. Engineers, Thermal environmental conditions for human occupancy, American Society of Heating, Refrigerating and Air-Conditioning Engineers, 2004.
[24] I.J.M.t.e.D.o.t.P. Standard, P. indices, s.o.t.c.f.t. comfort, 7730, (1994).
[25] A.J.A.S.o.H. Standard, Refrigerating, I. Air-Conditioning Engineers, Standard 62.1-2010 (2010). ventilation for acceptable indoor air quality, atlanta, ga, (2010).
[26] T.C.T.e. standards, CALIFORNIA ENERGY COMMISSION, (2016).
[27] 行政院環境保護署, 室內空氣品質標準, (2012).
[28] L. Wang, Z. Wang, R.J.I.t.o.s.g. Yang, Intelligent multiagent control system for energy and comfort management in smart and sustainable buildings, 3(2) (2012) 605-617.
[29] K.J.R.j. Ashton, That ‘internet of things’ thing, 22(7) (2009) 97-114.
[30] L. Atzori, A. Iera, G.J.C.n. Morabito, The internet of things: A survey, 54(15) (2010) 2787-2805.
[31] T. Simonini, A Free course in Deep Reinforcement Learning from beginner to expert, in, 2018, pp. Github.io, [Online]. Available: https://simoninit homas.github.io/Deep_reinforcement_learning_Course/. [Accessed 08 August 2018].
[32] V. Mnih, K. Kavukcuoglu, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, A. Graves, M. Riedmiller, A.K. Fidjeland, G.J.N. Ostrovski, Human-level control through deep reinforcement learning, 518(7540) (2015) 529.
[33] D.B. Crawley, L.K. Lawrie, F.C. Winkelmann, W.F. Buhl, Y.J. Huang, C.O. Pedersen, R.K. Strand, R.J. Liesen, D.E. Fisher, M.J.J.E. Witte, buildings, EnergyPlus: creating a new-generation building energy simulation program, 33(4) (2001) 319-331.
[34] T.I. T. Inc., SketchUp, in, pp. [Online]. Available: https://www.sketchup.com/es. [Accessed 04 August 2018].
[35] D.M.a.N.L. R. Guglielmetti, "OpenStudio: an open source integrated analysis platform," Proceedings of the 12th conference of international building performance simulation association, (2011).