疲勞駕駛是引發交通事故的重要因素之一，而疲勞駕駛監測系統是防範並降低交通安全的危害。在目前的研究中，多數系統是以單一疲勞特徵來斷定駕駛的疲勞程度，此方法難以應用在複雜的環境裡；而目前更大的問題是缺乏一個標準且齊的資料庫，因此無法公平地評估疲勞監測系統的好壞。
此篇論文提出了兩個疲勞駕駛監測系統: 離散化組件深度信念網絡和多層次時間性深度信念網絡。在離散化組件深度信念網絡中，幾個疲勞特徵會先被抓取並且收集更細部的資訊來組成多個組件，離散化的組件接著會隨著時間相加，平均之後將資料輸入至深度信念網路的可見層，最後再加softmax函式去估測駕駛的疲勞程度。而在多層次時間性深度信念網絡中，幾個疲勞特徵(打哈欠，點頭和閉眼)會先用深度信念網絡來偵測，並以Hash方法來形成可觀察的輸出，接著在網絡上層有兩個隱馬可夫模型用來描述時間資訊以及描述疲勞特徵之間的互動關係，最後將隱馬可夫模型產生的估計值相減並隨著時間相加後，輸入至S型函數來判斷駕駛的疲勞程度。
為了解決資料庫問題與評估系統的能力，我們也收集了模擬疲勞駕駛的影片資料庫。影片囊括不同膚色的人、姓別、環境光亮程度以及開車時況，目的是希望可以達到資料庫的多樣性與可信度。最後我們將提出的兩個系統用收集好的資料庫去做測試與分析，實驗結果與幾個示範証實系統的實用性。

Drowsy driver alert systems have been developed to reduce and prevent car accidents. Existing vision-based systems are usually restricted to using visual cues, and they usually depend on tedious parameter tuning or cannot work under general conditions. One additional crucial issue is the lack of public datasets that can be used to evaluate the performance of different methods. In this thesis, we develop two novel systems, i.e. a Component-wise Discretized Deep Belief Network (CDDBN) system and a novel Hierarchical Temporal Deep Belief Network (HTDBN) system, for drowsy driver detection.
In CDDBN, after detecting drowsiness-related symptoms using traditional DHMMs and SVM, detailed facial feature are computed to construct several discretized components. The input visible units for DBN are formed by the average of the discretized vectors over a time duration and the softmax layer at the last hidden layer of DBN is to predict the level of drowsiness. In HTDBN, our scheme first extracts high-level facial and head feature representations and then uses them to recognize drowsiness-related symptoms. Two discrete-hidden Markov models that utilize a hash-based scheme are constructed on top of the DBNs. They are used to model and capture the interactive relations between eyes, mouth and head motions. Finally, the summed difference of DHMM likelihoods is used to determine the drowsiness level. To evaluate the performance of the drowsy driver detection systems, we also collect a large comprehensive video dataset containing driver videos of various ethnicities, genders, lighting conditions and driving scenarios. Experimental results demonstrate the feasibility of the proposed CDDBN and HTDBN framework for detecting drowsiness based on different visual cues.

CHAPTER 1 INTRODUCTION 1
1.1 Motivation 1
1.2 Problem Description 2
1.3 Proposed Approaches 2
1.3.1 Component-wise Discretized Deep Belief Network (CDDBN) 3
1.3.2 Hierarchical Temporal Deep Belief Network (HTDBN) 4
1.3.3 Drowsy Driver Detection Dataset 4
1.3.4 Summary of Main Contributions 5
1.4 Thesis Organization 6
CHAPTER 2 LITERATURE REVIEW 8
2.1 Drowsiness-related Symptom Measurement 8
2.2 Drowsy Driver Detection 9
2.3 Background 12
2.3.1 Vector Quantization (VQ) 12
2.3.2 Discrete Hidden Markov Model (DHMM) 13
2.3.3 Local Binary Pattern (LBP) 14
2.3.4 Deep Belief Network (DBN) 15
CHAPTER 3 PROPOSED SYSTEMS 16
3.1 Preprocessing 16
3.1.1 Frontalization 16
3.1.2 Calibration 18
3.2 Component-wise Discretized Deep Belief Network 20
3.2.1 System Overview 20
3.2.2 Drowsiness-related Symptom Extraction 20
3.2.3 Component-wise Discretization 24
3.2.4 Deep Belief Network for Drowsy Driver Detection 27
3.3 Hierarchical Temporal Deep Belief Network 30
3.3.1 System Overview 30
3.3.2 Drowsiness-related Symptom Extraction 30
3.3.3 Pairwise Feature Extraction 32
3.3.4 Observation Vector Hashing 34
3.3.5 Discrete Hidden Markov Model for Drowsy Driver Detection 36
CHAPTER 4 DRIVER DROWSY DETECTION DATASET ACQUISITION 40
4.1 Camera Setting 40
4.2 Environment Setting 41
4.3 Participants 41
4.4 Driver Videos 42
CHAPTER 5 EXPERIMENTAL RESULT 44
5.1 Parameters 44
5.1.1 Component-wise Discretized Deep Belief Network 45
5.1.2 Hierarchical Temporal Deep Belief Network 46
50
5.2 Evaluation 50
5.2.1 Drowsiness-related Symptom Detection 50
5.2.2 Driver Drowsiness Detection 52
5.2.3 Demonstration 54
5.2.4 Computational Complexity 56
CHAPTER 6 CONCLUSION AND FUTURE WORK 58
REFERENCES 60

[1]. L. Bergasa, J. Nuevo, M. Sotelo, R. Barea, and M. Lopez. Real-time system for monitoring driver vigilance. IEEE Transactions on Intelligent Transportation Systems, 7(1):63–77, March.
[2]. W. H. Organization. Global status report on road safety 2013: Supporting a decade of action: Summary, 2013. World Health Organization.
[3]. G. Wheaton and R. Shults. Drowsy driving and risk behaviors 10 states and Puerto Rico, 2014. Online article.
[4]. M.L. Jackson, R.J. Croft, G.A. Kennedy, K. Owens and M.E. Howard. Cognitive components of simulated driving performance: Sleep loss effects and predictors. Accident Analysis & Prevention, 50:438–444, Jan. 2013.
[5]. N. S. Foundation. Drowsy driving reduction act of 2015, 2014.
[6]. A. Colic, O. Marques, and B. Furht. Driver Drowsiness Detection: Systems and Solutions. Springer Publishing Company, Incorporated, 2014.
[7]. Mercedes-Benz. Attention assist: Drowsiness-detection system warns drivers to prevent them falling asleep momentarily, 2008. Online article.
[8]. I. Teyeb, O. Jemai, M. Zaied, and C. Ben Amar. A drowsy driver detection system based on a new method of head posture estimation. Intelligent Data Engineering and Automated Learning, 8669:362–369, 2014.
[9]. Q. Ji, P. Lan, and C. Looney. A probabilistic framework for modeling and real-time monitoring human fatigue. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 36(5):862–875, Sept 2006.
[10]. P. Lucey, J. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2010.
[11]. D. Wu and L. Shao. Leveraging hierarchical parametric networks for skeletal joints based action segmentation and recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 724–731, June 2014.
[12]. A. Colic, O. Marques, and B. Furht. Driver Drowsiness Detection: Systems and Solutions. Springer Publishing Company, Incorporated, 2014.
[13]. G. Yang, Y. Lin, and P. Bhattacharya. A driver fatigue recognition model based on information fusion and dynamic bayesian network. Information Sciences, 180:1942–1954, 2010.
[14]. I. Teyeb, O. Jemai, M. Zaied, and C. Ben Amar. A drowsy driver detection system based on a new method of head posture estimation. Intelligent Data Engineering and Automated Learning, 8669:362–369, 2014.
[15]. Q. Wu, B.X. Sun, X. Bin and J. Zhao. A PERCLOS-Based Driver Fatigue Recognition Application for Smart Vehicle Space. Proceedings of the Third International Symposium on Information Processing, 437-441, 2010.
[16]. T. D’Orazio, M. Leo, C. Guaragnella and A. Distante. A visual approach for driver inattention detection. Pattern Recognition, 40: 2341-2355, 2007.
[17]. A. Mohamed, G. Dahl, and G. Hinton. Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing, 20(1):14–22, Jan 2012.
[18]. A. Dasgupta, A. George, S. Happy, and A. Routray. A vision-based system for monitoring the loss of attention in automotive drivers. IEEE Transactions on Intelligent Transportation Systems, 14(4):1825–1838, Dec 2013.
[19]. N. Alioua, A. Amine, and M. Rziza1. Drivers fatigue detection based on yawning extraction. International Journal of Vehicular Technology, 2014, 2014.
[20]. M. Rezaei and R. Klette. Look at the driver, look at the road: No distraction! no accident! In IEEE Conference on Computer Vision and Pattern Recognition, pages 129–136, June 2014.
[21]. P. Smith, M. Shah, and N. da Vitoria Lobo. Determining driver visual attention with one camera. IEEE Transactions on Intelligent Transportation Systems, 4:205–218, Dec 2003.
[22]. Q. Ji, Z. Zhu, and P. Lan. Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Transactions on Vehicular Technology, 53(4):1052–1068, July 2004.
[23]. L. Jin, Q. Niu, H. Hou, H. Xian, Y. Wang and D. Shi. Driver Cognitive Distraction Detection Using Driving Performance Measures. Discrete Dynamics in Nature and Society, 8669: 362-369, 2012.
[24]. A. Eskandarian and R. Sayed. Analysis of driver impairment, fatigue, and drowsiness and an unobtrusive vehicle-based detection scheme. In Proceeding of International Conference on Traffic Accidents, Dec 2005.
[25]. L. Rabiner, S. Levinson, and M. Sondhi. On the application of vector quantization and hidden Markov models to speaker-independent, isolated word recognition. Bell System Technical Journal, The, 62(4):1075–1105, April 1983.
[26]. L. Yang, B. Widjaja, and R. Prasad. Application of hidden Markov models for signature verification. Pattern Recognition, 28(2):161 – 170, 1995.
[27]. P. A. Devijver. Baum’s forward-backward algorithm revisited. Pattern Recognition Letters, 3(6):369 – 373, 1985.
[28]. T. Ojala, M. Pietikainen, and T. Maenpaa, “Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971–987, 2002.
[29]. G. Hinton and S. Osindero. A fast learning algorithm for deep belief nets. Neural Computation, 18:2006, 2006.
[30]. G. E. Hinton. A practical guide to training restricted Boltzmann machines. In Neural Networks: Tricks of the Trade (2nd ed.), volume 7700, pages 599–619. Springer, 2012.
[31]. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. Deepface: Closing the gap to human-level performance in face verification. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1701–1708, June 2014.
[32]. X. Xiong and F. de la Torre. Supervised descent method and its application to face alignment. In IEEE Conference on Computer Vision and Pattern Recognition, pages 532–539, June 2013.
[33]. F. DeMenthon and L. Davis. Model-based object pose in 25 lines of code. International Journal of Computer Vision, 15:123–141.
[34]. J. Heo and M. Savvides. Gender and ethnicity specific generic elastic models from a single 2d image for novel 2d pose face synthesis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12):2341– 2350, Dec 2012.
[35]. A. Eskandarian and R.A. Sayed, “Detecting Driver Fatigue by Monitoring Eye and Steering Activity”, Proceeding of Annual Intelligent Vehicles Systems Symposium, 2003.
[36]. A. Mohamed, G. Dahl, and G. Hinton. Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing, 20(1):14–22, Jan 2012.
[37]. G. Taylor, G. Hinton, and S. Roweis. Modeling human motion using binary latent variables. In Neural Information Processing Systems, pages 1345–1352, 2006.
[38]. Y. Freund and D. Haussler. Unsupervised learning of distributions on binary vectors using two layer networks. Technical report, Santa Cruz, CA, USA, 1994.
[39] C. Chung Chang and C. Jen Lin, “LIBSVM: A library for support vector machines,” ACM Transactions on Intelligent Systems and Technology, vol. 2, pp. 27:1–27:27, 2011, Software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm.
[40]. X. Yang and Y. Tian. Eigenjoints-based action recognition using naive-bayes-nearest-neighbor. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pages 14–19, June 2012.
[41]. S. Abtahi, M. Omidyeganeh, S. Shirmohammadi, and B. Hariri. Yawdd: A yawning detection dataset. In Proceedings of the 5th ACM Multimedia Systems Conference, pages 24–28. ACM, 2014.
[42]. D.W. Hosmer, S. Lemeshow and R.X. Sturdivant. Applied Logistic Regression (3rd edition). A Wiley Company, 2013.