中風在台灣一直是一個不可忽視的疾病，其發病以及致死率一直居高不下，中風病患需要接受適當的治療以及漫長的復健之路才能回復，然而，多數的病患因為肌肉的無力以及復健帶來的疼痛而中途放棄，現行的復健器材無法有效的輔助病患出力做出特定的復健動作，因此，復健之路通常以失敗告終。這篇論文意在研究、發展出一套低成本、高效率的機械系統能夠使病患更加輕易且省力的做出指定動作。如此，將能提升病患的復健意願。
人工智慧在醫療產業的應用日新月異，在此研究中，運用Kinect感測器來感測人體動作，再以卷積神經網路進行高準確率的人體動作識別並將結果輸出至一個雙自由度的機械手臂上，並使用線性二次調節控制器控制，使其能夠進行雙軸的軌跡追蹤以此達成研究目的。

Stroke is one of the most devastating medical emergencies in Taiwan, which can cause neurological disorder and permanent impairment. Stroke patients need to receive proper treatment in time and rehabilitate to recover from their illness. However, many patients give up their treatment due to the painful process, heavy financial burden, and countless visits to clinic or medical centers. The long-term goal of this study aims to design an upper limb assistive robotic system to assist stroke patients for their rehabilitation and daily activities. In this thesis, the motion prediction models using deep learning algorithm and controller synthesis of multi-axial robotic device are focused.
To develop the models of motion prediction, the algorithm of Convolutional Neural Networks (CNNs) has been adopted. The experimental results indicate that the type of a motion can be predicted after 33% of a motion cycle being completed. The trajectories of involved joints can then be synthesized as tracking references of assistive robotic devices. To emulate the derived trajectories on a physical robotic device, adequate controller needs to be designed. A Linear Quadratic Regulator (LQR) controller and a controller based on pole-placement were adopted on a two degrees-of-freedom (2-DOF’s) robotic arm in this research. As a result, CNN can provide an excellent accuracy of motion prediction. The developed controller can also precisely compensate for the tracking performance of the selected motions.

摘要 I
ABSTRACT II
TABLE OF CONTENTS IV
LIST OF TABLES VI
LIST OF FIGURES VII
INTRODUCTION 1
1.1 RESEARCH BACKGROUND 1
1.2 MOTIVATION 2
1.3 PROBLEM STATEMENT 5
1.4 RESEARCH OBJECTIVES AND ORGANIZATION OF THESIS 7
LITERATURE REVIEW 9
2.1 INTRODUCTION 9
2.2 ROBOTIC ASSISTANCE OF STROKE REHABILITATION 10
2.3 ACQUISITION OF HUMAN MOTIONS 14
2.3.1 Application of Human Motions 15
2.3.2 Recognition of Human Motions for Medical Purposes 16
2.4 APPLICATIONS OF ROBOTIC DEVICES 19
2.4.1 Medical Applications – Robotic Assistive Devices 21
2.4.2 Medical Applications – Wearable Assistive Devices 23
2.4.3 Other Applications of Wearable Robotic Devices 26
2.5 MEDICAL APPLICATIONS WITH ARTIFICIAL INTELLIGENCE 28
2.5.1 Artificial Intelligence in Healthcare 28
2.5.2 Machine Perception and Machine Learning 30
MOTION ACQUISITION OF ARM TRAJECTORIES 32
3.1 EXPERIMENTAL SETUP 32
3.1.1 Setup of Acquisition Environment 33
3.1.2 Data Acquisition with Depth Camera 34
3.1.3 Data Acquisition with Regular Camcorder 37
3.2 RECRUITMENT OF TESTING SUBJECTS 38
3.2.1 Adopted Motions 39
3.2.2 Consideration of Different Body Conditions 41
DERIVATION OF MOTION MODEL 43
4.1 TRAJECTORY TRACKING RESULTS OF MOTIONS 43
4.1.1 Preprocessing of Recorded Trajectories 44
4.1.2 Phase Portraits of Recorded Motions 46
4.2 CONVOLUTIONAL NEURAL NETWORK 48
4.2.1 CNN Structure and Adjusted Algorithms 49
4.2.2 RESULT OF CNN MODEL 56
4.3 DETECTION OF MOTION IN REAL TIME 62
4.3.1 Motion Identification with Partial Trajectories 62
MODELING AND CONTROLLER SYNTHESIS OF EXPERIMENTAL PLATFORM 64
5.1 MECHANICAL HARDWARE AND ELECTRICAL DEVICES 64
5.2 MODELING OF ROBOTIC DEVICE 68
5.3 CONTROLLER DESIGN 71
5.3.1 Ackermann’s Formula 71
5.3.2 Linear Quadratic Regulator (LQR) 72
SIMULATION AND EXPERIMENTAL RESULTS 75
6.1 TRACKING RESULTS OF DUAL AXIAL MOVEMENT WITH CONTINUOUS METHOD 75
6.1.1 Simulation of Controlled System with Continuous LQR Method and Ackermann’s Formula 75
6.1.2 Issues of Controller Implementation Using Microcontroller 82
6.1.3 Nonlinear Conditions of the Experimental Platform 84
6.2 EXPERIMENTAL RESULTS OF DUAL-AXIAL MOVEMENTS WITH DISCRETE METHODS 85
6.2.1 Simulation of Discrete LQR and Ackermann’s Control 85
6.2.2 Experimental Results of Discrete Time Controller Using Ackermann’s Formula 89
6.2.2 Experimental Results of Discrete Time LQR Controller 98
CONCLUSION 111
7.1 SUMMARY OF THE RESEARCH 111
7.2 FUTURE WORKS 114
REFERENCES 116

Asavari Patil. (2018). Artificial Intelligence (AI) Market by Technology (Machine Learning, Natural Language Processing, Image Processing, Speech Recognition), and Industry Vertical (Media & Advertising, BFSI, IT & Telecom, Retail, Healthcare, Automotive & Transportation, and Others) - Global Opportunity Analysis and Industry Forecast, 2018-2025. Allied Market Research.

Bashir, F. I., Khokhar, A. A., & Schonfeld, D. (2007). Object trajectory-based activity classification and recognition using hidden Markov models. IEEE transactions on Image Processing, 16(7), 1912-1919.

Blackburn, J., & Ribeiro, E. (2007). Human motion recognition using isomap and dynamic time warping. In Human motion–understanding, modeling, capture and animation (pp. 285-298). Springer, Berlin, Heidelberg.

Bobick, A. F., & Davis, J. W. (2001). The recognition of human movement using temporal templates. IEEE Transactions on pattern analysis and machine intelligence, 23(3), 257-267.

Bogue, R. (2015). Robotic exoskeletons: a review of recent progress. Industrial Robot: An International Journal, 42(1), 5-10.

Chakrabortty, A., Nagle, H. T., & Phillips, C. L. (2015). Digital Control System Analysis & Design: Global Edition.

Cheng, M. H., Chang, Y.J., Huang, P. L., & Chu, H. C. (2019). Path Planning for Assistive Robotic Devices. ACC. American control conference, submitted.

Cheng, M. H., Chen, C. Y., & Bakhoum, E. G. SYNCHRONIZATION CONTROLLER SYNTHESIS OF MULTI-AXIS MOTION SYSTEM.

Cheng, M. H., Jiang, L., Mou, Q., Wheeler, S., Banta, L., Shisheie, R., & Bakhoum, E. (2016, July). Design, fabrication, and control of twisted actuated robotic device. In 2016 American Control Conference (ACC)(pp. 1215-1220). IEEE.

Esquenazi, A., Talaty, M., & Jayaraman, A. (2017). Powered exoskeletons for walking assistance in persons with central nervous system injuries: a narrative review. PM&R, 9(1), 46-62.

Goodfellow, I., Bengio, Y., Courville, A., & Bengio, Y. (2016). Deep learning (Vol. 1). Cambridge: MIT press.

Hall, D. L., & Llinas, J. (1997). An introduction to multisensor data fusion. Proceedings of the IEEE, 85(1), 6-23.

Haritaoglu, I., Harwood, D., & Davis, L. S. (2000). W4: Real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis & Machine Intelligence, (8), 809-830.

Helfand, L. A., & Bridger, C. (1971). Hippocrates, Thermal Stress, and Stroke Mortality—1966. Weatherwise, 24(3), 100-104.

Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. (2018). Artificial intelligence in radiology. Nature Reviews Cancer, 1.

Jack, D., Boian, R., Merians, A., Adamovich, S. V., Tremaine, M., Recce, M., ... & Poizner, H. (2000, November). A virtual reality-based exercise program for stroke rehabilitation. In Proceedings of the fourth international ACM conference on Assistive technologies (pp. 56-63). ACM.

Jovanov, E., Milenkovic, A., Otto, C., & De Groen, P. C. (2005). A wireless body area network of intelligent motion sensors for computer assisted physical rehabilitation. Journal of NeuroEngineering and rehabilitation, 2(1), 6.

Julesz, B. (1964). Binocular Depth Perception without Familiarity Cues: Random-dot stereo images with controlled spatial and temporal properties clarify problems in stereopsis. Science, 145(3630), 356-362.

Kan, P., Huq, R., Hoey, J., Goetschalckx, R., & Mihailidis, A. (2011). The development of an adaptive upper-limb stroke rehabilitation robotic system. Journal of neuroengineering and rehabilitation, 8(1), 33.

Khoshelham, K., & Elberink, S. O. (2012). Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors, 12(2), 1437-1454.

Kim, S. S., Dohler, M., & Dasgupta, P. (2018). The Internet of Skills: The use of 5th Generation Telecommunications, Haptics, and Artificial Intelligence in Robotic Surgery. BJU international.

Kline, R. (2011). Cybernetics, automata studies, and the Dartmouth conference on artificial intelligence. IEEE Annals of the History of Computing, 33(4), 5-16.

Kopp, C. (2011). Exoskeletons for warriors of the future. Defence Today, 9(2), 38-40.

Kramer, J. P., Lindener, P., & George, W. R. (1991). U.S. Patent No. 5,047,952. Washington, DC: U.S. Patent and Trademark Office.

Krebs, H. I., Volpe, B. T., Aisen, M. L., & Hogan, N. (2000). Increasing productivity and quality of care: Robot-aided neuro-rehabilitation. Journal of rehabilitation research and development, 37(6), 639-652.

Langen, P. A., Katz, J. S., Dempsey, G., & Pompano, J. (1994). U.S. Patent No. 5,357,427. Washington, DC: U.S. Patent and Trademark Office.

Lauritzen, S. L., & Spiegelhalter, D. J. (1988). Local computations with probabilities on graphical structures and their application to expert systems. Journal of the Royal Statistical Society. Series B (Methodological), 157-224.

Lee, J., Kim, S., Lee, J., Yang, D., Park, B. C., Ryu, S., & Park, I. (2014). A stretchable strain sensor based on a metal nanoparticle thin film for human motion detection. Nanoscale, 6(20), 11932-11939.

Linder, S. M., Rosenfeldt, A. B., Bay, R. C., Sahu, K., Wolf, S. L., & Alberts, J. L. (2015). Improving quality of life and depression after stroke through telerehabilitation. American Journal of Occupational Therapy, 69(2), 6902290020p1-690229002.

Magoulas, G. D., & Prentza, A. (1999, July). Machine learning in medical applications. In Advanced Course on Artificial Intelligence (pp. 300-307). Springer, Berlin, Heidelberg.

Mao, Y., & Agrawal, S. K. (2012). Design of a cable-driven arm exoskeleton (CAREX) for neural rehabilitation. IEEE Transactions on Robotics, 28(4), 922-931.
Mayr, O. (1970). The origins of feedback control. Scientific American, 223(4), 110-119.

Nef, T., Mihelj, M., Kiefer, G., Perndl, C., Muller, R., & Riener, R. (2007, June). ARMin-Exoskeleton for arm therapy in stroke patients. In Rehabilitation Robotics, 2007. ICORR 2007. IEEE 10th International Conference on (pp. 68-74). IEEE.

Ong, M. E. H., Ng, C. H. L., Goh, K., Liu, N., Koh, Z. X., Shahidah, N., ... & Lin, Z. (2012). Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Critical Care, 16(3), R108.

Popescu, V. G., Burdea, G. C., Bouzit, M., & Hentz, V. R. (2000). A virtual-reality-based telerehabilitation system with force feedback. IEEE transactions on Information Technology in Biomedicine, 4(1), 45-51.

Reinkensmeyer, D. J., Kahn, L. E., Averbuch, M., McKenna-Cole, A., Schmit, B. D., & Rymer, W. Z. (2014). Understanding and treating arm movement impairment after chronic brain injury: progress with the ARM guide.

Ren, Y., Wu, Y. N., Yang, C. Y., Xu, T., Harvey, R. L., & Zhang, L. Q. (2017). Developing a wearable ankle rehabilitation robotic device for in-bed acute stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(6), 589-596.

Simonyan, K., & Zisserman, A. (2014). Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems (pp. 568-576).

Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

Sun, W. H., & Tang, J. W. (2017). Smart Clothing Integrates Both Health and Maximum Comfort. Journal of Fiber Science and Technology, 73(12), 348-352.
Szolovits, P. (Ed.). (1982). Artificial intelligence in medicine (pp. 1-226). Boulder, CO: Westview Press.

Volpe, B. T., Krebs, H. I., Hogan, N., Edelstein, L., Diels, C., & Aisen, M. (2000). A novel approach to stroke rehabilitation robot-aided sensorimotor stimulation. Neurology, 54(10), 1938-1944.

Vyborny, C. J., & Giger, M. L. (1994). Computer vision and artificial intelligence in mammography. AJR. American journal of roentgenology, 162(3), 699-708.

Wang, Y., Wang, L., Yang, T., Li, X., Zang, X., Zhu, M., ... & Zhu, H. (2014).

Wearable and highly sensitive graphene strain sensors for human motion monitoring. Advanced Functional Materials, 24(29), 4666-4670.

Weindelt, B. (2016, January). Digital Transformation of Industries: Automotive Industry. In World Economic Forum in collaboration with Accenture (p. 4).

Zhang, Z. (2012). Microsoft kinect sensor and its effect. IEEE multimedia, 19(2), 4-10.