本篇論文提出一個以動作感測為主的手指穿戴式裝置，建構出平面上的手寫辨識系統。
我們將裝置放在使用者食指上，利用手指環上的慣性感測器收集加速度以及角速度的資料，透過藍芽通訊協定的方式傳送到電腦端，進行資料處理。我們提出了一系列的動作分段以及字母辨識的方法。實驗結果顯示，我們的演算法能準確的切割出與辨試出手寫單字中的每個字母。字母錯誤率以及單字錯誤率分別可以達到1.05%與7.28%，讓我們的系統可以做為一個有效率的文字輸入介面。

This thesis proposes a finger-worn device based on an inertial sensor for an on-surface finger-writing recognition system. The user wears the ring sensor on the index finger to transmit the acceleration and the angular velocity data to a host computer by Bluetooth Low Energy (BLE) for conversion into words. We propose a series of segmentation and letter recognition methods based on Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) variant of Recurrent Neural Network (RNN) models. Experiment results show good precision for letter segmentation high accuracy for letter recognition, making it a viable system as a text input interface.

Contents
Contents i
Acknowledgments vi
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Related Work 3
2.1 Vision-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Motion-based . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 Background 6
3.1 Recurrent Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.2 Long Short-Term Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4 Technical Approach 11
4.1 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.1.1 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.1.2 Gravity Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.1.3 Signal Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.1.4 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2.1 Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2.2 LSTM Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2.3 Post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.3 Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.3.1 Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.3.2 LSTM Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.3.3 CNN Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5 System Architecture and Implementation 24
5.1 Node Subsystesm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.2 Host Subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
6 Evaluation 26
6.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.1.1 System Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.1.2 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.2 Word Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.2.1 Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.2.2 Segmentation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.3 Word and Character Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.3.1 Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.3.2 Recognition Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.3.3 Performance Comparison Between CNN Model and LSTM Model . . . . . . 34
6.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
7 Conclusions and Future Work 35
7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Bibliography 37

[1] Gauri Katiyar and Shabana Mehfuz. MLPNN based handwritten character recognition using combined feature extraction. In International Conference on Computing, Communication & Automation, pages 1155–1159. IEEE, 2015.
[2] Theodore Bluche, Hermann Ney, and Christopher Kermorvant. Tandem HMM with convolutional neural network for handwritten word recognition. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 2390–2394. IEEE, 2013.
[3] Jorge Sueiras, Victoria Ruiz, Angel Sanchez, and Jose F Velez. Offline continuous handwriting recognition using sequence to sequence neural networks. Neurocomputing, 289:119–128, 2018.
[4] Leap Motion. https://www.leapmotion.com, 2015.
[5] Dae-Won Kim, Jaesung Lee, Hyunki Lim, Jeongbong Seo, and Bo-Yeong Kang. Efficient dynamic time warping for 3D handwriting recognition using gyroscope equipped smartphones. Expert Systems with Applications, 41(11):5180–5189, 2014.
[6] Christoph Amma, Marcus Georgi, and Tanja Schultz. Airwriting: Hands-free mobile text input by spotting and continuous recognition of 3D-space handwriting with inertial sensors. In 2012 16th International Symposium on Wearable Computers, pages 52–59. IEEE, 2012.
[7] Mingyu Chen, Ghassan AlRegib, and Biing-Hwang Juang. Air-writing recognition—Part I: Modeling and recognition of characters, words, and connecting motions. IEEE Transactions on Human-Machine Systems, 46(3):403–413, 2016.
[8] Mingyu Chen, Ghassan AlRegib, and Biing-Hwang Juang. Air-writing recognition—Part II: Detection and recognition of writing activity in continuous stream of motion data. IEEE Transactions
on Human-Machine Systems, 46(3):436–444, 2016.
[9] Ji Gan, Weiqiang Wang, and Ke Lu. A Unified CNN-RNN Approach for in-Air Handwritten English Word Recognition. In 2018 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2018.
[10] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735–1780, 1997.
[11] Embedded Platform Lab (EPL) at National Tsing Hua University (NTHU) in Taiwan. Eco Mini. http://epl.tw/ecomini/, 2014.
[12] Python interface to Bluetooth LE on Linux. https://github.com/IanHarvey/bluepy.
[13] François Chollet et al. Keras. https://keras.io, 2015.