指紋是獨特且相當可靠的生物特徵，近年來已經被廣泛使用到手持裝置上來做為身分辨識功能。而在某些情況下的指紋辨識，需要將受測者的指紋與龐大的指紋資料庫做比對來辨認出使用者的真實身分，因此比對速度與辨識準確度變成相當重要的課題。
在傳統指紋辨識系統中，指紋比對方式通常是採用區域描述子(local descriptor)配對，這種方法雖然擁有相當不錯的辨識率，但是因為區域描述子間的配對必須嘗試過所有排列組合後，才能得到較好的配對，而這部分的時間通常是非常長的。為了克服配對時間過長的缺點，我們可以利用全域描述子(global descriptor)來加速指紋影像間的配對。但是全域描述子相當難以設計，因此我們提出使用深度卷積神經網路(deep convolutional neural network)的方式，去學習如何將一張指紋影像直接轉換成一個特徵向量，進而製作成全域描述子。
另外一個難題是，指紋影像通常稀少且珍貴，而要使深度卷積神經網路學習效果好，提供夠多的影像是必需的。為了克服這個問題，我們使用了資料擴增(data augmentation)結合課程式學習(curriculum learning)的訓練方式，使得在資料沒有這麼龐大的情況下也能得到不錯的訓練結果。
我們在FVC資料庫上做實驗，並且嘗試了幾種不同的模型以及不同的全域描述子大小。使用Inception-ResNet-v2模型並經過了約52小時的訓練之後，在使用512位元全域描述子的情況下，對於15個FVC資料庫平均配對準確度達到了95.4%，平均EER達到了4.434%。執行速度方面，在有GPU加速的情況下，提取一張影像的全域描述子的時間僅需20毫秒，比對兩個512位元全域描述子的時間僅需2微秒。

Fingerprint is a unique and reliable biometric feature that has been widely used in recent years for personal identification recognition. In some cases, fingerprint recognition needs to match the query fingerprints in a large database to identify the user's identity, so the matching speed and the identification accuracy becomes a very important issue.

In a traditional fingerprint identification system, the fingerprint recognition is done by local descriptors matching and can get a very good recognition rate usually. However, to get the best local descriptor pairs, it needs to try all combinations of all the local descriptors, and its computational cost is high. In order to solve the long matching time, we can use global descriptors to speed up the matching between fingerprint images. However, a global descriptor is quite difficult to design, so we propose to use deep convolutional neural networks to learn how to map a fingerprint image directly into a feature vector, and then convert it into a global descriptor.
Another problem for deep learning fingerprint recognition is that the amount of fingerprint images are usually not enough to have satisfactory learning effect. Therefore, providing enough of training images is important. In order to overcome this problem, we use data augmentation combined with the curriculum learning training methods to get good training results even the training data is not enough.
We perform experiments on the FVC databases, and we have tried several different models and different global descriptor sizes. While using Inception-ResNet-v2 model and training for about 52 hours, we get an average 95.4% accuracy, with an average 4.434% equal error rate (EER) with 512 bits global descriptors on 15 FVC databases. To extract a global descriptor from an image, it takes only 20 milliseconds with GPU acceleration and comparing two 512 bits global descriptors for just 2 microseconds without GPU acceleration.

1 Introduction 1
1.1 Motivation and Problem Description . . . . . . . . . . . . . . . . . 1
1.2 Goal and Contribution . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Related Works 6
2.1 Fingerprint Global Descriptors . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Reference-Based Approach . . . . . . . . . . . . . . . . . . . 8
2.1.2 Score-Based Approach . . . . . . . . . . . . . . . . . . . . . 8
2.1.3 Histogram-Based Approach . . . . . . . . . . . . . . . . . . 9
2.1.4 Spectral Transform-Based Approach . . . . . . . . . . . . . 10
2.2 Extract Global Descriptors Using Deep Learning . . . . . . . . . . 11
2.3 Supervised Semantics-Preserving Deep Hashing (SSDH) . . . . . . 14
2.4 Face Recognition Using Deep Learning . . . . . . . . . . . . . . . . 16
3 Learning Global Descriptor Using Supervised Deep Convolutional
Neural Networks for Fingerprint Recognition 19
3.1 Proposed Network Structure for Fingerprint Recognition . . . . . . 20
3.2 Objective Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2.1 Learning Discriminative Global Descriptor . . . . . . . . . . 25
3.2.2 Learning Binary Global Descriptor . . . . . . . . . . . . . . 26
3.2.3 Learning Compact Global Descriptor . . . . . . . . . . . . . 27
3.2.4 Overall Objective Function . . . . . . . . . . . . . . . . . . 29
4 Training Skills 30
4.1 Data Augmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2 Curriculum Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5 Experimental Results 36
5.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.2 Evaluation Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3 Training Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.3.1 Evaluation Results for Different Learning Strategies . . . . . 42
5.3.2 Evaluation Results for Different Descriptor Sizes . . . . . . 47
5.3.3 Evaluation Results for Different CNN Models . . . . . . . . 50
5.3.4 Evaluation Results for Computational Time . . . . . . . . . 52
5.3.5 Comparison with the State-of-the-art Methods . . . . . . . . 53
6 Conclusions and Future Works 57
6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.2 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

[1] H.-F. Yang, K. Lin, and C.-S. Chen, “Supervised learning of semanticspreserving
hash via deep convolutional neural networks,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, 2017.
[2] Improving Inception and Image Classification in TensorFlow, https:
//research.googleblog.com/2016/08/improving-inception-and-image.html,
2017.
[3] K. Simonyan and A. Zisserman, “Very deep convolutional networks for largescale
image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[4] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings
of the IEEE conference on computer vision and pattern recognition,
2015, pp. 1–9.
[5] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4, inceptionresnet
and the impact of residual connections on learning.” in AAAI, 2017,
pp. 4278–4284.
[6] D. Maltoni, D. Maio, A. Jain, and S. Prabhakar, Handbook of fingerprint
recognition. Springer Science & Business Media, 2009.
[7] R. Cappelli, M. Ferrara, and D. Maltoni, “Minutia cylinder-code: A new representation
and matching technique for fingerprint recognition,” IEEE Transactions
on Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp.
2128–2141, 2010.
[8] T.-T. Chu and C.-T. Chiu, “A cost-effective minutiae disk code for fingerprint
recognition and its implementation,” in Acoustics, Speech and Signal Processing
(ICASSP), 2016 IEEE International Conference on. IEEE, 2016, pp.
981–985.
[9] A. Nagar, S. Rane, and A. Vetro, “Privacy and security of features extracted
from minutiae aggregates,” in Acoustics Speech and Signal Processing
(ICASSP), 2010 IEEE International Conference on. IEEE, 2010, pp. 1826–
1829.
[10] E. Liu, H. Zhao, J. Liang, L. Pang, H. Chen, and J. Tian, “Random local
region descriptor (rlrd): A new method for fixed-length feature representation
of fingerprint image and its application to template protection,” Future
Generation Computer Systems, vol. 28, no. 1, pp. 236–243, 2012.
[11] Z. Jin, M.-H. Lim, A. B. J. Teoh, B.-M. Goi, and Y. H. Tay, “Generating
fixed-length representation from minutiae using kernel methods for fingerprint
authentication,” IEEE Transactions on Systems, Man, and Cybernetics: Systems,
vol. 46, no. 10, pp. 1415–1428, 2016.
[12] F. Farooq, R. M. Bolle, T.-Y. Jea, and N. Ratha, “Anonymous and revocable
fingerprint recognition,” in Computer Vision and Pattern Recognition, 2007.
CVPR’07. IEEE Conference on. IEEE, 2007, pp. 1–7.
[13] Z. Jin, A. B. J. Teoh, T. S. Ong, and C. Tee, “A revocable fingerprint template
for security and privacy preserving.” KSII Transactions on Internet &
Information Systems, vol. 4, no. 6, 2010.
[14] H. Xu, R. N. Veldhuis, A. M. Bazen, T. A. Kevenaar, T. A. Akkermans,
and B. Gokberk, “Fingerprint verification using spectral minutiae representations,”
IEEE Transactions on Information Forensics and Security, vol. 4,
no. 3, pp. 397–409, 2009.
[15] K. Nandakumar, “A fingerprint cryptosystem based on minutiae phase spectrum,”
in Information Forensics and Security (WIFS), 2010 IEEE International
Workshop on. IEEE, 2010, pp. 1–6.
[16] Y. Tang, F. Gao, and J. Feng, “Latent fingerprint minutia extraction using
fully convolutional network,” arXiv preprint arXiv:1609.09850, 2016.
[17] L. Jiang, T. Zhao, C. Bai, A. Yong, and M. Wu, “A direct fingerprint minutiae
extraction approach based on convolutional neural networks,” in Neural
Networks (IJCNN), 2016 International Joint Conference on. IEEE, 2016,
pp. 571–578.
[18] H.-R. Su, K.-Y. Chen, W. J. Wong, and S.-H. Lai, “A deep learning approach
towards pore extraction for high-resolution fingerprint recognition,”
in Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International
Conference on. IEEE, 2017, pp. 2057–2061.
[19] R. D. Labati, A. Genovese, E. Muñoz, V. Piuri, and F. Scotti, “A novel pore
extraction method for heterogeneous fingerprint images using convolutional
neural networks,” Pattern Recognition Letters, 2017.
[20] K. Lin, J. Lu, C.-S. Chen, and J. Zhou, “Learning compact binary descriptors
with unsupervised deep neural networks,” in Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2016, pp. 1183–1192.
[21] V. Erin Liong, J. Lu, G. Wang, P. Moulin, and J. Zhou, “Deep hashing for
compact binary codes learning,” in Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, 2015, pp. 2475–2483.
[22] R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, “Supervised hashing for image
retrieval via image representation learning.” in AAAI, vol. 1, 2014, pp. 2156–
2162.
[23] F. Zhao, Y. Huang, L. Wang, and T. Tan, “Deep semantic ranking based
hashing for multi-label image retrieval,” in Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2015, pp. 1556–1564.
[24] R. Zhang, L. Lin, R. Zhang, W. Zuo, and L. Zhang, “Bit-scalable deep hashing
with regularized similarity learning for image retrieval and person reidentification,”
IEEE Transactions on Image Processing, vol. 24, no. 12, pp.
4766–4779, 2015.
[25] H. Lai, Y. Pan, Y. Liu, and S. Yan, “Simultaneous feature learning and hash
coding with deep neural networks,” in Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2015, pp. 3270–3278.
[26] W.-J. Li, S. Wang, and W.-C. Kang, “Feature learning based deep supervised
hashing with pairwise labels,” arXiv preprint arXiv:1511.03855, 2015.
[27] H. Zhu, M. Long, J. Wang, and Y. Cao, “Deep hashing network for efficient
similarity retrieval.” in AAAI, 2016, pp. 2415–2421.
[28] T. Yao, F. Long, T. Mei, and Y. Rui, “Deep semantic-preserving and rankingbased
hashing for image retrieval.” in IJCAI, 2016, pp. 3931–3937.
[29] W. Liu, H. Ma, H. Qi, D. Zhao, and Z. Chen, “Deep learning hashing for
mobile visual search,” EURASIP Journal on Image and Video Processing,
vol. 2017, no. 1, p. 17, 2017.
[30] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with
deep convolutional neural networks,” in Advances in neural information processing
systems, 2012, pp. 1097–1105.
[31] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in Proceedings of the IEEE conference on computer vision and
pattern recognition, 2016, pp. 770–778.
[32] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding
for face recognition and clustering,” in Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2015, pp. 815–823.
[33] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial
examples,” arXiv preprint arXiv:1412.6572, 2014.
[34] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,”
in Proceedings of the 26th annual international conference on machine
learning. ACM, 2009, pp. 41–48.
[35] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S.
Corrado, A. Davis, J. Dean, M. Devin et al., “Tensorflow: Large-scale machine
learning on heterogeneous distributed systems,” arXiv preprint arXiv:
1603.04467, 2016.
[36] T. Tieleman and G. Hinton, “Rmsprop: Divide the gradient by a running
average of its recent magnitude. coursera: Neural networks for machine learning,”
Technical report, 2012. 31, Tech. Rep., 2012.
[37] Fingerprint Verification Competition (FVC2000), http://bias.csr.unibo.it/
fvc2000/databases.asp.
[38] Fingerprint Verification Competition (FVC2002), http://bias.csr.unibo.it/
fvc2002/databases.asp.
[39] Fingerprint Verification Competition (FVC2004), http://bias.csr.unibo.it/
fvc2004/databases.asp.
[40] Fingerprint Verification Competition (FVC2006), http://bias.csr.unibo.it/
fvc2006/databases.asp.
[41] R. Cappelli, D. Maio, and D. Maltoni, “Sfinge: an approach to synthetic
fingerprint generation,” in International Workshop on Biometric Technologies
(BT2004), 2004, pp. 147–154.