在目前業界常見的自動光學檢測(Automated Optical Inspection)領域中，主要針對大量應用工業自動化製程的元件，如LCD面板/TFT、電晶體與PCB等進行光學檢測，判斷製程過程中是否有缺陷、瑕疵甚至其他干擾物體的影響，並對其進行統計。然而，現有大多數自動光學檢測系統，雖然誤判、漏判的發生機率很低，卻無法對不同種類或發生原因的缺陷進行有效分類，僅能確認在元件下的各部分子區域是否有缺陷發生。
本研究基於影像處理及深度學習技術，希望由軟體面對進行AOI檢測後的缺陷影像進行判斷並實現分類，使得業界之自動檢測技術更為完善。本研究目前以總數6770張樣品影像為實驗數據，分成三種不同之缺陷類別，在搭配深度學習的訓練下，普遍已達到至少85%的分類準確度。

So far the common AOI (Automatic Optical Inspection) system field of industry mainly focus on the components manufactured by automatic equipment such as LCD panel, transistor, PCB, etc. The AOI system detect the defects appear in the components/chips during the manufacturing process, then output the statistic as a feedback. Although the misjudgement ratio of the AOI system is pretty low nowadays, the AOI system have yet to classify the different type of defects validly.
Although AOI system can detect the appearance of defect, the system can hardly analyze and define the reason from defects initiatively but only find it out. Nevertheless, the occurrence of different types of flaws or defects, or even the proportion of their occurrence may have different effects on the subsequent manufacturing process. The optimized methods are also quite different depends on different situations.
This research is based on image processing and deep learning techniques. It is hoped that the program will judge and classify the defect images after AOI detection, making the industry’s automatic detection technology more perfect and complete. This study currently uses a total of 6770 sample images as experimental data, and is divided into three different defect categories. Under the training with deep learning, at least 85% of the accuracy of classification has been achieved.

摘要 1
致謝 3
目錄 6
圖目錄 9
表目錄 13
第一章 緒論 15
1.1 前言 15
1.2 研究動機 16
第二章 文獻回顧 18
2.1 影像預處理 18
2.1.1 清晰度分析 18
2.1.2 色彩概念化 19
2.1.3 霍夫變換 20
2.1.4 模板匹配 21
2.2 缺陷提取 22
2.2.1 大津演算法 22
2.2.2 形態學處理 24
2.2.3 邊緣偵測 26
2.3 深度學習 27
2.3.1 深度學習簡介 27
2.3.2 多層神經網路(Multilayer Perceptron ) 29
2.3.3 卷積神經網路(Convolution Neural Network) 32
第三章 研究方法 35
3.1 分析影像清晰度 37
3.2 影像預處理 40
3.2.1 影像尺寸正規化 40
3.2.2 影像旋轉 41
3.2.3 模板定位 43
3.3 缺陷提取 44
3.3.1 初步影像相減 44
3.3.2 卡通圖建立 46
3.3.3 區域生長的邏輯與實現 50
3.4 深度學習 51
3.4.1 建構深度學習模型 51
3.4.2 缺陷種類的訓練及分類流程 54
第四章 研究結果與討論 56
4.1 預處理結果分析 57
4.1.1 旋轉結果分析 57
4.1.2 定位結果分析 58
4.2 缺陷檢測結果觀察 59
4.2.1 影像相減結果分析 59
4.2.2 Buffer區的優劣結果分析 60
4.3 深度學習訓練成果 61
4.3.1 不同訓練資料集之結果討論 63
4.3.2 樣本數量平衡之結果討論 73
4.3.3 不同層數模型之結果討論 76
4.4 系統整合 78
4.4.1 C++影像處理及Python深度學習的整合 78
4.4.2 運算時間統計 79
第五章 結論與未來展望 81
5.1 本研究之貢獻 81
5.2 未來展望 81
參考文獻 83
 

[1] S. Pertuz, D. Puig, and M. A. Garcia, "Analysis of focus measure operators for shape-from-focus," Pattern Recognition, vol. 46, no. 5, pp. 1415-1432, 2013.
[2] J. L. Pech-Pacheco, G. Cristóbal, J. Chamorro-Martinez, and J. Fernández-Valdivia, "Diatom autofocusing in brightfield microscopy: a comparative study," Proceedings 15th International Conference on Pattern Recognition (ICPR) , vol. 3, pp. 314-317, 2000.
[3] X. Hou and L. Zhang, "Color conceptualization," Proceedings of the 15th ACM international conference on Multimedia, pp. 265-268, 2007.
[4] D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, and Y.-Q. Xu, "Color harmonization," ACM Transactions on Graphics (TOG), vol. 25, no. 3, pp. 624-630, 2006.
[5] H. P. VC, "Method and means for recognizing complex patterns," U.S. Patent No. 3,069,654. 18 Dec. 1962.
[6] R. O. Duda and P. E. Hart, "Use of the Hough transformation to detect lines and curves in pictures," Communications of the ACM, vol. 15, no. 1, pp. 11-15, 1972.
[7] R. N. Nagel and A. Rosenfeld, "Ordered search techniques in template matching," Proceedings of the IEEE, vol. 60, no. 2, pp. 242-244, 1972.
[8] A. Coates, B. Carpenter, C. Case, S. Satheesh, B. Suresh, T. Wang, D. J. Wu, and A. Y. Ng, "Text detection and character recognition in scene images with unsupervised feature learning," International Conference on Document Analysis and Recognition (ICDAR), pp. 440-445, 2011.
[9] K. Bhosale, J. Jadhav, S. Kalyankar, and P. Bhambare, "Number Plate Recognition System for Toll Collection," International Journal of Emerging Technology and Advanced Engineering, vol. 4, no. 4, pp. 729-732, 2014.
[10] N. Otsu, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, 1979.
[11] M. H. J. Vala and A. Baxi, "A review on Otsu image segmentation algorithm," International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), vol. 2, no. 2, pp. pp: 387-389, 2013.
[12] E. R. Dougherty and R. A. Lotufo, "Binary Opening and Closing," Hands-on Morphological Image Processing. SPIE press,pp. 25-33, 2003.
[13] F. Zana and J.-C. Klein, "Segmentation of vessel-like patterns using mathematical morphology and curvature evaluation," IEEE Transactions on Image Processing, vol. 10, no. 7, pp. 1010-1019, 2001.
[14] J. Canny, "A computational approach to edge detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, no. 6, pp. 679-698, 1986.
[15] J. Flusser, S. Farokhi, C. Höschl, T. Suk, B. Zitová, and M. Pedone, "Recognition of images degraded by Gaussian blur," IEEE Transactions on Image Processing, vol. 25, no. 2, pp. 790-806, 2016.
[16] W. Rong, Z. Li, W. Zhang, and L. Sun, "An improved CANNY edge detection algorithm," IEEE International Conference on Mechatronics and Automation (ICMA), pp. 577-582, 2014.
[17] F. Ardhy and F. I. Hariadi, "Development of SBC based machine-vision system for PCB board assembly Automatic Optical Inspection," International Symposium on Electronics and Smart Devices (ISESD), pp. 386-393, 2016.
[18] K. R. Kim, J. H. Cho, N. Y. Lee, H. J. Kim, S. H. Cho, H. J. Park, and B. Choi "High-precision and ultrafast UV laser system for next-generation flexible PCB drilling," Journal of Manufacturing Systems, vol. 38, pp. 107-113, 2016.
[19] W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," The Bulletin of Mathematical Biophysics, vol. 5, no. 4, pp. 115-133, 1943.
[20] G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006.
[21] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[22] Hensman, P., & Masko, D, "The Impact of Imbalanced Training Data for Convolutional Neural Networks," Degree Project in Computer Science, KTH Royal Institute of Technology in Stockholm, pp. 1-28, 2015.
[23] L. Xie, "Disturblabel: Regularizing cnn on the loss layer," IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 4753-4762, 2016.