Defect Classification Using CNN Transfer Learning and Data Augmentation

運用卷積神經網路遷移學習與資料擴增方法於瑕疵之分類

10.29416/JMS.202204_29(2).0004

陳麗妃(Li-Fei Chen)；黃韻雅(Yun-Ya Huang)；張麗雪(Li-Hsueh Chang)；蘇朝墩(Chao-Ton Su)

CNN ； Transfer Learning ； Data Augmentation ； Defect Classification ； 卷積神經網路 ； 遷移學習 ； 資料擴增 ； 瑕疵分類

管理與系統

29卷2期（2022 / 04 / 01）

223 - 239

英文

With the development of industrial technology, it is common for factories to introduce automatic optical inspection systems to identify defects in production lines. However, there are still some difficulties in training image classification systems, such as unbalanced data and time-consuming labeling. In order to solve these problems, this study proposes an efficient defect classification modeling procedure by using convolutional neural networks, transfer learning, and data augmentation methods. LED lead frame defect images provided by a Taiwanese LED display manufacturer were used to construct the models and analyze model accuracy and training time. Finally, following comparative analysis, this study suggests applying an online data augmentation method to increase the variability of the dataset and then fine-tuning the pre-trained model to learn and classify the defect features; the classification accuracy of the proposed model can be as high as 98%. This result shows that, using the proposed procedure, a factory can quickly establish a defect classification system to monitor the production process.

隨著工業技術的發展，工廠通常導入自動光學檢查系統，以識別生產線中的產品缺陷。但是，訓練圖像分類系統仍然存在一些困難，例如數據不平衡和費時的標記。為了解決這些問題，本研究透過使用卷積神經網絡、遷移學習和資料擴增方法，提出了一有效的缺陷分類建模程序。使用台灣某LED顯示器製造商提供的LED板材瑕疵影像資料來構建模型，並分析模型的準確性和訓練時間。在進行比較分析之後，本研究建議使用線上資料擴增手法來增加資料集的變化性，然後再微調預訓練模型以進行瑕疵分類，該模型分類準確度可高達98%。此結果說明，使用所提出的程序，工廠可以快速建立缺陷分類系統以監視生產流程。

1. Bankar, J.,Gavai, N. R.(2018).Convolutional Neural Network Based Inception-V3 Model for Animal Classification.International Journal of Advanced Research in Computer and Communication Engineering,7(5),142-146.
2. Chen, W.,Gao, Y.,Gao, L.,Li, X.(2018).A New Ensemble Approach Based on Deep Convolutional Neural Networks for Steel Surface Defect Classification.Procedia CIRP,72,1069-1072.
3. Cheon, S.,Lee, H.,Kim, C. O.,Lee, S. H.(2019).Convolutional Neural Network for Wafer Surface Defect Classification and the Detection of Unknown Defect Class.IEEE Transactions on Semiconductor Manufacturing,32(2),163-170.
4. Ding, J.,Chen, B.,Liu, H.,Huang, M.(2016).Convolutional Neural Network with Data Augmentation for SAR Target Recognition.IEEE Geoscience and Remote Sensing Letters,13(3),364-368.
5. Fröhlich, H. B.,Fantin, A. V.,Oliveira, B. C. F.,Willemann, D. P.,Iervolino, L. A.,Benedet, M. E.,Júnior, A. A. G.(2018).Defect Classification in Shearography Images Using Convolutional Neural Networks.International Joint Conference on Neural Networks (IJCNN),Rio de Janeiro, Brazil:
6. Goodfellow, I.,Pouget-Abadie, J.,Mirza, M.,Xu, B.,WardeFarley, B.,Ozair, S.,Courville, A.,Bengio, Y.(2014).Generative Adversarial Networks.Advances in Neural Information Processing Systems 27 (NIPS 2014)
7. Han, D.,Liu, Q.,Fan, W.(2018).A New Image Classification Method using CNN Transfer Learning and Web Data Augmentation.Expert Systems with Applications,95,43-56.
8. He, K.,Zhang, X.,Ren, S.,Sun, J.(2015).,未出版
9. He, Y.,Song, K.,Meng, Q.,Yan, Y.(2020).An End-to-end Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features.IEEE Transactions on Instrumentation and Measurement,69(4),1493-1504.
10. Khumaidi, A.,Yuniarno, E. M.,Purnomo, M. H.(2017).Welding Defect Classification Based on Convolution Neural Network (CNN) and Gaussian Kernel.International Seminar on Intelligent Technology and Its Application,Surabaya, Indonesia:
11. Kim, M.,Lee, M.,An, M.,Lee, H.(2020).Effective Automatic Defect Classification Process Based on CNN with Stacking Ensemble Model for TFT-LCD Panel.Journal of Intelligent Manufacturing,31,1165-1174.
12. Kingma, D. P.,Ba, J.(2015).ADAM: A Method for Stochastic Optimization.Proceedings of the 3rd International Conference on Learning Representations
13. Krizhevsky, A.,Sutskever, I.,Hinton, G. E.(2012).ImageNet Classification with Deep Convolutional Neural Networks.Advances in Neural Information Processing Systems,25(2),1097-1105.
14. Kumar, S. S.,Abraham, D. M.,Jahanshahi, M. R.,Iseley, T.,Starr, J.(2018).Automated Defect Classification in Sewer Closed Circuit Television Inspections Using Deep Convolutional Neural Networks.Automation in Construction,91,273-283.
15. LeCun, Y.,Bottou, L.,Bengio, Y.,Haffner, P.(1998).Gradient-Based Learning Applied to Document Recognition.Proceedings of the IEEE,86(11),2278-2324.
16. Lin, C. Y.,Chen, C. H.,Yang, C. Y.,Akhyar, F.,Hsu, C. Y.,Ng, H. F.(2019).Cascading Convolutional Neural Network for Steel Surface Defect Detection.Advances in Artificial Intelligence, Software and Systems Engineering
17. Lin, H.,Li, B.,Wang, X.,Shu, Y.,Niu, S.(2019).Automated Defect Inspection of LED Chip Using Deep Convolutional Neural Network.Journal of Intelligent Manufacturing,30,2525-2534.
18. Lu, H. P.,Su, C. T.,Yang, S. Y.,Lin, Y. P.(2020).Combination of Convolutional and Generative Adversarial Networks for Defect Image Demoiréing of Thin-Film Transistor Liquid-Crystal Display Images.IEEE Transactions on Semiconductor Manufacturing,33(3),413-423.
19. Montserrat, D. M.,Lin, Q.,Allebach, J. P.,Delp, E.(2017).Training Object Detection and Recognition CNN Models Using Data Augmentation.Electronic Imaging,27-36.
20. Nakazawa, T.,Kulkarni, D. V.(2018).Wafer Map Defect Pattern Classification and Image Retrieval Using Convolutional Neural Network.IEEE Transactions on Semiconductor Manufacturing,31(2),309-314.
21. Ogawa, R.,Kido, T.,Kido, T.,Mochizuki, T.(2019).Effect of Augmented Datasets on Deep Convolutional Neural Networks Applied to Chest Radiographs.Clinical Radiology,74,697-701.
22. Shang, L.,Yang, Q.,Wang, J.,Li, S.,Lei, W.(2018).Detection of Rail Surface Defects Based on CNN Image Recognition and Classification.ICACT Conference,Huncheon, South Korea:
23. Soukup, D.,Huber-Mörk, R.(2014).Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images.Advances in Visual Computing
24. Sousa, M. J.,Moutinho, A.,Almeida, M.(2020).Wildfire Detection Using Transfer Learning on Augmented Datasets.Expert Systems with Applications,142,1-14.
25. Szegedy, C.,Liu, W.,Jia, Y.,Sermanet, P.,Reed, S.,Anguelov, D.,Erhan, D.,Vanhoucke, V.,Rabinovich, A.(2015).Going Deeper with Convolutions.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Boston, MA:
26. Szegedy, C.,Vanhoucke, V.,Ioffe, S.,Shlens, J.,Wojna, Z.(2016).Rethinking the Inception Architecture for Computer Vision.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas, NV:
27. Tanner, M. A.,Wong, W. H.(1987).The Calculation of Posterior Distributions by Data Augmentation.Journal of the American Statistical Association,398,528-540.
28. Taylor, L.,Nitschke, G.(2018).Improving Deep Learning with Generic Data Augmentation.8th IEEE Symposium Series on Computational Intelligence,Bengaluru, India:
29. Yang, H.,Mei, S.,Song, K.,Tao, B.,Yin, Z.(2018).Transfer-Learning-Based Online Mura Defect Classification.IEEE Transactions on Semiconductor Manufacturing,31(1),116-123.
30. Yang, J.,Yu, K.,Gong, Y.,Huang, T.(2009).Linear Spatial Pyramid Matching Using Sparse Coding for Image Classification.IEEE Conference on Computer Vision and Pattern Recognition,Miami, FL, USA:
31. Yunani, J. L.,Medvedev, M. V.(2019).Steel Defects Recognition Using Deep Learning.International Multi-Conference on Industrial Engineering and Modern Technologies,Vladivostok, Russia:
32. Zagoruyko, S. and Komodakis, N., Wide Residual Networks, 2016. http://www.bmva.org/bmvc/2016/papers/paper087/paper087.pdf