Real-Time Monitoring of the Quality of Multivariate Processes with a SVM Based Classifier Ensemble Approach

應用以支援向量機為基之集成式分類器於多變量製程之即時監控

10.6220/joq.2014.21(6).02

顧瑞祥(Ruey-Shiang Guh)；薛友仁(Yeou-Ren Shiue)；余豐榮(Fong-Jung Yu)

集成式分類 ； 支援向量機 ； 多變量製程 ； 統計製程管制 ； 類神經網路 ； classifier ensemble ； support vector machine ； multivariate process ； statistical process control ； neural network

品質學報

21卷6期（2014 / 12 / 31）

427 - 454

英文

由於製程資料自動擷取系統己普遍運用於現代化之製程環境中，同時監控數個相關的製程（或品質）變數之多變量統計製程管制技術己普遍受到重視。近年來，機器學習技術（特別是類神經網路）已被用來偵測多變量製程中平均值變動之狀態，也獲致不錯的成果，但是類神經網路常有「過度學習」的困擾，而無法順利將訓練結果一般化。支援向量機是機器學習領域中，另一種較新的技術，在其學習過程中，採用結構風險最小化的原則，來避免過度學習的陷阱，所以常能有較佳的一般化能力。集成式分類器模仿人類在作重大決策前，會先諮詢多位專家意見的行為，其核心原理在於整合多個單一分類器的分類結果後，所作的決策，常比單一分類器的分類結果準確，在許多複雜的模式辨識問題中，集成式分類器的績效往往比單一分類器好。本研究應用以支援向量機為基之集成式分類技術構建一個在多變量製程中，線上即時監控平均值變動的模式。模擬數據顯示，本研究提出的模式可有效率地偵測到多變量製程中平均值的變動，而且能準確地指出那些變數的平均值已變動及其變動方向，與文獻中其他的類神經網路模式、支援向量機模式及傳統多變量管制圖相較，本研究提出的以支援向量機為基之集成式分類模式具有較佳的偵測速度（即較短的平均串連長度）。本研究提出的模式，可使品管人員更有效率且更準確地在多變量製程中，監控平均值的變動。

Using data acquisition systems and computers in on-line process control has led to increased interest in multivariate statistical process control (SPC) in which several interrelated quality variables are simultaneously monitored. Learning based techniques, especially neural networks, have been applied to detect mean shifts in multivariate processes with promising results. However, neural networks suffer from generalization problems due to overfitting. Support vector machines (SVMs) avoid the overfitting problem by adopting the structure risk minimization principle in the learning process. Classifier ensembles (i.e., combining of multiple classifiers) have been proven to be a method superior to single classifiers in many complex pattern recognition problems. With the SVM based classifier ensemble technique, this study proposes a straightforward and effective model to on-line recognize mean shifts in multivariate processes. Empirical results using simulation show that the proposed classifier ensemble model can not only efficiently recognize the mean shifts but also accurately identify the variables that have deviated from their original means. The shift direction of each of the deviated variables can also be simultaneously determined. Numerical comparisons in a bivariate scenario indicate that the proposed SVM based classifier ensemble model outperforms neural network models, SVM models, and conventional multivariate SPC approaches reported in the literature in terms of average run length. This study is useful for quality practitioners who seek efficient methods for on-line recognizing mean shifts in multivariate processes, where the investigation resulting from a false recognition is costly.

1. Hotelling, H. H., 1947, Multivariate quality control, Techniques of Statistical Analysis, edited by Eisenhart, C., Hastay, M. W., and Wallis, W. A., McGraw-Hill, New York, 111-184.
2. Breiman, L.(1996).Bagging predictors.Machine Learning,24(2),123-140.
3. Bruzzone, L.,Cossu, R.,Vernazza, G.(2004).Detection of land-cover transitions by combining multidate classifiers.Pattern Recognition letters,25(13),1491-1500.
4. Burges, C. J. C.(1998).A tutorial on support vector machines for pattern recognition.Data Mining and Knowledge Discovery,2(2),121-167.
5. Chang, C.-C.,Lin, C.-J.(2010).,Taipei:.
6. Chen, L.-H.,Wang, T.-Y.(2004).Artificial neural networks to classify mean shifts from multivariate x2 chart signals.Computers & Industrial Engineering,47(2-3),195-205.
7. Chen, W.-H.,Shih, J.-Y.(2006).A study of Taiwan's issuer credit rating systems using support vector machines.Expert Systems with Applications,30(3),427-435.
8. Cheng, C.-S.(1997).A neural network approach for the analysis of control chart patterns.International Journal of Production Research,35(3),667-697.
9. Cheng, C.-S.,Cheng, H.-P.(2008).Identifying the source of variance shifts in the multivariate process using neural networks and support vector machines.Expert Systems with Applications,35(1-2),198-206.
10. Cheng, Z.-Q.,Ma, Y.-Z.,Bu, J.(2011).Variance shifts identification model of bivariate process based on LS-SVM pattern recognizer.Communications in Statistics-Simulation and Computation,40(2),286-296.
11. Chiu, C.-C.,Shao, Y. E.,Lee, T.-S.,Lee, K.-M.(2003).Identification of process disturbance using SPC/EPC and neural networks.Journal of Intelligent Manufacturing,14(3-4),379-388.
12. Cristianini, N.,Shawe-Taylor, J.(2000).An Introduction to Support Vector Machines: and Other Kernel-Based Learning Methods.New York:Cambridge University Press.
13. Crosier, R. B.(1988).Multivariate generalizations of cumulative sum quality control schemes.Technometrics,30(3),291-303.
14. Das, P.,Banerjee, I.(2011).An hybrid detection system of control chart patterns using cascaded SVM and neural network-based detector.Neural Computing and Applications,20(2),287-296.
15. Dreiseitl, S.,Ohno-Machado, L.,Kittler, H.,Vinterbo, S.,Billhardt, H.,Binder, M.(2001).A comparison of machine learning methods for the diagnosis of pigmented skin lesions.Journal of Biomedical Informatics,34(1),28-36.
16. Du, S.,Lv, J.,Xi, L.(2010).An integrated system for on-line intelligent monitoring and identifying process variability and its application.International Journal of Computer Integrated Manufacturing,23(6),529-542.
17. Edwards, C.,Raskutti, B.(2004).The effect of attribute scaling on the performance of support vector machines.Proceedings of the 17th Australian Joint Conference on Advances in Artificial Intelligence
18. Finej, S.,Navratil, J.,Gopinath, R. A.(2001).A hybrid GMM/SVM approach to speaker identification.International Conference on Acoustics Speech and Signal Processing (ICASSP)
19. Freund, Y.,Schapire, R. E.(1996).Experiments with a new boosting algorithm.Machine Learning: Proceedings of the Thirteenth International Conference
20. Fuchs, C.,Benjamini, Y.(1994).Multivariate profile charts for statistical process control.Technometrics,36(2),182-195.
21. Guh, R.-S.(2010).Simultaneous process mean and variance monitoring using artificial neural networks.Computers & Industrial Engineering,58(4),739-753.
22. Guh, R.-S.(2007).On-line identification and Quantification of mean shifts in bivariate processes using a neural network-based approach.Quality and Reliability Engineering International,23(3),367-385.
23. Guh, R.-S.,Shiue, Y.-R.(2005).On-line identification of control chart pattern using self-organizing approaches.International Journal of Production Research,43(6),1225-1254.
24. Hachicha, W.,Ghorbel, A.(2012).A survey of control-chart pattern-recognition literature (1991-2010) based on a new conceptual classification scheme.Computers & Industrial Engineering,63(1),204-222.
25. Hall, M.,Frank, E.,Holmes, G.,Pfahringer, B.,Reutemann, P.,Witten, I. H.(2009).The WEKA data mining software: an update.SIGKDD Explorations,11(1),10-18.
26. Hawkins, D. M.(1991).Multivariate quality control based on regression-adjusted variables.Technometrics,33(1),61-75.
27. Hawkins, D. M.(1993).Regression adjustment for variables in multivariate quality control.Journal of Quality Technology,25(3),170-182.
28. Hayter, A. J.,Tsui, K.-L.(1994).Identification and quantification in multivariate quality control problems.Journal of Quality Technology,26(3),197-208.
29. Hsu, C.-W.,Chang, C.-C.,Lin, C.-J.(2010).,Taipei:.
30. Jack, L. B.,Nadi, A. K.(2001).Support vector machine for detection and characterization of rolling element bearing faults.Proceedings of the Institution of Mechanical Engineers. Part C: Journal of Mechanical Engineering Science,215(9),1065-1074.
31. Jackson, J. E.(1985).Multivariate quality control.Communications in Statistics-Theory and Methods,14(110),2657-2688.
32. Jackson, J. E.(1991).A user's Guide to Principal Components.New York:John Wiley.
33. Johnson, R. A.,Wichern, D. W.(1992).Applied Multivariate Statistical Analysis.Englewood Cliffs, NJ:Prentice-Hall.
34. Kennedy, J.,Eberhart, R. C.(1997).A discrete binary version of the particle swarm optimization.Proceedings of IEEE International Conference on Computational Cybernetics and Simulation
35. Kittler, J.,Hatef, M.,Duin, R. P. W.,Matas, J.(1998).On combining classifiers.IEEE Transactions on Pattern Analysis and Machine Intelligence,20(3),226-239.
36. Kourti, T.,MacGregor, J. F.(1996).Multivariate SPC methods for process and product monitoring.Journal of Quality Technology,28(4),409-428.
37. Kuncheva, L. I.,Whitaker, C. J.(2003).Measures of diversity in classifier ensembles and their relationship with ensemble accuracy.Machine Learning,51(2),181-207.
38. Kwok, J. T.-Y.(1998).Automated text categorization using support vector machine.Proceedings of the 5th International Conference on Neural Information Processing (ICONIP 98) / 1998 Annual Conference of the Japanese-Neural-Network-Society (JNNS 98)
39. Law, A. M.,Kelton, W. D.(1982).Simulation Modeling and Analysis.New York:McGraw-Hill.
40. Lawrence, S.,Ciles, C. L.,Tsoi, A. C.(1997).Lessons in neural network training: overfitting may be harder than expected.Proceedings of the Fourteenth National Conference on Artificial Intelligence, AAAI-97
41. Lehman, R. S.(1977).Computer Simulation and Modeling: An Introduction.Hillsdale, N.J.:Lawrence Erlbaum Associates.
42. Leigh, W.,Purvis, R.,Ragusa, J. M.(2002).Forecasting the NYSE composite index with technical analysis, pattern recognizer, neural networks, and genetic algorithm: a case study in romantic decision support.Decision Support Systems,32(4),361-377.
43. Li, T.-F.,Hu, S.,Wei, Z.-Y.,Liao, Z.-Q.(2013).A framework for diagnosing the out-of-control signals in multivariate process using optimized support vector machines.Mathematical Problems in Engineering,2013
44. Lin, H.-J.,Kao, Y.-T.,Yang, F.-W.,Wang, P. S. P.(2006).Content-based image retrieval trained by AdaBoost for mobile application.International Journal of Pattern Recognition and Artificial Intelligence,20(4),525-541.
45. Liu, Y.,Yao, X.,Tetsuya, H.(2000).Evolutionary ensemble with negative correlation learning.IEEE Transaction on Evolutionary Computation,4(4),380-387.
46. Lowry, C. A.,Montgomery, D. C.(1995).A review of multivariate control charts.IIE Transactions,27(6),800-810.
47. Lowry, C. A.,Woodall, W. H.,Champ, C. W.,Rigdon, S. E.(1992).Multivariate exponentially weighted moving average control chart.Technometrics,34(1),46-53.
48. Maimon, O.,Rokach, L.(2004).Ensemble of decision trees for mining manufacturing data sets.Machine Engineering,4(1-2),32-57.
49. Mangiameli, P.,West, D.,Rampal, R.(2004).Model selection for medical diagnosis decision support systems.Decision Support Systems,36(3),247-259.
50. Mason, R. L.,Champ, C. W.,Tracy, N. D.,Wierda, S. J.,Young, J. C.(1997).Assessment of multivariate process control techniques.Journal of Quality Technology,29(2),140-143.
51. Mason, R. L.,Tracy, N. D.,Young, J. C.(1997).A practical approach for interpreting multivariate T2 control chart signals.Journal of Quality Technology,29(4),396-406.
52. Mason, R. L.,Tracy, N. D.,Young, J. C.(1995).Decomposition of T2 for multivariate control chart interpretation.Journal of Quality Technology,27(2),99-108.
53. Mason, R. L.,Tracy, N. D.,Young, J. C.(1996).Monitoring a multivariate step process.Journal of Quality Technology,28(1),39-50.
54. Merkwirth, C.,Mauser, H.,Schulz-Gasch, T.,Roche, O.,Stahl, M.,Lengauer, T.(2004).Ensemble methods for classification in cheminformatics.Journal of Chemical Information and Modeling,44(6),1971-1978.
55. Mitchell, T. M.(1997).Machine Learning.New York:McGraw-Hill.
56. Morrison, D. F.(1976).Multivariate Statistical Methods.New York:McGraw-Hill.
57. Pignatiello, J. J., Jr.,Runger, G. C.(1990).Comparisons of multivariate CUSUM charts.Journal of Quality Technology,22(3),173-186.
58. Rokach, L.(2010).Pattern Classification Using Ensemble Method.Singapore:World Scientific.
59. Ryan, T. P.(1989).Statistical Methods for Quality Improvement.New York:Wiley.
60. Salehi, M.,Bahreininejad, A.,Nakhai, I.(2011).On-line analysis of out-of-control signals in multivariate manufacturing processes using a hybrid learning-based model.Neurocomputing,74(12-13),2083-2095.
61. Salehi, M.,Kazemzadeh, R. B.,Salmasnia, A.(2012).On line detection of mean and variance shift using neural networks and support vector machine in multivariate processes.Applied Soft Computing,12(9),2973-2984.
62. Sarle, W. S.(1995).Stopped training and other remedies for overfitting.Proceedings of the 27th Symposium on the Interface of Computing Science and Statistics
63. Schölkopf, B.,Burges, C.,Vapnik, V.(1995).Extracting support data for a given task.Proceedings, First International Conference on Knowledge Discovery & Data Mining
64. Shao, Y. E.,Lu, C.-J.,Wang, Y.-C.(2012).A hybrid ICA-SVM approach for determining the quality variables at fault in a multivariate process.Mathematical Problems in Engineering,2012(12)
65. Silver, G. A.,Silver, M.(1989).Systems Analysis and Design.Reading, MA:Addison-Wesley.
66. Sun, J.,Rahman, M.,Wong, Y. S.,Hong, G. S.(2004).Multiclassification of tool wear with support vector machine by manufacturing loss consideration.International Journal of Machine Tools & Manufacture,44(11),1179-1187.
67. Tan, A. C.,Gilbert, D.,Deville, Y.(2003).Multi-class protein fold classification using new ensemble machine learning approach.Genome Informatics,14,206-217.
68. Tay, F. E. H.,Cao, L.(2001).Application of support vector machines in financial time series forecasting.Omega,29(4),309-317.
69. Tracy, N.,Young, J.,Mason, R.(1992).Multivariate control charts for individual observations.Journal of Quality Technology,24(2),88-95.
70. Vapnik, V. N.(2000).The Nature of Statistical Learning Theory.New York:Springer.
71. Western Electric Company(1958).Statistical Quality Control Handbook.New York:Western Electric Company.
72. Weston, J.,Watkins, C.(1998).,England:.
73. Wierda, S. J.(1994).Multivariate statistical process control-recent results and directions for future research.Statistica Neerlandica,48(2),147-168.
74. Wu, B.,Yu, J.(2010).A neural network ensemble model for on-line monitoring of process mean and variance shifts in correlated processes.Expert Systems with Applications,37(6),4058-4065.
75. Yu, J.,Xi, L.(2009).A hybrid learning-based model for on-line monitoring and diagnosis of out-of-control signals in multivariate manufacturing processes.International Journal of Production Research,47(15),4077-4108.
76. Yu, J.,Xi, L.,Zhou, X.(2009).Identifying source(s) of out-of-control signals in multivariate manufacturing processes using selective neural network ensemble.Engineering Application of Artificial Intelligence,22(1),141-152.
77. Zhou, Z.-H.,Wu, J.,Tang, W.(2002).Ensembling neural networks: many could be better than all.Artificial Intelligence,137(1-2),239-263.

1. Ruey-Shiang Guh(2016).On-Line Recognition of Mean Shifts in Multivariate Processes Using a Hybrid Artificial Bee Colony Algorithm/Classifier Ensemble-Based Approach.品質學報,23(6),375-402.
2. (2022)。提升機械加工品管：智慧製造在即時量測與異常警示的實證。產業管理評論，13(1)，27-46。