Effect of Bias and Variance on Estimation and Classification Error for Prediction

預測誤差與變異效應之估計與分類

10.6338/JDA.200606_1(3).0008

Mostafizur Rahman；朱建平(Jian-Ping Zhu)；鄭宇庭(Yu-Ting Cheng)

Journal of Data Analysis

1卷3期（2006 / 06 / 01）

113 - 135

英文

我們的目標是呈現出預測平方誤差與分類誤差的變異以及誤差的影響。我們發現估計平方誤差與分類誤差的變異以及偏誤是不同的。對於給予的誤差／變異，估計平方誤差是與各變異與誤差對稱的。分類誤差則是依誤差範圍的量而定。若誤差範圍是負的，那麼分類誤差則會不論預測誤差是否增加而減少。誤差範圍是正的情況下，分類誤差會增加預測距離的1/2。誤差範圍的影響在分類誤差中可以因小的變異而減少。相似的變異誤差則視誤差範圍的值而定。我們使用最方便的方法來使這些影響降至最小。

Our aim is to show the effect of bias and variance on squared estimation error and classification error. We found that the bias and variance affect squared estimation error and classification error differently. For a given bias/ variance, squared estimation error is proportional to variance/ bias respectively. Classification error depends on the relevant quantity of boundary bias. If the boundary bias is negative then classification error decreases with increasing irrespective of the estimation bias. For positive boundary bias the classification error increases with the distance of estimation from 1/2. The effect of boundary bias on classification error can be mitigated by low variance. Similarly the affect of the variance depends on the value of the boundary bias. And we use nearest neighbor methods for minimizing these effects.

1. Leo Breiman (1996). Statistics Department, University of California ,Berkeley, CA 94720. leo@stat.berkeley.edu
2. M. Di Marzio and C.C. Taylor(2004). Kluwer Academic Publishers. Printed in the Netherlands. kdcb4.tex; 23/12/2004; 16:34; p.1-18..
3. Bianca Zadrozny (2004). IBM T.J. Watson Research Center, Yorktown Heights, NY 10598..
4. Bellman, R. E.(1961).Adaptive Control Process.Princeton University Press.
5. Breiman, L.(1996).,Dept. of Statistics, University of California.
6. Chow, W. S.,Chen, Y. C.(1992).A new fast algorithm for effective training of neural classifiers.Pattern Recognition,25,423-429.
7. Dietterich, T. G.,Kong, E. B.(1995).,Dept. of Computer Science, Oregon State University.
8. Domungos, P.(2000).A Unified Bias-Variance Decomposition for zero-one and squared loss.Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence
9. Efron, B.,Tibshirani, R.(1995).,Dept. of Statistics, Stanford University.
10. Fix, E.,Hodges, J. L.(1951).U.S. Airforce School of Aviation Medicine Technical ReportU.S. Airforce School of Aviation Medicine Technical Report,Randolf Field Taxes:U.S. Airforce School of Aviation Medicine.
11. Friedman, J. H.(1985).Dept. of Statistics, Stanford University Technical ReportDept. of Statistics, Stanford University Technical Report,Dept. of Statistics, Stanford University.
12. Friedman, Jerome H.(1997).On Bias, Variance, 0/1-Loss, and the Curse-of-Dimensionality.Data Mining and Knowledge Discovery-DATAMINE,1(1),55-77.
13. Geman, S.,Bienenstock, E.,Doursat, R.(1992).Neural networks the bias/variance dilemma.Neural Comp.,4,1-48.
14. Habbema, J. D. F.,Gelpke, G. J.(1981).Comparison of discrimination techniques applied to a complex data set of head injured patients.J. Roy. Statist. Soc. A.,144,145-175.
15. Hand, D. J.(1982).Kernel Discriminant Analysis.Chichester:Research Studies Press.
16. Heckerman, D.,Geiger, D.,Chickering, D.(1994).Learning Bayesian networks: The combination of knowledge and statistical data.Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence,:
17. Heckman, J.(1979).Sample selection bias as a specification error.Econometrica,47,153-161.
18. Henley, W. E.,Hand, D. J.(1996).A k-nearest neighbor classifier for assessing consumer credit risk.The Statistician,45,77-95.
19. Holte, R. C.(1993).Very simple classification rules perform well on most commonly used data sets.Machine Learning,11,63-90.
20. Kohavi, R.,Wolpert, D. H.(1996).Dept. of Computer Science, Stanford University Technical ReportDept. of Computer Science, Stanford University Technical Report,未出版
21. Kohonen, T.(1990).The self-organizing map.Proceedings of the IEEF,78,1464-1480.
22. Langley, P.,Iba, W.,Thompshon, K.(1992).An analysis of Bayesian classifiers.Proceedings of the Tenth National Conference on Artificial Intelligence,:
23. Lippmann, R.(1989).Pattern classification using neural networks.IEEF Communication Magazine,11,47-64.
24. McLachlan, G. J.(1992).Discriminant Analysis and Statistical Pattern Recognition.Wiley.
25. Quinlan, J. R.(1993).C4.5 : programs for machine learning.Morgan Kaufmann.
26. Rosen, D. B.,Burke, H. B.,Goodman, P. H.(1995).Local learning methods in high dimension: Beating the bias-variance dilemma via recalibration.Workshop Machines That Learn-Neural Networks for Computing,Snowbird Utah:
27. Tibshirani, R.(1996).,Dept. of Statistics, University of Toronto.