Causal Discovery Based Neural Network Ensemble Method
Affiliation:

  • Article
  • | |
  • Metrics
  • |
  • Reference [27]
  • |
  • Related
  • |
  • Cited by [4]
  • | |
  • Comments
    Abstract:

    Current neural network ensemble methods usually generate accurate and diverse component networks by disturbing the training data, and therefore achieve strong generalization ability. In this paper, causal discovery is employed to discover the ancestor attributes of the class attribute on the results of the sampling process. Then, component neural networks are trained on the samples with only the ancestor attributes being used as inputs. Thus, the mechanism of disturbing the training data and the input attribute is combined to help generate accurate and diverse component networks. Experiments show that the generalization ability of the proposed method is better than or comparable to that of the ensembles generated by some prevailing methods.

    Reference
    [1]Zhou ZH, Chen SF. Neural network ensemble. Chinese Journal of Computers, 2002,25(1):1~8 (in Chinese with English abstract).
    [2]Hansen LK, Salamon P. Neural network ensembles. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1990,12(10):993~1001.
    [3]Krogh A, Vedelsby J. Neural network ensembles, cross validation, and active learning. In: Tesanro D, Touretzky D, Leen T, eds.Advances in Neural Information Processing Systems 7. Cambridge: MIT Press, 1995.231~238.
    [4]Schapire RE. The strength of weak learnability. Machine Learning, 1990,5(2):197~227.
    [5]Breiman L. Bagging predictors. Machine Learning, 1996,24(2):123~140.
    [6]Wright S. Correlation and causation. Journal of Agricultural Research, 1921,20:557~585.
    [7]Geman S, Geman D. Stochastic relaxation, gibbs distribution and the bayesian restoration of images. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1984,6(6):721~741.
    [8]Kass R, Tierney L, Kadane J. Asymptotics in bayesian computation. In: Bernardo J, DeGroot M, Lindley D, Smith A, eds. Bayesian Statistics 3. Oxford: Oxford University Press, 1988.261~278.
    [9]Saul LK, Jaakkola TS, Jordan MI. Mean field theory for sigmoid belief networks. Journal of Artificial Intelligence Research, 1996,4:61~76.
    [10]Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society, 1977,39(1):1~38.
    [11]Russell S, Norvig P. Artificial Intelligence: An Modern Approach. Englewood Cliffs: Prentice-Hall, 1995.
    [12]Robinson RW. Counting unlabeled acyclic digraphs. In: Little CHC, ed. Lecture Notes in Mathematics 622, Berlin:Springer-Verlag, 1977.28~43.
    [13]Heckerman D, Geiger D. Learning Bayesian networks. Technical Report, MSR-TR-95-02, Microsoft Research Advanced Technology Division, 1995.
    [14]Heckerman D, Geiger D, Chichering M. Learning Bayesian networks: The combination of knowledge and statistical data. Machine Leaning, 1995,20(3): 197~234.
    [15]Chickering M. Learning equivalence classes of bayesian networks structures. In: Horvitz E, Jensen F, eds. Proc. of the 12th Conf.on Uncertainty in Artificial Intelligence. Porland, OR, 1996. 150~157.
    [16]Cooper G, Herskovits E. A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 1992,9(4):309~347.
    [17]Thomas A, Spiegelhalter D, Gilks W. Bugs: A program to perform Bayesian inference using gibbs sampling. In: Bernardo J, Berger J, Dawid A, Smith A, eds. Bayesian Statistics 4. Oxford: Oxford University Press, 1992. 837~842.
    [18]Badsberg J. Model search in contingency tables by CoCo. In: Dodge Y, Whittaker J, eds. Computational Statistics. Heidelberg:Physica Verlag, 1992.251~256.
    [19]Hojsgaard S, Skjoth F, Thiesson B. User's guide to BIOFROST. Technical Report, Aalborg: Department of Mathematics and Computer Science, 1994.
    [20]Scheines R, Spirtes P, Glymour C, Meek C. Tetrad II: Tools for causal modeling. Hillsdale: Lawrence Erlbaum Associates, 1994.
    [21]Wallace C, Korb K, Dai H. Causal discovery via MML. In: Saitta L, ed. Proc. of the 13th Int'l Conf. on Machine Learning. 1996.516~524.
    [22]Freund Y, Schapire R. Experiments with a new boosting algorithm. In: Saitta L, ed. Proc. of the 13th Int'l Conf. on Machine Learning. 1996. 148~156.
    [23]Murphy K. The bayes net toolbox for Matlab. Computing Science and Statistics, 2001,33:331~351.
    [24]Rumelhart D, Hinton G, Williams R. Learning representations by backpropagating errors. Nature, 1986,323(9):533~536.
    [25]Blake C, Keogh E, Merz C. UCI repository of machine learning data bases. Department of Information and Computer Science,University of California, 1998. http://www.ics.uci.edu/~mlearn/Mlrepository.html
    [26]Dai H, Li G, Zhou ZH. Ensembling MML causal discovery. In: Dai H, Srikant R, Zhang C, eds. Lecture Notes in Artificial Intelligence 3056. Berlin: Springer-Verlag, 2004. 260~271.
    [27]周志华,陈世福.神经网络集成.计算机学报,2002,25(1):1~8.
    Related
    Comments
    Comments
    分享到微博
    Submit
Get Citation

凌锦江,周志华.基于因果发现的神经网络集成方法.软件学报,2004,15(10):1479-1484

Copy
Share
Article Metrics
  • Abstract:4797
  • PDF: 6399
  • HTML: 0
  • Cited by: 0
History
  • Received:August 04,2003
  • Revised:June 10,2004
You are the first2038544Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063