Convergence of the Belief Propagation Algorithm for RB Model Instances
Author:
Affiliation:

Fund Project:

National Natural Science Foundation of China (61462001, 61262006)

  • Article
  • | |
  • Metrics
  • |
  • Reference [37]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    Belief propagation algorithm is very effective in finding satisfying assignments for RB(k,n,α,rc,p) model instances where hard region becomes narrower. However, belief propagation algorithm does not always converge for factor graphs with cycles. Unfortunately, rigorous theoretical proof of this phenomenon is still lacking. Belief propagation algorithm is the most basic message passing algorithms. This study derives the convergence conditions of the belief propagation algorithm for solving RB(k,n,α,rc,p) model instances. In the RB(k,n,α,rc,p) model with k=2, α>(1/k), rc>0 and which proves that BP will be converged with high probability if p∈(0,n-2α). Experimental results show that such convergence conditions of belief propagation algorithm are very effective in two different group data from the random RB(k,n,α,rc,p) model instances. In the RB(k,n,α,rc,p) model, when n increases, the experimental convergence interval is fixed range, while the theory convergence interval become narrower. It is because the number of incompatible assignment are determined by n and p in the RB(k,n,α,rc,p).

    Reference
    [1] Martin D, Alan F, Michael M. A probabilistic analysis of randomly generated binary constraint satisfaction. Theory Computer Science, 2003,290(3):1815-1828.[doi:10.1016/S0304-3975(02)00317-1]
    [2] Creignou N, Eaude H. The SAT-UNSAT transition for random constraint satisfaction problems. Discrete Mathematics, 2009,309(8):2085-2099.[doi:10.1016/j.disc.2008.04.025]
    [3] Martin OC, Monasson R, Zecchina R. Statistical mechanics methods and phase transitions in optimization. Theory Computer Science, 2001,265(1-2):3-67.[doi:10.1016/S0304-3975(01)00149-9]
    [4] Tsang E. A glimpse of constraint satisfaction. Artificial Intelligence Review, 1999,13(3):215-277.[doi:10.1023/A:1006558104682]
    [5] Xu K, Li W. The SAT phase transition. Science in China, Series E, 1999,42(5):494-501.[doi:10.1007/BF02917402]
    [6] Fan Y, Shen J. On the phase transitions of random k-constraint satisfaction problems. Artificial Intelligence, 2011,175(3-4):914-927.[doi:10.1016/j.artint.2010.11.004]
    [7] Achlioptas D, Naro A, Peres Y. Rigorous location of phase transitions in hard optimization problems. Nature, 2005,435(7043):759-764.[doi:10.1038/nature03602]
    [8] Mertens S, Mèzard M, Zecchina R. Threshold values of random k-SAT from the cavity method. Random Structure Algorithms, 2006,28(3):340-373.[doi:10.1002/rsa.20090]
    [9] Moskewicz MW, Madigan CF, Zhao Y. Chaff:Engineering an efficient SAT solver. In:Proc. of the 38th Annual Design Automation Conf. New York:ACM Press, 2001. 530-535.[doi:10.1145/378239.379017]
    [10] Aurell E, Gordon U, Kirkkpatrick S. Comparing beliefs, surveys, and random walks. Advances in Neural Information Processing Systems, 2004,17(1):1-8.
    [11] Mezard M, Parisi G. The cavity method at zero temperature. Journal of Statistical Physics, 2003,111(1-2):1-34.[doi:10.1023/A:1022221005097]
    [12] Mezard M, Zecchina R. Random k-satisifability problem:From an analytic solution to an efficient algorithm. Physical Review E, 2002,66(5):No.056126.[doi:10.1103/PhysRevE.66.056126]
    [13] Maneva E, Mossel E, Wainwright M. A new look at survey propagation and its generalizations. Journal of the ACM, 2007,54(4):1089-1098.[doi:10.1145/1255443.1255445]
    [14] Braunstein A, Mezard M, Zecchina R. Survey propagation:An algorithm for satisfiability. Random Structures and Algorithms, 2005,27(2):201-226.[doi:10.1002/rsa.20057]
    [15] Braunstein A, Zecchina R. Survey and belief propagation on random k-SAT. Lecture Notes in Computer Science, 2004,2919(1):519-528.[doi:10.1007/978-3-540-24605-3_38]
    [16] Yedidia JS, Freeman WT, Weiss Y. Understanding belief propagation and its generalizations. Artificial Intelligence, 2003,8(1):239-269.
    [17] Mezard M, Parisi G, Zecchina R. Analytic and algorithmic solution of random satisfiability problems. Science, 2002,297(5582):812-815.[doi:10.1126/science.1073287]
    [18] Xu K, Li W. Exact phase transitions in random constraint satisfaction problems. Journal of Artificial Intelligence Research, 2000, 12(1):93-103.
    [19] Liu T, Lin XX, Wang CY, Su KL, Xu K. Large hinge width on sparse random hyper graphs. In:Proc. of the 22nd Int'l Joint Conf. on Artificial Intelligence. Barcelona:AAAI, 2011. 611-616.[doi:10.5591/978-1-57735-516-8/IJCAI11-109]
    [20] Xu K, Li W. Many hard examples in exact phase transitions.Theory Computer Science, 2006,355(1):291-302.[doi:10.1016/j.tcs. 2006.01.001]
    [21] Xu K, Boussemart F, Hemery F, Lecoutre C. Random constraint satisfaction:Easy generation of hard instances. Artificial Intelligence, 2007,171(1):514-534.[doi:10.1016/j.artint.2007.04.001]
    [22] Zhao CY, Zheng ZM. A belief propagation algorithm based on variable entropy for constraint satisfaction problems. Science China Information Sciences, 2012,42(9):1170-1180(in Chinese).
    [23] Kschischang F, Frey B, Loeliger H. Factor graph and the sum-product algorithm. Information Theory, 2001,47(1):498-519.[doi:10.1109/18.910572]
    [24] Weiss Y. Correctness of local probability propagation in graphical models with loops. Neural Computation, 2000,12(1):1-41.[doi:10.1162/089976600300015880]
    [25] Weiss Y, Freeman WT. Correctness of belief propagation in Gaussian graphical models of arbitrary topology. Neural Computation, 2001,13(10):2173-2200.[doi:10.1162/089976601750541769]
    [26] Tatikonda S, Jordan MI. Loopy belief propagation and Gibbs measure. In:Proc. of the 18th Annual Conf. on Uncertainty in Artificial Intelligence. 2002. 493-500.
    [27] Heskes T. On the uniqueness of loopy belief propagation fixed points. Neural Computation, 2004,16(11):2379-2413.[doi:10.1162/0899766041941943]
    [28] Ihler AT, Fisher JW, Willsky AS. Loopy belief propagation:Convergence and effects of message errors. Machine Learning Research, 2005,6(1):905-936.
    [29] Mooij JM, Kappen HJ. Sufficient conditions for convergence of the sum-product algorithm. IEEE Trans. on Information Theory, 2007,53(12):4422-4437.[doi:10.1109/TIT.2007.909166]
    [30] Shi XQ, Schonfeld D, Tuninetti D. Message error analysis of loopy belief propagation for the sum-product algorithm. Computer and Information Science, 2010,1009(2):1-30.[doi:10.1109/ICASSP.2010.5495075]
    [31] Gamarnik D, Shah D, Wei Y. Belief propagation for min-cost network flow:Convergence and correctness. Operations Research, 2012,60(2):410-428.[doi:10.1287/opre.1110.1025]
    [32] Brunsch T, Cornelissen K, Manthey B, Roglin H. Smoothed analysis of BP for minimum cost flow and matching. Journal of Graph Algorithms and Applications, 2013,17(6):647-670.[doi:10.7155/jgaa.00310]
    [33] Norshams N, Wainwright MJ. Stochastic belief propagation:A low complexity alternative to the sum product algorithm. IEEE Trans. on Information Theory, 2013,59(4):1981-2000.[doi:10.1109/TIT.2012.2231464]
    [34] Lecoute. http://www.cril.univ-artois.fr/~lecoutre/benchmarks.html
    [35] Dieudonné J. Foundations of modern analysis. Vol.10-I, New York:Academic Press, 1969.
    附中文参考文献:
    [22] 赵春艳,郑志明.一种基于变量熵求解约束满足问题的置信传播算法.中国科学:信息科学,2012,42(9):1170-1180.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

王晓峰,许道云. RB模型实例集上置信传播算法的收敛性.软件学报,2016,27(11):2712-2724

Copy
Share
Article Metrics
  • Abstract:3099
  • PDF: 5057
  • HTML: 2173
  • Cited by: 0
History
  • Received:March 09,2014
  • Revised:March 10,2015
  • Online: March 30,2016
You are the first2034050Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063