Survey of Sparse Exchangeable Graph Modeling
Author:
Affiliation:

Fund Project:

National Natural Science Foundation of China (61332005, 61725205, 61402369, 61462001, 61762002); National Program on Key Basic Research Project of China (973) (2015CB352401); "Computer Application" Ningxia Provincial Key Discipline Project; Research Project of Beifang University of Nationalities (2014XBZ04)

  • Article
  • | |
  • Metrics
  • |
  • Reference [61]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    Exchangeability is a key to model network data with Bayesian model. The Aldous-Hoover representation theorem based exchangeable graph model can't generate sparse network, while empirical studies of networks indicate that many real-world complex networks have a power-law degree distribution. Kallenberg representation theorem based exchangeable graph model can admit power-law behavior while retaining desirable exchangeability. This article offers an overview of the emerging literature on concept, theory and methods related to the sparse exchangeable graph model with the Caron-Fox model and the Graphex model as examples. First, developments of random graph models, Bayesian non-parametric mixture models, exchangeability representation theorem, Poisson point process, discrete non-parametric prior etc. are discussed. Next, the Caron-Fox model is introduced. Then, simulation of the sparse exchangeable graph model and related methods such as truncated sampler, and marginalized sampler are summarized. In addition, techniques of model posterior inference are viewed. Finally, state-of-the-art and the prospects for development of the sparse exchangeable graph model are demonstrated.

    Reference
    [1] Yu ZW, Zhou XS. Relocation and discussion about pervasive computing. Communications of the CCF, 2011,7(7):49-56(in Chinese).
    [2] Yu ZW, Yu ZY, Zhou XS. Socially aware computing. Chinese Journal of Computers, 2012,35(1):16-26(in Chinese with English abstract).[doi:10.3724/SP.J.1016.2012.0001]
    [3] Freer CE, Roy DM. Computable exchangeable sequences have computable de Finetti measures. In:Ambos-Spies K, Lőwe B, Merkle W, eds. Proc. of the CiE 2009. LNCS 5635, Berlin, Heidelberg:Springer-Verlag, 2009. 218-231.
    [4] Aldous DJ. Representations for partially exchangeable arrays of random variables. Journal of Multivariate Analysis, 1981,11(4):581-598.[doi:10.1016/0047-259X(81)90099-3]
    [5] Hoover DN, Keisler HJ. Adapted probability distributions. Trans. of the American Mathematical Society, 1984,286(1):159-201.[doi:10.1090/S0002-9947-1984-0756035-8]
    [6] Kallenberg O. Exchangeable random measures in the plane. Journal of Theoretical Probability, 1990,3:81-136.[doi:10.1007/BF01063330]
    [7] Erdős P, Rényi A. On the evolution of random graphs. Trans. of the American Mathematical Society, 2011,286(1):257-274.
    [8] Airoldi EM, Costa TB, Chan SH. Stochastic blockmodel approximation of a graphon:Theory and consistent estimation. In:Advances in Neural Information Processing Systems, 2013,26:692-700.
    [9] Airoldi EM, Blei D, Fienberg SE, Xing E. Mixed membership stochastic blockmodels. Journal of Machine Learning Research, 2007,9(5):1981-2014.
    [10] Kemp C, Tenenbaum JB, Griffiths TL, Yamada T, Ueda N. Learning systems of concepts with an infinite relational model. In:Proc. of the National Conf. on Artificial Intelligence. 2006. 381-388.
    [11] Veitch V, Roy DM. The class of random graphs arising from exchangeable random measures. Electronic pre-print, arXiv:1512. 03099, 2015. http://arxiv.org/abs/1512.03099
    [12] Orbanz P, Roy DM. Bayesian models of graphs, arrays and other exchangeable random structures. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2015,37(2):437-461.[doi:10.1109/TPAMI.2014.2334607]
    [13] Barabási AR, Albert L. Statistical mechanics of complex networks. Reviews of Modern Physics, 2002,74(1):47-97.
    [14] Herlau T, Schmidt MN, Mørup M. Completely random measures for modelling block-structured sparse networks. In:Advances in Neural Information Processing Systems (NIPS). Barcelona, 2016.
    [15] Caron F, Fox E. Sparse graphs using exchangeable random measures. Journal of the Royal Statistical Society B, 2017,79:1295-1366.
    [16] Borgs C, Chayes JT, Cohn H, Ganguly S. Consistent nonparametric estimation for heavy-tailed sparse graphs. Electronic pre-print, arXiv:1508.06675, 2015. http://arxiv.org/abs/1508.06675
    [17] Kingman JFC, Poisson processes. London:Oxford University Press, 1992. 79-98.
    [18] Lloyd JR, Orbanz P, Ghahramani Z, Roy DM. Random function priors for exchangeable arrays. Advances in Neural Information Processing Systems (NIPS), 2012,25(6):1007-1015.
    [19] Lijoi A, Prünster I. Models beyond the Dirichlet process. SSRN Electronic Journal, 2009,(23):80-136.[doi:10.2139/ssrn.1526505]
    [20] Kingman JFC. Completely random measures. Pacific Journal of Mathematics, 1967,21(1):59-78.[doi:10.2140/pjm.1967.21.59]
    [21] Hougaard P. Survival models for heterogeneous populations derived from stable distributions. Biometrika, 1986,73(2):387-396.[doi:10.1093/biomet/73.2.387]
    [22] Caron F, Teh YW, Murphy TB. Bayesian nonparametric Plackett-Luce models for the analysis of preferences for college degree programmes. The Annals of Applied Statistics, 2014,8(2):1145-1181.[doi:10.1214/14-AOAS717]
    [23] Muliere P, Tardella L. Approximating distributions of random functionals of Ferguson-Dirichlet priors. The Canadian Journal of Statistics, 1998,26(2):283-297.[doi:10.2307/3315511]
    [24] Ishwaran H, Zarepour M. Exact and approximate sum represenations for the Dirichlet process. The Canadian Journal of Statistics, 2002,30(2):269-283.[doi:10.2307/3315951]
    [25] Ishwaran H, James L. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 2001, 96(453):161-173.[doi:10.2307/2670356]
    [26] Papaspiliopoulos O, Roberts GO. Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Biometrika, 2008,95(1):169-186.
    [27] Walker SG. Sampling the Dirichlet mixture model with slices. Communications in Statistics:Simulation and Computation, 2007, 36(1):45-54.[doi:10.2139/ssrn.945330]
    [28] Kalli M,Griffin JE,Walker SG. Slice sampling mixture models. Statistics and Computing, 2011,21(1):93-105.[doi:10.1007/s11222-009-9150-y]
    [29] Favaro S, Walker SG. Slice sampling s-stable Poisson-Kingman mixture models. Journal of Computational and Graphical Statistics, 2013,22(4):830-847.[doi:10.1080/10618600.2012.681211]
    [30] Favaro S, Teh YW. MCMC for normalized random measure mixture models. Statistical Science, 2013,28(3):335-359.[doi:10. 1214/13-STS422]
    [31] Lewis PAW, Shedler GS. Simulation of nonhomogeneous Poisson processes by thinning. Naval Research Logistics, 1979,26(3):403-413.[doi:10.1002/nav.3800260304]
    [32] Favaro S, Lijoi A, Nava C, Nipoti B, Prünster I, Teh YW. On the stick-breaking representation for homogeneous NRMIs. Bayesian Analysis, 2016,11(3):697-724.[doi:10.1214/15-BA964]
    [33] Orbanz P, Williamson S. Unit-Rate Poisson representations of completely random measures. Electronic Journal of Statistics, 2011,1-12. http://stat.columbia.edu/~porbanz/reports/OrbanzWilliamson2012.pdf
    [34] Ferguson TS, Klass MJ. A representation of independent increment processes without Gaussian components. The Annals of Mathematical Statistics, 1972,43(5):1634-1643.[doi:10.1214/aoms/1177692395]
    [35] Blackwell D, MacQueen JB. Ferguson distributions via Pólya urn schemes. The Annals of Statistics, 1973,1(2):353-355.
    [36] Griffin JE. An adaptive truncation method for inference in Bayesian nonparametric models. Statistics and Computing, 2016,26(1):423-441.[doi:10.1007/s11222-014-9519-4]
    [37] Neal RM. Markov chain sampling methods for dirichlet process mixture models. Journal of Computational and Graphical Statistics, 2000,9(2):249-265.[doi:10.1111/j.1467-9469.2008.00609.x]
    [38] Bishop CM. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, 2006. 548-556.
    [39] Neal RM. MCMC using Hamiltonian dynamics. In:Brooks S, Gelman A, Jones G, Meng XL, eds. Handbook of Markov Chain Monte Carlo, Vol.2. Chapman & Hall/CRC Press, 2011. 113-160.
    [40] Chen T, Fox EB, Guestrin C. Stochastic gradient Hamiltonian Monte Carlo. In:Proc. of the Int'l Conf. on Machine Learning. 2014. 1683-1691.
    [41] James L, Lijoi A, Prünster I. Posterior analysis for normalized random measures with independent increments. Scandinavian Journal of Statistics, 2009,36(1):76-97.[doi:10.1111/j.1467-9469.2008.00609.x]
    [42] Hofert M. Sampling exponentially tilted stable distributions. ACM Trans. on Modeling and Computer Simulation, 2011,22(1):Article No.3.[doi:10.1145/2043635.2043638]
    [43] Kharroubi SA, Sweeting TJ. Exponential tilting in Bayesian asymptotics. Biometrika, 2016,103(2):337-349.
    [44] Hofert M. Sampling Archimedean copulas. Computational Statistics and Data Analysis, 2008,52(12):5163-5174.[doi:10.1016/j. csda.2008.05.019]
    [45] Devroye L. Random variate generation for exponentially and polynomially tilted stable distributions. ACM Trans. on Modeling and Computer Simulation, 2009,19(4):1-20.[doi:10.1145/1596519.1596523]
    [46] Borgs C, Chayes JT, Cohn H, Holden N. Sparse exchangeable graphs and their limits via graphon processes. Electronic pre-print, arXiv:1601.07134, 2016. http://arxiv.org/abs/1601.07134
    [47] Janson S. On edge exchangeable random graphs. Journal of Statistical Physics, 2017,8:1-37.[doi:10.1007/s10955-017-1832-9]
    [48] Jacobs AZ, Clauset A. A unified view of generative models for networks:Models, methods, opportunities and challenges. Electronic pre-print, arXiv:1411.4070, 2014. http://arxiv.org/abs/1411.4070
    [49] Todeschini A, Miscouridou X, Caron F. Exchangeable random measures for sparse and modular graphs with overlapping communities. Electronic pre-print, arXiv:1602.02114, 2017. http://arxiv.org/abs/1602.02114
    [50] Zhou MY. Infinite edge partition models for overlapping community detection and link prediction. In:Proc. of the 18th Int'l Conf. on Artificial Intelligence and Statistics (AISTATS). 2015. 1135-1143.
    [51] Griffin JE, Leisen F. Compound random measures and their use in Bayesian nonparametrics. Journal of the Royal Statistical Society-Series B, 2017,79:525-545.[doi:10.1111/rssb.12176].
    [52] Palla K,Caron F,Teh YW. Bayesian nonparametrics for sparse dynamic networks. Electronic pre-print, arXiv:1607.01624, 2014. http://arxiv.org/abs/1607.01624
    [53] Matias C, Rebafka T, Villers F. A semiparametric extension of the stochastic block model for longitudinal networks. Electronic pre-print, arXiv:1512.07075v2, 2016. http://arxiv.org/abs/1512.07075v2
    [54] Roychowdhury A, Kulis B. Gamma processes, stick-breaking, and variational inference. In:Proc. of the 18th Int'l Conf. on Artificial Intelligence and Statistics (AISTATS). San Diego, 2015. 800-809.
    [55] Tank A, Foti N, Fox EB. Streaming variational inference for Bayesian nonparametric mixture models. In:Proc. of the 18th Int'l Conf. on Artificial Intelligence and Statistics. 2014. 968-976.
    [56] Salimans T, Kingma DP, Welling M. Markov chain Monte Carlo and variational inference:Bridging the gap. In:Proc. of the 32nd Int'l Conf. on Machine Learning (ICML). Lille, 2015. 1218-1226.
    [57] Wolf C, Karl M, Smagt PVD. Variational inference with Hamiltonian Monte Carlo. Electronic pre-print, arXiv:1609. 08203, 2016. http://arxiv.org/abs/1609.08203
    [58] Karrer B, Newman ME. Stochastic blockmodels and community structure in networks. Physical Review E, 2011,83(2):96-107.[doi:10.1103/PhysRevE.83.016107]
    附中文参考文献:
    [1] 於志文,周兴社.普适计算的重定位与探讨.中国计算机学会通讯,2011,7(7):49-56.
    [2] 於志文,於志勇,周兴社.社会感知计算:概念、问题及其研究进展.计算机学报,2012,35(1):16-26.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

于千城,於志文,王柱,王晓峰.稀疏可交换图建模研究综述.软件学报,2018,29(8):2448-2469

Copy
Share
Article Metrics
  • Abstract:4425
  • PDF: 6759
  • HTML: 5053
  • Cited by: 0
History
  • Received:March 16,2017
  • Revised:December 11,2017
  • Online: February 08,2018
You are the first2036798Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063