mmsb模型PPT课件

1、.,1,Mixed Membership Stochastic Blockmodels,Journal of Machine Learning Research 9 (2008) 1981-2014 Submitted 5/07; Revised 6/08; Published 9/08,.,2,问题,.,3,问题方法,静态社区检测,.,4,模型评价,模型评价,HQ准则,.,5,文章脉络图,.,6,Edoardo M Airoldi,Research Interests Theory and methods for combinatorial and network data Design a

2、nd analysis of experiments and observational studies with interfering units Design and analysis of network sampling designs, including inference from non-ignorable designs Geometry and inference in ill-posed inverse problems, including network tomography and contingency tables Distributed inference

3、Modeling and inference in high-throughput biology, including sequencing and mass spectrometry Applications to computational social science, including text Sample Publications EM Airoldi, AW Blocker.Estimating latent processes on a network from indirect measurements.Journal of the American Statistica

4、l Association, 108, 149-164, 2013. N Slavov, EM Airoldi, A van Oudenaarden, D Botstein.A conserved cell growth cycle can account for the environmental stress responses of divergent eukaryotes.Molecular Biology of the Cell. 23, 1986-1997, 2012. H Azari, EM Airoldi.Graphlet decomposition of a weighted

5、 network.Journal of Machine Learning Research, W Bayesian approaches to confidentiality and data disclosure; causation; foundations of statistical inference; history of statistics; sample surveys and randomized experiments; statistics and the law; inference for multiple-media data. principal researc

6、h interests lie in the development of statistical methodology, especially for problems involving categorical variables. Publication Fienberg, Stephen E. (2006). When Did Baysian Inference Become Bayesian?Bayesian Analysis, 1, 1-40. For a pdf version of this paper,click here. For a related annotated

7、bibliography, see: Fienberg, Stephen E. (2005). A “Bayesian Classics” Reading List.ISBA Bulletin, 12(3), September 2005, 9-14. For a pdf version,click here. Airoldi, Edoardo M., Anderson, Analise G., Fienberg, Stephen E., and Skinner Kiron K. (2006). Who Wrote Ronald Reagans Radio Addresses?,Bayesia

8、n Analysis,1, 289-320. For a pdf version, Fienberg, Stephen E. (2006).Privacy and Confidentiality in an e-Commerce World: Data Mining, Data Warehousing, Matching and Disclosure Limitation.Statistical Science, 21, 143-154. For a pdf version,click here. Araneda, Anita, Fienberg, Stephen E. and Soto, A

9、lvaro (2007).A statistical approach to simultaneous mapping and localization for mobile robots,Annals of Applied Statistics, 1, No. 1, 66-84. Fienberg, Stephen E. (2007). Editorial - Expanding the statistical toolkit with algebraic statistics.Statistica Sinica, 17(4),1261-1272. Erosheva, Elena A., F

10、ienberg, Stephen E., and Joutard, Cyrille (2007). Describing disability through individual-level mixture models for multivariate binary data,Annals of Applied Statistics, 1, No. 2, 502-537. Fienberg, Stephen E. and Kim, Sung-Ho (2007). Positive association among three binary variables and cross-prod

11、uct ratios,Biometrika, 94, 999-1005. Fienberg, Stephen E. (2008). The Early Statistical Years: 19471967. A conversation with Howard Raiffa,Statistical Science, 23, No. 1, 136-149. Jackson, L. Fraser, Gray, Alistair G., and Fienberg, Stephen E. (2008). Sequential category aggregation and partitioning

12、 approaches for multi-way contingency tables based on survey and census data,Annals of Applied Statistics, 2, No. 3, 955-981.,.,9,Eric P. Xing,research interests research interests lie in the development ofmachine learning and statistical methodology, andlarge-scale computational system and architec

13、ture, for solving problems involving automated learning, reasoning, and decision-making in high-dimensional, multimodal, and dynamic possible worlds in artificial, biological, and social systems. Publication(2014) W. Wang, Y. Liang and E. P. Xing,Sharp Threshold for Multivariate Multi-Response Linea

14、r Regression via Block Regularized Lasso,IEEE Transactions on Information Theory, in press, 2014. J. Eisenstein, B. OConnor, N. A. Smith, and E. P. Xing,Diffusion of Lexical Change in Social Media,PLoS One, volume 9, Issue 11, e113114, 2014. (arXiv:1210.5268, communicated 18 Oct 2012.) S. Lee, J. K.

15、 Kim, X. Zheng, Q. Ho, G. A. Gibson, and E. P. Xing,On Model Parallelization and Scheduling Strategies for Distributed Machine Learning,Advances in Neural Information Processing Systems 28 (eds. Corinna Cortes and Neil Lawrence), MIT Press, 2014.(NIPS 2014).(arXiv:1312.5766, communicated 19 Dec 2013

16、.) A. Dubey, Q. Ho, S.Williamson, and E. P. Xing,Dependent Nonparametric Trees for Dynamic Hierarchical Clustering,Advances in Neural Information Processing Systems 28 (eds. Corinna Cortes and Neil Lawrence), MIT Press, 2014.(NIPS 2014). Supplemental Material A. P. Parikh, A. Saluja, C. Dyer and E.

17、P. Xing,Language Modeling with Power Low Rank Ensembles,Proceeding of the 2014 Conference on Empirical Methods on Natural Language Processing.(EMNLP 2014).(Recipient of the runner-up for BEST PAPER Award)(arXiv:1312.7077, communicated 26 Dec 2013.) W. Neiswanger, C. Wang and E. P. Xing,Asymptoticall

18、y Exact, Embarrassingly Parallel MCMC,Proceedings of the 30th International Conference on Conference on Uncertainty in Artificial Intelligence(UAI 2014).(arXiv:1311.4780, communicated 19 Nov 2013.) W. Neiswanger, C. Wang, Q. Ho and E. P. Xing,Modeling Citation Networks using Latent Random Offsets,Pr

19、oceedings of the 30th International Conference on Conference on Uncertainty in Artificial Intelligence(UAI 2014).,.,10,content,Introduction The Mixed Membership Stochastic Blockmodel Parameter Estimation and Posterior Inference Experiments and Results,.,11,content,Introduction The Mixed Membership S

20、tochastic Blockmodel Parameter Estimation and Posterior Inference Experiments and Results,.,12,Introduction,Standard model-based clustering methodsThey assume that the objects are conditionally independent giventheir cluster assignments latent stochastic blockmodelPlay a single latent role mixed mem

21、bership model They assume that the data are conditionally independent given their latent membership vectors,.,13,content,Introduction The Mixed Membership Stochastic Blockmodel Parameter Estimation and Posterior Inference Experiments and Results,.,14,mmsb实现流程图,.,15,.,16,The Mixed Membership Stochast

22、ic Blockmodel,For each node p N: For each pair of nodes（ p, q） N* N:,the joint probability of the data Y and the latent variables:,.,17,The Mixed Membership Stochastic Blockmodel,Modeling Sparsity we introduce a sparsity parameter 0, 1 in the MMSB to characterize the source of non-interaction assumi

23、ng that the probability of a non-interaction comes from a mixture, the probability of successful interaction to Summarizing and De-Noising Pairwise Measurements MMSB can be used to summarize the data, Y, in terms of B and s. MMSB can be used to de-noise the data,Y, in terms of B and Zs. The posterio

24、r expectations of an interaction is computed as follows, in the two cases,.,18,node-specific (s) For each node p N: Collected from individual objects mixed memberships,interaction-specific(Zs) For each pair of nodes（ p, q） N* N: Collected from relational data single memberships,VS,.,19,An Illustrati

25、on: Crisis in a Cloister,Sampson suggested the existence of tight factions among the novices: the loyal opposition,the young turks ,the outcasts, and the waverers,.,20,An Illustration: Crisis in a Cloister,.,21,An Illustration: Crisis in a Cloister,.,22,content,Introduction The Mixed Membership Stoc

26、hastic Blockmodel Parameter Estimation and Posterior Inference Experiments and Results,.,23,.,24,Parameter Estimation and Posterior Inference,Two computational problems are central to the MMSB: posterior inference of the per-node mixed membership vectors and per-pair roles employ a mean-field approx

27、imation scheme parameter estimation of the Dirichlet parameters and Bernoulli rate matrix. empirical Bayes estimates of the parameters ( , B),.,25,Posterior Inference,The normalizing constant of the posterior distribution is the marginal probability of the data, which requires an integral over the s

28、implicial vectors Which is not solvable in closed form . We appeal to variational methods.,.,26,The main idea behind variational methods,posit a distribution of the latent variables with free parameters fit those parameters such that the distribution is close in Kullback-Leibler divergence to the tr

29、ue posterior,.,27,Posterior Inference,In the MMSB, we begin by bounding the log of the marginal probability of the data with Jensens inequality, We specify q as the mean-field fully-factorized family,q1 is a Dirichlet, q2 is a multinomial,.,28,Jensens inequality,.,29,Lower Bound for the Likelihood,=

30、,其中,.,30,Variational E Step,The approximate lower bound for the likelihood can be maximized using exponential family arguments and coordinate ascent,The updates for the variational multinomial parameters are,.,31,A nested variational inference scheme,To improve convergence, we employed a nested vari

31、ational inference scheme based on an alternative schedule of updates to the traditional ordering Scalars：,.,32,Variational M Step,We compute the empirical Bayes estimates of the model hyper- parameters , B We use a linear-time Newton-Raphson method, where the Hessian and gradient are The approximate MLE of B is the approximate MLE of the sparsity parameter is,H=,g=,.,33,content,Introduction The Mixed