Tutorials
An introduction to Expectation Propagation (video lecture) (slides)
Video lectures on Approximate Inference
Expectation Propagation: CUED Tutorial slides
A family of algorithms
for approximate Bayesian inference
How to construct EP algorithms, illustrated with a variety of different
approximations and factor grouping schemes.
From Belief
Propagation to Expectation Propagation
K. Murphy, 2001
A quick skim of thesis Chapter 3, with some more derivations.
EP
Summary
E. Sudderth, 2002
An even shorter summary.
EP: A quick reference
A list of equations useful for constructing Gaussian EP algorithms.
Notes
on Minka's Expectation Propagation for Gaussian Process classification
M. Seeger, 2002
EP in practice
Extending expectation propagation for graphical models
Y. Qi, 2004
An overview of techniques used to implement EP in practice.
Expectation Propagation for Exponential Families
M. Seeger, 2007
Expectation propagation
J. Raymond, A. Manoel, and M. Opper, 2014
Theoretical developments
 The impact of different divergence measures

Divergence measures and message passing
 Raising factors to powers

Power EP
 Kikuchi approximation

Structured Region Graphs: Morphing EP into GBP
 Convergence control

Damping and skipping:
ExpectationPropagation for the Generative Aspect
Model
Doubleloop:
Expectation propagation for approximate inference in dynamic Bayesian networks
T. Heskes and O. Zoeter, UAI'2002
 EP within EM

ExpectationPropagation for the Generative Aspect
Model
Predictive Automatic Relevance Determination by Expectation Propagation
 Sparse approximation

Gaussian Processes  Iterative Sparse Approximations
L. Csato, 2002
Sparseposterior Gaussian Processes for general likelihoods
 The objective function

Expectation Propagation for approximate Bayesian
inference
The EP energy function and minimization
schemes
TAP Gibbs Free Energy,
Belief Propagation and Sparsity
L. Csato, M. Opper, and O. Winther, NIPS'2001
Expectation propagation for approximate inference in dynamic Bayesian networks
T. Heskes and O. Zoeter, UAI'2002
Approximate Inference Techniques with Expectation Constraints
T. Heskes, M. Opper, W. Wiegerinck, O. Winther and O. Zoeter,
Journal of Statistical Mechanics: Theory and Experiment, 11015 (2005)
 Perturbation Corrections

Approximate marginals in latent Gaussian models
B. Cseke, T. Heskes, JMLR 2011
Perturbation Corrections in Approximate Inference: Mixture Modelling Applications
Ulrich Paquet, Ole Winther, Manfred Opper, JMLR 2009
Perturbative Corrections for Approximate Inference in Gaussian Latent Variable Models
M. Opper, U. Paquet, and O. Winther, JMLR 2013
 Approximating the messages

Gaussian quadrature based expectation propagation
O. Zoeter and T. Heskes, AISTATS 2005
ABCEP: Expectation Propagation for Likelihoodfree Bayesian Computation
S. Barthelmé and N. Chopin, ICML 2011
Distributed Bayesian Posterior Sampling via Moment Sharing
M. Xu et al, NIPS 2014
Learning to Pass Expectation Propagation Messages
N. Heess, D. Tarlow, J. Winn, NIPS 2013
JustInTime Learning for Fast and Flexible Inference
S. M. Ali Eslami, D. Tarlow, P. Kohli, and J. Winn, NIPS 2014
KernelBased JustInTime Learning for Passing Expectation Propagation Messages
W. Jitkrittum et al, UAI 2015


Uses of EP
 Perceptrons, Gaussian process classifiers

Assessing Approximate
Inference for Binary Gaussian Process Classification
M. Kuss and C. E. Rasmussen, JMLR 2005
Predictive Automatic Relevance Determination by Expectation Propagation
Fast Sparse
Gaussian Process Methods: The Informative Vector Machine
N. Lawrence, M. Seeger, and R. Herbrich, NIPS'2002
Gaussian Processes  Iterative Sparse Approximations
L. Csato, 2002
A family of algorithms
for approximate Bayesian inference
Gaussian Processes for Classification: Mean Field
Algorithms
M. Opper and O. Winther, Neural Computation 12: 26552684, 2000
Bayes Machines for Binary Classification
D. HernándezLobato and J.M. HernándezLobato, Pattern Recognition Letters 29(10): 14661473, 2008
Expectation Propagation for Microarray Data Classification
D. HernándezLobato, J.M. HernándezLobato, and A. Suárez, Pattern Recognition Letters 31(12): 16181626, 2010
Robust MultiClass Gaussian Process Classification
D. HernándezLobato, J.M. HernándezLobato, and P. Dupont, NIPS 2011
Expectation Propagation for Bayesian Multitask Feature Selection
D. HernándezLobato, J.M. HernándezLobato, T. Helleppute, and P. Dupont, PKDD 2010
Generalized SpikeandSlab Priors for Bayesian Group Feature Selection Using Expectation Propagation
D. HernándezLobato, J.M. HernándezLobato, and P. Dupont, JMLR 2013
 Ordinal regression

Gaussian processes for ordinal regression
W. Chu and Z. Ghahramani, JMLR 2005
 Bayesian Conditional Random Fields
 Discrete Bayes nets and Markov random fields

Expectation Consistent Approximate Inference
M. Opper and O. Winther, JMLR 2005
Treestructured approximations by expectation
propagation
 Density estimation with Gaussian processes

Gaussian Processes  Iterative Sparse Approximations
L. Csato, 2002
 Neural networks, Multilayer perceptrons

Computing with
Finite and Infinite Networks
O. Winther, NIPS'2001
(ADATAP)
Expectation Propagation for Neural Networks with Sparsitypromoting Priors
P. Jylänki, A. Nummenmaa and A. Vehtari, JMLR 2013
Expectation backpropagation: Parameterfree training of multilayer neural networks with continuous or discrete weights
D. Soudry, I. Hubara and R. Meir, NIPS 2014
Probabilistic backpropagation for scalable learning of bayesian neural networks
J. M. HernandezLobato and R. P. Adams, ICML 2015
 Independent Components Analysis (ICA)

TAP Gibbs Free Energy,
Belief Propagation and Sparsity
L. Csato, M. Opper, and O. Winther, NIPS'2001
Mean Field
Approaches to Independent Component Analysis
P. A.d.F.R. HøjenSørensen, O. Winther, and L. K. Hansen,
Neural Computation 14: 889918 (2002)
 Text modeling, latent Dirichlet allocation

ExpectationPropagation for the Generative Aspect
Model
 Hybrid dynamic systems (continuous + discrete state)

Expectation propagation for approximate inference in dynamic Bayesian networks
T. Heskes and O. Zoeter, UAI'2002

Windowbased expectation propagation for adaptive signal detection in flatfading channels
(Fixedlag smoothing with EP)
 Nonlinear dynamic systems

Gaussian quadrature based expectation propagation
O. Zoeter and T. Heskes, AISTATS 2005

Expectation propagation for inference in nonlinear dynamical models with Poisson observations
B. M. Yu, K. V. Shenoy, M. Sahani, NSSPW'2006

Efficient computation of the maximum a posteriori path and parameter estimation in integrateandfire and more general statespace models
S. Koyama and L. Paninski, J Comp Neuroscience 2009

Iterated extended Kalman smoothing with ExpectationPropagation
A. Ypma and T. Heskes, NNSP'2003
 LowComplexity Iterative Detection for LargeScale Multiuser MIMOOFDM Systems Using Approximate Message Passing
 S. Wu et al, 2014
 Bayesian Inference for Sparse Generalized Linear Models
 M. Seeger, S. Gerwinn, M. Bethge, ECML 2007
 Bayesian inference for PlackettLuce ranking models
 J. Guiver, E. Snelson, ICML 2009
 A Distributed Message Passing Algorithm for Sensor Localization
 M. Welling and J. J. Lim, ICANN 2007
 Expectation Propagation for Continuous Time Bayesian Networks
 U. Nodelman, D. Koller, and C. R. Shelton, UAI 2005
 Visualization of timeseries data
 Hierarchical visualization of timeseries data using switching linear
dynamical systems
O. Zoeter and T. Heskes, PAMI 2003
 Expectation Propagation for Infinite Mixtures

