Expectation-Maximization as lower bound maximization

Thomas Minka
(1998; revised 11/29/99)

The Expectation-Maximization algorithm given by Dempster et al (1977) has enjoyed considerable popularity for solving MAP estimation problems. This note derives EM from the lower bounding viewpoint (Luttrell, 1994), which better illustrates the convergence properties of the algorithm and its variants. The algorithm is illustrated with two examples: pooling data from multiple noisy sources and fitting a mixture density.

PDF . Matlab code

Thomas P Minka