Comparison of MCL and MCEM

The gradient of (1.46) is

 

where the empirical expectation operator for the distribution with unnormalized density using samples having unnormalized density is defined by (1.34). MCL solves equation (1.47) to determine the MCMLE .

MCEM uses the same formula, but much less efficiently. It only uses the special case where in we set the current iterate of the MCEM algorithm. This simplifies the formulas by eliminating the importance weights. But the cost of this simplification is that (1.47) becomes

and only the 's inside the integrands are considered variables. The 's subscripting the expectation operators should also be variable, but EM leaves them fixed. The result is that MCEM takes hundreds of iterations, each involving the collection of two Monte Carlo samples to do what MCL does with no iteration as long as is close enough to the exact MLE so that the importance weights behave. The advantages of MCL methods over MCEM are

• MCL methods are faster, requiring many fewer iterations.
• MCL methods permit calculation of everything. MCEM only calculates MLEs, not likelihood ratios, not Fisher information.
• MCL methods permit calculation of Monte Carlo standard errors, MCEM doesn't.

The advantages of MCEM are weak

• MCEM allows one to avoid the importance sampling theory presented in the preceding section.
• Most statisticians have heard of EM, so MCEM sounds reasonable to people unfamiliar with the area.
• MCEM is better than MCNR because it is more stable.

But there are no real advantages of MCEM over MCL. The best that can be said for it is that it might be reasonable to do a few MCEM iterations to get close to MLE before switching to MCL. However, it must compete with stochastic approximation in that role, and it is not clear that MCEM has any advantages over stochastic approximation in providing crude estimates.