Le Cam Made Simple: Asymptotics of Maximum Likelihood
without the LLN or CLT or Sample Size Going to Infinity

University of Minnesota, Twin Cities     School of Statistics     Charlie's Home Page

Paper

Tech Report

Now of historical interest only (if that) is the original tech report

Abstract: If the log likelihood is approximately quadratic with constant Hessian, then the maximum likelihood estimator (MLE) is approximately normally distributed. No other assumptions are required. We do not need independent and identically distributed data. We do not need the law of large numbers (LLN) or the central limit theorem (CLT). We do not need sample size going to infinity or anything going to infinity.

The theory presented here is a combination of Le Cam style involving local asymptotic normality (LAN) and local asymptotic mixed normality (LAMN) and Cramér style involving derivatives and Fisher information. The main tool is convergence in law of the log likelihood function and its derivatives considered as random elements of a Polish space of continuous functions with the metric of uniform convergence on compact sets. We obtain results for both one-step-Newton estimators and Newton-iterated-to-convergence estimators.

Keywords: Locally asymptotically normal (LAN), Maximum likelihood, Newton's method, No-n asymptotics, Parametric bootstrap, Quadraticity.

The whole document (PDF format), an errata sheet (PDF format), and yet another errata sheet (PDF format).

Detailed Examples

The examples below illustrate the theory described in the paper. The are done using Sweave so anyone can redo the results and play with the examples. The first two examples are discussed in the paper. The third was done after the paper was submitted.

The text in the two quantitative genetics examples are the same, but the numbers are different. The text matches the smaller (n = 500) example. Most of the way through, it doesn't matter, but at the end there is serious discrepancy between words and numbers in the n = 2000 example. When it says the bootstrap confidence interval is better than the asymptotic confidence interval (which is correct at n = 500), this is clearly false at n = 2000, since the intervals are the same when reasonably rounded. Conclusion: the n = 2000 example is in asymptopia, at least as far as this confidence interval requires. The earlier comparisons of one-step and infinite-step Newton estimators still show some departure from asymptopia at n = 2000.

The simple structure of the Cauchy location model allows more detailed investigation of quadraticity than is possible in more complicated models.

Original Tech Report

For historical reasons only the original version (PDF format) dated April 20, 2005 is still available.

Readers should be warned that the proof of Theorem B.1 in the original version is incorrect, although the assertions of the theorem are correct, as shown in Theorem B.3 of the new version.