without the LLN or CLT or Sample Size Going to Infinity

University of Minnesota, Twin Cities School of Statistics Charlie's Home Page

Geyer, C. J. (2013).

Asymptotics of Maximum Likelihood without the LLN or CLT or Sample Size Going to Infinity.

In *Advances in Modern Statistical Theory and Applications:
A Festschrift in honor of Morris L. Eaton*, G. L. Jones
and X. Shen eds.

Institute of Mathematical Statistics: Hayward, CA.

Now of historical interest only (if that) is the original tech report

Le Cam Made Simple: Asymptotics of Maximum Likelihood

without the LLN or CLT or Sample Size Going to Infinity

By

Charles J. Geyer

Technical Report No. 643 (revised)

School of Statistics

University of Minnesota

May 20, 2005

without the LLN or CLT or Sample Size Going to Infinity

By

Charles J. Geyer

Technical Report No. 643 (revised)

School of Statistics

University of Minnesota

May 20, 2005

**Abstract:**
If the log likelihood is approximately quadratic with constant Hessian,
then the maximum likelihood estimator (MLE) is approximately
normally distributed. No other assumptions are required.
We do not need independent and identically distributed data.
We do not need the law of large numbers (LLN)
or the central limit theorem (CLT).
We do not need sample size going to infinity or
anything going to infinity.

The theory presented here is a combination of Le Cam style involving local asymptotic normality (LAN) and local asymptotic mixed normality (LAMN) and Cramér style involving derivatives and Fisher information. The main tool is convergence in law of the log likelihood function and its derivatives considered as random elements of a Polish space of continuous functions with the metric of uniform convergence on compact sets. We obtain results for both one-step-Newton estimators and Newton-iterated-to-convergence estimators.

**Keywords:** Locally asymptotically normal (LAN), Maximum likelihood,
Newton's method, No-n asymptotics, Parametric bootstrap, Quadraticity.

The whole document (PDF format), an errata sheet (PDF format), and yet another errata sheet (PDF format).

The examples below illustrate the theory described in the paper. The are done using Sweave so anyone can redo the results and play with the examples. The first two examples are discussed in the paper. The third was done after the paper was submitted.

The text in the two quantitative genetics examples are the same,
but the numbers are different. The
text matches the smaller (`n` = 500) example. Most of the
way through, it doesn't matter, but at the end there is serious discrepancy
between words and numbers in the `n` = 2000 example. When
it says the bootstrap confidence interval is better than the asymptotic
confidence interval (which is correct at `n` = 500), this is
clearly false at `n` = 2000, since the intervals are the same
when reasonably rounded. Conclusion: the `n` = 2000 example
is in asymptopia, at least as far as this confidence interval
requires. The earlier comparisons of one-step and infinite-step Newton
estimators still show some departure from asymptopia at `n` = 2000.

The simple structure of the Cauchy location model allows more detailed investigation of quadraticity than is possible in more complicated models.

Model | n
| Sweave | |
---|---|---|---|

quantitative genetics | 500 | Example/fisher.pdf | Example/fisher.Rnw |

quantitative genetics | 2000 | Example2/fisher.pdf | Example2/fisher.Rnw |

Cauchy location | 30 | Cauchy/cauchy.pdf | Cauchy/cauchy.Rnw |

For historical reasons only the original version (PDF format) dated April 20, 2005 is still available.

Readers should be warned that the proof of Theorem B.1 in the original version is incorrect, although the assertions of the theorem are correct, as shown in Theorem B.3 of the new version.