This web site goes with a paper and technical report.

Paper

Geyer, Charles J. (2009).
Likelihood inference in exponential families and directions of recession.
Electronic Journal of Statistics 3, (2009), 259-289 (electronic).

Abstract

When in a full exponential family the maximum likelihood estimate (MLE) does not exist, the MLE may exist in the Barndorff-Nielsen completion of the family. We propose a practical algorithm for finding the MLE in the completion based on repeated linear programming using the R contributed package \texttt{rcdd} and illustrate it with three generalized linear model examples. When the MLE for the null hypothesis lies in the completion, likelihood ratio tests of model comparison are almost unchanged from the usual case. Only the degrees of freedom need to be adjusted. When the MLE lies in the completion, confidence intervals are changed much more from the usual case. The MLE of the natural parameter can be thought of as having gone to infinity in a certain direction, which we call a generic direction of recession. We propose a new one-sided confidence interval which says how close to infinity the natural parameter may be. This maps to one-sided confidence intervals for mean values showing how close to the boundary of their support they may be.

Technical Reports

TR 673

More Supporting Data Analysis for Likelihood Inference
in Exponential Families and Directions of Recession

Charles J. Geyer
Technical Report No. 673
School of Statistics
University of Minnesota
March 24, 2009

PDF and RNW

Here is the PDF file for the technical report and the RNW (R noweb) file from which it was created. The entire technical report and all the R examples are done using the R command Sweave, so we know all the examples actually work and can be reproduced exactly by anyone who has the RNW file. See my Sweave page for more info.

This is the analysis for the third example in the revised resubmission of the paper.

TR 672

Supporting Theory and Data Analysis for Likelihood Inference
in Exponential Families and Directions of Recession

Charles J. Geyer
Technical Report No. 672
School of Statistics
University of Minnesota
September 29, 2008

PDF and RNW

Here is the PDF file for the technical report and the RNW (R noweb) file from which it was created. The entire technical report and all the R examples are done using the R command Sweave, so we know all the examples actually work and can be reproduced exactly by anyone who has the RNW file. See my Sweave page for more info.

Talks

Charlie Geyer is giving a departmental seminar in the School of Statistics, University of Minnesota on January 22, 2009. Here are the slides (PDF) for the talk (the typo Jim Hodges found during the talk has been corrected).

Charlie Geyer is giving a departmental seminar in the Department of Statistics, Penn State University on January 29, 2009. Here are the slides (PDF) for the talk.

Charlie Geyer is giving talk at the Second Midwest Statistics Research Colloquium on March 27, 2009. Here are the slides (PDF) for the talk.

Charlie Geyer is giving talk at the Thompson Symposium celebrating Elizabeth Thompson's 60th birthday, her election to the National Academy of Sciences, and her continuing contributions to statistical genetics on June 27, 2009. Here are the slides (PDF) for the talk.

What This is All About

In discrete multivariate analysis we use exponential family models and maximum likelihood estimates (MLE). Examples are logistic regression, Poisson regression, and log-linear models for categorical data.

Mostly this works great. We know about the large sample properties of MLE, and can use them to construct hypothesis tests and confidence intervals.

But there is a problem. The MLE need not exist, and when it does not all bets are off. Available software doesn't know what to do. It may give a warning, or it may produce nonsense with no warning or error message. Even if it does give a warning, that's no help, because there is — or wasn't before this technical report and paper — any software that would do something defensible.

Now there is software that will (1) find out whether or not the MLE exists and (2) perform valid hypothesis tests and confidence intervals even when it does not.