Student Seminar Series - July 17, 2007
University of Minnesota
School of Statistics
College of Liberal Arts

Inference after Model Selection


Yingwen Dong


Tuesday, July 17, 2007
1:30 PM, 127 Ford Hall
Minneapolis, East Bank Campus

Refreshments at 1:00 PM
300 Ford Hall


Abstract


Model selection is often employed in many scientific studies and has a significant effect on inference of parameter estimates. One common 
practice in modeling is to select a single best model from a set of candidate models using one type of model selection method. After the
model has been selected, one usually applies the standard inferential methods to an estimator directly as if the model had been known in
advance. This process simply ignores the uncertainty introduced by model selection, and could yield overoptimistic inferences, resulting in
misleading conclusions.
 
In this dissertation, we first discuss the issues of inference after model selection in the context of dose-response analysis. A general 
methodology (Shen et al. 2004) is reviewed for estimating the variance of the complex statistics that involve the process of model selection.
Simulation results are used to demonstrate the performance of data perturbation in estimating the standard error of parameter estimates
and the prediction error, as well as the impact of model selection on estimation. The application of this method to a phase I clinical trial data
set is considered.
 
A simple example of linear regression is shown to highlight the difficulty of deriving the statistical properties of estimators after model 
selection, as well as the effects of the uncertainty of model selection. A data perturbation approach is proposed to obtain an estimate of
model selection probability. The performance of this method and its application in hypothesis testing are demonstrated through simulation.
We also proposed a method to construct a confidence interval through data perturbation. The performance, as compared to the common
alternatives, is examined through simulation in various situations.