Assignment

Solve each problem. Explain your reasoning. No credit for answers with no explanation.

2-1

Agresti, problem 1.10. Calculate both likelihood ratio and Pearson chi-square test statistics and the corresponding $P$-values. Should we expect good asymptotic approximation here?

Hint: This problem is like the six-ace-flats example in that the null hypothesis of the goodness of fit test has a parameter or parameters to estimate. But it is not that six-ace-flats model. What model is it?

Hint: For IID Poisson data, the MLE for the population mean is the sample mean. This follows from the observed equals expected principle for exponential families, which we will eventually get to, or from writing down the log likelihood and maximizing it, which requires a bit of calculus.

2-2

For the Poisson distribution with data $x$ and mean $\mu$, the log likelihood is \[ l(\mu) = x \log(\mu) - \mu, \] the score (first derivative of log likelihood) is \[ l'(\mu) = \frac{x}{\mu} - 1, \] the MLE is \[ \hat{\mu} = x, \] observed Fisher information is \[ J(\mu) = \frac{x}{\mu^2}, \] and expected Fisher information is \[ I(\mu) = \frac{1}{\mu} \] (all of these were derived in class but are not in the likelihood handout). There is no $n$, but we can think of any Poisson random variable as the sum of IID Poisson random variables (because sum of independent Poisson is Poisson). This tells us that the normal approximation for $x$ is good when $\mu$ is large and bad when $\mu$ is small, and similarly for the chi-square approximation for the null distribution of Wald, Wilks, and Rao test statistics.

What are the Wilks, Rao, and Wald two-tailed tests for testing $H_0 : \mu = \mu_0$ versus $H_1 : \mu \neq \mu_0$? Does it make a difference whether observed or expected Fisher information is used? If it does, give both forms. Apply these to do the hypothesis test for data $x = 123$ and null hypothesis $\mu_0 = 100$. (If observed or expected Fisher information makes a difference, there may be 5 tests. If not, 3 tests. If it makes a difference for one but not the other, 4 tests.)

2-3

What are the confidence intervals obtained by inverting each of the tests found in the 2.2? Apply them to the same data as in 2.2. Do 95% confidence intervals.

2-4

The negative binomial distribution is one model for overdispersed Poisson. The negative binomial has PMF in its usual parameterization \[\begin{equation} \tag{$*$} f(x) = \binom{r + x - 1}{x} p^r (1 - p)^x, \qquad x = 0, 1, \ldots \end{equation}\] where $r > 0$ and $0 < p < 1$ are parameters and where \[ \binom{r}{k} = \frac{r \cdot (r - 1) \cdots (r - k + 1)}{k !} \] (this generalizes the usual definition of binomial coefficient so it makes sense for any nonnegative integer $k$ and any positive real number $r$). The R function that evaluates the negative binomial PMF is dnbinom, and the expression that evaluates ($*$) is

dnbinom(x, r, p)

The mean and variance of a random variable with PMF ($*$) are \[\begin{align*} E(X) & = \frac{r (1 - p)}{p} \\ \mathop{\rm var}(X) & = \frac{r (1 - p)}{p^2} \end{align*}\]

Writing $E(X) = \mu$ and $\mathop{\rm var}(X) = \nu$ solving for the other parameters gives \[\begin{align*} p & = \frac{\mu}{\nu} \\ r & = \frac{\mu^2}{\nu - \mu} \end{align*}\] Since $r > 0$ we always have $\nu > \mu$.

For comparison the Poisson distribution has PMF \[ f(x) = \frac{\mu^x}{x !} e^{- \mu}, \qquad x = 0, 1, \ldots \] and mean and variance \[\begin{align*} E(X) & = \mu \\ \mathop{\rm var}(X) & = \mu \end{align*}\]

Hence the variance of a negative binomial with mean $\mu$ is always greater than the variance of a Poisson with mean $\mu$, so in this sense the negative binomial is a model of overdispersed Poisson. But the connection between the two distributions is stronger than that. It can be shown (it is one of my 5101 homework problems) that, if the conditional distribution of $X$ given $Y$ is $\text{Poisson}(Y)$ and $Y$ has a gamma distribution, then the marginal distribution of $X$ is negative binomial. So the negative binomial can arise as a mixture of Poissons with different means. Conversely, if we take negative binomial distributions with parameters $r$ and $\mu$ and let $r$ go to infinity holding $\mu$ fixed, they converge to a $\text{Poisson}(\mu)$ distribution.

Thus negative binomial with parameters $r$ and $\mu$ is a two-parameter model, and Poisson with parameter $\mu$ is a one-parameter submodel, but it is a funny submodel. It is nested within negative binomial but not nested as a submanifold. Testing negative binomial (alternative) versus Poisson (null) does not give quite the same asymptotics because this is really a one-tailed test. There is no way to let $r$ go past Poisson (which we can think of as $r = \infty$) to some sort of underdispersed Poisson. This does not satisfy the “usual regularity conditions” for MLE, Wald, Wilks, Rao. If we use the $(\mu, \nu)$ parameterization, we might have one-sided partial derivatives with respect to $\nu$ at $\nu = \mu$, but this is not obvious. Nor do the “usual regularity conditions” talk about one-sided derivatives. Thus it is unclear whether Rao tests or likelihood ratio tests make sense.

The Wald test however does make sense, since it requires only the “usual regularity conditions” for the big model. Even the Wald test is a bit strange, however, because it is a one-sided multivariate test. In this model it is impossible to have $\nu < \mu$, so we must be testing \[\begin{align*} H_0 & : \nu = \mu \\ H_1 & : \nu > \mu \end{align*}\] and this means the $P$-value is half the “usual” $P$-value, which assumes a two-tailed null hypothesis.

The data for this problem are at http://www.stat.umn.edu/geyer/5421/mydata/hw2-4.txt.

Find the MLE for the null hypothesis.
Find the MLE for the alternative hypothesis.
Perform the Wald test for these hypotheses. For the $g$ function in the the definition of the Wald test use $g(\mu, \nu) = \nu - \mu$. What does the test say about whether the data are Poisson or negative binomial?
Assuming the likelihood ratio test is OK (even though we are not sure about this), also perform a likelihood ratio test (also one-sided) and interpret that. Compare with the Wald test. Does it look like these may be asymptotically equivalent in this situation?

Hint: Parts (a) and (b) are just like the examples using Cauchy models in the notes on maximum likelihood, changing what needs to be changed. Obviously this problem is about Poisson and negative binomial rather than Cauchy (so if you use R function dcauchy you are making a serious mistake) and the parameters of the Cauchy distribution have nothing whatsoever to do with the parameters of the Poisson and negative binomial distributions. But if you can correctly code an mlogl function (minus log likelihood) and find good starting values for the parameter or parameters, then the rest of the computations in the cited notes are the same for the notes and this problem.

Hint: In part (d) one has to be careful to define the log likelihood the same way for both null and alternative hypotheses. (One can add or subtract terms that do not depend on parametets to the log likelihood, and obviously if one does this differently for the two terms in the likelihood ratio test statistic, then this makes for arbitrary results.) In aid of this, note that R function dnbinom gives results for the Poisson distribution when size = Inf if (a big if) the other parameter is mu

dpois(0:4, lambda = pi)

## [1] 0.04321392 0.13576053 0.21325214 0.22331712 0.17539285

dnbinom(0:4, mu = pi, size = Inf)

## [1] 0.04321392 0.13576053 0.21325214 0.22331712 0.17539285

2-5

Suppose we observe Poisson data $x = 13$ and want to do a Bayesian analysis with conjugate priors, and we choose a prior distribution with $\alpha = 1$ and $\lambda = 1 / 20$.

Make a plot of the prior and posterior probability density functions.
For this posterior distribution, find the posterior mean, median, and mode (three different Bayesian point estimates). The formula for the mean of the gamma distribution is given in the notes. There is no formula for the median but R knows how to calculate it similar to how we calculated the posterior median for the binomial distribution. To derive the formula for the mode of the gamma distribution you need to know calculus, so we will just tell you it is $(\alpha - 1) / \lambda$ if this is greater than zero and zero otherwise.
Suppose we now observe new data $x = 19$ from the same Poisson distribution. Use the Bayesian-learning-paradigm to give a plot for this application of Bayes’ rule (prior is the posterior from part (a) and the posterior incorporates this new data point).

2-6

Suppose we are choosing a conjugate prior for Bayesian inference for Poisson data (this will say the mean of the Poisson distribution has a gamma distribution), and suppose we want this prior distribution to have

mean 10 and
standard deviation 20.

What gamma distribution has this mean and standard deviation? That is, what are the corresponding $\alpha$ and $\lambda$?

Stat 5421 Homework 2

Charles J. Geyer

Instructions

Assignment

2-1

2-2

2-3

2-4

2-5

2-6