Statistics 8701 (Geyer) Spring 2002 Homework No. 3

The policy is the same as the policy for Homework No. 2.

This assignment is due (initially) Friday, February 22, 2002.

Problem 1

Problem 3.1 in Robert and Casella.

For part (a) you can use either the strong law of large numbers (almost sure convergence) or the weak law of large numbers (convergence in probability), whichever you feel more comfortable with.

For part (b) you need the delta method, contrary to the dismissal of the problem as trivial in class. Or (as Bernardo pointed out to me) you can use the other ratio estimation method now clearly explained in the new statement of Problem 5.

Problem 2

Problem 3.8 in Robert and Casella.

It's not clear what Robert and Casella are asking here since no functional of interest, no h(x), is specified. The performance of importance sampling depends on the functional of interest. Thus the question as stated cannot be answered.

For the purposes of doing the question, consider the variance of the importance weights f(x) / g(x) themselves. These would be of interest if we were trying to estimate the number 1 using the expectation

E_f(1) = 1

Since we actually know the value of the number 1, there is little point to this exercise. It might be argued that the variance of the importance weights is important if we know that we will be calculating a probability, that is, h(x) is the indicator function of some event, but the particular event is not specified.

Another problem with this problem is that, according to Mathematica, the so-called density f is not, because it doesn't integrate to one. However, that doesn't matter if we are only looking at the variance of the importance weights. It would mean we would have to use normalized importance weights (as described in problem 5 below) if we were actually to do some computing with this problem.

Hmmmmmmmm. Some attempts to do the problem myself lead to the following comments.

Mathematica can't do these integrals numerically.
Some of these importance sampling distributions do a really bad job of estimating
E_{g_i}{ f(x) / g_i(x) }
which is the integral of f, which Mathematica says is 0.938515.
If they do a really bad job of estimating the mean, then they will do an even worse job of estimating the variance. Thus brute force and ignorance may not work here. Hint: importance sampling.

Triple Hmmmmmmmm. Some fiddling in Mathematica convinces me that some of these variances do not exist. Thus the problem is simpler than it appears at first. The variances that do not exist do not need to be estimated by Monte Carlo.

Problem 3

Problem 3.11 in Robert and Casella.

For part (a) the bounded means bounded above (likelihoods are rarely bounded below), that is that

sup_θ l(θ | x)

is finite.

For part (b) the hint should really read consider conjugate priors, that is, priors proportional to the likelihood.

Problem 4

Problem 3.13 in Robert and Casella.

Problem 5

This one is too complicated to explain in HTML. It is available either as a DVI file or as a PDF file.