The course slides (Deck 2,
Slide 93
and
Slides 112–121)
derives three different asymptotic
confidence intervals for binomial data
and mentions an exact

one (that involves heavy
computing and is not derived). This web page examines the exact performance
of all four (plots coverage as a function of the parameter value).

As discussed in the course slides (Deck 2,
Slide 92)
the coverage probability considered as a function of the parameter can never
be a constant function when the data have a discrete distribution. So even
the exact

confidence interval does not achieve exactly the nominal
coverage for all parameter values but only *at least* the nominal
coverage, so is perhaps better termed conservative-exact

.

## Usual Binomial Confidence Intervals

The usual

confidence interval is the one shown on
Slide 113, Deck 2
of the course slides. It is the one taught in all intro stats courses.

The following R code plots the coverage as a function of the parameter. The solid line is the graph of this function (the vertical sections are not, strictly speaking, part of the graph, but it is easiest to plot it this way). The dashed horizontal line is the nominal coverage level.

## Score Binomial Confidence Intervals

The score

confidence interval is the one shown on
Slide 116, Deck 2
of the course slides. It is the computed by the R function
`prop.test`

(on-line
help).
This one is beginning to be taught in some intro stats courses.

The following R code plots the coverage as a function of the parameter. The solid line is the graph of this function (the vertical sections are not, strictly speaking, part of the graph, but it is easiest to plot it this way). The dashed horizontal line is the nominal coverage level.

## Variance Stabilized Binomial Confidence Intervals

The confidence interval using the variance stabilizing transformation is the one shown on Slide 120, Deck 2 of the course slides.

The following R code plots the coverage as a function of the parameter. The solid line is the graph of this function (the vertical sections are not, strictly speaking, part of the graph, but it is easiest to plot it this way). The dashed horizontal line is the nominal coverage level.

## Clopper-Pearson Binomial Confidence Intervals

The Clopper-Pearson confidence interval is not derived on the course
slides. It is the computed by the R function
`binom.test`

(on-line
help).

## Modified Usual Binomial Confidence Intervals

Geyer (2009,
*Electronic
Journal of Statistics* **3**, 259 289)
proposed
a simple modification that fixes the behavior of binomial confidence intervals
near zero and one (and also applies to other distributions). For the binomial
distribution it says that when we observe zero successes, the confidence
interval should be from zero
to 1 − α^{1 ⁄ n},
where coverage 1 − α is wanted and `n` is the sample
size.
And it says that when we observe all successes
(`n` out of `n`),
the confidence interval should be from
α^{1 ⁄ n} to one.

In this section we use this modification when zero or `n` successes
are observed and use the usual

confidence interval (section
Usual Binomial Confidence Intervals above)
for other data.

## Modified Variance Stabilized Binomial Confidence Intervals

We use the same modification used in the preceding section but now apply it to the Variance Stabilized Binomial Confidence Intervals described above.

## Likelihood-Based Confidence Intervals

This section is way out of order. This web page was originally designed
to go with deck 2 of the course notes, and did not include this section
(which is not covered in deck 2). In fact, the procedure illustrated
in this section is not covered at all in this course, although it almost

is.

This section covers confidence intervals obtained by inverting the likelihood
ratio test (deck 6, slides 32–39 of the course slides). If the
likelihood ratio test compares models differing in dimension by one (so one
has one more parameter than the other), the confidence interval obtained by
inverting the test is what is illustrated here. These intervals are done
by the R function `confint`

, or at least it does them when applied
to generalized linear models (since this is a generic function authors of R
packages can make it do what they please for models fit by their packages).

Unfortunately, the R function `confint.glm`

is broken when the
data are `x` = 0 or `x` = `n`. So we have to
write our own code. Annoying.

## Summary

Of the three confidence intervals with simple definitions, only the
score intervals behave well for all values of the
unknown parameter. But the usual

intervals
and the variance-stabilized intervals can
be modified by patching up what they do when `x = 0` or
`x = n` is observed, as described in the sections
Modified Usual Binomial Confidence Intervals
and
Modified Variance Stabilized Binomial
Confidence Intervals above.

The Clopper-Pearson Binomial Confidence Intervals, of course, also have good performance, because they are defined to be that way (guaranteed conservative).

Similar sorts of plots and similar sorts of modifications can be cooked up for other confidence intervals, but we do not belabor the subject and leave it here.