General Instructions

To do each example, just click the Submit button. You do not have to type in any R instructions or specify a dataset. That's already done for you.

The Sign Test

Example 3.5 in Hollander and Wolfe

Summary

Comments

The Associated Point Estimate (Median)

Example 3.6 in Hollander and Wolfe

Summary

The Associated Confidence Interval

Example 3.6 in Hollander and Wolfe

Summary

Comments

The Zero Fudge

Many authorities recommend (at least lukewarmly) the following procedure for dealing with zero differences (differences equal to the hypothesized value μ if not zero) in the sign test. After defining the vector z of differences, do

z <- z[z != mu]
n <- length(z)

which treates zero differences as if they were not part of the data (and the sample size is reduced accordingly).

The best that can be said for this is

The alternative to the zero fudge, what Hollander and Wolfe call the conservative approach is to count the zero differences as evidence in favor of the null hypothesis. This is a bit tricky, since no matter how you do it, the recipe for the test must be altered.

For the upper tailed test, assuming the vector of differences z, the sample size n, and the hypothesisized value of the median mu have already been defined,

b <- sum(z > mu)
pbinom(b - 1, n, 1 / 2, lower.tail=FALSE)

calculates the P-value for the upper-tailed test. (This is the same code we used for the upper-tailed test when there were no ties.) Using the strict inequality (>) to excludes the zero differences from the tail area calculated by these statements.

Reversing the inquality gives the conservative lower-tailed test.

blow <- sum(z < mu)
pbinom(blow - 1, n, 1 / 2, lower.tail=FALSE)

Note that this isn't the test statistic described in the book, but obviously does the right thing by symmetry.

Fuzzy Procedures

Fuzzy P-Values

A recent paper by your humble instructor and another member of this department resurrected an old idea, randomized tests, and gave it a new spin as fuzzy tests, taking some terminology from fuzzy set theory.

In a simple situation where there are no ties, a fuzzy P-value for for a sign test (or other rank tests we will meet in a few weeks) can be thought of as a P-value smeared out over an interval. If t is the observed value of the test statistic and T is a random variable having the distribution of the test statistic assuming the null hypothesis is true, then

Note that the fuzzy P-value gives more information. The upper end point of the fuzzy P-value interval is the conventional P-value. Hence fans of conventional P-values cannot object to fuzzy P-values. The fuzzy P-value tells you more than a conventional P-value, but it does not tell you less.

The interpretation of a fuzzy P-value is just like the interpretation of a conventional P-value.

The only difference is that what is now low, high, or intermediate is a range of values. So long as the range isn't too wide, this makes little difference to the interpretation. Anyone who thinks there is an important difference between P = 0.051 and P = 0.049 understands neither science nor statistics. A fuzzy P-value smeared out over the interval (0.049, 0.051) isn't different in any practical sense.

Example 3.5 in Hollander and Wolfe Redone

Summary

Comments

Theoretically, a fuzzy P-value is a random variable whose randomness comes not from the sampling process that generated the data but is artificial, introduced by the theoretical statistician. We can say here that the fuzzy P-value is a random variable uniformly distributed on the interval (0.000078, 0.000455), which is what the summary says.

If there are ties, then the fuzzy.sign.test automatically does the right thing (or at least a right thing). Then the probability distribution of the fuzzy P-value becomes non-uniform. Details are given in the paper cited above and also on the fuzzy P-values and confidence intervals page.

An Example With Ties

Summary

Fuzzy Decisions

If one likes the decision theoretic view of hypothesis testing (pick an alpha level, say 0.05, and accept or reject the null hypothesis at that level, ignoring all other levels), then the fuzzy analog is to report the probability that the fuzzy test rejects (which is the same as the probability that the randomized test rejects), which is the probability that the fuzzy P-value is less than alpha.

An Example With Ties Redone Decisively

Summary

Not much difference between rejecting the null hypothesis (that the true population median is zero) with probability 0.94 and with probability 1.00. In either case, moderately strong, but not extremely strong, evidence against the null hypothesis.

Fuzzy Confidence Intervals

Example 3.5 in Hollander and Wolfe Redone with Confidence

Comments

The fuzzy confidence interval is a function that gives a number between zero and one for each parameter value. The values for which it is one (the core of the interval) are the ones for which it gets full credit if the true unknown parameter value is among them. The values for which it is nonzero (the support of the interval) are the ones for which it gets partial credit if the true unknown parameter value is among them. The reported confidence level (here 95%) is the expected amount of credit it gets (full or partial) averaged over samples from the population.

In most cases (and here) fuzzy and conventional confidence intervals are not so different. The core of the fuzzy confidence interval, which is (7.5, 23.8) in the example, counts in 9 from each end, which would be an 89.22% confidence interval by itself, what

n <- length(x)
k <- 9
1 - 2 * pbinom(k - 1, n, 1 / 2)

would calculate.

The core of the fuzzy confidence interval, which is (7.1, 24.7) in the example, counts in 8 from each end, which would be a 95.67% confidence interval by itself, what the above code would calculate if we set k <- 8 instead. The partial credit is carefully arranged to give exactly 95% coverage regardless of what the true parameter value may be.

More details are given in the paper cited above and also on the fuzzy P-values and confidence intervals page.