University of Minnesota, Twin Cities     School of Statistics     Stat 5601     Rweb

Stat 5601 (Geyer) Counterexample for the Sign Test

Suppose the pair differences Zi, have a distribution satisfying

pr(Zi > 0) = 0.01
pr(Zi = 0) = 0.90
pr(Zi < 0) = 0.09
Other details about the distribution do not matter. If N is the number of nonzero pair differences in a sample and B is the number of these nonzero pair differences that are positive, then the conditional distribution of B given N is Bin(N, 0.1).

In contrast, the rule used by the zero fudge assumes a quite different (hence wildly wrong) conditional sampling distribution of B given N, that is, Bin(N, 0.5).

For a concrete example suppose we observe B = 9 and N = 10. Then then the probability that is the P-value for the lower-tailed test

pr(N - B ≤ 1 | N = 10) = 0.7361
is calculated in R by 
> pbinom(1, 10, 0.1)
[1] 0.736099
if we use the actual distribution assumed for this example and by 
> pbinom(1, 10, 0.5)
[1] 0.01074219
under the zero fudge rule.

Quite a difference!