# Statistics 3011 (Geyer and Jones, Spring 2006) Probability Rules

This page is a short summary of the probability rules on pages 230–231 and on page 283 in the textbook. We also add two more rules.

See page 228 in the textbook for explanation of sample space, outcome, and event.

In our numbering, we follow the textbook.

Rule 1. All probabilities are between zero and one. For any event A, we have 0 ≤ P(A) ≤ 1.

Rule 2. The probability of the entire sample space is one. If S is the sample space (which is a set of outcomes, hence an event), then we have P(S) = 1.

Rule 2-extra. The probability of the empty event is zero. If ∅ is the empty set (which is a set of outcomes, hence an event), then we have P(∅) = 0.

Comment about Rules 2 and 2-extra. Probability 1 represents certainty. An event having probability one always occurs. Probability 0 represents impossibility. An event having probability zero never occurs. Events other than the sample space can have probability one in some models. Events other than the empty set can have probability zero in some models.

Preliminary Definition for Rule 3. For any event A, the set of outcomes not in A is called the complement of A and denoted not A.

Rule 3 (Complement Rule). For any event A, we have P(not A) = 1 − P(A).

Preliminary Definition for Rule 4. For any events A and B, the set of outcomes in either A or B or both is called the union of A and B and denoted A or B.

Another Preliminary Definition for Rule 4. Events A and B are called disjoint if they have no outcomes in common.

Rule 4 (Addition Rule). If A and B are disjoint events, then we have we have P(A or B) = P(A) + P(B).

Rule 4-extra (Subadditivity Rule). For any events A and B, we have P(A or B) ≤ P(A) + P(B).

Comment about Rules 4 and 4-extra. The addition rule holds for disjoint events. If the events are not disjoint, then you cannot apply Rule 4. Whether or not the events are disjoint, you can apply Rule 4-extra.

Rule 4-induction (Additivity Rule for Multiple Events). If A1, A2, …, and Ak are disjoint events, then we have

P(A1 or A2 or … or Ak) = P(A1) + P(A2) + … + P(Ak)

Rule 4-variant (Subadditivity Rule for Multiple Events). For any events A1, A2, …, and Ak we have

P(A1 or A2 or … or Ak) ≤ P(A1) + P(A2) + … + P(Ak)

Preliminary Definition for Rule 5. For any events A and B, the set of outcomes in both A and B is called the intersection of A and B and denoted A and B.

Rule 5 (Multiplication Rule). Events A and B are independent if and only if P(A and B) = P(A) P(B).

Comment about Rule 5. Mathematically, Rule 5 is a definition. If the events are independent, then the multiplication rule holds. If the multiplication rule holds, then the events are independent. Mathematically, independent is just short for the multiplication rule holds.

In applications, however, we apply the independence notion to events that involve random processes that have no effect on each other.

• outcomes of a sequence of coin flips
• outcomes of a sequence of (substitute your favorite randomization device here — dice rolls, well-shuffled deck of cards, slot machine, lottery machine).
• outcomes of a sequence of free throws by one basketball player.

The last is interesting because many people, including players and coaches, do not believe the sequence is independent but available evidence supports or at least does not disprove independence.

Rule 5-extra (Multiplication Rule for Multiple Events). A family of events is independent if and only if for every A1, A2, …, Ak distinct events chosen from the family we have

P(A1 and A2 and … and Ak) = P(A1) × P(A2) × … × P(Ak)

Moreover, if all k events have the same probability, this reduces to

P(A1 and A2 and … and Ak) = P(A1)k

Rule 5-random-variables (Multiplication Rule for Random Variables). A sequence of random variables X1, X2, … is independent if and only if for every k and every A1, A2, …, Ak we have

P(X1 in A1 and X2 in A2 and … and Xk in Ak) = P(X1 in A1) × P(X2 in A2) × … × P(Xk in Ak)