Probability

See disclaimer about the slides.
See learning objectives and sources.

Probability (definition)

$S$ is a set called space whose elements are called points.
$P : S \to [0,1]$ is called a probability distribution iff $\sum_{x \in S} P(x) = 1$.
A subset $E$ of $S$ is called an event.
$P(E)=\sum_{x \in E} P(x)$.

Probability (example)

$S = \{1,2,3,4,5,6\}$.
$P(x)=1/6$, for all $x \in S$.
$E=\{1,2\}$.
Compute during the lecture that $P(E)=1/3$.

Probability (example)

$S$ is the set of outcomes of throwing two fair dice.
$E$ is ''sum of (the top faces of) an outcome is 7''.
Compute during the lecture that $P(E)=1/6$.

Probability (example)

Compute the probability of getting a four-of-a-kind in a poker hand.
Hint: How many poker hands? Home many score a four-of-a-kind?
There are $52 \cdot 51 \cdot 50 \cdot 49 \cdot 48 = 311,875,200$ hands (counting order).
The are $13 \cdot 4! \cdot 48 \cdot 5 = 74,880$ four-of-a-kind (counting order).
13 ranks, 4! permutations, 48 other card, 5 positions for other card.
The probability is about $2.4~10^{-4}$.

If you discount the order, there are ${52 \choose 5} = 2,598,960$ choices of 5 cards.
There are $13 \choose 1$ choices of rank of the four-of-a-kind, and a forced choice (or $4 \choose 4$) of cards.
There are $48 \choose 1$ choices of the 5-th card.
So, the total number of four-of-a-kind is $13 \times 48 = 624$.

Properties of probability distributions

$\forall E \subseteq S ~(0 \leq P(E) \leq 1)$.
$P(\varnothing)=0$, $P(S)=1$.
$P(S-E)=1-P(E)$.
$P(E_1 \cup E_2) = P(E_1) + P(E_2) - P(E_1 \cap E_2)$.

Expected value (definition)

$S$ is a space.
$V : S \to {\mathbb R}$ is a function (value).
$P : S \to [0,1]$ is a probability distribution.
$\sum_{x \in S} V(x)P(x)$ is the expected value.

Expected value (examples)

In American Roulette, there are two zeroes and 36 non-zero numbers (18 red and 18 black). If a player bets \$1 on red, her chance of winning \$1 is therefore 18/38 and his chance of losing \$1 (or winning -\$1) is 20/38 (this is the value).
The player's expected value is $(18/38 \times 1) + (20/38 \times -1) = 18/38 - 20/38 = -2/38 = -5.26\%$. reference

Play game in class. Throw 2 dice: win 100 for 7, lose 100 for 2 and 12, lose 10 for rest.
Is it a fair game? Compute the expected value during the lecture.

Conditional Probability (definition)

Probability of event $E$ given $F$: $P(E \mid F)=\displaystyle \frac {P(E \cap F)}{P(F)}$.

Conditional Probability (example)

80% of students in a class pass the first midterm and 60% pass both first and second midterm.
What is the probability to pass the second midterm if a student passes the first one.
$P(S | F)= P(S \cap F) / P(F) = 60/80 = 75%$.

Conditional Probability (example)

Toss two fair dice.
What is the probability of getting a double (top faces same value)?
What is the probability of getting a double, if the sum is 4?
Compute during the lecture these probabilities.

Independent Events

$P(E \cap F)=P(E) \cdot P(F)$.
Ex.: $E =\,$ a card rank is Ace, $F =\,$ a card suite is Club.
Compute during the lecture whether $E$ and $F$ are independent.

Bayes' Theorem

Simple case: two events, $E$ and $F$.
$P(E|F) P(F) = P(F|E) P(E)$, because both are $P(E \cap F)$.

More complex case: consider both $F$ and $\bar F$ (the complement of $F$).
Observe $E = E \cap (F \cup \bar F) = (E \cap F) \cup (E \cap \bar F)$.
Thus, $P(E) = P(E \cap F) + P(E \cap \bar F)$ because disjoint.
Finally, $P(F|E) = \displaystyle \frac {P(E|F) P(F)} {P(E|F) P(F) + P(E|\bar F) P(\bar F)}$

Generalizes to partitions with more than 2 elements.

Example

A drug test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people are users of the drug. If a randomly selected individual tests positive, what is the probability that s/he is NOT a user? reference

bayes_drug_6.png

Tree diagram illustrating drug testing example. $\mathrm U$, $\overline {\mathrm U}$, "+" and "−" are the events representing user, non-user, positive result and negative result. Percentages in parentheses are calculated.

${\begin{aligned}P({\overline {\text {U}}}\mid {\text{+}})&={\frac {P({\text{+}}\mid {\overline {\text {U}}})P({\overline {\text {U}}})}{P({\text{+}}\mid {\overline {\text {U}}})P({\overline {\text {U}}})+P({\text{+}}\mid {\text{U}})P({\text{U}})}} \\[8pt]&={\frac {0.01\times 0.995}{0.01\times 0.995+0.99\times 0.005}} \\[8pt]&\approx 66.8\%\end{aligned}}$

Bertrand paradox

There are three boxes, each with one drawer on each of two sides. Each drawer contains a coin. One box has a gold coin on each side (GG), one a silver coin on each side (SS), and the other a gold coin on one side and a silver coin on the other (GS). A box is chosen at random, a random drawer is opened, and a gold coin is found inside it. What is the chance of the coin on the other side being gold? reference

Explanation

First some reasoning: Let $F$ be the event of finding a gold coin and let $GG$ the probability of choosing the box with two gold coins. If you choose at random box and drawer, $P(F)=1/2$ (because there are 6 drawers and 3 contain a gold coin) and $P(GG)=1/3$ (because there are 3 boxes and 1 is $GG$).

The space of choosing box and drawer is partitioned into $\{\{GG\},\{GS\},\{SS\}\}$. We want to compute $P(GG \mid F)$, the (conditional) probability that the choice was $GG$ given $F$, the fact we found a gold coin. This is the probability that the other drawer of the box we chose contains a gold coin.

Bayes' Theorem applied to this problem says $P(GG \mid F)=P(F \mid GG) \cdot P(GG) / P(F)$. Observe that $P(F \mid GG) = 1$ because if we choose $GG$ we certainly find a gold coin regardless of the drawer. Thus, $P(GG \mid F)=P(GG) / P(F)=(1/3)/(1/2)=2/3$.