====== - #2 Probability of two events ====== ===== - Mutually exclusive events ===== If the probability distribution of an experiment/process is given, finding the probability of __any__ event is really simple due to **the law of mutually exclusive events**. **The law of mutually exclusive events.** If the events $A$ and $B$ are __mutually exclusive__, then the probability that happens either $A$ or $B$ (denoted: $\text{Pr}[A\dot{\cup} B]$) is equal to the sum of $\text{Pr}[A]$ and $\text{Pr}[B]$, i.e. $$\text{Pr}[A\dot{\cup} B]=\text{Pr}[A]+\text{Pr}[B].$$ Events $A$ and $B$ are called **mutually exclusive**, if their simultaneous occurrence is impossible (these events exclude each other). It can be denoted also $A\cap B=\emptyset$, i.e. $A$ and $B$ do not intersect as sets. Recall that an event is defined as a subset, thus, the usage of set-theoretic notation is quite natural. ^ Notation ^ Graphical meaning ^ Intuitive meaning ^ | $A\dot{\cup} B$ | {{ :probability:venni_1.png?250 |}} | Either the event $A$ happens or the event $B$ happens, and the events $A$ and $B$ exclude each other. | | $A{\cup} B$ | {{ :probability:venni_2.png?200 |}} | Either the event $A$ happens or the event $B$ happens, and the events $A$ and $B$ do not exclude each other.| | $A{\cap} B$ | {{ :probability:venni_3.png?200 |}} | Both the events $A$ and $B$ happen simultaneously. | For example, the event $C:=\text{"we roll a even number"}$ means $\text{"we roll a two"}$, or $\text{"we roll a four"}$, or $\text{"we roll a six"}$, i.e. $C=\{2\}\dot{\cup} \{4\} \dot{\cup} \{6\}=\{2, 4, 6\}$. Or, $\text{"we roll a six"}=\text{"the result in multiple of three"}\cap\text{"the result in multiple of two"}$. Represent the event $\text{"we roll a five"}$ by two events. ++Answer| E.g. $\text{"we roll a five"}=\text{"the result is a prime number"}\cap\text{"the result is greater then 4"}$.++ Let us have an algorithm, where $x$ is evaluated by rolling a dice:\\ $x\leftarrow\{1,2,3,4,5,6\}$\\ if $2$ divides $x$ return $0$\\ if $3$ divides $x$ return $0$\\ return $1$\\ and let $A=\text{"2 divides}\ x \text{"}$ and $B=\text{"3 divides}\ x \text{"}$. Are $A$ and $B$ mutually exclusive? If not, then what is $A\cap B$. As we know, all the elementary events of an experiment/process are mutually exclusive events. And since every event consist of elementary events, calculating probability, according to the law of mutually exclusive events, of any event means just summing probabilities of all the elementary events of which the event consists of. For example, for calculating $\text{Pr}[\text{"we roll an even number"}]$ we can just sum $\text{Pr}[\text{"we roll a two"}]$, $\text{Pr}[\text{"we roll a four"}]$, and $\text{Pr}[\text{"we roll a six"}]$, i.e. $$\text{Pr}[\text{"we roll an even number"}]=\frac{1}{6}+\frac{1}{6}+\frac{1}{6}=\frac{1}{2}$$ for a normal six-faced dice. For the dice with two fives, $$\text{Pr}[\text{"we roll an even number"}]=\text{Pr}[\text{"we roll a two"}]+\text{Pr}[\text{"we roll a four"}]=\frac{1}{6}+\frac{1}{6}=\frac{1}{3}.$$ Another natural example of mutually exclusive events is an event $A$ and its complement $A^c$. That is, the occurrence of the event $A^c$ means that the event $A$ does not occur. Mostly $A^c$ is state just as negation of the event $A$, but not always. Some examples: if $A:=\text{"tomorrow it rains"}$, then $A^c:= \text{"tomorrow it does not rain"}$. Or, if $A:=\text{"we roll a result smaller then 3"}$ for a dice roll, then $A^c=\text{"we roll a result greater then or equal to 3"}$, i.e. $A^c=\text{"we roll a result 3, 4, 5, or 6"}$. Clearly we can also say that $A^c:=\text{"we do not roll a result smaller then 3"}$, however, the first and second formulations are more informative. Note, that $A\dot{\cup} A^c=\Omega$. Hence, $$1=\text{Pr}[\Omega]=\text{Pr}[A\dot{\cup} A^c]=\text{Pr}[A]+\text{Pr}[A^c],$$ providing us with formula $$\text{Pr}[A]=1-\text{Pr}[A^c].$$ Which of the following events are mutually exclusive, which are not, and for which is it unknown? (Here $x,y$ are the results of two fair dice rolls.) - Tomorrow's temperature is below zero. - $x>4$. - $y>6$. - $x<4$. - Tomorrow, the sun is shining. - Tomorrow's temperature is above the melting point of water. The wheel of fortune {{:probability:wheel_fortune.png?100|}} gives you a free ticket to cinema if it stops on red or on orange. What is the probability that you win the ticket? ++Answer| $\text{Pr}[\text{"you win the ticket"}]=\text{Pr}[\text{"the wheel stops on red"}]+ \text{Pr}[\text{"the wheel stops on orange"}]=\frac{2}{8}+\frac{1}{8}=\frac{3}{8}$++ Let us have an algorithm, where $x$ is evaluated by rolling a dice:\\ $x\leftarrow\{1,2,3,4,5,6\}$\\ if $x=3$ then $y=x+3$\\ if $x\geq 5$ then $y=8-x$.\\ Calculate $\text{Pr}[y=6]$. ++Answer| $\text{Pr}[y=6]=\text{Pr}[x=3]+\text{Pr}[x=5]=\frac{1}{6}+\frac{1}{6}= \frac{1}{3}$++ ===== - Mutually non exclusive events ===== If $A$ and $B$ are NOT mutually exclusive, then $$\text{Pr}[A\cup B]=\text{Pr}[A]+\text{Pr}[B]-\text{Pr}[A\cap B].$$ For example, viewing again the dice, if $A=\text{"the result is a prime"}$ and $B=\text{"the result is multiple of 3"}$ we have {{ :probability:probability_dice_union.png |}} Clearly $\text{Pr}[A]=\text{Pr}[\text{"we roll 1"}]+\text{Pr}[\text{"we roll 2"}]+ \text{Pr}[\color{red}{\text{"we roll 3"}}]+\text{Pr}[\text{"we roll 5"}]=\frac{1}{6}+\frac{1}{6} +\frac{1}{6}+\frac{1}{6}= \frac{4}{6}$\\ and\\ $\text{Pr}[B]=\text{Pr}[\color{blue}{\text{"we roll 3"}}]+\text{Pr}[\text{"we roll 6"}]= \frac{1}{6}+\frac{1}{6}= \frac{2}{6}$. If we add $\text{Pr}[A]+\text{Pr}[B]=$\\ $\text{Pr}[\text{"we roll 1"}]+\text{Pr}[\text{"we roll 2"}]+ \text{Pr}[\color{red}{\text{"we roll 3"}}]+\text{Pr}[\text{"we roll 5"}]+ \text{Pr}[\color{blue}{\text{"we roll 3"}}]+\text{Pr}[\text{"we roll 6"}]=$\\ $\frac{4}{6}+\frac{2}{6}=1$, we take $3$ into account two times, however, $A\cup B=\{1, 2, 3, 5, 6\}$ contains only one $3$. Thus, $\text{Pr}[A\cup B]=\text{Pr}[A]+\text{Pr}[B]-\text{Pr}[\text{"we roll 3"}]=\text{Pr}[A]+\text{Pr}[B]- \text{Pr}[A\cap B]=1-\frac{1}{6}=\frac{5}{6}$. What is the probability of drawing a red card or a six in a deck of cards? ++Answer| We have to calculated $\text{Pr}[\text{"we draw a red card"}\cup\text{"we draw a five"}]$.\\ There are $4$ fives in a deck of cards, thus $\text{Pr}[\text{"we draw a five"}]=\frac{4}{52}$ and\\ there are $26$ read cards in a deck, so $\text{Pr}[\text{"we draw a red card"}]=\frac{26}{52}$.\\ Two fives out four are red cards: $\text{Pr}[\text{"we draw a red five"}]=\frac{2}{52}$.\\ Hence, $\text{Pr}[\text{"we draw a red card"}\cup\text{"we draw a five"}]= \frac{26}{52}+\frac{4}{52}-\frac{2}{52}=\frac{28}{52}=\frac{7}{13}$. ++ On Midsummer's Day, a probability of a person having a car accident is $0,08$. The probability of a person driving intoxicated is $0,35$ and probability that a person has a car accident while intoxicated is $0,02$. What is the probability of a person driving while intoxicated or having a car accident? ++Answer| $\text{Pr}[\text{"a person driving while intoxicated or having a car accident"}]=0,35+0,08-0,02=0,41$++ Let us have an algorithm, where $x$ is evaluated by picking randomly a number from $1$ to $11$:\\ $x\leftarrow\{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11\}$\\ if $2$ divides $x$ return $0$\\ if $5$ divides $x$ return $0$\\ return $1$.\\ What is the probability that the algorithm returns $0$? What is the probability that the algorithm returns $1$? ++Answer| $\text{Pr}[\text{"the algorithm returns 0"}]=\text{Pr}[\text{"2 divides}\ x\text{"}]+ \text{Pr}[\text{"5 divides}\ x\text{"}]-\text{Pr}[\text{"2 and 5 divide}\ x\text{"}]=$\\ $\text{Pr}[\text{"}x\ \text{is 2, 4, 6, 8, or 10 "}]+ \text{Pr}[\text{"}x\ \text{is 5, or 10"}]-\text{Pr}[\text{"}x\ \text{is 10"}]= \frac{5}{11}+\frac{2}{11}-\frac{1}{11}=\frac{6}{11}$++ Two events $A$ and $B$ have probabilities given below: $\text{Pr}[A]=\frac{1}{3} \qquad \text{Pr}[B]=\frac{1}{2}\qquad \text{Pr}[A\cup B]=\frac{5}{6}$ Are events $A$ and $B$ mutually exclusive or not? ++Answer| From the formula in the beginning of this section, we know that $\text{Pr}[A\cap B]= \text{Pr}[A]+\text{Pr}[B]-\text{Pr}[A\cup B]$. Thus, $\text{Pr}[A\cap B]=\frac{1}{3}+\frac{1}{2}- \frac{5}{6}=0$, i.e. the events $A$ and $B$ are mutually exclusive.++ In general, if we do not know anything about the events $A$ and $B$. It is always correct to estimate $$\text{Pr}[A\cup B]\leq\text{Pr}[A]+\text{Pr}[B].$$ ===== - Independent events ===== Now, let us roll the dice two times in the row. First roll: {{:probability:dice_1.gif|}} Second roll: {{:probability:dice_3.gif|}} The event consist of $2$ events: $A:=\text{"the result of the first roll"}$ and $B:=\text{"the result of the second roll"}$. Moreover, the events $A$ and $B$ can be viewed as results of two different random processes. Note, that we can obtain by first roll a one and also by the second roll a one, actually, the result of the first roll does not influence the result of the second roll - these events are **independent**. So, how to calculated the probability now? For answering the question, let us list up all the possible results: {{:probability:two_dices_results.jpg?350|}} Thus, $\text{Pr}[\text{"the first result is a one and the second result is a three"}]=\frac{1}{36}$, but also $\text{Pr}[\text{"the first result is a five and the second result is a four"}]=\frac{1}{36}$, and so on. We can note a rule $$\text{Pr}[\text{"the first result is x and the second result is y"}]= \text{Pr}[\text{"the first result is x"}]\cdot\text{Pr}[\text{"the second result is y"}].$$ Actually, it is a general rule. Two events $A$ and $B$ are **independent** if and only if $$\text{Pr}[A\cap B]=\text{Pr}[A]\cdot\text{Pr}[B].$$ Note, that in case of an event which consists of results of two different random processes (e.g. $\text{"user logs into some computer system"}\cap\text{"the system has a failure"}$), there is no set-theoretic intersection between the events. However, $A\cap B$ means that the events $A$ and $B$ happen simultaneously (as one event), and this is precisely the case. Thus, by tossing a coin two times in a row, the probability $\text{Pr}[\text{"we obtain two tails in a row"}]=$ $\text{Pr}[\text{"1st throw gives tails"}\cap\text{"2nd throw gives tails"}]=$ $\text{Pr}[\text{"1st throw gives tails"}]\cdot\text{Pr}[\text{"2nd throw gives tails"}]= \frac{1}{2}\cdot\frac{1}{2}= \frac{1}{4}.$ Or, by tossing a coin and rolling a six-faced normal dice, the probability $\text{Pr}[\text{"we land on heads"}\cap\text{"we roll a six"}]=$ $\text{Pr}[\text{"we land on heads"}]\cdot\text{Pr}[\text{"we roll a six"}]=\frac{1}{2}\cdot\frac{1}{6}= \frac{1}{12}.$ It can be very useful to represent such events as tree diagram. For example, a card is chosen blindly from a pack of $52$ playing cards, and then replaced. Then also a second card is drawn at random. What is the probability that both of the cards are Diamonds? Or, at least one card is a Diamond? Or, exactly one card is a Diamond? First note, that for a single card\\ $\text{Pr}[\text{"we obtain a Diamond"}]=\frac{13}{52}=\frac{1}{4}$ and\\ $\text{Pr}[\text{"we do not obtain a Diamond"}]=1-\text{Pr}[\text{"we obtain a Diamond"}]=1-\frac{1}{4} =\frac{3}{4}$. Let us put these probabilities on the branches of the tree diagram: {{:probability:tree_diagram.png?600|}} The probabilities are calculated by multiplying along the branches. Clearly,\\ $\text{Pr}[\text{"both cards are Diamonds"}]=\frac{1}{16}$,\\ $\text{Pr}[\text{"at least one card is a Diamond"}]=\frac{1}{16}+\frac{3}{16}+\frac{3}{16}=\frac{7}{16}$,\\ $\text{Pr}[\text{"exactly one card is a Diamond"}]=\frac{3}{16}+\frac{3}{16}=\frac{3}{8}$. A useful property of independent events is the following. If $A$ and $B$ are independent, then also $A$ and $B^c$ are independent. Thus, showing that $A$ and $B$ are independent can be replaced by showing that $A$ and $B^c$ are independent or $A^c$ and $B$ are independent ($(B^c)^c=B$ and $(A^c)^c=A$), if calculation of $B^c$ or $A^c$ is more easy. A natural observation concerning independent events is: if $A$ holds always (or holds never), then $A$ and $B$ are independent. Which of the following events are independent, which are not? Explain your answer. (Here $x,y$ are the results of two fair dice rolls.) - Tomorrow's temperature is below zero. - $x>4$. - $y>6$. - $x<4$. - Tomorrow, the sun is shining. - Tomorrow's temperature is above the melting point of water. Let $x,\ y$ be random bits, each chosen using a coin flip, and let $A:=\text{"$x=0$"}$ and $B:=\text{"$x=y$"}$. Are $A$ and $B$ independent? ++Answer| The events $A$ and $B$ are independent because\\ $\text{Pr}[A\cap B]=\text{Pr}[\text{"$x=0$"}\cap \text{"$x=y$"}]= \text{Pr}[\text{"$x=0$"}\cap \text{"$y=0$"}]=\frac{1}{4}=\text{Pr}[A]\cdot\text{Pr}[B]$.++ Let us have the following algorithm: x = 0, y = 0 while (x = 0) x <- {0,1,2} y = y+1 where **x** is evaluated randomly with one of the values $0$, $1$, $2$. Calculate $\text{Pr}[y=3]$. Let us have the following algorithm, where $x$ is evaluated by picking randomly a number from $1$ to $12$:\\ $x\leftarrow\{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12\}$\\ if $2$ divides $x$ return "boy"\\ if $4$ divides $x$ return "girl" Are the events of obtain "boy" and obtaining "girl" independent? What is the answer to the same question in the following case:\\ $x\leftarrow\{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12\}$\\ if $6$ divides $x$ return "boy"\\ if $4$ divides $x$ return "girl" ===== - Additional exercises ===== Assume the following game of chance: two random numbers $x,y$ are picked from $1,\dots,100$. We win if $x\neq y$ and $x,y> 90$. What is the probability of winning? ++Answer| Picking a number $x$ and picking a number $y$ are independent events, thus $\text{Pr}[x> 90,y>90]=0,1\cdot 0,1=0,01$. Similarly, $\text{Pr}[x=y\ \text{and}\ x,y> 90]=0,1\cdot\frac{1}{100}=0,001$ (the probability that $x> 90$ is $0,1$, and with probability $1/100$, $y$ has the same value). Note, that the events $(x\neq y\ \text{and}\ x,y> 90)$ and $(x=y\ \text{and}\ x,y> 90)$ are mutually exclusive. Thus,\\ \\ $0,01=\text{Pr}[x,y>90]= \text{Pr}[x\neq y\ \text{and}\ x,y> 90]+\text{Pr}[x=y\ \text{and}\ x,y> 90]= \text{Pr}[x\neq y\ \text{and}\ x,y> 90]+0,001$,\\ \\ and hence $\text{Pr}[\text{"win"}]=\text{Pr}[x\neq y\ \text{and}\ x,y> 90]=0,01-0,001=0,009$. ++ Assume that you have the following information: with probability $0,1$, my new friend is female and at most $29$ years old. With probability $0,3$, my new friend is female and at least $30$ years old. What $\text{Pr}[\text{"my new friend is female"}]$? ++Answer| Note, that $\text{"my new friend is female"}=$\\ $\text{"my new friend is female and at most $29$ years"}\cup \text{"my new friend is female and at least $30$ years"}$, and these events are mutually exclusive. Thus, $\text{Pr}[\text{"my new friend is female"}]=0,1+0,3=0,4$.++ In a ($6$ out of $49$) lottery, the probability of winning is $\frac{1}{13\,983\,816}$. The probability that $49$ is one of the numbers is $\frac{6}{49}$. You play in the lottery and at the same time roll a die. What is the probability that you win the lottery and roll a six? ++Answer| $\text{"Winning the lottery"=:A}$ and $\text{"rolling a six"=:B}$ are two independent events, thus $\text{Pr}[A\cap B]=\frac{1}{13\,983\,816}\cdot\frac{1}{6}.$++ A company makes computer discs. It tested a random sample of discs from a large batch and found that the probability of any disc being defective is $0,025$. Bob buys two discs. Calculated the probability that * both discs are defective; * that only one disc is defective. The company found $4$ defective discs in the sample they tested. How many discs were likely tested? [[probability:03_conditional_probability]]\\