Imagine a box containing two bags. Bag A contains 60% of the balls, while Bag B contains 40% of the balls. In Bag A, 90% of the balls are blue, and 10% are red. In contrast, Bag B has an even distribution of balls, with 50% blue and 50% red. The data distributions can be visualized as below:

The subfigure on the right illustrates the concept of swapping conditional probabilities. In this scenario, focusing on blue balls, we find that 72% of them ( $\frac{54}{54+20} \times 100\% = 72\%$) are located in Bag A, while the remaining 28% are in Bag B. Similarly, for red balls, 23% ($\frac{6}{6+20} \times 100\% = 23\%$) are in Bag A, and the majority, 77%, are in Bag B. These percentages reflect the distribution of blue and red balls across the two bags. This convertion is also known as Bayes theorem.

Probability Concepts

Marginal Probability

Marginal probability $P(A)$ is the probability of an event $A$ occurring, regardless of the outcomes of other variables. It's the "overall" or "unconditional" probability of an event $A$. In our example, $P(\text{Blue})$ is the marginal probability of event the picked ball is blue. It's the probability of picking a blue ball from all the balls in the box, without considering which bag the ball comes from.

Joint Probability

Joint probability refers to the probability of two events occurring together. It's denoted as $P(A, B)$. In our example, $P(\text{Bag A, Blue})$ is a joint probability, representing the probability of a ball being blue and coming from Bag A.

For independent events, the joint probability, which is the probability of both events $A$ and $B$ occurring together, can be calculated using the following formula:

$$ P(A, B) = P(A) \times P(B) $$

This equation implies that the occurrence of event $A$ has no influence on the occurrence of event $B$, and vice versa. In other words, the events are independent if the outcome of one does not affect the outcome of the other. However, this assumption of independence is not always applicable.

For example, in our scenario, the event of a ball being from Bag A influences the probability of the ball being red. Therefore, the events 'a ball is from Bag A' and 'a ball is red' are not independent, and the equation does not work.

When dealing with dependent events, the joint probability of two events $A$ and $B$ happening together is calculated differently than for independent events. The formula for dependent events is:

$$ P(A, B) = P(A) \times P(B|A) $$

In our scenario with the balls and bags, the events are dependent. The probability of a ball being a certain color (say, red or blue) is dependent on which bag (A or B) it is drawn from. Let's break down this concept:

1. Probability of Picking from a Bag $P(A)$: This is the marginal probability $P(\text{Bag A})$ of choosing a ball from a specific bag, say Bag A.
2. Conditional Probability $P(B∣A)$: This is the case we know the ball is from Bag A, the probability picking a ball is red. This probability $P(\text{Red|Bag A})$ is dependent on the composition of Bag A.
3. Joint Probability $P(A,B)$: Combining these, the joint probability $P(\text{Red,Bag A})$ of picking a red ball from Bag A is the product of the probability of picking from Bag A and the conditional probability of the ball being red given it's from Bag A.