Suppose we have a fair standard die which we roll twice. The die has no magical properties that cause the outcome of one roll to influence the outcome of a subsequent roll, so we can assume that the outcomes of each roll are independent.
Let and denote the maximum and minimum of the two rolls, respectively. By definition of maximum and minimum, . For example, if we know that the minimum of the two rolls is , then we know that the maximum is at least .
This example illustrates that and are not independent: knowing provides us information on the distribution of . Additionally, the above example tells us that larger values of are associated with larger values of , since the minimum of the rolls is a lower bound for the maximum of the rolls. Given this observation, we might suspect that and are positively correlated. To validate this suspicion, we compute the joint distribution of and using conditional probability.
Let and denote the outcomes of the first and second rolls, respectively. For integers with , observe that
Using this formula, we can compute the marginal distributions of and . For ,
We summarize these results with the table below.
One key observation about the conditional distribution of given , the distribution of the random variable , is that it is not uniform. In the first draft of this post, I wrote, as an example, that is uniformly distributed over the integers , , and . This is false:
The reason for this is that there is only one way for the maximum of the two rolls to be , given that the minimum is : both rolls have outcomes of . However, there are two ways for the maximum of the two rolls to be or , given that minimum that the minimum is : the first roll is and the second roll is or , or the the first roll is or and the second roll is .
Now that we have the joint pmf of and , we can return to the original question of the correlation between and . Observe that
Using the marginal distributions of and , we compute the expectations to be
Putting these results together, we see that and have positive covariance:
I leave it as an exercise to the reader to compute the Pearson -correlation coefficient of and , which is defined by
It’s not hard; since you know , you just need to compute the variances of and using their respective marginal distributions.
One of the reasons I like this problem is it illustrates that covariance is a measure of association, not causation. By association, I mean how two random variables vary together. Statistics teachers try to knock in students’ head this fact with the oft-repeated statement “correlation does not imply causation,” but all too often I read descriptions of studies that measure the correlation between two variables and and summarize the conclusions by saying that causes or causes . The media are the worst offenders when it comes to confounding correlation and causation; just read the headlines in the Health section of your Google News page.
Even though the minimum and maximum of the die rolls are positively correlated, as shown above, no one would say that a large minimum causes a large maximum; remember that we assume the rolls are independent. Knowing the minimum of the rolls only gives information on the possible values of the maximum, by definition of minimum and maximum. The maximum is tautologically at least as big as the minimum, so if , then we know that . The positive correlation stems from the information one random variable provides about one another.