Conditional distributions and independence
The concept of conditional probability extends naturally to distributions of random variables. This extension is useful for describing probability distributions that depend on another variable. For example: data on health outcomes by age provides information about the conditional distribution of outcomes as it depends on age; the distribution of daytime temperatures depends on time of year and location; the distributions of measures of educational attainment depend on socioeconomic indicators. In all of these cases, we are considering the conditional distribution of a variable of interest (health outcomes, daytime temperatures, educational attainment) given other variables (age, time/location, socioeconomic variables); this may be of use in modeling the dependence, correcting by conditioning variables to obtain joint distributions, or a variety of other purposes. Here we’ll focus on the relationship between joint probability distributions and conditional distributions, and illustrate some applications of conditional distributions.
If
If
In the continuous case, the contional PDF is constructed in analogous fashion:
Let
Check your understanding Show that
Conditional expectation
The conditional expectation of
That is, conditional expectation is simply an expected value computed in the usual way but using the conditional mass or density function in place of the marginal. Similarly, the conditional variance is defined as:
In the previous example, the conditional expectations are easy to find based on the properties of the uniform distribution:
Consider the example immediately above, and notice that the conditional means are functions of the value of the other “conditioning” variable. That will be generally true, that is:
Theorem (Total expectation). For any random variables
We will review the proof in class. Check that the result holds in the example above.
Theorem (Total variance). For any random variables
We will review the proof in class. Check that the result holds in the example above.
Independence
Intuitively, random variables are independent if the value of one does not affect the distribution of another. In other words,
We write
Theorem (factorization theorem).
Let
Lemma. If
The contrapositive tells us that if the joint support set of any random variables is not a Cartesian product, then they cannot be independent. Moreover, the lemma makes it rather straightforward to establish the following result.
Corrollary. If
Conditional probability models
Notice that the definitions of conditional PDF/PMFs entail that the joint PDF/PMFs can be obtained from either conditional and the remaining marginal:
This allows one to construct models for multivariate processes from conditional distributions in a hierarchical fashion. For example: