Unconscious Statistics
Law of the Unconscious Statistician
In probability theory and statistics, the law of the unconscious statistician (LOTUS), is a theorem used to calculate the expected value of a function \(g(X)\) of a random variable \(X\) when one knows the probability distribution of \(X\) but one does not know the distribution of \(g(X)\).
If the probability mass function is known, \[ \E[g(X)] = \sum_x g(x) p(x) \] If the probability density function is known, \[ \E[g(X)] = \int_{-\infty}^{+\infty} g(x)p(x)\ \d x \] If the cumulative distribution function is known, \[ \E[g(X)] = \int_{-\infty}^{+\infty} g(x)\ \d F(x) \]
Marginal Expectation
If the joint distribution of two random variables \(X\) and \(Y\) is known, then the expectation of one component can be calculated as \[ \E[X] = \int_{-\infty}^{+\infty} x p_X(x)\; \d x = \int_{-\infty}^{+\infty} x \int_{-\infty}^{+\infty} p(x,y)\; \d y\; \d x = \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} xp(x,y)\ \d y\ \d x \] On the other hand, \[ \E [X] = \E_{y \sim p_Y} [\E_{x \sim p(X|Y=y)}] = \int_{-\infty}^{+\infty} p(y) \bigg( \int_{-\infty}^{+\infty} x p(x|y)\ \d x \bigg) \d y \] StackExchange Discussion
Expectation of Non-negative Random Variables
If \(X\) is a random variable whose value is non-negative, and its expectation exists, and
when \(X\) is continuous, \[ \begin{aligned}[b] \E (X) &= \int_{0}^{+\infty} x p(x)\ \d x = \int_{0}^{+\infty} x\ \d \big( P(x) - 1 \big) \\ &= [x \big( P(x) - 1 \big)]\bigg|_{x=0}^{+\infty} - \int_0^{+\infty} \big( P(x) - 1 \big)\ \d x \end{aligned} \] Because the expectation exists, the above expression and especially the \([x \big( P(x) - 1 \big)]\bigg|_{x=0}^{+\infty}\) term must converge: \[ \begin{gather} [x \big( P(x) - 1 \big)]\bigg|_{x=0} = 0 \\ [P(x) - 1]\bigg|_{x \to +\infty} = 0 \Rightarrow [x \big( P(x) - 1 \big)]\bigg|_{x \to +\infty} = 0 \end{gather} \] Therefore, \[ \E(X) = \int_{0}^{+\infty} \big (1 - P(x) \big)\ \d x \]
when \(X\) is discrete and \(X\) only takes on integer values, supposing the max value of \(X\) is \(N\), \[ \begin{aligned} &\E(X) = \sum_{k=0}^{N} [k P(X = k)] \\ &= \sum_{k=0}^{N} [(\sum_{j=0}^{k-1} 1) P(X = k)] \\ &= \sum_{k=0}^{N} [\sum_{j=0}^{k-1} P(X = k)] \\ &= \sum_{j=0}^{N-1} [\sum_{k=j+1}^{N} P(X = k)] \\ &= \sum_{j=0}^{N-1} P(X > j) \end{aligned} \]
StackExchange Discussion || Summation by Parts
Expectation and Quantile Function
Let \(f\) be the PDF and \(F\) be the CDF of a random variable \(X\). Let \(Q = F^{-1}\) be the inverse of \(F\). \(Q\) is called the quantile function of \(X\), and \[ \int_0^1 Q(p)\ \d p \stackrel{p=F(x)}{\Longrightarrow} = \int_{-\infty}^{+\infty} x f(x)\ \d x = \E(X) \] StackExchange Answer