Gaussian Distribution
Gaussian Distribution
One-dimensional
Suppose \(1\)-d random variable \(x \sim N(\mu, \sigma^2)\), then its density function is \[ p(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}(\frac{x - \mu}{\sigma})^2} \]
To verify that it integrates to \(1\), \[ \begin{aligned} \int_{-\infty}^{+\infty} p(x) \d x &= \sqrt{(\int_{-\infty}^{+\infty} p(x) \d x) \cdot (\int_{-\infty}^{+\infty} p(y) \d y)} \\ &= \sqrt {\int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} p(x)p(y) \d x \d y} \\ \end{aligned} \]
Let \(I^2 = \int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty} p(x)p(y) \d x \d y\), \[ \begin{gather} \int_{-\infty}^{+\infty} p(x) \d x = \sqrt{I^2} \\ I^2 = \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2} (\frac{x - \mu}{\sigma})^2} \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2} (\frac{y - \mu}{\sigma})^2} \d x \d y \end{gather} \] Let \(u = \frac{x-\mu}{\sigma}, v = \frac{y-\mu}{\sigma}\), \[ \begin{aligned} I^2 &= \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}u^2}\frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}v^2} \d u \d v \\ &= \frac{1}{2\pi} \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} e^{-\frac{1}{2}(u^2 + v^2)} \d u \d v \end{aligned} \] Let \(u = r \sin \theta, v = r \cos \theta\), \[ \begin{aligned} \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} e^{-\frac{1}{2}(u^2+v^2)} \d u \d v &= \int_{0}^{2\pi} \int_{0}^{+\infty} e^{-\frac{1}{2} (r^2 \sin^2\theta + r^2 \cos^2\theta)} r \d r \d\theta \\ &= \int_{0}^{2\pi} \int_{0}^{+\infty} -e^{-\frac{1}{2} r^2} \d(-\frac{1}{2} r^2) \d\theta \\ &= \int_{0}^{2\pi}-e^{t}\Big|_{t=0}^{t=-\infty}d\theta \\ &= \int_{0}^{2\pi}d\theta \\ &= 2\pi \end{aligned} \] Therefore, \(I^2 = \frac{1}{2\pi}2\pi = 1\), \(\int_{-\infty}^{+\infty}p(x)dx = \sqrt{I^2} = 1\)
Independent standard \(n\)-dimensional
Let \(Z = [Z_1, Z_2, ..., Z_n]^T\), suppose \(Z_i, Z_j (i,j=1,...,n \and i \ne j)\) are independent and \(Z_i (i=1,...,n)\) observes standard Gaussian distribution, we can derive the joint distribution density function for random variable \(Z\) to be \[ \begin{aligned} \newcommand{z}{\mathrm{z}} p(\z) &= p(z_1, z_2, ..., z_n) = \prod_{i=1}^n \frac{1}{\sqrt{2\pi}} e^{-\frac{1}{2} z_i^2} \\ &= \frac{1}{(2\pi)^{\frac{n}{2}}} e^{-\frac{1}{2}\z^T\z} \\ \end{aligned} \]
First-order correlated \(n\)-dimensional
We have given the joint distribution function of independent \(n\)-dimensional standard Gaussian distribution. What if \(n\) dimensions of \(Z\) are not standard, are not independent with each other, but are correlated only in first order?
We may begin with standard Gaussian random variables \(X = [X_1, \dots, X_n]\). Then we can shift \(X\) by \(\mu\) and linearly transform it with an invertible matrix \(B^{-1}\). \(Z\) mentioned above will just be such a matrix as \(Z = B^{-1} (X - \mu)\) and \[ \begin{gather} X = B^{-1}(Z - \mu) \sim \mathcal{N}(0, I) \\ p_X(\x) = \frac{1}{(2\pi)^{\frac{n}{2}}} e^{-\frac{1}{2} \x^T\x} \end{gather} \]
Suppose \(Z\) is to take on values in \(\mathcal{Z}\), which is a subset of \(\R^{n}\), \[ \newcommand{z}{\mathrm{z}} P_Z(Z \in \mathcal{Z}) = \int_\mathcal{Z} p_Z(\z) \d \z \] Since \(Z = f(X) = BX + \mu\) and \(B\) is invertible, the mapping \(X \to Z\) is one-to-one, therefore the multivariate Jacobian transformation is \[ J(X \to Z) = B^{-1} \\ \] with its determinant \(J = |J(X \to Z)| = |B^{-1}| = |B|^{-1}\). Note that \[ \begin{aligned} |J| &= \sqrt{|B|^{-1}|B|^{-1}} \\ &= \sqrt{|B|^{-1}|B^T|^{-1}} \\ &= \sqrt{|BB^T|^{-1}} \\ &= |BB^T|^{-\frac{1}{2}} \end{aligned} \] Therefore, \[ \begin{aligned} P_Z (Z \in \mathcal{Z}) &= P_X (X \in f^{-1}(\mathcal{Z})) \\ P_Z (Z \in \mathcal{Z}) &= \int_{f^{-1}(\mathcal{Z})} p_X(\x) \d \x \\ &\Downarrow_{\x = f^{-1}(\z)} \\ \int_\mathcal{Z} p_Z (\z) \d \z &= \int_\mathcal{Z} p_X (f^{-1}(\z)) |J| \d \z \\ \end{aligned} \]
\[ \begin{aligned} &p_Z (\z) = p_X (f^{-1}(\z))|J| = p_X (B^{-1} (\z - \mu) |J| \\ &= \frac{1}{(2\pi)^{\frac{n}{2}}} e^{-\frac{1}{2} (\z-\mu)^T(B^{-1})^T B^{-1} (\z-\mu)} |BB^T|^{-\frac{1}{2}} \\ &= \frac{1}{(2\pi)^{\frac{n}{2}}} e^{-\frac{1}{2} (\z-\mu)^T (B^T)^{-1} B^{-1} (\z-\mu)} |BB^T|^{-\frac{1}{2}} \\ &= \frac{1}{(2\pi)^{\frac{n}{2}}} e^{-\frac{1}{2} (\z-\mu)^T (BB^T)^{-1} (\z-\mu)} |BB^T|^{-\frac{1}{2}} \\ &= \frac{1}{(2\pi)^{\frac{n}{2}}|BB^T|^\frac{1}{2}} e^{-\frac{1}{2} (\z-\mu)^T (BB^T)^{-1} (\z-\mu)} \\ \end{aligned} \]
Also note that \[ \begin{aligned} \Sigma_Z &= E[(\z-\mu) (\z-\mu)^T] \\ &= E[B X X^T B^T] \\ &= B E[X X^T] B^T \\ &= B I B^T \\ &= B B^T \end{aligned} \]
Thus, \[ p_Z(\z) = \frac{1}{\sqrt{(2\pi)^n|\Sigma_Z|}} e^{-\frac{1}{2} (\z-\mu)^T \Sigma_Z^{-1} (\z-\mu)} \]
External Materials
为什么高斯分布概率密度函数的积分等于1 - 知乎 (zhihu.com) || 多元高斯分布完全解析 - 知乎 (zhihu.com)