Metrics
-
Spectral Normalization
Spectral normalization of an \(M \times N\) matrix \(A\) is defined as \[ ||A||_2 = \max_{\mathrm z} \frac{||A\mathrm z||_2}{||\mathrm z||_2} = \sqrt{\lambda_{\max}(A^TA)} = \sigma_{\max}(A) \] where \(\rm z \in \R^N\), \(\lambda_{\max}(A^TA)\) is the maximum eigenvalue of matrix \(A^TA\), which is exactly \(A\)’s largest singular value \(\sigma_{\max}(A)\).
-
Frobenius Normalization
Frobenius Normalization of an \(m \times n\) matrix \(A\) is defined as \[ ||A||_F \triangleq \sqrt{\sum_{i,j}A_{ij}^2} \] It can be found that \[ \begin{aligned} ||A||_F^2 = \sum_{ij}A_{ij}^2 &= \sum_{i=1}^m\sum_{j=1}^nA_{ij}A_{ij} = \sum_{i=1}^n\sum_{j=1}^mA_{ji}A_{ji} \\ &= \sum_{i=1}^m(\sum_{j=1}^nA_{ij}A_{ji}^T) = \sum_{i=1}^n(\sum_{j=1}^mA_{ij}^TA_{ji}) \\ &= \sum_{i=1}^m(A_{i:}A_{:i}^T) = \sum_{i=1}^n(A_{i:}^TA_{:i})\\ &= \sum_{i=1}^m(AA^T)_{ii} = \sum_{i=1}^n(A^TA)_{ii}\\ &= \tr(AA^T) = \tr(A^TA) \end{aligned} \]
-
Chebyshev Distance
Chebyshev distance is a specific form of Minkowski norm (\(l_p\) norm): \[ d_p(x, x^\prime) = ||x - x^\prime||_p \coloneq (\sum_{i=1}^n|x_i - x^\prime_i|^p)^{1/p} \] where \(p \to \infty\). Chebyshev distance is