Convex Conjugate

For a convex function $f$, its convex conjugate $f^$ is defined as $$ f^(t) = \sup_x [x^T \cdot t - f(x)] $$ By definition, a Convex Conjugate pair $(f,f^)$ has the following property: $$ f(x) + f^(t) \ge x^T \cdot t $$ As a conjugate, $f^{} = f$: $$ \begin{aligned} f^{}(t) &= \sup_x [x^T \cdot t - f^*(x)] \ &= \sup_x [x^T \cdot t - \sup_y [y^T \cdot x - f(y)]] \ &= \sup_x [x^T \cdot t + \inf_y [f(y) - y^T \cdot x]] \ &= \sup_x \inf_y [x^T (t-y) + f(y)] \ &\Downarrow_\text{Mini-max Theorem} \ &= \inf_y \sup_x [x^T (t-y) + f(y)] \end{aligned} $$ The above reaches the infimum only if $y=t$. Otherwise, $\sup_x [x^T (t-y) + f(y)]$ can make it to infinity. Therefore, $$ f^{**}(t) = \inf_y \sup_x [f(t)] = f(t) $$ Wiki || 凸优化-凸共轭