This is a summary of material from the course ``Advanced topics in machine learning’’ by Arthur Gretton with slightly more intermediate steps.

Theorem Let and be normed linear spaces. If $L:\mathcal{F}\mapsto\mathcal{G}$ is a linear operator, then the following three conditions are equivalent.

  • $L$ is a bounded operator.
  • $L$ is continuous on $\mathcal{F}$.
  • $L$ is continuous at one point of $\mathcal{F}$.

Proof Equivalence of these conditions can be proved by proving $1\Rightarrow2,2\Rightarrow3$ and $3\Rightarrow1$.

Firstly we prove $1\Rightarrow2$. Assume $L$ is bounded. Then by definition of a bounded operator, for all $f\in\mathcal{F}$, , where $|L|$ is the operator norm. Set $f:=f_{1}-f_{2}$. So, we have which is the definition of Lipschitz continuity with Lipschitz constant $|L|$. Since Lipschitz continuity implies continuity, $1\Rightarrow2$ holds.

The implication from 2 to 3 is obvious since by definition an operator is continuous if it is continuous at every point.

Proving $3\Rightarrow1$ is a bit tricky. Let us recall the definition of continuity. An operator is said to be continuous at $f_{0}\in\mathcal{F}$ if for all $\epsilon>0$, there exists $\delta(\epsilon,f_{0})>0$ such that for all $f\in\mathcal{F}$, we have

Assume $L$ is continuous at $f_{0}$. Then, for $\epsilon=1$, there exists $\delta>0$, such that for any $\Delta$ implies . Here $f$ in the definition of continuity is set to $f_{0}+\Delta$, and $\epsilon$ is chosen to be 1. Now we try to show that $L$ is bounded.

Here we used the fact that $L$ is linear. Since , if we set , then we have $|\Delta|_{\mathcal{F}}\leq\delta$. So,

where we used the fact that $\left\Vert L\Delta\right\Vert \leq1$. Since $f$ is arbitrary, we have the definition of a bounded operator with operator norm $|L|=\frac{1}{\delta}$.