This is a summary of material from the course ``Advanced topics in machine learning’’ by Arthur Gretton with slightly more intermediate steps.

Theorem Let (F,F) and (G,G) be normed linear spaces. If L:FG is a linear operator, then the following three conditions are equivalent.

  • L is a bounded operator.
  • L is continuous on F.
  • L is continuous at one point of F.

Proof Equivalence of these conditions can be proved by proving 12,23 and 31.

Firstly we prove 12. Assume L is bounded. Then by definition of a bounded operator, for all fF, LfGLfF, where |L| is the operator norm. Set f:=f1f2. So, we have L(f1f2)GLf1f2F which is the definition of Lipschitz continuity with Lipschitz constant |L|. Since Lipschitz continuity implies continuity, 12 holds.

The implication from 2 to 3 is obvious since by definition an operator is continuous if it is continuous at every point.

Proving 31 is a bit tricky. Let us recall the definition of continuity. An operator is said to be continuous at f0F if for all ϵ>0, there exists δ(ϵ,f0)>0 such that for all fF, we have ff0F<δLfLf0G<ϵ.

Assume L is continuous at f0. Then, for ϵ=1, there exists δ>0, such that (f0+Δ)f0F=ΔFδ for any Δ implies L(f0+Δ)Lf0G=LΔϵ=1. Here f in the definition of continuity is set to f0+Δ, and ϵ is chosen to be 1. Now we try to show that L is bounded.

LfG=δδfFfFLfG=1δfFδfFL(f)G=1δfFL(δfFf)G

Here we used the fact that L is linear. Since δfFfF=δ, if we set Δ=δfFf, then we have |Δ|Fδ. So,

LfG=1δfFLΔG1δfF.

where we used the fact that LΔ1. Since f is arbitrary, we have the definition of a bounded operator with operator norm |L|=1δ.