Equivalent Properties of a Linear Operator
This is a summary of material from the course ``Advanced topics in machine learning’’ by Arthur Gretton with slightly more intermediate steps.
Theorem Let (F,‖⋅‖F) and (G,‖⋅‖G) be normed linear spaces. If L:F↦G is a linear operator, then the following three conditions are equivalent.
- L is a bounded operator.
- L is continuous on F.
- L is continuous at one point of F.
Proof Equivalence of these conditions can be proved by proving 1⇒2,2⇒3 and 3⇒1.
Firstly we prove 1⇒2. Assume L is bounded. Then by definition of a bounded operator, for all f∈F, ‖Lf‖G≤‖L‖‖f‖F, where |L| is the operator norm. Set f:=f1−f2. So, we have ‖L(f1−f2)‖G≤‖L‖‖f1−f2‖F which is the definition of Lipschitz continuity with Lipschitz constant |L|. Since Lipschitz continuity implies continuity, 1⇒2 holds.
The implication from 2 to 3 is obvious since by definition an operator is continuous if it is continuous at every point.
Proving 3⇒1 is a bit tricky. Let us recall the definition of continuity. An operator is said to be continuous at f0∈F if for all ϵ>0, there exists δ(ϵ,f0)>0 such that for all f∈F, we have ‖f−f0‖F<δ⇒‖Lf−Lf0‖G<ϵ.
Assume L is continuous at f0. Then, for ϵ=1, there exists δ>0, such that ‖(f0+Δ)−f0‖F=‖Δ‖F≤δ for any Δ implies ‖L(f0+Δ)−Lf0‖G=‖LΔ‖≤ϵ=1. Here f in the definition of continuity is set to f0+Δ, and ϵ is chosen to be 1. Now we try to show that L is bounded.
‖Lf‖G=δδ‖f‖F‖f‖F‖Lf‖G=1δ‖f‖F‖δ‖f‖FL(f)‖G=1δ‖f‖F‖L(δ‖f‖Ff)‖GHere we used the fact that L is linear. Since ‖δ‖f‖Ff‖F=δ, if we set Δ=δ‖f‖Ff, then we have |Δ|F≤δ. So,
‖Lf‖G=1δ‖f‖F‖LΔ‖G≤1δ‖f‖F.where we used the fact that ‖LΔ‖≤1. Since f is arbitrary, we have the definition of a bounded operator with operator norm |L|=1δ.