# Moreau's decomposition theorem

(Difference between revisions)
 Revision as of 01:53, 19 November 2011 (edit) (→Proof)← Previous diff Revision as of 01:59, 19 November 2011 (edit) (undo) (→Moreau's theorem)Next diff → Line 48: Line 48: under the addition of vectors and multiplication of vectors by positive scalars. under the addition of vectors and multiplication of vectors by positive scalars. - '''Theorem (Moreau).''' Let $\mathcal K$ be a closed convex cone in the Hilbert space $(\mathbb H,\langle\cdot,\cdot\rangle)$ and $\mathcal K^\circ$ its '''polar cone'''; that is, the closed convex cone defined by $\mathcal K^\circ=\{a\in\mathbb H\mid\langle a,b\rangle\leq0,\,\forall b\in\mathcal K\}.$ + '''Theorem (Moreau).''' Let $\mathcal{K}$ be a closed convex cone in the Hilbert space $(\mathbb{H},\langle\cdot,\cdot\rangle)$ and $\mathcal{K}^\circ$ its '''polar cone'''; that is, the closed convex cone defined by $\mathcal{K}^\circ=\{a\in\mathbb{H}\,\mid\,\langle a,b\rangle\leq0,\,\forall b\in\mathcal{K}\}.$ - For $x,y,z\in\mathbb H$ the following statements are equivalent: + For $x,y,z\in\mathbb{H}$ the following statements are equivalent:
-
1. $z=x+y,\,x\in\mathcal K,\,y\in\mathcal K^\circ$ and $\langle x,y\rangle=0,$
2. +
3. $z=x+y,\,x\in\mathcal{K},\,y\in\mathcal{K}^\circ$ and $\langle x,y\rangle=0,$
4. -
5. $x=P_{\mathcal K}z$ and $y=P_{\mathcal K^\circ}z.$ +
6. $x=P_{\mathcal{K}}z$ and $y=P_{\mathcal{K}^\circ}z.$
Line 60: Line 60: === Proof of Moreau's theorem === === Proof of Moreau's theorem ===
-
• 1$\Rightarrow$2: For all $p\in K$ we have +
• 1$\Rightarrow$2: For all $p\in\mathcal{K}$ we have
Line 66: Line 66:
- Then, by the characterization of the projection, it follows that $x=P_{\mathcal K}z.$ Similarly, for all $q\in K^\circ$ we have + Then, by the characterization of the projection, it follows that $x=P_{\mathcal{K}}z.$ Similarly, for all $q\in\mathcal{K}^\circ$ we have
Line 72: Line 72:
- and thus $y=P_{\mathcal K^\circ}z.$
• + and thus $y=P_{\mathcal{K}^\circ}z.$ -
• 2$\Rightarrow$1: By using the characterization of the projection, we have $\langle z-x,p-x\rangle\leq0,$ for all $p\in\mathcal K.$ In particular, if $p=0,\,$ then $\langle z-x,x\rangle\geq0$ and if $p=2x,\,$ then $\langle z-x,x\rangle\leq0.$ Thus, $\langle z-x,x\rangle=0.$ Denote $u=z-x.\,$ Then $\langle x,u\rangle=0.$ It remains to show that $u=y.\,$ First, we prove that $u\in\mathcal K^\circ.$ For this we have to show that $\langle u,p\rangle\leq0,$ for +
• 2$\Rightarrow$1: By using the characterization of the projection, we have $\langle z-x,p-x\rangle\leq0,$ for all $p\in\mathcal K.$ In particular, if $p=0,\,$ then $\langle z-x,x\rangle\geq0$ and if $p=2x,\,$ then $\langle z-x,x\rangle\leq0.$ Thus, $\langle z-x,x\rangle=0.$ Denote $u=z-x.\,$ Then $\langle x,u\rangle=0.$ It remains to show that $u=y.\,$ First, we prove that $u\in\mathcal{K}^\circ.$ For this we have to show that $\langle u,p\rangle\leq0,$ for - all $p\in\mathcal K.$ By using the characterization of the projection, we have + all $p\in\mathcal{K}.$ By using the characterization of the projection, we have
Line 82: Line 82:
- for all $p\in\mathcal K.$ Thus, $u\in\mathcal K^\circ.$ We also have + for all $p\in\mathcal{K}.$ Thus, $u\in\mathcal{K}^\circ.$ We also have
Line 90: Line 90:
- for all $q\in K^\circ,$ because $x\in K.$ By using again the characterization of the projection, it follows that $u=y.\,$ + for all $q\in\mathcal{K}^\circ,$ because $x\in\mathcal{K}.$ By using again the characterization of the projection, it follows that $u=y.\,$

## Revision as of 01:59, 19 November 2011

(In particular, we can have $LaTeX: \mathbb H=\mathbb R^n$ everywhere in this page.)

## Projection on closed convex sets

### Projection mapping

Let $LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle)$ be a Hilbert space and $LaTeX: \mathcal{C}$ a closed convex set in $LaTeX: \mathbb{H}.$ The projection mapping $LaTeX: P_{\mathcal{C}}$ onto $LaTeX: \mathcal{C}$ is the mapping $LaTeX: P_{\mathcal{C}}:\mathbb{H}\to\mathbb{H}$ defined by $LaTeX: P_{\mathcal{C}}(x)\in\mathcal{C}$ and $LaTeX: \parallel x-P_{\mathcal{C}}(x)\parallel=\min\{\parallel x-y\parallel\,:\,y\in\mathcal{C}\}.$

### Characterization of the projection

Let $LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle)$ be a Hilbert space, $LaTeX: \mathcal{C}$ a closed convex set in $LaTeX: \mathbb{H},\,u\in\mathbb{H}$ and $LaTeX: v\in\mathcal{C}.$ Then $LaTeX: v=P_{\mathcal{C}}(u)$ if and only if $LaTeX: \langle u-v,w-v\rangle\leq0$ for all $LaTeX: w\in\mathcal{C}.$

### Proof

Suppose that $LaTeX: v=P_{\mathcal{C}}u.$ Let $LaTeX: w\in\mathcal{C}$ and $LaTeX: t\in (0,1)$ be arbitrary. By using the convexity of $LaTeX: \mathcal{C},$ it follows that $LaTeX: (1-t)v+tw\in\mathcal{C}.$ Then, by using the definition of the projection, we have $LaTeX: \parallel u-v\parallel^2\leq\parallel u-((1-t)v+tw)\parallel^2=\parallel u-v-t(w-v)\parallel^2=\parallel u-v\parallel^2-2t\langle u-v,w-v\rangle+t^2\parallel w-v\parallel^2,$

Hence, $LaTeX: \langle u-v,w-v\rangle\leq\frac t2\parallel w-v\parallel^2.$

By tending with $LaTeX: t\,$ to $LaTeX: 0,\,$ we get $LaTeX: \langle u-v,w-v\rangle\leq0.$

Conversely, suppose that $LaTeX: \langle u-v,w-v\rangle\leq0,$ for all $LaTeX: w\in\mathcal C.$ Then $LaTeX: \parallel u-w\parallel^2=\parallel u-v-(w-v)\parallel^2=\parallel u-v\parallel^2-2\langle u-v,w-v\rangle+\parallel w-v\parallel^2\geq \parallel u-v\parallel^2,$

for all $LaTeX: w\in\mathcal C.$ Hence, by using the definition of the projection, we get $LaTeX: v=P_{\mathcal C}u.$

## Moreau's theorem

Moreau's theorem is a fundamental result characterizing projections onto closed convex cones in Hilbert spaces.

Recall that a convex cone in a vector space is a set which is invariant under the addition of vectors and multiplication of vectors by positive scalars.

Theorem (Moreau). Let $LaTeX: \mathcal{K}$ be a closed convex cone in the Hilbert space $LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle)$ and $LaTeX: \mathcal{K}^\circ$ its polar cone; that is, the closed convex cone defined by $LaTeX: \mathcal{K}^\circ=\{a\in\mathbb{H}\,\mid\,\langle a,b\rangle\leq0,\,\forall b\in\mathcal{K}\}.$

For $LaTeX: x,y,z\in\mathbb{H}$ the following statements are equivalent:

1. $LaTeX: z=x+y,\,x\in\mathcal{K},\,y\in\mathcal{K}^\circ$ and $LaTeX: \langle x,y\rangle=0,$
2. $LaTeX: x=P_{\mathcal{K}}z$ and $LaTeX: y=P_{\mathcal{K}^\circ}z.$

### Proof of Moreau's theorem

• 1 $LaTeX: \Rightarrow$2: For all $LaTeX: p\in\mathcal{K}$ we have $LaTeX: \langle z-x,p-x\rangle=\langle y,p-x\rangle=\langle y,p\rangle\leq0.$

Then, by the characterization of the projection, it follows that $LaTeX: x=P_{\mathcal{K}}z.$ Similarly, for all $LaTeX: q\in\mathcal{K}^\circ$ we have $LaTeX: \langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0$

and thus $LaTeX: y=P_{\mathcal{K}^\circ}z.$
• 2 $LaTeX: \Rightarrow$1: By using the characterization of the projection, we have $LaTeX: \langle z-x,p-x\rangle\leq0,$ for all $LaTeX: p\in\mathcal K.$ In particular, if $LaTeX: p=0,\,$ then $LaTeX: \langle z-x,x\rangle\geq0$ and if $LaTeX: p=2x,\,$ then $LaTeX: \langle z-x,x\rangle\leq0.$ Thus, $LaTeX: \langle z-x,x\rangle=0.$ Denote $LaTeX: u=z-x.\,$ Then $LaTeX: \langle x,u\rangle=0.$ It remains to show that $LaTeX: u=y.\,$ First, we prove that $LaTeX: u\in\mathcal{K}^\circ.$ For this we have to show that $LaTeX: \langle u,p\rangle\leq0,$ for all $LaTeX: p\in\mathcal{K}.$ By using the characterization of the projection, we have $LaTeX: \langle u,p\rangle=\langle u,p-x\rangle=\langle z-x,p-x\rangle\leq0,$

for all $LaTeX: p\in\mathcal{K}.$ Thus, $LaTeX: u\in\mathcal{K}^\circ.$ We also have $LaTeX: \langle z-u,q-u\rangle=\langle x,q-u\rangle=\langle x,q\rangle\leq0,$

for all $LaTeX: q\in\mathcal{K}^\circ,$ because $LaTeX: x\in\mathcal{K}.$ By using again the characterization of the projection, it follows that $LaTeX: u=y.\,$

### notes

For definition of convex cone in finite dimension see Convex cones, Wikimization.

For definition of polar cone in finite dimension, see Convex Optimization & Euclidean Distance Geometry. $LaTeX: \mathcal K^{\circ\circ}=K$ see Extended Farkas' lemma.

## Applications

### References

• J. J. Moreau, Décomposition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires, C. R. Acad. Sci., volume 255, pages 238–240, 1962.