Moreau's decomposition theorem

From Wikimization

(Difference between revisions)
Jump to: navigation, search
(→Proof)
(→Moreau's theorem)
Line 48: Line 48:
under the addition of vectors and multiplication of vectors by positive scalars.
under the addition of vectors and multiplication of vectors by positive scalars.
-
'''Theorem (Moreau).''' Let <math>\mathcal K</math> be a closed convex cone in the Hilbert space <math>(\mathbb H,\langle\cdot,\cdot\rangle)</math> and <math>\mathcal K^\circ</math> its '''polar cone'''; that is, the closed convex cone defined by <math>\mathcal K^\circ=\{a\in\mathbb H\mid\langle a,b\rangle\leq0,\,\forall b\in\mathcal K\}.</math>
+
'''Theorem (Moreau).''' Let <math>\mathcal{K}</math> be a closed convex cone in the Hilbert space <math>(\mathbb{H},\langle\cdot,\cdot\rangle)</math> and <math>\mathcal{K}^\circ</math> its '''polar cone'''; that is, the closed convex cone defined by <math>\mathcal{K}^\circ=\{a\in\mathbb{H}\,\mid\,\langle a,b\rangle\leq0,\,\forall b\in\mathcal{K}\}.</math>
-
For <math>x,y,z\in\mathbb H</math> the following statements are equivalent:
+
For <math>x,y,z\in\mathbb{H}</math> the following statements are equivalent:
<ol>
<ol>
-
<li><math>z=x+y,\,x\in\mathcal K,\,y\in\mathcal K^\circ</math> and <math>\langle x,y\rangle=0,</math></li>
+
<li><math>z=x+y,\,x\in\mathcal{K},\,y\in\mathcal{K}^\circ</math> and <math>\langle x,y\rangle=0,</math></li>
-
<li><math>x=P_{\mathcal K}z</math> and <math>y=P_{\mathcal K^\circ}z.</math>
+
<li><math>x=P_{\mathcal{K}}z</math> and <math>y=P_{\mathcal{K}^\circ}z.</math>
</li>
</li>
</ol>
</ol>
Line 60: Line 60:
=== Proof of Moreau's theorem ===
=== Proof of Moreau's theorem ===
<ul>
<ul>
-
<li>1<math>\Rightarrow</math>2: For all <math>p\in K</math> we have
+
<li>1<math>\Rightarrow</math>2: For all <math>p\in\mathcal{K}</math> we have
<center>
<center>
Line 66: Line 66:
</center>
</center>
-
Then, by the characterization of the projection, it follows that <math>x=P_{\mathcal K}z.</math> Similarly, for all <math>q\in K^\circ</math> we have
+
Then, by the characterization of the projection, it follows that <math>x=P_{\mathcal{K}}z.</math> Similarly, for all <math>q\in\mathcal{K}^\circ</math> we have
<center>
<center>
Line 72: Line 72:
</center>
</center>
-
and thus <math>y=P_{\mathcal K^\circ}z.</math></li>
+
and thus <math>y=P_{\mathcal{K}^\circ}z.</math></li>
-
<li>2<math>\Rightarrow</math>1: By using the characterization of the projection, we have <math>\langle z-x,p-x\rangle\leq0,</math> for all <math>p\in\mathcal K.</math> In particular, if <math>p=0,\,</math> then <math>\langle z-x,x\rangle\geq0</math> and if <math>p=2x,\,</math> then <math>\langle z-x,x\rangle\leq0.</math> Thus, <math>\langle z-x,x\rangle=0.</math> Denote <math>u=z-x.\,</math> Then <math>\langle x,u\rangle=0.</math> It remains to show that <math>u=y.\,</math> First, we prove that <math>u\in\mathcal K^\circ.</math> For this we have to show that <math>\langle u,p\rangle\leq0,</math> for
+
<li>2<math>\Rightarrow</math>1: By using the characterization of the projection, we have <math>\langle z-x,p-x\rangle\leq0,</math> for all <math>p\in\mathcal K.</math> In particular, if <math>p=0,\,</math> then <math>\langle z-x,x\rangle\geq0</math> and if <math>p=2x,\,</math> then <math>\langle z-x,x\rangle\leq0.</math> Thus, <math>\langle z-x,x\rangle=0.</math> Denote <math>u=z-x.\,</math> Then <math>\langle x,u\rangle=0.</math> It remains to show that <math>u=y.\,</math> First, we prove that <math>u\in\mathcal{K}^\circ.</math> For this we have to show that <math>\langle u,p\rangle\leq0,</math> for
-
all <math>p\in\mathcal K.</math> By using the characterization of the projection, we have
+
all <math>p\in\mathcal{K}.</math> By using the characterization of the projection, we have
<center>
<center>
Line 82: Line 82:
</center>
</center>
-
for all <math>p\in\mathcal K.</math> Thus, <math>u\in\mathcal K^\circ.</math> We also have
+
for all <math>p\in\mathcal{K}.</math> Thus, <math>u\in\mathcal{K}^\circ.</math> We also have
<center>
<center>
Line 90: Line 90:
</center>
</center>
-
for all <math>q\in K^\circ,</math> because <math>x\in K.</math> By using again the characterization of the projection, it follows that <math>u=y.\,</math>
+
for all <math>q\in\mathcal{K}^\circ,</math> because <math>x\in\mathcal{K}.</math> By using again the characterization of the projection, it follows that <math>u=y.\,</math>
</ul>
</ul>

Revision as of 01:59, 19 November 2011

LaTeX: - Sándor Zoltán Németh

(In particular, we can have LaTeX: \mathbb H=\mathbb R^n everywhere in this page.)

Contents

Projection on closed convex sets

Projection mapping

Let LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle) be a Hilbert space and LaTeX: \mathcal{C} a closed convex set in LaTeX: \mathbb{H}. The projection mapping LaTeX: P_{\mathcal{C}} onto LaTeX: \mathcal{C} is the mapping LaTeX: P_{\mathcal{C}}:\mathbb{H}\to\mathbb{H} defined by LaTeX: P_{\mathcal{C}}(x)\in\mathcal{C} and

LaTeX: \parallel x-P_{\mathcal{C}}(x)\parallel=\min\{\parallel x-y\parallel\,:\,y\in\mathcal{C}\}.

Characterization of the projection

Let LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle) be a Hilbert space, LaTeX: \mathcal{C} a closed convex set in LaTeX: \mathbb{H},\,u\in\mathbb{H} and LaTeX: v\in\mathcal{C}. Then LaTeX: v=P_{\mathcal{C}}(u) if and only if LaTeX: \langle u-v,w-v\rangle\leq0 for all LaTeX: w\in\mathcal{C}.

Proof

Suppose that LaTeX: v=P_{\mathcal{C}}u. Let LaTeX: w\in\mathcal{C} and LaTeX: t\in (0,1) be arbitrary. By using the convexity of LaTeX: \mathcal{C}, it follows that LaTeX: (1-t)v+tw\in\mathcal{C}. Then, by using the definition of the projection, we have

LaTeX: \parallel u-v\parallel^2\leq\parallel u-((1-t)v+tw)\parallel^2=\parallel u-v-t(w-v)\parallel^2=\parallel u-v\parallel^2-2t\langle u-v,w-v\rangle+t^2\parallel w-v\parallel^2,

Hence,

LaTeX: \langle u-v,w-v\rangle\leq\frac t2\parallel w-v\parallel^2.

By tending with LaTeX: t\, to LaTeX: 0,\, we get LaTeX: \langle u-v,w-v\rangle\leq0.

Conversely, suppose that LaTeX: \langle u-v,w-v\rangle\leq0, for all LaTeX: w\in\mathcal C. Then

LaTeX: \parallel u-w\parallel^2=\parallel u-v-(w-v)\parallel^2=\parallel u-v\parallel^2-2\langle u-v,w-v\rangle+\parallel w-v\parallel^2\geq \parallel u-v\parallel^2,

for all LaTeX: w\in\mathcal C. Hence, by using the definition of the projection, we get LaTeX: v=P_{\mathcal C}u.

Moreau's theorem

Moreau's theorem is a fundamental result characterizing projections onto closed convex cones in Hilbert spaces.

Recall that a convex cone in a vector space is a set which is invariant under the addition of vectors and multiplication of vectors by positive scalars.

Theorem (Moreau). Let LaTeX: \mathcal{K} be a closed convex cone in the Hilbert space LaTeX: (\mathbb{H},\langle\cdot,\cdot\rangle) and LaTeX: \mathcal{K}^\circ its polar cone; that is, the closed convex cone defined by LaTeX: \mathcal{K}^\circ=\{a\in\mathbb{H}\,\mid\,\langle a,b\rangle\leq0,\,\forall b\in\mathcal{K}\}.

For LaTeX: x,y,z\in\mathbb{H} the following statements are equivalent:

  1. LaTeX: z=x+y,\,x\in\mathcal{K},\,y\in\mathcal{K}^\circ and LaTeX: \langle x,y\rangle=0,
  2. LaTeX: x=P_{\mathcal{K}}z and LaTeX: y=P_{\mathcal{K}^\circ}z.

Proof of Moreau's theorem

  • 1LaTeX: \Rightarrow2: For all LaTeX: p\in\mathcal{K} we have

    LaTeX: \langle z-x,p-x\rangle=\langle y,p-x\rangle=\langle y,p\rangle\leq0.

    Then, by the characterization of the projection, it follows that LaTeX: x=P_{\mathcal{K}}z. Similarly, for all LaTeX: q\in\mathcal{K}^\circ we have

    LaTeX: \langle z-y,q-y\rangle=\langle x,q-y\rangle=\langle x,q\rangle\leq0

    and thus LaTeX: y=P_{\mathcal{K}^\circ}z.
  • 2LaTeX: \Rightarrow1: By using the characterization of the projection, we have LaTeX: \langle z-x,p-x\rangle\leq0, for all LaTeX: p\in\mathcal K. In particular, if LaTeX: p=0,\, then LaTeX: \langle z-x,x\rangle\geq0 and if LaTeX: p=2x,\, then LaTeX: \langle z-x,x\rangle\leq0. Thus, LaTeX: \langle z-x,x\rangle=0. Denote LaTeX: u=z-x.\, Then LaTeX: \langle x,u\rangle=0. It remains to show that LaTeX: u=y.\, First, we prove that LaTeX: u\in\mathcal{K}^\circ. For this we have to show that LaTeX: \langle u,p\rangle\leq0, for all LaTeX: p\in\mathcal{K}. By using the characterization of the projection, we have

    LaTeX: \langle u,p\rangle=\langle u,p-x\rangle=\langle z-x,p-x\rangle\leq0,

    for all LaTeX: p\in\mathcal{K}. Thus, LaTeX: u\in\mathcal{K}^\circ. We also have

    LaTeX: \langle z-u,q-u\rangle=\langle x,q-u\rangle=\langle x,q\rangle\leq0,

    for all LaTeX: q\in\mathcal{K}^\circ, because LaTeX: x\in\mathcal{K}. By using again the characterization of the projection, it follows that LaTeX: u=y.\,

notes

For definition of convex cone in finite dimension see Convex cones, Wikimization.

For definition of polar cone in finite dimension, see Convex Optimization & Euclidean Distance Geometry.

LaTeX: \mathcal K^{\circ\circ}=K see Extended Farkas' lemma.

Applications

For applications see Every nonlinear complementarity problem is equivalent to a fixed point problem, Every implicit complementarity problem is equivalent to a fixed point problem, and Projection on isotone projection cone.

References

  • J. J. Moreau, Décomposition orthogonale d'un espace hilbertien selon deux cones mutuellement polaires, C. R. Acad. Sci., volume 255, pages 238–240, 1962.
Retrieved from "http://www.convexoptimization.com/wikimization/index.php/Moreau%27s_decomposition_theorem"
Personal tools